How to setup Elasticsearch+Kibana and enable Authentication in Ubuntu 20.04

The ELK stack is a popular stack for retrieving,visualizing and managing log files in a centralized location. It is a collection of three open-source tools, Elasticsearch, Kibana, and Logstash. Logstash is a free and open source tool for gathering, processing, and archiving logs. Kibana is a web interface that allows you to search and see the logs indexed by Logstash. Both of these tools are built on Elasticsearch, a log storage system.

Today, we’ll look at how to install Elasticsearch and Kibana on Ubuntu 20.04, as well as how to activate Auth in the free basic license.

Prerequisites

• OS: Ubuntu 20.04
• User account with root privileges

Step 1: Install Java 8

update apt package database and install java 8 openjdk

sudo apt-get update && sudo apt-get install openjdk-8-jdk -y

verify JAVA installation

java --version

Now that Java 8 is installed, let’s install Elasticsearch

Step2: Install Elasticsearch

import the Elasticsearch PGP key

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

Create Elasticsearch source list

echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-7.x.list

update and install Elasticsearch

sudo apt-get update && sudo apt-get install elasticsearch -y

Elasticsearch is now installed. Let’s modify the configuration

sudo vi /etc/elasticsearch/elasticsearch.yml

I’m modifying below configurations

cluster.name: my-elk-stack #To change elasticsearch cluster name
network.host: 0.0.0.0 #set 0.0.0.0 to listen on all network interface or you can restrict to specific network interface
discovery.type: single-node #for single node cluster or testing env. NOT RECOMMENDED IN PRODUCTION.
#below options are for enable Auth in elasticsearch. if you don't want to enable Auth, ignore below configuration
xpack.security.enabled: true 
xpack.security.transport.ssl.enabled: true 
xpack.security.transport.ssl.verification_mode: certificate 
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/elastic-certificates.p12 
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/elastic-certificates.p12

below is the full configuration file

now, generate certificate using below command

sudo /usr/share/elasticsearch/bin/elasticsearch-certutil cert -out elastic-certificates.p12 -pass ""

copy this certificate to /etc/elasticsearch directory and change owner

sudo cp /usr/share/elasticsearch/elastic-certificates.p12 /etc/elasticsearch/elastic-certificates.p12
sudo chown elasticsearch: /etc/elasticsearch/elastic-certificates.p12

you can use jvm.options file to adjust java options. I’m limiting memory for elasticsearch to 2gb

sudo vi /etc/elasticsearch/jvm.options

add below lines to adjust memory and save the file

-Xmx2g 
-Xms2g

Now restart the cluster

sudo systemctl restart elasticsearch.service

you can check status using below command

sudo systemctl status elasticsearch.service

To enable elasticsearch on system boot, run below command

sudo systemctl enable elasticsearch.service

if case of any error you can check logs in /var/log/elasticsearch/my-elk-stack.log

Now, it’s time to generate password for elasticsearch users

sudo /usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto

type y and press enter

Don’t forget to save the passwords at somewhere safe place

check elasticsearch status by using below CURL format in terminal. Replace username and password with your actual username and password

curl http://username:password@localhost:9200/_cluster/health?pretty

OUTPUT:

check elasticsearch status

Step 3: Install Kibana

Run below command to install Kibana

sudo apt-get install kibana -y

Kibana is now installed. Let’s modify the configuration

sudo vi /etc/kibana/kibana.yml

I’m modifying below configurations

server.port: 5601 #port on which kibana listen
server.host: "0.0.0.0" #set 0.0.0.0 to listen on all network interface or you can restrict to specific network interface
elasticsearch.hosts: ["http://localhost:9200"] #elasticsearch address.
elasticsearch.username: "kibana_user" #kibana user to connect with elasticsearch
elasticsearch.password: "passworod" #enter passowrd generated by elasticsearch-setup-passwords tool

Start kibana using below command

sudo systemctl start kibana

check status using below command

sudo systemctl status kibana

To enable kibana on system boot, run below command

sudo systemctl enable kibana

you can check logs of kibana at /var/log/kibana/kibana.log

To open kibana type below address in browser

http://localhost:5601/

To login enter username and password of elasticsearch.

Now you have successfully installed Elasticsearch and Kibana in ubuntu with Authentication enabled.