Logging - Monitoring C21: Difference between revisions

From ICO wiki
Jump to navigationJump to search
Line 24: Line 24:
Ubuntu 14.04
Ubuntu 14.04


Install MongoDB
Prerequisites:
The MongoDB installation is simple and quick. Run the following command to import the MongoDB public GPG key into apt:


sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
Since the Elasticsearch is based on java, we would require to install either openJDK or Oracle JDK. It is recommended to install Oracle JDK, verify the java version by using the following command.
Create the MongoDB source list:


echo 'deb http://downloads-distro.mongodb.org/repo/debian-sysvinit dist 10gen' | sudo tee /etc/apt/sources.list.d/mongodb.list
Remove the OpenJDK from the system, if you have it already installed.
Update your apt package database:


sudo apt-get update
$ sudo apt-get remove --purge openjdk*
Install the latest stable version of MongoDB with this command:
Add repository.


sudo apt-get install mongodb-org
$ sudo add-apt-repository -y ppa:webupd8team/java
MongoDB should be up and running now. Let's move on to installing Java 7.
Run the following command to pull the packages information from the newly added repository.


Install Java 7
$ sudo apt-get update
Elasticsearch requires Java 7, so we will install that now. We will install Oracle Java 7 because that is what is recommended on elasticsearch.org. It should, however, work fine with OpenJDK, if you decide to go that route.
Issue the following command to install Java jdk 1.8.


Add the Oracle Java PPA to apt:
$ sudo apt-get -y install oracle-java8-installer


sudo add-apt-repository ppa:webupd8team/java
$ java -version
Update your apt package database:


sudo apt-get update
Java version "1.8.0_60"
Install the latest stable version of Oracle Java 7 with this command (and accept the license agreement that pops up):
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
Install Elasticsearch:


sudo apt-get install oracle-java7-installer
Now that Java 7 is installed, let's install Elasticsearch.


Install Elasticsearch
Let’s install the Elasticsearch, it can be downloaded from official website.
Graylog2 v0.20.2 requires Elasticsearch v.0.90.10. Download and install it with these commands:


cd ~; wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.10.deb
'''Download and install GPG signing key'''.
sudo dpkg -i elasticsearch-0.90.10.deb
We need to change the Elasticsearch cluster.name setting. Open the Elasticsearch configuration file:


sudo vi /etc/elasticsearch/elasticsearch.yml
$ sudo wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
Find the section that specifies cluster.name. Uncomment it, and replace the default value with "graylog2", so it looks like the following:


cluster.name: graylog2
Save the repository definition to /etc/apt/sources.list.d/elasticsearch.list
You will also want to restrict outside access to your Elasticsearch instance (port 9200), so outsiders can't read your data or shutdown your Elasticseach cluster through the HTTP API. Find the line that specifies network.bind_host and uncomment it so it looks like this:


network.bind_host: localhost
$ echo "deb http://packages.elastic.co/elasticsearch/1.7/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch.list
Then add the following line somewhere in the file, to disable dynamic scripts:


script.disable_dynamic: true
Update repository cache.
Save and quit. Next, restart Elasticsearch to put our changes into effect:


sudo service elasticsearch restart
$ sudo apt-get update
After a few seconds, run the following to test that Elasticsearch is running properly:


curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'
'''Install Elasticsearch.'''
Now that Elasticsearch is up and running, let's install the Graylog2 server.


Install Graylog2 server
$ sudo apt-get install elasticsearch
Now that we have installed the other required software, let's install the Graylog2 server. We will install Graylog2 Server v0.20.2 in /opt. First, download the Graylog2 archive to /opt with this command:
Configure Elasticsearch to start during system startup.


cd /opt; sudo wget https://github.com/Graylog2/graylog2-server/releases/download/0.20.2/graylog2-server-0.20.2.tgz
$ sudo update-rc.d elasticsearch defaults
Then extract the archive:
The only important thing is to set a cluster name as “graylog2“, that is being used by graylog. Now edit the configuration file of Elasticsearch.


sudo tar xvf graylog2-server-0.20.2.tgz
$ sudo nano /etc/elasticsearch/elasticsearch.yml
Let's create a symbolic link to the newly created directory, to simplify the directory name:


sudo ln -s graylog2-server-0.20.2 graylog2-server
cluster.name: graylog2
Copy the example configuration file to the proper location, in /etc:
Disable dynamic scripts to avoid remote execution, that can be done by adding the following line at the end of above file.


sudo cp /opt/graylog2-server/graylog2.conf.example /etc/graylog2.conf
script.disable_dynamic: true
Install pwgen, which we will use to generate password secret keys:
Once it is done, we are good to go. Before that, restart the Elasticsearch services to load the modified configuration.


sudo apt-get install pwgen
$ sudo service elasticsearch restart
Now we must configure the admin password and secret key. The password secret key is configured in graylog2.conf, by the password_secret parameter. We can generate a random key and insert it into the Graylog2 configuration with the following two commands:
Wait at least a minute to let the Elasticsearch get fully restarted, otherwise testing will fail. Elastisearch should be now listen on 9200 for processing HTTP request, we can use CURL to get the response. Ensure that it returns with cluster name as “graylog2”


SECRET=$(pwgen -s 96 1)
$ curl -X GET http://localhost:9200
sudo -E sed -i -e 's/password_secret =.*/password_secret = '$SECRET'/' /etc/graylog2.conf
The admin password is assigned by creating an shasum of the desired password, and assigning it to the root_password_sha2 parameter in the Graylog2 configuration file. Create shasum of your desired password with the following command, substituting the highlighted "password" with your own. The sed command inserts it into the Graylog2 configuration for you:


PASSWORD=$(echo -n password | shasum -a 256 | awk '{print $1}')
{
sudo -E sed -i -e 's/root_password_sha2 =.*/root_password_sha2 = '$PASSWORD'/' /etc/graylog2.conf
  "status" : 200,
Now that the admin password is setup, let's open the Graylog2 configuration to make a few changes:
  "name" : "Pistol",
  "cluster_name" : "graylog2",
  "version" : {
    "number" : "1.7.1",
    "build_hash" : "b88f43fc40b0bcd7f173a1f9ee2e97816de80b19",
    "build_timestamp" : "2015-07-29T09:54:16Z",
    "build_snapshot" : false,
    "lucene_version" : "4.10.4"
  },
  "tagline" : "You Know, for Search"
}


sudo vi /etc/graylog2.conf
Optional: Use the following command to check the Elasticsearch cluster health, you must get a cluster status as “green” for graylog to work.
You should see that password_secret and root_password_sha2 have random strings to them, because of the commands that you ran in the steps above. Now we will configure the rest_transport_uri, which is how the Graylog2 web interface will communicate with the server. Because we are installing all of the components on a single server, let's set the value to 127.0.0.1, or localhost. Find and uncomment rest_transport_uri, and change it's value so it looks like the following:


rest_transport_uri = http://127.0.0.1:12900/
$ curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'
Next, because we only have one Elasticsearch shard (which is running on this server), we will change the value of elasticsearch_shards to 1:


elasticsearch_shards = 1
{
Save and quit. Now our Graylog2 server is configured and ready to be started.
  "cluster_name" : "graylog2",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0
}
 
 
'''Install MongoDB:'''
 
MongoDB is available in dep format and same can be downloaded from the official website. Add the following repository information on the system to install MongoDB. Before that we must import public key.
 
$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
Add repository by creating the /etc/apt/sources.list.d/mongodb-org-3.0.list list file using the command.
 
$ echo "deb http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.0.list
Update repository cache.
 
$ sudo apt-get update
Install MongoDB using the following command.
 
$ sudo apt-get install mongodb-org
Start the MongoDB service and enable it to start automatically during the system start-up.
 
$ sudo service mongod start
$ sudo update-rc.d mongod defaults
Install Graylog2:
 
Graylog-server accepts and process the log messages, also spawns the RESTAPI for the requests that comes from graylog-web-interface. Download the latest version of graylog from graylog.org,
 
Use the following command to install graylog2 repository.
 
$ wget https://packages.graylog2.org/repo/packages/graylog-1.2-repository-ubuntu14.04_latest.deb
 
$ sudo dpkg -i graylog-1.2-repository-ubuntu14.04_latest.deb
Install https suppport and update the repository cache.
 
$ sudo apt-get install apt-transport-https
 
$ sudo apt-get update
Install Graylog server using following command.
 
$ sudo apt-get install graylog-server
Edit the server.conf file.
 
$ sudo nano /etc/graylog/server/server.conf
Configure the following variables in the above file.
 
Set a secret to secure the user passwords, use the following command to generate a secret, use at least 64 character’s.
 
$ pwgen -N 1 -s 96
 
OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP
If you get a “pwgen: command not found“, use the following command to install pwgen.
 
$ sudo apt-get install pwgen
Place the secret.
 
password_secret = OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP
Next is to set a hash password for the root user (not to be confused with system user, root user of graylog is admin). You will use this password for login into the web interface, admin’s password can not be changed using web interface, must edit this variable to set.
 
Replace “yourpassword” with the choice of your’s.


Optional: If you want to test it out, run the following command:
# echo -n yourpassword | sha256sum


sudo java -jar /opt/graylog2-server/graylog2-server.jar --debug
e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951
You should see a lot of output. Once you see output similar to the following lines, you will know that your Graylog2 server was configured correctly:
Place the hash password.


2014-06-06 14:16:13,420 INFO : org.graylog2.Core - Started REST API at <http://127.0.0.1:12900/>
root_password_sha2 = e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951
2014-06-06 14:16:13,421 INFO : org.graylog2.Main - Graylog2 up and running.
You can setup email address root (admin) user.
Press CTRL-C to kill the test and return to the shell.


Now let's install the Graylog2 init script. Copy graylog2ctl to /etc/init.d:
root_email = "itzgeek.web@gmail.com"
Set timezone of root (admin) user.


sudo cp /opt/graylog2-server/bin/graylog2ctl /etc/init.d/graylog2
root_timezone = UTC
Update the startup script to put the Graylog2 logs in /var/log and to look for the Graylog2 server JAR file in /opt/graylog2-server by running the two following sed commands:
Graylog will try to find the Elasticsearch nodes automatically, it uses multicast mode for the same. But when it comes to larger network, it is recommended to use unicast mode which is best suited one for production setups. So add the following two entries to graylog server.conf file, replace ipaddress with live hostname or ipaddress. Multiple hosts can be added with comma separated.


sudo sed -i -e 's/GRAYLOG2_SERVER_JAR=\${GRAYLOG2_SERVER_JAR:=graylog2-server.jar}/GRAYLOG2_SERVER_JAR=\${GRAYLOG2_SERVER_JAR:=\/opt\/graylog2-server\/graylog2-server.jar}/' /etc/init.d/graylog2
elasticsearch_http_enabled = false
sudo sed -i -e 's/LOG_FILE=\${LOG_FILE:=log\/graylog2-server.log}/LOG_FILE=\${LOG_FILE:=\/var\/log\/graylog2-server.log}/' /etc/init.d/graylog2
elasticsearch_discovery_zen_ping_unicast_hosts = ipaddress:9300
Next, install the startup script:
Set only one master node by defining the below variable, default setting is true, you must set it as a false to make the particular node as a slave. Master node performs some periodic tasks that slave won’t perform.


sudo update-rc.d graylog2 defaults
is_master = true
Now we can start the Graylog2 server with the service command:
The following variable sets the number of log messages to keep per index, it is recommended to have several smaller indices instead of larger ones.


sudo service graylog2 start
elasticsearch_max_docs_per_index = 20000000
The next step is to install the Graylog2 web interface. Let's do that now!
The following parameter defines to have total number of indices, if the this number is reached old index will be deleted.


Install Graylog2 Web Interface
elasticsearch_max_number_of_indices = 20
We will download and install the Graylog2 v.0.20.2 web interface in /opt with the following commands:
Shards setting is really depends on the number of nodes in the Elasticsearch cluster, if you have only one node, set it as 1.


cd /opt; sudo wget https://github.com/Graylog2/graylog2-web-interface/releases/download/0.20.2/graylog2-web-interface-0.20.2.tgz
elasticsearch_shards = 1
sudo tar xvf graylog2-web-interface-0.20.2.tgz
The number of replicas for your indices, if you have only one node in Elasticsearch cluster; set it as 0.
Let's create a symbolic link to the newly created directory, to simplify the directory name:


sudo ln -s graylog2-web-interface-0.20.2 graylog2-web-interface
elasticsearch_replicas = 0
Next, we want to configure the web interface's secret key, the application.secret parameter in graylog2-web-interface.conf. We will generate another key, as we did with the Graylog2 server configuration, and insert it with sed, like so:
Restart Graylog service.


SECRET=$(pwgen -s 96 1)
$ sudo service graylog-server restart
sudo -E sed -i -e 's/application\.secret=""/application\.secret="'$SECRET'"/' /opt/graylog2-web-interface/conf/graylog2-web-interface.conf
Enable auto start of graylog server service during system startup.
Now open the web interface configuration file, with this command:


sudo vi /opt/graylog2-web-interface/conf/graylog2-web-interface.conf
$ sudo update-rc.d graylog-server defaults
Now we need to update the web interface's configuration to specify the graylog2-server.uris parameter. This is a comma delimited list of the server REST URIs. Since we only have one Graylog2 server node, the value should match that of rest_listen_uri in the Graylog2 server configuration (i.e. "http://127.0.0.1:12900/").
You can check out the server startup logs, it will be useful for you to troubleshoot graylog in case of any issue.


graylog2-server.uris="http://127.0.0.1:12900/"
# tailf /var/log/graylog-server/server.log
The Graylog2 web interface is now configured. Let's start it up to test it out:
On successful start of graylog-server, you should get the following message in the log file.


sudo /opt/graylog2-web-interface-0.20.2/bin/graylog2-web-interface
2015-09-17T09:35:22.895+02:00 INFO  [ServerBootstrap] Graylog server up and running.
You will know it started properly when you see the following two lines:
Install Graylog web interface:


[info] play - Application started (Prod)
To configure graylog-web-interface, you must have at least one graylog-server node. Install Graylog web interface using “apt-get”.
[info] play - Listening for HTTP on /0:0:0:0:0:0:0:0:9000
Hit CTRL-C to kill the web interface. Now let's install a startup script. You can either create your own, or download one that I created for this tutorial. To download the script to your home directory, use this command:


cd ~; wget https://assets.digitalocean.com/articles/graylog2/graylog2-web
$ sudo apt-get install graylog-web
Next, you will want to copy it to /etc/init.d, and change its ownership to root and its permissions to 755:
Edit the configuration file and set the following parameters.


sudo cp ~/graylog2-web /etc/init.d/
$ sudo nano /etc/graylog/web/web.conf
sudo chown root:root /etc/init.d/graylog2-web
This is the list of graylog-server nodes, you can add multiple nodes, separate by commas.
sudo chmod 755 /etc/init.d/graylog2-web
Now you can install the web interface init script with this command:


sudo update-rc.d graylog2-web defaults
graylog2-server.uris="http://127.0.0.1:12900/"
Start the Graylog2 web interface:
Set the application scret and can be generated using pwgen -N 1 -s 96.


sudo service graylog2-web start
application.secret="sNXyFf6B4Au3GqSlZwq7En86xp10JimdxxYiLtpptOejX6tIUpUE4DGRJOrcMj07wcK0wugPaapvzEzCYinEWj7BOtHXVl5Z"
Now we can use the Graylog2 web interface. Let's do that now.
Set Web interface timezone.


Configure Graylog2 to Receive syslog messages
timezone="Europe/Berlin"
Log into Graylog2 Web Interface
Restart the gralog-web-interface using following command,


In your favorite browser, go to the port 9000 of your VPS's public IP address:
$ sudo service graylog-web restart
Enable auto start of web interface service during system startup.


http://gl2_public_IP:9000/
$ sudo update-rc.d graylog-web defaults
You should see a login screen. Enter "admin" as your username and the password the admin password that you set earlier.


= Summary =
= Summary =


= References =
= References =

Revision as of 17:57, 5 October 2016

Logging and Monitoring with Graylog


Course: Logging and Monitoring - Lecturer: Margus Ernits

Group : Cyber Security Engineering (C21)

Team members: Ender Phan, Kustas Kurval, Sheela Gowry Sumathi Raju, Artur Vincent Kerge

Page created by : October 05, 2016

Abstract

In order to understand how to set up the Graylog service as well as understand its crucial roles. We decided to choose Graylog as our application for Logging and Monitoring. Below are our objectives which would be expected to achieve later on:

- How to install Graylog on Ubuntu 14.04/16.0.

- How to use Graylog to protect servers.

(..more)

Installation Guide

Ubuntu 14.04

Prerequisites:

Since the Elasticsearch is based on java, we would require to install either openJDK or Oracle JDK. It is recommended to install Oracle JDK, verify the java version by using the following command.

Remove the OpenJDK from the system, if you have it already installed. 

$ sudo apt-get remove --purge openjdk*

Add repository. 

$ sudo add-apt-repository -y ppa:webupd8team/java

Run the following command to pull the packages information from the newly added repository. 

$ sudo apt-get update

Issue the following command to install Java jdk 1.8. 

$ sudo apt-get -y install oracle-java8-installer

$ java -version

Java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
Install Elasticsearch:


Let’s install the Elasticsearch, it can be downloaded from official website.

Download and install GPG signing key. 

$ sudo wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

Save the repository definition to /etc/apt/sources.list.d/elasticsearch.list 

$ echo "deb http://packages.elastic.co/elasticsearch/1.7/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch.list

Update repository cache. 

$ sudo apt-get update

Install Elasticsearch. 

$ sudo apt-get install elasticsearch

Configure Elasticsearch to start during system startup. 

$ sudo update-rc.d elasticsearch defaults

The only important thing is to set a cluster name as “graylog2“, that is being used by graylog. Now edit the configuration file of Elasticsearch. 

$ sudo nano /etc/elasticsearch/elasticsearch.yml

cluster.name: graylog2

Disable dynamic scripts to avoid remote execution, that can be done by adding the following line at the end of above file. 

script.disable_dynamic: true

Once it is done, we are good to go. Before that, restart the Elasticsearch services to load the modified configuration. 

$ sudo service elasticsearch restart

Wait at least a minute to let the Elasticsearch get fully restarted, otherwise testing will fail. Elastisearch should be now listen on 9200 for processing HTTP request, we can use CURL to get the response. Ensure that it returns with cluster name as “graylog2” 

$ curl -X GET http://localhost:9200

{
  "status" : 200,
  "name" : "Pistol",
  "cluster_name" : "graylog2",
  "version" : {
   "number" : "1.7.1",
   "build_hash" : "b88f43fc40b0bcd7f173a1f9ee2e97816de80b19",
   "build_timestamp" : "2015-07-29T09:54:16Z",
   "build_snapshot" : false,
   "lucene_version" : "4.10.4"
 },
 "tagline" : "You Know, for Search"
}

Optional: Use the following command to check the Elasticsearch cluster health, you must get a cluster status as “green” for graylog to work. 

$ curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'

{
 "cluster_name" : "graylog2",
 "status" : "green",
 "timed_out" : false,
 "number_of_nodes" : 1,
 "number_of_data_nodes" : 1,
 "active_primary_shards" : 0,
 "active_shards" : 0,
 "relocating_shards" : 0,
 "initializing_shards" : 0,
 "unassigned_shards" : 0,
 "delayed_unassigned_shards" : 0,
 "number_of_pending_tasks" : 0,
 "number_of_in_flight_fetch" : 0
}


Install MongoDB:

MongoDB is available in dep format and same can be downloaded from the official website. Add the following repository information on the system to install MongoDB. Before that we must import public key.

$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10 Add repository by creating the /etc/apt/sources.list.d/mongodb-org-3.0.list list file using the command.

$ echo "deb http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.0.list Update repository cache.

$ sudo apt-get update Install MongoDB using the following command.

$ sudo apt-get install mongodb-org Start the MongoDB service and enable it to start automatically during the system start-up.

$ sudo service mongod start $ sudo update-rc.d mongod defaults Install Graylog2:

Graylog-server accepts and process the log messages, also spawns the RESTAPI for the requests that comes from graylog-web-interface. Download the latest version of graylog from graylog.org,

Use the following command to install graylog2 repository.

$ wget https://packages.graylog2.org/repo/packages/graylog-1.2-repository-ubuntu14.04_latest.deb

$ sudo dpkg -i graylog-1.2-repository-ubuntu14.04_latest.deb Install https suppport and update the repository cache.

$ sudo apt-get install apt-transport-https

$ sudo apt-get update Install Graylog server using following command.

$ sudo apt-get install graylog-server Edit the server.conf file.

$ sudo nano /etc/graylog/server/server.conf Configure the following variables in the above file.

Set a secret to secure the user passwords, use the following command to generate a secret, use at least 64 character’s.

$ pwgen -N 1 -s 96

OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP If you get a “pwgen: command not found“, use the following command to install pwgen.

$ sudo apt-get install pwgen Place the secret.

password_secret = OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP Next is to set a hash password for the root user (not to be confused with system user, root user of graylog is admin). You will use this password for login into the web interface, admin’s password can not be changed using web interface, must edit this variable to set.

Replace “yourpassword” with the choice of your’s.

  1. echo -n yourpassword | sha256sum

e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951 Place the hash password.

root_password_sha2 = e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951 You can setup email address root (admin) user.

root_email = "itzgeek.web@gmail.com" Set timezone of root (admin) user.

root_timezone = UTC Graylog will try to find the Elasticsearch nodes automatically, it uses multicast mode for the same. But when it comes to larger network, it is recommended to use unicast mode which is best suited one for production setups. So add the following two entries to graylog server.conf file, replace ipaddress with live hostname or ipaddress. Multiple hosts can be added with comma separated.

elasticsearch_http_enabled = false elasticsearch_discovery_zen_ping_unicast_hosts = ipaddress:9300 Set only one master node by defining the below variable, default setting is true, you must set it as a false to make the particular node as a slave. Master node performs some periodic tasks that slave won’t perform.

is_master = true The following variable sets the number of log messages to keep per index, it is recommended to have several smaller indices instead of larger ones.

elasticsearch_max_docs_per_index = 20000000 The following parameter defines to have total number of indices, if the this number is reached old index will be deleted.

elasticsearch_max_number_of_indices = 20 Shards setting is really depends on the number of nodes in the Elasticsearch cluster, if you have only one node, set it as 1.

elasticsearch_shards = 1 The number of replicas for your indices, if you have only one node in Elasticsearch cluster; set it as 0.

elasticsearch_replicas = 0 Restart Graylog service.

$ sudo service graylog-server restart Enable auto start of graylog server service during system startup.

$ sudo update-rc.d graylog-server defaults You can check out the server startup logs, it will be useful for you to troubleshoot graylog in case of any issue.

  1. tailf /var/log/graylog-server/server.log

On successful start of graylog-server, you should get the following message in the log file.

2015-09-17T09:35:22.895+02:00 INFO [ServerBootstrap] Graylog server up and running. Install Graylog web interface:

To configure graylog-web-interface, you must have at least one graylog-server node. Install Graylog web interface using “apt-get”.

$ sudo apt-get install graylog-web Edit the configuration file and set the following parameters.

$ sudo nano /etc/graylog/web/web.conf This is the list of graylog-server nodes, you can add multiple nodes, separate by commas.

graylog2-server.uris="http://127.0.0.1:12900/" Set the application scret and can be generated using pwgen -N 1 -s 96.

application.secret="sNXyFf6B4Au3GqSlZwq7En86xp10JimdxxYiLtpptOejX6tIUpUE4DGRJOrcMj07wcK0wugPaapvzEzCYinEWj7BOtHXVl5Z" Set Web interface timezone.

timezone="Europe/Berlin" Restart the gralog-web-interface using following command,

$ sudo service graylog-web restart Enable auto start of web interface service during system startup.

$ sudo update-rc.d graylog-web defaults

Summary

References

Retrieved from "https://wiki.itcollege.ee/index.php?title=Logging_-_Monitoring_C21&oldid=106110"