Logging - Monitoring C21: Difference between revisions

From ICO wiki
Jump to navigationJump to search
 
(98 intermediate revisions by 3 users not shown)
Line 1: Line 1:
'''Logging and Monitoring with Graylog'''
'''Logging and Monitoring with Graylog'''


Course: Logging and Monitoring - Lecturer: Margus Ernits
Course: Logging and Monitoring - Lecturer: Margus Ernits


Group : Cyber Security Engineering (C21)
Group: Cyber Security Engineering (C21)


Team members: Ender Phan, Kustas Kurval, Sheela Gowry Sumathi Raju, Artur Vincent Kerge
Team members: Ender Phan, Kustas Kurval, Sheela Gowry Sumathi Raju, Artur Vincent Kerge


Page created by : October 05, 2016
Page created on: October 05, 2016


= Abstract =
= Abstract =


In order to understand how to set up the Graylog service as well as understand its crucial roles. We decided to choose Graylog as our application for Logging and Monitoring. Below are our objectives which would be expected to achieve later on:
In order to understand how to set up the Graylog service as well as understand its crucial roles. We decided to choose Graylog as our application for logging and monitoring. Below are our objectives which would be expected to achieve later on:
 
* '''How to install Graylog on Ubuntu 14.04.'''
 
* '''How to use Graylog to protect servers.'''
 
* '''Upgrading and configuring Graylog at first, and know how to secure Graylog.'''
 
* '''Threats and security during logging.'''
 
* '''How to install Netdata and Monitoring with Netdata.'''
 
* '''How to install Nagios and Monitoring with Nagios.'''
 
* '''Trouble shooting and experiences.'''
 
= Installation Guide =
 
'''Ubuntu 14.04'''
 
 
[[File:Graylog_simple_setup_v2.png]]
 
== Logging Solutions ==
 
Since the Elasticsearch is based on java, we would require to install either openJDK or Oracle JDK. It is recommended to install Oracle JDK, verify the java version by using the following command.
 
Remove the OpenJDK from the system, if you have it already installed. From now on it is presumed that user has root privileges.
 
apt-get remove --purge openjdk*
Add repository.
 
add-apt-repository -y ppa:webupd8team/java
Run the following command to pull the packages information from the newly added repository.
 
apt-get update
Issue the following command to install Java jdk 1.8.
 
apt-get -y install oracle-java8-installer
 
java -version
 
Output:
 
Java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)


- How to install Graylog on Ubuntu 14.04/16.0.
=== Installing extras ===


- How to use Graylog to protect servers.
=== Elasticsearch ===


(..more)
Let’s install Elasticsearch, it can be downloaded from the official website.


= Installation Guide=
'''Download and install GPG signing key'''.


Ubuntu 14.04
wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -


Install MongoDB
Note that the -qO argument is lowercase Quebec followed by capital Oscar.
The MongoDB installation is simple and quick. Run the following command to import the MongoDB public GPG key into apt:


sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
Save the repository definition to /etc/apt/sources.list.d/elasticsearch.list
Create the MongoDB source list:


echo 'deb http://downloads-distro.mongodb.org/repo/debian-sysvinit dist 10gen' | sudo tee /etc/apt/sources.list.d/mongodb.list
echo "deb http://packages.elastic.co/elasticsearch/2.x/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch.list
Update your apt package database:


sudo apt-get update
Update repository cache and install Elasticsearch
Install the latest stable version of MongoDB with this command:


sudo apt-get install mongodb-org
apt-get update && apt-get install elasticsearch
MongoDB should be up and running now. Let's move on to installing Java 7.


Install Java 7
Configure Elasticsearch to start during system startup.
Elasticsearch requires Java 7, so we will install that now. We will install Oracle Java 7 because that is what is recommended on elasticsearch.org. It should, however, work fine with OpenJDK, if you decide to go that route.


Add the Oracle Java PPA to apt:
update-rc.d elasticsearch defaults
The only important thing is to set a cluster name as “graylog2“, that is being used by graylog. Now edit the configuration file of Elasticsearch.


sudo add-apt-repository ppa:webupd8team/java
nano /etc/elasticsearch/elasticsearch.yml
Update your apt package database:


sudo apt-get update
cluster.name: graylog2
Install the latest stable version of Oracle Java 7 with this command (and accept the license agreement that pops up):


sudo apt-get install oracle-java7-installer
In the same file disable dynamic scripts to avoid remote execution. That can be done by adding the following line:
Now that Java 7 is installed, let's install Elasticsearch.


Install Elasticsearch
'''FOLLOWING LINE GOES INTO GRAYLOG CONF!'''
Graylog2 v0.20.2 requires Elasticsearch v.0.90.10. Download and install it with these commands:


cd ~; wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.10.deb
script.disable_dynamic: true
sudo dpkg -i elasticsearch-0.90.10.deb
We need to change the Elasticsearch cluster.name setting. Open the Elasticsearch configuration file:


sudo vi /etc/elasticsearch/elasticsearch.yml
Once it is done, we are good to go. Before that, restart the Elasticsearch services to load the modified configuration.
Find the section that specifies cluster.name. Uncomment it, and replace the default value with "graylog2", so it looks like the following:


cluster.name: graylog2
service elasticsearch restart
You will also want to restrict outside access to your Elasticsearch instance (port 9200), so outsiders can't read your data or shutdown your Elasticseach cluster through the HTTP API. Find the line that specifies network.bind_host and uncomment it so it looks like this:


network.bind_host: localhost
Wait at least a minute to let the Elasticsearch get fully restarted, otherwise testing will fail. Elastisearch should be now listen on 9200 for processing HTTP request, we can use CURL to get the response. Ensure that it returns with cluster name as “graylog2”
Then add the following line somewhere in the file, to disable dynamic scripts:


script.disable_dynamic: true
curl -X GET http://localhost:9200
Save and quit. Next, restart Elasticsearch to put our changes into effect:


sudo service elasticsearch restart
{
After a few seconds, run the following to test that Elasticsearch is running properly:
  "status" : 200,
  "name" : "Pistol",
  "cluster_name" : "'''graylog2'''",
  "version" : {
    "number" : "1.7.1",
    "build_hash" : "b88f43fc40b0bcd7f173a1f9ee2e97816de80b19",
    "build_timestamp" : "2015-07-29T09:54:16Z",
    "build_snapshot" : false,
    "lucene_version" : "4.10.4"
  },
  "tagline" : "You Know, for Search"
}


curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'
Optional: Use the following command to check the Elasticsearch cluster health, you must get a cluster status as “green” for graylog to work.
Now that Elasticsearch is up and running, let's install the Graylog2 server.


Install Graylog2 server
curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'
Now that we have installed the other required software, let's install the Graylog2 server. We will install Graylog2 Server v0.20.2 in /opt. First, download the Graylog2 archive to /opt with this command:


cd /opt; sudo wget https://github.com/Graylog2/graylog2-server/releases/download/0.20.2/graylog2-server-0.20.2.tgz
{
Then extract the archive:
  "cluster_name" : "'''graylog2'''",
  "status" : "'''green'''",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0
}


sudo tar xvf graylog2-server-0.20.2.tgz
=== MongoDB ===
Let's create a symbolic link to the newly created directory, to simplify the directory name:


sudo ln -s graylog2-server-0.20.2 graylog2-server
MongoDB is available in deb format and same can be downloaded from the official website. Add the following repository information on the system to install MongoDB. Before that we must import public key.
Copy the example configuration file to the proper location, in /etc:


sudo cp /opt/graylog2-server/graylog2.conf.example /etc/graylog2.conf
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
Install pwgen, which we will use to generate password secret keys:
Add repository by creating the /etc/apt/sources.list.d/mongodb-org-3.0.list list file using the command.


sudo apt-get install pwgen
echo "deb http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.0.list
Now we must configure the admin password and secret key. The password secret key is configured in graylog2.conf, by the password_secret parameter. We can generate a random key and insert it into the Graylog2 configuration with the following two commands:
Update repository cache.


SECRET=$(pwgen -s 96 1)
apt-get update
sudo -E sed -i -e 's/password_secret =.*/password_secret = '$SECRET'/' /etc/graylog2.conf
Install MongoDB using the following command.
The admin password is assigned by creating an shasum of the desired password, and assigning it to the root_password_sha2 parameter in the Graylog2 configuration file. Create shasum of your desired password with the following command, substituting the highlighted "password" with your own. The sed command inserts it into the Graylog2 configuration for you:


PASSWORD=$(echo -n password | shasum -a 256 | awk '{print $1}')
apt-get install mongodb-org
sudo -E sed -i -e 's/root_password_sha2 =.*/root_password_sha2 = '$PASSWORD'/' /etc/graylog2.conf
Start the MongoDB service and enable it to start automatically during the system start-up.
Now that the admin password is setup, let's open the Graylog2 configuration to make a few changes:


sudo vi /etc/graylog2.conf
service mongod start
You should see that password_secret and root_password_sha2 have random strings to them, because of the commands that you ran in the steps above. Now we will configure the rest_transport_uri, which is how the Graylog2 web interface will communicate with the server. Because we are installing all of the components on a single server, let's set the value to 127.0.0.1, or localhost. Find and uncomment rest_transport_uri, and change it's value so it looks like the following:


rest_transport_uri = http://127.0.0.1:12900/
ln -s /usr/bin/mongod /etc/init.d/mongod
Next, because we only have one Elasticsearch shard (which is running on this server), we will change the value of elasticsearch_shards to 1:


elasticsearch_shards = 1
Create a symbolic link
Save and quit. Now our Graylog2 server is configured and ready to be started.


Optional: If you want to test it out, run the following command:
update-rc.d mongod defaults


sudo java -jar /opt/graylog2-server/graylog2-server.jar --debug
=== Graylog2 ===
You should see a lot of output. Once you see output similar to the following lines, you will know that your Graylog2 server was configured correctly:


2014-06-06 14:16:13,420 INFO : org.graylog2.Core - Started REST API at <http://127.0.0.1:12900/>
Graylog-server accepts and process the log messages, also spawns the RESTAPI for the requests that comes from graylog-web-interface. Download the latest version of graylog from graylog.org,
2014-06-06 14:16:13,421 INFO : org.graylog2.Main - Graylog2 up and running.
Press CTRL-C to kill the test and return to the shell.


Now let's install the Graylog2 init script. Copy graylog2ctl to /etc/init.d:
Use the following command to install graylog2 repository.


sudo cp /opt/graylog2-server/bin/graylog2ctl /etc/init.d/graylog2
wget https://packages.graylog2.org/repo/packages/graylog-1.2-repository-ubuntu14.04_latest.deb
Update the startup script to put the Graylog2 logs in /var/log and to look for the Graylog2 server JAR file in /opt/graylog2-server by running the two following sed commands:


sudo sed -i -e 's/GRAYLOG2_SERVER_JAR=\${GRAYLOG2_SERVER_JAR:=graylog2-server.jar}/GRAYLOG2_SERVER_JAR=\${GRAYLOG2_SERVER_JAR:=\/opt\/graylog2-server\/graylog2-server.jar}/' /etc/init.d/graylog2
dpkg -i graylog-1.2-repository-ubuntu14.04_latest.deb
sudo sed -i -e 's/LOG_FILE=\${LOG_FILE:=log\/graylog2-server.log}/LOG_FILE=\${LOG_FILE:=\/var\/log\/graylog2-server.log}/' /etc/init.d/graylog2
Install https suppport and update the repository cache.
Next, install the startup script:


sudo update-rc.d graylog2 defaults
apt-get install apt-transport-https
Now we can start the Graylog2 server with the service command:


sudo service graylog2 start
apt-get update
The next step is to install the Graylog2 web interface. Let's do that now!
Install Graylog server using following command.


Install Graylog2 Web Interface
apt-get install graylog-server
We will download and install the Graylog2 v.0.20.2 web interface in /opt with the following commands:
Edit the server.conf file.


cd /opt; sudo wget https://github.com/Graylog2/graylog2-web-interface/releases/download/0.20.2/graylog2-web-interface-0.20.2.tgz
nano /etc/graylog/server/server.conf
sudo tar xvf graylog2-web-interface-0.20.2.tgz
Configure the following variables in the above file.
Let's create a symbolic link to the newly created directory, to simplify the directory name:


sudo ln -s graylog2-web-interface-0.20.2 graylog2-web-interface
Set a secret to secure the user passwords, use the following command to generate a secret, use at least 64 character’s.
Next, we want to configure the web interface's secret key, the application.secret parameter in graylog2-web-interface.conf. We will generate another key, as we did with the Graylog2 server configuration, and insert it with sed, like so:


SECRET=$(pwgen -s 96 1)
pwgen -N 1 -s 96
sudo -E sed -i -e 's/application\.secret=""/application\.secret="'$SECRET'"/' /opt/graylog2-web-interface/conf/graylog2-web-interface.conf
Now open the web interface configuration file, with this command:


sudo vi /opt/graylog2-web-interface/conf/graylog2-web-interface.conf
OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP
Now we need to update the web interface's configuration to specify the graylog2-server.uris parameter. This is a comma delimited list of the server REST URIs. Since we only have one Graylog2 server node, the value should match that of rest_listen_uri in the Graylog2 server configuration (i.e. "http://127.0.0.1:12900/").


graylog2-server.uris="http://127.0.0.1:12900/"
If you get a “pwgen: command not found“, use the following command to install pwgen.
The Graylog2 web interface is now configured. Let's start it up to test it out:


sudo /opt/graylog2-web-interface-0.20.2/bin/graylog2-web-interface
apt-get install pwgen
You will know it started properly when you see the following two lines:


[info] play - Application started (Prod)
Place the secret.
[info] play - Listening for HTTP on /0:0:0:0:0:0:0:0:9000
Hit CTRL-C to kill the web interface. Now let's install a startup script. You can either create your own, or download one that I created for this tutorial. To download the script to your home directory, use this command:


cd ~; wget https://assets.digitalocean.com/articles/graylog2/graylog2-web
password_secret = OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP
Next, you will want to copy it to /etc/init.d, and change its ownership to root and its permissions to 755:


sudo cp ~/graylog2-web /etc/init.d/
Next is to set a hash password for the root user (not to be confused with system user, root user of graylog is admin). You will use this password for login into the web interface, admin’s password can not be changed using web interface, must edit this variable to set.
sudo chown root:root /etc/init.d/graylog2-web
sudo chmod 755 /etc/init.d/graylog2-web
Now you can install the web interface init script with this command:


sudo update-rc.d graylog2-web defaults
Replace “yourpassword” with the choice of your’s.
Start the Graylog2 web interface:


sudo service graylog2-web start
echo -n yourpassword | sha256sum
Now we can use the Graylog2 web interface. Let's do that now.


Configure Graylog2 to Receive syslog messages
e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951
Log into Graylog2 Web Interface


In your favorite browser, go to the port 9000 of your VPS's public IP address:
Place the hash password.


http://gl2_public_IP:9000/
root_password_sha2 = e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951
You should see a login screen. Enter "admin" as your username and the password the admin password that you set earlier.
 
You can setup email address root (admin) user.
 
root_email = "cyber.web@gmail.com"
 
Set timezone of root (admin) user.
 
root_timezone = UTC
 
Graylog will try to find the Elasticsearch nodes automatically, it uses multicast mode for the same. But when it comes to larger network, it is recommended to use unicast mode which is best suited one for production setups. So add the following two entries to graylog server.conf file, replace ipaddress with live hostname or ipaddress. Multiple hosts can be added with comma separated.
 
elasticsearch_http_enabled = false
elasticsearch_discovery_zen_ping_unicast_hosts = ipaddress:9300
 
Set only one master node by defining the below variable, default setting is true, you must set it as a false to make the particular node as a slave. Master node performs some periodic tasks that slave won’t perform.
 
is_master = true
 
The following variable sets the number of log messages to keep per index, it is recommended to have several smaller indices instead of larger ones.
 
elasticsearch_max_docs_per_index = 20000000
 
The following parameter defines to have total number of indices, if the this number is reached old index will be deleted.
 
elasticsearch_max_number_of_indices = 20
 
Shards setting is really depends on the number of nodes in the Elasticsearch cluster, if you have only one node, set it as 1.
 
elasticsearch_shards = 1
 
The number of replicas for your indices, if you have only one node in Elasticsearch cluster; set it as 0.
 
elasticsearch_replicas = 0
 
''Restart Graylog service.''
 
service graylog-server restart
 
Enable auto start of graylog server service during system startup.
 
update-rc.d graylog-server defaults
 
You can check out the server startup logs, it will be useful for you to troubleshoot graylog in case of any issue.
 
tailf /var/log/graylog-server/server.log
 
On successful start of graylog-server, you should get the following message in the log file.
 
2015-09-17T09:35:22.895+02:00 INFO  [ServerBootstrap] Graylog server up and running.
 
'''Install Graylog web interface:'''
 
To configure graylog-web-interface, you must have at least one graylog-server node. Install Graylog web interface using “apt-get”.
 
apt-get install graylog-web
 
Edit the configuration file and set the following parameters.
 
nano /etc/graylog/web/web.conf
 
This is the list of graylog-server nodes, you can add multiple nodes, separate by commas.
 
graylog2-server.uris="http://127.0.0.1:12900/"
 
Set the application scret and can be generated using pwgen -N 1 -s 96.
 
application.secret="sNXyFf6B4Au3GqSlZwq7En86xp10JimdxxYiLtpptOejX6tIUpUE4DGRJOrcMj07wcK0wugPaapvzEzCYinEWj7BOtHXVl5Z"
 
Set Web interface timezone.
 
Timezone="Europe/Tallinn"
 
Restart the gralog-web-interface using following command,
 
service graylog-web restart
 
Enable auto start of web interface service during system startup.
 
update-rc.d graylog-web defaults
 
Add log forwarding to ryslog.
nano /etc/rsyslog.d/50-default.conf
*.*@192.168.56.200:5555
 
== Monitoring Solutions ==
* Netdata
 
* Nagios
=== Netdata Monitoring ===
[[File:Netdata.gif]]
 
Linus Distribution: Debian Linux and its derivatives (including Ubuntu, Mint)
 
'''Install nessesary packages:'''
 
Install the packages for having a basic netdata installation (system monitoring and many applications, without mysql / mariadb, postgres, named, hardware sensors and SNMP):
 
  curl -Ss 'https://raw.githubusercontent.com/firehol/netdata-demo-site/master/install-required-packages.sh' >/tmp/kickstart.sh && bash /tmp/kickstart.sh netdata
 
Install all the required packages for monitoring everything netdata can monitor:
 
  curl -Ss 'https://raw.githubusercontent.com/firehol/netdata-demo-site/master/install-required-packages.sh' >/tmp/kickstart.sh && bash /tmp/kickstart.sh netdata-all
 
  apt-get install zlib1g-dev uuid-dev libmnl-dev gcc make git autoconf autoconf-archive autogen automake pkg-config curl
 
'''Install Netdata'''
 
Download it - the directory 'netdata' will be created
  git clone https://github.com/firehol/netdata.git --depth=1
  cd netdata
 
Build it, install it, start it
  ./netdata-installer.sh
 
'''starting netdata at boot'''
 
Copy the netdata startup file to /etc/init.d
  cp system/netdata-lsb /etc/init.d/netdata
 
Make sure it is executable
  chmod +x /etc/init.d/netdata
 
Enable it
  update-rc.d netdata defaults
 
Access to web interface
http://<ipaddress>:19999
 
''Referenced : https://github.com/firehol/netdata/wiki/Installation''
 
=== Nagios Monitoring ===
 
[[File:Nagios.png]]
 
 
'''Choosing Nagios implementation'''
 
Nagios core and Nagios XI. Depending on what version of Linux you are running you might only be able to support Nagios core. On Ubuntu 15 and ahead PHP5 is unavailable which means that Installing Nagios XI is not possible since it depends on PHP5. In this case you have to install Nagios core using PHP7.:
 
 
'''For Nagios XI'''
 
The following steps are for Nagios XI. For Nagios Core install please go below.
Install PHP (version 5):
 
  apt-get install php5-common libapache2-mod-php5 php5-cli
 
 
'''Downloading the Latest Release'''
 
Make sure to download the Nagios XI installation package to the /tmp directory of the server on which you wish to install it, as shown in
the following commands:
 
  cd /tmp
 
To download the latest stable release, use the following command:
 
  wget http://assets.nagios.com/downloads/nagiosxi/xi-latest.tar.gz
 
  tar xzf xi-latest.tar.gz
 
  cd /tmp/nagiosxi
 
  ./fullinstall
 
''Note: It will take around 5 minutes to finish the installation.''
 
You are access the Nagios XI interface by pointing your web browser to:
 
  http://<ipaddress>/nagiosxi OR http://<ipaddress>/nagiosql/install
 
''Note: An empty file named ENABLE_INSTALLER is necessary to create in install/ directory to continue with installation.
 
'''Troubleshooting'''
 
I got the problem with the given IP after finishing installation, it gave me wrong IP to access. But just ignore it and go to your correct IP of your web server:
 
  http://<ipaddress>
 
  without "/nagiosxi"
 
It will show the installation GUI and you just need to follow and login after all.
 
''Referenced: https://assets.nagios.com/downloads/nagiosxi/docs/XI_Manual_Installation_Instructions.pdf''
 
==== Install Plugins ====
 
'''Locate Your Plugin'''
 
Visit this website to download plugins: http://exchange.nagios.org.
 
'''Install Plugin'''
 
Click the Admin menu --> Manage Plugins
 
Click the Browse button --> click the Upload Plugin button.
 
'''Test Your Plugin From The Command Line'''
 
  cd /usr/local/nagios/libexec
 
  ./plugin_name arg1 arg2 ...
 
*See the document of plugin to get more detail of its arguments
 
 
'''Define A Command'''
 
browse to Configure → Core
 
Configuration Manager → Commands and click Add New.
 
Enter a Command Name, usually the same as the plugin
filename. Next in the Command Line field, enter
$USER1$/plugin_name $ARG1$. The $USER1$ macro is
replaced with the path to the plugin directory, while the
$ARG1$ macro is a placeholder for an argument you want to
specify later on a per-service basis. If you want, you can omit
$ARG1$ or add more ($ARG2$, $ARG3$, etc.)
 
 
Click Save --> Apply Configuration.
 
'''Add A Service'''
 
In the left-hand menu
 
Click Services --> click Add New.
 
*'''Config Name''' field, enter the file name where the
configuration will be stored. Next enter a description of the
service in the Description field. Associate the service with a
host by clicking the Manage Host button.
 
*'''Check command''' field, select the command you defined
above from the dropdown list. If you had any argument
placeholders when you defined the command earlier, enter
replacement text in the appropriate argument fields.
 
*'''Manage Templates''' button allows you to apply a template
to this service which will save you from having to fill in all the
other configuration options, or you can enter the required
Check and Alert settings manually.
 
Save --> Apply Configuration again
 
 
'''Verify Your Service Is Working'''
 
Navigate to
 
  Home → Service Detail 
 
Look for your new service in the list. It will probably take a few minutes before it runs its first check.
 
''Referenced: https://assets.nagios.com/downloads/nagiosxi/docs/Managing-Plugins-in-Nagios-XI.pdf
''
 
 
'''For Nagios Core'''
 
Install Nagios on Ubuntu 16.04
 
Step 1. First make sure that all your system packages are up-to-date by running these following apt-get commands in the terminal.
 
sudo apt-get update
sudo apt-get upgrade
 
Step 2. Install LAMP (Linux, Apache, MariaDB, PHP) server.
 
A Ubuntu 16.04 LAMP server is required. If you do not have LAMP installed, you can follow our guide here. Also install all required PHP modules:
 
apt-get install php7.0 openssl perl make php7.0-gd libgd2-xpm-dev libapache2-mod-php7.0 libperl-dev libssl-dev daemon wget apache2-utils unzip
 
Step 3. Create users and groups for Nagios.
 
Now create a new nagios user account and setup a password to this account:
 
 
useradd nagios
groupadd nagcmd
usermod -a -G nagcmd nagios
usermod -a -G nagcmd www-data
 
 
Step 4. Installing Nagios and plugins.
 
First thing to do is to go to Nagios’s download page and download the latest stable version of Nagios, At the moment of writing this article it is version 4.1.1:
 
 
wget  https://assets.nagios.com/downloads/nagioscore/releases/nagios-4.1.1.tar.gz
tar -zxvf /tmp/nagios-4.1.1.tar.gz
cd /tmp/nagios-4.1.1/
 
Perform below steps to compile the Nagios from the source code:
 
 
./configure --with-nagios-group=nagios --with-command-group=nagcmd --with-  httpd_conf=/etc/apache2/sites-enabled/
make all
make install
make install-init
make install-config
make install-commandmode
make install-webconf
 
Next steps, Download latest nagios-plugins source and install using following commands:
 
wget http://www.nagios-plugins.org/download/nagios-plugins-2.1.1.tar.gz
tar xzf nagios-plugins-2.1.1.tar.gz
cd nagios-plugins-2.1.1
./configure --with-nagios-user=nagios --with-nagios-group=nagios
make
make install
 
Step 5. Configure Nagios.
 
Edit the /usr/local/nagios/etc/objects/contacts.cfg config file with your favorite editor and change the email address associated with the nagiosadmin contact definition to the address you’d like to use for receiving alerts.
 
 
nano /usr/local/nagios/etc/objects/contacts.cfg
 
Change the email address field to receive the notification:
 
 
[...]
define contact{
contact_name nagiosadmin ; Short name of userus
generic-contact ; Inherit default values from generic-contact template (defined above)
alias Nagios Admin ; Full name of useremail
admin@idroot.net ; <<***** CHANGE THIS TO YOUR EMAIL ADDRESS ******
[...]
 
Step 6. Configure Apache web server for Nagios.
 
Now create nagios apache2 configuration file:
 
 
nano /etc/apache2/sites-enabled/nagios.conf
 
Edit the following lines if you want to access nagios administrative console from a particular IP series, Here, I want to allow nagios administrative access from 192.168.1.0/24 series only:
 
 
[...]
## Comment the following lines ##
#  Order allow,deny
#  Allow from all
 
## Uncomment and Change lines as shown below ##
Order deny,allow
Deny from all
Allow from 127.0.0.1 192.168.1.0/24
[...]
 
Enable Apache’s rewrite and cgi modules:
 
 
sudo a2enmod rewrite
sudo a2enmod cgi
 
Configure Apache authentication:
 
We need to setup the password for the user nagiosadmin. This username will be used to access the web interface so it is important to remember the password that you will input here. Set the password running the following command and enter the password twice:
 
 
# sudo htpasswd -s -c /usr/local/nagios/etc/htpasswd.users nagiosadmin
New password:
Re-type new password:
Adding password for user nagiosadmin
 
Restart Apache for the changes to take effect:
 
 
systemctl restart apache2
 
Step 7. Verify and Start Nagios service.
 
Next we have to make Nagios start at boot time, so first verify that the configuration file has no errors running the following command:
 
 
sudo /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
 
And you should get the output:
 
 
[...]
Checking objects...
    Checked 8 services.
    Checked 1 hosts.
    Checked 1 host groups.
    Checked 0 service groups.
    Checked 1 contacts.
    Checked 1 contact groups.
    Checked 24 commands.
    Checked 5 time periods.
    Checked 0 host escalations.
    Checked 0 service escalations.
Checking for circular paths...
    Checked 1 hosts
    Checked 0 service dependencies
    Checked 0 host dependencies
    Checked 5 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...
 
Total Warnings: 0
Total Errors:  0
 
Things look okay - No serious problems were detected during the pre-flight check
[...]
 
Ubuntu 16.04 uses systemd for starting / stopping all the services, so, we need to create nagios.service file:
 
 
nano /etc/systemd/system/nagios.service
 
Add the following lines:
 
 
[Unit]
Description=Nagios
BindTo=network.target
 
[Install]
WantedBy=multi-user.target
 
[Service]
User=nagios
Group=nagios
Type=simple
ExecStart=/usr/local/nagios/bin/nagios /usr/local/nagios/etc/nagios.cfg
 
 
Enable Nagios to start automatically at system startup:
 
 
systemctl enable /etc/systemd/system/nagios.service
 
Now, start Nagios service:
 
systemctl start nagios
 
Step 8. Accessing Nagios.
 
Nagios will be available on HTTP port 80 by default. Open your favorite browser and navigate to http://yourdomain.com/install.php or http://server-ip/install.php and complete the required the steps to finish the installation. When prompted for username and password you will introduce the username “nagiosadmin” and the password that you entered in step 6.
 
= Experiences =
 
'''Graylog'''
 
In during installation time, we have found some problems which will impact on our your installation as well.
 
- The version of Ubuntu and Graylog might conflict each other as well as Graylog's packages ( e.g : java )
 
 
'''Nagios'''
 
I have faced some of certain problem with installing Plugin for Nagios. First of all, you need to check if the plugins you want to use is supported for the OS of the server you are running, for example the plugin is written for Windows Server but you are running Linux server, thus it will give you an error about its format or bunch of error which is difficult to determine exactly its trouble.
 
Furthermore, once you wish to add the plugin as a service. Please pay attention on the "Service Name" and "Manage Host" which might effect to your installation as well.


= Summary =
= Summary =
'''Graylog'''
During the installation Graylogs in both versions of Ubuntu ( 14.04 and 16.0 ). We realized that it had many differences between these versions. Listed above our objectives what we expect to achieve after installing and using Graylog, along with its useful information and its interaction between Administrator (users) with Graylog.
its advantages:
+ Free.
+ Easy to interact with web interface.
+ Easy to install with the good support from its sources.
+ Help Administrator (user) to collect information in during logging and monitoring straightforwardly.
+ Many useful tools ( plugins ) which supports to work on.
'''Nagios'''
The reason I choose Nagios as my monitoring solution is because it is quite fully functions which help people can handle the services on the server for example it points out detail notification of crucial services such as manage users, HTTP, SSH..so forth. I am using a trial version, but it's seem pretty enough for beginner to "hangout" with Monitoring tool at first.
This is my opinion of its advantages:
+ Easy to install with many ways with official guide from Nagios Supports.
+ Good trial version to barely use for beginner.
+ There are some useful extensions, plugins which are free and easy to install.
+ Easy to handle with web interface.
+ Does not take many space of hard disk.
Another hand, it also has disadvantages:
+ Too expensive for license version.
+ Lack of plugins development, some of them is quite old as well.


= References =
= References =
''http://www.itzgeek.com/how-tos/linux/ubuntu-how-tos/how-to-install-graylog2-on-ubuntu-14-04.html''
''https://assets.nagios.com/downloads/nagiosxi/docs/Managing-Plugins-in-Nagios-XI.pdf''
''https://assets.nagios.com/downloads/nagiosxi/docs/XI_Manual_Installation_Instructions.pdf''
''https://github.com/firehol/netdata/wiki/Installation''
''http://docs.graylog.org/en/2.1/''
''https://assets.nagios.com/downloads/nagioscore/docs/Installing_Nagios_Core_From_Source.pdf''
''http://idroot.net/linux/install-nagios-ubuntu-16-04/''

Latest revision as of 11:09, 15 December 2016

Logging and Monitoring with Graylog

Course: Logging and Monitoring - Lecturer: Margus Ernits

Group: Cyber Security Engineering (C21)

Team members: Ender Phan, Kustas Kurval, Sheela Gowry Sumathi Raju, Artur Vincent Kerge

Page created on: October 05, 2016

Abstract

In order to understand how to set up the Graylog service as well as understand its crucial roles. We decided to choose Graylog as our application for logging and monitoring. Below are our objectives which would be expected to achieve later on:

  • How to install Graylog on Ubuntu 14.04.
  • How to use Graylog to protect servers.
  • Upgrading and configuring Graylog at first, and know how to secure Graylog.
  • Threats and security during logging.
  • How to install Netdata and Monitoring with Netdata.
  • How to install Nagios and Monitoring with Nagios.
  • Trouble shooting and experiences.

Installation Guide

Ubuntu 14.04


Logging Solutions

Since the Elasticsearch is based on java, we would require to install either openJDK or Oracle JDK. It is recommended to install Oracle JDK, verify the java version by using the following command.

Remove the OpenJDK from the system, if you have it already installed. From now on it is presumed that user has root privileges. 

apt-get remove --purge openjdk*

Add repository. 

add-apt-repository -y ppa:webupd8team/java

Run the following command to pull the packages information from the newly added repository. 

apt-get update

Issue the following command to install Java jdk 1.8. 

apt-get -y install oracle-java8-installer

java -version

Output: 

Java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

Installing extras

Elasticsearch

Let’s install Elasticsearch, it can be downloaded from the official website.

Download and install GPG signing key. 

wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

Note that the -qO argument is lowercase Quebec followed by capital Oscar.

Save the repository definition to /etc/apt/sources.list.d/elasticsearch.list 

echo "deb http://packages.elastic.co/elasticsearch/2.x/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch.list

Update repository cache and install Elasticsearch 

apt-get update && apt-get install elasticsearch

Configure Elasticsearch to start during system startup. 

update-rc.d elasticsearch defaults

The only important thing is to set a cluster name as “graylog2“, that is being used by graylog. Now edit the configuration file of Elasticsearch. 

nano /etc/elasticsearch/elasticsearch.yml

cluster.name: graylog2

In the same file disable dynamic scripts to avoid remote execution. That can be done by adding the following line:

FOLLOWING LINE GOES INTO GRAYLOG CONF! 

script.disable_dynamic: true

Once it is done, we are good to go. Before that, restart the Elasticsearch services to load the modified configuration. 

service elasticsearch restart

Wait at least a minute to let the Elasticsearch get fully restarted, otherwise testing will fail. Elastisearch should be now listen on 9200 for processing HTTP request, we can use CURL to get the response. Ensure that it returns with cluster name as “graylog2” 

curl -X GET http://localhost:9200

{
  "status" : 200,
  "name" : "Pistol",
  "cluster_name" : "graylog2",
  "version" : {
   "number" : "1.7.1",
   "build_hash" : "b88f43fc40b0bcd7f173a1f9ee2e97816de80b19",
   "build_timestamp" : "2015-07-29T09:54:16Z",
   "build_snapshot" : false,
   "lucene_version" : "4.10.4"
 },
 "tagline" : "You Know, for Search"
}

Optional: Use the following command to check the Elasticsearch cluster health, you must get a cluster status as “green” for graylog to work. 

curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'

{
 "cluster_name" : "graylog2",
 "status" : "green",
 "timed_out" : false,
 "number_of_nodes" : 1,
 "number_of_data_nodes" : 1,
 "active_primary_shards" : 0,
 "active_shards" : 0,
 "relocating_shards" : 0,
 "initializing_shards" : 0,
 "unassigned_shards" : 0,
 "delayed_unassigned_shards" : 0,
 "number_of_pending_tasks" : 0,
 "number_of_in_flight_fetch" : 0
}

MongoDB

MongoDB is available in deb format and same can be downloaded from the official website. Add the following repository information on the system to install MongoDB. Before that we must import public key. 

apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10

Add repository by creating the /etc/apt/sources.list.d/mongodb-org-3.0.list list file using the command. 

echo "deb http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.0.list

Update repository cache. 

apt-get update

Install MongoDB using the following command. 

apt-get install mongodb-org

Start the MongoDB service and enable it to start automatically during the system start-up. 

service mongod start

ln -s /usr/bin/mongod /etc/init.d/mongod

Create a symbolic link 

update-rc.d mongod defaults

Graylog2

Graylog-server accepts and process the log messages, also spawns the RESTAPI for the requests that comes from graylog-web-interface. Download the latest version of graylog from graylog.org,

Use the following command to install graylog2 repository. 

wget https://packages.graylog2.org/repo/packages/graylog-1.2-repository-ubuntu14.04_latest.deb

dpkg -i graylog-1.2-repository-ubuntu14.04_latest.deb

Install https suppport and update the repository cache. 

apt-get install apt-transport-https

apt-get update

Install Graylog server using following command. 

apt-get install graylog-server

Edit the server.conf file. 

nano /etc/graylog/server/server.conf

Configure the following variables in the above file.

Set a secret to secure the user passwords, use the following command to generate a secret, use at least 64 character’s. 

pwgen -N 1 -s 96

OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP

If you get a “pwgen: command not found“, use the following command to install pwgen. 

apt-get install pwgen

Place the secret. 

password_secret = OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP

Next is to set a hash password for the root user (not to be confused with system user, root user of graylog is admin). You will use this password for login into the web interface, admin’s password can not be changed using web interface, must edit this variable to set.

Replace “yourpassword” with the choice of your’s. 

echo -n yourpassword | sha256sum

e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951

Place the hash password. 

root_password_sha2 = e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951

You can setup email address root (admin) user. 

root_email = "cyber.web@gmail.com"

Set timezone of root (admin) user. 

root_timezone = UTC

Graylog will try to find the Elasticsearch nodes automatically, it uses multicast mode for the same. But when it comes to larger network, it is recommended to use unicast mode which is best suited one for production setups. So add the following two entries to graylog server.conf file, replace ipaddress with live hostname or ipaddress. Multiple hosts can be added with comma separated. 

elasticsearch_http_enabled = false
elasticsearch_discovery_zen_ping_unicast_hosts = ipaddress:9300

Set only one master node by defining the below variable, default setting is true, you must set it as a false to make the particular node as a slave. Master node performs some periodic tasks that slave won’t perform. 

is_master = true

The following variable sets the number of log messages to keep per index, it is recommended to have several smaller indices instead of larger ones. 

elasticsearch_max_docs_per_index = 20000000

The following parameter defines to have total number of indices, if the this number is reached old index will be deleted. 

elasticsearch_max_number_of_indices = 20

Shards setting is really depends on the number of nodes in the Elasticsearch cluster, if you have only one node, set it as 1. 

elasticsearch_shards = 1

The number of replicas for your indices, if you have only one node in Elasticsearch cluster; set it as 0. 

elasticsearch_replicas = 0

Restart Graylog service. 

service graylog-server restart

Enable auto start of graylog server service during system startup. 

update-rc.d graylog-server defaults

You can check out the server startup logs, it will be useful for you to troubleshoot graylog in case of any issue. 

tailf /var/log/graylog-server/server.log

On successful start of graylog-server, you should get the following message in the log file. 

2015-09-17T09:35:22.895+02:00 INFO  [ServerBootstrap] Graylog server up and running.

Install Graylog web interface:

To configure graylog-web-interface, you must have at least one graylog-server node. Install Graylog web interface using “apt-get”. 

apt-get install graylog-web

Edit the configuration file and set the following parameters. 

nano /etc/graylog/web/web.conf

This is the list of graylog-server nodes, you can add multiple nodes, separate by commas. 

graylog2-server.uris="http://127.0.0.1:12900/"

Set the application scret and can be generated using pwgen -N 1 -s 96.

application.secret="sNXyFf6B4Au3GqSlZwq7En86xp10JimdxxYiLtpptOejX6tIUpUE4DGRJOrcMj07wcK0wugPaapvzEzCYinEWj7BOtHXVl5Z"

Set Web interface timezone. 

Timezone="Europe/Tallinn"

Restart the gralog-web-interface using following command, 

service graylog-web restart

Enable auto start of web interface service during system startup. 

update-rc.d graylog-web defaults

Add log forwarding to ryslog. 

nano /etc/rsyslog.d/50-default.conf
*.*@192.168.56.200:5555

Monitoring Solutions

  • Netdata
  • Nagios

Netdata Monitoring

Linus Distribution: Debian Linux and its derivatives (including Ubuntu, Mint)

Install nessesary packages:

Install the packages for having a basic netdata installation (system monitoring and many applications, without mysql / mariadb, postgres, named, hardware sensors and SNMP): 

 curl -Ss 'https://raw.githubusercontent.com/firehol/netdata-demo-site/master/install-required-packages.sh' >/tmp/kickstart.sh && bash /tmp/kickstart.sh netdata

Install all the required packages for monitoring everything netdata can monitor: 

 curl -Ss 'https://raw.githubusercontent.com/firehol/netdata-demo-site/master/install-required-packages.sh' >/tmp/kickstart.sh && bash /tmp/kickstart.sh netdata-all

 apt-get install zlib1g-dev uuid-dev libmnl-dev gcc make git autoconf autoconf-archive autogen automake pkg-config curl

Install Netdata

Download it - the directory 'netdata' will be created 

 git clone https://github.com/firehol/netdata.git --depth=1
 cd netdata

Build it, install it, start it 

 ./netdata-installer.sh

starting netdata at boot

Copy the netdata startup file to /etc/init.d 

 cp system/netdata-lsb /etc/init.d/netdata

Make sure it is executable 

 chmod +x /etc/init.d/netdata

Enable it 

 update-rc.d netdata defaults

Access to web interface 

http://<ipaddress>:19999

Referenced : https://github.com/firehol/netdata/wiki/Installation

Nagios Monitoring


Choosing Nagios implementation

Nagios core and Nagios XI. Depending on what version of Linux you are running you might only be able to support Nagios core. On Ubuntu 15 and ahead PHP5 is unavailable which means that Installing Nagios XI is not possible since it depends on PHP5. In this case you have to install Nagios core using PHP7.:


For Nagios XI

The following steps are for Nagios XI. For Nagios Core install please go below. Install PHP (version 5): 

 apt-get install php5-common libapache2-mod-php5 php5-cli


Downloading the Latest Release

Make sure to download the Nagios XI installation package to the /tmp directory of the server on which you wish to install it, as shown in the following commands: 

 cd /tmp

To download the latest stable release, use the following command: 

 wget http://assets.nagios.com/downloads/nagiosxi/xi-latest.tar.gz

 tar xzf xi-latest.tar.gz

 cd /tmp/nagiosxi

 ./fullinstall

Note: It will take around 5 minutes to finish the installation.

You are access the Nagios XI interface by pointing your web browser to: 

 http://<ipaddress>/nagiosxi OR http://<ipaddress>/nagiosql/install

Note: An empty file named ENABLE_INSTALLER is necessary to create in install/ directory to continue with installation.

Troubleshooting

I got the problem with the given IP after finishing installation, it gave me wrong IP to access. But just ignore it and go to your correct IP of your web server: 

 http://<ipaddress>

 without "/nagiosxi"

It will show the installation GUI and you just need to follow and login after all.

Referenced: https://assets.nagios.com/downloads/nagiosxi/docs/XI_Manual_Installation_Instructions.pdf

Install Plugins

Locate Your Plugin

Visit this website to download plugins: http://exchange.nagios.org.

Install Plugin 

Click the Admin menu --> Manage Plugins 

Click the Browse button --> click the Upload Plugin button.

Test Your Plugin From The Command Line 

 cd /usr/local/nagios/libexec
 
 ./plugin_name arg1 arg2 ...
  • See the document of plugin to get more detail of its arguments


Define A Command 

browse to Configure → Core

Configuration Manager → Commands and click Add New.

Enter a Command Name, usually the same as the plugin filename. Next in the Command Line field, enter $USER1$/plugin_name $ARG1$. The $USER1$ macro is replaced with the path to the plugin directory, while the $ARG1$ macro is a placeholder for an argument you want to specify later on a per-service basis. If you want, you can omit $ARG1$ or add more ($ARG2$, $ARG3$, etc.) 


Click Save --> Apply Configuration.

Add A Service

In the left-hand menu 

Click Services --> click Add New.
  • Config Name field, enter the file name where the

configuration will be stored. Next enter a description of the service in the Description field. Associate the service with a host by clicking the Manage Host button.

  • Check command field, select the command you defined

above from the dropdown list. If you had any argument placeholders when you defined the command earlier, enter replacement text in the appropriate argument fields.

  • Manage Templates button allows you to apply a template

to this service which will save you from having to fill in all the other configuration options, or you can enter the required Check and Alert settings manually. 

Save --> Apply Configuration again


Verify Your Service Is Working

Navigate to 

 Home → Service Detail

Look for your new service in the list. It will probably take a few minutes before it runs its first check.

Referenced: https://assets.nagios.com/downloads/nagiosxi/docs/Managing-Plugins-in-Nagios-XI.pdf


For Nagios Core

Install Nagios on Ubuntu 16.04

Step 1. First make sure that all your system packages are up-to-date by running these following apt-get commands in the terminal. 

sudo apt-get update
sudo apt-get upgrade

Step 2. Install LAMP (Linux, Apache, MariaDB, PHP) server.

A Ubuntu 16.04 LAMP server is required. If you do not have LAMP installed, you can follow our guide here. Also install all required PHP modules: 

apt-get install php7.0 openssl perl make php7.0-gd libgd2-xpm-dev libapache2-mod-php7.0 libperl-dev libssl-dev daemon wget apache2-utils unzip

Step 3. Create users and groups for Nagios.

Now create a new nagios user account and setup a password to this account: 


useradd nagios
groupadd nagcmd
usermod -a -G nagcmd nagios
usermod -a -G nagcmd www-data


Step 4. Installing Nagios and plugins.

First thing to do is to go to Nagios’s download page and download the latest stable version of Nagios, At the moment of writing this article it is version 4.1.1: 


wget  https://assets.nagios.com/downloads/nagioscore/releases/nagios-4.1.1.tar.gz
tar -zxvf /tmp/nagios-4.1.1.tar.gz
cd /tmp/nagios-4.1.1/

Perform below steps to compile the Nagios from the source code: 


./configure --with-nagios-group=nagios --with-command-group=nagcmd --with-   httpd_conf=/etc/apache2/sites-enabled/
make all
make install
make install-init
make install-config
make install-commandmode
make install-webconf

Next steps, Download latest nagios-plugins source and install using following commands: 

wget http://www.nagios-plugins.org/download/nagios-plugins-2.1.1.tar.gz
tar xzf nagios-plugins-2.1.1.tar.gz
cd nagios-plugins-2.1.1
./configure --with-nagios-user=nagios --with-nagios-group=nagios
make
make install

Step 5. Configure Nagios.

Edit the /usr/local/nagios/etc/objects/contacts.cfg config file with your favorite editor and change the email address associated with the nagiosadmin contact definition to the address you’d like to use for receiving alerts. 


nano /usr/local/nagios/etc/objects/contacts.cfg

Change the email address field to receive the notification:


[...] define contact{ contact_name nagiosadmin ; Short name of userus generic-contact ; Inherit default values from generic-contact template (defined above) alias Nagios Admin ; Full name of useremail admin@idroot.net ; <<***** CHANGE THIS TO YOUR EMAIL ADDRESS ****** [...]

Step 6. Configure Apache web server for Nagios.

Now create nagios apache2 configuration file: 


nano /etc/apache2/sites-enabled/nagios.conf

Edit the following lines if you want to access nagios administrative console from a particular IP series, Here, I want to allow nagios administrative access from 192.168.1.0/24 series only:


[...]

    1. Comment the following lines ##
  2. Order allow,deny
  3. Allow from all
    1. Uncomment and Change lines as shown below ##

Order deny,allow Deny from all Allow from 127.0.0.1 192.168.1.0/24 [...]

Enable Apache’s rewrite and cgi modules: 


sudo a2enmod rewrite
sudo a2enmod cgi

Configure Apache authentication:

We need to setup the password for the user nagiosadmin. This username will be used to access the web interface so it is important to remember the password that you will input here. Set the password running the following command and enter the password twice: 


# sudo htpasswd -s -c /usr/local/nagios/etc/htpasswd.users nagiosadmin
New password:
Re-type new password:
Adding password for user nagiosadmin

Restart Apache for the changes to take effect: 


systemctl restart apache2

Step 7. Verify and Start Nagios service.

Next we have to make Nagios start at boot time, so first verify that the configuration file has no errors running the following command: 


sudo /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

And you should get the output: 


[...]
Checking objects...
   Checked 8 services.
   Checked 1 hosts.
   Checked 1 host groups.
   Checked 0 service groups.
   Checked 1 contacts.
   Checked 1 contact groups.
   Checked 24 commands.
   Checked 5 time periods.
   Checked 0 host escalations.
   Checked 0 service escalations.
Checking for circular paths...
   Checked 1 hosts
   Checked 0 service dependencies
   Checked 0 host dependencies
   Checked 5 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...

Total Warnings: 0
Total Errors:   0

Things look okay - No serious problems were detected during the pre-flight check
[...]

Ubuntu 16.04 uses systemd for starting / stopping all the services, so, we need to create nagios.service file: 


nano /etc/systemd/system/nagios.service

Add the following lines: 


[Unit]
Description=Nagios
BindTo=network.target

[Install]
WantedBy=multi-user.target

[Service]
User=nagios
Group=nagios
Type=simple
ExecStart=/usr/local/nagios/bin/nagios /usr/local/nagios/etc/nagios.cfg


Enable Nagios to start automatically at system startup: 


systemctl enable /etc/systemd/system/nagios.service

Now, start Nagios service: 

systemctl start nagios

Step 8. Accessing Nagios.

Nagios will be available on HTTP port 80 by default. Open your favorite browser and navigate to http://yourdomain.com/install.php or http://server-ip/install.php and complete the required the steps to finish the installation. When prompted for username and password you will introduce the username “nagiosadmin” and the password that you entered in step 6.

Experiences

Graylog

In during installation time, we have found some problems which will impact on our your installation as well.

- The version of Ubuntu and Graylog might conflict each other as well as Graylog's packages ( e.g : java )


Nagios

I have faced some of certain problem with installing Plugin for Nagios. First of all, you need to check if the plugins you want to use is supported for the OS of the server you are running, for example the plugin is written for Windows Server but you are running Linux server, thus it will give you an error about its format or bunch of error which is difficult to determine exactly its trouble.

Furthermore, once you wish to add the plugin as a service. Please pay attention on the "Service Name" and "Manage Host" which might effect to your installation as well.

Summary

Graylog

During the installation Graylogs in both versions of Ubuntu ( 14.04 and 16.0 ). We realized that it had many differences between these versions. Listed above our objectives what we expect to achieve after installing and using Graylog, along with its useful information and its interaction between Administrator (users) with Graylog.

its advantages: 

+ Free.
+ Easy to interact with web interface.
+ Easy to install with the good support from its sources.
+ Help Administrator (user) to collect information in during logging and monitoring straightforwardly.
+ Many useful tools ( plugins ) which supports to work on.

Nagios

The reason I choose Nagios as my monitoring solution is because it is quite fully functions which help people can handle the services on the server for example it points out detail notification of crucial services such as manage users, HTTP, SSH..so forth. I am using a trial version, but it's seem pretty enough for beginner to "hangout" with Monitoring tool at first.

This is my opinion of its advantages: 

+ Easy to install with many ways with official guide from Nagios Supports.
+ Good trial version to barely use for beginner.
+ There are some useful extensions, plugins which are free and easy to install.
+ Easy to handle with web interface.
+ Does not take many space of hard disk.

Another hand, it also has disadvantages: 

+ Too expensive for license version.
+ Lack of plugins development, some of them is quite old as well.

References

http://www.itzgeek.com/how-tos/linux/ubuntu-how-tos/how-to-install-graylog2-on-ubuntu-14-04.html

https://assets.nagios.com/downloads/nagiosxi/docs/Managing-Plugins-in-Nagios-XI.pdf

https://assets.nagios.com/downloads/nagiosxi/docs/XI_Manual_Installation_Instructions.pdf

https://github.com/firehol/netdata/wiki/Installation

http://docs.graylog.org/en/2.1/

https://assets.nagios.com/downloads/nagioscore/docs/Installing_Nagios_Core_From_Source.pdf

http://idroot.net/linux/install-nagios-ubuntu-16-04/

Retrieved from "https://wiki.itcollege.ee/index.php?title=Logging_-_Monitoring_C21&oldid=115861"