Sguil

Author: Kustas Kurval

Cyber Security Engineering C11

Written 06.06.2016

Introduction

This tutorial was made to make an introduction to Sguil. Sguil (pronounced sgweel) is built by network security analysts for network security analysts. It is a collection of free software components for Network Security Monitoring (NSM) and event driven analysis of IDS alerts. Sguil's main component is an intuitive GUI that provides access to realtime events, session data, and raw packet captures. Sguil facilitates the practice of Network Security Monitoring and event driven analysis.

The Sguil client is written in tcl / tk and can be run on any operating system that supports tcl / tk (including Linux, *BSD, Solaris, MacOS, and Win32).

It is provided by Q Public License

Sguil integrates alert data from Snort, session data from SANCP, and full content data from a second instance of Snort running in packet logger mode.

In this introduction I will be covering Sguil in Xbuntu. You will need to know basic Linux syntax and terminology also some terminology concerning overall intrusion detection and prevention systems (IDPS) and overall basic networking.

Software architecture

A sguil system is composed of a single sguil server and an arbitrary number of sguil network sensors. The sensors perform all the security monitoring tasks and feed information back to the server on a regular basis. The server coordinates this information, stores it in a database and communicates with sguil clients running on administrators' desktop machines. It can also issue requests for specific information from the sensors.

Each sensor monitors a single network link (although you can have multiple sensors on one physical machine). They collect several different types of information:

1. Snort monitors the link for security events, and logs them to a file on the local disk.
2. Barnyard takes events from the snort log file and sends them to the sensor agent, which inserts them into database running on the sguil server in near real-time
3. A separate instance of snort logs the full content of all network packets to the local disk (this typically requires a large separate data partition)
4. SANCP records TCP/IP sessions and forwards them to the database on the sguil server
5. The sguil agent also listens for commands from the sguil server. These commands are typically requests for packet data previously logged by Snort.

Tools that usually make up Sguil

Tool	Purpose
MySQL 4.x or 5.x	Data storage and retrieval
Snort 2.x / Suricata	Intrusion detection alerts, scan detection, packet logging
Barnyard / Barnyard2	Decodes IDS alerts and sends them to sguil
SANCP	TCP/IP session records
Tcpflow	Extract an ASCII dump of a given TCP session
p0f	Operating system fingerprinting
tcpdump	Extracts individual sessions from packet logs
Wireshark	Packet analysis tool (used to be called Ethereal)

Contents

Since Sguil is dependant on many other types of software to gather, facilitate, store ,decode and analyze I will be using Xbuntu based distribution Security Onion which saves massive amount of time to set up the entire environment. Security Onion has all this and more build in and is able to quickly configure which software to tie to Sguil.

Setup

• First, review the Hardware Requirements page.
• Review the Release Notes page.
• Download and verify our Security Onion ISO image.
• Boot the ISO image and select the Install option.

I Installed this on Oracle Virutalbox as a 64 bit Ubuntu operating system, with 4GB of memory and a single processor. I set the network adapter as bridged with promiscuous mode allowed. This ensured that I am able to capture network traffic from the host machine

• Follow the prompts in the Xubuntu installer. If prompted with an encrypt home folder or encrypt partition option, DO NOT enable this feature. If asked about * * * automatic updates, DO NOT enable automatic updates. Reboot into your new installation. Login using the username/password you specified during installation.
• Verify that you have Internet connectivity. If necessary, configure your proxy settings.
• Install updates and reboot.
• Double-click the Setup icon on the desktop. The Setup wizard will walk you through configuring /etc/network/interfaces and will then reboot.
• After rebooting, log back in and start the Setup wizard again. It will detect that you have already configured /etc/network/interfaces and will walk you through the rest of the configuration. When prompted for Evaluation Mode or Production Mode, choose Evaluation Mode.

Security Onion usually expects at least two networking interfaces. One for monitoring the other for management. Since I only had access to a single interface on the virtual machine I set it as management. I used the static IP address 192.168.1.111 with regular /24 subnet mask for ease of use. After this I was prompted for a gateway address and DNS server.

• Once you've completed the Setup wizard, use the Desktop icons to login to Sguil.

Post Installation

Verify services are running:

sudo service nsm status

If any services are not running, try starting them:

sudo service nsm start

Tuning / Miscellaneous

• Are you monitoring network traffic that has VLAN tags? If so, take a look at our VLAN page.
• If you’re monitoring IP address ranges other than private RFC1918 address space (192.168.0.0/16, 10.0.0.0/8, 172.16.0.0/12), you should update your sensor configuration with the correct IP ranges. Sensor configuration files can be found in /etc/nsm/$HOSTNAME-$INTERFACE/. Modify either snort.conf or suricata.yaml (depending on which IDS engine you chose during sosetup) and update the HOME_NET variable. Also update the home_nets variable in prads.conf. Then update Bro’s network configuration in /opt/bro/etc/networks.cfg. Restart the sensor processes:

sudo nsm_sensor_ps-restart

• If you have Internet access, create an IDS alert by typing the following at a terminal:

curl http://testmyids.com

• As of securityonion-setup - 20120912-0ubuntu0securityonion201, Setup now defaults to only opening port 22 in the firewall. If you need to connect OSSEC agents, syslog devices, or analyst VMs, you can run the new so-allow utility which will walk you through creating firewall rules to allow these devices to connect. For more information, please see the firewall page.
• Full-time analysts should install Security Onion in a VM on their workstation (run through the Ubuntu installer, but do not run our Setup wizard). This gives you a local copy of Wireshark, NetworkMiner, and our customized Sguil client. Launch the Sguil client and connect to the IP/hostname of your production Sguil sensor (you may need to run so-allow as described in the previous step). This allows you to investigate pcaps without fear of impacting your production server/sensors. To change the resolution of your Security Onion VM, install the Virtual Tools for your virtualization solution or use xrandr. For a list of available screen resolutions, simply execute “xrandr”. To set the screen resolution (replace W and H with the actual Width and Height desired):

xrandr -s WxH

• Login to Sguil and review your IDS alerts. Squert and ELSA can be accessed by visiting https://server/ for additional in-depth analysis.
• Run the following to see how your sensor is coping with the load. You should check this on a daily basis to make sure your sensor is not dropping packets. Consider adding it to a cronjob and having it emailed to you (see the “configure email” link below).

sudo sostat | less

• Please note that any IDS/NSM system needs to be tuned for the network it’s monitoring. Please see ManagingAlerts. You should only run the signatures you really care about.
• Also note that you should be looking at and categorizing events every day with the goal being to categorize all events every day. Even if you don’t use the Sguil console for your primary analysis, you need to log into it periodically and F8 old events to keep the real time queue from getting too big. Neglecting to do so will result in database/Sguil issues as the number of uncategorized events continues to increase on a daily basis. Please see the Sguil client page on NSMwiki.
• On the server running the Sguil database, set the DAYSTOKEEP variable in /etc/nsm/securityonion.conf to however many days you want to keep in your archive. The default is 30, but you may need to adjust it based on your organization’s detection/response policy and your available disk space.
• If you enabled http_agent, you should tune it using http_agent.conf. If you're running ELSA, you already have all the Bro HTTP logs available there, so you might want to disable http_agent to avoid duplicating those logs in the Sguil database:

# Terminate the running http_agent

sudo nsm_sensor_ps-stop --only-http-agent

# Disable http_agent

sudo sed -i 's|HTTP_AGENT_ENABLED="yes"|HTTP_AGENT_ENABLED="no"|g' /etc/nsm//sensor.conf

• Disable any unneeded sensor processes
• Tune the number of PF_RING instances for Snort/Suricata and Bro: PF_RING
• Optional:* exclude unnecessary traffic from your monitoring using BPF.
• Optional: add new Sguil user accounts with the following:

sudo nsm_server_user-add

• Optional, but highly recommended: configure Email for alerting and reporting.
• Optional, but highly recommended: place /etc under version control. If your organization doesn't already have a standard version control tool, you can use bazaar, git, etckeeper:

sudo apt-get install etckeeper

• Optional: need “remote desktop” access to your Security Onion sensor or server? We recommend SSH X-Forwarding as shown above, but if you want something more rdp-like, you can install FreeNX or xrdp:

sudo apt-get install xrdp

Please note that we do not support FreeNX or xrdp.

• Read more about the tools contained in Security Onion: Tools

See also

• Sagan
• Intrusion detection system (IDS)
• Intrusion prevention system (IPS)
• Network intrusion detection system (NIDS)
• Metasploit Project
• nmap

References

• Sguil Homepage
• Sguil FAQ
• NSMWiki: The official wiki for the Sguil project.