Auditd

Auditd - is the userspace component of the Linux Auditing System. It is responsible for writing audit records to the disk. The Linux Auditing System helps system administrators to create an audit trail, a log for every action on the server. Since Auditd operates at the kernel level, this gives system administrators an opportunity to get detailed information about any system operation. The Linux Audit System provides a way to track security-relevant information on the system. Based on pre-configured rules, Auditd generates log entries to record as much information about the events that are happening on the system as possible. System administrators can choose which actions on the server should be monitored and to what extent. This information is crucial for mission-critical environments to determine the violator of the security policy and the actions they performed. Auditd does not provide additional security to the system, rather it can be used to discover violations of security policies used on the system. These violations can further be prevented by additional security measures such as SELinux. The Linux Audit System is developed and maintained by RedHat. For using Linux Auditing System the user should have system administration experience and auditd packages installed on the system. For executing auditd-related commands the root permissions are required.

Usage and use cases

Usage

Auditd is a powerful tool for system administrators as it allows them to monitor access to any file, network traffic and almost everything they would need. The level of detail is phenomenal, since the daemon operates at kernel level and the granularity of the information has been proven to be very useful. The following list summarizes some of the information that Audit can record in its log files:

• Date and time, type, and outcome of an event.

• Sensitivity labels of subjects and objects.

• Association of an event with the identity of the user who triggered the event.

• All modifications to Audit configuration and attempts to access Audit log files.

• All uses of authentication mechanisms, such as SSH, Kerberos, and others.

• Changes to any trusted database, such as /etc/passwd.

• Attempts to import or export information into or from the system.

• Include or exclude events based on user identity, subject and object labels, and other attributes.

The use of Audit system is also a requirement for a number of security related certifications:

• Controlled Access Protection Profile (CAPP)

• Labeled Security Protection Profile (LSPP)

• Rule Set Base Access Control (RSBAC)

• National Industrial Security Program Operating Manual (NISPOM)

• Federal Information Security Management Act (FISMA)

• Payment Card Industry — Data Security Standard (PCI-DSS)

• Security Technical Implementation Guides (STIG)

Use cases

Watching file access

Audit can track whether a file or a directory has been accessed, modified, executed, or the file's attributes have been changed. This is useful, for example, to detect access to important files and have an Audit trail available in case one of these files is corrupted. For example, if we create a test file called test.txt in the home directory and with auditctl utility define the following rule:

# auditctl -w ~/test.txt -p w -k test_changes

we will get the following log messages in our log file:

In the end of SYSCALL type log messages we can see string key="test_changes" which means that this message was triggered by the rule that we defined in the previous step.

Monitoring system calls

Audit can be configured to generate a log entry every time a particular system call is used. This can be used, for example, to track changes to the system time by monitoring the settimeofday, clock_adjtime, and other time-related system calls. For example, if we define the following rule:

# auditctl -a always,exit -F arch=b64 -S clock_settime -k time_change

we get the following log message in the log file:

Recording commands run by a user

Because Audit can track whether a file has been executed, a number of rules can be defined to record every execution of a particular command. For example, a rule can be defined for every executable in the /bin directory. The resulting log entries can then be searched by user ID to generate an audit trail of executed commands per user. For example, we can track all the commands executed by the user if we define the following rule:

# auditctl -a always,exit -F arch=b64 -S execve -k command-executed

After executing rm command to remove a test.txt file created in previous examples, we get the following log messages in the log file:

Searching for events

Audit provides the ausearch utility, which can be used to filter the log entries and provide a complete audit trail based on a number of conditions. For example, if we define a rule for watching test.txt file changes, change the file content and then run the following command:

# ausearch -ul 0

Based on the UID, in our case 0 which is an UID of root, we will get the following output:

Running summary reports

The aureport utility can be used to generate, among other things, daily reports of recorded events. A system administrator can then analyze these reports and investigate suspicious activity furthermore. For example, we can run the following command:

# aureport --summary

We will get the following output:

Monitoring network access

The iptables and ebtables utilities can be configured to trigger Audit events, allowing system administrators to monitor network access.

Audit system architecture

Audit components

Audit System components

The following list describes Audit system components and their functionalities:

auditd

The audit daemon is responsible for writing the audit messages that were generated through the audit kernel interface and triggered by application and system activity to disk. The way the audit daemon is started is controlled by its configuration file, /etc/sysconfig/auditd. The audit system functions (once started) are controlled by /etc/audit/auditd.conf.

auditctl

The auditctl utility controls the audit system. It controls the log generation parameters and kernel settings of the audit interface as well as the rule sets that determine which events are tracked.

audit rules

The file /etc/audit/audit.rules contains a sequence of auditctl commands that are loaded at system boot time immediately after the audit daemon is started.

aureport

The aureport utility allows you to create custom reports from the audit event log. This report generation can easily be scripted, and the output can be used by various other applications, for example, to plot these results.

ausearch

The ausearch utility can search the audit log file for certain events using various keys or other characteristics of the logged format.

audispd

The audit dispatcher daemon (audispd) can be used to relay event notifications to other applications instead of (or in addition to) writing them to disk in the audit log.

autrace

The autrace utility traces individual processes in a fashion similar to strace. The output of autrace is logged to the audit log.

Installation of Audit packages

Audit system uses following packages: audit and audit-libs. These packages are available for almost every major Linux distribution. If the packages are not installed by default, they can be installed with the following commands:

• For Debian, Ubuntu or Linux Mint:

$ sudo apt-get install auditd

• For Fedora, CentOS or RHEL:

$ sudo yum install audit

Starting and configuring Audit service

The Audit daemon can be configured in the /etc/audit/auditd.conf configuration file. This file consists of configuration parameters that modify the behavior of the Audit daemon. Any empty lines or any text following a hash sign (#) is ignored. A complete listing of all configuration parameters and their explanation can be found in the audit.conf(5) man page.

Usually, default configurations are sufficient for most setups, but depending on whether the environment has to satisfy the requirements of 'Controlled Access Protection Profile(CAPP), then the audit daemon configuration needs to be extra restrictive. In this case audit daemon has to be configured with the following settings:

• The directory that holds the Audit log files (usually /var/log/audit/) should reside on a separate partition. This prevents other processes from consuming space in this directory, and provides accurate detection of the remaining space for the Audit daemon.

• The max_log_file parameter, which specifies the maximum size of a single Audit log file, must be set to make full use of the available space on the partition that holds the Audit log files.

• The max_log_file_action parameter, which decides what action is taken once the limit set in max_log_file is reached, should be set to keep_logs to prevent Audit log files from being overwritten.

• The space_left parameter, which specifies the amount of free space left on the disk for which an action that is set in the space_left_action parameter is triggered, must be set to a number that gives the administrator enough time to respond and free up disk space. The space_left value depends on the rate at which the Audit log files are generated.

• It is recommended to set the space_left_action parameter to email or exec with an appropriate notification method.

• The admin_space_left parameter, which specifies the absolute minimum amount of free space for which an action that is set in the admin_space_left_action parameter is triggered, must be set to a value that leaves enough space to log actions performed by the administrator.

• The admin_space_left_action parameter must be set to single to put the system into single-user mode and allow the administrator to free up some disk space.

• The disk_full_action parameter, which specifies an action that is triggered when no free space is available on the partition that holds the Audit log files, must be set to halt or single. This ensures that the system is either shut down or operating in single-user mode when Audit can no longer log events.

• The disk_error_action, which specifies an action that is triggered in case an error is detected on the partition that holds the Audit log files, must be set to syslog, single, or halt, depending on your local security policies regarding the handling of hardware malfunctions.

• The flush configuration parameter must be set to sync or data. These parameters assure that all Audit event data is fully synchronized with the log files on the disk.

Once auditd is properly configured, service can be started to collect Audit information and store it in the log files. To start the srvice, following command should be executed as a root user:

• # service auditd start

Auditd can also be configured to start at boot time using the following command as a root user:

• # chkconfig auditd on

A number of other actions can be performed on auditd using the service auditd 'action' where 'action' can be one of the following:

• stop — stops auditd

• restart — restarts auditd

• reload or force-reload — reloads the configuration of auditd from the /etc/audit/auditd.conf file

• rotate — rotates the log files in the /var/log/audit/ directory

• resume — resumes logging of Audit events after it has been previously suspended, for example, when there is not enough free space on the disk partition that holds the Audit log files

• status — displays the running status of auditd

Defining Audit rules

Audit rules are used to specify which components of the system should be audited. There are three basic types of audit rules:

• Basic audit system parameters

• File and directory watches

• System call audits

Before creating audit rules and rolling them out to the system, system administrators should carefully determine which components to audit as extensive auditing causes substantial logging load. Also, system should provide enough disk space to store large audit logs.

Audit rules can either be passed to the audit system by the command line using auditctl, or defined in a rules file located under /etc/audit/audit.rules.

Defining Audit rules with auditctl utility

The auditctl command allows to control basic functionality of the Audit system. Command supplied by auditctl are not persistent across restarts.

Defining control rules

The following are some of the control rules that allow to modify the behavior of the Audit system:

• -b - sets the maximum amount of existing Audit buffers in the kernel, for example:

# auditctl -b 8192

• -f - sets the action that is performed when the critical error is detected, for example the following command triggers a kernel panic in case of a critical error:

# auditctl -f 2

• -e - enables and disables the Audit system or locks its configuration, for example:

# auditctl -e 2>

• -r - sets the rate of generated messages per second, for example the following command sets no limit for generated messages:

# auditctl -r 0

• -s - reports the status of the Audit system, for example:

# auditctl -s

AUDIT_STATUS: enabled=1 flag=2 pid=0 rate_limit=0 backlog_limit=8192 lost=259 backlog=0

• -l - lists all currently loaded Audit rules, for example:

# auditctl -l

LIST_RULES: exit,always watch=/etc/localtime perm=wa key=time-change

LIST_RULES: exit,always watch=/etc/group perm=wa key=identity

LIST_RULES: exit,always watch=/etc/passwd perm=wa key=identity

LIST_RULES: exit,always watch=/etc/gshadow perm=wa key=identity

• -D - deletes all currently loaded Audit rules

Defining file system rules

For defining a file system rule, the following syntax should be used:

# auditctl -w path_to_file -p permissions -k key_name

where:

• path_to_file is the file or directory that is audited

• permissions are the permissions that are logged

• • r - read access to the file or directory

• • w - write access to the file or directory

• • x - execute access to the file or directory

• • a - change in the file's or directory's attribute

• key_name is an optional string that helps to identify which rule or set of rules generated a particular log

For example the following rule logs all write access to the /etc/passwd file:

# auditctl -w /etc/passwd -p w -k passwd_changes

In the example /etc/passwd is the file we want to log access to, w parameter describes that we want to log write access and passwd_changes is a key which helps us for identifying the rule which generated the log.

Defining system call rules

For defining system call rules the following syntax should be used:

# auditctl -a action,filter -S system_call -F field=value -k key_name

where:

• action and filter specify when a certain event is logged. action can be either always or never. filter specifies which kernel rule-matching filter is applied to the event. The rule-matching filter can be one of the following: task, exit, user, and exclude

• system_call specifies the system call by its name.Several system calls can be grouped into one rule, each specified after the -S option

• field=value specifies additional options that furthermore modify the rule to match events based on a specified architecture, group ID, process ID, and others

• key_name is an optional string that helps to identify which rule or a set of rules generated a particular log entry

Defining persistent Audit rules and controls

For defining Audit rules that are persistent across reboots, the rules should be included in the /etc/audit/audit.rules file which uses the same auditctl command line syntax. Any empty lines or any text following the hash(#) sign is ignored. This file is read during the start of the audit daemon. Here we can see a possible basic configuration of the audit.rules file:

Understanding Audit log files

By default the Audit system log entries are stored in /var/log/audit/audit.log file. Understanding Audit logs is crucial for using the system auditing daemon. The following rule logs every attempt to read or modify the /etc/ssh/sshd_config file:

-w /etc/ssh/sshd_config -p warx -k sshd_config

If the auditd daemon is running, the following command creates a new event in the Audit log file:

# cat /etc/ssh/sshd_config

The event in the audit.log file looks as follows:

The above event consists of three records each starting with the type= keyword. All three records share time stamp and serial number. Each records consists of several name=value pairs separated by a white space or a comma. Each value has its meaning:

type

The type of the event recorded. For a list of all possible values and their explanations see "Audit Record Types"

msg

A time stamp and a unique ID of the record in the form audit(time_stamp:ID)

arch

CPU architecture of the system call

syscall

The type of system call

success

Whether the system call succeeded or failed

exit

The exit value returned by a system call

a0 to a3

First four arguments of the system call encoded in hexadecimal notation

items

The number of strings passed to the application

ppid

The parent process ID

pid

The process ID

auid

The audit ID which is given on user login. This means that even if the user changes his identity you can still trace actions to the original user who logged in

uid

The user ID of the user who started the process

gid

The group ID of the user

euid, suid, fsuid

Effective user ID, set user ID and file system user ID

egid, sgid, fsgid

Effective group ID, set group ID and file system group ID

tty

Terminal from which the application is started

ses

The login session ID

comm

Command line name of the command that was used

exe

The resolved pathname to the binary program

subj

auditd records whether the process is subject to any security context

key

Administrator-defined string associated with the rule that generated the event

item

The item field indicates which item, of the total number of items referenced in the SYSCALL type record, the current record is

name

Full path of the file or directory that was passed to the system call as an argument

inode

Contains the inode number associated with the file or directory recorded in this event

dev

Specifies device on which the file is stored

mode

Numerical representation of the file's access permissions

ouid and ogid

Object owner's user ID and group ID

rdev

Contains a recorded device identifier for special files only

obj

The obj field records the SELinux context with which the recorded file or directory was labeled at the time of execution

This is just a subset of all the possible event fields. For a list of all fields see "Audit Event Fields"

Searching the Audit log files

The ausearch utility allows to search Audit log files for specific events.Ausearch allows to search the audit logs using special keys and search phrases that relate to most of the flags that appear in event messages in /var/log/audit/audit.log.By default, ausearch searches the /var/log/audit/audit.log file. You can specify a different file using the ausearch options -if file_name command. We can search for events with multiple options:

Search by Audit Event ID

An audit event ID is the second part of an audit message ID, which consists of a UNIX epoch time stamp and the audit event ID separated by a colon. All events that are logged from one application's system call have the same event ID. Use this event ID with ausearch to retrieve this event's trail from the log with ausearch -a event_ID command

Search by Message Type

To search for audit records of a particular message type, use the ausearch -m message_type command

Search by Login ID

To view records associated with a particular login user ID, use the ausearch -ul uid command

Search by User ID

View reports related to a particular user ID with ausearch -ui uid. Search for records related to a particular effective user ID, use the ausearch -ue euid

Search by Group ID

View reports related to a particular user ID with ausearch -gi gid. Search for records related to a particular effective group ID, use ausearch -ge egid

Search by Command Line Name

View records related to a certain command, using the ausearch -c comm_name command

Search by Executable Name

View records related to a certain executable with the ausearch -x exe command

Search by System Call Name

View records related to a certain system call with the ausearch -sc syscall command

Search by Process ID

View records related to a certain process ID with the ausearch -p pid command

Search by Event or System Call Success Value

View records containing a certain system call success value with ausearch -sv success_value command

Search by Filename

View records containing a certain filename with ausearch -f filename command

Search by Terminal

View records of events related to a certain terminal only with ausearch -tm term command

Search by Hostname

View records related to a certain remote hostname with ausearch -hn hostname command

Search by Key Field

View records that contain a certain key assigned in the audit rule set to identify events of a particular type with ausearch -k key_field command

Search by Word

View records that contain a certain string assigned in the audit rule set to identify events of a particular type with ausearch -w word command

Limit a search to a Certain Time Frame

Use -ts and -te to limit the scope of your searches to a certain time frame. The -ts option is used to specify the start date and time and the -te option is used to specify the end date and time

Creating Audit reports

The aureport utility allows you to generate summary and columnar reports on the events recorded in Audit log files. To avoid having to read the raw audit log, it is possible to configure custom audit reports with aureport and run them regularly. Because the aureport commands are scriptable, you can easily create custom report scripts to run at certain intervals to gather the audit information. With aureport it is possible to generate various report types:

aureport --summary

Run this report to get a rough overview of the current audit statistics (events, logins, processes, etc.). To get detailed information about any of the event categories listed, run individual reports for the event type.

aureport --success

Run this report to get statistics of successful events on your system.To get detailed information for a particular event type, run the individual report adding the --success option to filter for successful events of this type.

aureport --failed

Run this report to get statistics of failed events on your system.

aureport -l

Run this command to generate a numbered list of all login-related events. The report includes date, time, audit ID, host and terminal used, as well as name of the executable, success or failure of the attempt, and an event ID.

aureport -p

Run this report to generate a numbered list of all process-related events. This command generates a numbered list of all process events including date, time, process ID, name of the executable, system call, audit ID, and event number.

aureport -f

Run this report to generate a numbered list of all file-related events. This command generates a numbered list of all process events including date, time, process ID, name of the executable, system call, audit ID and event number.

aureport -u

Run this report to find out which users are running what executables on your system. This command generates a numbered list of all user-related events including date, time, audit ID, terminal used, host, name of the executable, and an event ID.

aureport -s

This command generates a numbered list of all system call events including date, time, number of the system call, process ID, name of the command that used this call, audit ID, and event number.

aureport -e

This command generates a numbered list of all events including date, time, event number, event type, and audit ID.

aureport -x

This command generates a numbered list of all executable events including date, time, name of the executable, the terminal it is run in, the host executing it, the audit ID, and event number.

Summary

Linux Auditing System is one of the most powerful tools for system administrators for monitoring the whole system. The variety of commands and rules gives administrators incredible flexibility and almost endless possibilities to monitor every single action and event on the system. Using Auditing System makes it easier to find and track malicious users and security policy violators on the system and this gives opportunity to take further actions against violators. As the Auditing System is working on the kernel level, its ability to give an information of incredible granularity, makes this system an essential tool for system administrators and this is one of the main reasons why knowledge of using this system is required for many security-related certifications.

Further Reading

For more information follow the links:

A Brief Introduction to auditd

Customized File Monitoring with Auditd

Linux System Monitoring and More with Auditd

Auditd - Tool for Security Auditing on Linux Server

How To Use the Linux Auditing System on CentOS 7

How to monitor file access on Linux with auditd

Investigating Kernel Return Codes with the Linux Audit System

Other Details

Author: Nika Ptskialadze

Curriculum: Cyber Security Engineering

Group: C11

Date created: March 11, 2017

Last modification: April 7, 2017