BASH shell en

Using shell scripting to automate Linux maintenance tasks

Author

Translated from Estonian by Steven Rugam

Short introduction

Bash (Bourne-again shell) is a well-known shell. Bash scripts are used to simplify and automate system work. When talking about bash in few words, then the structure of the language and possible usages are being checked. At length, however, focus lies on the certain administrating assignments which are solved with bash scripting language. It is a powerful shell, and features, among other things:

• • Command line editing
  • Command history
  • A directory stack (pushd, popd)
  • Command substitution
  • Special variables, like $PPID
  • Autocompletion
  • In-process integer arithmetic: $((...))
  • In-process regexes
  • Aliases
  • Functions
  • Arrays
  • Expansions: tilde, brace, variable
  • Substring awesomeness
  • Conditional expressions
  • Security (restricted shell mode)
  • Job Control
  • Timing
  • Prompt customization

Purpose

Providing skills for the bash scripting language to simplify and to improve efficiency towards our daily work.

Prerequisites/Assumptions

History with GNU/Linux operation system and acquired skills, which are being tested by automated self-test.

To test prerequisites, you should pass self-test.

Target group

Support engineers and system administrators.

Educational materials for learning BASH shell

Bash for the beginners

Additional learning materials

https://bash.cyberciti.biz/guide/Main_Page

Using Bash for administrating

Useful commands

Command: find

Search for files in the given directory, hierarchically starting at the parent directory and moving to sub-directories.

root@user:~# find -name *.sh

Command: grep

root@user:~# grep user /etc/passwd 
user:x:1000:1000:user,,,:/home/user:/bin/bash

Ignore word case and all other combination with ‘-i‘ option.

root@user:~# grep -i USER /etc/passwd 
user:x:1000:1000:User,,,:/home/user:/bin/bash

Search recursively (-r) i.e. read all files under each directory for a string “127.0.0.1“.

root@user:~# grep -r "127.0.0.1" /etc/

Command: man

The ‘man‘ is the system’s manual pager. Man provides online documentation for all the possible options with a command and its usages. Almost all the command comes with their corresponding manual pages. For example,

root@user:~# man bash

Manual page for man page itself, similarly ‘man cat‘ (Manual page for cat command) and ‘man ls‘ (Manual page for command ls).

Command: ps

ps (Process) gives the status of running processes with a unique Id called PID.

root@user:~# ps

To list status of all the processes along with process id and PID, use option ‘-A‘.

root@user:~# ps -A

Note: This command is very useful when you want to know which processes are running or may need PID sometimes, for process to be killed. You can use it with ‘grep‘ command to find customised output. For example,

root@user:~# ps -A | grep -i ssh

Here ‘ps‘ is pipelined with ‘grep‘ command to find customised and relevant output of our need.

Command: kill

You need a process’s pid (ps) to kill it. Let suppose you want to kill program ‘apache2‘ that might not be responding. Run ‘ps -A‘ along with grep command.

root@user:~# ps -A | grep -i apache2
1285 ?        00:00:00 apache2

Find process ‘apache2‘, note its pid and kill it. For example, in that case ‘apache2‘ pid is ‘1285‘.

root@user:~# kill 1285 (to kill the process apache2)

Note: Every time you re-run a process or start a system, a new pid is generated for each process and you can know about the current running processes and its pid using command ‘ps‘. Another way to kill the same process is.

root@user:~# pkill apache2

Note: Kill requires job id / process id for sending signals, where as in pkill, you have an option of using pattern, specifying process owner, etc.

Command: whereis

The ‘whereis‘ command is used to locate the Binary, Sources and Manual Pages of the command. For example, to locate the Binary, Sources and Manual Pages of the command ‘ls‘ and ‘kill‘.

root@user:~# whereis ls

root@user:~# whereis kill

Note: This is useful to know where the binaries are installed for manual editing sometimes.

Command: service

The ‘service‘ command controls the Starting, Stopping or Restarting of a ‘service‘. This command make it possible to start, restart or stop a service without restarting the system, for the changes to be taken into effect.

Startting an apache2 server on Ubuntu

root@user:~# service apache2 start

Restarting a apache2 server on Ubuntu

root@user:~# service apache2 restart

Stopping a apache2 server on Ubuntu

Stopping a apache2 server on Ubuntu

Note: All the process script lies in ‘/etc/init.d‘, and the path might needs to be included on certain system, i.e., in spite of running “service apache2 start” you would be asked to run “/etc/init.d/apache2 start”.

Command: alias

alias is a built in shell command that lets you assign name for a long command or frequently used command. For example.

alias root='sudo -i'

Command: df

Report disk usages of file system. Useful for user as well as System Administrator to keep track of their disk usages. ‘df‘ works by examining directory entries, which generally are updated only when a file is closed.

root@user:~# df
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/sda1       47929224 7811908  37675948  18% /
none                   4       0         4   0% /sys/fs/cgroup
udev             1005916       4   1005912   1% /dev
tmpfs             202824     816    202008   1% /run
none                5120       0      5120   0% /run/lock
none             1014120     628   1013492   1% /run/shm
none              102400      44    102356   1% /run/user
/dev/sda5         184307   79852     94727  46% /boot
/dev/sda7       95989516   61104  91045676   1% /data
/dev/sda8       91953192   57032  87218528   1% /personal

Command: du

Estimate file space usage. Output the summary of disk usages by ever file hierarchically, i.e., in recursive manner.

root@user:~# du
308   ./.local/share/Trash/files/jdk1.8.0_121/jre/lib/desktop/icons
344   ./.local/share/Trash/files/jdk1.8.0_121/jre/lib/desktop
4     ./.local/share/Trash/files/jdk1.8.0_121/jre/lib/desktop/applet
548   ./.local/share/Trash/files/jdk1.8.0_121/jre/lib/desktop/oblique-fonts
2048  ./.local/share/Trash/files/jdk1.8.0_121/jre/lib/desktop/fonts

Note: ‘df‘ only reports usage statistics on file systems, while ‘du‘, on the other hand, measures directory contents.

Bash scripts

Cpu monitoring

#!/bin/bash
while [ true ] ;do
used=`free -m |awk 'NR==3 {print $4}'`

if [ $used -lt 1000 ] && [ $used -gt 800 ]; then
echo "Free memory is below 1000MB. Possible memory leak!!!" | /bin/mail -s "HIGH MEMORY ALERT!!!" user@itcollege.ee


fi
sleep 5
done

Adding new users to a Linux system

This script allows the root user or admin to add new users to the system in an easier way by just typing the user name and password (The password is entered in an encrypted manner).

#!/bin/bash
# Script to add a user to Linux system
if [ $(id -u) -eq 0 ]; then
    read -p "Enter username : " username
    read -s -p "Enter password : " password
    egrep "^$username" /etc/passwd >/dev/null
    if [ $? -eq 0 ]; then
        echo "$username exists!"
        exit 1
    else
        pass=$(perl -e 'print crypt($ARGV[0], "password")' $password)
        useradd -m -p $pass $username
        [ $? -eq 0 ] && echo "User has been added to system!" || echo "Failed to add a user!"
    fi
else
    echo "Only root may add a user to the system"
    exit 2
fi

Login Script

When bash is invoked, it runs /etc/profile if that file exists. Next, it looks for these files (in this order) ~/.bash_profile, ~/.bash_login , and ~/.profile. The first one that is found gets executed (any others are ignored). In the case of Ubuntu, ~/.profile is found and executed. (All of ~/.bash_profile, ~/.bash_login , and ~/.profile are ignored if --noprofile is used as an option to the underlying bash call). So for example, if you login via a virtual console, or if you change to another user e.g:

sudo su student2

or if u run:

bash --login

or if you login to a machine via ssh e.g:

ssh user@machine

You'll be invoking ~/.profile Note that ~/.profile (or if not there, one of ~/.bash_profile or ~/.bash_login) normally contains relevant commands to run another script called ~/.bashrc (if said file exists):

if [ -f ~/.bashrc ]; then
    . ~/.bashrc
fi

=Checking Server Utilization

Checking the server utilization is one of the important task of an administrator, and a good administrator is one who knows how to automate his day to day task. Below is the script that will give many such information about your server

#!/bin/bash
date;
echo "uptime:"
uptime
echo "Currently connected:"
w
echo "--------------------"
echo "Last logins:"
last -a |head -3
echo "--------------------"
echo "Disk and memory usage:"
df -h | xargs | awk '{print "Free/total disk: " $11 " / " $9}'
free -m | xargs | awk '{print "Free/total memory: " $17 " / " $8 " MB"}'
echo "--------------------"
start_log=`head -1 /var/log/messages |cut -c 1-12`
oom=`grep -ci kill /var/log/messages`
echo -n "OOM errors since $start_log :" $oom
echo ""
echo "--------------------"
echo "Utilization and most expensive processes:"
top -b |head -3
echo
top -b |head -10 |tail -4
echo "--------------------"
echo "Open TCP ports:"
nmap -p- -T4 127.0.0.1
echo "--------------------"
echo "Current connections:"
ss -s
echo "--------------------"
echo "processes:"
ps auxf --width=200
echo "--------------------"
echo "vmstat:"
vmstat 1 5

Topics

• Bash in general
  • What is bash and shell
  • Bash script writing
  • Executing bash scripts
• Bash configuration files
  • .bash_profile: .bash_login ja .profile
  • .bashrc
  • .bash_logout
  • /etc/profile, /etc/profile.d/
  • Bash command history in .bash_history
• Bash Syntax
  • Commenting
  • Symbols with special meaning(metacharacters, escaping characters, wildcards)
  • Difference between quotation marks and single quotation mark
  • Parameters
  • Catching user input and interactive scripts
  • Exit codes
  • Multiple commands on the same row
  • Piping
  • If sentences (if-else, case)
  • Cycles (for, while, until, select)
  • Control of strings (larger than, larger/smaller than or equal, smaller than)
  • File descriptors (stdin, stdout, stderr)
  • File existance scripts
  • Functions
  • Arithmetic operations, rounding
  • Regular expressions
  • Usage of arrays
• Basics of bash
  • Utilities and their usage (echo, cat, cut, wc, sed, awk, mv, cp, mkdir, ls, file, head, uniq, tail, etc.)
  • Processes and executing them (ps, pstree, top, &)
  • Detecting shell script mistakes and fixing them

Example exercises

• Calculation exercises
• String manipulations
• Exercises with files
• Writing scripts, which show system configurations (logged in users, OS version, RAM info, Partitions etc)
• Stopping processes and restarting processes
• Adding new domains to DNS servers from the shell
• Converting file lowercase characters to uppercase characters

Links

http://gnome-look.org/content/show.php/Ultimate+Bashrc+File?content=129746 hacking .bashrc