- 1 Firewalls and VPN/IPSec
- 1.1 General information
- 1.2 Scenario
- 1.3 Setting up virtual machine hosts
- 1.4 Setting up router
- 1.5 Domain names
- 1.6 Monitoring
- 1.7 Teams
- 1.8 Point-to-point tunnels between routers
- 1.9 Generating certificates
- 1.10 Remote logging
- 1.11 Internal DNS
- 2 Boring stuff
Firewalls and VPN/IPSec
Lecturer: Lauri Võsandi
In this course we will attempt to set up a network similar to a corporate network with multiple offices, eg http://docplayer.it/docs-images/20/596222/images/25-0.png
We will use VPN software to connect subnets to each other and we will use VPN software to connect our personal computers to the intranet.
Setting up virtual machine hosts
For this course we have 3 Sun servers, each with 16GB of RAM. In each server we should be able to create 3 or more virtual machines. As host operating system we will install Ubuntu 16.04 server. On disks set up ext4 on mdraid set up in RAID1 configuration.
For virtualization let's use libvirt and virt-manager on your Ubuntu laptops, for Windows and Mac unfortunately there is no decent UI available instead installing web interface such as Kimchi is recommended.
To install libvirt on the server:
apt install libvirt-bin qemu-kvm
Also add the primary user to the libvirt group:
sudo gpasswd -a $USER libvirt
On your laptop first set up SSH keys between the laptop and server with ssh-keygen and ssh-copy-id. And then you can install virt-manager with:
sudo apt install virt-manager
Copy CD ISO files into /var/lib/libvirt/images using scp or FileZilla.
Continue with creating a virtual machine for each service. For Windows 2012 server virtual machines use 2G of RAM and 50G of storage. For Ubuntu 16.04 server installations use 1G of memory and 50G storage. For Ubuntu 16.04 MATE desktop installations use 2G of RAM.
In order to set up virtual switch inside the server use Linux's built-in bridges, start with installing bridge-utils:
apt install bridge-utils
Reconfigure your server's /etc/network/interfaces, replace X with number relevant to your server. Also replace eth0, eth1 and eth2 with the network interfaces available in your machine:
# The loopback network interface auto lo iface lo inet loopback # Wide area network interface auto br-wan iface br-wan inet dhcp # Until we set up router in a VM we will use DHCP so we can have internet access in 417 bridge_ports eth0 # Local area network interface auto br-lan iface br-lan inet static address 172.16.X.1 netmask 255.255.255.0 bridge_ports eth1 # Management interface auto eth2 iface eth2 inet static address 192.168.12.1X netmask 255.255.255.0
When creating virtual machines, configure network as shown in the screenshot below:
This way your VM-s should be able to access the Internet as the physical machine can
Setting up router
On Wednesday 14. September we will configure OpenWrt as a router in a virtual machine. Download the OpenWrt image and uncompress it:
cd /var/lib/libvirt/images/ wget https://downloads.openwrt.org/chaos_calmer/15.05-rc3/x86/kvm_guest/openwrt-15.05-rc3-x86-kvm_guest-combined-ext4.img.gz gunzip openwrt-15.05.1-x86-kvm_guest-combined-ext4.img.gz
Add second network interface to your router's VM. Configure first NIC as connected to br-wan and second one connected to br-lan.
After that you should end up with topology similar to this:
To clarify: 'srv1.office' and 'srv2.office' are the Ubuntu 16.04 servers, you should have configured static IP addresses or set a static lease from the router. The 'router.office' refers to the OpenWrt router you just set up. The router serves IP addresses using DHCP to 'ubuntu-mate.office' Ubuntu MATE 16.04 workstation and 'windows.office' refers to Windows workstation. Your physical server 'host.office' can be accessed as well. The 'office' throughout the diagram refers to your domain name, use abbrevations such as hq, rnd, devops for that.
Arti will be your DNS registrar (like Godaddy or Zone.ee). Currently added DNS records:
- http://www.biz.wut.ee - 184.108.40.206 for Wut Incorporated website
- http://gw.biz.wut.ee - 220.127.116.11 for OpenVPN gateway
- http://wiki.biz.wut.ee - 18.104.22.168 for Wut Inc internal wiki
- http://git.biz.wut.ee - 22.214.171.124 for Wut Inc source code hosting
- http://paste.biz.wut.ee - 126.96.36.199 for Wut Inc code snippets
- http://chat.biz.wut.ee - 188.8.131.52 for Wut Inc IRC chatroom
- http://pad.biz.wut.ee - 184.108.40.206 for Wut Inc etherpad
- http://ca.biz.wut.ee - 220.127.116.11 for Wut Inc certificate authority web endpoint
- http://mail.biz.wut.ee - 18.104.22.168 for MX entry of biz.wut.ee
(Re)configure your services to make use of these DNS records.
It is also possible to access the services with the domain mareti.ee
Use this **only** on the physical hosts.
You can (ab)use Lauri's collectd at http://log.koodur.com/cgp
apt install collectd
Reconfigure service in /etc/collectd/collectd.conf:
FQDNLookup true LoadPlugin syslog LoadPlugin cpu LoadPlugin df LoadPlugin disk LoadPlugin interface LoadPlugin load LoadPlugin memory LoadPlugin network LoadPlugin processes LoadPlugin swap LoadPlugin uptime LoadPlugin users LoadPlugin dns LoadPlugin ping LoadPlugin sensors <Plugin df> FSType rootfs FSType sysfs FSType proc FSType devtmpfs FSType devpts FSType tmpfs FSType fusectl FSType cgroup IgnoreSelected true </Plugin> <Plugin disk> Disk "/[sv]d[a-z]/" </Plugin> <Include "/etc/collectd/collectd.conf.d"> Filter "*.conf" </Include> <Plugin network> Server "22.214.171.124" </Plugin>
DNS: 126.96.36.199, 188.8.131.52
Public IP address (port no 0 = enp6s4f0): 184.108.40.206/24
Management network IP address (port no 1 = enp6s4f1), accessible from robotics club: 192.168.12.10
Internal IP address of the physical server (port no 2 = enp0s9): 172.16.1.1/24
- Hypervisor, access to physical box - (Mohanad)
- BIND9 as public DNS server, also figure out what domain name we should/can use (Arti)
- domain controller, at this point primarily for user accounts (Keijo)
- nginx web server, for company's homepage (Etienne)
- SMB/CIFS fileserver, join to domain (Etienne)
- VPN server for other subnets, presumably OpenVPN (Mohanad Aly)
Research & development
DNS: 220.127.116.11, 18.104.22.168
Public IP address (port no 0 = enp6s4f0): 22.214.171.124/24
Management network IP address (port no 1 = enp6s4f1), accessible from robotics club: 192.168.12.11
Internal IP address of the physical server (port no 2 = enp0s9): 172.16.2.1/24
Team members: Marvin, Madis, Taavi, Berit, Joosep
- Hypervisor, access to physical box - Marvin
- Git hosting, for sharing scripts, set up LDAP to authenticate with domain controller (Madis)
- Wiki, for exchanging information, set up LDAP to authenticate with domain controller and later possibly configure web server to authenticate with Kerberos (Taavi)
- Windows XP workstation, join to domain
- Ubuntu 16.04 MATE workstation, join to domain.
- LTSP server (Joosep)
- OpenVPN connection to headquarters, use shared secret at first, later X509 certificates
- Pastebin, possibly later implement Kerberos support (Berit)
Network interface example file:
auto lo iface lo inet loopback # Wide area network interface (port 0) auto br-wan iface br-wan inet manual bridge_ports enp6s4f0 # Local area network interface (port 3) auto br-lan iface br-lan inet static address 172.16.2.1 gateway 172.16.2.254 dns-nameserver 172.16.2.254 netmask 255.255.255.0 bridge_ports enp0s8 # Management interface (port 1) auto enp6s4f1 iface enp6s4f1 inet static address 192.168.12.11 netmask 255.255.255.0
Openwrt interface file working example /etc/config/network:
config interface 'lan' option ifname 'eth0' option type 'bridge' option proto 'static' option netmask '255.255.255.0' option ip6assign '60' option ipaddr '172.16.2.254' config interface 'wan' option ifname 'eth1' option proto 'static' option ipaddr '126.96.36.199' option gateway '188.8.131.52' option netmask '255.255.255.0' option dns '184.108.40.206 220.127.116.11'
To create poor man's vpn. Install on your computer
apt install sshuttle
sshuttle --dns -HNvr username@server:port
no you should be able to connect local network computers and services. NB! ping is not working with sshuttle you can read more here http://teohm.com/blog/using-sshuttle-in-daily-work/
DNS: 18.104.22.168, 22.214.171.124
Public IP address (port no 0 = enp6s4f0): 126.96.36.199/24
Management network IP address (port no 1 = enp6s4f1), accessible from robotics club: 192.168.12.12
Internal IP address of the physical server (port no 2 = enp0s9): 172.16.3.1/24
- Hypervisor, access to physical box - Artur O
- IRC, for chatting (Meelis Hass)
- Etherpad for collaborating (Sheela)
- Certificate management for roadwarriors, usecase number #1 (Artur O)
- Monitoring software of your choice to make sure that services are up and running, possibly use LDAP for authentication (Artur O)
- E-mail for sending notifications from monitoring software at first (Ilja), this needs MX records in DNS (Ilja, Mohanad helps)
- Later, in the beginning just monitor public services: OpenVPN connection to headquarters
Find security issues in the deployed services, attempt to plant backdoors, malware etc.
Team members: Kustas, Ender
Point-to-point tunnels between routers
Since routers are the default route for all the internal machines the easiest way to set up routing between internal networks is to set up OpenVPN instances on each router.
In router install OpenVPN module for OpenWrt:
opkg update opkg install luci-app-openvpn openvpn-openssl
In the OpenWrt web interface there should pop up Services section with OpenVPN underneath it.
The topology for routers:
For each tunnel configure on one end "Simple server configuration for a routed point-to-point VPN" and on the other end "Simple client configuration for a routed point-to-point VPN" the configuration for connection on hq could look something like this:
To upload secret select secret under --Additional fields-- and hit add. To generate secret use following command on your laptop:
openvpn --genkey --secret static.key
Under Switch to advanced configuration --> Networking add route field for each subnet you want to make accessible via that tunnel. For each tunnel a new interface pops up under Interfaces section. Assign firewall rules as appropriate. To test I guess you can just insert the interface into LAN zone.
On your Ubuntu laptop install OpenVPN plugin for NetworkManager:
sudo apt install network-manager-openvpn-gnome
On Windows laptop install OpenVPN client and TAP driver from https://openvpn.net/index.php/download/community-downloads.html
To generate key, use your computer hostname as filename.
openssl genrsa -out lauri-acer-c720p.key 4096
To generate signing request, again use computer hostname as filename.
openssl req -new -key lauri-acer-c720p.key -out lauri-acer-c720p.csr
The command expects interactive input, set common name to your computer hostname again:
lauri@lauri-c720p ~ $ openssl req -new -key lauri-acer-c720p.key -out lauri-acer-c720p.csr You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [AU]:EE State or Province Name (full name) [Some-State]:Harjumaa Locality Name (eg, city) :Tallinn Organization Name (eg, company) [Internet Widgits Pty Ltd]:Wut Incorporated Organizational Unit Name (eg, section) :Headquarters Common Name (e.g. server FQDN or YOUR name) :lauri-acer-c720p Email Address :firstname.lastname@example.org Please enter the following 'extra' attributes to be sent with your certificate request A challenge password : An optional company name :
Send lauri-acer-c720p.csr to Artur. Artur has to send you back three things: OpenVPN configuration file, signed certificate lauri-acer-c720p.crt and just in case: server.crt, ca.crt, ta.key
To sign the request:
openssl ca -config /etc/openvpn/easy-rsa/openssl-1.0.cnf -in lauri-acer-c720p.csr -extensions client_cert -out lauri-acer-c720p.crt
To dump certificate contents in human-readable format:
openssl x509 -in lauri-acer-c720p.crt -noout -text
To test web server's TLS configuration:
openssl s_client -connect www.koodur.com:443
To make a HTTP request as well:
(echo -en "GET / HTTP/1.0\n\n"; sleep 1) | openssl s_client -connect www.koodur.com:443
See here for more about Estonian ID-card certificates: https://www.sk.ee/en/repository/ldap/ldap-kataloogi-kasutamine/
In Ubuntu, the configuration of the VPN goes as shown below. "User certificate" is the user's signing request signed by the CA (that CA gave to user after receiving the signing request). "CA certificate" is the CA public key. "Private Key" is the user's private key.
In order to send logs to Graylog server, put those lines into new file: /etc/rsyslog.d/client.conf
$ActionQueueType LinkedList $ActionQueueFileName srvrfwd $ActionResumeRetryCount -1 $ActionQueueSaveOnShutdown on *.* @@logging.office:1514
and then restart the service:
sudo service rsyslog restart
All machines should have hostnames in accordance to this page, Keijo uses it to insert DNS entries for the DNS server that is accessible via VPN.
When you reconfigure hostname on your Linux box: set /etc/hostname to the (short) hostname eg. 'monitoring' and set /etc/hosts line 127.0.1.1 to fully qualified hostname and (short) hostname '127.0.1.1 monitoring.office monitoring'
- dc01.office - 172.16.1.128 domain services
- http://intranet.office - 172.16.x.x intranet website
- http://wiki.office - 172.16.2.30 internal wiki
- http://git.office - 172.16.2.171 source code hosting
- http://paste.office - 172.16.2.20 code snippets
- http://chat.office - 172.16.3.149 IRC server
- http://pad.office - 172.16.3.247 etherpad
- http://ca.office - 172.16.x.x certificate authority web endpoint
- http://mail.office - 172.16.3.235 webmail
- http://logging.office - 172.16.3.228 Artur's graylog
- http://monitoring.office - 172.16.1.212/nagios Mohanad's nagios
- http://router-hq.office - 172.16.1.254 hq router
- http://router-rnd.office - 172.16.2.254 rnd router
- http://router-devops.office - 172.16.3.254 devops router
- host-hq.office - 172.16.1.1 hq host
- host-rnd.office - 172.16.2.1 rnd host
- host-devops.office - 172.16.3.1 devops host
Send report as a plaintext e-mail to Lauri, in the title include: Report #number - your name - your team
In the content make sure you specify the timespan you're talking about (September of 2016, first half of October 2016 etc)
The content, no need for formal speech:
- What have been done so far by the team (eg. server hardware setup, virtual machine setup, service setup)
- What was your role for this timespan, note that we will shuffle the teams now and then
- What was your contribution, or in other words what did you do during this timespan
- What (security) incidents happened - red team found messing around with the servers, passwords changed, backdoor found etc.
September wrapup & iptables lecture
October wrapup & X.509/TLS lecture
Last sessions 7. december and 14. december, no session on Robotex week (30. november).
Last steps to pass the course:
- Make sure blah.office DNS records work and services are accessible on default ports, eg http://blah.office:9001 is not cool. If necessary set up proxying web server or use iptables DNAT rule to overwrite the port number.
- Make sure your service is not running as root user or regular user.
- Make sure services are being monitored by Nagios
- Make sure service logs are forwarded to Graylog
- Check port forward rules on the routers, make sure only necessary services are accessible from the Internet
- Check listening services on each machine, make sure only necessary services are running using netstat -lntup if necessary not only stop the services but disable it as well so it's not started during next boot
- If a service can't be disabled prevent access using iptables, to save firewall rules over reboots: apt install iptables-persistent
- Make sure there are no user accounts with simple passwords
- Make sure there are no random user accounts with passwords, to check: cat /etc/shadow
- Preferrably use SSH public key authentication
- Run port scans on the public and internal IP addresses using nmap
- Make sure ~/.ssh/authorized_keys and /root/.ssh/authorized_keys does not contain any unrecognized keys, if necessary remove them
- Prevent brute force SSH attacks using fail2ban
- Make sure security updates are installed, make sure machine gets rebooted if kernel was upgraded and make sure service is restarted when service is upgraded.
- Make sure it's not open relay, meaning that it won't accept mail for foreign domains
For webservers, eg if your service has a web interface:
- In your webserver check /etc/apache2/sites-enabled contents, make sure that only what's necessary is there.
- Set up certificates if you haven't done so using Let's Encrypt
- Make sure HSTS is enabled
- Make sure weak ciphers are disabled, use some SSL test to check
By the last session on 14th of December prepare a comprehensive e-mail about the state of your service
- How to administer the service if applicable - what is the administrator username and password
- How to access the virtual machine - what DNS record or IP address should be used, what is the username and password. If public key authentication is used instead of passwords, figure out who will take over your service after this course and give him/her access to the machine.