Meeskond "VRAM"

Intensiivnädala ülesanne

Mis põhjustab arvutiklasside Linuxite hangumist, kui on palju kasutajaid korraga ning kas seda on võimalik parandada?

---

Võimalikud põhjused

1. Võrgu läbilaskevõime on liiga madal sellise hulga andmete jaoks?

• Võrgu failisüsteem on aeglane
• Ei tea miks failisüsteem on aeglane
• Pole teada, kui kiire peaks võrgufailisüsteem olema, et programmide kasutamise kogemus oleks võrdväärne kohalikku ketta kasutamisega.

Eesmärgid

1. Mõõta, kui palju kasutab Firefox ja Eclipse ketta ressurssi
2. Mõõta, kui palju langeb võrgukiirus mitmete kasutajate samaaegsel kirjutamisel võrgukettale

Mida on vaja eesmärkide täitmiseks teha

• Õppida kasutama võrgufailisüsteemi mõõtmise ja testimise tarkvarasid (Iozone, dbench, bonnie++)
• Õppida mõõtma võrgufailisüsteemi kirjutamise kiirust

Tegevused

ESMASPÄEV 28.03.2011

Eesmärgid:

• Probleemi püstitamine ja analüüsimine.
  
• Tutvumine arvutiklassi Linuxiga - [1].
  
• Probleemi lahendamiseks vajalike tööriistadega tutvumine ning nende hankimine.
  
• Testimiseks vajaliku töökeskkonna loomine.
  

Tegevus:

Esmalt tutvusime arvutiklassi Linuxiga, et saada üldpilt probleemi olemusest. Järgnevalt jagasime meeskonnasiseselt tööjaotuse. Risto ja Vadim loovad probleemi lahendamiseks vajaliku test töökeskkonna, võttes arvutiklassis asuva estobuntu versiooni tõmmise ning luues virtuaalse koopia arvutiklassi Linuxist. Arvi ja Meelis tutvuvad testimiseks vajalike tööriistadega, IOzone ja teiste võimalike alternatiividega ning hiljem tutvustavad ideid ka teistele meeskonnaliikmetele.

Testimiseks vajaliku töökeskkonna lõime wiki artikli "Arvutiklassi Linux - Installeerimine" juhendi järgi.

• Paigaldame subversion
• Paigaldame ruby
• Uuendame süsteemi

sudo -i
aptitude install subversion ruby
svn co http://robot.itcollege.ee/svn/marcus/estobuntu-post-install/
cd estobuntu-post-install
apt-get update
apt-get dist-upgrade
./estobuntu.rb

Erinevate pakendamata programmide installeerimiseks muutsime local_install.sh eemaldades sealt read, mis puudutasid windows operatsiooni süsteemi ning seejärel käivitasime antud skripti.

Päeva lõpuks saime erinevad test töökeskkonnad valmis testimiseks.
Estobuntu 10.05 x64 - Arvutiklassi Linux
Ubuntu 10.10 x64 - Linuxi uuem versioon

TEISIPÄEV 29.03.2011

Järgnevalt tegime 2 katset:

• Tõmbasime ubuntu.iso kohalikule kettale, mille tulemuseks oli keskmiselt 47,9MB/s
• Tõmbasime ubuntu.iso võrgu kettale, mille tulemuseks oli keskmiselt 600K/s

Alustasime testimist Iozone-ga kuid testimine ebaõnnestus, kuna püüdisme iga testi järel võrgukausta uuesti külge võtta (unmount > mount), et cache tühjendada.
Pärast -U võtme eemaldamist käsust, hakkas testimine tööle.

Bonnie++'ga testimine ei ole võimalik, kuna programm nõuab, et testimisfail oleks vähemalt kaks korda suurem, kui on arvutil muutmälu. Aga nii mahukas testimine võtab aeglase võrgukiiruse juures liiga palju aega ning lisaks on kodukataloogides ainult 500MB ruumi.

Alustasime tarkvara otsimist, millega saaks jäädvustada, kui palju kirjutatakse ja loetakse võrgukettalt, kui töötavad õppetöös kasutatavad programmid (N: firefox, eclipse)

Leidsime programmid: IOtop, speedometer, nethogs, iopp, pidstat (sysstat)
Homme vaatame, mis nendega teha annab ja sooritame paar mõõtmist.

Over&Out!

KOLMAPÄEV 30.03.2011

Hommikul tekkis probleem VirtualBoxiga.
Probleem:

Kernel driver not installed (rc=-1908) 

The VirtualBox Linux kernel driver (vboxdrv) is either not loaded or there is a permission problem with /dev/vboxdrv. Please reinstall the kernel module
 by executing '/etc/init.d/vboxdrv setup' as root. Users of Ubuntu, Fedora or Mandriva should install the DKMS package first. This package keeps
 track of Linux kernel changes and recompiles the vboxdrv kernel module if necessary.

Failed to open a session for the virtual machine estobuntu.

The virtual machine 'estobuntu' has terminated unexpectedly during startup with exit code 1.

Lahendus: Uuele kernerile headeri lisamine ning kõikidele klassiarvutitele käsu '/etc/init.d/vboxdrv setup' rakendamine.

Eile leitud programmidest võtsime kasutusele IOtop'i ja pidstat'i. Iopp ei sobinud, kuna seda ei ole ametlikes pakihaldurites ja vajab eraldi kompileerimist ning manuaalset paigaldamist. Nethogs käivitub ainult juurkasutaja õigustes, seega ei sobi meie puhul. Speedometer mõõtis kogu võrguliidese liiklust, mitte meile vajalikku kettale kirjutamist ja kettalt lugemist.

Sooritasime järgmised mõõtmised:

Jälgisime, kui palju kirjutatakse võrgukettale teatud aja jooksul

Firefoxi ja Eclipse käivitamise puhul ühes masinas kirjutatakse võrgukettale väga vähe andmeid (50-100 kB).

Võrgukasutuse mõõtmise näiteskript

#!/bin/bash
 
LC_ALL=C
 
RX1=$(ifconfig | grep RX | head -1 | cut -d: -f2 | cut -d" " -f1)
TX1=$(ifconfig | grep TX | head -1 | cut -d: -f2 | cut -d" " -f1)

echo "Stardin Eclipse'i"
eclipse > /dev/null 2>&1 &
ECLIPSE_PID=$(ps -ef | grep eclipse | grep java | awk '{print $2;}')
sleep 10
echo "Tapan Eclipse'i"
killall eclipse
kill $ECLIPSE_PID

RX2=$(ifconfig | grep RX | head -1 | cut -d: -f2 | cut -d" " -f1)
TX2=$(ifconfig | grep TX | head -1 | cut -d: -f2 | cut -d" " -f1)
 
RX=$(($RX2-$RX1))
TX=$(($TX2-$TX1))

#echo "kokku RX baitides: $RX"
#echo "kokku TX baitides: $TX"

RXKB=$(echo "scale=2;$RX/1024"|bc -l)
TXKB=$(echo "scale=2;$TX/1024"|bc -l)

echo "kokku RX: $RXKB/KB"
echo "kokku TX: $TXKB/KB"

Testimise skripti link

# Käsureal täitmiseks

wget "http://enos.itcollege.ee/~metamm/m66tja.sh"

chmod +x m66tja.sh

./m66tja.sh

Koormuse skripti link

wget "http://enos.itcollege.ee/~mernits/failid/small-file-test.pl"

chmod +x small-file-test.pl

./small-file-test.pl > mingi_nimi.txt

WireSharkiga uurimine

Probleem: Kuidas käivitada WireShark ilma root õigusi omamata?

Setting network privileges for dumpcap

1. Ensure your linux kernel and filesystem supports File Capabilities and also you have installed necessary tools.
2. "setcap 'CAP_NET_RAW+eip CAP_NET_ADMIN+eip' /usr/bin/dumpcap"
3. Start Wireshark as non-root and ensure you see the list of interfaces and can do live capture.

Limiting capture permission to only one group

1. Create user "wireshark" in group "wireshark".
2. "chgrp wireshark /usr/bin/dumpcap"
3. chmod 754 /usr/bin/dumpcap
4. "setcap 'CAP_NET_RAW+eip CAP_NET_ADMIN+eip' /usr/bin/dumpcap"
5. Ensure Wireshak works only from root and from a user in the "wireshark" group

Link: http://wiki.wireshark.org/CaptureSetup/CapturePrivileges

NELJAPÄEV 31.03.2011

Lõime virtuaalse Ubuntu serveri samade parameetritega nagu Solarise server

• 4GB operatiivmälu
• 2-tuumaline protsessor
• 32GB ketta suurus

Klassiarvutis konfigureerisime ssh ühenduse kaudu virtuaalset Ubuntu serverit

• Konfigureerisime staatilise ip aadressi, võrgumaski, vaikelüüsi

nano /etc/network/interfaces

• Uuendasime virtuaalset Ubuntu serverit

sudo apt-get update
sudo apt-get upgrade

• Muutsime kasutaja student parooli

passwd student

LDAP kliendi audentimise konfigureerimine

Paigaldasime järgmised pakid:

apt-get install libpam-ldap

Paigaldamise käigus küsiti järgmised küsimused:

• The address of the LDAP server used: ldaps:///host 172.16.0.148
• The distinguished name of the search base: dc=itcollege,dc=ee
• The LDAP version to use: 3
• Make local root Database admin: no
• Does the LDAP database require login? no

apt-get install libnss-ldap nss-updatedb libnss-db nscd

Tegime Name Service Cache Daemon'le restardi:

service nscd restart

Installisime samba:

apt-get install samba smbfs

Vajadusel käivitamine:

service smbd start

Konfigureerisime samba konfiguratsiooni faili:

nano /etc/samba/smb.conf

Tuleb lisada:

[global]

        passdb backend = ldapsam:"ldap://172.16.0.148"
        ldap admin dn = "uid=robot,ou=People,o=itcollege.ee,dc=itcollege,dc=ee"
	ldap group suffix = ou=Groups
	ldap idmap suffix = ou=Idmap
	ldap machine suffix = ou=Computers
	ldap replication sleep = 2000
	ldap suffix = dc=itcollege,dc=ee
	ldap ssl = no
	ldap timeout = 3
	ldap user suffix = ou=People
	usershare allow guests = Yes
	panic action = /usr/share/samba/panic-action %d

[earl global]
        unix charset = iso8859-1
        workgroup = ITCOLLEGE
        interfaces = 172.16.0.175/24
;       password server = 172.16.0.175
        passdb backend = ldapsam:"ldap://172.16.0.148"
;       passdb backend = ldapsam:"ldap://172.16.0.150"
;       log level = 9
        log file = /opt/samba/var/log.%m
        logon path = 
        logon drive = C:
        logon home = C:\Documents and Settings\%U
        domain logons = Yes
        os level = 200 
        wins support = Yes
        ldap admin dn = "cn=Directory Manager"
        ldap group suffix = ou=Groups
        ldap idmap suffix = ou=Idmap
        ldap machine suffix = ou=Computers
        ldap suffix = o=itcollege.ee,dc=itcollege,dc=ee
        ldap ssl = no
        ldap timeout = 60
        ldap user suffix = ou=People
        ldapsam:trusted = yes
        client ntlmv2 auth = yes

[homes]
	comment = Home Directories
	valid users = %S
	create mask = 0700
	directory mask = 0700
	browseable = No
	browsable = No
        read only = No

[lab]
	comment = Labori kaust
	path = /var/data/lab
	valid users = @libuuid
	read list = @libuuid
	write list = @libuuid
	force group = libuuid
	read only = No
	create mask = 0664
	directory mask = 0770

Muutsime failide konfiguratsiooni

/etc/pam.d/common-account

Lisasime:

account sufficient pam_ldap.so

Muutsime faili:

/etc/pam.d/common-auth

Lisasime:

auth sufficient pam_ldap.so

Muutsime faili:

/etc/pam.d/common-password

Lisasime:

password sufficient pam_ldap.so

Tsharki: tulemus mitme kasutajaga
===================================================================
SMB RTT Statistics:
Filter: ip.addr==192.168.6.4
Commands Calls Min RTT Max RTT Avg RTT
Close 194 0.00016 0.00585 0.00038
Rename 11 0.00079 0.00153 0.00101
Read AndX 28 0.00017 0.00089 0.00042
Write AndX 228 0.00018 0.01307 0.00137
NT Create AndX 182 0.00031 0.00282 0.00054

Transaction2 Commands Calls Min RTT Max RTT Avg RTT
FIND_FIRST2 528 0.00051 0.01478 0.00151
FIND_NEXT2 1 0.00297 0.00297 0.00297
QUERY_PATH_INFO 30051 0.00016 0.01881 0.00048
SET_PATH_INFO 60 0.00019 0.00260 0.00069
SET_FILE_INFO 34 0.00014 0.00100 0.00035

NT Transaction Commands Calls Min RTT Max RTT Avg RTT
===================================================================

Katsetulemused

Tegime test 10 arvuti kaupa Arvuti nr 1:

Creating ........................................ 9.10193 sec
Deleting ... 8.858836 sec
Creating ........................................ 6.074982 sec
Deleting ... 8.251794 sec
Creating ........................................ 7.86962 sec
Deleting ... 7.069565 sec
Creating ........................................ 9.429803 sec
Deleting ... 8.10277 sec
Creating ........................................ 8.655039 sec
Deleting ... 7.480237 sec
Creating ........................................ 8.775523 sec
Deleting ... 8.291478 sec
Creating ........................................ 6.422976 sec
Deleting ... 6.810017 sec
Creating ........................................ 8.99275 sec
Deleting ... 9.306371 sec
Creating ........................................ 10.497752 sec
Deleting ... 7.697717 sec
Creating ........................................ 6.229081 sec
Deleting ... 7.433747 sec


Arvuti nr 2:

Creating ........................................ 18.023821 sec Deleting ... 4.753462 sec Creating ........................................ 12.650268 sec Deleting ... 7.747844 sec Creating ........................................ 15.582884 sec Deleting ... 3.55386 sec Creating ........................................ 16.05862 sec Deleting ... 4.783962 sec Creating ........................................ 17.174822 sec Deleting ... 5.052176 sec Creating ........................................ 15.956232 sec Deleting ... 5.171336 sec Creating ........................................ 15.724377 sec Deleting ... 5.502351 sec Creating ........................................ 12.887717 sec Deleting ... 4.991567 sec Creating ........................................ 11.220797 sec Deleting ... 6.137639 sec Creating ........................................ 9.458247 sec Deleting ... 2.718366 sec Arvuti nr 3:

Creating ........................................ 15.729418 sec Deleting ... 8.198384 sec Creating ........................................ 13.535778 sec Deleting ... 6.554081 sec Creating ........................................ 13.573645 sec Deleting ... 7.629296 sec Creating ........................................ 13.88095 sec Deleting ... 5.748886 sec Creating ........................................ 16.439681 sec Deleting ... 9.304297 sec Creating ........................................ 15.25325 sec Deleting ... 6.19263 sec Creating ........................................ 14.664501 sec Deleting ... 4.338816 sec Creating ........................................ 11.886828 sec Deleting ... 7.095056 sec Creating ........................................ 9.77776 sec Deleting ... 5.542512 sec Creating ........................................ 9.448197 sec Deleting ... 1.962213 sec

Arvuti nr 4:

Creating ........................................ 7.676064 sec Deleting ... 6.02757 sec Creating ........................................ 9.263942 sec Deleting ... 4.905135 sec Creating ........................................ 8.163424 sec Deleting ... 6.988196 sec Creating ........................................ 7.979979 sec Deleting ... 8.241237 sec Creating ........................................ 9.832357 sec Deleting ... 9.175535 sec Creating ........................................ 7.510824 sec Deleting ... 5.904447 sec Creating ........................................ 6.592056 sec Deleting ... 4.19613 sec Creating ........................................ 8.749334 sec Deleting ... 5.599045 sec Creating ........................................ 5.359211 sec Deleting ... 3.467353 sec Creating ........................................ 7.537781 sec Deleting ... 8.451127 sec

Arvuti nr 5:

Creating ........................................ 13.169464 sec Deleting ... 6.836029 sec Creating ........................................ 13.925972 sec Deleting ... 5.677754 sec Creating ........................................ 14.811662 sec Deleting ... 7.513486 sec Creating ........................................ 12.983368 sec Deleting ... 8.468889 sec Creating ........................................ 18.058816 sec Deleting ... 6.941241 sec Creating ........................................ 10.452872 sec Deleting ... 7.653394 sec Creating ........................................ 15.36756 sec Deleting ... 6.070592 sec Creating ........................................ 11.245425 sec Deleting ... 5.09349 sec Creating ........................................ 13.024059 sec Deleting ... 4.026848 sec Creating ........................................ 10.292623 sec Deleting ... 3.309758 sec

Arvuti nr 6:

Creating ........................................ 16.649613 sec Deleting ... 5.90282 sec Creating ........................................ 16.513232 sec Deleting ... 6.474622 sec Creating ........................................ 12.514955 sec Deleting ... 6.540716 sec Creating ........................................ 14.317602 sec Deleting ... 5.422598 sec Creating ........................................ 12.40729 sec Deleting ... 5.617731 sec Creating ........................................ 16.06474 sec Deleting ... 7.304758 sec Creating ........................................ 17.421312 sec Deleting ... 6.33096 sec Creating ........................................ 13.030796 sec Deleting ... 6.132165 sec Creating ........................................ 11.349273 sec Deleting ... 4.383546 sec Creating ........................................ 10.157172 sec Deleting ... 1.868885 sec

Arvuti nr 7:

Creating ........................................ 18.039663 sec Deleting ... 4.161979 sec Creating ........................................ 14.502472 sec Deleting ... 5.579537 sec Creating ........................................ 14.990685 sec Deleting ... 5.382131 sec Creating ........................................ 13.286984 sec Deleting ... 5.973559 sec Creating ........................................ 15.245604 sec Deleting ... 3.385832 sec Creating ........................................ 12.821683 sec Deleting ... 6.865197 sec Creating ........................................ 14.187354 sec Deleting ... 5.839637 sec Creating ........................................ 14.416255 sec Deleting ... 3.093757 sec Creating ........................................ 10.937081 sec Deleting ... 5.085945 sec Creating ........................................ 9.893626 sec Deleting ... 5.173803 sec

Arvuti nr 8:

Creating ........................................ 16.364517 sec Deleting ... 6.239315 sec Creating ........................................ 13.928007 sec Deleting ... 5.21715 sec Creating ........................................ 14.595766 sec Deleting ... 6.093827 sec Creating ........................................ 14.606533 sec Deleting ... 5.183041 sec Creating ........................................ 12.90902 sec Deleting ... 5.189499 sec Creating ........................................ 13.892223 sec Deleting ... 6.237047 sec Creating ........................................ 13.312935 sec Deleting ... 6.411764 sec Creating ........................................ 13.875612 sec Deleting ... 3.421691 sec Creating ........................................ 11.570193 sec Deleting ... 4.880627 sec Creating ........................................ 13.045542 sec Deleting ... 1.768013 sec

Arvuti nr 9:

Creating ........................................ 16.289831 sec Deleting ... 6.293338 sec Creating ........................................ 15.250328 sec Deleting ... 4.298625 sec Creating ........................................ 19.849191 sec Deleting ... 1.468677 sec Creating ........................................ 14.427785 sec Deleting ... 4.8108 sec Creating ........................................ 12.979617 sec Deleting ... 4.77237 sec Creating ........................................ 12.943957 sec Deleting ... 7.405112 sec Creating ........................................ 11.134989 sec Deleting ... 10.306092 sec Creating ........................................ 10.724982 sec Deleting ... 4.481664 sec Creating ........................................ 17.842688 sec Deleting ... 2.417205 sec Creating ........................................ 12.064586 sec Deleting ... 0.338769 sec

Arvuti nr 10:

Creating ........................................ 14.26751 sec Deleting ... 9.117601 sec Creating ........................................ 15.482384 sec Deleting ... 5.015514 sec Creating ........................................ 16.197769 sec Deleting ... 7.708947 sec Creating ........................................ 13.49954 sec Deleting ... 5.823264 sec Creating ........................................ 12.400706 sec Deleting ... 6.303302 sec Creating ........................................ 15.563541 sec Deleting ... 5.900704 sec Creating ........................................ 13.540804 sec Deleting ... 3.621184 sec Creating ........................................ 14.359361 sec Deleting ... 5.54229 sec Creating ........................................ 10.889221 sec Deleting ... 4.749881 sec Creating ........................................ 11.360072 sec Deleting ... 3.339618 sec