User talk:Ksaareme

Reference page for I600 exam

---

Exam help wiki: [1][2]

Useful linux commands

Checking information

*lsb_release -a to check OS
*uname -sr to check OS kernel
*cat /proc/cpuinfo and check processor information. 
*arch to check CPU architecture 
*cat /proc/meminfo to check memory usage, RAM
*lspci -t -v -nn to enumerate PCI and PCI Express devices 
*lsusb and lsusb -t to enumerate USB devices
*fdisk -l to enumerate disks and partitions
*lsblk to enumerate block devices 
*xrandr to enumerate display outputs
*cat /proc/asound/cards to check which audio devices are available 
*dmidecode to see even more information about your computer
*ifconfig -a or ip addr list to list all network interfaces
*iwconfig or iw list to list all wireless network interfaces
*hcitool dev to list bluetooth host controller.
*glxinfo to check 3D rendering capabilities, GPU
*lscpu  info about cpu and processing units
*lshw - short info about hardware units, such as cpu, memory, disk, usb controllers, network adapters
*hwinfo –short info about hardware utility stuff
*lspci info about pci buses and devices connected to them
*lsscsi info about sata/scsi devices
*lsusb info about usb controllers and devices
*lsblk block information, harddrive partitions
*df –H info about disk space and partitions that are mounted
*fdisk –l  info about partitions
*free –m info about used, free and total amount of RAM
*hdparm –i info about sata devices

General Commands
cat [filename] 	Display file’s contents to the standard output device (usually your monitor).
cd /directorypath 	Change to directory.
chmod [options] mode filename 	Change a file’s permissions.
chown [options] filename 	Change who owns a file.
clear 	Clear a command line screen/window for a fresh start.
cp [options] source destination 	Copy files and directories.
date [options] 	Display or set the system date and time.
df [options] 	Display used and available disk space.
du [options] 	Show how much space each file takes up.
file [options] filename 	Determine what type of data is within a file.
find [pathname] [expression] 	Search for files matching a provided pattern.
grep [options] pattern [filesname] 	Search files or output for a particular pattern.
kill [options] pid 	Stop a process. If the process refuses to stop, use kill -9 pid.
less [options] [filename] 	View the contents of a file one page at a time.
ln [options] source [destination] 	Create a shortcut.
locate filename 	Search a copy of your filesystem for the specified filename.
lpr [options] 	Send a print job.
ls [options] 	List directory contents.
man [command] 	Display the help information for the specified command.
mkdir [options] directory 	Create a new directory.
mv [options] source destination 	Rename or move file(s) or directories.
passwd [name [password]] 	Change the password or allow (for the system administrator) to change any password.
ps [options] 	Display a snapshot of the currently running processes.
pwd 	Display the pathname for the current directory.
rm [options] directory 	Remove (delete) file(s) and/or directories.
rmdir [options] directory 	Delete empty directories.
ssh [options] user@machine 	Remotely log in to another Linux machine, over the network. Leave an ssh session by typing exit.
su [options] [user [arguments]] 	Switch to another user account.
tail [options] [filename] 	Display the last n lines of a file (the default is 10).
tar [options] filename 	Store and extract files from a tarfile (.tar) or tarball (.tar.gz or .tgz).
top 	Displays the resources being used on your system. Press q to exit.
touch filename 	Create an empty file with the specified name.
who [options] 	Display who is logged on.

Linux Command Line

ls list files and directories

ls -a list all files and directories

mkdir make a directory

cd directory change to named directory

cd change to home-directory

cd ~ change to home-directory

cd .. change to parent directory

pwd display the path of the current directory

cp (copy)

cp file1 file2 is the command which makes a copy of file1 in the current working directory and calls it file2

What we are going to do now, is to take a file stored in an open access area of the file system, and use the cp command to copy it to your unixstuff directory.

First, cd to your unixstuff directory.

% cd ~/unixstuff

Then at the UNIX prompt, type,

% cp /vol/examples/tutorial/science.txt .

Note: Don't forget the dot . at the end. Remember, in UNIX, the dot means the current directory.

The above command means copy the file science.txt to the current directory, keeping the name the same.

(Note: The directory /vol/examples/tutorial/ is an area to which everyone in the school has read and copy access. If you are from outside the University, you can grab a copy of the file here. Use 'File/Save As..' from the menu bar to save it into your unixstuff directory.)

Exercise 2a

Create a backup of your science.txt file by copying it to a file called science.bak

2.2 Moving files

mv (move)

mv file1 file2 moves (or renames) file1 to file2

To move a file from one place to another, use the mv command. This has the effect of moving rather than copying the file, so you end up with only one file rather than two.

It can also be used to rename a file, by moving the file to the same directory, but giving it a different name.

We are now going to move the file science.bak to your backup directory.

First, change directories to your unixstuff directory (can you remember how?). Then, inside the unixstuff directory, type

% mv science.bak backups/.

Type ls and ls backups to see if it has worked.

2.3 Removing files and directories

rm (remove), rmdir (remove directory)

To delete (remove) a file, use the rm command. As an example, we are going to create a copy of the science.txt file then delete it.

Inside your unixstuff directory, type

% cp science.txt tempfile.txt % ls % rm tempfile.txt % ls

You can use the rmdir command to remove a directory (make sure it is empty first). Try to remove the backups directory. You will not be able to since UNIX will not let you remove a non-empty directory.

Exercise 2b

Create a directory called tempstuff using mkdir , then remove it using the rmdir command.

2.4 Displaying the contents of a file on the screen

clear (clear screen)

Before you start the next section, you may like to clear the terminal window of the previous commands so the output of the following commands can be clearly understood.

At the prompt, type

% clear

This will clear all text and leave you with the % prompt at the top of the window.

cat (concatenate)

The command cat can be used to display the contents of a file on the screen. Type:

% cat science.txt

As you can see, the file is longer than than the size of the window, so it scrolls past making it unreadable.

less

The command less writes the contents of a file onto the screen a page at a time. Type

% less science.txt

Press the [space-bar] if you want to see another page, and type [q] if you want to quit reading. As you can see, less is used in preference to cat for long files.

head

The head command writes the first ten lines of a file to the screen.

First clear the screen then type

% head science.txt

Then type

% head -5 science.txt

What difference did the -5 do to the head command?

tail

The tail command writes the last ten lines of a file to the screen.

Clear the screen and type

% tail science.txt

Q. How can you view the last 15 lines of the file?

2.5 Searching the contents of a file

Simple searching using less

Using less, you can search though a text file for a keyword (pattern). For example, to search through science.txt for the word 'science', type

% less science.txt

then, still in less, type a forward slash [/] followed by the word to search

/science

As you can see, less finds and highlights the keyword. Type [n] to search for the next occurrence of the word.

grep (don't ask why it is called grep)

grep is one of many standard UNIX utilities. It searches files for specified words or patterns. First clear the screen, then type

% grep science science.txt

As you can see, grep has printed out each line containg the word science.

Or has it ????

Try typing

% grep Science science.txt

The grep command is case sensitive; it distinguishes between Science and science.

To ignore upper/lower case distinctions, use the -i option, i.e. type

% grep -i science science.txt

To search for a phrase or pattern, you must enclose it in single quotes (the apostrophe symbol). For example to search for spinning top, type

% grep -i 'spinning top' science.txt

Some of the other options of grep are:

-v display those lines that do NOT match -n precede each matching line with the line number -c print only the total count of matched lines Try some of them and see the different results. Don't forget, you can use more than one option at a time. For example, the number of lines without the words science or Science is

% grep -ivc science science.txt

wc (word count)

A handy little utility is the wc command, short for word count. To do a word count on science.txt, type

% wc -w science.txt

To find out how many lines the file has, type

% wc -l science.txt

NMAP and SSH

NMAP

• Find my ip: ifconfig
• Scan networks: nmap <ip>
• Find live hosts: nmap -sP 192.168.0.*
• Scan specific port: nmap -p 80 server2.tecmint.com
• Scan a single ip address: nmap 192.168.1.1
• Scan a host name: nmap server1.cyberciti.biz
• Scan a host name with more info : nmap -v server1.cyberciti.biz
• Scan with disabled port scan: nmap -sn 192.168.2.1/24
• Read and scan from txt file: nmap -iL /tmp/test.txt
• Create the txt file to read from: cat > /tmp/test.txt

• Shut down a network: <sudo> shutdown -s -m \\192.168.1.1 or shutdown -h now

• nmap --iflist

SSH

• Connect: ssh collie.stanford.edu
• Log-in: ssh jhawkins@collie.stanford.edu

Instruction sets

nstruction sets may be categorized by the maximum number of operands explicitly specified in instructions.

(In the examples that follow, a, b, and c are (direct or calculated) addresses referring to memory cells, while reg1 and so on refer to machine registers.)

C = A+B

  * 0-operand (zero-address machines), so called stack machines: All arithmetic operations take place using the top one or two positions on 
   the stack: push a, push b, add, pop c.
   C = A+B needs four instructions. For stack machines, the terms "0-operand" and "zero-address" apply to arithmetic instructions, 
   but not to all instructions, as 1-operand push and pop instructions are used to access memory.

   *1-operand (one-address machines), so called accumulator machines, include early computers and many small microcontrollers: most instructions 
   specify a single right operand (that is, constant, a register, or a memory location), with the implicit accumulator as the left operand 
   (and the destination if there is one): load a, add b, store c.
   C = A+B needs three instructions.

   *2-operand — many CISC and RISC machines fall under this category:
       CISC — move A to C; then add B to C.
           C = A+B needs two instructions. This effectively 'stores' the result without an explicit store instruction.
           CISC — Often machines are limited to one memory operand per instruction: load a,reg1; add b,reg1; store reg1,c; 
           This requires a load/store pair for any memory movement regardless of whether the add result is an augmentation 
           stored to a different place, as in C = A+B, or the same memory location: A = A+B.
           C = A+B needs three instructions.
           RISC — Requiring explicit memory loads, the instructions would be: load a,reg1; load b,reg2; add reg1,reg2; store reg2,c.
           C = A+B needs four instructions.

   *3-operand, allowing better reuse of data:[5]
       CISC — It becomes either a single instruction: add a,b,c
           C = A+B needs one instruction.
           or more typically: move a,reg1; add reg1,b,c as most machines are limited to two memory operands.
               C = A+B needs two instructions.
       RISC — arithmetic instructions use registers only, so explicit 2-operand load/store instructions are needed: 
       load a,reg1; load b,reg2; add reg1+reg2->reg3; store reg3,c;
           C = A+B needs four instructions.
           Unlike 2-operand or 1-operand, this leaves all three values a, b, and c in registers available for further reuse.[5]

   LDA - Loads the contents of the memory address or integer into the accumulator
   ADD - Adds the contents of the memory address or integer to the accumulator
   STO - Stores the contents of the accumulator into the addressed location

 ADD ;add one number to another number
 SUB ;subtract one number to another number
 INC ;increment a number by 1
 DEC ;decrement a number by 1
 MUL ;multiply numbers together
 OR  ;boolean algebra function
 AND ;boolean algebra function
 NOT ;boolean algebra function
 XOR ;boolean algebra function
 JNZ ;jump to another section of code if a number is not zero (used for loops and ifs)
 JZ  ;jump to another section of code if a number is zero (used for loops and ifs)
 JMP ;jump to another section of code (used for loops and ifs)

 1 LDA #12 ;loads the number 12 into the accumulator
 2 MUL #2  ;multiplies the accumulator by 2 = 24
 3 SUB #6  ;take 6 away from the accumulator = 18
 4 JNZ 6   ;if the accumulator <> 0 then goto line 6
 5 SUB #5  ;take 5 away from the accumulator (this line isn't executed!)
 6 STO 34  ;saves the accumulator result (18) to the memory address 34

Addressing Mode 	Symbol 	Example 	Description
Memory Location 		LOAD 15 	15 is treated as an address
Integer 	        # 	LOAD #15 	15 is treated as a number
Nothing 		        HALT 	        Some inst. dont need operands

Truth Tables

Logical Conjunction(AND)
p 	q 	p ∧ q
T 	T         T
T 	F 	  F
F 	T 	  F
F 	F 	  F

Logical Disjunction(OR)
p 	q 	p ∨ q
T 	T 	  T
T 	F 	  T
F 	T 	  T
F 	F 	  F

Logical NAND
p 	q 	p ↑ q
T 	T 	  F
T 	F 	  T
F 	T 	  T
F 	F 	  T

Logical  NOR
p 	q 	p ↓ q
T 	T 	  F
T 	F 	  F
F 	T 	  F
F 	F 	  T

Compression

Data compression ratio is defined as the ratio between the uncompressed size and compressed size:

   Compression Ratio = Uncompressed Size / Compressed Size

Thus a representation that compresses a 10MB file to 2MB has a compression ratio of 10/2 = 5, often notated as an explicit ratio, 5:1 (read "five" to "one"), or as an implicit ratio, 5/1. Note that this formulation applies equally for compression, where the uncompressed size is that of the original; and for decompression, where the uncompressed size is that of the reproduction.

Sometimes the space savings is given instead, which is defined as the reduction in size relative to the uncompressed size:

  Space Savings = 1 - Compressed Size / Uncompressed Size

Thus a representation that compresses a 10MB file to 2MB would yield a space savings of 1 - 2/10 = 0.8, often notated as a percentage, 80%.

For signals of indefinite size, such as streaming audio and video, the compression ratio is defined in terms of uncompressed and compressed data rates instead of data sizes:

    Compression Ratio = Uncompressed Data Rate/ Compressed Data Rate

and instead of space savings, one speaks of data-rate savings, which is defined as the data-rate reduction relative to the uncompressed data rate:

 Data Rate Savings = 1 - Compressed Data Rate / Uncompressed Data Rate

For example, uncompressed songs in CD format have a data rate of 16 bits/channel x 2 channels x 44.1 kHz ≅ 1.4 Mbit/s, whereas AAC files on an iPod are typically compressed to 128 kbit/s, yielding a compression ratio of 10.9, for a data-rate savings of 0.91, or 91%.

When the uncompressed data rate is known, the compression ratio can be inferred from the compressed data rate.

Bits and Bytes

 	 bit 	 byte 	         
bit 	  1 	0.125 	      
byte 	  8 	  1

Notes:The value of K (Kilo) during calculations can take two values 1024 or 1000, depends on which type of calculation you want to perform. 
Consider using K = 1024 when you are considering storage capacity whether in hard disk, DVDs, flash drives or other devices and storage media.
K = 1000 should be used when you are thinking of throughput, ie the speed at which information is transferred.

Example: If your computer has 1 KB of disk space is says that he has 1024 B of space, now the throughput of your network card is 1 KB/s then 
it is said that it transmits data to 1000 B/s.

 Bit (b)
 Byte (B)
 Kilobit (Kb)
 Kilobyte (KB)
 Megabit (Mb)
 Megabyte (MB)
 Gigabit (Gb)
 Gigabyte (GB)

   Binary Equivalents
   1 Nybble (or nibble)  =  4 bits 
   1 Byte  =  2 nybbles  =  8 bits 
   1 Kilobyte (KB)  =  1024 bytes 
   1 Megabyte (MB)  =  1024 kilobytes  =  1,048,576 bytes 
   1 Gigabyte (GB)  =  1024 megabytes  =  1,073,741,824 bytes 

Show addition of X and Y in binary

   0 + 0 = 0
   0 + 1 = 1
   1 + 0 = 1
   1 + 1 = 0, and carry 1 to the next more significant bit 

Show subtraction of X and Y in binary
   0 - 0 = 0
   0 - 1 = 1, and borrow 1 from the next more significant bit
   1 - 0 = 1
   1 - 1 = 0 

Show multiplication of X and Y in binary

   0 x 0 = 0
   0 x 1 = 0
   1 x 0 = 0
   1 x 1 = 1, and no carry or borrow bits

Git

   The object database contains four types of objects:

   A blob (binary large object) is the content of a file. Blobs have no file name, time stamps, or other metadata.
   A tree object is the equivalent of a directory. It contains a list of file names, each with some type bits and the name of a blob 
   or tree object that is that file, symbolic link, or directory's contents. This object describes a snapshot of the source tree.
   A commit object links tree objects together into a history. It contains the name of a tree object (of the top-level source directory),
   a time stamp, a log message, and the names of zero or more parent commit objects.
   A tag object is a container that contains reference to another object and can hold additional meta-data related to another object. 
   Most commonly, it is used to store a digital signature of a commit object corresponding to a particular release of the data being tracked by Git.

   heads
   refers to an object locally.
   remotes
   refers to an object which exists in a remote repository.
   stash
   refers to an object not yet committed.
   meta
   e.g. a configuration in a bare repository, user rights. The refs/meta/config namespace was introduced resp gets 
   used by Gerrit (software)[clarification needed][45]
   tags
   see above.

Common Usage

create a new repository
create a new directory, open it and perform a
git init
to create a new git repository.

checkout a repository
create a working copy of a local repository by running the command
git clone /path/to/repository
when using a remote server, your command will be
git clone username@host:/path/to/repository

workflow
your local repository consists of three "trees" maintained by git. the first one is your Working Directory 
which holds the actual files. the second one is the Index which acts as a staging area and finally the HEAD 
which points to the last commit you've made. 

add & commit
You can propose changes (add it to the Index) using
git add <filename>
git add *
This is the first step in the basic git workflow. To actually commit these changes use
git commit -m "Commit message"
Now the file is committed to the HEAD, but not in your remote repository yet.

pushing changes
Your changes are now in the HEAD of your local working copy. To send those changes to your 
remote repository, execute
git push origin master
Change master to whatever branch you want to push your changes to.
If you have not cloned an existing repository and want to connect your repository to a remote 
server, you need to add it with
git remote add origin <server>
Now you are able to push your changes to the selected remote server

update & merge
to update your local repository to the newest commit, execute
git pull
in your working directory to fetch and merge remote changes.
to merge another branch into your active branch (e.g. master), use
git merge <branch>
in both cases git tries to auto-merge changes. Unfortunately, this is not always possible and results in conflicts. 
You are responsible to merge those conflicts manually by  editing the files shown by git. After changing, you need to mark them as merged with
git add <filename>
before merging changes, you can also preview them by using
git diff <source_branch> <target_branch>

replace local changes
In case you did something wrong, which for sure never happens ;), you can replace local changes using the command
git checkout -- <filename>
this replaces the changes in your working tree with the last content in HEAD. Changes already added to the index, 
as well as new files, will be kept.
If you instead want to drop all your local changes and commits, fetch the latest history from the server and point 
your local master branch at it like this
git fetch origin
git reset --hard origin/master

log
in its simplest form, you can study repository history using.. git log
You can add a lot of parameters to make the log look like what you want. To see only the commits of a certain author:
git log --author=bob
To see a very compressed log where each commit is one line:
git log --pretty=oneline
Or maybe you want to see an ASCII art tree of all the branches, decorated with the names of tags and branches:
git log --graph --oneline --decorate --all
See only which files have changed:
git log --name-status
These are just a few of the possible parameters you can use. For more, see git log --help

tagging
it's recommended to create tags for software releases. this is a known concept, which also exists in SVN.
You can create a new tag named 1.0.0 by executing
git tag 1.0.0 1b2e1d63ff
the 1b2e1d63ff stands for the first 10 characters of the commit id you want to reference with your tag. 
You can get the commit id by looking at the...

Decimal <--> Hexadecimal

Hexadecimal	Decimal
    0	          0
    1	          1
    2	          2
    3	          3
    4	          4
    5	          5
    6	          6
    7	          7
    8	          8
    9	          9
    A	          10
    B	          11
    C	          12
    D	          13
    E	          14
    F	          15

Radio station is streaming MP3 audio at 192kbps. How many 500GB harddisks are required in order to archive 8 years of shows?

192 kbps = 192 000 bps 1 536 000 Bps

1500 Kb/s 1.46484375 MB/s

8 years = 2920 days = 70080 hours = 4204800 minutes = 252288000 seconds

172 228608 MB 16 8192 GB 336.384 500GB harddrives

Audio and Images

Audio
In digital audio, 44,100 Hz (alternately represented as 44.1 kHz) is a common sampling frequency. 
Analog audio is recorded by sampling it 44,100 times per second, and then these samples are used 
to reconstruct the audio signal when playing it back.

Images
PNG supports palette-based images (with palettes of 24-bit RGB or 32-bit RGBA colors), grayscale images (with or without alpha channel),
and full-color non-palette-based RGB[A] images (with or without alpha channel). PNG was designed for transferring images on the Internet,
not for professional-quality print graphics, and therefore does not support non-RGB color spaces such as CMYK.

JPEG/JFIF supports a maximum image size of 65,535×65,535 pixels, hence up to 4 gigapixels (for an aspect ratio of 1:1).
JPEG is a commonly used method of lossy compression for digital images, particularly for those images 
produced by digital photography. The degree of  compression can be adjusted, allowing a selectable tradeoff between storage size 
and image quality. JPEG typically achieves 10:1 compression with little perceptible loss in image quality.

---

Compact Disc
Capacity 	Typically up to 700 MiB (up to 80 minutes audio)
Read mechanism 	780 nm wavelength (infrared and red edge) semiconductor laser, 1200 Kibit/s (1×)
Write mechanism 	1200 Kibit/s (1×)

Sampling 4 bit audio (2^4) gives us only 16 values, a far cry from 16-bit audio's 65,536! sample rate. 
Sample rate refers to the number of samples or measurements taken each  second from a recording. 
The typical CD sample rate is 44.1kHz, or 44,100 samples per second.

The bit rate is quantified using the bits per second unit bit/s, often in conjunction with an SI prefix such as "kilo" 
(1 kbit/s = 1000 bit/s), "mega" (1 Mbit/s = 1000 kbit/s), "giga" (1 Gbit/s = 1000 Mbit/s) or "tera" (1 Tbit/s = 1000 Gbit/s).
The non-standard abbreviation "bps" is often used to replace the standard symbol "bit/s", so that, for example, "1 Mbps" is
used to mean one million bits per second.
One byte per second (1 B/s) corresponds to 8 bit/s.

The RGB565 color format is the same as the RGB555 color format, except that 6 bits are used for 
the green value instead of 5. Therefore, all 16 bits are in use. The  organization of the pixels 
in the image buffer is from left to right and bottom up.
RGB888 --> 24-bit RGB (888)

Statistics Lecture One, 02.01.16

Primary Material:
Course Homepage: http://www.cs.ioc.ee/ITKStat
e-Book:           http://onlinestatbook.com/Online_Statistics_Education.pdf

Primary Software
R: https://www.r-project.org/

Introduction to Statistics
Descriptive statistics are used for presenting, organizing and summarizing data. 
Inferential statistics are about drawing conclusions about a population based on data observed in a sample.

Data Analysis Process
Data collection and preparation: Collect Data, prepare codebook, set up structure of data, enter data, screen data for errors
Exploration of data: Descriptive statistics, graphs
Analysis: Explore relationship between variables, compare groups

Python Lecture and Practicum Notes

Lecture 1

---

Get an idea what to do by next week. Ideas:

// 1. Pyglet or pygame lib for a game or simulation:
    1.1. MP beat em' up
    1.2. Survival space sim, use webcam for dynamic view
    1.3. Planet rotation simulation
    Check out http://nehe.gamedev.net/

// 2. WebApp
    2.1. Budget Management with a GUI
    2.2. Personal Blog

// Notes: 
   * Set up RPi web-server
   * Use XMind to map the idea first
   * Run basic website and deploy webapp to it. 
   * Check out RPi components and sensors. 
   * Buy a breadboard.
   * Check out Notepad++