Advanced Python (Fall 2017)/lecture6

Lecture 6: Concurrency

Concurrency is doing to things at once, not necessarily simultaneously.

Task 1.

Make a socket echo server

Solution

use google! this one is from https://pymotw.com/3/socket/tcp.html

# socket_echo_server.py

import socket
import sys

# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# Bind the socket to the port
server_address = ('localhost', 10000)
print('starting up on {} port {}'.format(*server_address))
sock.bind(server_address)

# Listen for incoming connections
sock.listen(1)

while True:
    # Wait for a connection
    print('waiting for a connection')
    connection, client_address = sock.accept()
    try:
        print('connection from', client_address)

        # Receive the data in small chunks and retransmit it
        while True:
            data = connection.recv(16)
            print('received {!r}'.format(data))
            if data:
                print('sending data back to the client')
                connection.sendall(data)
            else:
                print('no data from', client_address)
                break

    finally:
        # Clean up the connection
        connection.close()

You can connect with this echo server with the command nc localhost 10000

Need for concurrency

What happens if you try to open multiple connections to this socket server, and write a message?

Does it respond with an echo?

Async

python 3.4 adds a new library called asyncio which adds the ability to do ansychronous programming with python, using only the standard library. Asynchronous python, like javascript, has the ability to do something else when it is waiting.

function with async def are coroutines. actions which the async coroutine waits for are preceded with await and are generators.

Examples

Here is an example of doing things out of order, and sleeping without blocking: https://asyncio.readthedocs.io/en/latest/hello_clock.html

Here is a tcp echo server: https://asyncio.readthedocs.io/en/latest/tcp_echo.html

here is an example that will stay connected until the connection is closed by the client

import asyncio


async def handle_echo(reader, writer):
    while True:
        data = await reader.read(100)
        message = data.decode()
        addr = writer.get_extra_info('peername')
        print("Received %r from %r" % (message, addr))
        if len(message.strip()) == 0:
            print("Close the client socket")
            writer.close()
            break
        print("Send: %r" % message)
        writer.write(data)
        await writer.drain()


loop = asyncio.get_event_loop()
coro = asyncio.start_server(handle_echo, '127.0.0.1', 8888, loop=loop)
server = loop.run_until_complete(coro)

# Serve requests until Ctrl+C is pressed
print('Serving on {}'.format(server.sockets[0].getsockname()))
try:
    loop.run_forever()
except KeyboardInterrupt:
    pass

# Close the server
server.close()
loop.run_until_complete(server.wait_closed())
loop.close()

What happens if you make multiple connections with this server?

Threads

Threads can run concurrently, and they share the same memory.

this will take about 1 second to run using threads. without threads it would take 100 seconds

import threading
import time
threads = []


def foo():
    time.sleep(1)


for i in range(100):
    t = threading.Thread(target=foo)
    threads.append(t)

for t in threads:
    t.start()

for t in threads:
    t.join()

Thread safety

If you run this code with python 2, you will get different results each time you run it. It is not thread safe.

import threading
threads = []


n = 0

def foo():
    global n
    n += 1

for i in range(1000):
    t = threading.Thread(target=foo)
    threads.append(t)

for t in threads:
    t.start()

for t in threads:
    t.join()

print(n)