View source for Advanced Python (Fall 2017)/lecture5

= lecture 5 =

== piping stdout to stdin ==

you can send the outputs (stdout) of commands in shell to other commands, or write the to files

http://www.tldp.org/LDP/abs/html/io-redirection.html

To receive the output of a pipe into your python script, use <code>sys.stdin</code>. <code>sys.stdin</code> is an iterable.

To write to stdout, you can use <code>print</code> or <code>sys.stdout.write</code> -- which does not add the newline.

To print all stdin to stdout in a python script, you could use.

<source lang="python">import sys


def main()
    for line in sys.stdin:
        sys.stdout.write(line)


main()</source>
and to print out a file: <code>cat ./file.txt | python3 my_script.py</code>

To make an stream of a file that waits for new lines to be added, use the command

<code>tail -f ./file.txt</code>

and you can pipe this output to your python script.

== Task 1: Make a basic grep ==

something that can function as such <code>tail -f ./file.txt | grep $SEARCH_TERM</code> where lines with the search term are written to stdout

=== A Solution ===

Normal function

<source lang="python">import argparse
import sys


def grep(term, line):
    if term in line:
        sys.stdout.write(line)


def main():

    parser = argparse.ArgumentParser()
    parser.add_argument('term')
    args = parser.parse_args()
    term = args.term
    for line in sys.stdin:
        grep(term, line)


if __name__ == '__main__':
    main()</source>
== Event driven: Using a generator ==

the function <code>grep</code> is a generator. When the file object <code>sys.stdin</code> yields another line, then the generator <code>grep</code> yields another line. When <code>grep</code> yeilds another line, then the <code>for</code> loop prints another line.

<source lang="python">import argparse
import sys


def grep(term, lines):
    for line in lines:
        if term in line:
            print('grep executed')
            yield line


def main():

    parser = argparse.ArgumentParser()
    parser.add_argument('term')
    args = parser.parse_args()
    term = args.term
    grep_lines = grep(term, sys.stdin)
    for line in grep_lines:
        print('print executed')
        sys.stdout.write(line)


if __name__ == '__main__':
    main()
</source>
== Coroutine ==

Please see this tutorial for a better and fuller explanation of coroutines. NOTE: this tutorial is written in Python 2 code. http://www.dabeaz.com/coroutines/

Here is our grep written with a coroutine:

<source lang="python">import argparse
import sys


def grep(pattern):
    while True:
        line = (yield)
        if pattern in line:
            sys.stdout.write(line)


def main():

    parser = argparse.ArgumentParser()
    parser.add_argument('term')
    args = parser.parse_args()
    term = args.term
    g = grep(term)
    next(g)
    for line in sys.stdin:
        g.send(line)


if __name__ == '__main__':
    main()
</source>