Eroman: Created page with "= lecture 5 = == piping stdout to stdin == you can send the outputs (stdout) of commands in shell to other commands, or write the to files http://www.tldp.org/LDP/abs/html/..."

2017-10-10T09:12:54Z

Created page with "= lecture 5 = == piping stdout to stdin == you can send the outputs (stdout) of commands in shell to other commands, or write the to files http://www.tldp.org/LDP/abs/html/..."

New page

= lecture 5 =

== piping stdout to stdin ==

you can send the outputs (stdout) of commands in shell to other commands, or write the to files

http://www.tldp.org/LDP/abs/html/io-redirection.html

To receive the output of a pipe into your python script, use <code>sys.stdin</code>. <code>sys.stdin</code> is an iterable.

To write to stdout, you can use <code>print</code> or <code>sys.stdout.write</code> -- which does not add the newline.

To print all stdin to stdout in a python script, you could use.

<source lang="python">import sys

def main()
for line in sys.stdin:
sys.stdout.write(line)

main()</source>
and to print out a file: <code>cat ./file.txt | python3 my_script.py</code>

To make an stream of a file that waits for new lines to be added, use the command

<code>tail -f ./file.txt</code>

and you can pipe this output to your python script.

== Task 1: Make a basic grep ==

something that can function as such <code>tail -f ./file.txt | grep $SEARCH_TERM</code> where lines with the search term are written to stdout

=== A Solution ===

Normal function

<source lang="python">import argparse
import sys

def grep(term, line):
if term in line:
sys.stdout.write(line)

def main():

parser = argparse.ArgumentParser()
parser.add_argument('term')
args = parser.parse_args()
term = args.term
for line in sys.stdin:
grep(term, line)

if __name__ == '__main__':
main()</source>
== Event driven: Using a generator ==

the function <code>grep</code> is a generator. When the file object <code>sys.stdin</code> yields another line, then the generator <code>grep</code> yields another line. When <code>grep</code> yeilds another line, then the <code>for</code> loop prints another line.

<source lang="python">import argparse
import sys

def grep(term, lines):
for line in lines:
if term in line:
print('grep executed')
yield line

def main():

parser = argparse.ArgumentParser()
parser.add_argument('term')
args = parser.parse_args()
term = args.term
grep_lines = grep(term, sys.stdin)
for line in grep_lines:
print('print executed')
sys.stdout.write(line)

if __name__ == '__main__':
main()
</source>
== Coroutine ==

Please see this tutorial for a better and fuller explanation of coroutines. NOTE: this tutorial is written in Python 2 code. http://www.dabeaz.com/coroutines/

Here is our grep written with a coroutine:

<source lang="python">import argparse
import sys

def grep(pattern):
while True:
line = (yield)
if pattern in line:
sys.stdout.write(line)

def main():

parser = argparse.ArgumentParser()
parser.add_argument('term')
args = parser.parse_args()
term = args.term
g = grep(term)
next(g)
for line in sys.stdin:
g.send(line)

if __name__ == '__main__':
main()
</source>

Advanced Python (Fall 2017)/lecture5 - Revision history

Eroman: Created page with "= lecture 5 = == piping stdout to stdin == you can send the outputs (stdout) of commands in shell to other commands, or write the to files http://www.tldp.org/LDP/abs/html/..."