SED eng

SED - sed is a stream editor. The major use of stream editor is to perform basic text transformations on an input stream (a file or input from a pipeline). While in some ways similar to an editor which permits scripted edits (such as ed), sed works by making only one pass over the input(s), and is consequently more efficient. But it is sed’s ability to filter text in a pipeline which particularly distinguishes it from other types of editors. it basically means that all the editing are made by calling the command and sed will execute the directions automatically. sed is a very powerful and fast way to transform text.

Introduction

sed stream editor is a text editor that performs editing operations on information coming from standard input or a file. Sed reads and edits text line-by-line and in a non-interactive way.

Main Operation and usage

sed is a line-oriented text processing utility: it reads text, line by line, from an input stream or file, into an internal buffer called the pattern space. Each line read starts a cycle. To the pattern space, sed applies one or more operations which have been specified via a sed script. sed implements a programming language with about 25 commands that specify the operations on the text. For each input line, after running the script sed ordinarily outputs the pattern space (the line as modified by the script) and begins the cycle again with the next line. Other end-of-script behaviors are available through sed options and script commands, e.g. d to delete the pattern space, q to quit, N to add the next line to the pattern space immediately, and so on. Thus a sed script corresponds to the body of a loop that iterates through the lines of a stream, where the loop itself and the loop variable (the current line number) are implicit and maintained by sed.

The sed script can either be specified on the command line (-e option) or read from a separate file (-f option). Commands in the sed script may take an optional address, in terms of line numbers or regular expressions. The address determines when the command is run. For example, 2d would only run the d (delete) command on the second input line (printing all lines but the second), while /^ /d would delete all lines beginning with a space.

In general, sed operates on a stream of text that it reads from either standard input or from a file.which means that all the editing are made by calling the command and sed will execute the directions automatically. Have it in mind that sed outputs everything to standard out by default. which means that, unless redirected, sed print its output to the screen instead of saving it in a file.

RUNNING SED

Sed can be run and invoked as follow

   sed SCRIPT INPUTFILE.

In order to replace all occurrences of ‘hello’ to ‘world’ in the file input.txt:

   sed 's/hello/world/' input.txt > output.txt.

If an INPUTFILE is not specified, or if INPUTFILE is -, sed filters the contents of the standard input. The following commands are thesame:

   sed 's/hello/world/' input.txt > output.txt
   sed 's/hello/world/' < input.txt > output.txt
   cat input.txt | sed 's/hello/world/' - > output.txt.

sed writes output to standard output. Use -i to edit files in-place instead of printing to standard output. See also the W and s///w commands for writing output to other files. The following command modifies file.txt and does not produce any output:

   sed -i 's/hello/world' file.txt

By default sed prints all processed input (except input that has been modified/deleted by commands such as d). Use -n to suppress output, and the p command to print specific lines. The following command prints only line 50 of the input file

   sed -n '50p' file.txt

sed treats multiple input files as one long stream. The following example prints the first line of the first file (one.txt) and the last line of the last file (three.txt). Use -s to reverse this behavior.

   sed -n  '1p ; $p' one.txt two.txt three.txt

Without -e or -f options, sed uses the first non-option parameter as the script, and the following non-option parameters as input files. If -e or -f options are used to specify a script, all non-option parameters are taken as input files. Options -e and -f can be combined, and can appear multiple times (in which case the final effective script will be concatenation of all the individual scripts).

The following examples are equivalent:

   sed 's/hello/world/' input.txt > output.txt
  
   sed -e 's/hello/world/' input.txt > output.txt
   sed --expression='s/hello/world/' input.txt > output.txt
  
   echo 's/hello/world/' > myscript.sed
   sed -f myscript.sed input.txt > output.txt
   sed --file=myscript.sed input.txt > output.txt

USE AND EDITTING OF FILES

sed enable you to work on a file that you've already created and also sed print its output to the screen instead of saving it in a file. it give you a flexible way to edit a file to the standard you want it to be.

Taking an instance, let's copy some files into our home directory to practice some editing.

   cd
   cp /usr/share/common-licenses/BSD .
   cp /usr/share/common-licenses/GPL-3 .

Now, Let's use sed to view the contents of the BSD license file we copied.

   sed  BSD
   

The output of the file is:

   Copyright (c) The Regents of the University of California.
   All rights reserved.
   
   Redistribution and use in source and binary forms, with or without
   modification, are permitted provided that the following conditions
   are met:
   1. Redistributions of source code must retain the above copyright
      notice, this list of conditions and the following disclaimer.
   2. Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.

Demonstrate how sed can use standard input by piping the output of the "cat" command into sed to produce the same result.

   cat BSD | sed 

   Copyright (c) The Regents of the University of California.
   All rights reserved.
   
   Redistribution and use in source and binary forms, with or without
   modification, are permitted provided that the following conditions
   are met:
   1. Redistributions of source code must retain the above copyright
      notice, this list of conditions and the following disclaimer.
   2. Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.
   . . .
   . . .

As shown below, files can be worked on or streams of text (as is produced when piping output with the "|" character)

Printing Lines

from the previous example, input passed into sed without any operations and the results was prited directly to standard output. when using sed to print, the "print" command, is specified by using the "p" character within single quotes.

   sed 'p' filename

With this print command, sed will print each line twice because it automatically prints each line, note that we've told it to print explicitly with the "p" command. sed operates line by line. It accepts a line, operates on it, and outputs the resulting text before repeating the process on the next line.

The printing results can be clean by passing the "-n" option to sed, which suppresses the automatic printing and output the actual command that its given to print:

   sed -n 'p' filename

This command will print out all the lines in the file as it was without duplicating or printing out a specific line. the example shown below can not be considered to be 'Editing', it has to at least print a specific line or make changes to the file before it can be seen as being edited.

Address specification

Sed can print just the first line by using this command:

   sed -n '1p' filename

placing the number "1" before the print command"p", it enable sed to know the line number to operate on. more than one line can easily be printed by indicating the number of lines to be printed in the command example

   sed -n '1,6p' filename

in this example, sed will print from line 1 through to line 6, the command gives sed an address range and will only execute the commands that follow on those line. this can be done in another way by by giving the first address and then using an offset to tell sed how many additional lines to travel:

   sed -n '1,+5p' filename

This will basically result in the same output, sed will start at line 1 and then operate on the next 5 lines as well.

To print every other line, specify the interval after the "~" character. The following line will print every other line starting with line 1:

   sed -n '1~3p' filename

Deleting Text

A text can easily be deleted by changing the previous text printing "p" command to "d" command.

The "-n" command will not not needed because with the delete command, sed will print everything that is not deleted, which will what's going on.

Using the last command from the previous section to make it delete every other line starting with the first.

   sed '1~3d' filename

It is important to note that the source file is will not be affected. It is still intact. The edits are output to the screen.

To save the edits, redirect standard output to a file like so:

   sed '1~2d' filename > anotherfile.txt

To create a backup file prior to editing, add the backup extension directly after the "-i" option:

   sed -i.bak '1~2d' anotherfile.txt

This will create a backup file with the ".bak" extension, and then edit the regular file in-place.

Substituting Text

one of the main use of sed is substituting of text. Sed has the ability to search for text patterns using regular expressions, and then replace the found text.

- click for more information about text patterns

the following syntax can change one word to another word

   's/old_word/new_word/'

The "s" is the substitute command. The three slashes (/) are used to separate the different text fields. other characters can be use to delimit the fields if it would be more helpful. For instance, when trying to change a website name, using another delimiter would be helpful since URLs contain slashes. use echo to pipe in an example:

   echo "http://www.example.com/index.html" | sed 's_com/index_org/home_'

   http://www.example.org/home.html

Do not forget the final delimiter, or sed will complain.

   echo "http://www.example.com/index.html" | sed 's_com/index_org/home'

   sed: -e expression #1, char 22: unterminated `s' command

Let's create a file to practice substitutions on:

   echo "i want to be good
   it actually takes nothing to be good actually
   good music help me think positively
   that could result to my good deed
   and motivate me to stand by you
   that is absolutely life..." > funny.txt

substitute the expression "actually" with"really".

   sed 's/actully/really/' funny.txt

Output

   i want to be good
   it really take nothing to be good actually
   good music help me think positively
   that could result to my good deed
   and motivate me to stand by you
   that is absolutely life..." > funny.txt

First note that, patterns is replaced , not words. The first "actually" on the second line changed to "really" but the second actually did not.

this is because the "s" command operates on the first match in a line and then moves to the next line.

To make sed replace every instance of "on" instead of just the first on each line, an optional flag can be pass to the substitute command.

"g" flag will affect the substitute command by placing it after the substitution set.

   sed 's/actually/really/g' funny.txt

   i want to be good
   it really take nothing to be good really
   good music help me think positively
   that could result to my good deed
   and motivate me to stand by you
   that is absolutely life..." > funny.txt

To change the second instance of "actually" that sed finds on each line, then use the number "2" instead of the "g".

   sed 's/actually/really/2' funny.txt

   i want to be good
   it actually take nothing to be good really
   good music help me think positively
   that could result to my good deed
   and motivate me to stand by you
   that is absolutely life..." > funny.txt

To see only the lines that were substituted, use the "-n" option again to suppress automatic printing. then pass the "p" flag to the substitute command to print lines where substitution took place.

   sed -n 's/actually/really/2p' funny.text

   it actually take nothing to be good really

For the search process to ignore case, pass it the "i" flag.

   sed 's/MUSIC/life/i' funny.txt

   i want to be good
   it actually take nothing to be good really
   good life help me think positively
   that could result to my good deed
   and motivate me to stand by you
   that is absolutely life..." > funny.txt

Matching Text

To find more complex patterns with regular expressions, there are number of different methods of referencing that matched pattern in the replacement text. For instance,to match the from the beginning of the line to "to" we can use the expression:

   sed 's/^.*to/REPLACED/' funny.txt

   REPLACED be good
   it actually take nothing to be good really
   good life help me think positively
   that could result to my good deed
   and motivate me to stand by you
   that is absolutely life..." > funny.txt

You can see that the wildcard expression matches from the beginning of the line to the last instance of "to"

Since you don't know the exact phrase that will match in the search string, you can use the "&" character to represent the matched text in the replacement string.

This example shows how to put parentheses around the matched text:

   sed 's/^.*to/(&)/' annoying.txt

   (i want to) be good
   it actually take nothing to be good really
   good life help me think positively
   that could result to my good deed
   and motivate me to stand by you
   that is absolutely life..." > funny.txt