SED – Stream Editor
Following
drawbacks of vi editor are overcomed using sed.
Doesn’t
work on large files
Only
one command at a time
Multiple
files cannot be edited simultaneously
Interactive
– makes changes to the source.
Advantages
of sed:
Line
oriented editor
Non
interactive – Changes to copy of the source
Multiple
files can be modified and multiple commands can be given.
sed - Replace or
substitute file contents
Let us consider a sample file, sample1.txt, as shown below:
apple
orange
banana
pappaya
1. To add something to the beginning of a every line in a file, say to add a word Fruit:
$ sed 's/^/Fruit: /' sample1.txt
Fruit: apple
Fruit: orange
Fruit: banana
Fruit: pappaya
The character 's' stands for substitution. What
follows 's' is the character, word or regular expression to replace followed by
character, word or regular expression to replace with. '/' is used to separate
the substitution character 's', the content to replace and the content to
replace with. The '^' character tells replace in the beginning and hence
everyline gets added the phrase 'Fruit: ' in the beginning of the line.
2. Similarly, to add something to the end of the file:
2. Similarly, to add something to the end of the file:
$ sed 's/$/ Fruit/' sample1.txt
apple Fruit
orange Fruit
banana Fruit
pappaya Fruit
The character '$' is used to denote the end of
the line. And hence this means, replace the end of the line with 'Fruit' which
effectively means to add the word 'Fruit' to the end of the line.
3. To replace or substitute a particular character, say to replace 'a' with 'A'.
3. To replace or substitute a particular character, say to replace 'a' with 'A'.
$ sed 's/a/A/' sample1.txt
Apple
orAnge
bAnana
pAppaya
Please note in every line only the
first occurrence of 'a' is being replaed, not all. The example shown
here is just for a single character replacement, which can be easily be done
for a word as well.
4. To replace or substitute all occurrences of 'a' with 'A'
4. To replace or substitute all occurrences of 'a' with 'A'
$ sed 's/a/A/g' sample1.txt
Apple
orAnge
bAnAnA
pAppAyA
5. Replace the first occurrence or all
occurrences is fine. What if we want to replace the second occurrence or
third occurrence or in other words nth occurrence.
To replace only the 2nd occurrence of a character :
To replace only the 2nd occurrence of a character :
$ sed 's/a/A/2' sample1.txt
apple
orange
banAna
pappAya
Please note above. The 'a' in apple has not
changed, and so is in orange since there is no 2nd occurrence of 'a' in this.
However, the changes have happened appropriately in banana and pappaya
6. Now, say to replace all occurrences from 2nd occurrence onwards:
6. Now, say to replace all occurrences from 2nd occurrence onwards:
$ sed 's/a/A/2g' sample1.txt
apple
orange
banAnA
pappAyA
7. Say, you want to replace 'a'
only in a specific line say 3rd line, not in the entire file:
$ sed '3s/a/A/g' sample1.txt
apple
orange
bAnAnA
pappaya
'3s' denotes the substitution to be done is only
for the 3rd line.
8. To replace or substitute 'a' on a range of lines, say from 1st to 3rd line:
8. To replace or substitute 'a' on a range of lines, say from 1st to 3rd line:
$ sed '1,3s/a/A/g' sample1.txt
Apple
orAnge
bAnAnA
pappaya
9. To replace the entire line
with something. For example, to replace 'apple' with 'apple is a Fruit'.
$ sed 's/.*/& is a Fruit/'
sample1.txt
apple is a Fruit
orange is a Fruit
banana is a Fruit
pappaya is a Fruit
The '&' symbol denotes the entire pattern
matched. In this case, since we are using '.*' which means matching the entire
line, '&' contains the entire line. This type of matching will be really
useful when you a file containing list of file names and you want to say rename
them as we have shown in one of our earlier articles:
10.
Using sed, we can also do multiple substitution. For example, say to
replace all 'a' to 'A', and 'p' to 'P':
$ sed 's/a/A/g; s/p/P/g' sample1.txt
APPle
orAnge
bAnAnA
PAPPAyA
OR This can also be done as:
$ sed -e 's/a/A/g' -e 's/p/P/g'
sample1.txt
APPle
orAnge
bAnAnA
PAPPAyA
The option '-e' is used when you have more than
one set of substitutions to be done.
OR The multiple substitution can also be done as shown below spanning multiple lines:
OR The multiple substitution can also be done as shown below spanning multiple lines:
$ sed -e 's/a/A/g' \
> -e 's/p/P/g' sample1.txt
APPle
orAnge
bAnAnA
PAPPAyA
Let us consider a file with the following contents:
$ cat file
RE01:EMP1:25:2500
RE02:EMP2:26:2650
RE03:EMP3:24:3500
RE04:EMP4:27:2900
1. To replace the first two(2) characters of a string or a line with say "XX":
$ sed 's/^../XX/' file
XX01:EMP1:25:2500
XX02:EMP2:26:2650
XX03:EMP3:24:3500
XX04:EMP4:27:2900
The "^" symbol indicates from
the beginning. The two dots indicate 2 characters.
The same thing can also be achieved without using the carrot(^) symbol as shown below. This also works because by default sed starts any operation from the beginning.
The same thing can also be achieved without using the carrot(^) symbol as shown below. This also works because by default sed starts any operation from the beginning.
sed 's/../XX/' file
2. In the same lines, to remove
or delete the first two characters of a string or a line.
$ sed 's/^..//' file
01:EMP1:25:2500
02:EMP2:26:2650
03:EMP3:24:3500
04:EMP4:27:2900
Here the string to be substituted is empty, and
hence gets deleted.
3. Similarly, to remove/delete the last two characters in the string:
3. Similarly, to remove/delete the last two characters in the string:
$ sed 's/..$//' file
RE01:EMP1:25:25
RE02:EMP2:26:26
RE03:EMP3:24:35
RE04:EMP4:27:29
4. To add a string to the end of a
line:
$ sed 's/$/.Rs/' file
RE01:EMP1:25:2500.Rs
RE02:EMP2:26:2650.Rs
RE03:EMP3:24:3500.Rs
RE04:EMP4:27:2900.Rs
Here the string ".Rs" is being
added to the end of the line.
5. To add empty spaces to the beginning of every line in a file:
5. To add empty spaces to the beginning of every line in a file:
$ sed 's/^/ /' file
RE01:EMP1:25:Rs.2500
RE02:EMP2:26:Rs.2650
RE03:EMP3:24:Rs.3500
RE04:EMP4:27:Rs.2900
To make any of the sed command change
permanent to the file OR in other words, to save or update the changes in the
same file, use the option "-i"
$ sed -i 's/^/ /' file
$ cat file
RE01:EMP1:25:Rs.2500
RE02:EMP2:26:Rs.2650
RE03:EMP3:24:Rs.3500
RE04:EMP4:27:Rs.2900
6. To remove empty spaces from the
beginning of a line:
$ sed 's/^ *//' file
RE01:EMP1:25:2500
RE02:EMP2:26:2650
RE03:EMP3:24:3500
RE04:EMP4:27:2900
"^ *"(space followed by a *)
indicates a sequence of spaces in the beginning.
7. To remove empty spaces from beginning and end of string.
7. To remove empty spaces from beginning and end of string.
$ sed 's/^ *//; s/ *$//' file
RE01:EMP1:25:2500
RE02:EMP2:26:2650
RE03:EMP3:24:3500
RE04:EMP4:27:2900
This example also shows to use multiple sed
command substitutions as part of the same command.
The same command can also be written as :
The same command can also be written as :
sed -e 's/^ *//' -e 's/ *$//' file
8. To add a character before and after a string. Or in other words, to encapsulate the string with something:
$ sed 's/.*/"&"/' file
"RE01:EMP1:25:Rs.2500"
"RE02:EMP2:26:Rs.2650"
"RE03:EMP3:24:Rs.3500"
"RE04:EMP4:27:Rs.2900"
".*" matches the entire line.
'&' denotes the pattern matched. The substitution pattern "&"
indicates to put a double-quote at the beginning and end of the string.
9. To remove the first and last character of a string:
9. To remove the first and last character of a string:
$ sed 's/^.//;s/.$//' file
RE01:EMP1:25:2500
RE02:EMP2:26:2650
RE03:EMP3:24:3500
RE04:EMP4:27:2900
10. To remove everything till the
first digit comes :
$ sed 's/^[^0-9]*//' file
01:EMP1:25:2500
02:EMP2:26:2650
03:EMP3:24:3500
04:EMP4:27:2900
Similarly, to remove everything till the first
alphabet comes:
sed 's/^[^a-zA-Z]*//' file
11. To remove a numerical word from
the end of the string:
$ sed 's/[0-9]*$//' file
RE01:EMP1:25:
RE02:EMP2:26:
RE03:EMP3:24:
RE04:EMP4:27:
12. To get the last column of a
file with a delimiter. The delimiter in this case is ":".
$ sed 's/.*://' file
2500
2650
3500
2900
For a moment, one can think the output of the
above command to be the same contents without the first column and the delim.
sed is greedy. When we tell, '.*:' it goes to the last column and consumes
everything. And hence, we only the get the content after the last colon.
13. To convert the entire line into lower case:
13. To convert the entire line into lower case:
$ sed 's/.*/\L&/' file
re01:emp1:25:rs.2500
re02:emp2:26:rs.2650
re03:emp3:24:rs.3500
re04:emp4:27:rs.2900
\L is the sed switch to convert to
lower case. The operand following the \L gets converted. Since &(the
pattern matched, which is the entire line in this case) is following \L, the
entire line gets converted to lower case.
14. To convert the entire line or a string to uppercase :
14. To convert the entire line or a string to uppercase :
$ sed 's/.*/\U&/' file
RE01:EMP1:25:RS.2500
RE02:EMP2:26:RS.2650
RE03:EMP3:24:RS.3500
RE04:EMP4:27:RS.2900
sed - Selective printing
In this sed article, we are going to see the different
options sed provides to selectively print contents in a file. Let us take
a sample file with the following contents:
$ cat file
Gmail 10
Yahoo 20
Redif 18
1. To print the entire file contents:
$ sed '' file
Gmail 10
Yahoo 20
Redif 18
2. To print only the line containing
'Gmail'. In other words, to simulate the grep command:
$ sed '/Gmail/p' file
Gmail 10
Gmail 10
Yahoo 20
Redif 18
Within the slashes, we specify the pattern which we
try to match. The 'p' command tells to print the line. Look at the above result
properly, the line Gmail got printed twice. Why? This is because the
default behavior of sed is to print every line after parsing it. On top of it,
since we asked sed to print the line containing the pattern 'Gmail' explicitly
by specifying 'p", the line 'Gmail' got printed twice. How to get the
desired result now?
$ sed -n '/Gmail/p' file
Gmail 10
The desired result can be obtained by
suppressing the default printing which can be done by using the option
"-n". And hence the above result.
3. To delete the line containing the pattern 'Gmail'. In other words, to simulate the "grep -v" command option in sed:
3. To delete the line containing the pattern 'Gmail'. In other words, to simulate the "grep -v" command option in sed:
$ sed '/Gmail/d' file
Yahoo 20
Redif 18
The "d" command denotes the delete
the pattern. As said earlier, the default action of sed is to print. Hence, all
the other lines got printed, and the line containing the pattern 'Gmail' got
deleted since we have specified explicit "d" option.
In the same lines, say to delete the first line of the file:
In the same lines, say to delete the first line of the file:
$ sed '1d' file
Yahoo 20
Redif 18
4. Print lines till you encounter a
specific pattern, say till 'Yahoo' is encountered.
$ sed '/Yahoo/q' file
Gmail 10
Yahoo 20
The "q" command tells to quit from
that point onwards. This sed command tells to keep printing(which is default)
and stop processing once the pattern "Yahoo" is encountered.
Printing Range of Lines:
Till now, what we saw is to retrieve a line or a set of lines based on a condition. Now, we will see how to get the same for a given range:
Consider the below sample file:
Printing Range of Lines:
Till now, what we saw is to retrieve a line or a set of lines based on a condition. Now, we will see how to get the same for a given range:
Consider the below sample file:
$ cat file
Gmail 10
Yahoo 20
Redif 18
Inbox 15
Live 23
Hotml 09
5. To print the first 3 lines, or from
lines 1 through 3:
$ sed -n '1,3p' file
Gmail 10
Yahoo 20
Redif 18
The option "-n" suppresses the default
printing. "1,3p" indicates to print from lines 1 to 3.
The same can also be achieved through:
The same can also be achieved through:
$ sed '3q' file
Gmail 10
Yahoo 20
Redif 18
3q denotes to quit after reading the 3rd line.
Since the "-n" option is not used, the first 3 lines get printed.
6. Similar to give line number ranges, sed can also work on pattern ranges. Say, to print from lines between patterns "Yahoo" and "Live":
6. Similar to give line number ranges, sed can also work on pattern ranges. Say, to print from lines between patterns "Yahoo" and "Live":
$ sed -n '/Yahoo/,/Live/p' file
Yahoo 20
Redif 18
Inbox 15
Live 23
The pattern is always specified between the slashes.
The comma operator is used to specify the range. This command tells to print
all those lines between the patterns "Yahoo" and 'Live".
7. To print the lines from pattern "Redif" till the end of the file.
7. To print the lines from pattern "Redif" till the end of the file.
$ sed -n '/Redif/,$p' file
Redif 18
Inbox 15
Live 23
Hotml 09
The earlier examples were line number ranges
and pattern ranges. sed allows us to use both (line number and pattern)
in the same command itself. This command indicates to print the lines from
pattern "Redif" till the end of the file($).
8. Similarly, to print contents from the beginning of the file till the pattern "Inbox":
8. Similarly, to print contents from the beginning of the file till the pattern "Inbox":
$ sed -n '1,/Inbox/p' file
Gmail 10
Yahoo 20
Redif 18
Inbox 15
sed - Include or Append a
line to a file
sed is one of the most important editors we use in UNIX. It
supports lot of file editing tasks. In this article, we will see a specific set
of sed options.
Assume I have a flat file, empFile, containing employee name and employee id as shown below:
Assume I have a flat file, empFile, containing employee name and employee id as shown below:
Hilesh, 1001
Bharti, 1002
Aparna, 1003
Harshal, 1004
Keyur, 1005
$ sed '1i Employee, EmpId'
empFile
Employee, EmpId
Hilesh, 1001
Bharti, 1002
Aparna, 1003
Harshal, 1004
Keyur, 1005
This command does the following: The number '1'
tells the operation is to be done only for the first line. 'i' stands for
including the following content before reading the line. So, '1i' means to
include the following before reading the first line and hence we got the header
in the file.
However, the file with the header is displayed only in the output, the file contents still remain the old file. So, if the user's requirement is to update the original file with this output, the user has to re-direct the output of the sed command to a temporary file and then move it to the original file.
The UNIX system which has the GNU version contains sed with the '-i' option. This option of the sed command is used to edit the file in-place. Let us see the same above example using '-i' option:
However, the file with the header is displayed only in the output, the file contents still remain the old file. So, if the user's requirement is to update the original file with this output, the user has to re-direct the output of the sed command to a temporary file and then move it to the original file.
The UNIX system which has the GNU version contains sed with the '-i' option. This option of the sed command is used to edit the file in-place. Let us see the same above example using '-i' option:
$ cat empFile
Hilesh, 1001
Bharti, 1002
Aparna, 1003
Harshal, 1004
Keyur, 1005
$ sed -i '1i Employee, EmpId'
empFile
$ cat empFile
Employee, EmpId
Hilesh, 1001
Bharti, 1002
Aparna, 1003
Harshal, 1004
Keyur, 1005
As shown above, the '-i' option edits the file
in-place without the need of a temporary file.
2. How to add a line '-------' after the header line or the 1st line?
2. How to add a line '-------' after the header line or the 1st line?
$ sed -i '1a ---------------'
empFile
$ cat empFile
Employee, EmpId
---------------
Hilesh, 1001
Bharti, 1002
Aparna, 1003
Harshal, 1004
Keyur, 1005
'1i' is similar to '1a' except that 'i'
tells to include the content before reading the line, 'a' tells to include the
content after reading the line. And hence in this case, the '----' line gets
included after the 1st line. As you thought correctly, even if you had used
'2i', it will work well and fine.
3. How to add a trailer line to this file?
3. How to add a trailer line to this file?
$ sed -i '$a ---------------'
empFile
$ cat empFile
Employee, EmpId
---------------
Hilesh, 1001
Bharti, 1002
Aparna, 1003
Harshal, 1004
Keyur, 1005
---------------
To add to the last line of the file, we need to
know the total line count of the file to use in the above mentioned methods.
However, sed has the '$' symbol which denotes the last line. '$a' tells to
include the following content after reading the last line of the file.
4. How to add a record after a particular record?
Let us assume the sample file contains only 3 records as shown below:
4. How to add a record after a particular record?
Let us assume the sample file contains only 3 records as shown below:
Employee, EmpId
---------------
Hilesh, 1001
Harshal, 1004
Keyur, 1005
---------------
Now, if I want to insert the record for the employee
'Bharti' after the employee 'Hilesh':
$ sed -i '/Hilesh/a Bharti, 1002'
empFile
$ cat empFile
Employee, EmpId
---------------
Hilesh, 1001
Bharti, 1002
Harshal, 1004
Keyur, 1005
---------------
If you note the above sed command
carefully, all we have done is in place of a number, we have used a pattern.
/Hilesh/a tells to include the following contents after finding the pattern
'Hilesh', and hence the result.
5. How to add a record before a particular record? Say, add the record for the employee 'Aparna' before the employee record of 'Harshal'
5. How to add a record before a particular record? Say, add the record for the employee 'Aparna' before the employee record of 'Harshal'
$ sed -i '/Harshal/i Aparna, 1003'
empFile
$ cat empFile
Employee, EmpId
---------------
Hilesh, 1001
Bharti, 1002
Aparna, 1003
Harshal, 1004
Keyur, 1005
---------------
Similarly, /Harshal/i
tells to include the following contents before reading the line containing the
pattern 'Harshal'.
Note: As said above, the '-i' option will only work if the sed is GNU sed. Else the user has to re-direct the output to a temporary file and move it to the original file.
Note: As said above, the '-i' option will only work if the sed is GNU sed. Else the user has to re-direct the output to a temporary file and move it to the original file.
sed - Read from a file or
write into a file
In this sed article, we will see how to read a file into a
sed output, and also how to write a section of a file content to a different
file.
Let us assume we have 2 files, file1 and file2 with the following content:
Let us assume we have 2 files, file1 and file2 with the following content:
$ cat file1
1apple
1banana
1mango
$ cat file2
2orange
2strawberry
sed has 2 options for reading and writing:
r filename : To read a file name content specified in the filename
w filename : To write to a file specified in the filename
Let us see some examples now:
1. Read the file2 after every line of file1.
r filename : To read a file name content specified in the filename
w filename : To write to a file specified in the filename
Let us see some examples now:
1. Read the file2 after every line of file1.
$ sed 'r file2' file1
1apple
2orange
2strawberry
1banana
2orange
2strawberry
1mango
2orange
2strawberry
r file2 reads the file contents of file2. Since
there is no specific number before 'r', it means to read the file contents of
file2 for every line of file1. And hence the above output.
2. The above output is not very useful. Say, we want to read the file2 contents after the 1st line of file1:
2. The above output is not very useful. Say, we want to read the file2 contents after the 1st line of file1:
$ sed '1r file2' file1
1apple
2orange
2strawberry
1banana
1mango
'1r' indicates to read the contents of file2
only after reading the line1 of file1.
3. Similarly, we can also try to read a file contents on finding a pattern:
3. Similarly, we can also try to read a file contents on finding a pattern:
$ sed '/banana/r file2' file1
1apple
1banana
2orange
2strawberry
1mango
The file2 contents are read on finding the
pattern banana and hence the above output.
4. To read a file content on encountering the last line:
4. To read a file content on encountering the last line:
$ sed '$r file2' file1
1apple
1banana
1mango
2orange
2strawberry
The '$' indicates the last line, and hence the
file2 contents are read after the last line. Hey, hold on. The above example is
put to show the usage of $ in this scenario. If your requirement is really
something like above, you need not use sed. cat file1 file2 will do :) .
Let us now move onto the writing part of sed. Consider a file, file1, with the below contents:
Let us now move onto the writing part of sed. Consider a file, file1, with the below contents:
$ cat file1
apple
banana
mango
orange
strawberry
1. Write the lines from 2nd to 4th to a file, say file2.
$ sed -n '2,4w file2' file1
The option '2,4w' indicates to write the lines
from 2 to 4. What is the option "-n" for? By default, sed prints
every line it reads, and hence the above command without "-n" will
still print the file1 contents on the standard output. In order to suppress
this default output, "-n' is used. Let us print the file2 contents
to check the above output.
$ cat file2
banana
mango
orange
Note: Even after running the above command, the file1 contents
still remain intact.
2. Write the contents from the 3rd line onwards to a different file:
2. Write the contents from the 3rd line onwards to a different file:
$ sed -n '3,$w file2' file1
$ cat file2
mango
orange
strawberry
As explained earlier, the '3,$' indicates from
3 line to end of the file.
3. To write a range of lines, say to write from lines apple through mango :
3. To write a range of lines, say to write from lines apple through mango :
$ sed -n '/apple/,/mango/w file2'
file1
$ cat file2
apple
banana
mango
sed - Delete a line or
pattern in a file
Let us consider a file with the sample contents as
below:
$ cat file
Cygwin
Unix
Linux
Solaris
AIX
1. Delete the 1st line or the
header line:
$ sed '1d' file
Unix
Linux
Solaris
AIX
d command is to delete a line. 1d means to delete the first line.
The above command will show the file content by deleting the first line.
However, the source file remains unchanged. To update the original file itself
with this deletion or to make the changes permanently in the source file, use
the -i option. The same is applicable for all the other examples.
sed -i '1d' file
Note: -i option in sed is
available only if it is GNU sed. If not GNU, re-direct the sed output to a
file, and rename the output file to the original file.
2. Delete a particular line, 3rd line in this case:
2. Delete a particular line, 3rd line in this case:
$ sed '3d' file
Cygwin
Unix
Solaris
AIX
3. Delete the last line or the
trailer line of the file:
$ sed '$d' file
Cygwin
Unix
Linux
Solaris
$ indicates the last line.
4. Delete a range of lines, from 2nd line till 4th line:
4. Delete a range of lines, from 2nd line till 4th line:
$ sed '2,4d' file
Cygwin
AIX
The range is specified using the
comma operator.
5. Delete lines other than the specified range, line other than 2nd till 4th here:
5. Delete lines other than the specified range, line other than 2nd till 4th here:
$ sed '2,4!d' file
Unix
Linux
Solaris
The ! operator indicates negative
condition.
6. Delete the first line AND the last line of a file, i.e, the header and trailer line of a file.
6. Delete the first line AND the last line of a file, i.e, the header and trailer line of a file.
$ sed '1d;$d' file
Unix
Linux
Solaris
Multiple conditions are separated using
the ';' operator. Similarly, say to delete 2nd and 4th line, you can use:
'2d;3d'.
7. Delete all lines beginning with a particular character, 'L' in this case:
7. Delete all lines beginning with a particular character, 'L' in this case:
$ sed '/^L/d' file
Cygwin
Unix
Solaris
AIX
'^L' indicates lines beginning with L.
8. Delete all lines ending with a particular character, 'x' in this case:
8. Delete all lines ending with a particular character, 'x' in this case:
$ sed '/x$/d' file
Cygwin
Solaris
AIX
'x$' indicates lines ending with 'x'. AIX
did not get deleted because the X is capital.
9. Delete all lines ending with either x or X, i.e case-insensitive delete:
9. Delete all lines ending with either x or X, i.e case-insensitive delete:
$ sed '/[xX]$/d' file
Cygwin
Solaris
[xX] indicates either 'x' or 'X'. So, this
will delete all lines ending with either small 'x' or capital 'X'.
10. Delete all blank lines in the file
10. Delete all blank lines in the file
$ sed '/^$/d' file
Cygwin
Unix
Linux
Solaris
AIX
'^$' indicates lines containing nothing
and hence the empty lines get deleted. However, this wont delete lines
containing only some blank spaces.
11. Delete all lines which are empty or which contains just some blank spaces:
11. Delete all lines which are empty or which contains just some blank spaces:
$ sed '/^ *$/d' file
Cygwin
Unix
Linux
Solaris
AIX
'*' indicates 0 or more occurrences of
the previous character. '^ *$' indicates a line containing zero or more spaces.
Hence, this will delete all lines which are either empty or lines with only
some blank spaces.
12. Delete all lines which are entirely in capital letters:
12. Delete all lines which are entirely in capital letters:
$ sed '/^[A-Z]*$/d' file
Cygwin
Unix
Linux
Solaris
[A-Z] indicates any character matching
the alphabets in capital.
13. Delete the lines containing the pattern 'Unix'.
13. Delete the lines containing the pattern 'Unix'.
$ sed '/Unix/d' file
Cygwin
Linux
Solaris
AIX
The pattern is specified within a pair
of slashes.
14. Delete the lines NOT containing the pattern 'Unix':
14. Delete the lines NOT containing the pattern 'Unix':
$ sed '/Unix/!d' file
Unix
15. Delete the lines containing the pattern 'Unix' OR
'Linux':
$ sed '/Unix\|Linux/d' file
Cygwin
Solaris
AIX
The OR condition is specified using the
| operator. In order not to get the pipe(|) interpreted as a literal, it is
escaped using a backslash.
16. Delete the lines starting from the 1st line till encountering the pattern 'Linux':
16. Delete the lines starting from the 1st line till encountering the pattern 'Linux':
$ sed '1,/Linux/d' file
Solaris
AIX
Earlier, we saw how to delete a range of
lines. Range can be in many combinations: Line ranges, pattern ranges, line and
pattern, pattern and line.
17. Delete the lines starting from the pattern 'Linux' till the last line:
17. Delete the lines starting from the pattern 'Linux' till the last line:
$ sed '/Linux/,$d' file
Cygwin
Unix
18. Delete the last line ONLY if it contains the pattern
'AIX':
$ sed '${/AIX/d;}' file
Cygwin
Unix
Linux
Solaris
$ is for the last line. To delete a
particular line only if it contains the pattern AIX, put the line number in
place of the $. This is how we can implement the 'if' condition in sed.
19. Delete the last line ONLY if it contains either the pattern 'AIX' or 'HPUX':
19. Delete the last line ONLY if it contains either the pattern 'AIX' or 'HPUX':
$ sed '${/AIX\|HPUX/d;}' file
Cygwin
Unix
Linux
Solaris
20. Delete the lines containing the pattern 'Solaris' only
if it is present in the lines from 1 to 4.
$ sed '1,4{/Solaris/d;}' file
Cygwin
Unix
Linux
AIX
This will only delete the lines
containing the pattern Solaris only if it is in the 1st four lines, nowhere
else.
21. Delete the line containing the pattern 'Unix' and also the next line:
21. Delete the line containing the pattern 'Unix' and also the next line:
$ sed '/Unix/{N;d;}' file
Cygwin
Solaris
AIX
N command reads the next line in the
pattern space. d deletes the entire pattern space which contains the current
and the next line.
22. Delete only the next line containing the pattern 'Unix', not the very line:
22. Delete only the next line containing the pattern 'Unix', not the very line:
$ sed '/Unix/{N;s/\n.*//;}' file
Cygwin
Unix
Solaris
AIX
Using the substitution command s, we
delete from the newline character till the end, which effective deletes the
next line after the line containing the pattern Unix.
23. Delete the line containing the pattern 'Linux', also the line before the pattern:
23. Delete the line containing the pattern 'Linux', also the line before the pattern:
$ sed -n
'/Linux/{s/.*//;x;d;};x;p;${x;p;}' file | sed '/^$/d'
Cygwin
Solaris
AIX
A little tricky ones.
In order to delete the line prior to the pattern,we store every line in a
buffer called as hold space. Whenever the pattern matches, we delete the content
present in both, the pattern space which contains the current line, the hold
space which contains the previous line.
Let me explain this
command: 'x;p;' ; This gets executed for every line. x exchanges the content of
pattern space with hold space. p prints the pattern space. As a result, every
time, the current line goes to hold space, and the previous line comes to
pattern space and gets printed. When the pattern /Linux/ matches, we
empty(s/.*//) the pattern space, and exchange(x) with the hold space(as a
result of which the hold space becomes empty) and delete(d) the pattern space
which contains the previous line. And hence, the current and the previous line
gets deleted on encountering the pattern Linux. The ${x;p;} is to print the
last line which will remain in the hold space if left.
The second part of sed is to
remove the empty lines created by the first sed command.
24. Delete only the line prior to the line containing the pattern 'Linux', not the very line:
$
sed -n '/Linux/{x;d;};1h;1!{x;p;};${x;p;}' file
Cygwin
Linux
Solaris
AIX
This is almost
same as the last one with few changes. On encountering the pattern /Linux/, we
exchange(x) and delete(d). As a result of exchange, the current line remains in
hold space, and the previous line which came into pattern space got deleted.
1h;1!{x;p;} - 1h
is to move the current line to hold space only if it first line. Exchange and
print for all the other lines. This could easily have been simply: x;p . The
drawback is it gives an empty line at the beginning because during the first
exchange between the pattern space and hold space, a new line comes to pattern
space since hold space is empty.
25. Delete the line containing the pattern 'Linux', the line before, the line after:
$ sed -n '/Linux/{N;s/.*//;x;d;};x;p;${x;p;}' file | sed '/^$/d'
Cygwin
AIX
With the
explanations of the last 2 commands, this should be fairly simple to
understand.
Different ways to print
the next few lines after pattern match
grep, awk or a sed command is used to print the line
matching a particular pattern. However, at times, we need to print a few
more lines following the lines matching the pattern. In this article, we will
see the different ways in which we can get this done. The first part explains
ways to print one line following the pattern along with the pattern, few
examples to print only the line following the pattern and in the end we have
ways to print multiple lines following the pattern.
Let us consider a file with the following contents as shown below:
Let us consider a file with the following contents as shown below:
$ cat file
Unix
Linux
Solaris
AIX
SCO
1. The simplest is using the grep
command. In GNU grep, there is an option -A which prints lines following
the pattern.
$ grep -A1 Linux file
Linux
Solaris
In the above example, -A1 will print one line following
the pattern along with the line matching the pattern. To print 2 lines after
the pattern, it is -A2.
2. sed has the N command which will read the next line into the pattern space.
2. sed has the N command which will read the next line into the pattern space.
$ sed -n '/Linux/{N;p}' file
Linux
Solaris
First, the line containing the pattern /Linux/
is found. The command within the braces will run on the pattern found. {N;p}
means read the next line and print the pattern space which now contains the
current and the next line. Similarly, to print 2 lines, you can simply put: {N;N;p}.
Example 7 onwards explains for printing multiple lines following the
pattern.
3. awk has the getline command which reads the next line from the file.
3. awk has the getline command which reads the next line from the file.
$ awk
'/Linux/{print;getline;print;}' file
Linux
Solaris
Once the line containing the pattern Linux is
found, it is printed. getline command reads the next line into $0. Hence, the
second print statement prints the next line in the file.
4. In this, the same thing is achieved using only one print statement.
4. In this, the same thing is achieved using only one print statement.
$ awk '/Linux/{getline x;print $0 RS
x;}' file
Linux
Solaris
getline x reads the next line
into variable x. x is used in order to prevent the getline from overwriting the
current line present in $0. The print statement prints the current line($0),
record separator(RS) which is the newline, and the next line which is in x.
5. To print only the line following the pattern without the line matching the pattern:
5. To print only the line following the pattern without the line matching the pattern:
$ sed -n '/Linux/{n;p}' file
Solaris
The n command reads the next
line into the pattern space thereby overwriting the current line. On printing
the pattern space using the p command, we get the next line printed.
6. Same using awk:
6. Same using awk:
$ awk '/Linux/{getline;print;}' file
Solaris
Multiple lines after the pattern:
GNU grep may not available in every Unix box. Excluding grep option, the above examples are good only to print a line or two following the pattern. Say, if you have to print some 10 lines after the pattern, the command will get clumsy. Let us now see how to print n lines following the pattern along with the pattern:
7. To print multiple(2) lines following the pattern using awk:
GNU grep may not available in every Unix box. Excluding grep option, the above examples are good only to print a line or two following the pattern. Say, if you have to print some 10 lines after the pattern, the command will get clumsy. Let us now see how to print n lines following the pattern along with the pattern:
7. To print multiple(2) lines following the pattern using awk:
$ awk
'/Linux/{x=NR+2}(NR<=x){print}' file
Linux
Solaris
Aix
To print 5 lines after the pattern, simply
replace the number 2 with 5. This above example is a little tricky. Once the
pattern Linux is found, x is calculated which is current line number(NR) plus
2. So, we will print lines from the current line till the line number(NR)
reaches x.
8. To print 2 lines following the pattern without the line matching the pattern:
8. To print 2 lines following the pattern without the line matching the pattern:
$ awk '/Linux/{x=NR+2;next}(NR<=x){print}'
file
Solaris
Aix
The next command makes the current line, which
is the pattern matched, to get skipped. In this way, we can exclude the line
matching the pattern from getting printed.
9. To print 2 lines following the pattern along with the pattern matching line in another way.
9. To print 2 lines following the pattern along with the pattern matching line in another way.
$ x=`grep -n Linux file | cut -f1 -d:`
$ awk -v ln=$x 'NR>=ln
&& NR<=ln+2' file
Using grep and cut command, the line number of
the pattern in the file is found. By passing the shell variable to awk,
we make it print only those lines whose line number is between x
and x+2.
10. One more way using sed and awk combination. First we calculate the from and to line numbers in the variables x and y. Using sed printing range of lines, we can get the same. sed can not only deal with numbers, it can work with variables as well:
10. One more way using sed and awk combination. First we calculate the from and to line numbers in the variables x and y. Using sed printing range of lines, we can get the same. sed can not only deal with numbers, it can work with variables as well:
$ x=`awk '/Linux/{print NR}' file`
$ y=`awk '/Linux/{print NR+2}' file`
$ sed -n "$x,$y p" file
OR
$ x=`awk '/Linux/{print NR+2}' file`
$ sed -n "/Linux/,$x p"
file
Sed – Printing Line number
‘=’ command
Line number are
printed because of ‘=’ command
Sed ‘=’ file1
Sed –n ‘=’ file1 -> prints only line numbers
Sed –n ‘$=’ file1 -> prints last line number
Sed –n ‘/^root/=’ file1 -> prints line number of the line
starting with root