grep piped into awk

This has been one of my bug bears for years and it comes up in email all the time. I blame it on the VAX I used years ago that was so slow that this really mattered at the time but now, mostly it is just because it is wrong. This is what a mean, grep piped into awk:

grep foo | awk '{ print $1 }'



Why? Because awk can do it all for you, saving a pipe and a for and exec:

nawk '/foo/ { print $1 }' 



is exactly the same, I use nawk as it is just better. It gets worse when I see:

grep foo | grep -v bar | awk '{ print $1 }'

which should be:

nawk '/foo/ && ! /bar/ { print $1 }'

Mostly these just come about when typed on the command line but when I see them in scripts it just caused me to roll my eyes. They lead me to come up with these short rules for pipes:

If your pipe line contains

Use

grep and awk

nawk

grep and sed

nawk

awk and sed

nawk

awk sed and grep

nawk



Like the pirates code these are not so much rules as guide lines.

Comments:

Nice tip, thanks.

dl

Posted by Dan Lacher on April 01, 2005 at 05:48 AM BST #

Time to write a book Chris on the correct use of Unix utilities.I think you should follow Kimberley's example. The only question left is what to use as a title?

Posted by Paul Humphreys on April 01, 2005 at 07:50 AM BST #

You've caught one of my pet hates with this blog, pipelining greps into awk. What worries me is that the same people may well have progressed into Perl, etc, with similar lack of efficiency. It took me a long time to outgrow ksh and nawk! One of the most informative chapters in any O'Reilly book is that covering regular expressions in "sed and awk".

Posted by Don Thompson on April 01, 2005 at 08:31 AM BST #

[Trackback] Peter gives an undocumented way to find out the firmware version of your Sun Ray. I have three problems with this. It uses an undocumented interface which therefore is likely to change It does not work for me. It involved Cut...

Posted by The dot in ... --- ... on May 22, 2007 at 05:23 AM BST #

If your pipe line contains any combination of (n)awk, cat, grep, sed, tr then use Perl.
Now can you please write a rant about the useless use of cat. From a live example:

cat somefile | grep -v '\^#' | grep foo | awk '{print $2}'

Posted by Arjen Bax on December 05, 2007 at 04:21 AM GMT #

If the answer is perl then the question is wrong;-)

Don't start me on cat piped into things. Actually what is worse is cat < file | grep .... so close but so far.

Posted by Chris Gerhard on December 05, 2007 at 08:05 AM GMT #

A nice lesson to learn!

Posted by Prasanna S on March 16, 2008 at 04:42 AM GMT #

thanks

Posted by guest on September 17, 2008 at 12:36 AM BST #

Post a Comment:
Comments are closed for this entry.
About

This is the old blog of Chris Gerhard. It has mostly moved to http://chrisgerhard.wordpress.com

Search

Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today