How to Write AWK Commands and Scripts

The awk command is a powerful method for processing or analyzing text files—in particular, data files that are organized by lines (rows) and columns.

Simple awk commands can be run from the command line. More complex tasks should be written as awk programs (so-called awk scripts) to a file.

The basic format of an awk command looks like this:

This means: take each line of the input file; if the line contains the pattern apply the action to the line and write the resulting line to the output-file. If the pattern is omitted, the action is applied to all line. For example:

This statement takes the element of the 5th column of each line and writes it as a line in the output file "output.txt". The variable '$4' refers to the second column. Similarly you can access the first, second, and third column, with $1, $2, $3, etc. By default columns are assumed to be separated by spaces or tabs (so called white space). So, if the input file "table1.txt" contains these lines:

Then the command would write the following lines to the output file "output1.txt":

If the column separator is something other than spaces or tabs, such as a comma, you can specify that in the awk statement as follows:

This will select the element from column 3 of each line if the columns are considered to be separated by a comma. Therefore the output, in this case, would be:

The list of statements inside the curly brackets ('{','}') is called a block. If you put a conditional expression in front of a block, the statement inside the block will be executed only if the condition is true.

In this case, the condition is $7=="\$7.30", which means that the element at column 7 is equal to $7.30. The backslash in front of the dollar sign is used to prevent the system from interpreting $7 as a variable and instead take the dollar sign literally.

So this awk statement prints out the element at the 3rd column of each line that has a "$7.30" at column 7.

You can also use regular expressions as the condition. For example:

The string between the two slashes ('/') is the regular expression. In this case, it is just the string "30." This means if a line contains the string "30", the system prints out the element at the 3rd column of that line. The output in the above example would be:

If the table elements are numbers awk can run calculations on them as in this example:

Besides the variables that access elements of the current row ($1, $2, etc.) there is the variable $0 which refers to the complete row (line), and the variable NF which holds to the number of fields.

You can also define new variables as in this example:

This computes and prints the sum of all the elements of each row.

Awk statements are frequently combined with sed commands.