GAWK: Effective AWK Programming

44 GAWK: Effective AWK Programming3.5.1 Using Regular Expressions to Separate FieldsThe previous subsection discussed the use of single characters or simple strings as the valueof FS. More generally, the value of FS may be a string containing any regular expression. Inthis case, each match in the record for the regular expression separates fields. For example,the assignment:FS = ", \t"makes every area of an input line that consists of a comma followed by a space and a TABinto a field separator.For a less trivial example of a regular expression, try using single spaces to separatefields the way single commas are used. FS can be set to "[ ]" (left bracket, space, rightbracket). This regular expression matches a single space and nothing else (see Chapter 2[Regular Expressions], page 24).There is an important difference between the two cases of ‘FS = " "’ (a single space) and‘FS = "[ \t\n]+"’ (a regular expression matching one or more spaces, TABs, or newlines).For both values of FS, fields are separated by runs (multiple adjacent occurrences) of spaces,TABs, and/or newlines. However, when the value of FS is " ", awk first strips leading andtrailing whitespace from the record and then decides where the fields are. For example, thefollowing pipeline prints ‘b’:$ echo ’ a b c d ’ | awk ’{ print $2 }’⊣ bHowever, this pipeline prints ‘a’ (note the extra spaces around each letter):$ echo ’ a b c d ’ | awk ’BEGIN { FS = "[ \t\n]+" }> { print $2 }’⊣ aIn this case, the first field is null or empty.The stripping of leading and trailing whitespace also comes into play whenever $0 isrecomputed. For instance, study this pipeline:$ echo ’ a b c d’ | awk ’{ print; $2 = $2; print }’⊣ a b c d⊣ a b c dThe first print statement prints the record as it was read, with leading whitespace intact.The assignment to $2 rebuilds $0 by concatenating $1 through $NF together, separated bythe value of OFS. Because the leading whitespace was ignored when finding $1, it is notpart of the new $0. Finally, the last print statement prints the new $0.There is an additional subtlety to be aware of when using regular exressions for fieldsplitting. It is not well-specified in the POSIX standard, or anywhere else, what ‘^’ meanswhen splitting fields. Does the ‘^’ match only at the beginning of the entire record? Oris each field separator a new string? It turns out that different awk versions answer thisquestion differently, and you should not rely on any specific behavior in your programs.As a point of information, the Bell Labs awk allows ‘^’ to match only at the beginningof the record. Versions of gawk after 3.1.6 also work this way. For example:$ echo ’xxAA xxBxx C’ |> nawk -F ’(^x+)|( +)’ ’{ for (i = 1; i