Introduction to Awk Awk is a convenient and expressive programming language that can be applied to a wide variety of computing and data manipulation tasks Awk Works well on record type data Reads input file s a line at a time Parses each line into fields Performs user defined tests against each line performs actions on matches Other Common Uses Input validation Every record have same of fields Do values make sense negative time hourly wage 1000 etc Filtering out certain fields Searches Who got a zero on lab 3 Who got the highest grade Many others Invocation Can write little one liners on the command line very handy print the 3rd field of every line awk print 3 input txt Execute an awk script file awk f script awk input txt Or use this sha bang as the first line and give your script execute permissions bin awk f Form of an AWK program AWK programs are entries of the form pattern action pattern some test looking for a pattern regular expressions or C like conditions if null actions are applies to every line action a statement or set of statements if not provided the default action is to print the entire line much like grep Form of an AWK program Input files are parsed a record line at a time Each line is checked against each pattern in order There are 2 special patterns BEGIN true before any records are read END true at end of input after all records have been read Awk Features Patterns can be regular expressions or C like conditions Each line of the input is matched against the patterns one after the next If a match occurs the corresponding action is performed Input lines are parsed and split into fields which are accessed by 1 NF where NF is a variable set to the number of fields The variable 0 contains the entire line and by default lines are split by white space blanks tabs Variables Not declared nor typed No character type Only strings and floats support for ints n refers to the nth field where n is some integer value prints each field on the line for i 1 i NF i print i Some Built in Variables FS the input field separator OFS the output field separator NF of fields changes w each record NR the of records read so far So the current record FNR the of records read so far reset for each named file 0 the entire input line Example Print pay for those employees who actually worked awk 3 0 print 1 2 3 emp data Kathy Mark Mary Susie 40 100 121 76 5 cat emp data Beth 4 00 0 Dan 3 75 0 Kathy 4 00 10 Mark 5 00 20 Mary 5 50 22 Susie 4 25 18 Example CSV file cat students csv smith john js12 jones fred fj84 bee sue sb23 fife ralph rf86 james jim jj22 cook nancy nc54 banana anna ab67 russ sam sr77 loeb lisa guitarHottie cat getEmails awk bin awk f getEmails awk students csv john s email is js12 school edu fred s email is fj84 school edu sue s email is sb23 school edu ralph s email is rf86 school edu jim s email is jj22 school edu nancy s email is nc54 school edu anna s email is ab67 school edu sam s email is sr77 school edu lisa s email is guitarHottie schoo BEGIN FS printf s s email is s school edu n 2 3 Example output separator cat out awk bin awk f BEGIN FS OFS print 1 2 3 out awk students csv smith john js12 jones fred fj84 bee sue sb23 fife ralph rf86 james jim jj22 cook nancy nc54 banana anna ab67 russ sam sr77 loeb lisa guitarHottie Flow Control Awk syntax is much like C Same loops if statements etc AWK Aho Weinberger Kernighan Kernighan and Ritchie wrote the C language Associative Arrays Awk also supports arrays that can be indexed by arbitrary strings They are implemented using hash tables Total Sue 100 It is possible to loop over all indices that have currently been assigned values for name in Total print name Total name Example using Associative Arrays cat scores Fred 90 Sue 100 Fred 85 Sam 70 Sue 98 Sam 50 Fred 70 cat total awk Total 1 2 END for i in Total print i Total i awk f total awk scores Sue 198 Sam 120 Fred 245 Useful one liners Line count awk END print NR grep awk pat head awk NR 10 Add line s to a file awk print NR 0 awk printf 5d s NR 0 Many more See the resources tab on the course webpage for links to more examples
View Full Document
Unlocking...