Awk/tips and tricks

From Christoph's Personal Wiki
Jump to: navigation, search

This article will list random "tips & tricks" in using awk/gawk.

  • List the directories and files in the current directory such that they can be distinguished with a trailing '/' (-F flag), identify the directories by grepping for the trailing '/' and pull out only those rows, remove the trailing slash with sed, use awk to build a command line that will tar up the directory, and finally execute the whole shebang by piping it to bash.
ls -F | grep '\/$' | sed 's/\/$//g' | awk '{print "tar -xf "$0" "$0".tar";}' | bash
  • The "proper" way to do this is with the while construct; however, the disadvantage of this is that you can't preview the command sequence quite as easily.
ls -F | grep '\/$' | sed 's/\/$//g' | while read filename; do tar -xf "${filename}" "${filename}.tar"; done
  • Find the files in the subdirectories below the current directory, grep for those that end with a trailing '.txt', take the top 30 lines of each of these files, and pipe the result to less so you can page through the results.
find ./ | grep '\.txt$' | xargs head -30 | less
  • Find all the rows where the 5th column does not equal "-1" in the file foo.txt, cut out the resulting columns 1-5 among the resulting rows, sort the result by the 1st column with a reverse (-r) numeric (-n) sort, retain only those lines which are not adjacent to identical lines, and print out the resulting number of lines with wc -l.
awk '{if ($5 != -1) { print;}}' demo.txt | cut -f1-5 | sort -k1 -r -n | uniq | wc -l