Difference between revisions of "Miscellaneous commands"

From Christoph's Personal Wiki
Jump to: navigation, search
(→‎Selecting random lines from a file)
(→‎cat tricks)
Line 177: Line 177:
 
The second command, [[tr]], just says delete ("<code>-d</code>") any form feeds (you would use this if you want to print the file).
 
The second command, [[tr]], just says delete ("<code>-d</code>") any form feeds (you would use this if you want to print the file).
  
−
* cat the contents of file <code>foo</code>, then the STDOUT ("<code>bar</bar>"), then the contents of file <code>baz</code>:
+
* cat the contents of file <code>foo</code>, then the STDOUT ("<code>bar</code>"), then the contents of file <code>baz</code>:
 
  echo "bar" | cat foo - baz
 
  echo "bar" | cat foo - baz
  

Revision as of 21:17, 23 November 2013

This article will present various miscellaneous commands tested in Linux (running SuSE 10.2+ and Mandriva). Most of these will be simple command line tools. Eventually, many of these will have their own article. For now, they are presented as is with absolutely no guarantee and zero responsibility on my part if they cause loss of information or files. Use at your own risk.

Google tricks

  • Finding files:
-inurl:(htm|html|php) intitle:"index of" +"last modified" 
+"parent directory" +description +size +(jpg|png) "Lion"
  • Finding documents:
-inurl:(htm|html|php) intitle:"index of" +"last modified"
+"parent directory" +description +size +(pdf|doc) "shakespeare"
  • Phonebook:
rphonebook: John Doe Portland OR
bphonebook: Blue Sushi Seattle WA

Note: See Google Search Manual and Google command line help for more tricks.

ImageMagick tips & tricks

Bulk image resize

If you are like me and have a high resolution digital camera, it is often necessary to resize the images before emailing them to friends and family. It is, of course, possible to manually resize them using Adobe Photoshop, The Gimp, or any other image editing program. However, it is possible to automate this task using simple command line tools.

For an example, say you want to resize all of the jpeg images in your current directory to 800x600 and place them in a sub-directory called, "resized". Then you would execute the following commands: 

find . -maxdepth 1 -name '*.jpg' -type f -exec convert -resize 800x600 {} resized/{} \;

It is also possible to have the above commands run recursively through a directory and its sub-directories like so: 

find . -follow -name '*.jpg' -type f -exec convert -resize 800x600 {} ../resized/{} \;

Note that the program convert is part of the ImageMagick suite and you will need to have it installed to use the above commands (it is, by default, in SuSE Linux and most distributions).

Misc

  • Show photo filenames where no physical flash device was used:
identify -format "%f F:%[EXIF:Flash]\n" *.jpg | egrep " F:(0|16|24|32)$"

Tracking down large files

Note: See find for details. Sometimes it is necessary to find files over a certain size and it can be somewhat tedious ls-ing through your many directories. The following command will list only those files over a certain size and only within the specified directory (and sub-directories): 

find some_directory/ -size +2000k -ls

which will only list files over 2000 kb (2 MB).

Finding files containing a string in a directory hierarchy

In this example, all .php files will be searched for the string "MySQL" (case-insensitive with -i) and the line numbers will also be returned (using -n): 

find . -name '*.php' -type f | xargs grep -n -i 'MySQL'

ps2png

gs -sDEVICE=ppmraw -sOutputFile=- -sNOPAUSE -q foo.ps -c showpage -c quit |\
pnmcrop| pnmmargin -white 10 | pnmtopng >foo.png

nrg2iso

There is a command line utility, nrg2iso, for converting to ISO files in Linux. Linux users can easily convert the NRG file using the dd command in the following way: 

dd bs=1k if=myfile.nrg of=image.iso skip=300

Or, you can just mount it like so: 

mount -o loop, offset=307200 /path/to/image.nrg /path/to/mount

Selecting random lines from a file

This example could be used for printing random quotes from a file: 

FILE="/some/file_name"; nlines=$(wc -l < "$FILE"); \
IFS=$'\n'; array=($(<"$FILE")); echo "${array[$((RANDOM%nlines))]}"

Here, nlines holds the total number of lines in the file. The file is read into an array (note the use of IFS — this splits the lines based on '\n'). Then, once the array has been populated, print a random line from it.

Printing a block of text from a file

Say you have a file, foo, and it contains the following lines (note the capital letters and the full stop in line six): 

one blah blah
Two blah blah
three blah blah 3, 4
Four blah blah
five blah blah 5, 6
six blah blah.
  • If you only want to print out lines 3, 4, and 5, execute the following command:
awk "/three/,/five/" < foo
  • If you only want to print out lines starting with a capital "F", execute the following command:
awk "/^F/" < foo
  • If you only want to print out lines ending in a full stop, execute the following command:
awk "/\.$/" < foo
  • Finally, if you only want to print out lines containing the numbers "5" and "6", execute the following command:
awk "/[5-6]/" < foo

Note: See awk for details.

  • The following commands will print (to an actual printer) a file, starting at page 4, with a header of "ERRORS", with a line length of 80 characters, and print the whole thing 3 times:
pr +4 -h"ERRORS" -l80 logfile | lpr -# 3

Linux I/O redirection

  • The following command saves stdout and stderr to the files "out.txt" and "err.txt", respectively:
./cmd 1>out.txt 2>err.txt
  • The following command appends stdout and stderr to the files "out.txt" and "err.txt", respectively:
./cmd 1>>out.txt 2>>err.txt
  • The following command functions similar to the above two commands, but also copies stdout and stderr to the files "stdout.txt" and "stderr.txt", respectively:
(((./cmd | tee stdout.txt) 3>&1 1>&2 2>&3\
| tee stderr.txt) 3>&1 1>&2 2>&3) 1>out.txt 2>err.txt

Note: The above should be entered as one command (ie, the line that ends with a backslash is only continued on the next line because of the formatting constraints of this page).

Also note that Linux uses the following redirection codes/handles (see: redirection):

  • 0 = stdin
  • 1 = stdout
  • 2 = stderr

Count number of n-word lengths

for i in `seq 1 32` {
     egrep '^.{'$i'}$' /usr/share/dict/words | wc -l
}
# OR (depending on your shell),
for i in `seq 1 32`; do egrep '^.{'$i'}$' /usr/share/dict/words | wc -l; done

Then paste the numbers together (or, just add them to the above for-loop): 

seq 1 32 | paste - tmp_n-word_lenghts.dat >n-word_lengths.dat

Random number generation

The cli variable RANDOM returns a random number between a range of 0 to 32767. The following will generate three random numbers from within this range: 

r=0;while [ $r -lt 3 ];do echo $RANDOM;let r=$r+1;done

You can create your own versions of /dev/random and /dev/urandom with the following commands: 

mknod /tmp/random c 1 8 &&
mknod /tmp/urandom c 1 9 &&

Or, you can use the RANDOM variable, like so: 

r=0;while [ $r -lt 92160 ];do echo $RANDOM >>/tmp/urandom;let r=$r+1;done

You can also copy the original /dev/urandom like so (Note: Be careful with this command!): 

dd if=/dev/urandom of=/tmp/urandom count=1 >/dev/null 2>&1

You can also use openssl to generate a random key 

openssl rand -base64 512 | tr -d '\r\n' > /tmp/my_random_key

Other methods include: 

bash -c 'echo $RANDOM'
perl -e 'print int(rand(65535))'

Command-line calculator

% echo "111111111 * 111111111" | bc
  12345678987654321
% echo -e "sqrt(25)\nquit\n" | bc -q -i
  5

Also, 

% let x=3*4-6; echo $x

% echo $[ (10 + 5) / 2]
  7
% echo "(10+5)/2" | bc -l
  7.50000000000000000000
  • Convert 10MB to KB from within bash by running the following:
expr $((size_in_MB << 10))
# E.g.,
expr $((10MB << 10))
10240

Split large files into small pieces

% ls -lh mylargefile
  -rw-r--r--  1 foo users 800M Feb 18 11:17 mylargefile
% split -b 2m largefile mylargefile_
% ls -lh mylargefile_* | head 3
  -rw-r--r--  1 foo users 2.0M Feb 18 11:19 mylargefile_aa
  -rw-r--r--  1 foo users 2.0M Feb 18 11:19 mylargefile_ab
  -rw-r--r--  1 foo users 2.0M Feb 18 11:19 mylargefile_ac
  ...

Stego trick using built-in Linux utilities

cat foo.zip >> bar.gif  # "hides" 'foo.zip' inside 'bar.gif'
xv bar.gif     # views just fine
unzip bar.gif  # extracts 'foo.zip'

cat tricks

  • cat and squeeze-blank:

The cat utility has a nice little feature that allows you to "squeeze" multiple blank lines into a single line. In other words, never print more than one single blank line. 

cat -s foo | tr -d '\f'

The second command, tr, just says delete ("-d") any form feeds (you would use this if you want to print the file).

  • cat the contents of file foo, then the STDOUT ("bar"), then the contents of file baz:
echo "bar" | cat foo - baz

Create a favicon

  • Step 1: Download png2ico and install (just type make)
  • Step 2: Create a PNG using something like GIMP (make sure the image is square and small; e.g. 128x128, 300x300, etc).
  • Step 3: Convert the image to a 32x32 PNG (or 16x16, if you wish)
convert foo.png -resize 32x32 favicon.png
  • Step 3: Convert to icon
png2ico favicon.ico favicon.png
# ~ OR ~
png2ico favicon.ico logo16x16.png logo32x32.png

see: Creative Favicons: When Small Is Beautiful for ideas

Check if your OS is 32- or 64-bit

$ uname -a

If 32-bit, it will look something like: 

Linux hostname 3.8.0-33-generic #48-Ubuntu SMP Wed Oct 23 17:26:34 UTC 2013 i686 i686 i686 GNU/Linux

If 64-bit: 

Linux hostname 3.8.0-33-generic #48-Ubuntu SMP Wed Oct 23 17:26:34 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

$ file /sbin/init
/sbin/init: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV)...
# ~OR~
/sbin/init: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV)...

$ uname -m
i686    # <- 32-bit
# ~OR~
x86_64  # <- 64-bit

$ arch 
$ getconf LONG_BIT
32
# ~OR~
64

Misc

  • Save man pages as plain text:
man grep | col -b > grep.txt
  • Display the total number of files in the current working directory and all of its subdirectories:
find . -type f -print | wc  -l
  • Display a list of directories and how much space they consume, sorted from the largest to the smallest:
du | sort -nr
  • Format text for printing:
cat poorly_formatted_report.txt | fmt | pr | lpr
cat unsorted_list_with_dupes.txt | sort | uniq | pr | lpr
  • Delete files older than n days:
find /path/to/files* -mtime +5 -exec rm {} \;
  • Download multiple files in bash using for-loop:
for (( a=1; a<=96; a++ )) ;do wget -c http://www.example.com/images/$a.jpg; done
  • Multiple unzip: The following command will unzip all zip files in the current directory:
for i in $(ls *.zip); do unzip $i; done
  • Copying files through tar filter:
(cd /path/dir/from && tar -cvf - .) | (cd /path/dir/to && tar -xvf -)
  • Look up the definition of a word from the CLI:
curl dict://dict.org/d:word
  • Timed read-input using bash:
answer="yes";read -p "Do you wish to install now? " -t 10 answer;echo " Timed out; assuming $answer";
  • Check current battery charge system temperature:
acpi -t
  • Display information about all system users:
finger -l
  • Display distribution and version:
cat /etc/issue
# OR
cat /etc/release
  • What is the longest word you can think of where no letter appears more than once?
egrep -vi '(.).*\1' /usr/share/dict/words | awk '{ print length($0),$0 }' | sort -nr | head -n 8

See also

To Do

Lifehacker

Howtoforge

Sourceforge

Wikipedia

Other

External links

Retrieved from "http://wiki.christophchamp.com/index.php?title=Miscellaneous_commands&oldid=5945"