View source for Miscellaneous commands

This article will present various '''miscellaneous commands''' tested in [[Linux]] (running [[SuSE|SuSE 10.2]]+ and [[Mandriva Linux|Mandriva]]). Most of these will be simple [[:Category:Linux Command Line Tools|command line tools]]. Eventually, many of these will have their own article. For now, they are presented as is with absolutely ''no guarantee'' and ''zero responsibility'' on my part if they cause loss of information or files. Use at your own risk.

==Google tricks==
*Finding files:
 -inurl:(htm|html|php) intitle:"index of" +"last modified" 
 +"parent directory" +description +size +(jpg|png) "Lion"

*Finding documents:
 -inurl:(htm|html|php) intitle:"index of" +"last modified"
 +"parent directory" +description +size +(pdf|doc) "shakespeare"

*Phonebook:
 rphonebook: John Doe Portland OR
 bphonebook: Blue Sushi Seattle WA

*Find YouTube videos with your "word" in the video ID:
 allinurl:[your word here] site:youtube.com/watch

Note: See [http://www.googletutor.com/google-manual/ Google Search Manual] and [http://projects.felipc.com/gcl/ Google command line help] for more tricks.

==ImageMagick tips & tricks==
===Bulk image resize===
If you are like me and have a high resolution digital camera, it is often necessary to resize the images before emailing them to friends and family. It is, of course, possible to manually resize them using [[Adobe Photoshop]], [[Gimp|The Gimp]], or any other image editing program. However, it is possible to automate this task using simple command line tools.

For an example, say you want to resize all of the jpeg images in your current directory to 800x600 and place them in a sub-directory called, "<tt>resized</tt>". Then you would execute the following commands:
 find . -maxdepth 1 -name '*.jpg' -type f -exec convert -resize 800x600 {} resized/{} \;

It is also possible to have the above commands run recursively through a directory and its sub-directories like so:
 find . -follow -name '*.jpg' -type f -exec convert -resize 800x600 {} ../resized/{} \;

Note that the program <tt>convert</tt> is part of the [[ImageMagick]] suite and you will need to have it installed to use the above commands (it is, by default, in SuSE Linux and most distributions).

===Misc===
*Show photo filenames where no physical flash device was used:
 identify -format "%f F:%[EXIF:Flash]\n" *.jpg | egrep " F:(0|16|24|32)$"

==Tracking down large files==
''Note: See [[find]] for details.''
Sometimes it is necessary to find files over a certain size and it can be somewhat tedious <tt>ls</tt>-ing through your many directories. The following command will list only those files over a certain size and only within the specified directory (and sub-directories):

 find some_directory/ -size +2000k -ls
which will only list files over 2000 kb (2 MB).

==Finding files containing a string in a directory hierarchy==
In this example, all <tt>.php</tt> files will be [[find|searched]] for the string "<tt>MySQL</tt>" (case-insensitive with <tt>-i</tt>) and the line numbers will also be returned (using <tt>-n</tt>):
 find . -name '*.php' -type f | xargs grep -n -i 'MySQL'

==ps2png==
*Convert [[PostScript]] (*.ps) files to PNG:
 $ gs -sDEVICE=ppmraw -sOutputFile=- -sNOPAUSE -q foo.ps -c showpage -c quit |\
      pnmcrop | pnmmargin -white 10 | pnmtopng >foo.png

==nrg2iso==
There is a command line utility, [http://gregory.kokanosky.free.fr/v4/linux/nrg2iso.en.html nrg2iso], for converting to [[ISO Images|ISO]] files in Linux. Linux users can easily convert the NRG file using the [[dd]] command in the following way:
 dd bs=1k if=myfile.nrg of=image.iso skip=300
Or, you can just mount it like so:
 mount -o loop, offset=307200 /path/to/image.nrg /path/to/mount

==Selecting random lines from a file==
This example could be used for printing random quotes from a file:

<pre>FILE="/some/file_name"; nlines=$(wc -l < "$FILE"); \
IFS=$'\n'; array=($(<"$FILE")); echo "${array[$((RANDOM%nlines))]}"</pre>

Here, <tt>nlines</tt> holds the total number of lines in the file. The file is read into an array (note the use of <tt>IFS</tt> &mdash; this splits the lines based on '<tt>\n</tt>'). Then, once the array has been populated, print a random line from it.

==Printing a block of text from a file==
Say you have a file, <tt>foo</tt>, and it contains the following lines (note the capital letters and the full stop in line six):
<pre>
one blah blah
Two blah blah
three blah blah 3, 4
Four blah blah
five blah blah 5, 6
six blah blah.
</pre>

*If you only want to print out lines 3, 4, and 5, execute the following command:
 awk "/three/,/five/" < foo
*If you only want to print out lines starting with a capital "F", execute the following command:
 awk "/^F/" < foo
*If you only want to print out lines ''ending'' in a full stop, execute the following command:
 awk "/\.$/" < foo
*Finally, if you only want to print out lines containing the numbers "5" ''and'' "6", execute the following command:
 awk "/[5-6]/" < foo

''Note: See [[awk]] for details.''

*The following commands will print (to an actual printer) a file, starting at page 4, with a header of "ERRORS", with a line length of 80 characters, and print the whole thing 3 times:
 pr +4 -h"ERRORS" -l80 logfile | lpr -# 3

==Linux I/O redirection==
* The following command saves <tt>stdout</tt> and <tt>stderr</tt> to the files "<tt>out.txt</tt>" and "<tt>err.txt</tt>", respectively:
 ./cmd 1>out.txt 2>err.txt

* The following command ''appends'' <tt>stdout</tt> and <tt>stderr</tt> to the files "<tt>out.txt</tt>" and "<tt>err.txt</tt>", respectively:
 ./cmd 1>>out.txt 2>>err.txt

* The following command functions similar to the above two commands, but also ''copies'' <tt>stdout</tt> and <tt>stderr</tt> to the files "<tt>stdout.txt</tt>" and "<tt>stderr.txt</tt>", respectively:

 (((./cmd | tee stdout.txt) 3>&1 1>&2 2>&3\
 | tee stderr.txt) 3>&1 1>&2 2>&3) 1>out.txt 2>err.txt
''Note: The above should be entered as one command (ie, the line that ends with a backslash is only continued on the next line because of the formatting constraints of this page).''

Also note that Linux uses the following redirection codes/handles (see: [[Redirection (Linux)|redirection]]):
*0 = <tt>stdin</tt>
*1 = <tt>stdout</tt>
*2 = <tt>stderr</tt>

==Count number of n-word lengths==
 for i in `seq 1 32` {
      grep -E '^.{'$i'}$' /usr/share/dict/words | wc -l
 }
 # OR (depending on your shell),
 for i in `seq 1 32`; do grep -E '^.{'$i'}$' /usr/share/dict/words | wc -l; done

Then paste the numbers together (or, just add them to the above for-loop):
 seq 1 32 | paste - tmp_n-word_lenghts.dat > n-word_lengths.dat

==Random number generation==
The CLI variable <code>RANDOM</code> returns a random number between a range of 0 to 32767. The following will generate three random numbers from within this range:
 $ r=0;while [ $r -lt 3 ];do echo $RANDOM;let r=$r+1;done

You can create your own versions of <code>/dev/random</code> and <code>/dev/urandom</code> with the following commands:
 $ mknod /tmp/random c 1 8 &&
 $ mknod /tmp/urandom c 1 9 &&
Or, you can use the <code>RANDOM</code> variable, like so:
 $ r=0;while [ $r -lt 92160 ];do echo $RANDOM >>/tmp/urandom;let r=$r+1;done

You can also copy the original <code>/dev/urandom</code> like so (Note: Be careful with this command!):
 $ dd if=/dev/urandom of=/tmp/urandom count=1 >/dev/null 2>&1

You can also use <code>openssl</code> to generate a random key
 $ openssl rand -base64 512 | tr -d '\r\n' > /tmp/my_random_key

Other methods include:
 $ bash -c 'echo $RANDOM'
 $ perl -e 'print int(rand(65535))'

* Generate a random 8-character password
 $ dd if=/dev/urandom count=1 2> /dev/null | uuencode -m - | sed -ne 2p | cut -c-8
* Or, generate 10 random 8-character passwords:
 $ for ((n=0;n<10;n++)); do dd if=/dev/urandom count=1 2> /dev/null | uuencode -m -| sed -ne 2p | cut -c-8; done 
Or, you could just use the <code>`pwgen`</code> utility, if you are lazy.

==Command-line calculator==
 $ echo "111111111 * 111111111" | bc  # => 12345678987654321
 $ echo "sqrt(25)" | bc -qi  # => 5
 $ let x=3*4-6; echo $x  # => 6
 $ echo $[ (10 + 5) / 2]  # => 7
 $ echo "(10+5)/2" | bc -l  # => 7.50000000000000000000

* Convert 10MB to KB from within bash by running the following:
 $ expr $((size_in_MB << 10))
 $ # E.g.,
 $ expr $((10MB << 10))  # => 10240

==Split large files into small pieces==
 $ ls -lh mylargefile
   -rw-r--r--  1 foo users 800M Feb 18 11:17 mylargefile
 $ split -b 2m largefile mylargefile_
 $ ls -lh mylargefile_* | head 3
   -rw-r--r--  1 foo users 2.0M Feb 18 11:19 mylargefile_aa
   -rw-r--r--  1 foo users 2.0M Feb 18 11:19 mylargefile_ab
   -rw-r--r--  1 foo users 2.0M Feb 18 11:19 mylargefile_ac
   ...

==Stego trick using built-in Linux utilities==
 # see: https://gist.github.com/christophchamp/8e086f73c148510c0fc6
 $ cat foo.zip >> bar.gif  # "hides" 'foo.zip' inside 'bar.gif'
 $ qiv bar.gif     # views just fine (note: you can use any image viewer you like here)
 $ unzip bar.gif   # extracts 'foo.zip'

==cat tricks==

* cat and squeeze-blank:
The [[cat]] utility has a nice little feature that allows you to "squeeze" multiple blank lines into a single line. In other words, never print more than one single blank line.
 cat -s foo | tr -d '\f'
The second command, [[tr]], just says delete ("<code>-d</code>") any form feeds (you would use this if you want to print the file).

* cat the contents of file <code>foo</code>, then the STDOUT ("<code>bar</code>"), then the contents of file <code>baz</code>:
 echo "bar" | cat foo - baz

==Create a favicon==
*Step 1: Download [http://www.winterdrache.de/freeware/png2ico/ png2ico] and install (just type <code>make</code>)
*Step 2: Create a PNG using something like [[GIMP]] (make sure the image is square and small; e.g. 128x128, 300x300, etc).
*Step 3: Convert the image to a 32x32 PNG (or 16x16, if you wish)
 convert foo.png -resize 32x32 favicon.png
*Step 3: Convert to icon
 png2ico favicon.ico favicon.png
 # ~ OR ~
 png2ico favicon.ico logo16x16.png logo32x32.png

 see: [http://www.smashingmagazine.com/2007/06/14/creative-favicons-when-small-is-beautiful/ Creative Favicons: When Small Is Beautiful] for ideas

==Check if an OS is 32- or 64-bit==

 $ uname -a

:If 32-bit, it will look something like:
 Linux hostname 3.8.0-33-generic #48-Ubuntu SMP Wed Oct 23 17:26:34 UTC 2013 i686 i686 i686 GNU/Linux
:If 64-bit:
 Linux hostname 3.8.0-33-generic #48-Ubuntu SMP Wed Oct 23 17:26:34 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

* Note: The following does not work on OSes using [[systemd]]:
 $ file /sbin/init
 /sbin/init: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV)...
 # ~OR~
 /sbin/init: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV)...

 $ uname -m
 i686    # <- 32-bit
 # ~OR~
 x86_64  # <- 64-bit

 $ arch 
 $ getconf LONG_BIT
 32
 # ~OR~
 64

* To see if your processor is 64-bit, you can run this command:
 $ egrep -c ' lm ' /proc/cpuinfo

:If <code>0</code> is printed, it means that your CPU is not 64-bit.
:If <code>1</code> or higher, it is. Note: <code>lm</code> stands for "Long Mode" which equates to a 64-bit CPU. 

* Check if a remote host is running a 32- or a 64-bit [[Linux]] kernel:
 $ ssh $REMOTE_HOST getconf LONG_BIT

==Misc==
*Save man pages as plain text:
 $ man grep | col -b > grep.txt
*Display the total number of files in the current working directory and all of its subdirectories:
 $ find . -type f -print | wc  -l
*Display a list of directories and how much space they consume, sorted from the largest to the smallest:
 $ du | sort -nr
*Format text for printing:
 $ cat poorly_formatted_report.txt | fmt | pr | lpr
 $ cat unsorted_list_with_dupes.txt | sort | uniq | pr | lpr
*Delete files older than ''n'' days:
 $ find /path/to/files* -mtime +5 -exec rm {} \;
*Download multiple files in bash using for-loop:
 $ for (( a=1; a<=96; a++ )) ;do wget -c <nowiki>http://www.example.com/images/$a.jpg</nowiki>; done
*Multiple unzip: The following command will <tt>unzip</tt> all zip files in the current directory:
 $ for i in $(ls *.zip); do unzip $i; done
*Copying files through [[tar]] filter:
 $ (cd /path/dir/from && tar -cvf - .) | (cd /path/dir/to && tar -xvf -)
*Look up the definition of a word from the CLI:
 $ curl dict://dict.org/d:word
*Timed read-input using [[bash]]:
 $ answer="yes";read -p "Do you wish to install now? " -t 10 answer;echo " Timed out; assuming $answer";
*Check current battery charge system temperature:
 $ acpi -t
*Display information about all system users:
 $ finger -l
*Display distribution and version:
 $ cat /etc/issue
 # OR
 $ cat /etc/release
*What is the longest word you can think of where no letter appears more than once?
 $ grep -Evi '(.).*\1' /usr/share/dict/words | awk '{ print length($0),$0 }' | sort -nr | head -n 8
*urlencode a given URL:
 $ URL="foo bar";echo $(perl -MURI::Escape -e 'print uri_escape($ARGV[0]);' "$URL")
 $ #~OR~
 $ URL=$(php -r "echo urlencode(\"foo bar\");")
*Create your own sub-mlocate database to search only specific paths:
 $ updatedb -l 0 -U /usr/lib/python2.6/ -o /root/python.db
 $ locate -d /root/python.db httplib
 # OR:
 $ alias pylocate='locate -d /root/python.db'
 $ pylocate httplib
 # OR:
 $ updatedb -i -l 0 -U `python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"` \
       -o /home/bob/.mlocate/python-dist-packages.db
 $ alias pylocate='locate -i -d /root/python.db'  # case insensitive search
 $ pylocate mysql
 # OR:
 $ alias hlocate='locate -d /home/home.db'
 $ hlocate foobar
 # OR:
 $ locate -d /home/home.db -d /root/python.db foobar
*Tell mlocate to ignore certain paths by editing the <code>/etc/updatedb.conf</code> file and add the path you wish to ignore to:
 $ PRUNEPATHS="/tmp /var/spool /media /home/.ecryptfs"
*Get the number of days since the root password was last changed:
 $ echo "$(echo `date +%s`/86400|bc) - $(sudo awk -F: /^root/'{print $3}' /etc/shadow)" | bc
 #~OR~
 $ echo $(($((`date +%s`/86400))-$(sudo awk -F: /^root/'{print $3}' /etc/shadow)))

==See also==
*[[awk]]/gawk
*[[Findutils|find]]
*[[Findutils|xargs]]
*[[ImageMagick]]
*[[Convert (command)|convert]]
*[[Bash]]
*[[Tee (command)|tee]]

==To Do==
===Lifehacker===
*[http://lifehacker.com/software/gmail/geek-to-live--back-up-gmail-with-fetchmail-235207.php Geek to Live: Back up Gmail with fetchmail]
*[http://www.lifehacker.com/software/top/geek-to-live--keep-your-calendar-in-plain-text-with-remind-186661.php Geek to Live: Keep your calendar in plain text with Remind]
*[http://lifehacker.com/software/command-line/getting-things-done-with-rulebased-list-processing-217063.php Getting things done with rule-based list processing]
*[http://lifehacker.com/software/bookmarks/hack-attack-firefox-and-the-art-of-keyword-bookmarking-196779.php Hack Attack: Firefox and the art of keyword bookmarking]
===Howtoforge===
*[http://www.eps.mcgill.ca/jargon/jargon.html The New Hacker's Dictionary]
*[http://www.howtoforge.com/video_streaming_lighttpd_flowplayer Build Your Own Video Community With Lighttpd And FlowPlayer (Debian Etch)]
*[http://www.howtoforge.com/point_in_time_restoration_mysql_zrm How to perform a point in time restoration using ZRM for MySQL]
*[http://www.howtoforge.com/ipp_based_print_server_cups Step-by-Step IPP based Print Server using CUPS]
*[http://www.howtoforge.com/intrusion_detection_with_ossec_hids Securing Your Server With A Host-based Intrusion Detection System]
*[http://www.howtoforge.com/sysstat_monitoring_centos How To Monitor A System With Sysstat On Centos 4.3]
*[http://www.howtoforge.com/sysstat_monitoring_centos Accessing Windows Or Samba Shares Using AutoFS]
===Sourceforge===
*[http://ark.sourceforge.net/index.html The Arusha Project (ARK)]
*[http://clpbar.sourceforge.net/ Command Line Progress Bar (bar)]
*[http://sourceforge.net/projects/smbnetfs SMBNetFS] &mdash; a Linux/FreeBSD filesystem that allow you to use samba/microsoft network in the same manner as the network neighborhood in Microsoft Windows.
===Wikipedia===
*[[wikipedia:Category:GNU project software]]
*[[wikipedia:GNU Scientific Library]]
*[[wikipedia:DotGNU]]
*[[wikipedia:Patch_%28Unix%29]] (patch)
*[[wikipedia:Filesystem in Userspace]] (aka "FUSE")
===Other===
*[http://www.scottklarr.com/topic/115/linux-unix-cheat-sheets---the-ultimate-collection/ Linux-Unix cheat sheets - The ultimate collection]
*[http://cb.vu/unixtoolbox.xhtml Unix Toolbox]
*[http://www.shell-fu.org/ shell-fu] &mdash; collects as many little CLI commands as possible
*[http://phpgroupware.org/ phpGroupWare]
*[http://gentoo-wiki.com/HOWTO_Auto_mount_filesystems_(AUTOFS) HOWTO Auto mount filesystems (AUTOFS)]
*[http://linux.inet.hr/sshfs_secure_and_transparent_access_to_remote_filesystems.html sshfs - secure and transparent access to remote filesystems]
*[http://applications.linux.com/article.pl?sid=07/03/05/1949213&from=rss Become a digital video editing guru using Linux tools]
*[http://www.cyberciti.biz/ nixCraft]
*[http://www.linux.ie/ Linux.ie]
*[http://www.keller.com/html-quickref/latin1.html 8-Bit ASCII Codes and HTML Equivalents]
*[http://oss.oetiker.ch/rrdtool/ RRDtool] &mdash; data logging and graphing application.
*[http://www.igglo.co.uk/6/15-javascript-snippets-you-cant-live-without/ 15 Javascript Snippets You Can't Live Without]
*[http://www.kexi-project.org/ Kexi Project] &mdash; "Microsoft Access for Linux"
*[http://www.tuxmachines.org/node/14738 The Lazy Guide to Installing Knoppix on a USB Key]
*[http://www.gnome.org/projects/tracker/ Tracker]
*[http://www.norvig.com/spell-correct.html How to Write a Spelling Corrector]
*[http://www-128.ibm.com/developerworks/aix/library/au-speakingunix8/?ca=dgr-lnxw01speakunix8 Speaking UNIX, Part 8: UNIX processes]
*[http://www.scribd.com/collection/41 Free Linux books] (on-line only)
*[http://tutorialblog.org/25-code-snippets-for-web-designers-part5/ 25 Code Snippets for Web Designers (Part5)]
*[http://recaptcha.net/ reCAPTCHA]
*[http://www.openjsan.org/ JSAN.org] &mdsash; JavaScript Archive Network is a comprehensive resource for Open Source JavaScript libraries and software.
*[http://polishlinux.org/howtos/truecrypt-howto/ TrueCrypt Tutorial: Truly Portable Data Encryption]
*[http://dmiessler.com/study/lsof/ lsof]
*[http://jquery.com/ jQuery] &mdash; a fast, concise, [[JavaScript]] Library that simplifies how you traverse HTML documents, handle events, perform animations, and add Ajax interactions to your web pages.
*[http://www.linuxdevcenter.com/pub/a/linux/2007/08/02/an-introduction-to-linux-audio.html An Introduction to Linux Audio]
*[http://instantfundas.blogspot.com/2007/08/install-any-linux-distro-directly-from.html Install any Linux distro directly from hard disk without burning any DVD]
*[http://www.catonmat.net/blog/designing-digg-picture-website/ Designing Digg Picture Website in a Matter of Hours]
*[http://dailypackage.fedorabook.com/index.php?/archives/102-Productive-Monday-Incron-Execute-commands-based-on-filesystem-activity.html incron] &mdash; execute commands based on filesystem activity
*[http://www-128.ibm.com/developerworks/aix/library/au-badunixhabits.html Learn 10 good UNIX usage habits]
*[http://www.xs4all.nl/~carlo17/howto/ Carlo Wood's HOWTOs]
*[http://www.andrews-corner.org/mutt.html Using Mutt with Gmail]
*[http://code.google.com/p/memcachedb/wiki/Performance memcachedb] &mdash; a very fast to get/set an object
*[http://wordaligned.org/articles/shell-script-sets He Sells Shell Scripts to Intersect Sets]
*[http://polishlinux.org/apps/cli/defragmentation-of-linux-filesystems/ Defragmentation of Linux Filesystems]

==External links==
*[http://www.cs.usfca.edu/~parrt/course/601/lectures/unix.util.html How To Look Like A UNIX Guru]
*[http://www.sns.ias.edu/~jns/wp/2006/03/17/scripting-partition-creation-in-linux-using-fdisk/ Scripting Partition Creation In Linux Using Fdisk]
*[http://www.sns.ias.edu/~jns/wp/2006/03/21/backup-restore-scripts-for-imaging-cloning-disks-using-partimage-under-linux/Backup/Restore Scripts For Imaging/Cloning Disks Using Partimage Under Linux]
*[http://www.sns.ias.edu/~jns/wp/2006/01/31/configuring-vacation-mail-with-procmail/ Configuring Vacation Mail With Procmail]
*[http://en.literateprograms.org/LiteratePrograms:Welcome LiteratePrograms]
*[http://www.krugle.org/ krugle] &mdash; code search
*[http://www.koders.co.uk/ koders] &mdash; code search

[[Category:Linux Command Line Tools| ]]