GNU parallel - Revision history

Christoph: /* Usage examples / tutorial */

2015-03-20T02:36:47Z

‎Usage examples / tutorial

Christoph: /* Usage examples / tutorial */

2015-03-20T02:35:24Z

‎Usage examples / tutorial

Christoph: /* Usage examples / tutorial */

2015-03-20T02:35:03Z

‎Usage examples / tutorial

Christoph: /* Usage examples / tutorial */

2015-03-20T02:28:01Z

‎Usage examples / tutorial

Christoph: New page: '''GNU parallel''' is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in t...

2015-03-20T02:25:09Z

New page: '''GNU parallel''' is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in t...

New page

'''GNU parallel''' is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU parallel can then split the input and pipe it into commands in parallel.

==Usage examples / tutorial==

* Install GNU parallel from the CLI (or, just use your distro's repo):
$ wget pi.dk/3 -qO - | bash -x

$ cat foo.fasta
>RECORD1
ATGGCTGTCTTCTTGCTTGCCACTTCCACCATAATGTTCCCAACGAAGATAGAAGCAGCA
GATTGTAATGGTGCATGTTCACCTTTCGAGGTGCCACCGTGCCGCTCAAGTGATTGTCGT
TGTGTCCCTATAGGACTATTTGTTGGTTTCTGCATACATCCAACTGGACTTTCATCTGTT
>RECORD2
GCGAAGATGGTCGACGAACATCCCAACTTATGTCAATCTGATGATGAATGCATGAAGAAA
GGAAGTGGCAATTTTTGCGCTCGTTACCCTAATAATTATATCGATTATGGATGGTGTTTT
GACTCTGATTCTGAAGCACTGAAAGGCTTCTTGGCCATGCCTAGGGCAACCACCAAGTAA

$ cat foo.fasta | parallel --pipe --recstart '>' -N1 cat';' echo =====
>RECORD1
ATGGCTGTCTTCTTGCTTGCCACTTCCACCATAATGTTCCCAACGAAGATAGAAGCAGCA
GATTGTAATGGTGCATGTTCACCTTTCGAGGTGCCACCGTGCCGCTCAAGTGATTGTCGT
TGTGTCCCTATAGGACTATTTGTTGGTTTCTGCATACATCCAACTGGACTTTCATCTGTT
=====
>RECORD2
GCGAAGATGGTCGACGAACATCCCAACTTATGTCAATCTGATGATGAATGCATGAAGAAA
GGAAGTGGCAATTTTTGCGCTCGTTACCCTAATAATTATATCGATTATGGATGGTGTTTT
GACTCTGATTCTGAAGCACTGAAAGGCTTCTTGGCCATGCCTAGGGCAACCACCAAGTAA
=====

$ printf '=%.0s' {1..79}
$ printf %79s | tr " " "="
$ seq -s= 79 | tr -d '[:digit:]'
$ perl -E 'say "=" x 79'

$ cat /usr/share/dict/words | parallel --pipe --blocksize 500k wc

$ seq 1 10 |shuf | parallel --pipe --files -N 3 sort -n | parallel -mj1 sort -nm
$ rm -f /tmp/*.par
$ seq 1 10 |shuf | parallel --pipe --files -N 3 sort -n | parallel -mj1 sort -nm {} ";"rm {}

==External links==
*[http://www.gnu.org/software/parallel/ Official website]
*[http://www.gnu.org/software/parallel/parallel_tutorial.html GNU parallel tutorial] (same as running: <code>`man parallel_tutorial`</code>)
*[http://pi.dk/1/ GNU Parallel videos] on YouTube

[[Category:Linux Command Line Tools]]

← Older revision		Revision as of 02:36, 20 March 2015
Line 30:		Line 30:

	however, the former command which uses <code>find</code>/<code>parallel</code>/<code>cp</code> is more resource efficient and will not halt with an error if the expansion of *.ogg is too large for the shell.		however, the former command which uses <code>find</code>/<code>parallel</code>/<code>cp</code> is more resource efficient and will not halt with an error if the expansion of *.ogg is too large for the shell.
		+
		+	* Multiple commands as arguments:
		+	$ cat a.txt \| xargs -I % sh -c 'command1; command2; ...'
		+	# ~OR~
		+	$ cat a.txt \| parallel 'command1 {}; command2 {}; ...; '

	* Example of using "pipes" and "records" to separate STDIN/STDOUT:		* Example of using "pipes" and "records" to separate STDIN/STDOUT:

@@ Line 17: / Line 17: @@
   $ find . -name "*.foo" -print0 | parallel -0 grep bar
-The above command uses the [[null character]] to delimit file names.
+The above command uses the null character to delimit file names.
   $ find . -name "*.foo" | parallel -X mv {} /tmp/trash

@@ Line 6: / Line 6: @@
   $ wget pi.dk/3 -qO - | bash -x
   $ cat foo.fasta
   >RECORD1
@@ Line 39: / Line 65: @@
   $ rm -f /tmp/*.par
   $ seq 1 10 |shuf | parallel --pipe --files -N 3 sort -n | parallel -mj1 sort -nm {} ";"rm {}
 ==External links==

@@ Line 32: / Line 32: @@
   $ seq -s= 79 | tr -d '[:digit:]'
   $ perl -E 'say "=" x 79'
   $ cat /usr/share/dict/words | parallel --pipe --blocksize 500k wc