GNU parallel

Last updated
Parallel
Developer GNU Project
Stable release
20241022 [1] / 21 October 2024;14 months ago (2024-10-21)
Repository
Written in Perl
Operating system GNU
Type Utility
License GPLv3
Website www.gnu.org/software/parallel/   OOjs UI icon edit-ltr-progressive.svg

GNU parallel is a command-line utility for Linux and other Unix-like operating systems which allows the user to execute shell scripts or commands in parallel. GNU parallel is free software, written by Ole Tange in Perl. It is available under the terms of GPLv3. [2]

Contents

Usage

Introduction video, Part 1
Introduction video, Part 2

The most common usage is to replace the shell loop, for example

whilereadx;dodo_something"$x"done<list 

to the form of

paralleldo_something<list 

where the file list contains arguments for do_something and where process_output may be empty.

Scripts using parallel are often easier to read than scripts using pexec.

The program parallel features also

By default, parallel runs as many jobs in parallel as there are CPU cores.

Examples

find.-name"*.foo"|parallelgrepbar 

The above is the parallel equivalent to:

find.-name"*.foo"-execgrepbar{}+ 

This searches in all files in the current directory and its subdirectories whose name end in .foo for occurrences of the string bar. The parallel command will work as expected unless a file name contains a newline. In order to avoid this limitation one may use:

find.-name"*.foo"-print0|parallel-0grepbar 

The above command uses the null character to delimit file names.

find.-name"*.foo"|parallel-Xmv{}/tmp/trash 

The above command expands {} with as many arguments as the command line length permits, distributing them evenly among parallel jobs if required. This can lower process overhead for short-lived commands that take less time to finish than they do to launch.

find.-maxdepth1-typef-name"*.ogg"|parallel-X-rcp-v-p{}/home/media 

The command above does the same as:

cp-v-p*.ogg/home/media 

However, the former command which uses find/parallel/cp is more resource efficient and will not halt with an error if the expansion of *.ogg is too large for the shell.

See also

References

  1. Tange, Ole (21 Oct 2024). "GNU Parallel 20241022 ('Sinwar Nasrallah') released [stable]". parallel (Mailing list).
  2. "GNU Parallel". GNU.org.