xargs.md

Xargs and Command Argument Processing

The xargs command builds command lines from standard input. It is commonly used when one command produces a list of items and another command expects those items as command-line arguments rather than as text on standard input.

This distinction is important in Unix pipelines:

Producer                    xargs                         Command
+-------------+             +------------------+          +------------------+
| writes text | --stdout--> | reads stdin      | --argv-> | receives args    |
| to stdout   |             | builds arguments |          | and executes     |
+-------------+             +------------------+          +------------------+

For example, rm expects filenames as arguments:

rm file1.txt file2.txt file3.txt

If a previous command prints those filenames, xargs can turn the printed values into arguments for rm:

printf '%s\n' file1.txt file2.txt file3.txt | xargs rm

Conceptually, xargs transforms the input into a command resembling:

rm file1.txt file2.txt file3.txt

Syntax

The general form is:

command-producing-items | xargs [XARGS_OPTIONS] command [COMMAND_OPTIONS]

For example:

printf '%s\n' one two three | xargs echo

produces:

one two three

Here, printf writes three items to standard output. xargs reads them and executes a command equivalent to:

echo one two three

Standard Input Versus Command Arguments

A pipe (|) connects the standard output of one process to the standard input of another process. It does not automatically turn each line into a command-line argument.

For example:

printf '%s\n' alpha beta | cat

works because cat can read data from standard input.

By contrast, commands such as rm, mkdir, and many uses of cp need pathnames supplied as arguments:

rm alpha beta

This is where xargs is useful:

printf '%s\n' alpha beta | xargs rm

A useful mental model is:

pipe:   stdout -> stdin
xargs:  stdin  -> argv

where argv is the argument list passed to a program.

Common Options

Option	Description	Example
-n N	Pass at most N input items to each command invocation.	xargs -n 2 echo
-I {}	Replace {} in the command with one input item at a time.	xargs -I {} mv {} backup/
-P N	Run up to N command invocations in parallel.	xargs -P 4 -n 1 gzip
-0	Read NUL-separated input instead of whitespace-separated input.	`find . -print0
-t	Print each generated command before executing it.	xargs -t echo
-r	On GNU xargs, do not run the command when input is empty.	xargs -r rm --

The -r option is provided by GNU xargs and is not specified by POSIX, so scripts intended for multiple Unix systems should not assume it is available.

Controlling How Many Arguments Are Passed

By default, xargs tries to place as many input items as practical into each command invocation, up to system command-line size limits.

Suppose the input is:

printf '%s\n' one two three four

Using -n 2 runs echo with two arguments at a time:

printf '%s\n' one two three four | xargs -n 2 echo

Output:

one two
three four

This is useful when a command should process only a fixed number of arguments per invocation.

Replacement with -I

The -I option defines a replacement token. A common convention is {}:

printf '%s\n' server1 server2 server3 | xargs -I {} echo "checking {}"

Output:

checking server1
checking server2
checking server3

Each input item causes a separate command invocation.

Another example copies files into a backup directory:

printf '%s\n' config.ini app.conf | xargs -I {} cp -- {} backup/

The generated commands are conceptually:

cp -- config.ini backup/
cp -- app.conf backup/

For simple commands where arguments can appear at the end, -I is often unnecessary. For example:

printf '%s\n' file1 file2 | xargs rm --

is simpler than:

printf '%s\n' file1 file2 | xargs -I {} rm -- {}

Parallel Execution with -P

The -P option allows multiple command invocations to run concurrently:

printf '%s\n' archive1 archive2 archive3 archive4 | xargs -n 1 -P 4 gzip

In this example:

• -n 1 gives one archive name to each gzip invocation.
• -P 4 permits up to four gzip processes to run at the same time.

Parallel execution can reduce processing time for independent CPU-bound or I/O-bound tasks, but it should not be used when commands depend on running in a specific order or modify the same resource.

Filenames, Whitespace, and the -0 Option

The most important safety issue with xargs is filename parsing.

By default, xargs treats blanks, tabs, newlines, quotes, and backslashes as input syntax. Therefore, this pattern is unsafe for arbitrary filenames:

find . -type f | xargs rm

A pathname such as:

./old reports/report 1.txt

contains spaces and can be interpreted incorrectly.

A robust pattern is to use NUL-delimited records:

find . -type f -print0 | xargs -0 rm --

Here:

• find ... -print0 terminates each pathname with a NUL byte.
• xargs -0 reads NUL-separated items.
• Spaces, tabs, quotes, and newlines inside filenames remain part of the filename.

This is the preferred find + xargs pattern when arbitrary pathnames are possible.

The Meaning of --

Many Unix commands use -- to mark the end of command options. Arguments appearing after it are treated as operands even if they begin with -.

Consider a file literally named:

-rf

This is dangerous:

rm -rf

because rm interprets -rf as options.

To explicitly treat it as a filename:

rm -- -rf

The same rule applies when xargs invokes rm:

printf '%s\n' '-rf' | xargs rm --

In this command, the -- belongs to rm, not to xargs. xargs constructs a command similar to:

rm -- -rf

Many tools support this convention, including rm, cp, mv, grep, and others.

Using find with xargs

A common pattern is to let find select files and xargs invoke another command.

Compress all .log files:

find logs -type f -name '*.log' -print0 | xargs -0 -n 1 gzip

Remove matching temporary files:

find . -type f -name '*.tmp' -print0 | xargs -0 rm --

Calculate checksums in parallel:

find images -type f -print0 | xargs -0 -n 1 -P 4 sha256sum

find -exec Versus xargs

The find command can execute commands directly, so xargs is not always necessary.

Run once per file:

find . -type f -name '*.tmp' -exec rm -- {} \;

Batch many files into each invocation:

find . -type f -name '*.tmp' -exec rm -- {} +

The {} + form is often a clean alternative to:

find . -type f -name '*.tmp' -print0 | xargs -0 rm --

Both approaches safely handle unusual filenames.

A practical comparison is:

Approach	Characteristics
find ... -exec command {} \;	Executes once per pathname; simple but may create many processes.
find ... -exec command {} +	Batches pathnames safely without requiring xargs.
`find ... -print0	xargs -0 command`

Shell Globbing Versus xargs

Sometimes xargs is unnecessary because the shell can already generate the argument list.

To remove entries beginning with soi in the current directory:

rm -rf -- soi*

The shell expands soi* before rm runs, producing a command conceptually similar to:

rm -rf -- soi-audio soi-beards soi-maps soi-minimap

A pipeline such as:

ls | grep soi | xargs rm -rf --

is less robust. Parsing ls output is generally unnecessary, and unusual filenames can make text-based pipelines ambiguous. When a shell glob expresses the selection directly, prefer the glob.

Previewing Commands Before Running Them

Before using xargs with destructive commands, replace the destructive command with printf or echo, or use xargs -t.

For example:

find . -type f -name '*.tmp' -print0 | xargs -0 -t rm --

The -t option prints commands as they are executed.

For an entirely non-destructive preview:

find . -type f -name '*.tmp' -print0 | xargs -0 -n 1 printf 'would remove: %s\n'

This makes it easier to confirm the selected pathnames before deleting anything.

Empty Input

An important detail is what happens when the producer returns no items.

On GNU systems, xargs normally runs the command once even when there is no input. GNU xargs -r prevents this:

find . -type f -name '*.tmp' -print0 | xargs -0 -r rm --

For scripts that need portability across systems where -r may not exist, find -exec ... {} + is often a convenient alternative.

Practical Patterns

Change Permissions on Matching Files

find public -type f -name '*.html' -print0 | xargs -0 chmod 644

Search a Generated File List

find logs -type f -name '*.log' -print0 | xargs -0 grep -n -- 'ERROR'

Run Four Jobs in Parallel

printf '%s\n' host1 host2 host3 host4 | xargs -n 1 -P 4 ping -c 1

Process One Item at a Time with a Placeholder

printf '%s\n' alpha beta gamma | xargs -I {} sh -c 'printf "item: %s\n" "$1"' _ {}

When a shell command is needed inside xargs, pass the item as a positional parameter rather than interpolating untrusted text directly into shell code.

Common Mistakes

I. Parsing ls output

Avoid:

ls | grep '.log' | xargs rm

Prefer a shell glob when possible:

rm -- *.log

or use find with NUL delimiters:

find . -maxdepth 1 -type f -name '*.log' -print0 | xargs -0 rm --

II. Forgetting unusual filenames

Avoid newline-delimited filename processing when filenames may contain spaces or newlines. Use -print0 together with -0.

III. Confusing pipe input with arguments

This:

printf '%s\n' file1 file2 | rm

does not make file1 and file2 arguments to rm.

This does:

printf '%s\n' file1 file2 | xargs rm --

IV. Assuming -- belongs to xargs

In:

xargs rm -rf --

xargs is being asked to run the command rm -rf -- .... The -- terminates option processing for rm.

V. Using parallel execution for dependent operations

Do not use -P when one invocation must finish before the next one starts.

Challenges

1. Use printf and xargs to create three directories named alpha, beta, and gamma.
2. Use xargs -n 2 to print six words two at a time. Observe how many times the command runs.
3. Use xargs -I {} to transform a list of usernames into messages such as checking alice.
4. Create files whose names contain spaces, then compare plain find | xargs with the safe find -print0 | xargs -0 form.
5. Find all .tmp files below a test directory and preview their deletion without deleting them.
6. Repeat the previous task using find -exec ... {} + instead of xargs.
7. Use xargs -P 4 to calculate checksums for several files concurrently.
8. Create a file whose name begins with - and remove it safely using rm -- filename.
9. Explain the difference between command1 | command2 and command1 | xargs command2 in terms of standard input and command arguments.
10. For a set of files sharing a simple prefix, compare a shell glob with an ls | grep | xargs pipeline and identify which approach is simpler and safer.