Don't understand german? Read or subscribe to my english-only feed.

argument list too long

A problem everyone stumbles across (at least) once:

% tar zcf stats_2005.tar.gz stats_2005*
zsh: argument list too long: tar

The reason? Too many files for the ARG_MAX limit. ARG_MAX defines the maximum length of arguments to the exec function (more precise: bytes of args plus environment for exec), defined in /usr/include/linux/limits.h on your Linux system:

% ls -la | grep -c stats_2005
% getconf ARG_MAX
% cpp << EOF | tail -1
#include <limits.h>

Ok. But how to work around the issue? Possible solution for GNU:

% find . -name 'stats_2005*' > filelist
% tar zcf stats_2005.tar.gz --files-from filelist

Without the temporary file (filelist in our example):

% find . -name 'stats_2005*' -print | tar zcf stats_2005.tar.gz --files-from -

zsh provides zargs (but AFAIK it doesn’t work with gzip in the same command line due to the way zargs works, so create the archive in one step and compress it in another step later):

% autoload zargs
% zargs -- stats_2005* -- tar --ignore-failed-read -rpf stats_2005.tar

The approach works with other tools than tar as well of course, usage example for afio:

find . -name 'stats_2005*' -print | afio -o -Z stats_2005.afio

Depending on the type of problem (rm, cp, mv,…) you can choose differented approaches as well, some further examples:

% for file in stats_2005* ; do rm -f -- "$file" ; done
% perl -e 'unlink <stats_2005*>'
% zsh -c 'zmodload zsh/files && rm -f -- stats_2005*'
% zsh -c 'autoload -U zargs; zargs -r ./stats_2005* -- rm -f'
% ksh93 -c 'command -x rm -f -- stats_2005*'
% find . -maxdepth 1 -name 'stats_2005*' -print0 | xargs -r0 rm -f
% find . -name 'stats_2005*' -print0 | xargs -0 mv -i --target-dir=/destination
% zmodload zsh/files ; mv -- stats_2005* /destination

2 Responses to “argument list too long”

  1. tinus Says:

    Or you can use xargs, which automatically breaks the command into pieces shorter than 131072 characters. If you want it secure, be sure to use — and the -0 options to find and xargs.

  2. strcat Says:

    “-0” isn’t POSIX; neither for find or xargs.