Context for the question: According to POSIX specs, ARG_MAX is maximum length of command-line arguments to exec() family of functions. Which lead me to believe that’s actual number of arguments, however that clearly didn’t work:
$ ulimit -s
8192
$ touch {1..18000}.jpg
$ rm *.jpg
$
Clearly, this works fine, despite being in length over 8192 items. According to D.W.’s answer, the 8192 is supposedly size in kB. So clearly the previous assumption was wrong.
This is where the actual question comes in: How do I figure out the amount of items that actually will get above 8192 kB limit ? In other words, what sort of computation I have to perform to ensure that *.jpg type of glob will result into Argument list too long error ?
Please note, this isn’t a duplicate of What defines the maximum size of single command argument. I know about getconf ARG_MAX and ulimit -s values, that’s not my question. I need to know how to generate enough arguments in size that will be above the limit. In other words, I need to find a way to get the error, not avoid it.
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
Using getconf ARG_MAX to generate a long list of x and calling an external utility with that as its argument would generate an “Argument list too long” error:
$ /bin/echo $( perl -e 'print "x" x $ARGV[0]' "$(getconf ARG_MAX)" ) /bin/sh: /bin/echo: Argument list too long
The environment and the length of the string /bin/echo will be included in what makes the error occur, so we can try to find the biggest possible number by subtracting these:
$ env PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/X11R6/bin:/usr/local/bin
(I started this shell with env -i sh, so there’s only the PATH variable in the environment)
$ /bin/echo $( perl -e 'print "x" x ($ARGV[0] - length($ENV{"PATH"}) - length("/bin/echo"))' "$(getconf ARG_MAX)" )
sh: /bin/echo: Argument list too long
Still too long. By how much?
i=0
while ! /bin/echo $( perl -e 'print "x" x ($ARGV[0] - length($ENV{"PATH"}) - length("/bin/echo") - $ARGV[1])' "$(getconf ARG_MAX)" "$i" )
do
i=$(( i + 1 ))
done
This loop exits for i=8.
So there’s four bytes that I can’t immediately account for (four of the eight must be for the name of the PATH environment variable). These are the null terminators for the four strings PATH, the value of PATH, /bin/echo and the long string of x characters.
Note that each argument is null terminated, so the more arguments you have to the command, the shorter the combined length of them can be.
Also, just to show the effect of a big environment:
$ export BIG=$( perl -e 'print "x" x $ARGV[0]' "$( getconf ARG_MAX )" ) $ /bin/echo hello sh: /bin/echo: Argument list too long $ /bin/echo sh: /bin/echo: Argument list too long
Method 2
On many Linux distributions, you can find out what’s the current value of ARG_MAX by running
getconf ARG_MAX
A simple way to generate the message “Argument list too long” is to actually supply a long argument list; for example, on my system
/bin/echo "$(find / -xdev 2> /dev/null)"
works, but
/bin/echo "$(find / -xdev 2> /dev/null)" "$(find / -xdev 2> /dev/null)"
produces
bash: /bin/echo: Argument list too long
For an in-depth discussion, see “Argument list too long error for rm, cp, mv commands” on Stack Overflow.
P.S. The length in question is the actual amount of memory needed to store the arguments and environment. It is not related to the number of arguments.
Method 3
To make it easier start a new shell (exit when done) and set the limit to a lower value (100kbytes):
$ bash $ ulimit -s 100
With the example you used, the error could be triggered:
$ touch {1..100000}.jpg
bash: /usr/bin/touch: Argument list too long
But better to use something like echo. Understand that a builtin might not trigger an error, this command should work correctly:
$ echo {000001..100000}6789
Also note that the amount of bytes is well over the limit set above (1.1Mbyte):
$ echo {000001..100000}6789 | wc -c
1100000
But this command won’t work:
$ /bin/echo {000001..100000}6789
bash: /bin/echo: Argument list too long
The limit is set by the size of the argument list in bytes added to the environment size of the shell.
Method 4
Instead of listing all the files on the system, which takes forever and doesn’t work on sparse systems, the command
/bin/echo {1..200000}
is a much faster (163ms) way to generate the error. For me ARG_MAX is 2097152 (2,097,152 on the order of 2 million) but the command still errors.
If your shell doesn’t have {start..end}, you can use the somewhat slower (but still faster than listing files)
/bin/echo $(seq 1 200000)
If either command doesn’t work, just increase the end value until it does, until bash runs out of memory, or until seq can’t count any higher.
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0