What command can I use to create zips with a file number limit? I have a folder (no subfolders) of, say, 5000 files, so I would want a command that could divide that number and create 10 individual zip archives, each consisting of no more than 500 files.
I also don’t want the resulting 10 zip files to be connected with each other, so that I can open them individually and won’t need to open all 10 at the same time.
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
You can use GNU parallel to do that as it can limit the number of elements to a job as well as provide a job number (for a unique zip archive name):
$ touch $(seq 20)
$ find . ! -name "*.zip" -type f -print0 | parallel -0 -N 5 zip arch{#} {}
adding: 1 (stored 0%)
adding: 10 (stored 0%)
adding: 11 (stored 0%)
adding: 12 (stored 0%)
adding: 13 (stored 0%)
adding: 14 (stored 0%)
adding: 15 (stored 0%)
adding: 16 (stored 0%)
adding: 17 (stored 0%)
adding: 18 (stored 0%)
adding: 19 (stored 0%)
adding: 2 (stored 0%)
adding: 20 (stored 0%)
adding: 3 (stored 0%)
adding: 4 (stored 0%)
adding: 5 (stored 0%)
adding: 6 (stored 0%)
adding: 7 (stored 0%)
adding: 8 (stored 0%)
adding: 9 (stored 0%)
$ ls
1 11 13 15 17 19 20 4 6 8 arch1.zip arch3.zip
10 12 14 16 18 2 3 5 7 9 arch2.zip arch4.zip
The option -N 5 limits the number of files to 5 per archive and is presented to zip in place of {}
The {#} (verbatim, not to be replaced by you during the invocation), is replaced by the job number, resulting in arch1.zip, arch2.zip etc.
The -print0 option to find and -0 option to parallel in tandem make sure that filenames with special characters are correctly handled.
Method 2
A shell-only alternative: process batches of COUNT files via "${@:START:COUNT}" (range of positional parameters) and shift COUNT while incrementing a counter c to name the archives:
set -- *
c=1
while (($#)); do
if [ $# -ge COUNT ]; then
zip ${c}.zip "${@:1:COUNT}"
c=$((c+1))
shift COUNT
else
zip ${c}.zip "${@}"
shift $#
fi
done
Method 3
The accepted answer worked perfectly fine for me. 🙂 BUT, in case you don’t have access to parallel (who knows why), here’s an alternative I had come up with before:
find . ! -name '*.zip' -type f | xargs -n 500 | awk '{system("zip myarch"NR".zip "$0)}'
Which will create myarch1.zip, myarch2.zip, myarch3.zip, etc
You might want to use the -0 trick Anthon suggested, if you have weird filenames.
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0