I know I can solve this problem several ways, but I’m wondering if there is a way to do it using only bash built-ins, and if not, what is the most efficient way to do it.
I have a file with contents like
AAA B C DDD FOO BAR
by which I only mean it has several lines and each line may or may not have spaces. I want to run a command like
cmd AAA "B C DDD" "FOO BAR"
If I use cmd $(< file) I get
cmd AAA B C DDD FOO BAR
and if I use cmd "$(< file)" I get
cmd "AAA B C DDD FOO BAR"
How do I get each line treated a exactly one parameter?
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
Portably:
set -f # turn off globbing IFS=' ' # split at newlines only cmd $(cat <file) unset IFS set +f
Or using a subshell to make the IFS and option changes local:
( set -f; IFS=' '; exec cmd $(cat <file) )
The shell performs field splitting and filename generation on the result of a variable or command substitution that is not in double quotes. So you need to turn off filename generation with set -f, and configure field splitting with IFS to make only newlines separate fields.
There’s not much to be gained with bash or ksh constructs. You can make IFS local to a function, but not set -f.
In bash or ksh93, you can store the fields in an array, if you need to pass them to multiple commands. You need to control expansion at the time you build the array. Then "${a[@]}" expands to the elements of the array, one per word.
set -f; IFS=$'n'
a=($(cat <file))
set +f; unset IFS
cmd "${a[@]}"
Method 2
You can do this with a temporary array.
Setup:
$ cat input AAA A B C DE F $ cat t.sh #! /bin/bash echo "$1" echo "$2" echo "$3"
Fill the array:
$ IFS=$'n'; set -f; foo=($(<input))
Use the array:
$ for a in "${foo[@]}" ; do echo "--" "$a" "--" ; done
-- AAA --
-- A B C --
-- DE F --
$ ./t.sh "${foo[@]}"
AAA
A B C
DE F
Can’t figure out a way of doing that without that temporary variable – unless the IFS change isn’t important for cmd, in which case:
$ IFS=$'n'; set -f; cmd $(<input)
should do it.
Method 3
Looks like the canonical way to do this in bash is something like
unset args
while IFS= read -r line; do
args+=("$line")
done < file
cmd "${args[@]}"
or, if your version of bash has mapfile:
mapfile -t args < filename
cmd "${args[@]}"
The only difference I can find between the mapfile and the while-read loop versus the one-liner
(set -f; IFS=$'n'; cmd $(<file))
is that the former will convert a blank line to an empty argument, while the one-liner will ignore a blank line. In this case the one-liner behavior is what I’d prefer anyway, so double bonus on it being compact.
I would use IFS=$'n' cmd $(<file) but it doesn’t work, because $(<file) is interpreted to form the command line before IFS=$'n' takes effect.
Though it doesn’t work in my case, I’ve now learned that a lot of tools support terminating lines with null (00) instead of newline (n) which does make a lot of this easier when dealing with, say, file names, which are common sources of these situations:
find / -name '*.config' -print0 | xargs -0 md5
feeds a list of fully-qualified file names as arguments to md5 without any globbing or interpolating or whatever. That leads to the non-built-in solution
tr "n" "00" <file | xargs -0 cmd
although this, too, ignores empty lines, though it does capture lines that have only whitespace.
Method 4
You could use the bash built-in mapfile to read the file into an array
mapfile -t foo < filename
cmd "${foo[@]}"
or, untested, xargs might do it
xargs cmd < filename
Method 5
old=$IFS IFS=' #newline ' array=`cat Submissions` #input the text in this variable for ... #use parts of variable in the for loop ... done IFS=$old
Best way I could find. Just Works.
Method 6
File
The most basic loop (portable) to split a file on newlines is:
#!/bin/sh
while read -r line; do # get one line (n) at a time.
set -- "<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="1f3b5f">[email protected]</a>" "$line" # store in the list of positional arguments.
done <infile # read from a file called infile.
printf '<%s>' "<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="b094f0">[email protected]</a>" ; echo # print the results.
Which will print:
$ ./script <AAA><A B C><DE F>
Yes, with default IFS=spacetabnewline.
Why it works
- IFS will be used by the shell to split the input into several variables. As there is only one variable, no splitting is perform by the shell. So, no change of
IFSneeded. - Yes, leading and trailing spaces/tabs are being removed, but it doesn’t seem to be a problem in this case.
- No, no globbing is done as no expansion is unquoted. So, no
set -fneeded. - The only array used (or needed) is the array-like positional parameters.
- The
-r(raw) option is to avoid the removal of most backslash.
That will not work if splitting and/or globbing is needed. In such cases a more complex structure is needed.
If you need (still portable) to:
- Avoid removal of leading and trailing spaces/tabs, use:
IFS= read -r line - Split line to vars on some character, use:
IFS=':' read -r a b c.
Split the file on some other character (not portable, works with ksh, bash, zsh):
IFS=':' read -d '+' -r a b c
Expansion
Of course, the title of your question is about spliting a command execution on newlines avoiding the split on spaces.
The only way to get splitting from the shell is to leave an expansion without quotes:
echo $(< file)
That is controlled by the value of IFS, and, on unquoted expansions, globbing is also applied. To mkae that work, you need:
- Set IFS to new line only, to get splitting on newline only.
-
Unset the globbing shell option
set +f:set +f
IFS=’
‘
cmd $(< file)
Of course, that change the value of IFS and of globbing for the rest of the script.
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0