I’m looking to write a script that takes a .txt filename as an argument, reads the file line by line, and passes each line to a command. For example, it runs command --option "LINE 1", then command --option "LINE 2", etc. The output of the command is written to another file. How do I go about doing that? I don’t know where to start.
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
Another option is xargs.
With GNU xargs:
xargs -a file -I{} -d'n' command --option {} other args
{} is the place holder for the line of text.
Other xargs generally don’t have -a, -d, but some have -0 for NUL-delimited input. With those, you can do:
< file tr 'n' '' | xargs -0 -I{} command --option {} other args
On Unix-conformant systems (-I is optional in POSIX and only required for UNIX-conformant systems), you’d need to preprocess the input to quote the lines in the format expected by xargs:
< file sed 's/"/"\""/g;s/.*/"&"/' |
xargs -E '' -I{} command --option {} other args
However note that some xargs implementations have a very low limit on the maximum size of the argument (255 on Solaris for instance, the minimum allowed by the Unix specification).
Method 2
Use while read loop:
: > another_file ## Truncate file.
while IFS= read -r LINE; do
command --option "$LINE" >> another_file
done < file
Another is to redirect output by block:
while IFS= read -r LINE; do
command --option "$LINE"
done < file > another_file
Last is to open the file:
exec 4> another_file
while IFS= read -r LINE; do
command --option "$LINE" >&4
echo xyz ## Another optional command that sends output to stdout.
done < file
If one of the commands reads input, it would be a good idea to use another fd for input so the commands won’t eat it (here assuming ksh, zsh or bash for -u 3, use <&3 instead portably):
while IFS= read -ru 3 LINE; do
...
done 3< file
Finally to accept arguments, you can do:
#!/bin/bash
FILE=$1
ANOTHER_FILE=$2
exec 4> "$ANOTHER_FILE"
while IFS= read -ru 3 LINE; do
command --option "$LINE" >&4
done 3< "$FILE"
Which one could run as:
bash script.sh file another_file
Extra idea. With bash, use readarray:
readarray -t LINES < "$FILE"
for LINE in "${LINES[@]}"; do
...
done
Note: IFS= can be omitted if you don’t mind having line values trimmed of leading and trailing spaces.
Method 3
Keeping precisely to the question:
#!/bin/bash # xargs -n param sets how many lines from the input to send to the command # Call command once per line [[ -f $1 ]] && cat $1 | xargs -n1 command --option # Call command with 2 lines as args, such as an openvpn password file # [[ -f $1 ]] && cat $1 | xargs -n2 command --option # Call command with all lines as args # [[ -f $1 ]] && cat $1 | xargs command --option
Method 4
The best answer I found is:
for i in `cat`; do "$cmd" "$i"; done < $file
EDIT:
… four years later …
after several down votes and some more experience I’d recommend the following now:
xargs -l COMMAND < file
Method 5
sed "s/'/'\\''/g;s/.*/$* '&'/" <<FILE |
sh -s -- command echo --option
all of the{&}se li$nes 'are safely shell
quoted and handed to command as its last argument
following --option, and, here, before that echo
FILE
OUTPUT
--option all of the{&}se li$nes 'are safely shell
--option quoted and handed to command as its last argument
--option following --option, and, here, before that echo
Method 6
ed file.txt %g/^/s// / 2,$g/^/-,.j 1s/^/command/ wq chmod 755 file.txt ./file.txt
Take all the lines of a file and pass them as arguments to a single command i.e.,
command line1 line2 line3 ....
If you need the --option flag to precede each line change the second command to:
%g/^/s// --option /
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0