Split an input for different command and combine the result

I know how to combine the result of different command

paste -t',' <(commanda) <(commandb)

I know pipe same input to different command

cat myfile | tee >(commanda) >(commandb)

Now how to combine these command? So that I can do

cat myfile | tee >(commanda) >(commandb) | paste -t',' resulta resultb

Say I have a file

myfile:

I want to make a new file

I used

cat myfile | tee >(tac) >(awk '{print $1*2}') | paste

would gives me result vertically, where I really want paste them in horizontal order.

Contents hide

Answers:

Method 1

Method 2

Method 3

Method 4

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

When you tee to multiple process substitutions, you’re not guaranteed to get the output in any particular order, so you’d better stick with

paste -t',' <(commanda < file) <(commandb < file)

Assuming cat myfile stands for some expensive pipeline, I think you’ll have to store the output, either in a file or a variable:

output=$( some expensive pipeline )
paste -t',' <(commanda <<< "$output") <(commandb <<< "$output")

Using your example:

output=$( seq 4 )
paste -d' ' <(cat <<<"$output") <(tac <<<"$output") <(awk '$1*=2' <<<"$output")

Another thought: FIFOs, and a single pipeline

mkfifo resulta resultb
seq 4 | tee  >(tac > resulta) >(awk '$1*=2' > resultb) | paste -d ' ' - resulta resultb
rm resulta resultb

Method 2

The yash shell has unique features (pipeline redirection and process redirection) that make that easier there:

cat myfile | (
  exec 3>>|4
  tee /dev/fd/5 5>(commanda >&3 3>&-) 3>&- |
    commandb 3>&- |
    paste -d , /dev/fd/4 - 3>&-
)

3>>|4 (pipeline redirection) creates a pipe where the writing end is on fd 3 and the reading end on fd 4.

3>(commanda>&3) is process redirection, a bit like ksh/zsh/bash process substitution but just does the redirection and doesn’t substitute with the /dev/fd/n. ksh‘s >(cmd) is more or less the same as yash‘s n>(cmd) /dev/fd/n (there n is a file descriptor chosen by ksh on which you have no control).

Method 3

With zsh:

pee() (
  n=0 close_in= close_out= inputs=() outputs=()
  merge_command=$1; shift
  for cmd do
    eval "coproc $cmd $close_in $close_out"

    exec {i}<&p {o}>&p
    inputs+=($i) outputs+=($o)
    eval i$n=$i o$n=$o
    close_in+=" {i$n}<&-" close_out+=" {o$n}>&-"
    ((n++))
  done
  coproc :
  read -p
  eval tee /dev/fd/$^outputs $close_in "> /dev/null &
    " exec $merge_command /dev/fd/$^inputs $close_out
)

Then use as:

$ echo abcd | pee 'paste -d,' 'tr a A' 'tr b B' 'tr c C'
Abcd,aBcd,abCd

That’s adapted from this other question where you’ll find some detailed explanations and hints at the limitations (beware of deadlocks!).

Method 4

For your particular example there should be no need for paste and the rest. It is often true that when we encounter a limit with the standard toolset it is because what we want to do one way can be done another. Such as:

set 1 2 3 4
while [ "$#" -gt 0 ]
do    echo "$1" "$#" "$(($1*2))"
shift;done

…which prints…

You can get a file with contents like you mention into your shell "[email protected]" array like…

set -f; IFS='
'; set -- $(cat)

And to validate the arg values in a loop like the one above you can change the initial test a bit…

while { [ "$1" -eq "${1-1}" ] ;} 2>&"$((2+!$#))"
do    echo "$1" "$#" "$(($1*2))"
shift;done  3>/dev/null >outfile

…which prints an error to stderr only if a line read in with set -- $(cat) contains a line which does not consist entirely of a single integer.

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0

0 0 votes

Article Rating