Considering this Q&A about the order of execution in the shell as it relates to redirection, and despite the fact that if a file doesn’t exist it gets created first so that cat example.txt | shuf > example.txt doesn’t complain that the file doesn’t exist – which just confirms the order from what I understand – then why is it that about once every thousand times on my system the shuffle works when I do this (backup contains 15 static values each on their line)
for i in $(seq 1 1000); do
cp backup test
echo $i
cat test | shuf > test
cat test
done
How can there seemingly be an exception to the rule?
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
Here:
cat test | shuf > test
> takes precedence in the sense that this means:
(cat test) | (shuf > test)
and not:
(cat test | shuf) > test
Although if we used two different files, there wouldn’t be any difference between these two groupings anyway.
What becomes significant when you use the same file is that the two commands on either side of the pipe execute concurrently. Those commands are cat test and shuf > test. > means “open for writing and truncate”, whereas cat is going to “open for reading”, read, and close. Since those two things are happening concurrently, the relationship between the order of their collective operations is indeterminate. There’s a chance that cat will get to slurp the file in before the shell managing shuf > test truncates it. But that’s a slim chance, since there’s more involved; it will only happen if cat is very lucky in relation to the scheduling of shuf > test.
Moral of story: Don’t do this — use two files instead.
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0