Why does sed -i executed on symlink destroys that link and replaces it with destination file? How to avoid this?
eg.
$ ls -l pet* -rw-rw-r-- 1 madneon madneon 4 mar 23 16:46 pet lrwxrwxrwx 1 madneon madneon 6 mar 23 16:48 pet_link -> pet $ sed -i 's/cat/dog/' pet_link $ ls -l pet* -rw-rw-r-- 1 madneon madneon 4 mar 23 16:48 pet -rw-rw-r-- 1 madneon madneon 4 mar 23 16:49 pet_link
And why isn’t it considered a bug?
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
The -i/--in-place flag edits a file in place. By default, sed reads the given file, processes it outputting into a temporary file, then copies the temporary file over the original, without checking whether the original was a symlink.
GNU sed has a --follow-symlinks flag, which makes it behave as you want:
$ echo "cat" > pet $ ln --symbolic pet pet_link $ sed --in-place --follow-symlinks 's/cat/dog/' pet_link $ cat pet dog
Method 2
It’s not a bug, this is by design since sed is a Stream EDitor, not a file editor. It basically makes a copy and replaces the original file with the copy. BashFAQ
Alternatively you can use ex command instead which has similar syntax for substitution, e.g.
ex +%s/cat/dog/ge -scwq pet_link
or multiple files:
ex "+bufdo! %s/cat/dog/ge" -scxa **/pet_link*
It won’t destroy the symbolic links.
Related: How do I prevent sed from destroying hardinks?
Method 3
I find that this also works well (preserving both symbolic and hard links):
sed 's/cat/dog/' pet_link > pet_link.tmp cat pet_link.tmp > pet_link rm pet_link.tmp
Method 4
There is a solution that we sometimes use to write to the same file as is read from. Here is an excerpt from the man page:
sponge reads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before opening the output file. This allows constructing pipelines that read from and write to the same file. It also creates the output file atomically by renaming a temp file into place, and preserves the permissions of the output file if it already exists. If the output file is a special file or symlink, the data will be written to it.
Here is a snippet that shows that it can preserve symbolic links, although I usually use it to preserve inodes:
# Utility functions: print-as-echo, print-line-with-visual-space.
pe() { for _i;do printf "%s" "$_i";done; printf "n"; }
pl() { pe;pe "-----" ;pe "$*"; }
rm -f pet pet_link
echo "cat" > pet
pl " Input data file $FILE:"
head -v pet
pl " Results, before sed:"
ln --symbolic pet pet_link
ls -ligG pet pet_link
# sed --in-place --follow-symlinks 's/cat/dog/' pet_link
pe
pe " Results, after sed:"
sed 's/cat/dog/' pet_link | sponge pet_link
head -v pet
ls -ligG pet pet_link
which produces:
----- Input data file data1: ==> pet <== cat ----- Results, before sed: 1571283 -rw-r--r-- 1 4 Nov 26 23:03 pet 1571286 lrwxrwxrwx 1 3 Nov 26 23:03 pet_link -> pet Results, after sed: ==> pet <== cat 1571283 -rw-r--r-- 1 4 Nov 26 23:03 pet 1571286 lrwxrwxrwx 1 3 Nov 26 23:03 pet_link -> pet
On a system like:
OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64 Distribution : Debian 8.9 (jessie) bash GNU bash 4.3.30
The sponge code is available in a package moreutils — some details:
sponge soak up standard input and write to a file (man) Path : /usr/bin/sponge Package : moreutils Home : http://kitenet.net/~joey/code/moreutils/ Version : 0.52 Type : ELF 64-bit LSB executable, x86-64, version 1 (SYS ...)
At our shop, we wrote a version that writes to a temporary file for the case of very large files.
The package is available on Debian, Fedora, macOS (via brew), etc. … cheers,
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0