I have a general question, which might be a result of misunderstanding of how processes are handled in Linux.
For my purposes I am going to define a ‘script’ as a snippet of bash code saved to a text file with execute permissions enabled for the current user.
I have a series of scripts that call each other in tandem. For simplicity’s sake I’ll call them scripts A, B, and C. Script A carries out a series of statements and then pauses, it then executes script B, then it pauses, then it executes script C. In other words, the series of steps is something like this:
Run Script A:
- Series of statements
- Pause
- Run Script B
- Pause
- Run Script C
I know from experience that if I run script A until the first pause, then make edits in script B, those edits are reflected in the execution of the code when I allow it to resume. Likewise if I make edits to script C while script A is still paused, then allow it to continue after saving changes, those changes are reflected in the execution of the code.
Here is the real question then, is there any way to edit Script A while it is still running? Or is editing impossible once its execution begins?
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
In Unix, most editors work by creating a new temporary file containing the edited contents. When the edited file is saved, the original file is deleted and the temporary file renamed to the original name. (There are, of course, various safeguards to prevent dataloss.) This is, for example, the style used by sed or perl when invoked with the -i (“in-place”) flag, which is not really “in-place” at all. It should have been called “new place with old name”.
This works well because unix assures (at least for local filesystems) that an opened file continues to exist until it is closed, even if it is “deleted” and a new file with the same name is created. (It’s not coincidental that the unix system call to “delete” a file is actually called “unlink”.) So, generally speaking, if a shell interpreter has some source file open, and you “edit” the file in the manner described above, the shell won’t even see the changes since it still has the original file open.
[Note: as with all standards-based comments, the above is subject to multiple interpretations and there are various corner-cases, such as NFS. Pedants are welcome to fill the comments with exceptions.]
It is, of course, possible to modify files directly; it’s just not very convenient for editing purposes, because while you can overwrite data in a file, you cannot delete or insert without shifting all following data, which would imply quite a lot of rewriting. Furthermore, while you were doing that shifting, the contents of the file would be unpredictable and processes which had the file open would suffer. In order to get away with this (as with database systems, for example), you need a sophisticated set of modification protocols and distributed locks; stuff which is well beyond the scope of a typical file editing utility.
So, if you want to edit a file while its being processed by a shell, you have two options:
- You can append to the file. This should always work.
- You can overwrite the file with new contents of exactly the same length. This may or may not work, depending on whether the shell has already read that part of the file or not. Since most file I/O involves read buffers, and since all the shells I know read an entire compound command before executing it, it is pretty unlikely that you can get away with this. It certainly wouldn’t be reliable.
I don’t know of any wording in the Posix standard which actually requires the possibility of appending to a script file while the file is being executed, so it might not work with every Posix compliant shell, much less with the current offering of almost- and sometimes-posix-compliant shells. So YMMV. But as far as I know, it does work reliably with bash.
As evidence, here’s a “loop-free” implementation of the infamous 99 bottles of beer program in bash, which uses dd to overwrite and append (the overwriting is presumably safe because it substitutes the currently executing line, which is always the last line of the file, with a comment of exactly the same length; I did that so that the end result can be executed without the self-modifying behaviour.)
#!/bin/bash
if [[ $1 == reset ]]; then
printf "%sn%-16s#n" '####' 'next ${1:-99}' |
dd if=/dev/stdin of=$0 seek=$(grep -bom1 ^#### $0 | cut -f1 -d:) bs=1 2>/dev/null
exit
fi
step() {
s=s
one=one
case $beer in
2) beer=1; unset s;;
1) beer="No more"; one=it;;
"No more") beer=99; return 1;;
*) ((--beer));;
esac
}
next() {
step ${beer:=$(($1+1))}
refrain |
dd if=/dev/stdin of=$0 seek=$(grep -bom1 ^next $0 | cut -f1 -d:) bs=1 conv=notrunc 2>/dev/null
}
refrain() {
printf "%-17sn" "# $beer bottles"
echo echo ${beer:-No more} bottle$s of beer on the wall, ${beer:-No more} bottle$s of beer.
if step; then
echo echo Take $one down, pass it around, $beer bottle$s of beer on the wall.
echo echo
echo next abcdefghijkl
else
echo echo Go to the store, buy some more, $beer bottle$s of beer on the wall.
fi
}
####
next ${1:-99} #
Method 2
bash goes a long way to make sure it reads commands just before executing them.
For instance in:
cmd1 cmd2
The shell will read the script by blocks, so likely read both commands, interpret the first one and then seek back to the end of cmd1 in the script and read the script again to read cmd2 and execute it.
You can easily verify it:
$ cat a echo foo | dd 2> /dev/null bs=1 seek=50 of=a echo bar $ bash a foo
(though looking at the strace output on that, it seems it does some more fancy things (like read the data several times, seek back…) than when I tried the same a few years ago, so my statement above about lseeking back may not apply anymore on newer versions).
If however you write your script as:
{
cmd1
cmd2
exit
}
The shell will have to read up to the closing }, store that in memory and execute it. Because of the exit, the shell will not read from the script again so you can edit it safely while the shell is interpreting it.
Alternatively, when editing the script, make sure you write a new copy of the script. The shell will keep reading the original one (even if it’s deleted or renamed).
To do that, rename the-script to the-script.old and copy the-script.old to the-script and edit it.
Method 3
There is really no safe way to modify the script while it is running because the shell can use buffering to read the file. In addition, if the script is modified by replacing it with a new file, shells will typically only read the new file after performing certain operations.
Often, when a script is changed while executing, the shell ends up reporting syntax errors. This is due to the fact that, when the shell closes and reopens the script file, it uses the byte offset into the file to reposition itself on return.
Method 4
You could get around this by setting a trap on your script, and then using exec to pick up the new script contents. Note however, the exec call starts the script from scratch and not from where it has reached in the running process, and so script B will get called (on so forth).
#! /bin/bash
CMD="$0"
ARGS=("<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="604420">[email protected]</a>")
trap reexec 1
reexec() {
exec "$CMD" "${ARGS[@]}"
}
while : ; do sleep 1 ; clear ; date ; done
This will continue to display the date on the screen. I could then edit my script and
change date to echo "Date: $(date)". On writing that out the running script still just displays the date. How ever if I send the signal that I set the trap to capture, the script will exec (replaces the current running process with the command specified) which is the command $CMD and the arguments [email protected]. You can do this by issuing kill -1 PID – where PID is the PID of the running script – and the output changes to show Date: before the date command output.
You could store the “state” of your script in an external file (in say /tmp), and read the contents to know where to “resume” on the when the program is re-exec’ed. You could then add an additional traps termination (SIGINT/SIGQUIT/SIGKILL/SIGTERM) to clear up that tmp file so when you restart after interrupting the “Script A” it will start from the beginning. A stateful version would be something like:
#! /bin/bash
trap reexec 1
trap cleanup 2 3 9 15
CMD="$0"
ARGS=("<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="022642">[email protected]</a>")
statefile='/tmp/scriptA.state'
EXIT=1
reexec() { echo "Restarting..." ; exec "$CMD" "${ARGS[@]}"; }
cleanup() { rm -f $statefile; exit $EXIT; }
run_scriptB() { /path/to/scriptB; echo "scriptC" > $statefile; }
run_scriptC() { /path/to/scriptC; echo "stop" > $statefile; }
while [ "$state" != "stop" ] ; do
if [ -f "$statefile" ] ; then
state="$(cat "$statefile")"
else
state='starting'
fi
case "$state" in
starting)
run_scriptB
;;
scriptC)
run_scriptC
;;
esac
done
EXIT=0
cleanup
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0