Question
I’d like to be able to run a UNIX command precisely every second over a long time period.
I need a solution, which does not lag behind after a certain time, because of the time the command itself needs for execution. sleep, watch, and a certain python script all failed me in this regard.
On the microcontroller’s such as the http://Arduino.cc I’d do that through hardware clock interrupts.
I’d like to know whether there is a similar time-precise shell script solution. All the solutions which I found within StackExchange.com, resulted in a noticeable time lag, if run over hours. See details below.
Practical purpose / application
I want to test whether my network connection is continuously up by sending timestamps via nc (netcat) every 1 second.
Sender:
precise-timestamp-generator | tee netcat-sender.txt | nc $receiver $port
Receiver:
nc -l -p $port > netcat-receiver.txt
After completion, compare the two logs:
diff netcat-sender.txt netcat-receiver.txt
The diffs would be the untransmitted timestamps.
From this I would know at what time my LAN / WAN / ISP makes troubles.
Solution SLEEP
while [ true ]; do date "+%Y-%m-%d %H:%M:%S" ; sleep 1; done | tee timelog-sleep.txt
Gets a certain offset over time, as the command within the loop also takes a little time.
Precision
cat timelog-sleep.txt 2012-07-16 00:45:16 [...] 2012-07-16 10:20:36
Seconds elapsed: 34520
wc -l timelog-sleep.txt
Lines in file: 34243
Precision summarized:
- 34520-34243 = 277 timing problems
- 34520/34243 = 1.008 = 0.8 % off
Solution REPEAT PYTHON
Found at: Repeat a Unix command every x seconds forever
repeat.py 1 "date '+%Y-%m-%d %H:%M:%S'" >> timelog-repeat-py.txt
Supposed to avoid the time offset, but fails to do so.
Precision
wc -l timelog-repeat-py.txt 2012-07-16 13:42:44 [...] 2012-07-16 16:45:24
Seconds elapsed: 10960
wc -l timelog-repeat-py.txt
Lines in file: 10859
Precision summarized:
- 10960-10859 = 101 timing problems
- 10960/10859 = 1.009 = 0.9 % off
Solution WATCH
watch -n 1 "date '+%Y-%m-%d %H:%M:%S' >> ~/Desktop/timelog-watch.txt"
Precision
wc -l timelog-watch.txt 2012-07-16 11:04:08 [...] 2012-07-16 13:25:47
Seconds elapsed: 8499
wc -l timelog-watch.txt
Lines in file: 8366
Precision summarized:
- 8499-8366 = 133 timing problems.
- 8499/8366 = 1.016 = 1.6 % off.
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
The POSIX ualarm() function lets you schedule the kernel to periodically signal your process, with microsecond precision.
Whip up a simple program:
#include<unistd.h>
#include<signal.h>
void tick(int sig){
write(1, "n", 1);
}
int main(){
signal(SIGALRM, tick);
ualarm(1000000, 1000000); //alarm in a second, and every second after that.
for(;;)
pause();
}
Compile
gcc -O2 tick.c -o tick
Then attach it to whatever you need done periodically like so:
./tick | while read x; do
date "+%Y-%m-%d %H:%M:%S"
done | tee timelog-sleep.txt
Method 2
Have you tried watch with the parameter --precise?
watch -n 1 --precise "date '+%Y-%m-%d %H:%M:%S.%N' >> ~/Desktop/timelog-watch.txt"
From the man page:
Normally, this interval is interpreted as the amout of time between
the completion of one run of command and the beginning of the next
run. However, with the -p or –precise option, you can make
watch attempt to run command every interval seconds. Try it with ntptime and notice how the fractional seconds stays (nearly) the
same, as opposed to normal mode where they continuously increase.
The parameter might not be available on your system, though.
You should also consider what should happen when the execution of your program needs more than one second. Should the next scheduled execution be skipped, or should it be run late?
Update: I ran the script for some time, and it didn’t loose a single step:
2561 lines start: 2012-07-17 09:46:34.938805108 end: 2012-07-17 10:29:14.938547796
Update: The --precise flag is a Debian addition, the patch is however rather simple: http://patch-tracker.debian.org/patch/series/view/procps/1:3.2.8-9squeeze1/watch_precision_time.patch
Method 3
crontab has a resolution of 1 minute. If you’re fine with lag time accumulating per that minute and then resetting on the next minute, this basic idea could work:
* * * * * for second in $(seq 0 59); do /path/to/script.sh & sleep 1s;done
Note that script.sh is also run in the background. This should help to minimize the lag that accumulates with each iteration of the loop.
Depending on how much lag sleep generates, there is however the chance of second 59 overlapping with second 0 of the next minute.
EDIT to toss in some results, in the same format as in the question:
$ cat timelog-cron 2012-07-16 20:51:01 ... 2012-07-16 22:43:00
1 hour 52 minutes = 6720 seconds
$ wc -l timelog-cron 6720 timelog-cron
0 timing problems, 0% off. Any time accumulation resets every minute.
Method 4
Your problem is that you’re sleeping for a fixed amount of time after you run your program without taking in to consideration the amount of time that has elapsed since the last time you slept.
You can do this is bash or any other programming language, but the key is to use the clock to determine how long to schedule the next sleep. Before you sleep, check the clock, see how much time you have left, and sleep the difference.
Because of process scheduling compromises, you are not guaranteed to wake up right on the clock tick, but you should be fairly close (within a few ms unloaded, or within a few hundred ms under load). And you won’t accumulate error over time because each time you are re-synchronizing on every sleep cycle and removing any accumulated error.
If you need to hit the clock tick exactly, then what you’re looking for is a real time operating system, which are designed exactly for this purpose.
Method 5
How does this Perl script I just whipped up work?
#!/usr/bin/perl
use strict;
use warnings;
use Time::HiRes qw/time sleep/;
sub launch {
return if fork;
exec @_;
die "Couldn't exec";
}
$SIG{CHLD} = 'IGNORE';
my $interval = shift;
my $start = time();
while (1) {
launch(@ARGV);
$start += $interval;
sleep $start - time();
}
Use: perl timer.pl 1 date '+%Y-%m-%d %H:%M:%S'
It has been running 45 minutes without a single skip, and I suspect it will continue to do so unless a) system load becomes so high that fork() takes more than a second or b) a leap second is inserted.
It cannot guarantee, however, that the command runs at exact second intervals, as there is some overhead, but I doubt it is much worse than an interrupt-based solution.
I ran it for about an hour with date +%N (nanoseconds, GNU extension) and ran some statistics on it. The most lag it had was 1 155 microseconds. Average (arithmetic mean) 216 µs, median 219 µs, standard deviation 42 µs. It ran faster than 270 µs 95% of the time. I don’t think you can beat it except by a C program.
Method 6
I’ve always just given up on having something run exactly on interval. I think you’ll have to write a C program, and pay very careful attention to not exceeding the portion of the 1-second interval with your own code. You’ll probably have to use threading or multiple, inter-communicating processes to get this to work. Take care to avoid thread-starting or process-starting time overhead.
One reference that seems relevant dates to 1993: A Randomized Sampling Clock for CPU Utilization Estimation and Code Profiling You’ll want to take a look at the appendix “Adversary Source Code” to see how they accurately measured time intervals, and “woke up” their program at just the correct time. Since the code is 19 years old, it probably won’t port directly or easily, but if you read it and try to understand it, the principles involved might guide your code.
EDIT:
Found another reference that might help: Effects of Clock Resolution on the Scheduling of Interactive and Soft Real-Time Processes That should help you with any theoretical background.
Method 7
Take a look at nanosleep() (from http://linux.about.com/library/cmd/blcmdl2_nanosleep.htm). Instead of making your program sleep 1 second, make it sleep (1 – ammount it takes to run) seconds. You’ll get a much better resolution.
Method 8
Try running your command in the background so it does not affect the loop timing so much, but even that will not be enough if you don’t want any accumulation for long periods of time as there is surely a few millisecond cost associated with it.
So, this is likely better, but also likely still not good enough:
while [ true ]; do date "+%Y-%m-%d %H:%M:%S" & sleep 1; done | tee timelog-sleep.txt
On my computer this gave 2 errors in 20 minutes or 0,1 per minute, which is roughly a low five fold improvement over your run.
Method 9
Ugly but it works. You should probably rethink the design of your program if you need a loop like this. It basically checks if the current whole second is equal to the previous checked one and prints the number of nanoseconds since change of the second. The accuracy is influenced by the sleep .001.
while true; do T=$( date +%s ); while [[ $T -eq $( date +%s ) ]]; do sleep .001; done; date "+%N nanoseconds late"; done
Accuracy is in the milliseconds, provided that the ‘payload’ date "+%N nanoseconds late" does not take longer than just under a second. You can lower CPU load by increasing the sleep period or if you really don’t mind just replace the sleep command by true.
002112890 nanoseconds late 001847692 nanoseconds late 002273652 nanoseconds late 001317015 nanoseconds late 001650504 nanoseconds late 002180949 nanoseconds late 002338716 nanoseconds late 002064578 nanoseconds late 002160883 nanoseconds late
This is bad practice because you basically make thee CPU poll for an event and you are wasting CPU cycles. You probably want to attach to a timer interrupt (not possible from bash) or use dedicated hardware like a microcontroller. A PC and its operating system are not designed for high timing accuracy.
Method 10
Another method would be to use a suspend in a loop and send SIGCONT from a precise external program. Sending a signal is very lightweight and will have much less latency than executing something. You could also pre-queue a bunch of commands with the “at” command, hardly anybody uses “at” anymore I’m not sure how precise it is.
If precision is critical and you want to get serious about this, this sounds like the sort of application where typically you would use RTOS, which could be done under Linux with RT-Preempt patched kernel, that will give you the precision and some measure of interrupt control, but it may be more bother than it’s worth.
https://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO
Xenomai also might be helpful, it’s a full RTOS implementation and is ported for x86 and x86_64, but there’s some programming involved.
http://www.xenomai.org/index.php/Main_Page
Method 11
With ksh93 (which has a floating point $SECONDS and a builtin sleep)
typeset -F SECONDS=0 typeset -i i=0 while true; do cmd sleep "$((++i - SECONDS))" done
The same script will work with zsh as well but will invoke your system’s sleep command. zsh has a zselect builtin, but with 1/100 resolution only.
Method 12
I’d go with a small C program:
#include <sys/time.h>
#include <unistd.h>
int main(int argc, char **argv, char **envp)
{
struct timeval start;
int rc = gettimeofday(&start, NULL);
if(rc != 0)
return 1;
for(;;)
{
struct timeval now;
rc = gettimeofday(&now, NULL);
useconds_t delay;
if(now.tv_usec < start.tv_usec)
delay = start.tv_usec - now.tv_usec;
else
delay = 1000000 - now.tv_usec + start.tv_usec;
usleep(delay);
pid_t pid = fork();
if(pid == -1)
return 1;
if(pid == 0)
_exit(execve(argv[1], &argv[1], envp));
}
}
This program expects the program to call with full path as its first argument, and passes on any remaining arguments. It will not wait for the command to finish, so it will happily start multiple instances.
Also, the coding style here is really sloppy, and a number of assumptions are made that may or may not be guaranteed by applicable standards, i.e. the quality of this code is “works for me”.
This program will get somewhat longer or shorter intervals when the clock is adjusted by NTP or by manually setting it. If the program should handle this, POSIX provides timer_create(CLOCK_MONOTONIC, ...) which is unaffected by this.
Method 13
You should be keeping track of the current time and be comparing it to the start time. So you sleep a calculated ammount of time each iteration, not a fixed amount. In this way you wont accumulate timing errors and shift away from where you should be because you reset your timings each loop to absolute time from the start.
Also some sleep functions return early if there is an interupt, so in this case you will have to call your sleep method again until the full amount of time has passed.
Method 14
Here’s one that’s a bash script and highly accurate. It uses usleep for microsecond precision.
http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron
There’s 3 scrips there. Look at the one at the bottom. Will do up to about 2400 executions per minute with reasonable accuracy. And it’s really simple.
Method 15
This one can run at least 100 times a second with very accurate resolution.
The existence of the directory of the number of loops per minute creates the schedule. This version supports microsecond resolution assuming your computer can handle it. The number of executions per minute does not have to to evenly divisible by 60 nor is it limited to 60 I have tested it to 6000 and it works.
This version can be installed in the /etc/init.d directory and run as a service.
#! /bin/sh
# chkconfig: 2345 91 61
# description: This program is used to run all programs in a directory in parallel every X times per minute.
# Think of this program as cron with microseconds resolution.
# Microsecond Cron
# Usage: cron-ms start
# Copyright 2014 by Marc Perkel
# docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron"
# Free to use with attribution
# The scheduling is done by creating directories with the number of"
# executions per minute as part of the directory name."
# Examples:
# /etc/cron-ms/7 # Executes everything in that directory 7 times a minute
# /etc/cron-ms/30 # Executes everything in that directory 30 times a minute
# /etc/cron-ms/600 # Executes everything in that directory 10 times a second
# /etc/cron-ms/2400 # Executes everything in that directory 40 times a second
basedir=/etc/cron-ms
case "$1" in
start|restart|reload)
$0 stop
mkdir -p /var/run/cron-ms
for dir in $basedir/* ; do
$0 ${dir##*/} &
done
exit
;;
stop)
rm -Rf /var/run/cron-ms
exit
;;
esac
# Loops per minute is passed on the command line
loops=$1
interval=$((60000000/$loops))
# Just a heartbeat signal that can be used with monit to verify it's alive
touch /var/run/cron-ms
# After a restart the PIDs will be different allowing old processes to terminate
touch /var/run/cron-ms/$$
# Sleeps until a specific part of a minute with microsecond resolution. 60000000 is full minute
usleep $(( $interval - 10#$(date +%S%N) / 1000 % $interval ))
# Deleting the PID files exit the program
if [ ! -f /var/run/cron-ms/$$ ]
then
exit
fi
# Run all the programs in the directory in parallel
for program in $basedir/$loops/* ; do
if [ -x $program ]
then
$program &> /dev/null &
fi
done
exec $0 $loops
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0