Have you ever heard about Leap Seconds? No? Well, you are like me a couple of years ago, when some of our outdated OSs suddenly and inexplicably had various problems. One day (to be more precise, 1st of July 2012) I woke up and the monitoring system was flooding my mail client with tons of alert.
About five machines had some automatic jobs stopped, very high CPU usage, DB stuck, and god knows what else.
After a bit of investigation, I finally noticed something strange in a log. Time was 01:59:60 . WAIT WAIT WAIT. At my primary school they tought me this was wrong. I think I even received a bad valutation when I was 6 years old for an error like this. Well, maybe it's a bug of the logging system. But the next log entry was 02:00:00 . And then I found the same log on every machine. This was suspicious.
As always, I asked Mother Google "What's this?"
LEAP SECOND
The leap second is a second added occasionally to a day, like the leap day 29th February, to adjust UTC (Coordinated Universal Time) to be as similar as possible to solar time. What does it mean?
Ok, without citing too much history, once the second was defined as 1/86400 of the mean solar day. Then someone noted that the solar day is not constant, but slows down irregularly. In search for a more precise time definition, at the moment 1 second is defined as 9.192.631.770 oscillation between two states of an atom of Caesium 133. This value is far more precise. But this second, even if near to the 1/86400 of a mean day definition, is slightly longer: about 0.002 seconds per day. Some maths:
0.002s/d * 365d = 0.73s
1s / 0.002 s/d = 500d = 1,37 y = 1y 4months
So about every year and half day time and universal time differ for about a second. To adjust things, a great convention of time expert decided to insert a leap second whenever necessary. (see Leap Second Wikipedia)
To me, the more I look into this kind of thing, the more it seems a great and useless mess, but here it is. On the other way, many thing around the web and in general in computer science needs to be synchronized, so it is good to have a general overview of the whole thing. By the way, GPS satellites don't use the leap second, so at the moment they are about 16 seconds behind UTC time (US Navy - Leap Second).
SOLUTIONS
In any case, many systems doesn't manage well this situation, or doesn't manage it at all, resolving in strange behaviour. After all, it is a bug.
Preventive:
As far as I understood, there is no official solution to the bug of leap second in all OSs. The main solution is to have NTP (NTP Website) installed and scheduled on your system. NTP stands for Network Time Protocol. It's a protocol to handle time synchronization between machines, where clients connect to a NTP server requesting for the right time and adjust local time in consequence.
Corrective:
In case you have NTP, you just have to do these steps:
- Stop NTP daemon
- Force update of time using ntpdate <ntpserver>
- Start NTP daemon
How to do this, it depends very much on the OS you are using.
REFERENCES
- Marco Marongiu Blog - a deep analysis with tests and benchmarks on how to resolve leap second problem
- Time and Date - A simplified but clear explaination of leap second
- Google Leap Second Solution - The solution (and explanation of the solution) used by google to prevent any leap second related problem
Labels: linux, linux internals, ubuntu