join
donate

NTP Drift File Notes

Related Topics:

History

ntpd has long had the option of writing a drift file, usually ntp.drift. The value written there was the recent frequency adjustment ntpd was using to keep the clock in sync. This file was usually updated once an hour.

ntpd initializes its frequency adjustment based on the value stored in ntp.drift and if this value turns out to be accurate, ntpd will achieve SYNC in about 11 seconds' time. If not, or if there is no ntp.drift file, ntpd will embark on a 900 second (minimum?) analysis of the clock to determine the offset and frequency corrections.

If "enough" time has passed, or perhaps more accurately, if environmental conditions have changed significantly since the ntp.drift file was written, the value we'll use to predict the current frequency drift will be incorrect and ntpd will need to do a full-on analysis of the clock to determine the offset and frequency adjustments.

It will take a decent mount of work and effort to determine the amount and type of information we'd want to keep in the ntp.drift file to have the initial drift estimate be "good enough" in a wider range of situations. It is doubtful the added complexity will be worth it, especially with new startup schemes emerging.

Additionally, expanding this scope will likely require temperature data that is generally not available.

The primary reason to avoid writing the ntp.drift file every hour is that a number of embedded platforms want to avoid as many writes to the filesystem as possible.

We could only write the drift file on an orderly shutdown, but a) that doesn't mean the value has changed significantly, and b) there are plenty of times when ntpd will not be able to perform an orderly shutdown.

 
  • Harlen, Yes, sure THe SYNC state can be lost, BUT the drift file which is what I thought the subject was about only comes in to play at start up time. State changes after that are not influenced by what is written there AFIK. I agree that having more info on the long term evolution of local clock drift would be interesting, but maybe not that useful for a general case due to the widely differing devices on which NTP can be run. Studies of very long term crystal drift do exist but those I have looked at relate to timing grade oscillators. I should have said that my comment related to the small set of my own devices. -- MikeCook - 10 Mar 2015
  • Mike, the aim here is to get the drift file so that ntpd achieves the SYNC state in under 11 seconds' time (assuming iburst). I hear what you're saying, and if the initial drift converges at 100 you are saying it won't generally change by more than 10 over the course of time. However, if the initial drift converges on 10, that means it won't change by more than 1 over time. I'm not sure that this is true; it would be good to find out. And the issue is "how big a drift change will cause ntpd to change from the SYNC state?" By my read of the book and the code, this happens if the offset becomes greater than the step threshold (128 ms by default). More below. -- HarlanStenn - 10 Mar 2015
  • The ball park drift is known fairly soon after start up ( 900s max) and after that will only change by a relatively small amounts, say an order of magnitude or less than the initial estimation.). So it could be written early and then forgotten. -- MikeCook - 09 Mar 2015

Goals

The goal of the ntp.drift file is to turn a 900 second (or more) cold-start into something faster. Currently the best we know how to do is an 11-second startup.

We want to do this writing a new ntp.drift file as infrequently as reasonably possible.

 
  • The vast majority of folks don't care how often the drift file is written. The issue is more for folks who have embedded systems where the filestore has a limited number of writes and we want to be respectful of that. -- HarlanStenn - 10 Mar 2015
  • The current goals seem to work for the vast majority of implementations otherwise we would see more bug reports raised. It is however at the moment an ON/OFF switch. Either you create the file, expect writes and profit from getting a fast convergence, or you don't configure a file and expect/accept slow convergence. My tests show that allthough native clock drift can vary greatly over different devices, that of a particular device remains within limited bounds. So configuration either by command line argument or ntp.conf file option, without subsequent file update could add a finer tuning to help some corner case implimentations. -- MikeCook - 10 Mar 2015

How close does the frequency drift need to be?

 
  • I must be missing something, Mike. If the nominal drift is 200ppm, you are saying setting the drift to anywhere between 180 and 220 will get you to <1ms in a few seconds, and if the nominal drift is 10 that setting the drift to 9 or 11 will get you to <1ms in a few seconds? And in this latter case if the drift is set to 5 or 15 it will take 480 seconds to get the clock to <1ms? And you also raise a point - I'm focusing on keeping NTP in the SYNC state while there might be a similar but different goal of getting the clock accurate to some number of milli- or micro-seconds. -- HarlanStenn - 10 Mar 2015
  • My tests show that a value within 10% of the median drift will enable <1ms convergence in a few secs. A 50% error pushes that delay out to 480 secs. On the same systems which have nominal offsets of <5us, retrace is a few secs with accurate drift file, For a drift file 10% in error, 3000s, and for one with a 50% error, 6300s. So I think, for the purposes of serving/aquiring reasonable time , a value within 10% is a good enough, but it is clear that an accurate value is necessary for optimal retrace. -- MikeCook - 09 Mar 2015

Current implementation

The current implementation is documented in the write_stats() function in ntp_util.c .

That documentation says:

When the frequency file is written, initialize the prev_drift_comp and wander_resid. Thereafter, reduce the wander_resid by half each hour. When the difference between the prev_drift_comp and drift_comp is less than the wander_resid, update the frequency file. This minimizes the file writes to nonvolaile storage.

Harlan doesn't think that's what the code implements.

Regardless, this still seems overly complicated.

If we can get a good idea of how close the driftfile must be in order to achieve and hold sync, then perhaps the best we can do is to record the cold-start correction and the range of operational state corrections, and see if we can find a "happy medium".

It is not clear if it's worth this much effort, either...

 
  • Harlen, the code is not wrong, just the comments. Maybe it is me up the creek. Are we discussing keeping sync state or getting there in the first place? They are two different issues for me. When the subject was raised by you on the list it was related to a request by embedded system designers to limit updates to the file. The reason that I raise the % is that one method of limiting writes would be reducing the likelyhood of the trigger being hit. -- MikeCook - 10 Mar 2015
  • How can the code be wrong if the implementation is working as expected? And I'm still not seeing how the % is what we want to look at. -- HarlanStenn - 10 Mar 2015
  • Important to note that it is only helps to achieve sync. Not hold it.

The doc and code comments appear to be wrong, but the implimentation is working as expected. We only re-write the file if the difference between the previous and current values is GREATER than the resid. This works, as for a very stable clock I have a last file update of May 2014.

Could it be improved to further minimise writes without impacting startup times? I think so. Withouth changing the basic implimentation we could fix a lower bound on the absolute value of drift_comp which triggers a write. Say to 10% of the initially calculated/recorded value. -- MikeCook - 09 Mar 2015



This topic: Dev > WebHome > DevelopmentIssues > NtpDriftFileNotes
Topic revision: r7 - 2015-03-10 - 11:37:57 - MikeCook
 
SSL security by CAcert
Get the CAcert Root Certificate
This site is powered by the TWiki collaboration platform
IPv6 Ready
Copyright & 1999-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding the site? Send feedback