Problems with uk.pool.ntp.org stability

How Nutanix helped us uncover an unexpected issue with a public NTP source For many years now we have, by default, used uk.pool.ntp.org as an NTP source to synchronise time on all our systems. It is becoming increasingly important to keep all systems tightly integrated to a common time reference. This is especially the case with distributed clusters now becoming more common with the advent of Hyperconverged systems. Recently we had started to get alerts on our Nutanix cluster. It started with one cluster, so with Nutanix we were investigating any configuration or networking issues that could be causing it. Then when two other clusters on another site started displaying errors then we began to suspect it was the time source – unlikely as that seems. The problem was intermittent, and determined with a built in NTP configuration check performed hourly by the Nutanix system. We were getting errors reported between 5% and 15% of the time, which varied by node. Every time an error was detected it raised an alert, which was getting wearing, so we had to make some sort of change. All change! We therefore changed all systems over to time.google.com – Google’s new (as of last December) free NTP source. We use their Global DNS Cloud service, so assuming that the level of reliability would be the same we decided to at least try this alternate time source. Immediately the errors stopped. So this indicates that there is a problem server int he UK NTP Org pool. As we’d been having trouble for weeks on and off it doesn’t look like it’s going to get...