Saturday, May 26, 2018

ntp configuration to avoid KUDU ntptime error in lab/poc envronment

[root@cdh01 ~]# tail -n 5 /etc/ntp.conf
# Undisciplined Local Clock. This is a fake driver intended for backup
# and when no outside source of synchronized time is available.
server  127.127.1.0     # local clock
fudge   127.127.1.0 stratum 10


[root@cdh01 ~]# ntptime
ntp_gettime() returns code 0 (OK)
  time deb2b920.6f031890  Sat, May 26 2018  0:32:32.433, (.433641853),
  maximum error 133377 us, estimated error 527 us, TAI offset 0
ntp_adjtime() returns code 0 (OK)
  modes 0x0 (),
  offset 0.000 us, frequency -12.739 ppm, interval 1 s,
  maximum error 133377 us, estimated error 527 us,
  status 0x2000 (NANO),
  time constant 2, precision 0.001 us, tolerance 500 ppm,
=====================================

Alternative workaround:

Note: Use a Reliable clock in Linux.  Normally you need to install and configure NTP so Kudu has a reliable clock.  Without a reliable clock you may get errors starting and running Kudu that NTP would resolve. For development only, it is possible to avoid setting up and running ntpd with a reliable time server by running Kudu with the setting --use-hybrid-clock=false. However that setting has a serious effect on transactional consistency so it's not something we recommend for production or load testing.  Furthermore without a reliable clock provided by NTP, Kudu skips cleanup work it would normally do.  Your disk size will grow beyond what is normal and performance will be negatively impacted.5.For just this short lab only environment, because it is non-production, and we don’t care about performance in the lab, we will set the not recommended--use-hybrid-clock=false parameter.  Normally never forget to setup NTP for Kudu in production!

Click on the Kudu Service>Configuration. Look for the setting, “Kudu Service AdvancedConfiguration Snippet (Safety Valve) for gflagfile” and add the setting:

--use-hybrid-clock=false

Note: if you don’t set this clock setting correctly (or setup NTP), Kudu will fail to start intermittently and the full log file for the Kudu service will have errors indicating timesync problems such as:
Check failed:_s.ok() Bad status: Service unavailable: Cannot initialize clock: Error reading clock. Clock considered unsynchronized