Troubleshooting Time Synchronization Issues with SSH Access

Motivation:

Luminate's SSH solution uses short-lived certificates to perform authentication with the SSH server.

During the certificate validation, the SSH server uses the current time to make sure the certificate hasn't expired and therefore the time on the SSH server machine must be synced with the current global UTC time.

Troubleshooting

Generally speaking, a troubleshooting on the SSH server side should start off by examining the /var/log/auth.log file or /var/log/secure depends on the OS.

In some cases, you can make the debug logs more verbose, by adding LogLevel DEBUG3 line to the sshd_config file and restarting the sshd service (this will not disconnect active sessions).

In our case, the way to diagnose the issue:

• User experience:
  • Received disconnect from 10.0.6.11: 14: No more authentication methods available
  • Disconnected from 10.0.6.11

• This should lead you to find the following lines on the Server side in /var/log/auth.log:
  • error: key_cert_check_authority: invalid certificate
  • error: Certificate invalid: not yet valid

To fix the time synchronization issues we will use the ntpdate utility:

Installing the ntpdate tool:

$ sudo apt-get install ntpdate    [On Debian/Ubuntu]
$ sudo yum  install ntpdate       [On CentOS/RHEL]
$ sudo dnf install ntpdate        [On Fedora 22+]

1. First, run ntpdate in query-only mode, this way ntpdate will query the global NTP servers and will display the delta from local time:

Above, we can see that ntpdate detecting an offset of 759 seconds from global NTP time that means our local time is not correctly synced.

2. To fix the issue, execute ntpdate in update mode:

3. you can now verify again and make sure the time is synced correctly.

 To avoid future time synchronization issues we recommend using NTP Daemon for automatic time synchronization. This should be done together with the customers' IT department and can use manuals such as https://www.tecmint.com/install-ntp-server-in-centos/