feat(ntp): check clock sync on startup (#2251)

This adds method to validate the system clock is synchronized on startup. Configuration allows adjusting the server address, enabled state, desync limit, and if the error is fatal.

Co-authored-by: James Elliott <james-d-elliott@users.noreply.github.com>

config.template.yml

@@ -108,6 +108,29 @@ duo_api:
   ## Secret can also be set using a secret: https://www.authelia.com/docs/configuration/secrets.html
   secret_key: 1234567890abcdefghifjkl
 
+##
+## NTP Configuration
+##
+## This is used to validate the servers time is accurate enough to validate TOTP.
+ntp:
+  ## NTP server address.
+  address: "time.cloudflare.com:123"
+
+  ## NTP version.
+  version: 4
+
+  ## Maximum allowed time offset between the host and the NTP server.
+  max_desync: 3s
+
+  ## Disables the NTP check on startup entirely. This means Authelia will not contact a remote service at all if you
+  ## set this to true, and can operate in a truly offline mode.
+  disable_startup_check: false
+
+  ## The default of false will prevent startup only if we can contact the NTP server and the time is out of sync with
+  ## the NTP server more than the configured max_desync. If you set this to true, an error will be logged but startup
+  ## will continue regardless of results.
+  disable_failure: false
+
 ##
 ## Authentication Backend Provider Configuration
 ##

docs/configuration/ntp.md

@@ -0,0 +1,91 @@
+---
+layout: default
+title: NTP
+parent: Configuration
+nav_order: 9
+---
+
+# NTP
+
+Authelia has the ability to check the system time against an NTP server. Currently this only occurs at startup. This
+section configures and tunes the settings for this check which is primarily used to ensure [TOTP](./one-time-password.md)
+can be accurately validated.
+
+In the instance of inability to contact the NTP server Authelia will just log an error and will continue to run.
+
+## Configuration
+
+```yaml
+ntp:
+  address: "time.cloudflare.com:123"
+  version: 3
+  max_desync: 3s
+  disable_startup_check: false
+  disable_failure: false
+```
+
+## Options
+
+### address
+<div markdown="1">
+type: string
+{: .label .label-config .label-purple } 
+default: time.cloudflare.com:123
+{: .label .label-config .label-blue }
+required: no
+{: .label .label-config .label-green }
+</div>
+
+Determines the address of the NTP server to retrieve the time from. The format is `<host>:<port>`, and both of these are
+required.
+
+### version
+<div markdown="1">
+type: integer
+{: .label .label-config .label-purple } 
+default: 4
+{: .label .label-config .label-blue }
+required: no
+{: .label .label-config .label-green }
+</div>
+
+Determines the NTP verion supported. Valid values are 3 or 4.
+
+### max_desync
+<div markdown="1">
+type: duration
+{: .label .label-config .label-purple } 
+default: 3s
+{: .label .label-config .label-blue }
+required: no
+{: .label .label-config .label-green }
+</div>
+
+This is used to tune the acceptable desync from the time reported from the NTP server. This uses our 
+[duration notation](./index.md#duration-notation-format) format.
+
+### disable_startup_check
+<div markdown="1">
+type: boolean
+{: .label .label-config .label-purple } 
+default: false
+{: .label .label-config .label-blue }
+required: no
+{: .label .label-config .label-green }
+</div>
+
+Setting this to true will disable the startup check entirely.
+
+### disable_failure
+<div markdown="1">
+type: boolean
+{: .label .label-config .label-purple } 
+default: false
+{: .label .label-config .label-blue }
+required: no
+{: .label .label-config .label-green }
+</div>
+
+Setting this to true will allow Authelia to start and just log an error instead of exiting. The default is that if
+Authelia can contact the NTP server successfully, and the time reported by the server is greater than what is configured
+in [max_desync](#max_desync) that Authelia fails to start and logs a fatal error.

docs/configuration/one-time-password.md

@@ -2,7 +2,7 @@
 layout: default
 title: Time-based One-Time Password
 parent: Configuration
-nav_order: 15
+nav_order: 16
 ---
 
 # Time-based One-Time Password
@@ -80,3 +80,13 @@ For example the default of 1 has a total of 3 keys valid. A value of 2 has 5 one
 valid.
 
 It is recommended to keep this value set to 0 or 1, the minimum is 0.
+
+## System time accuracy
+
+It's important to note that if the system time is not accurate enough then clients will seemingly not generate valid
+passwords for TOTP. Conversely this is the same when the client time is not accurate enough. This is due to the Time-based
+One Time Passwords being time-based.
+
+Authelia by default checks the system time against an [NTP server](./ntp.md#address) on startup. This helps to prevent
+a time synchronization issue on the server being an issue. There is however no effective and reliable way to check the
+clients.

docs/configuration/regulation.md

@@ -2,7 +2,7 @@
 layout: default
 title: Regulation
 parent: Configuration
-nav_order: 9
+nav_order: 10
 ---
 
 # Regulation

docs/configuration/secrets.md

@@ -2,7 +2,7 @@
 layout: default
 title: Secrets
 parent: Configuration
-nav_order: 10
+nav_order: 11
 ---
 
 # Secrets

docs/configuration/server.md

@@ -2,7 +2,7 @@
 layout: default
 title: Server
 parent: Configuration
-nav_order: 11
+nav_order: 12
 ---
 
 # Server

docs/configuration/session/index.md

@@ -2,7 +2,7 @@
 layout: default
 title: Session
 parent: Configuration
-nav_order: 12
+nav_order: 13
 has_children: true
 ---
 

docs/configuration/storage/index.md

@@ -2,7 +2,7 @@
 layout: default
 title: Storage Backends
 parent: Configuration
-nav_order: 13
+nav_order: 14
 has_children: true
 ---
 

docs/configuration/theme.md

@@ -2,7 +2,7 @@
 layout: default
 title: Theme
 parent: Configuration
-nav_order: 14
+nav_order: 15
 ---
 
 # Theme

internal/commands/root.go

@@ -12,6 +12,7 @@ import (
 	"github.com/authelia/authelia/v4/internal/logging"
 	"github.com/authelia/authelia/v4/internal/middlewares"
 	"github.com/authelia/authelia/v4/internal/notification"
+	"github.com/authelia/authelia/v4/internal/ntp"
 	"github.com/authelia/authelia/v4/internal/oidc"
 	"github.com/authelia/authelia/v4/internal/regulation"
 	"github.com/authelia/authelia/v4/internal/server"
@@ -80,7 +81,10 @@ func cmdRootRun(_ *cobra.Command, _ []string) {
 	server.Start(*config, providers)
 }
 
+//nolint:gocyclo // TODO: Consider refactoring time permitting.
 func getProviders(config *schema.Configuration) (providers middlewares.Providers, warnings []error, errors []error) {
+	logger := logging.Logger()
+
 	autheliaCertPool, warnings, errors := utils.NewX509CertPool(config.CertificatesDirectory)
 	if len(warnings) != 0 || len(errors) != 0 {
 		return providers, warnings, errors
@@ -133,6 +137,11 @@ func getProviders(config *schema.Configuration) (providers middlewares.Providers
 		}
 	}
 
+	var ntpProvider *ntp.Provider
+	if config.NTP != nil {
+		ntpProvider = ntp.NewProvider(config.NTP)
+	}
+
 	clock := utils.RealClock{}
 	authorizer := authorization.NewAuthorizer(config)
 	sessionProvider := session.NewProvider(config.Session, autheliaCertPool)
@@ -143,12 +152,32 @@ func getProviders(config *schema.Configuration) (providers middlewares.Providers
 		errors = append(errors, err)
 	}
 
+	var failed bool
+	if !config.NTP.DisableStartupCheck && authorizer.IsSecondFactorEnabled() {
+		failed, err = ntpProvider.StartupCheck()
+
+		if err != nil {
+			logger.Errorf("Failed to check time against the NTP server: %+v", err)
+		}
+
+		if failed {
+			if config.NTP.DisableFailure {
+				logger.Error("The system time is outside the maximum desynchronization when compared to the time reported by the NTP server, this may cause issues in validating TOTP secrets")
+			} else {
+				logger.Fatal("The system time is outside the maximum desynchronization when compared to the time reported by the NTP server")
+			}
+		} else {
+			logger.Debug("The system time is within the maximum desynchronization when compared to the time reported by the NTP server")
+		}
+	}
+
 	return middlewares.Providers{
 		Authorizer:      authorizer,
 		UserProvider:    userProvider,
 		Regulator:       regulator,
 		OpenIDConnect:   oidcProvider,
 		StorageProvider: storageProvider,
+		NTP:             ntpProvider,
 		Notifier:        notifier,
 		SessionProvider: sessionProvider,
 	}, warnings, errors

internal/configuration/config.template.yml

@@ -108,6 +108,29 @@ duo_api:
   ## Secret can also be set using a secret: https://www.authelia.com/docs/configuration/secrets.html
   secret_key: 1234567890abcdefghifjkl
 
+##
+## NTP Configuration
+##
+## This is used to validate the servers time is accurate enough to validate TOTP.
+ntp:
+  ## NTP server address.
+  address: "time.cloudflare.com:123"
+
+  ## NTP version.
+  version: 4
+
+  ## Maximum allowed time offset between the host and the NTP server.
+  max_desync: 3s
+
+  ## Disables the NTP check on startup entirely. This means Authelia will not contact a remote service at all if you
+  ## set this to true, and can operate in a truly offline mode.
+  disable_startup_check: false
+
+  ## The default of false will prevent startup only if we can contact the NTP server and the time is out of sync with
+  ## the NTP server more than the configured max_desync. If you set this to true, an error will be logged but startup
+  ## will continue regardless of results.
+  disable_failure: false
+
 ##
 ## Authentication Backend Provider Configuration
 ##

internal/configuration/schema/configuration.go

@@ -22,6 +22,7 @@ type Configuration struct {
 	TOTP                  *TOTPConfiguration                 `koanf:"totp"`
 	DuoAPI                *DuoAPIConfiguration               `koanf:"duo_api"`
 	AccessControl         AccessControlConfiguration         `koanf:"access_control"`
+	NTP                   *NTPConfiguration                  `koanf:"ntp"`
 	Regulation            *RegulationConfiguration           `koanf:"regulation"`
 	Storage               StorageConfiguration               `koanf:"storage"`
 	Notifier              *NotifierConfiguration             `koanf:"notifier"`

internal/configuration/schema/ntp.go

@@ -0,0 +1,17 @@
+package schema
+
+// NTPConfiguration represents the configuration related to ntp server.
+type NTPConfiguration struct {
+	Address             string `koanf:"address"`
+	Version             int    `koanf:"version"`
+	MaximumDesync       string `koanf:"max_desync"`
+	DisableStartupCheck bool   `koanf:"disable_startup_check"`
+	DisableFailure      bool   `koanf:"disable_failure"`
+}
+
+// DefaultNTPConfiguration represents default configuration parameters for the NTP server.
+var DefaultNTPConfiguration = NTPConfiguration{
+	Address:       "time.cloudflare.com:123",
+	Version:       4,
+	MaximumDesync: "3s",
+}

internal/configuration/validator/configuration.go

@@ -65,4 +65,10 @@ func ValidateConfiguration(configuration *schema.Configuration, validator *schem
 	}
 
 	ValidateIdentityProviders(&configuration.IdentityProviders, validator)
+
+	if configuration.NTP == nil {
+		configuration.NTP = &schema.DefaultNTPConfiguration
+	}
+
+	ValidateNTP(configuration.NTP, validator)
 }

internal/configuration/validator/const.go

@@ -301,6 +301,13 @@ var ValidKeys = []string{
 	"identity_providers.oidc.clients[].scopes",
 	"identity_providers.oidc.clients[].grant_types",
 	"identity_providers.oidc.clients[].response_types",
+
+	// NTP keys.
+	"ntp.address",
+	"ntp.version",
+	"ntp.max_desync",
+	"ntp.disable_startup_check",
+	"ntp.disable_failure",
 }
 
 var replacedKeys = map[string]string{

internal/configuration/validator/ntp.go

@@ -0,0 +1,30 @@
+package validator
+
+import (
+	"fmt"
+
+	"github.com/authelia/authelia/v4/internal/configuration/schema"
+	"github.com/authelia/authelia/v4/internal/utils"
+)
+
+// ValidateNTP validates and update NTP configuration.
+func ValidateNTP(configuration *schema.NTPConfiguration, validator *schema.StructValidator) {
+	if configuration.Address == "" {
+		configuration.Address = schema.DefaultNTPConfiguration.Address
+	}
+
+	if configuration.Version == 0 {
+		configuration.Version = schema.DefaultNTPConfiguration.Version
+	} else if configuration.Version < 3 || configuration.Version > 4 {
+		validator.Push(fmt.Errorf("ntp: version must be either 3 or 4"))
+	}
+
+	if configuration.MaximumDesync == "" {
+		configuration.MaximumDesync = schema.DefaultNTPConfiguration.MaximumDesync
+	}
+
+	_, err := utils.ParseDurationString(configuration.MaximumDesync)
+	if err != nil {
+		validator.Push(fmt.Errorf("ntp: error occurred parsing NTP max_desync string: %s", err))
+	}
+}

internal/configuration/validator/ntp_test.go

@@ -0,0 +1,65 @@
+package validator
+
+import (
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+
+	"github.com/authelia/authelia/v4/internal/configuration/schema"
+)
+
+func newDefaultNTPConfig() schema.NTPConfiguration {
+	config := schema.NTPConfiguration{}
+	return config
+}
+
+func TestShouldSetDefaultNtpAddress(t *testing.T) {
+	validator := schema.NewStructValidator()
+	config := newDefaultNTPConfig()
+
+	ValidateNTP(&config, validator)
+
+	assert.Len(t, validator.Errors(), 0)
+	assert.Equal(t, schema.DefaultNTPConfiguration.Address, config.Address)
+}
+
+func TestShouldSetDefaultNtpVersion(t *testing.T) {
+	validator := schema.NewStructValidator()
+	config := newDefaultNTPConfig()
+
+	ValidateNTP(&config, validator)
+
+	assert.Len(t, validator.Errors(), 0)
+	assert.Equal(t, schema.DefaultNTPConfiguration.Version, config.Version)
+}
+
+func TestShouldSetDefaultNtpMaximumDesync(t *testing.T) {
+	validator := schema.NewStructValidator()
+	config := newDefaultNTPConfig()
+
+	ValidateNTP(&config, validator)
+
+	assert.Len(t, validator.Errors(), 0)
+	assert.Equal(t, schema.DefaultNTPConfiguration.MaximumDesync, config.MaximumDesync)
+}
+
+func TestShouldSetDefaultNtpDisableStartupCheck(t *testing.T) {
+	validator := schema.NewStructValidator()
+	config := newDefaultNTPConfig()
+
+	ValidateNTP(&config, validator)
+
+	assert.Len(t, validator.Errors(), 0)
+	assert.Equal(t, schema.DefaultNTPConfiguration.DisableStartupCheck, config.DisableStartupCheck)
+}
+
+func TestShouldRaiseErrorOnMaximumDesyncString(t *testing.T) {
+	validator := schema.NewStructValidator()
+	config := newDefaultNTPConfig()
+	config.MaximumDesync = "a second"
+
+	ValidateNTP(&config, validator)
+
+	assert.Len(t, validator.Errors(), 1)
+	assert.EqualError(t, validator.Errors()[0], "ntp: error occurred parsing NTP max_desync string: could not convert the input string of a second into a duration")
+}

internal/middlewares/types.go

@@ -9,6 +9,7 @@ import (
 	"github.com/authelia/authelia/v4/internal/authorization"
 	"github.com/authelia/authelia/v4/internal/configuration/schema"
 	"github.com/authelia/authelia/v4/internal/notification"
+	"github.com/authelia/authelia/v4/internal/ntp"
 	"github.com/authelia/authelia/v4/internal/oidc"
 	"github.com/authelia/authelia/v4/internal/regulation"
 	"github.com/authelia/authelia/v4/internal/session"
@@ -33,7 +34,7 @@ type Providers struct {
 	SessionProvider *session.Provider
 	Regulator       *regulation.Regulator
 	OpenIDConnect   oidc.OpenIDConnectProvider
-
+	NTP             *ntp.Provider
 	UserProvider    authentication.UserProvider
 	StorageProvider storage.Provider
 	Notifier        notification.Notifier

internal/ntp/const.go

@@ -0,0 +1,15 @@
+package ntp
+
+const (
+	ntpClientModeValue  uint8 = 3  // 00000011
+	ntpLeapEnabledValue uint8 = 64 // 01000000
+	ntpVersion3Value    uint8 = 24 // 00011000
+	ntpVersion4Value    uint8 = 40 // 00101000
+)
+
+const ntpEpochOffset = 2208988800
+
+const (
+	ntpV3 ntpVersion = iota
+	ntpV4
+)

internal/ntp/ntp.go

@@ -0,0 +1,55 @@
+package ntp
+
+import (
+	"encoding/binary"
+	"fmt"
+	"net"
+	"time"
+
+	"github.com/authelia/authelia/v4/internal/configuration/schema"
+	"github.com/authelia/authelia/v4/internal/utils"
+)
+
+// NewProvider instantiate a ntp provider given a configuration.
+func NewProvider(config *schema.NTPConfiguration) *Provider {
+	return &Provider{config}
+}
+
+// StartupCheck checks if the system clock is not out of sync.
+func (p *Provider) StartupCheck() (failed bool, err error) {
+	conn, err := net.Dial("udp", p.config.Address)
+	if err != nil {
+		return false, fmt.Errorf("could not connect to NTP server to validate the time desync: %w", err)
+	}
+
+	defer conn.Close()
+
+	if err := conn.SetDeadline(time.Now().Add(5 * time.Second)); err != nil {
+		return false, fmt.Errorf("could not connect to NTP server to validate the time desync: %w", err)
+	}
+
+	version := ntpV4
+	if p.config.Version == 3 {
+		version = ntpV3
+	}
+
+	req := &ntpPacket{LeapVersionMode: ntpLeapVersionClientMode(false, version)}
+
+	if err := binary.Write(conn, binary.BigEndian, req); err != nil {
+		return false, fmt.Errorf("could not write to the NTP server socket to validate the time desync: %w", err)
+	}
+
+	now := time.Now()
+
+	resp := &ntpPacket{}
+
+	if err := binary.Read(conn, binary.BigEndian, resp); err != nil {
+		return false, fmt.Errorf("could not read from the NTP server socket to validate the time desync: %w", err)
+	}
+
+	maxOffset, _ := utils.ParseDurationString(p.config.MaximumDesync)
+
+	ntpTime := ntpPacketToTime(resp)
+
+	return ntpIsOffsetTooLarge(maxOffset, now, ntpTime), nil
+}

internal/ntp/ntp_test.go

@@ -0,0 +1,26 @@
+package ntp
+
+import (
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+
+	"github.com/authelia/authelia/v4/internal/configuration/schema"
+	"github.com/authelia/authelia/v4/internal/configuration/validator"
+)
+
+func TestShouldCheckNTP(t *testing.T) {
+	config := schema.NTPConfiguration{
+		Address:             "time.cloudflare.com:123",
+		Version:             4,
+		MaximumDesync:       "3s",
+		DisableStartupCheck: false,
+	}
+	sv := schema.NewStructValidator()
+	validator.ValidateNTP(&config, sv)
+
+	NTP := NewProvider(&config)
+
+	checkfailed, _ := NTP.StartupCheck()
+	assert.Equal(t, false, checkfailed)
+}

internal/ntp/types.go

@@ -0,0 +1,30 @@
+package ntp
+
+import (
+	"github.com/authelia/authelia/v4/internal/configuration/schema"
+)
+
+// Provider type is the NTP provider.
+type Provider struct {
+	config *schema.NTPConfiguration
+}
+
+type ntpVersion int
+
+type ntpPacket struct {
+	LeapVersionMode       uint8
+	Stratum               uint8
+	Poll                  int8
+	Precision             int8
+	RootDelay             uint32
+	RootDispersion        uint32
+	ReferenceID           uint32
+	ReferenceTimeSeconds  uint32
+	ReferenceTimeFraction uint32
+	OriginTimeSeconds     uint32
+	OriginTimeFraction    uint32
+	RxTimeSeconds         uint32
+	RxTimeFraction        uint32
+	TxTimeSeconds         uint32
+	TxTimeFraction        uint32
+}

internal/ntp/util.go

@@ -0,0 +1,42 @@
+package ntp
+
+import "time"
+
+// ntpLeapVersionClientMode does the mathematics to configure the leap/version/mode value of an NTP client packet.
+func ntpLeapVersionClientMode(leap bool, version ntpVersion) (lvm uint8) {
+	lvm = ntpClientModeValue
+
+	if leap {
+		lvm += ntpLeapEnabledValue
+	}
+
+	switch version {
+	case ntpV3:
+		lvm += ntpVersion3Value
+	case ntpV4:
+		lvm += ntpVersion4Value
+	}
+
+	return lvm
+}
+
+// ntpPacketToTime converts a NTP server response into a time.Time.
+func ntpPacketToTime(packet *ntpPacket) time.Time {
+	seconds := float64(packet.TxTimeSeconds) - ntpEpochOffset
+	nanoseconds := (int64(packet.TxTimeFraction) * 1e9) >> 32
+
+	return time.Unix(int64(seconds), nanoseconds)
+}
+
+// ntpIsOffsetTooLarge return true if there is offset of "offset" between two times.
+func ntpIsOffsetTooLarge(maxOffset time.Duration, first, second time.Time) (tooLarge bool) {
+	var offset time.Duration
+
+	if first.After(second) {
+		offset = first.Sub(second)
+	} else {
+		offset = second.Sub(first)
+	}
+
+	return offset > maxOffset
+}

internal/ntp/util_test.go

@@ -0,0 +1,16 @@
+package ntp
+
+import (
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/assert"
+
+	"github.com/authelia/authelia/v4/internal/utils"
+)
+
+func TestShould(t *testing.T) {
+	maxOffset, _ := utils.ParseDurationString("1s")
+	assert.True(t, ntpIsOffsetTooLarge(maxOffset, time.Now(), time.Now().Add(time.Second*2)))
+	assert.False(t, ntpIsOffsetTooLarge(maxOffset, time.Now(), time.Now()))
+}

internal/suites/Standalone/configuration.yml

@@ -89,4 +89,13 @@ notifier:
     port: 1025
     sender: admin@example.com
     disable_require_tls: true
+ntp:
+  ## NTP server address
+  address: "time.cloudflare.com:123"
+  ## ntp version
+  version: 4
+  ## "maximum desynchronization" is the allowed offset time between the host and the ntp server
+  max_desync: 3s
+  ## You can enable or disable the NTP synchronization check on startup
+  disable_startup_check: false
 ...