Failed to call clock_adjtime(): Operation not permitted running systemd-timesyncd - probably permission related

systemd-timesyncd unit fails on my Pi 4B running Linux 5.10.103-v7l+

openhabian@openhab:~ $ systemctl -a --failed
  UNIT                      LOAD   ACTIVE SUB    DESCRIPTION                 
● systemd-timesyncd.service loaded failed failed Network Time Synchronization

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

1 loaded units listed.
To show all installed unit files use 'systemctl list-unit-files'.

openhabian@openhab:~ $

I can see in journalctl that systemd-timesyncd is starting and stopping until it fails for too many attempts:

@openhab:~ $ journalctl -u systemd-timesyncd
-- Logs begin at Sat 2022-12-03 07:12:50 CST, end at Sat 2022-12-03 11:23:56 CST. --
Dec 03 11:20:27 openhab systemd: Starting Network Time Synchronization...
Dec 03 11:20:27 openhab systemd: Started Network Time Synchronization.
Dec 03 11:20:27 openhab systemd-timesyncd: Synchronized to time server for the first time 198.137.202.56:123 (0.debian.pool.ntp.org).
Dec 03 11:21:44 openhab systemd: Stopping Network Time Synchronization...
Dec 03 11:21:44 openhab systemd: systemd-timesyncd.service: Succeeded.
Dec 03 11:21:44 openhab systemd: Stopped Network Time Synchronization.
Dec 03 11:21:44 openhab systemd: Starting Network Time Synchronization...
Dec 03 11:21:44 openhab systemd: Started Network Time Synchronization.
Dec 03 11:21:44 openhab systemd-timesyncd: Synchronized to time server for the first time 202.22.158.104:123 (0.debian.pool.ntp.org).
Dec 03 11:22:25 openhab systemd: Stopping Network Time Synchronization...
Dec 03 11:22:25 openhab systemd: systemd-timesyncd.service: Succeeded.
Dec 03 11:22:25 openhab systemd: Stopped Network Time Synchronization.
Dec 03 11:22:25 openhab systemd: Starting Network Time Synchronization...
Dec 03 11:22:26 openhab systemd: Started Network Time Synchronization.
Dec 03 11:22:26 openhab systemd-timesyncd: Synchronized to time server for the first time 198.206.184.196:123 (0.debian.pool.ntp.org).
Dec 03 11:22:37 openhab systemd: Stopping Network Time Synchronization...
Dec 03 11:22:37 openhab systemd: systemd-timesyncd.service: Succeeded.
Dec 03 11:22:37 openhab systemd: Stopped Network Time Synchronization.
Dec 03 11:22:37 openhab systemd: Starting Network Time Synchronization...
Dec 03 11:22:37 openhab systemd: Started Network Time Synchronization.
Dec 03 11:22:37 openhab systemd-timesyncd: Synchronized to time server for the first time 198.206.184.196:123 (0.debian.pool.ntp.org).
Dec 03 11:22:38 openhab systemd: Stopping Network Time Synchronization...
Dec 03 11:22:38 openhab systemd: systemd-timesyncd.service: Succeeded.
Dec 03 11:22:38 openhab systemd: Stopped Network Time Synchronization.
Dec 03 11:22:38 openhab systemd: Starting Network Time Synchronization...
Dec 03 11:22:38 openhab systemd: Started Network Time Synchronization.
Dec 03 11:22:38 openhab systemd-timesyncd: Synchronized to time server for the first time 147.182.158.78:123 (0.debian.pool.ntp.org).
Dec 03 11:22:39 openhab systemd: Stopping Network Time Synchronization...
Dec 03 11:22:39 openhab systemd: systemd-timesyncd.service: Succeeded.
Dec 03 11:22:39 openhab systemd: Stopped Network Time Synchronization.
Dec 03 11:22:39 openhab systemd: Starting Network Time Synchronization...
Dec 03 11:22:39 openhab systemd: Started Network Time Synchronization.
Dec 03 11:22:40 openhab systemd: Stopping Network Time Synchronization...
Dec 03 11:22:40 openhab systemd: systemd-timesyncd.service: Succeeded.
Dec 03 11:22:40 openhab systemd: Stopped Network Time Synchronization.
Dec 03 11:22:40 openhab systemd: Starting Network Time Synchronization...
Dec 03 11:22:40 openhab systemd: Started Network Time Synchronization.
Dec 03 11:22:41 openhab systemd: Stopping Network Time Synchronization...
Dec 03 11:22:41 openhab systemd: systemd-timesyncd.service: Succeeded.
Dec 03 11:22:41 openhab systemd: Stopped Network Time Synchronization.
Dec 03 11:22:41 openhab systemd: Starting Network Time Synchronization...
Dec 03 11:22:41 openhab systemd: Started Network Time Synchronization.
Dec 03 11:22:42 openhab systemd: Stopping Network Time Synchronization...
Dec 03 11:22:42 openhab systemd: systemd-timesyncd.service: Succeeded.
Dec 03 11:22:42 openhab systemd: Stopped Network Time Synchronization.
Dec 03 11:22:42 openhab systemd: systemd-timesyncd.service: Start request repeated too quickly.
Dec 03 11:22:42 openhab systemd: systemd-timesyncd.service: Failed with result 'start-limit-hit'.
Dec 03 11:22:42 openhab systemd: Failed to start Network Time Synchronization.

I tried running it with debug logging using the same user that is running it, systemd-timesync:

@openhab:~ $ sudo -u systemd-timesync SYSTEMD_LOG_LEVEL=debug /lib/systemd/systemd-timesyncd
Failed to get link NTP servers: No data available
Bus bus-api-timesync: changing state UNSET → OPENING
Bus bus-api-timesync: changing state OPENING → AUTHENTICATING
Added new server 0.debian.pool.ntp.org.
Added new server 1.debian.pool.ntp.org.
Added new server 2.debian.pool.ntp.org.
Added new server 3.debian.pool.ntp.org.
systemd-timesyncd running as pid 30632
Selected server 0.debian.pool.ntp.org.
Resolving 0.debian.pool.ntp.org...
Bus bus-api-timesync: changing state AUTHENTICATING → HELLO
Sent message type=method_call sender=n/a destination=org.freedesktop.DBus path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=Hello cookie=1 reply_cookie=0 signature=n/a error-name=n/a error-message=n/a
Sent message type=method_call sender=n/a destination=org.freedesktop.DBus path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=RequestName cookie=2 reply_cookie=0 signature=su error-name=n/a error-message=n/a
Got message type=method_return sender=org.freedesktop.DBus destination=:1.1680 path=n/a interface=n/a member=n/a cookie=1 reply_cookie=1 signature=s error-name=n/a error-message=n/a
Bus bus-api-timesync: changing state HELLO → RUNNING
Got message type=signal sender=org.freedesktop.DBus.Local destination=n/a path=/org/freedesktop/DBus/Local interface=org.freedesktop.DBus.Local member=Connected cookie=4294967295 reply_cookie=0 signature=n/a error-name=n/a error-message=n/a
Got message type=signal sender=org.freedesktop.DBus destination=:1.1680 path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=NameAcquired cookie=2 reply_cookie=0 signature=s error-name=n/a error-message=n/a
Got message type=signal sender=org.freedesktop.DBus destination=:1.1680 path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=NameAcquired cookie=3 reply_cookie=0 signature=s error-name=n/a error-message=n/a
Got message type=method_return sender=org.freedesktop.DBus destination=:1.1680 path=n/a interface=n/a member=n/a cookie=4 reply_cookie=2 signature=u error-name=n/a error-message=n/a
Successfully acquired requested service name.
Resolved address 168.61.215.74:123 for 0.debian.pool.ntp.org.
Resolved address 69.89.207.199:123 for 0.debian.pool.ntp.org.
Resolved address 73.239.136.185:123 for 0.debian.pool.ntp.org.
Resolved address 45.55.58.103:123 for 0.debian.pool.ntp.org.
Selected address 168.61.215.74:123 of server 0.debian.pool.ntp.org.
Connecting to time server 168.61.215.74:123 (0.debian.pool.ntp.org).
Sent NTP request to 168.61.215.74:123 (0.debian.pool.ntp.org).
NTP response:
  leap         : 0
  version      : 3
  mode         : 4
  stratum      : 3
  precision    : 0.000000 sec (-23)
  root distance: 0.025322 sec
  reference    : n/a
  origin       : 1669861973.494
  receive      : 1669861973.578
  transmit     : 1669861973.578
  dest         : 1669861973.522
  offset       : +0.070 sec
  delay        : +0.028 sec
  packet count : 1
  jitter       : 0.000
  poll interval: 32
  adjust (slew): +0.070 sec
Failed to call clock_adjtime(): Operation not permitted

This shows why it is failing. If I run it as root using sudo, I don't get that error and it appears to be working:

@openhab:~ $ sudo SYSTEMD_LOG_LEVEL=debug /lib/systemd/systemd-timesyncd
[sudo] password for openhabian: 
Failed to get link NTP servers: No data available
Bus bus-api-timesync: changing state UNSET → OPENING
Bus bus-api-timesync: changing state OPENING → AUTHENTICATING
Added new server 0.debian.pool.ntp.org.
Added new server 1.debian.pool.ntp.org.
Added new server 2.debian.pool.ntp.org.
Added new server 3.debian.pool.ntp.org.
systemd-timesyncd running as pid 7506
Selected server 0.debian.pool.ntp.org.
Resolving 0.debian.pool.ntp.org...
Bus bus-api-timesync: changing state AUTHENTICATING → HELLO
Sent message type=method_call sender=n/a destination=org.freedesktop.DBus path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=Hello cookie=1 reply_cookie=0 signature=n/a error-name=n/a error-message=n/a
Sent message type=method_call sender=n/a destination=org.freedesktop.DBus path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=RequestName cookie=2 reply_cookie=0 signature=su error-name=n/a error-message=n/a
Got message type=method_return sender=org.freedesktop.DBus destination=:1.1802 path=n/a interface=n/a member=n/a cookie=1 reply_cookie=1 signature=s error-name=n/a error-message=n/a
Bus bus-api-timesync: changing state HELLO → RUNNING
Got message type=signal sender=org.freedesktop.DBus.Local destination=n/a path=/org/freedesktop/DBus/Local interface=org.freedesktop.DBus.Local member=Connected cookie=4294967295 reply_cookie=0 signature=n/a error-name=n/a error-message=n/a
Got message type=signal sender=org.freedesktop.DBus destination=:1.1802 path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=NameAcquired cookie=2 reply_cookie=0 signature=s error-name=n/a error-message=n/a
Got message type=signal sender=org.freedesktop.DBus destination=:1.1802 path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=NameAcquired cookie=3 reply_cookie=0 signature=s error-name=n/a error-message=n/a
Got message type=method_return sender=org.freedesktop.DBus destination=:1.1802 path=n/a interface=n/a member=n/a cookie=4 reply_cookie=2 signature=u error-name=n/a error-message=n/a
Successfully acquired requested service name.
Resolved address 91.236.251.14:123 for 0.debian.pool.ntp.org.
Resolved address 212.26.18.43:123 for 0.debian.pool.ntp.org.
Resolved address 196.10.52.58:123 for 0.debian.pool.ntp.org.
Resolved address 91.209.0.17:123 for 0.debian.pool.ntp.org.
Selected address 91.236.251.14:123 of server 0.debian.pool.ntp.org.
Connecting to time server 91.236.251.14:123 (0.debian.pool.ntp.org).
Sent NTP request to 91.236.251.14:123 (0.debian.pool.ntp.org).
NTP response:
  leap         : 0
  version      : 4
  mode         : 4
  stratum      : 2
  precision    : 0.000000 sec (-22)
  root distance: 0.036522 sec
  reference    : n/a
  origin       : 1670079231.584
  receive      : 1670079230.147
  transmit     : 1670079230.147
  dest         : 1670079231.735
  offset       : -1.513 sec
  delay        : +0.151 sec
  packet count : 1
  jitter       : 0.000
  poll interval: 32
  adjust (jump): -1.513 sec
  status       : 8192 sync
  time now     : 1670079230.223
  constant     : 2
  offset       : +0.000 sec
  freq offset  : +243704 (+3 ppm)
interval/delta/delay/jitter/drift 32s/-1.513s/0.151s/0.000s/+3ppm
Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/timesync1 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=3 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Synchronized to time server for the first time 91.236.251.14:123 (0.debian.pool.ntp.org).
Sent NTP request to 91.236.251.14:123 (0.debian.pool.ntp.org).
NTP response:
  leap         : 0
  version      : 4
  mode         : 4
  stratum      : 2
  precision    : 0.000000 sec (-22)
  root distance: 0.037010 sec
  reference    : n/a
  origin       : 1670079262.294
  receive      : 1670079262.371
  transmit     : 1670079262.371
  dest         : 1670079262.448
  offset       : -0.000 sec
  delay        : +0.154 sec
  packet count : 2
  jitter       : 0.000
  poll interval: 64
  adjust (slew): -0.000 sec
  status       : 8193 sync
  time now     : 1670079262.448
  constant     : 2
  offset       : -0.000 sec
  freq offset  : +243704 (+3 ppm)
interval/delta/delay/jitter/drift 64s/-0.000s/0.154s/0.000s/+3ppm
Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/timesync1 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=4 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Sent NTP request to 91.236.251.14:123 (0.debian.pool.ntp.org).
NTP response:
  leap         : 0
  version      : 4
  mode         : 4
  stratum      : 2
  precision    : 0.000000 sec (-22)
  root distance: 0.037971 sec
  reference    : n/a
  origin       : 1670079326.544
  receive      : 1670079326.621
  transmit     : 1670079326.621
  dest         : 1670079326.699
  offset       : -0.000 sec
  delay        : +0.154 sec
  packet count : 3
  jitter       : 0.000
  poll interval: 128
  adjust (slew): -0.000 sec
  status       : 8193 sync
  time now     : 1670079326.699
  constant     : 3
  offset       : -0.000 sec
  freq offset  : +119863 (+1 ppm)
interval/delta/delay/jitter/drift 128s/-0.000s/0.154s/0.000s/+1ppm
Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/timesync1 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=5 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a

So I think I need to somehow give the system user systemd-timesync the right permission, but I have no idea how to do that. Here are the users: root:x:0:0:root:/root:/bin/bash daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin bin:x:2:2:bin:/bin:/usr/sbin/nologin sys:x:3:3:sys:/dev:/usr/sbin/nologin sync:x:4:65534:sync:/bin:/bin/sync games:x:5:60:games:/usr/games:/usr/sbin/nologin man:x:6:12:man:/var/cache/man:/usr/sbin/nologin lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin mail:x:8:8:mail:/var/mail:/usr/sbin/nologin news:x:9:9:news:/var/spool/news:/usr/sbin/nologin uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin proxy:x:13:13:proxy:/bin:/usr/sbin/nologin www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin backup:x:34:34:backup:/var/backups:/bin/bash list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin systemd-timesync:x:100:102:systemd Time Synchronization,,,:/run/systemd:/usr/sbin/nologin systemd-network:x:101:103:systemd Network Management,,,:/run/systemd:/usr/sbin/nologin systemd-resolve:x:102:104:systemd Resolver,,,:/run/systemd:/usr/sbin/nologin _apt:x:103:65534::/nonexistent:/usr/sbin/nologin messagebus:x:104:110::/nonexistent:/usr/sbin/nologin _rpc:x:105:65534::/run/rpcbind:/usr/sbin/nologin statd:x:106:65534::/var/lib/nfs:/usr/sbin/nologin sshd:x:107:65534::/run/sshd:/usr/sbin/nologin avahi:x:108:113:Avahi mDNS daemon,,,:/var/run/avahi-daemon:/usr/sbin/nologin systemd-coredump:x:996:996:systemd Core Dumper:/:/usr/sbin/nologin openhabian:x:1000:115:,,,:/home/openhabian:/bin/bash avahi-autoipd:x:109:114:Avahi autoip daemon,,,:/var/lib/avahi-autoipd:/usr/sbin/nologin openhab:x:110:115:openhab2 runtime user,,,:/var/lib/openhab:/bin/false dnsmasq:x:111:65534:dnsmasq,,,:/var/lib/misc:/usr/sbin/nologin mosquitto:x:112:118::/var/lib/mosquitto:/usr/sbin/nologin frontail:x:1001:1001::/var/tmp:/bin/bash Debian-exim:x:113:119::/var/spool/exim4:/usr/sbin/nologin