How to set `iptables` so NTP works while internet access is blocked

I want to configure iptables such that it blocks everything except 1. Date time synchronization over the internet using NTP and 2. Access from machines in the LAN. I wrote the following script:

# Reset firewall:
iptables -F

# Allow NTP so the hour syncs over the internet:
iptables -A OUTPUT -p udp --dport 123 -j ACCEPT
iptables -A INPUT -p udp --sport 123 -j ACCEPT

# Allow LAN:
iptables -A INPUT -s $NETWORK_ADDRESS/$MASK -j ACCEPT
iptables -A OUTPUT -d $NETWORK_ADDRESS/$MASK -j ACCEPT

# Block all the rest:
iptables -A INPUT -j DROP
iptables -A OUTPUT -j DROP

following [this answer](https://superuser.com/a/141795/1210408) . After the script, I run sudo date -s "2 OCT 2006 18:00:00" && sudo service ntp stop && sudo service ntp start && date && watch -n 1 date. However, the date and time never synchronize until I do iptables -F. What am I doing wrong?