DHCP lease renewal fails on AWS lightsail server
0
votes
2
answers
754
views
Tuesday the 12th of September 2021 out of the blue a root server of mine running debian hosted in AWS Lightsail crashed. After a reboot it would run fine for approximately 20-60 minutes, after that it would crash again. As it turned out the server was still running but it had lost all network connections. Since AWS Lightsail does not have a serial console and only supports SSH for server administration this means the only method of regaining access to it after a crash was a reboot.
An analysis of the syslog and other logs didn't reveal any relevant clues. Also no relevant packages were installed/removed in the last days.
Because I already had issues with dhcp in the past on that server I suspected that the issue might be related to the dhcp lease renewal. So I did run some diagnostic with dhclient and it turned out that dhcp was indeed the issue that caused the server to crash. The AWS dhcp server gives very short lease times of ~20-60 minutes and as soon as the lease runs out the server will loose all networking.
I was able to create a workaround by running
dhclient -d
in an endless loop:
#!/bin/bash
while true; do
bash -c 'dhclient -d 2>&1 | tee -a /var/log/dhclient.log' &
sleep 60
kill $(jobs -p) 2>/dev/null || true
kill -9 $(jobs -p) 2>/dev/null || true
done
I just run that script in a screen session and just let it run. This successfully kept the server from crashing. However this is just a dirty workaround and the real reasons for this issue still hadn't been found.
Asked by Gellweiler
(153 rep)
Oct 16, 2021, 09:16 AM
Last activity: Jun 7, 2022, 06:51 PM
Last activity: Jun 7, 2022, 06:51 PM