Sample Header Ad - 728x90

Why is this docker container process not triggering a mount for my systemd auto-mounted drive?

1 vote
1 answer
117 views
I've been struggling to make sense of something, so would appreciate some help. I am mounting a remote NFS drive onto my Debian system with the following fstab entry which uses the systemd automounter, and is set to auto-unmount after 120 seconds of inactivity:
192.168.0.67:/mnt/SSD_240GB/backups/TIG_backups  /mnt/nfs/SSD_240GB/backups/TIG_backups   nfs auto,_netdev,bg,soft,x-systemd.automount,x-systemd.idle-timeout=120,timeo=14,nofail,noatime,nolock,tcp,actimeo=1800 0 0
Now on this Debian host system I am running a docker container (Telegraf ), to monitor some metrics of the Debian host. To facilitate this, I am bind-mounting the host filesystem and proc directory (as recommended here in the docs ), as well as bind-mounting the NFS drive. The docker run command looks like this:
docker run -d \
--name telegraf_container \
--user 1001:1001 \
--network docker_monitoring_network \
--mount type=bind,source=/,destination=/hostfs \
--mount type=bind,source=/mnt/nfs/SSD_240GB/backups/TIG_backups/telegraf_backups,destination=/mnt/nfs/SSD_240GB/backups/TIG_backups/telegraf_backups \
-e HOST_MOUNT_PREFIX=/hostfs \
-e HOST_PROC=/hostfs/proc \
telegraf:latest
I am using the Telegraf Disk Input plugin because I want to gather disk usage metrics once every hour for the NFS drive (used, free, total). The problem is that the disk is unmounted automatically 120s after system boot as expected, *but it is never remounted*. I would expect the telegraf container to trigger an automount every hour. The reason I expect this is because the container is essentially running a .go program (as seen here in the source code) which is querying the filesystem. I believe under the hood it is calling some .go libraries (here and here ), which are essentially calling statfs(). I was under the impression that statfs() should trigger a systemd automount. Here in the Debian host's logs, I can see the NFS drive mounting correctly at boot up, and then unmounting after a couple of minutes automatically (but then it never remounts):
root@docker-debian:/home/monitoring/docker_files/scripts# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.automount -b
Jun 05 13:54:12 docker-debian systemd[1] : Set up automount mnt-nfs-SSD_240GB-backups-TIG_backups.automount.
Jun 05 13:54:18 docker-debian systemd[1] : mnt-nfs-SSD_240GB-backups-TIG_backups.automount: Got automount request for /mnt/nfs/SSD_240GB/backups/TIG_backups, triggered by 893 (runc:[2:INIT])

root@docker-debian:/home/monitoring/docker_files/scripts# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.mount -b
Jun 05 13:54:18 docker-debian systemd[1] : Mounting mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups...
Jun 05 13:54:18 docker-debian systemd[1] : Mounted mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups.
Jun 05 13:57:39 docker-debian systemd[1] : Unmounting mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups...
Jun 05 13:57:39 docker-debian systemd[1] : mnt-nfs-SSD_240GB-backups-TIG_backups.mount: Deactivated successfully.
Jun 05 13:57:39 docker-debian systemd[1] : Unmounted mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups.
After the drive has auto-unmounted, it is missing from the host as expected:
monitoring@docker-debian:/$ df
Filesystem     1K-blocks    Used Available Use% Mounted on
udev              983908       0    983908   0% /dev
tmpfs             201420     816    200604   1% /run
/dev/sda1       15421320 4779404   9836748  33% /
tmpfs            1007084       0   1007084   0% /dev/shm
tmpfs               5120       0      5120   0% /run/lock
tmpfs             201416       0    201416   0% /run/user/1001
But it is present in the container:
monitoring@docker-debian:/$ docker exec -it telegraf_container df
Filesystem                                                       1K-blocks     Used Available Use% Mounted on
overlay                                                           15421320  4779404   9836748  33% /
tmpfs                                                                65536        0     65536   0% /dev
shm                                                                  65536        0     65536   0% /dev/shm
/dev/sda1                                                         15421320  4779404   9836748  33% /hostfs
udev                                                                983908        0    983908   0% /hostfs/dev
tmpfs                                                              1007084        0   1007084   0% /hostfs/dev/shm
tmpfs                                                               201420      820    200600   1% /hostfs/run
tmpfs                                                                 5120        0      5120   0% /hostfs/run/lock
192.168.0.67:/mnt/SSD_240GB/backups/TIG_backups/telegraf_backups 229608448 42336256 175535104  20% /mnt/nfs/SSD_240GB/backups/TIG_backups/telegraf_backups
tmpfs                                                              1007084        0   1007084   0% /proc/acpi
tmpfs                                                              1007084        0   1007084   0% /sys/firmware
tmpfs                                                               201416        0    201416   0% /hostfs/run/user/1001
In case it's relevant, the Telegraf config is here:
# GLOBAL SETTINGS
[agent]
  hostname = "docker-debian"
  flush_interval = "60m"
  interval = "60m"

# COLLECT DISK USAGE OF THIS VM
[[inputs.disk]]
  mount_points = ["/", "/mnt/nfs/SSD_240GB/backups/TIG_backups"]  # Only these will be checked
  fieldpass = [ "free", "total", "used", "used_percent" ]
  ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"]

# VIEW COLLECTED METRICS
[[outputs.file]]
  files = ["stdout"]
Why is the container not triggering an automount, which leads to it not being able to collect the metrics on the drive? --- **EDIT** After the answer from @grawity, I did a simpler check: - I removed the idle timeout (by setting x-systemd.idle-timeout=0) - I removed explicit bind-mounts for these drives from the docker run command In this situation, I found the following: 1) Immediately after boot, an automount is set up, but nothing triggered it yet, as expected:
root@docker-debian:# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.automount -b
Jun 06 12:22:20 docker-debian systemd[1] : Set up automount mnt-nfs-SSD_240GB-backups-TIG_backups.automount.

root@docker-debian:# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.mount -b
-- No entries --
2) I start a simple container up, with no explicit bind mounts for those drives (only the hostfs structure) :
docker run -d \
--name telegraf_container \
--mount type=bind,source=/,destination=/hostfs \
-e HOST_MOUNT_PREFIX=/hostfs \
-e HOST_PROC=/hostfs/proc \
telegraf:latest
This still does not trigger any automounts on the host. 3) Now I manually trigger an automount on the host by accessing the drive:
ls /mnt/nfs/SSD_240GB/backups/TIG_backups/
The automount is triggered and mounts the drive successfully:
root@docker-debian:# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.automount -b
Jun 06 12:22:20 docker-debian systemd[1] : Set up automount mnt-nfs-SSD_240GB-backups-TIG_backups.automount.
Jun 06 12:35:20 docker-debian systemd[1] : mnt-nfs-SSD_240GB-backups-TIG_backups.automount: Got automount request for /mnt/nfs/SSD_240GB/backups/TIG_backups, triggered by 936 (ls)

root@docker-debian:# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.mount -b
Jun 06 12:35:21 docker-debian systemd[1] : Mounting mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups...
Jun 06 12:35:21 docker-debian systemd[1] : Mounted mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups.
Interestingly, the mounted drive now *automatically* appears inside the container (even though no bind-mounts have been used), but it appears under /hostfs instead:
monitoring@docker-debian:~$ docker exec -it telegraf_container df
Filesystem                                     1K-blocks    Used Available Use% Mounted on
overlay                                         15421320 4686888   9929264  33% /
tmpfs                                              65536       0     65536   0% /dev
shm                                                65536       0     65536   0% /dev/shm
/dev/sda1                                       15421320 4686888   9929264  33% /hostfs
udev                                              983908       0    983908   0% /hostfs/dev
tmpfs                                            1007084       0   1007084   0% /hostfs/dev/shm
tmpfs                                             201420     656    200764   1% /hostfs/run
tmpfs                                               5120       0      5120   0% /hostfs/run/lock
tmpfs                                             201416       0    201416   0% /hostfs/run/user/1001
tmpfs                                            1007084       0   1007084   0% /proc/acpi
tmpfs                                            1007084       0   1007084   0% /sys/firmware
192.168.0.67:/mnt/SSD_240GB/backups/TIG_backups  16337920 5799936   9682944  38% /hostfs/mnt/nfs/SSD_240GB/backups/TIG_backups
If I unmount the drive directly on the host (using umount), then it disappears from the container again. 4) I repeated this but instead using an idle timeout of 2mins now. What I found was that having the docker container running *prevents* the autounmount after 2 mins from happening (even though the container does not explicitly bind-mount in the drive, but instead appears automatically in the container under /hostfs). If I stop and remove the container, then the idle timeout unmounts the drive after the 2mins:
root@docker-debian:# journalctl -u mnt-nfs-SSD_240GB-backups-TIG_backups.mount -b
Jun 06 12:49:40 docker-debian systemd[1] : Mounting mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups...
Jun 06 12:49:41 docker-debian systemd[1] : Mounted mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups.
Jun 06 13:10:28 docker-debian systemd[1] : Unmounting mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups...
Jun 06 13:10:28 docker-debian systemd[1] : mnt-nfs-SSD_240GB-backups-TIG_backups.mount: Deactivated successfully.
Jun 06 13:10:28 docker-debian systemd[1] : Unmounted mnt-nfs-SSD_240GB-backups-TIG_backups.mount - /mnt/nfs/SSD_240GB/backups/TIG_backups.
This makes me think a couple of things: - If I want to use telegraf to monitor drives that are mounted on the host, I don't need to bind mount them in explicitly, because they are present already due to the /hostfs bind-mount. - I should never see what I was originally expecting - namely, a drive automatically unmounting due to the idle timeout, and then the container triggering a remount. Because I observed above that once a drive has been mounted in (in my case at /hostfs), the container actually prevents it from ever being auto-unmounted.
Asked by teeeeee (305 rep)
Jun 5, 2025, 03:04 PM
Last activity: Jun 6, 2025, 01:08 PM