Sample Header Ad - 728x90

Not all XEN VMs auto start after reboot of DOM 0

1 vote
4 answers
1502 views
On my Xen host (systemd in Debian Buster, Xen 4.11), I usually start all guest VMs automatically. I added all my VM configs as link in /etc/xen/auto/ to the underlying config like /etc/xen/auto/proxy.cfg -> /etc/xen/proxy.cfg /etc/xen/auto/vm04.matrix.cfg -> /etc/xen/vm04.matrix.cfg ... and sometimes some guest VMs start automatically on reboot of the DOM0 host and sometimes some fail and throw these errors in syslog: $ grep "Starting Xen domain" /var/log/syslog May 12 08:38:36 dom0 xendomains: Starting Xen domain proxy (from /etc/xen/auto/proxy.cfg)...done. May 12 08:38:36 dom0 xendomains: Starting Xen domain vm04.matrix.de (from /etc/xen/auto/vm04.matrix.cfg)...failed. If they fail, the errors in \var\log\syslog around the fail during boot are: Starting Xen domain vm04.matrix (from /etc/xen/auto/vm04.matrix.cfg)...failed. libxl: error: libxl_device.c:417:libxl__device_disk_set_backend: Disk vdev=xvda2 failed to stat: /dev/vg0/vm04.matrix-disk: No such file or directory libxl: error: libxl_create.c:983:initiate_domain_create: Domain 4:Unable to set disk defaults for disk 0 libxl: error: libxl_domain.c:1034:libxl__destroy_domid: Domain 4:Non-existant domain libxl: error: libxl_domain.c:993:domain_destroy_callback: Domain 4:Unable to destroy guest libxl: error: libxl_domain.c:920:domain_destroy_cb: Domain 4:Destruction of domain failed --- Starting Xen domain proxy (from /etc/xen/auto/proxy.cfg)...failed. libxl: error: libxl_device.c:417:libxl__device_disk_set_backend: Disk vdev=xvda2 failed to stat: /dev/vg0/proxy-disk: No such file or directory libxl: error: libxl_create.c:983:initiate_domain_create: Domain 1:Unable to set disk defaults for disk 0 libxl: error: libxl_domain.c:1034:libxl__destroy_domid: Domain 1:Non-existant domain libxl: error: libxl_domain.c:993:domain_destroy_callback: Domain 1:Unable to destroy guest libxl: error: libxl_domain.c:920:domain_destroy_cb: Domain 1:Destruction of domain failed The files /etc/xen/proxy.cfg and /etc/xen/vm04.matrix.cfg are identical apart from the mount-points. If I start xen create /etc/xen/vm04.matrix.cfg later manually it starts just fine, only not automatically during the boot of the host. ## debugging To debug the problem, I added this in /etc/default/grub GRUB_CMDLINE_XEN_DEFAULT="dom0_mem=4096M,max:4096M loglvl=all guest_loglvl=all" and called update-grub, but after a reboot the error messages in syslog are the same. I cannot find anything suspicious in dmesg either. No mentions of any of the guest hostnames. Only this shows, the problem: # dmesg|grep dev|tail [ 11.348931] xen:xen_evtchn: Event-channel device installed [ 22.006020] device vif3.0 entered promiscuous mode [ 25.362878] device vif5.0 entered promiscuous mode [ 28.636912] device vif6.0 entered promiscuous mode [ 1004.501654] device vif7.0 entered promiscuous mode ... You only see, that the first three VMs Network-interfaces did start during boot this time and 1000s later I started the next VM manually on the console. Xen loads just fine: $ systemctl --type=service | grep -i xen xen.service loaded active running LSB: Xen daemons xendomains.service loaded active running LSB: Start/stop secondary xen domains I think the best hint to the problem is that the volume is not there during boot: /dev/vg0/proxy-disk: No such file or directory maybe xen starts too early in the boot process?
systemctl cat xen.service|uniq
# /run/systemd/generator.late/xen.service
# Automatically generated by systemd-sysv-generator

[Unit]
Documentation=man:systemd-sysv-generator(8)
SourcePath=/etc/init.d/xen
Description=LSB: Xen daemons
Before=multi-user.target
Before=graphical.target
After=remote-fs.target

[Service]
Type=forking
Restart=no
TimeoutSec=5min
IgnoreSIGPIPE=no
KillMode=process
GuessMainPID=no
RemainAfterExit=yes
SuccessExitStatus=5 6
ExecStart=/etc/init.d/xen start
ExecStop=/etc/init.d/xen stop
systemctl cat xendomains.service|uniq
# /run/systemd/generator.late/xendomains.service
# Automatically generated by systemd-sysv-generator

[Unit]
Documentation=man:systemd-sysv-generator(8)
SourcePath=/etc/init.d/xendomains
Description=LSB: Start/stop secondary xen domains
Before=multi-user.target
Before=graphical.target
Before=corosync.service
Before=heartbeat.service
Before=libvirtd.service
After=network-online.target
After=remote-fs.target
After=xen.service
After=drbd.service
After=iscsi.service
After=openvswitch-switch.service
After=nfs-kernel-server.service
Wants=network-online.target

[Service]
Type=forking
Restart=no
TimeoutSec=5min
IgnoreSIGPIPE=no
KillMode=process
GuessMainPID=no
RemainAfterExit=yes
SuccessExitStatus=5 6
ExecStart=/etc/init.d/xendomains start
ExecStop=/etc/init.d/xendomains stop
ExecReload=/etc/init.d/xendomains reload
How can I further debug this?
Asked by rubo77 (30435 rep)
May 12, 2020, 01:06 PM
Last activity: Jun 12, 2024, 03:23 PM