I'm trying to boot a QEMU VM from a
vhost-user-blk-pci
device, which appears to be generally possible (https://github.com/spdk/spdk/issues/1728) . In my case, vhost gets the image via SPDK's NVMe-oF driver. However, QEMU does not find a bootable device. What I am doing:
1. Start vhost
bin/vhost -S /var/tmp -s 1024 -m 0x3 -A 0000:82:00.1
2. Connect to NVMe-oF server and create blk controller
./rpc.py bdev_nvme_attach_controller -t tcp -a 10.0.0.4 -s 4420 -f ipv4 -n nqn.2024-10.placeholder:bd --name placeholder
./rpc.py vhost_create_blk_controller --cpumask 0x1 vhost.0 placeholdern1
3. Attempt to launch QEMU with blk controller as boot device (does not find anything bootable)
taskset -c 2,3 qemu-system-x86_64 \
-enable-kvm \
-m 1G \
-smp 8 \
-nographic \
-object memory-backend-file,id=mem0,size=1G,mem-path=/dev/hugepages,share=on \
-numa node,memdev=mem0 \
-chardev socket,id=spdk_vhost_blk0,path=/var/tmp/vhost.0,reconnect=1 \
-device vhost-user-blk-pci,chardev=spdk_vhost_blk0,bootindex=1,num-queues=2
Things I've checked:
* I can mount an NMVe-oF disk to the VM just fine using the same sequence of commands (giving QEMU an additional bootable drive) (just booting from it won't work)
* the image on the NVMe-oF server boots just fine if I provide it locally (via the host-kernel NVMe-oF driver that I can't use in production) and declare it in the QEMU options as a drive
* QEMU does not appear to have an NVMe-oF driver itself that I could use instead (it does have an NVMe driver)
QEMU version 7.2.15 (Debian 1:7.2+dfsg-7+deb12u12)
SPDK version SPDK v25.01-pre git sha1 8d960f1d8
Asked by Slow
(1 rep)
Aug 4, 2025, 10:39 AM
Last activity: Aug 5, 2025, 12:05 PM
Last activity: Aug 5, 2025, 12:05 PM