Sample Header Ad - 728x90

What's the expected overhead of a passthrough device mapper?

3 votes
0 answers
152 views
I'm trying to establish a baseline throughput overhead for a passthrough device mapper; i.e. a device mapper that does nothing. Roughly following benchmarking procedures [from Cloudflare](https://blog.cloudflare.com/speeding-up-linux-disk-encryption) , I'm measuring roughly **30% throughput decrease** from using a passthrough device mapper over ramdisk with fio, as opposed to straight I/Os to ramdisk, running on Azure VMs, GCP VMs, and raw metal machines, with both Ubuntu 20.04 LTS and 22.04 LTS. **Is this expected?** I'm getting roughly 1000+MB/s over ramdisk across devices and 600+MB/s for passthrough. Here's my setup for those of you who would like to replicate my results: 1. Create a Ubuntu 20.04 or 22.04 VM or get access to such a machine. Turn off secure boot so you can load kernel modules. 2. Create 4GB of ramdisk: sudo modprobe brd rd_nr=1 rd_size=4194304 3. Install fio: sudo apt install -y fio 4. Run fio over ramdisk: sudo fio --filename=/dev/ram0 --readwrite=readwrite --bs=4k --direct=1 --loops=20 --name=plain 5. Record the read/write throughput at the bottom of the output: aggrb=?MB/s. This is the baseline. Now for the passthrough device mapper, using the "delay" device mapper. This is what was done by [Cloudflare](https://www.usenix.org/sites/default/files/conference/protected-files/vault20_slides_korchagin.pdf) : 1. Set up the passthrough device mapper: echo '0 8388608 delay /dev/ram0 0 0' | sudo dmsetup create plain 2. Run fio over it: sudo fio --filename=/dev/mapper/plain --readwrite=readwrite --bs=4k --direct=1 --loops=20 --name=plain 3. Record the throughput similarly. Alternatively, if you suspect the "delay" device mapper with 0 delay is not performant, you can use my implementation of a passthrough device [here](https://github.com/davidchuyaya/rollbaccine/blob/main/src/passthrough/passthrough.c) . Download both files [here](https://github.com/davidchuyaya/rollbaccine/tree/main/src/passthrough) , then compile and load the kernel module: 1. Run make 2. Load the module: sudo insmod passthrough.ko 3. Load the device mapper: `echo "0 sudo blockdev --getsz /dev/ram0 passthrough /dev/ram0" | sudo dmsetup create passthrough` 4. Run fio over it: sudo fio --filename=/dev/mapper/passthrough --readwrite=readwrite --bs=4k --direct=1 --loops=20 --name=passthrough 5. Record the throughput similarly.
Asked by davidchuyaya (81 rep)
Mar 13, 2024, 08:30 PM
Last activity: Mar 18, 2024, 09:18 AM