What's the expected overhead of a passthrough device mapper?
3
votes
0
answers
152
views
I'm trying to establish a baseline throughput overhead for a passthrough device mapper; i.e. a device mapper that does nothing. Roughly following benchmarking procedures [from Cloudflare](https://blog.cloudflare.com/speeding-up-linux-disk-encryption) , I'm measuring roughly **30% throughput decrease** from using a passthrough device mapper over ramdisk with fio, as opposed to straight I/Os to ramdisk, running on Azure VMs, GCP VMs, and raw metal machines, with both Ubuntu 20.04 LTS and 22.04 LTS.
**Is this expected?**
I'm getting roughly 1000+MB/s over ramdisk across devices and 600+MB/s for passthrough.
Here's my setup for those of you who would like to replicate my results:
1. Create a Ubuntu 20.04 or 22.04 VM or get access to such a machine. Turn off secure boot so you can load kernel modules.
2. Create 4GB of ramdisk:
sudo modprobe brd rd_nr=1 rd_size=4194304
3. Install fio: sudo apt install -y fio
4. Run fio over ramdisk: sudo fio --filename=/dev/ram0 --readwrite=readwrite --bs=4k --direct=1 --loops=20 --name=plain
5. Record the read/write throughput at the bottom of the output: aggrb=?MB/s
. This is the baseline.
Now for the passthrough device mapper, using the "delay" device mapper. This is what was done by [Cloudflare](https://www.usenix.org/sites/default/files/conference/protected-files/vault20_slides_korchagin.pdf) :
1. Set up the passthrough device mapper: echo '0 8388608 delay /dev/ram0 0 0' | sudo dmsetup create plain
2. Run fio over it: sudo fio --filename=/dev/mapper/plain --readwrite=readwrite --bs=4k --direct=1 --loops=20 --name=plain
3. Record the throughput similarly.
Alternatively, if you suspect the "delay" device mapper with 0 delay is not performant, you can use my implementation of a passthrough device [here](https://github.com/davidchuyaya/rollbaccine/blob/main/src/passthrough/passthrough.c) .
Download both files [here](https://github.com/davidchuyaya/rollbaccine/tree/main/src/passthrough) , then compile and load the kernel module:
1. Run make
2. Load the module: sudo insmod passthrough.ko
3. Load the device mapper: `echo "0
sudo blockdev --getsz /dev/ram0 passthrough /dev/ram0" | sudo dmsetup create passthrough
`
4. Run fio over it: sudo fio --filename=/dev/mapper/passthrough --readwrite=readwrite --bs=4k --direct=1 --loops=20 --name=passthrough
5. Record the throughput similarly.
Asked by davidchuyaya
(81 rep)
Mar 13, 2024, 08:30 PM
Last activity: Mar 18, 2024, 09:18 AM
Last activity: Mar 18, 2024, 09:18 AM