Based on https://unix.stackexchange.com/questions/784690/limit-file-io-speed I have built https://git.data.coop/tange/tangetools/src/branch/master/iothrottle
It works for some programs:
iothrottle -i 10M cat foo > bar # Yes, here you could just use pv
iothrottle -o 1M ffmpeg -i foo.mp3 foo.wav
iothrottle -o 1M cp -a foodir /other/fs/foodir
This does not:
iothrottle -o 1M cp -a foodir /same/fs/foodir
because on the same file system
cp
uses a single call to copy_file_range
per file.
I am OK with that limitation.
It also does not work for seq
. When I run strace seq 100000
I get:
~~~
write(1, "1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14"..., 8192) = 8192
write(1, "\n1861\n1862\n1863\n1864\n1865\n1866\n1"..., 4096) = 4096
write(1, "2680\n2681\n2682\n2683\n2684\n2685\n26"..., 4096) = 4096
write(1, "499\n3500\n3501\n3502\n3503\n3504\n350"..., 4096) = 4096
write(1, "18\n4319\n4320\n4321\n4322\n4323\n4324"..., 4096) = 4096
write(1, "7\n5138\n5139\n5140\n5141\n5142\n5143\n"..., 4096) = 4096
write(1, "\n5957\n5958\n5959\n5960\n5961\n5962\n5"..., 4096) = 4096
write(1, "6776\n6777\n6778\n6779\n6780\n6781\n67"..., 4096) = 4096
write(1, "595\n7596\n7597\n7598\n7599\n7600\n760"..., 4096) = 4096
write(1, "14\n8415\n8416\n8417\n8418\n8419\n8420"..., 4096) = 4096
write(1, "3\n9234\n9235\n9236\n9237\n9238\n9239\n"..., 3838) = 3838
~~~
So it seems to call write
(and not some mmap
or other magic), and thus ought to work.
But when I run: IOTHROTTLE_DEBUG=1 iothrottle -o 1M seq 10000 >/dev/null
I get:
~~~
init called: default read_limit=0, write_limit=0
IOTHROTTLE_READ=0
IOTHROTTLE_WRITE=1048576
Final read_limit=0, write_limit=1048576
allowed: 608 written: 0
allowed: 617 written: 0
~~~
This tells me that iothrottle
only intercepts 2 calls to write
.
What am I missing?
Asked by Ole Tange
(37348 rep)
Dec 29, 2024, 07:53 PM
Last activity: Dec 30, 2024, 07:42 AM
Last activity: Dec 30, 2024, 07:42 AM