dd: writing to Invalid argument with 4k blocksize on iSCSI LUN

Why can't use oflag=direct on the 4k blocksize iSCSI disk? **Could other applications also have a problem with the cause?** root@testvm02:~# dd if=/root/speedtest of=/mnt/8k_512/speedtest oflag=direct 51200+0 records in 51200+0 records out 26214400 bytes (26 MB, 25 MiB) copied, 59,4312 s, 441 kB/s root@testvm02:~# dd if=/root/speedtest of=/mnt/8k_4k/speedtest oflag=direct dd: writing to '/mnt/8k_4k/speedtest': Invalid argument 1+0 records in 0+0 records out 0 bytes copied, 0,000790648 s, 0,0 kB/s root@testvm02:~# dd if=/root/speedtest of=/mnt/4k_4k/speedtest oflag=direct dd: writing to '/mnt/4k_4k/speedtest': Invalid argument 1+0 records in 0+0 records out 0 bytes copied, 0.000139662 s, 0.0 kB/s my environment @storage server: /usr/sbin/zfs create -s -V 50GiB STORAGE01/4af1a9b7-0592-4707-a875-986d91fceac4 #<-- / & vda on testvm02 /usr/sbin/tgtadm --lld iscsi --op new --mode logicalunit --tid 19 --lun 1 -b /dev/zvol/STORAGE01/4af1a9b7-0592-4707-a875-986d91fceac4 # /usr/sbin/zfs create -s -V 50GiB STORAGE01/209afd7c-bdd6-4125-a899-b98758fcc6c0 #<-- /mnt/8k-512 & vdb on testvm02 /usr/sbin/tgtadm --lld iscsi --op new --mode logicalunit --tid 19 --lun 2 -b /dev/zvol/STORAGE01/209afd7c-bdd6-4125-a899-b98758fcc6c0 # /usr/sbin/zfs create -s -V 50GiB STORAGE01/6ae21fa3-df76-4843-ab65-0700af4f04f7 #<-- /mnt/8k-4k & vdc on testvm02 /usr/sbin/tgtadm --lld iscsi --op new --mode logicalunit --tid 19 --blocksize 4096 --lun 3 -b /dev/zvol/STORAGE01/6ae21fa3-df76-4843-ab65-0700af4f04f7 # /usr/sbin/zfs create -s -o volblocksize=4k -V 50GiB STORAGE01/24d38989-b47b-4e3c-b5ea-5d9a30d611f6 #<-- /mnt/4k-4k & vdd on testvm02 /usr/sbin/tgtadm --lld iscsi --op new --mode logicalunit --tid 19 --blocksize 4096 --lun 4 -b /dev/zvol/STORAGE01/24d38989-b47b-4e3c-b5ea-5d9a30d611f6 # # default zvol blocksize = 8k, default iSCSI LUN blocksize = 512 VM: virt-install --name 'testvm02.domain.de' --description 'desc' --os-type 'Linux' --os-variant 'debian9' --ram 2048 --vcpus 2 --cdrom '/var/lib/libvirt/boot/firmware-10.9.0-amd64-netinst.iso' --graphics vnc,password=foobar --network 'bridge:br540,model=virtio,virtualport_type=openvswitch' --disk 'vol=c0dcc42e-7805-4ab3-845f-363bdedada5b/unit:0:0:1,logical_block_size=512,physical_block_size=512' --disk 'vol=c0dcc42e-7805-4ab3-845f-363bdedada5b/unit:0:0:2,logical_block_size=512,physical_block_size=512' --disk 'vol=c0dcc42e-7805-4ab3-845f-363bdedada5b/unit:0:0:3,logical_block_size=4096,physical_block_size=4096' --disk 'vol=c0dcc42e-7805-4ab3-845f-363bdedada5b/unit:0:0:4,logical_block_size=4096,physical_block_size=4096' all disks formatted with ext4 root@testvm02:~# fdisk -l Disk /dev/vda: 50 GiB, 53687091200 bytes, 104857600 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x50172cb2 Device Boot Start End Sectors Size Id Type /dev/vda1 * 2048 999423 997376 487M 83 Linux /dev/vda2 999424 104855551 103856128 49,5G 83 Linux Disk /dev/vdb: 50 GiB, 53687091200 bytes, 104857600 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x5e831f4b Device Boot Start End Sectors Size Id Type /dev/vdb1 2048 104855551 104853504 50G 83 Linux Disk /dev/vdc: 50 GiB, 53687091200 bytes, 13107200 sectors Units: sectors of 1 * 4096 = 4096 bytes Sector size (logical/physical): 4096 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: dos Disk identifier: 0x791f5094 Device Boot Start End Sectors Size Id Type /dev/vdc1 256 13106943 13106688 50G 83 Linux Disk /dev/vdd: 50 GiB, 53687091200 bytes, 13107200 sectors Units: sectors of 1 * 4096 = 4096 bytes Sector size (logical/physical): 4096 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: dos Disk identifier: 0x6de2b945 Device Boot Start End Sectors Size Id Type /dev/vdd1 256 13106943 13106688 50G 83 Linux