Sample Header Ad - 728x90

Unix & Linux Stack Exchange

Q&A for users of Linux, FreeBSD and other Unix-like operating systems

Latest Questions

2 votes
3 answers
629 views
How to securely export a device (HDD)?
So I have a Scientific Linux 6.3 (RHEL clone so basically the question is Redhat related) machine called "B" (with an extra "A" HDD besides the system HDD) and a notebook with SL 6.3. They are in a /24 IPv4 subnet, and can fully reach each other. **Q**: How can I export the "A" HDD device to the not...
So I have a Scientific Linux 6.3 (RHEL clone so basically the question is Redhat related) machine called "B" (with an extra "A" HDD besides the system HDD) and a notebook with SL 6.3. They are in a /24 IPv4 subnet, and can fully reach each other. **Q**: How can I export the "A" HDD device to the notebook, so that on the notebook I could see the "A" HDD as a device /HDD/? (, and locally encrypt it using LUKS - I know this last encrypting part) The important thing is that I need the connection to be secured (SSL?) so that no one can intercept the data that I encrypt on the notebook. **OR**: is it already encrypted via LUKS? (and an SSL connection between the notebook and the "B" machine would be just an overhead?) - extra: I also need that the "exporting of the device" must be routable over network. **ps.: so the main question is: does encrypted communication needed between the notebook and the "B" machine or are ALL the datas on the HDD already encrypted when leaving the notebook (even LUKS pwd too??)**
gasko peter (5634 rep)
Sep 17, 2012, 09:00 AM • Last activity: Jul 13, 2025, 04:56 PM
0 votes
0 answers
34 views
Target iSCSI sometime drop connection
I'v got 2 Linux servers, Proxima and Lisa. Proxima is as iSCSI-target, Lisa as iSCSI-initiator connects to Proxima. An initial connection is successfully done. But while data transfering Proxima sometimes drops connection. **Proxima** \ Debian-11 +Proxmox, kernel 5.15.149-1-pve x86_64 \ iSCSI: targe...
I'v got 2 Linux servers, Proxima and Lisa. Proxima is as iSCSI-target, Lisa as iSCSI-initiator connects to Proxima. An initial connection is successfully done. But while data transfering Proxima sometimes drops connection. **Proxima** \ Debian-11 +Proxmox, kernel 5.15.149-1-pve x86_64 \ iSCSI: targetcli-fb-2.1.53 \ IP: aaa.bbb.ccc.21 \ System Log:
07:36:26.079 kernel:: [6942463.448080] Unable to recover from DataOut timeout while in ERL=0, closing iSCSI connection for I_T Nexus iqn.2015-07.cz.domain.lisa:client,i,0x00023d010000,iqn.2024-07.cz.domain.proxima:backup,t,0x01
07:36:26.494 kernel: Unable to recover from DataOut timeout while in ERL=0, closing iSCSI connection for I_T Nexus iqn.2015-07.cz.domain.lisa:client,i,0x00023d010000,iqn.2024-07.cz.domain.proxima:backup,t,0x01
**Lisa** \ Debian-7, kernel 3.16.0-7-amd64 x86_64 \ iSCSI: open-iscsi-2.0.873 \ IP: aaa.bbb.ccc.20 \ System Log:
07:36:26.089 kernel:: connection1:0: detected conn error (1020)
07:36:27.167 iscsid: Kernel reported iSCSI connection 1:0 error (1020 - ISCSI_ERR_TCP_CONN_CLOSE: TCP connection closed) state (3)
07:36:29.167 iscsid: connection1:0 is operational after recovery (1 attempts)
**tcpdump**
:36:26.065241 IP aaa.bbb.ccc.20.36751 > aaa.bbb.ccc.21.3260: Flags [.], seq 120408213:120409661, ack 214825, win 7568, length 1448
07:36:26.065254 IP aaa.bbb.ccc.20.36751 > aaa.bbb.ccc.21.3260: Flags [.], seq 120409661:120411109, ack 214825, win 7568, length 1448
07:36:26.065266 IP aaa.bbb.ccc.20.36751 > aaa.bbb.ccc.21.3260: Flags [.], seq 120411109:120412557, ack 214825, win 7568, length 1448
07:36:26.065278 IP aaa.bbb.ccc.20.36751 > aaa.bbb.ccc.21.3260: Flags [.], seq 120412557:120414005, ack 214825, win 7568, length 1448
07:36:26.065290 IP aaa.bbb.ccc.20.36751 > aaa.bbb.ccc.21.3260: Flags [.], seq 120414005:120415453, ack 214825, win 7568, length 1448
07:36:26.065303 IP aaa.bbb.ccc.20.36751 > aaa.bbb.ccc.21.3260: Flags [.], seq 120415453:120416901, ack 214825, win 7568, length 1448
07:36:26.088573 IP aaa.bbb.ccc.21.3260 > aaa.bbb.ccc.20.36751: Flags [R.], seq 214825, ack 120416901, win 24570, length 0
07:36:28.592030 IP aaa.bbb.ccc.20.36752 > aaa.bbb.ccc.21.3260: Flags [S], seq 492965647, win 29200, options [mss 1460,sackOK,TS val 1736195842 ecr 0,nop,wscale 2], length 0
07:36:28.592053 IP aaa.bbb.ccc.21.3260 > aaa.bbb.ccc.20.36752: Flags [S.], seq 1204188907, ack 492965648, win 65160, options [mss 1460,sackOK,TS val 4010571466 ecr 1736195842,nop,wscale 7], length 0
07:36:28.592222 IP aaa.bbb.ccc.20.36752 > aaa.bbb.ccc.21.3260: Flags [.], ack 1, win 7300, length 0
07:36:28.842712 IP aaa.bbb.ccc.20.36752 > aaa.bbb.ccc.21.3260: Flags [P.], seq 1:49, ack 1, win 7300, length 48
07:36:28.842769 IP aaa.bbb.ccc.21.3260 > aaa.bbb.ccc.20.36752: Flags [.], ack 49, win 509, length 0
07:36:28.842713 IP aaa.bbb.ccc.20.36752 > aaa.bbb.ccc.21.3260: Flags [P.], seq 49:501, ack 1, win 7300, length 452
---------- ***Addition*** (accordance with comment by waltinator): \ Hardware of both servers seem Ok, all other services are without problem. \ The problem with iSCSI is from beginning. No hardware changes since then. \ Previously iSCSI connection was between server Lisa and server Zion (Debian-6). It was without errors. Server Zion was replaced by server Proxima (Debian-11). I believe the cause is a difference in the iSCSI software versions.
Mysak (49 rep)
Jun 5, 2025, 07:44 PM • Last activity: Jun 8, 2025, 08:50 PM
2 votes
2 answers
2182 views
add new disk in /etc/tgt/targets.conf and reload without affect other disks/initiator hosts
I have added a new disk to the server: [root@ns1 tgt]# lsblk |grep sdh sdh 8:112 0 600M 0 disk Also, I have created a new entry for `/dev/sdh` in `/etc/tgt/targets.conf` [root@ns1 tgt]# cat /etc/tgt/targets.conf |grep /dev/ direct-store /dev/sdb direct-store /dev/sdc direct-store /dev/sdd direct-sto...
I have added a new disk to the server: [root@ns1 tgt]# lsblk |grep sdh sdh 8:112 0 600M 0 disk Also, I have created a new entry for /dev/sdh in /etc/tgt/targets.conf [root@ns1 tgt]# cat /etc/tgt/targets.conf |grep /dev/ direct-store /dev/sdb direct-store /dev/sdc direct-store /dev/sdd direct-store /dev/sde direct-store /dev/sdf direct-store /dev/sdg direct-store /dev/sdh [root@ns1 tgt]# How can I reload the new configuration and make available the new lun? I have tried systemctl reload tgtd, tgt-admin -e and tgt-admin --ready ALL, but none has worked. Bellow we can see there is no /dev/sdh yet. [root@ns1 tgt]# tgtadm --mode target --op show|grep /dev/ Backing store path: /dev/sdb Backing store path: /dev/sdc Backing store path: /dev/sdd Backing store path: /dev/sde Backing store path: /dev/sdf Backing store path: /dev/sdg [root@ns1 tgt]# I have tested with systemctl restart tgtd and it has worked, but it affect the initiator hosts. ie after restart tgtd (log from some initiator host): Jun 6 18:20:41 rac1 kernel: connection1:0: detected conn error (1020) Jun 6 18:20:41 rac1 iscsid: iscsid: Kernel reported iSCSI connection 1:0 error (1020 - ISCSI_ERR_TCP_CONN_CLOSE: TCP connection closed) state (3) Jun 6 18:20:43 rac1 iscsid: iscsid: Kernel reported iSCSI connection 1:0 error (1020 - ISCSI_ERR_TCP_CONN_CLOSE: TCP connection closed) state (2) Jun 6 18:20:43 rac1 kernel: connection1:0: detected conn error (1020) Jun 6 18:20:45 rac1 kernel: connection1:0: detected conn error (1020) Jun 6 18:20:45 rac1 iscsid: iscsid: Kernel reported iSCSI connection 1:0 error (1020 - ISCSI_ERR_TCP_CONN_CLOSE: TCP connection closed) state (2) Jun 6 18:20:46 rac1 kernel: session1: session recovery timed out after 5 secs Jun 6 18:20:47 rac1 multipathd: checker failed path 8:112 in map data1 Jun 6 18:20:47 rac1 multipathd: data1: remaining active paths: 0 Jun 6 18:20:47 rac1 kernel: device-mapper: multipath: Failing path 8:112. Jun 6 18:20:47 rac1 kernel: device-mapper: multipath: Failing path 8:80. Jun 6 18:20:47 rac1 kernel: device-mapper: multipath: Failing path 8:96. Jun 6 18:20:47 rac1 multipathd: checker failed path 8:80 in map fra2 Jun 6 18:20:47 rac1 multipathd: fra2: remaining active paths: 0 Jun 6 18:20:47 rac1 multipathd: checker failed path 8:96 in map fra3 Jun 6 18:20:47 rac1 multipathd: fra3: remaining active paths: 0 Jun 6 18:20:47 rac1 iscsid: iscsid: Kernel reported iSCSI connection 1:0 error (1020 - ISCSI_ERR_TCP_CONN_CLOSE: TCP connection closed) state (2) Jun 6 18:20:47 rac1 kernel: connection1:0: detected conn error (1020) Jun 6 18:20:49 rac1 iscsid: iscsid: connection1:0 is operational after recovery (4 attempts)
Astora (509 rep)
Jun 6, 2021, 10:09 PM • Last activity: Jun 3, 2025, 11:03 AM
2 votes
0 answers
66 views
Automatic cloning and iscsi sharing of zfs dataset on connect
Since iSCSI was not designed to support multiple concurrent clients accessing the same dataset, is it technically possible without modifying code/recompiling (but maybe create scripts and systemd sockets?) to snapshot and clone a dataset for each client, presenting it to the requesting client, and d...
Since iSCSI was not designed to support multiple concurrent clients accessing the same dataset, is it technically possible without modifying code/recompiling (but maybe create scripts and systemd sockets?) to snapshot and clone a dataset for each client, presenting it to the requesting client, and destroy it when the client disconnect? The data would be lost at disconnect unless the main version of the dataset was mounted which would be okay as it is intended to be used as read-only.
Zulgrib (1034 rep)
Oct 18, 2022, 12:09 AM • Last activity: May 15, 2025, 03:07 PM
0 votes
1 answers
36 views
How to find out iscsiadm command to remove a disk
For example, I have the following disks: [root@xxx-xxxx]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 47G 0 disk ├─sda1 8:1 0 100M 0 part /boot/efi ├─sda2 8:2 0 1G 0 part /boot └─sda3 8:3 0 45.5G 0 part ├─ocivolume-root 252:0 0 35.5G 0 lvm / └─ocivolume-oled 252:1 0 10G 0 lvm /var/oled s...
For example, I have the following disks: [root@xxx-xxxx]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 47G 0 disk ├─sda1 8:1 0 100M 0 part /boot/efi ├─sda2 8:2 0 1G 0 part /boot └─sda3 8:3 0 45.5G 0 part ├─ocivolume-root 252:0 0 35.5G 0 lvm / └─ocivolume-oled 252:1 0 10G 0 lvm /var/oled sdb 8:16 0 100G 0 disk └─sdb1 8:17 0 100G 0 part sdc 8:32 0 50G 0 disk └─sdc1 8:33 0 50G 0 part └─oraclevg-oraclebinlv 252:2 0 50G 0 lvm /u01 sdd 8:48 0 100G 0 disk └─sdd1 8:49 0 100G 0 part sde 8:64 0 100G 0 disk └─sde1 8:65 0 100G 0 part sdf 8:80 0 100G 0 disk └─sdf1 8:81 0 100G 0 part sdg 8:96 0 100G 0 disk └─sdg1 8:97 0 100G 0 part sdh 8:112 0 100G 0 disk └─sdh1 8:113 0 100G 0 part sdi 8:128 0 100G 0 disk └─sdi1 8:129 0 100G 0 part sdj 8:144 0 100G 0 disk └─sdj1 8:145 0 100G 0 part sdk 8:160 0 100G 0 disk └─sdk1 8:161 0 100G 0 part How to know which command should I run to remove /dev/sdk?? I know that the command is sudo iscsiadm -m node -T iqn.2015-12.com.oracleiaas:xxxxxxxxxxx-xxxxxxxxxx-p 169.xxx.x.xx:3260 -u sudo iscsiadm -m node -o delete -T iqn.2015-12.com.oracleiaas:xxxxxxxxxx-xxxx-xxxx-xxxx-xxxxxx-p 169.xxx.x.xx:3260 But how to know which iqn and IP to use to remove /dev/sdk?
Astora (509 rep)
Feb 10, 2025, 01:37 PM • Last activity: Feb 12, 2025, 02:50 PM
1 votes
0 answers
68 views
IO wait/failure timeout on iscsi device with multipath enablement
- I'm accessing a remote iscsi based SAN using multipath. - The network on the server side has known intermittent issues such that there are session failures and path failures/IO failures. I'm not trying to beat this problem as it's already a WIP. - Now, the issue i have is let's say I'm trying to f...
- I'm accessing a remote iscsi based SAN using multipath. - The network on the server side has known intermittent issues such that there are session failures and path failures/IO failures. I'm not trying to beat this problem as it's already a WIP. - Now, the issue i have is let's say I'm trying to format or partition the device via a process/service, the parted/mkfs cmd gets hung causing Kernel panic. This value is set to 240 secs. - Now, what i want to avoid is the kernel panic, i want parted/mkfs command to fail and return than cause kernel panic. - I have searched and tried changing various parameters ( iscsid, sysfs, multipath ) to no avail. This is my iscsid config
iscsid.startup = /bin/systemctl start iscsid.socket iscsiuio.socket
node.startup = automatic
node.leading_login = No
node.session.timeo.replacement_timeout = 30
node.conn.timeo.login_timeout = 30
node.conn.timeo.logout_timeout = 15
node.conn.timeo.noop_out_interval = 5
node.conn.timeo.noop_out_timeout = 5
node.session.err_timeo.abort_timeout = 15
node.session.err_timeo.lu_reset_timeout = 30
node.session.err_timeo.tgt_reset_timeout = 30
node.session.initial_login_retry_max = 8
node.session.cmds_max = 128
node.session.queue_depth = 2
node.session.xmit_thread_priority = -20
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 262144
node.conn.iscsi.MaxRecvDataSegmentLength = 262144
node.conn.iscsi.MaxXmitDataSegmentLength = 262144
discovery.sendtargets.iscsi.MaxRecvDataSegmentLength = 32768
node.conn.iscsi.HeaderDigest = CRC32C
node.conn.iscsi.DataDigest = CRC32C
node.session.nr_sessions = 1
node.session.reopen_max = 0
node.session.iscsi.FastAbort = Yes
node.session.scan = auto
multipath conf
defaults {
        path_checker none
        user_friendly_names yes          # To create ‘mpathn’ names for multipath devices
        path_grouping_policy multibus    # To place all the paths in one priority group
        path_selector "round-robin 0"    # To use round robin algorithm to determine path for next I/O operation
        failback immediate               # For immediate failback to highest priority path group with active paths
        no_path_retry 1                  # To disable I/O queueing after retrying once when all paths are down
    }
And I've set all sysfs timeout values of all slave paths to be 30 seconds. But still parted/mkfs never fail and return when there's network issue ( simulated ). What am i missing? My multipath version is tad old but i can't upgrade as this is supported version on Rocky 8. multipath-tools v0.8.4 (05/04, 2020) iscsid version 6.2.1.4-1
Neetz (111 rep)
Jan 21, 2025, 09:38 PM
0 votes
1 answers
403 views
Write QCOW2 image to iSCSI LUN?
I've been trying to write a QCOW2 image to an iSCSI LUN target but have been unsuccessful. Based on [this][1] link, this should be possible (and should be possible without having to mount the iSCSI LUN as a disk) but from what error I am getting back, it's unclear to me whether what I am trying to d...
I've been trying to write a QCOW2 image to an iSCSI LUN target but have been unsuccessful. Based on this link, this should be possible (and should be possible without having to mount the iSCSI LUN as a disk) but from what error I am getting back, it's unclear to me whether what I am trying to do simply can't be done or I have a syntax error in the URL. The OS is Debian (Proxmox). Based on this output, the LUN should be visible: root@delta-vm:~# pvesm list TrueNAS Volid Format Type Size VMID TrueNAS:0.0.0.scsi-36589cfc000000f18d78a50cff1dec18b raw images 48318386176 root@delta-vm:~# ls /dev/disk/by-path/ ip-10.0.50.1:3260-iscsi-iqn.2005-10.org.freenas.ctl:delta-proxmox-target-lun-0 pci-0000:00:1f.2-ata-5.0-part1 pci-0000:00:1f.2-ata-5-part2 ip-10.0.60.1:3260-iscsi-iqn.2005-10.org.freenas.ctl:delta-proxmox-target-lun-0 pci-0000:00:1f.2-ata-5.0-part2 pci-0000:00:1f.2-ata-5-part3 pci-0000:00:1f.2-ata-5 pci-0000:00:1f.2-ata-5.0-part3 pci-0000:00:1f.2-ata-6 pci-0000:00:1f.2-ata-5.0 But when I try this, I am confused by the error message root@delta-vm:~# qemu-img convert -O raw /var/lib/vz/template/iso/Rocky-9-GenericCloud-Base.latest.x86_64.qcow2.img iscsi://10.0.50.1:3260/iscs-iqn.2005-10.org.freenas.ct l/delta-proxmox-target-lun-0 qemu-img: iscsi://10.0.50.1:3260/iscsi-iqn.2005-10.org.freenas.ctl/delta-proxmox-target-lun-0: error while converting raw: Protocol driver 'iscsi' does not support image creation, and opening the image failed: Failed to parse URL : iscsi://10.0.50.1:3260/iscsi-iqn.2005-10.org.freenas.ctl/delta-proxmox-target-lun-0 I have tried a few permutations of the URL above, but nothing seems to be working. E.g. cutting out the iscsi- string in URL doesn't help (the iSCSI Global Config is iqn.2005-10.org.freenas.ctl): root@delta-vm:~# qemu-img convert -O raw /var/lib/vz/template/iso/Rocky-9-GenericCloud-Base.latest.x86_64.qcow2.img iscsi://10.0.50.1:3260/iqn.2005-10.org.freenas.ctl/del ta-proxmox-target-lun-0 qemu-img: iscsi://10.0.50.1:3260/iqn.2005-10.org.freenas.ctl/delta-proxmox-target-lun-0: error while converting raw: Protocol driver 'iscsi' does not support image creation, and opening the image failed: Failed to parse URL : iscsi://10.0.50.1:3260/iqn.2005-10.org.freenas.ctl/delta-proxmox-target-lun-0 Any clues what am I doing wrong? The iSCSI portal(s) are unauthenticated so I am providing no user/pass combination to the URL above. For full disclosure (and I ack that this is tangential to my question), my ultimate goal with these gyrations would be to see if it's possible to re-target a disk (i.e. move storage) in qemu from local storage to iSCSI LUN but for some reason, this was never properly implemented in Proxmox so I have to take the scenic route. In other words, given this... root@delta-vm:~# qm config 105 agent: 1 ... onboot: 1 scsi0: samsung:vm-105-disk-0,discard=on,iothread=1,replicate=0,size=12G,ssd=1 scsihw: virtio-scsi-single .... ...I'd like to substitute the scsi0 device with the iSCSI LUN. From what I gather , this currently isn't possible as the code to materialise the disk image onto the LUN simply doesn't exist and an error is thrown (storage migration failed: can't allocate space in iscsi storage). EDIT: Some progress - specifying the _size_ argument to the directive changes the error. It looks like there is something definitely wrong with the URL root@delta-vm:~# qemu-img convert -O raw /var/lib/vz/template/iso/Rocky-9-GenericCloud-Base.latest.x86_64.qcow2.img iscsi://10.0.50.1:3260/iscsi-iqn.2005-10.org.freenas.ctl/delta-proxmox-target-lun-0 10G qemu-img: Could not open 'iscsi://10.0.50.1:3260/iscsi-iqn.2005-10.org.freenas.ctl/delta-proxmox-target-lun-0': Failed to parse URL : iscsi://10.0.50.1:3260/iscsi-iqn.2005-10.org.freenas.ctl/delta-proxmox-target-lun-0
quantum (101 rep)
Mar 6, 2024, 01:16 AM • Last activity: Oct 24, 2024, 11:38 AM
0 votes
1 answers
86 views
Can not change ISCSI disk mount point to other directory
**Problem:** I can not change ISCSI disk mount point to other directory. Currently I have ISCSI disk mounted to /iscsi2 I would like to change it to /iscsi I changed /iscsi2 to /iscsi in /etc/fstab, but mounting to /iscsi is unsuccessful. **Reproducing steps:** 1. Mount /iscsi2 based on /etc/fstab....
**Problem:** I can not change ISCSI disk mount point to other directory. Currently I have ISCSI disk mounted to /iscsi2 I would like to change it to /iscsi I changed /iscsi2 to /iscsi in /etc/fstab, but mounting to /iscsi is unsuccessful. **Reproducing steps:** 1. Mount /iscsi2 based on /etc/fstab. Failed, because mountpoint is already renamed to /iscsi mount /iscsi2 mount: can't find /iscsi2 in /etc/fstab 2. Mount /iscsi2 based on device path, successful. mount /dev/sdd1 /iscsi2 3. Shows that it is mounted: lsblk -af | grep -i iscsi L¦sdd1 ext4 ISCSI_Backup2 3829ed05-f445-425d-8213-3b1c2d41fba /iscsi2 4. Unmount /iscsi2: umount /iscsi2 5. Shows that it is unmounted: lsblk -af | grep -i iscsi L¦sdd1 ext4 ISCSI_Backup2 3829ed05-f445-425d-8213-3b1c2d41fba 6. Mount /iscsi based on device path. Failed. mount /dev/sdd1 /iscsi 7. Shows that it is still unmounted: lsblk -af | grep -i iscsi L¦sdd1 ext4 ISCSI_Backup2 3829ed05-f445-425d-8213-3b1c2d41fba Any idea why mount is successful to /iscsi2 and fails to /iscsi ?
klor (426 rep)
Aug 5, 2024, 07:20 AM • Last activity: Aug 6, 2024, 12:16 AM
1 votes
2 answers
891 views
Fault-tolerant iSCSI root fs for Linux
I am in the process of moving some Linux servers onto a virtualized environment with their filesystems mounted from LVM volumes, which are in turn hosted on a remote NAS via iSCSI. I am able to start them up and they run perfectly with no issues. However, the NAS server is Windows-based and, when Mi...
I am in the process of moving some Linux servers onto a virtualized environment with their filesystems mounted from LVM volumes, which are in turn hosted on a remote NAS via iSCSI. I am able to start them up and they run perfectly with no issues. However, the NAS server is Windows-based and, when Microsoft issues patches, it automatically applies them and reboots. When it reboots, all of the virtual servers' filesystems detect errors and go into read-only mode. I have attempted to remount them as read/write, but the kernel has the filesystem flagged as write-protected, so this fails. The only way I've been able to find to recover is to shut the virt down, fsck its LVM volume, and restart it. The virts mount these LVM volumes with an fstab entry of the form: /dev/xvda2 / ext3 noatime,nodiratime,errors=remount-ro 0 1 or /dev/xvda2 / ext4 errors=remount-ro 0 1 The virtual host OS also has an LVM/iSCSI mount from the NAS server (in the same volume group, even) which continues working in read/write mode despite these interruptions. Its fstab entry is: /dev/mapper/nas6-dom0 /mnt/nas6 ext4 _netdev 0 0 This leads me to suspect that removing errors=remount-ro from the guests' fstab entries would provide fault-tolerance, but I'm a bit uneasy about doing that - if an actual error develops in the filesystem, I would expect that allowing continued writes to the fs could make things much worse in short order. What is the best practice for resolving this such that the virtual guests will be able to continue running after the NAS reboots itself?
Dave Sherohman (437 rep)
Apr 21, 2015, 08:52 AM • Last activity: Dec 20, 2023, 02:04 PM
1 votes
0 answers
121 views
iSCSI LIO target - how to resize lun?
What are the steps to resize a lun which is currently attached to an initiator? I mostly interested to know how to do that for file-based luns.
What are the steps to resize a lun which is currently attached to an initiator? I mostly interested to know how to do that for file-based luns.
Jiri B (549 rep)
Sep 21, 2023, 08:54 AM • Last activity: Sep 21, 2023, 09:44 AM
2 votes
1 answers
1246 views
Force NFS server to start after iSCSI targets connected
I have a Ubuntu machine that takes an iSCSI block, mounts it as ext4, then exports it as an NFS share. On boot, NFS fails to start as the iSCSI directory mounts have not loaded yet. "exportfs: Failed to stat /mnt/iscsi/nfs: No such file or directory" This works fine if I run nfs-kernel-server after...
I have a Ubuntu machine that takes an iSCSI block, mounts it as ext4, then exports it as an NFS share. On boot, NFS fails to start as the iSCSI directory mounts have not loaded yet. "exportfs: Failed to stat /mnt/iscsi/nfs: No such file or directory" This works fine if I run nfs-kernel-server after the server starts. Is there a way to force NFS to wait till the iSCSI block has been mounted? Edit: Further investigation.. By forcing nfs-server.service to wait for mnt-iscsi.mount, I've triggered a dependency loop.
Dec 02 09:16:09 on1 systemd: nfs-server.service: Found ordering cycle on mnt-iscsi.mount/start
Dec 02 09:16:09 on1 systemd: nfs-server.service: Found dependency on remote-fs-pre.target/start
Dec 02 09:16:09 on1 systemd: nfs-server.service: Found dependency on nfs-server.service/start
Dec 02 09:16:09 on1 systemd: nfs-server.service: Unable to break cycle starting with nfs-server.service/start
Am stuck trying to figure out what to change. Thanks!
Joshua (21 rep)
Dec 1, 2019, 10:51 PM • Last activity: Jul 19, 2023, 09:05 AM
1 votes
2 answers
471 views
iscsi target mount from NAS slow on Fedora 37 boot
I have a single iscsi target on Asustor NAS , but after adding it to fstab, F37 boot is 2 min slower, because the iscsi.service is reloaded twice during boot, and I don't know why. ``` $ sudo iscsiadm -m discovery -t st -p 192.168.0.90 192.168.0.90:3260,1 iqn.2011-08.com.asustor:as5304t-8d33b5.targe...
I have a single iscsi target on Asustor NAS , but after adding it to fstab, F37 boot is 2 min slower, because the iscsi.service is reloaded twice during boot, and I don't know why.
$ sudo iscsiadm -m discovery -t st -p 192.168.0.90
192.168.0.90:3260,1 iqn.2011-08.com.asustor:as5304t-8d33b5.target001
172.17.0.1:3260,1 iqn.2011-08.com.asustor:as5304t-8d33b5.target001

$ sudo blkid
/dev/sdc1: UUID="a8ba2edc-5c7c-41b3-bc78-6b8229dedb7d" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="primary" PARTUUID="19732655-96b8-439c-9ff1-07928f2486ef"

$ sudo iscsiadm -m session -o show
tcp:  192.168.0.90:3260,1 iqn.2011-08.com.asustor:as5304t-8d33b5.target001 (non-flash)
The slowest service is
$ systemd-analyze blame
2min 1.740s iscsi.service

# the slowest line is 
$ echo "$(
Dec 27 00:35:51 t450s iscsiadm: iscsiadm: initiator reported error (8 - connection timed out)
Dec 27 00:35:51 t450s iscsiadm: iscsiadm: Could not log into all portals
Dec 27 00:35:51 t450s iscsiadm: Logging in to [iface: default, target: iqn.2011-08.com.asustor:as5304t-8d33b5.target001, portal: 192>
Dec 27 00:35:51 t450s iscsiadm: Logging in to [iface: default, target: iqn.2011-08.com.asustor:as5304t-8d33b5.target001, portal: 172>
Dec 27 00:35:51 t450s iscsiadm: Login to [iface: default, target: iqn.2011-08.com.asustor:as5304t-8d33b5.target001, portal: 192.168.>
Dec 27 00:35:51 t450s systemd: Finished iscsi.service - Login and scanning of iSCSI devices.
Dec 27 00:35:51 t450s systemd: Reloading iscsi.service - Login and scanning of iSCSI devices...
Dec 27 00:37:54 t450s iscsiadm: iscsiadm: Could not login to [iface: default, target: iqn.2011-08.com.asustor:as5304t-8d33b5.target0>
Dec 27 00:37:54 t450s iscsiadm: iscsiadm: initiator reported error (8 - connection timed out)
Dec 27 00:37:54 t450s iscsiadm: iscsiadm: Could not log into all portals
Dec 27 00:37:54 t450s iscsiadm: Logging in to [iface: default, target: iqn.2011-08.com.asustor:as5304t-8d33b5.target001, portal: 172>
Dec 27 00:37:54 t450s systemd: Reloaded iscsi.service - Login and scanning of iSCSI devices.
I can see from above log that
Dec 27 00:35:51 t450s systemd: Finished iscsi.service - Login and scanning of iSCSI devices.
Dec 27 00:35:51 t450s systemd: Reloading iscsi.service - Login and scanning of iSCSI devices...
it finishes loading the service, then it started reloading the service, and then it took 2min to finish loading it. not sure why it wants to reload again. After booting the mount works fine, it is just that the boot time is slow, if I comment out the line fstab, the boot is fast again, but I want it to be auto mounted on boot. If I manually restart the service, it also takes 2 min
$ sudo systemctl restart iscsi.service

$ sudo journalctl -u iscsi
Dec 27 12:23:07 t450s systemd: iscsi.service: Deactivated successfully.
Dec 27 12:23:07 t450s systemd: Stopped iscsi.service - Login and scanning of iSCSI devices.
Dec 27 12:23:07 t450s systemd: Stopping iscsi.service - Login and scanning of iSCSI devices...
Dec 27 12:23:07 t450s systemd: Starting iscsi.service - Login and scanning of iSCSI devices...
Dec 27 12:25:09 t450s iscsiadm: iscsiadm: Could not login to [iface: default, target: iqn.2011-08.com.asustor:as5304t-8d33b5.target>
Dec 27 12:25:09 t450s iscsiadm: iscsiadm: initiator reported error (8 - connection timed out)
Dec 27 12:25:09 t450s iscsiadm: iscsiadm: Could not log into all portals
Dec 27 12:25:09 t450s iscsiadm: Logging in to [iface: default, target: iqn.2011-08.com.asustor:as5304t-8d33b5.target001, portal: 17>
Dec 27 12:25:09 t450s systemd: Finished iscsi.service - Login and scanning of iSCSI devices.
There might be an answer here but I can not see the hidden solution. https://access.redhat.com/solutions/3649231
Shuman (307 rep)
Dec 28, 2022, 02:27 AM • Last activity: Jan 6, 2023, 12:18 AM
0 votes
1 answers
315 views
iSCSI target implementation is User Space
I want to write my own implementation of iSCSI target where in I want to interrupt read/write calls to block device and serve them from my custom storage implementation. Is there any default library(preferably in GOLANG/Python) which gives this ability for an application running in user space. Note:...
I want to write my own implementation of iSCSI target where in I want to interrupt read/write calls to block device and serve them from my custom storage implementation. Is there any default library(preferably in GOLANG/Python) which gives this ability for an application running in user space. Note: This implementation is similar to fuse but I want to avoid fuse and directly integrate my storage in iSCSI target implementation.
Uday Swami (101 rep)
Nov 28, 2022, 05:58 AM • Last activity: Dec 11, 2022, 10:56 AM
1 votes
1 answers
8205 views
How to expand an iSCSI lun with targetcli on CentOS7?
I have a CentOS 7 machine with iSCSI configured. see ls output from targetcli below. I want to add disk02 to lun0 so that to the iSCSI client can see a total 869.16GB instead of 392.2GB Question: Is it possible to have lun0 use the combined capacity of disk0 and disk1 and if so how to do that with t...
I have a CentOS 7 machine with iSCSI configured. see ls output from targetcli below. I want to add disk02 to lun0 so that to the iSCSI client can see a total 869.16GB instead of 392.2GB Question: Is it possible to have lun0 use the combined capacity of disk0 and disk1 and if so how to do that with targetcli on CentOS7. /> ls o- / ....................................................................................................... [...] o- backstores ............................................................................................ [...] | o- block ................................................................................ [Storage Objects: 2] | | o- disk01 ...................................................... [/dev/sdb4 (392.2GiB) write-thru activated] | | o- disk02 ..................................................... [/dev/sda (476.9GiB) write-thru deactivated] | o- fileio ............................................................................... [Storage Objects: 0] | o- pscsi ................................................................................ [Storage Objects: 0] | o- ramdisk .............................................................................. [Storage Objects: 0] o- iscsi .......................................................................................... [Targets: 1] | o- iqn.2014-08.com.exmaple:nuc .................................................................... [TPGs: 1] | o- tpg1 ................................................................................ [gen-acls, no-auth] | o- acls ........................................................................................ [ACLs: 0] | o- luns ........................................................................................ [LUNs: 1] | | o- lun0 ..................................................................... [block/disk01 (/dev/sdb4)] | o- portals .................................................................................. [Portals: 1] | o- 0.0.0.0:3260 ................................................................................... [OK] o- loopback ....................................................................................... [Targets: 0] />
ams (1398 rep)
Oct 15, 2014, 04:43 PM • Last activity: Aug 24, 2022, 05:58 AM
3 votes
1 answers
19171 views
24 - iSCSI login failed due to authorization failure
When I run iscsiadm --mode node --targetname iqn.2018-12.dz.esi:iso --portal 10.11.0.2 --login I get this error: Logging in to [iface: default, target: iqn.2018-12.dz.esi:iso, portal: 10.11.0.2,3260] (multiple) iscsiadm: Could not login to [iface: default, target: iqn.2018-12.dz.esi:iso, portal: 10....
When I run iscsiadm --mode node --targetname iqn.2018-12.dz.esi:iso --portal 10.11.0.2 --login I get this error: Logging in to [iface: default, target: iqn.2018-12.dz.esi:iso, portal: 10.11.0.2,3260] (multiple) iscsiadm: Could not login to [iface: default, target: iqn.2018-12.dz.esi:iso, portal: 10.11.0.2,3260]. iscsiadm: initiator reported error (24 - iSCSI login failed due to authorization failure) iscsiadm: Could not log into all portals enter image description here My /etc/iscsi/initiatorname.iscsi file: > InitiatorName = iqn.2018-12.dz.esi:kvm1 Auth config in my target: /iscsi/iqn.20....esi:iso/tpg1> set auth userid=vcl Parameter userid is now 'vcl'. /iscsi/iqn.20....esi:iso/tpg1> set auth password=password Parameter password is now 'password'. My /etc/iscsi/iscsid.conf file: # To enable CHAP authentication set node.session.auth.authmethod # to CHAP. The default is None. node.session.auth.authmethod = CHAP # To configure which CHAP algorithms to enable set # node.session.auth.chap_algs to a comma seperated list. # The algorithms should be listen with most prefered first. # Valid values are MD5, SHA1, SHA256 # The default is MD5. # node.session.auth.chap_algs = SHA256,SHA1,MD5 # To set a CHAP username and password for initiator # authentication by the target(s), uncomment the following lines: node.session.auth.username = vcl node.session.auth.password = password Thanks for your help,
Yasmine (31 rep)
Nov 30, 2021, 05:00 AM • Last activity: Aug 18, 2022, 05:42 PM
1 votes
0 answers
183 views
ISCSI - how to connect only in one target session?
I have created a lab for databases, and when I am discovering the targets, it show two sessions (1,2): 10.0.0.17:3260,1 iqn.2003-01.org.linux-iscsi.san.x8664:sn.b5aad37c0404 10.0.0.17:3260,2 iqn.2003-01.org.linux-iscsi.san.x8664:sn.b5aad37c0404 How can I connect to only one session? I guess those se...
I have created a lab for databases, and when I am discovering the targets, it show two sessions (1,2): 10.0.0.17:3260,1 iqn.2003-01.org.linux-iscsi.san.x8664:sn.b5aad37c0404 10.0.0.17:3260,2 iqn.2003-01.org.linux-iscsi.san.x8664:sn.b5aad37c0404 How can I connect to only one session? I guess those sessions are for multiple paths, right? since is a lab, I don't have a multipath configured, not needed. I have tried something like it (,1 at the end): iscsiadm --mode node --targetname iqn.2003-01.org.linux-iscsi.san.x8664:sn.b5aad37c0404 --portal 10.0.0.17:3260,1 --login But the two sessions are connected. I' am using targetcli in a linux machine to simulate it. If possible, I can disable on target server side as well.
Astora (509 rep)
Jul 7, 2022, 07:52 PM • Last activity: Jul 7, 2022, 08:03 PM
0 votes
1 answers
1105 views
How to recover an iSCSI disk
I run an iSCSI disk on a Linux server (RHEL7). The disk has a normal ext4 partition containing a loop device called disk01.img, which is mapped to a virtual disk with a Windows NTFS file system. All was well untill the Ethernet connection between the Windows and Linux systems was interrupted. Window...
I run an iSCSI disk on a Linux server (RHEL7). The disk has a normal ext4 partition containing a loop device called disk01.img, which is mapped to a virtual disk with a Windows NTFS file system. All was well untill the Ethernet connection between the Windows and Linux systems was interrupted. Windows decided it should examine the NTFS volume, ultimately advising to re-format it. Before accepting that, I saved the file disk01.img into /savedir on the Linux server. I am now trying to restore disk01.img but I am running into problems.
[root@server ~]# losetup -P /dev/loop1 /savedir/disk01.img
[root@server ~]# mount -t ntfs /dev/loop1 /mount/point
Failed to read bootsector (size=0)
Failed to sync device /dev/loop1: Input/output error
Failed to mount '/dev/loop1': Ongeldig argument
The device '/dev/loop1' doesn't seem to have a valid NTFS.
Maybe the wrong device is used? Or the whole disk instead of a
partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?
I imagine the NTFS volume might be damaged and I will need to dd the file disk01.img onto a USB disk with an empty NTFS partition, then porting the USB disk to Windows, run chkdsk and if that fails, try recovering the files using something like PhotoRec. Or am I just using the wrong code? Edit: Result of fdisk (in Dutch)
[root@server ~]# fdisk -l /savedir/disk01.img

Schijf /savedir/disk01.img: 750.2 GB, 750155323392 bytes, 1465147116 sectoren
Eenheid = sectoren van 1 * 512 = 512 bytes

Sectorgrootte (logischl/fysiek): 512 bytes / 512 bytes
in-/uitvoergrootte (minimaal/optimaal): 512 bytes / 512 bytes

[root@server ~]#
Edit 2: The NTFS volume is damaged, so losetup -P was working but produced an empty partition. I will probably need to mount /savedir/disk01.img with an offset for the iSCSI sector, like
mount -o loop,offset=512 /savedir/disk01.img /mount/point
but I have difficulty finding the correct offset (looking for NTFS signatures, comparing the hexdump of the *.img file to the hexdump of the newly made virtual disk). BTW: I have also ported a dd copy of disk01.img back to Windows, trying to resolve the files. Still working on that cumbersome action. Finding the start of the NTFS filesystem on disk01.img seems a much better option. Edit 3. Having run TestDisk 4 times there seem to be geometry problems I cannot yet resolve. Output:
losetup -P /dev/loop1 /savedir/disk01.img
testdisk /dev/loop1

Disk /dev/loop1 - 750 GB / 698 GiB - 1465147116 sectors
... ... ...
  Linux filesys. data            0 1465147111 1465147112
  Linux filesys. data            0 1465147111 1465147112
Warning: number of heads/cylinder mismatches 255 (NTFS) != 1 (HD)
Warning: number of sectors per track mismatches 63 (NTFS) != 1 (HD)
Warning: number of heads/cylinder mismatches 255 (NTFS) != 1 (HD)
Warning: number of sectors per track mismatches 63 (NTFS) != 1 (HD)
  MS Data                100452352 1526511615 1426059264
  Linux filesys. data            0 1465147111 1465147112
... ... ...
  Linux filesys. data    730080712 2195227823 1465147112
  Linux filesys. data    730090720 2195237831 1465147112
... ... ...
  Linux filesys. data    730093536 2195240647 1465147112
  Linux filesys. data            0 1465147111 1465147112
Warning: number of heads/cylinder mismatches 255 (NTFS) != 1 (HD)
Warning: number of sectors per track mismatches 63 (NTFS) != 1 (HD)
Warning: number of heads/cylinder mismatches 255 (NTFS) != 1 (HD)
Warning: number of sectors per track mismatches 63 (NTFS) != 1 (HD)
  MS Data               1346517247 2772576510 1426059264

------------------------------------------------------

-- After Reboot --

  Linux                          0 1465147111 1465147112
  Linux                          0 1465147111 1465147112
check_FAT: can't read FAT boot sector
Invalid FAT boot sector
 0 D FAT16 LBA             3179978961 3795078107  615099147
  FAT16 LBA             3179978961 3795078107  615099147
  Linux                          0 1465147111 1465147112
Invalid FAT boot sector
 0 D FAT12                  202831329 1489073369 1286242041
  FAT12                  202831329 1489073369 1286242041
  Linux                          0 1465147111 1465147112
Invalid NTFS or exFAT boot
 0 D HPFS - NTFS           2833862451 3582746086  748883636
  HPFS - NTFS           2833862451 3582746086  748883636
Warning: number of heads/cylinder mismatches 255 (NTFS) != 1 (HD)
Warning: number of sectors per track mismatches 63 (NTFS) != 1 (HD)
Warning: number of heads/cylinder mismatches 255 (NTFS) != 1 (HD)
Warning: number of sectors per track mismatches 63 (NTFS) != 1 (HD)
  HPFS - NTFS            100452352 1526511615 1426059264


Disk /dev/loop1 - 750 GB / 698 GiB - 1465147116 sectors

The harddisk (750 GB / 698 GiB) seems too small! (  HPFS - NTFS            100452352 1526511615 1426059264
   FAT12                  202831329 1489073369 1286242041
   Linux                  730069008 2195218167 1465149160
   Linux                  730071112 2195218223 1465147112
   Linux                  730079744 2195226855 1465147112
   Linux                  730080712 2195227823 1465147112
   Linux                  730090720 2195237831 1465147112
   Linux                  730093536 2195240647 1465147112
   HPFS - NTFS           1346517247 2772576510 1426059264
   HPFS - NTFS           1465145007 2930290014 1465145008


NTFS, blocksize=4096, 730 GB / 679 GiB

--------------------------------------

-- After adding TestDisk MBR, Reboot

  Linux                          0 1465147111 1465147112
  Linux                          0 1465147111 1465147112
  Linux                          0 1465147111 1465147112
  Linux                          0 1465147111 1465147112
check_FAT: can't read FAT boot sector
Invalid FAT boot sector
 0 D FAT16 LBA             3179978961 3795078107  615099147
  FAT16 LBA             3179978961 3795078107  615099147
  Linux                          0 1465147111 1465147112
  Linux                          0 1465147111 1465147112
Invalid FAT boot sector
 0 D FAT12                  202831329 1489073369 1286242041
  FAT12                  202831329 1489073369 1286242041
  Linux                          0 1465147111 1465147112
Invalid NTFS or exFAT boot
 0 D HPFS - NTFS           2833862451 3582746086  748883636
  HPFS - NTFS           2833862451 3582746086  748883636
Warning: number of heads/cylinder mismatches 255 (NTFS) != 1 (HD)
Warning: number of sectors per track mismatches 63 (NTFS) != 1 (HD)
Warning: number of heads/cylinder mismatches 255 (NTFS) != 1 (HD)
Warning: number of sectors per track mismatches 63 (NTFS) != 1 (HD)
  HPFS - NTFS            100452352 1526511615 1426059264

-------------------------------------

-- After Reboot, running Deeper Analysis ---

Disk /dev/loop1 - 750 GB / 698 GiB - 1465147116 sectors


  No partition          36028797018963967 1465147115 1465147117

Enter the ending sector 

--------------------------------------
roelvdh (1 rep)
May 23, 2022, 12:47 PM • Last activity: May 25, 2022, 06:56 AM
14 votes
2 answers
50593 views
Mount iscsi drive at boot - system halts
I am running Oracle Linux 7 (CentOS / RedHat based distro) in a VirtualBox VM on a Mac with OS X 10.10. I have a Synology Diskstation serving as an iscsi target. I have successfully connected to the Synology, partitioned the disk and created a filesystem. It is refernced as `/dev/sdb` and the partit...
I am running Oracle Linux 7 (CentOS / RedHat based distro) in a VirtualBox VM on a Mac with OS X 10.10. I have a Synology Diskstation serving as an iscsi target. I have successfully connected to the Synology, partitioned the disk and created a filesystem. It is refernced as /dev/sdb and the partition is /dev/sdb1. Now, what I would like to do is create a mount point so I can easily access it: mount /dev/sdb1 /mnt/www That command works. But obviously, it isn't persistent across a reboot. No worries...into /etc/fstab we go. First, I got the UUID of the partition to ensure I am always using the correct device: blkid /dev/sdb1 Result: /dev/sdb1: UUID="723eb295-8fe0-409f-a75f-a26eede8904f" TYPE="ext3" Now, I inserted the following line into my /etc/fstab UUID=723eb295-8fe0-409f-a75f-a26eede8904f /mnt/www ext3 defaults 0 0 Upon reboot, the system crashes and goes into maintenance mode. If I remove the line I inserted, all works again. However, I am following the instructions verbatim from Oracle-Base I know I am missing something..can anyone point me in the right direction?
Allan (1090 rep)
Apr 8, 2015, 08:09 PM • Last activity: Dec 14, 2021, 04:47 PM
0 votes
1 answers
1093 views
dd: writing to Invalid argument with 4k blocksize on iSCSI LUN
Why can't use `oflag=direct` on the 4k blocksize iSCSI disk? **Could other applications also have a problem with the cause?** root@testvm02:~# dd if=/root/speedtest of=/mnt/8k_512/speedtest oflag=direct 51200+0 records in 51200+0 records out 26214400 bytes (26 MB, 25 MiB) copied, 59,4312 s, 441 kB/s...
Why can't use oflag=direct on the 4k blocksize iSCSI disk? **Could other applications also have a problem with the cause?** root@testvm02:~# dd if=/root/speedtest of=/mnt/8k_512/speedtest oflag=direct 51200+0 records in 51200+0 records out 26214400 bytes (26 MB, 25 MiB) copied, 59,4312 s, 441 kB/s root@testvm02:~# dd if=/root/speedtest of=/mnt/8k_4k/speedtest oflag=direct dd: writing to '/mnt/8k_4k/speedtest': Invalid argument 1+0 records in 0+0 records out 0 bytes copied, 0,000790648 s, 0,0 kB/s root@testvm02:~# dd if=/root/speedtest of=/mnt/4k_4k/speedtest oflag=direct dd: writing to '/mnt/4k_4k/speedtest': Invalid argument 1+0 records in 0+0 records out 0 bytes copied, 0.000139662 s, 0.0 kB/s my environment @storage server: /usr/sbin/zfs create -s -V 50GiB STORAGE01/4af1a9b7-0592-4707-a875-986d91fceac4 #<-- / & vda on testvm02 /usr/sbin/tgtadm --lld iscsi --op new --mode logicalunit --tid 19 --lun 1 -b /dev/zvol/STORAGE01/4af1a9b7-0592-4707-a875-986d91fceac4 # /usr/sbin/zfs create -s -V 50GiB STORAGE01/209afd7c-bdd6-4125-a899-b98758fcc6c0 #<-- /mnt/8k-512 & vdb on testvm02 /usr/sbin/tgtadm --lld iscsi --op new --mode logicalunit --tid 19 --lun 2 -b /dev/zvol/STORAGE01/209afd7c-bdd6-4125-a899-b98758fcc6c0 # /usr/sbin/zfs create -s -V 50GiB STORAGE01/6ae21fa3-df76-4843-ab65-0700af4f04f7 #<-- /mnt/8k-4k & vdc on testvm02 /usr/sbin/tgtadm --lld iscsi --op new --mode logicalunit --tid 19 --blocksize 4096 --lun 3 -b /dev/zvol/STORAGE01/6ae21fa3-df76-4843-ab65-0700af4f04f7 # /usr/sbin/zfs create -s -o volblocksize=4k -V 50GiB STORAGE01/24d38989-b47b-4e3c-b5ea-5d9a30d611f6 #<-- /mnt/4k-4k & vdd on testvm02 /usr/sbin/tgtadm --lld iscsi --op new --mode logicalunit --tid 19 --blocksize 4096 --lun 4 -b /dev/zvol/STORAGE01/24d38989-b47b-4e3c-b5ea-5d9a30d611f6 # # default zvol blocksize = 8k, default iSCSI LUN blocksize = 512 VM: virt-install --name 'testvm02.domain.de' --description 'desc' --os-type 'Linux' --os-variant 'debian9' --ram 2048 --vcpus 2 --cdrom '/var/lib/libvirt/boot/firmware-10.9.0-amd64-netinst.iso' --graphics vnc,password=foobar --network 'bridge:br540,model=virtio,virtualport_type=openvswitch' --disk 'vol=c0dcc42e-7805-4ab3-845f-363bdedada5b/unit:0:0:1,logical_block_size=512,physical_block_size=512' --disk 'vol=c0dcc42e-7805-4ab3-845f-363bdedada5b/unit:0:0:2,logical_block_size=512,physical_block_size=512' --disk 'vol=c0dcc42e-7805-4ab3-845f-363bdedada5b/unit:0:0:3,logical_block_size=4096,physical_block_size=4096' --disk 'vol=c0dcc42e-7805-4ab3-845f-363bdedada5b/unit:0:0:4,logical_block_size=4096,physical_block_size=4096' all disks formatted with ext4 root@testvm02:~# fdisk -l Disk /dev/vda: 50 GiB, 53687091200 bytes, 104857600 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x50172cb2 Device Boot Start End Sectors Size Id Type /dev/vda1 * 2048 999423 997376 487M 83 Linux /dev/vda2 999424 104855551 103856128 49,5G 83 Linux Disk /dev/vdb: 50 GiB, 53687091200 bytes, 104857600 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x5e831f4b Device Boot Start End Sectors Size Id Type /dev/vdb1 2048 104855551 104853504 50G 83 Linux Disk /dev/vdc: 50 GiB, 53687091200 bytes, 13107200 sectors Units: sectors of 1 * 4096 = 4096 bytes Sector size (logical/physical): 4096 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: dos Disk identifier: 0x791f5094 Device Boot Start End Sectors Size Id Type /dev/vdc1 256 13106943 13106688 50G 83 Linux Disk /dev/vdd: 50 GiB, 53687091200 bytes, 13107200 sectors Units: sectors of 1 * 4096 = 4096 bytes Sector size (logical/physical): 4096 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: dos Disk identifier: 0x6de2b945 Device Boot Start End Sectors Size Id Type /dev/vdd1 256 13106943 13106688 50G 83 Linux
FaxMax (726 rep)
May 17, 2021, 04:03 PM • Last activity: May 18, 2021, 10:02 AM
1 votes
0 answers
685 views
iscsi login issue - 32 target not likely connected
I'm experimenting with iSCSI. I set up a target on a VM and set up a second VM to UEFI network boot. In the `initramfs`, it tries to log in to the target and use a root filesystem from the iSCSI device. This all works. The problem is when I try to switch the initiator from a VM to a physical device....
I'm experimenting with iSCSI. I set up a target on a VM and set up a second VM to UEFI network boot. In the initramfs, it tries to log in to the target and use a root filesystem from the iSCSI device. This all works. The problem is when I try to switch the initiator from a VM to a physical device. It netboots just fine and I can see diagnostics I logged in the initramfs. Where the VM normally emitted a message like Connection1:0 to [target:...... to show that it logged in OK, the physical host instead logs a message: initiator reported error (32 - target likely not connected). On the server, I see a non-descript message. I believe I've reduced as many variables as I can. Both the VM initiator and the physical initiator are booting the same UEFI unified Linux kernel from TFTP, they're using the same initiator/target information, and the VM even has the same MAC as the physical host. (They are of course never both running at the same time). The only remaining variable I see is the network position. The VM's NIC is bridged to the same interface that the iSCSI target's NIC is bridged to. The physical host's NIC is on a different switch. Is there any way to increase the logging verbosity on the iSCSI target to see if maybe it would log a better error? Alternatively, does the debug logging from the physical initiator mean anything to anyone? The time delta between Setting login timer and poll not conntected 0/initiator reported error is pretty much instant. From a tcpdump, it's actually the initiator sending the FIN packet to the server, right after finishing the 3-way handshake. So what did iscsistart not like about the connection? This error is only emitted in two places in the code, and only [this call site](https://github.com/open-iscsi/open-iscsi/blob/bf035ee1784f28e81917c5bd9d930bf238f372e2/usr/iscsid_req.c#L159) would make sense in context. That is called only in two main paths, of which one [leads back to iscsistart.c: login_session](https://github.com/open-iscsi/open-iscsi/blob/b680f6e81f2f05f7e721f0aa97ce8aa885b3f0ba/usr/iscsistart.c#L223-L265) which has a comment about a race condition :( Initcpio hook:
modprobe iscsi_tcp
# ... dhcp ...
( 
  set -x
  ip route show
  ip route get 10.0.0.86
  ping -c1 10.0.0.86
  local INITIATOR=...
  local TARGET=...
  iscsistart -d6 -i "$INITIATOR" -t "$TARGET" -g 1 -a 10.0.0.86
) 2>&1 | while IFS='' read -r line; do
  # feed output slowly so I have a chance to read it
  echo "$line"
  sleep 1
done
set +x
# ... remainder of early userspace hooks ...
Physical host:
...
--- 10.0.0.86 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trp min/avg/max = 146.998/146.998/146.998 ms
Logging into iqn.....
Matched transport tcp
sysfs_attr_get_value: open /class/iscsi_transport/tcp/handle
sysfs_attr_get_value: open /class/iscsi_transport/tcp/caps
Allocated session 0x....
resolved 10.0.0.86 to 10.0.0.86
setting iface default, dev , set up , hw , transport tcp
set TCP recv window size to 524280, actually got 425984
set TCP send window size to 524280, actually got 425984
connecting to 10.0.0.86:3260
Setting login timer 0x... timeout 30
poll not connected 0
initiator reported error (32 - target likely not connected)
mgmt_ipc_write_rsp: rsp to fd 8
deleting a schelded/waiting thread!
Releasing session 0x...
iscsi child done
VM:
...
--- 10.0.0.86 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trp min/avg/max = 0.902/0.902/0.902 ms
Logging into iqn.....
Matched transport tcp
sysfs_attr_get_value: open /class/iscsi_transport/tcp/handle
sysfs_attr_get_value: open /class/iscsi_transport/tcp/caps
Allocated session 0x....
resolved 10.0.0.86 to 10.0.0.86
setting iface default, dev , set up , hw , transport tcp
set TCP recv window size to 524280, actually got 425984
set TCP send window size to 524280, actually got 425984
connecting to 10.0.0.86:3260
Setting login timer 0x... timeout 30
connected local port 39830 to 10.0.0.86:3260
expecting event 11, got 106, handling
....
Connection1:0 to [target:......
....
# system boots
Server (only when physical host tries to connect):
rx_data returned 0, expecting 48.
iSCSI Login negotiation failed.
tcpdump from Server:
21:48:19.467611 IP (tos 0x0, ttl 64, id 43318, offset 0, flags [DF], proto ICMP (1), length 84)
    10.0.0.74 > 10.0.0.86: ICMP echo request, id 54272, seq 0, length 64
21:48:19.467661 IP (tos 0x0, ttl 64, id 26507, offset 0, flags [none], proto ICMP (1), length 84)
    10.0.0.86 > 10.0.0.74: ICMP echo reply, id 54272, seq 0, length 64
21:48:20.695909 IP (tos 0x0, ttl 64, id 21194, offset 0, flags [DF], proto TCP (6), length 60)
    10.0.0.74.46440 > 10.0.0.86.3260: Flags [S], cksum 0x13b8 (correct), seq 1762553309, win 65535, options [mss 1460,sackOK,TS val 3433609716 ecr 0,nop,wscale 2], length 0
21:48:20.695974 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    10.0.0.86.3260 > 10.0.0.74.46440: Flags [S.], cksum 0x14ce (incorrect -> 0x9f88), seq 4180390614, ack 1762553310, win 65160, options [mss 1460,sackOK,TS val 2735285892 ecr 3433609716,nop,wscale 7], length 0
21:48:20.698368 IP (tos 0x0, ttl 64, id 21195, offset 0, flags [DF], proto TCP (6), length 52)
    10.0.0.74.46440 > 10.0.0.86.3260: Flags [.], cksum 0x8c0e (correct), ack 1, win 16384, options [nop,nop,TS val 3433609923 ecr 2735285892], length 0
21:48:21.515176 IP (tos 0x0, ttl 64, id 21196, offset 0, flags [DF], proto TCP (6), length 52)
    10.0.0.74.46440 > 10.0.0.86.3260: Flags [F.], cksum 0x88f4 (correct), seq 1, ack 1, win 16384, options [nop,nop,TS val 3433610716 ecr 2735285892], length 0
21:48:21.519590 IP (tos 0x0, ttl 64, id 17461, offset 0, flags [DF], proto TCP (6), length 52)
    10.0.0.86.3260 > 10.0.0.74.46440: Flags [.], cksum 0x14c6 (incorrect -> 0xc3bf), ack 2, win 510, options [nop,nop,TS val 2735286715 ecr 3433610716], length 0
21:48:21.524522 IP (tos 0x0, ttl 64, id 17462, offset 0, flags [DF], proto TCP (6), length 52)
    10.0.0.86.3260 > 10.0.0.74.46440: Flags [F.], cksum 0x14c6 (incorrect -> 0xc3b9), seq 1, ack 2, win 510, options [nop,nop,TS val 2735286720 ecr 3433610716], length 0
21:48:21.527019 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)
    10.0.0.74.46440 > 10.0.0.86.3260: Flags [.], cksum 0x8593 (correct), ack 2, win 16384, options [nop,nop,TS val 3433610752 ecr 2735286720], length 0
------------------- **Update:** After recompiling open-iscsi with additional logging and digging around the code a bit, I concluded something must be wrong with the way it is detecting the 1 second login timeout. It's a bit hard to follow the intended flow with all the actors / timeouts / alarms, but I managed to rip out enough of the code pertaining to login timeouts that it proceeded to the rest of the login process. The minimal change to get it to work is to remove the login timeout in login_session in iscsistart.c, but I suspect that's actually just working around the real issue, which appears to be related to the TCP connect returning EINPROGRESS (despite the code explicitly setting the socket to non-blocking). Some retry actor should be running before some timeout actor (or the parent process) kills it, but it doesn't seem to be happening. I couldn't help but notice that [this commit](https://github.com/open-iscsi/open-iscsi/commit/9258c8eae046d98511d92912983778ca57ba201f) is the most recent commit to the file and happens to change a return code. I can't prove it's related (yet), but it seems suspect.
static int login_session(struct node_rec *rec)
 {
        iscsiadm_req_t req;
        iscsiadm_rsp_t rsp;
        int rc, msec, err;
        struct timespec ts;
 
        rc = apply_params(rec);
        if (rc)
                return rc;
 
        printf("%s: Logging into %s %s:%d,%d\n", program_name, rec->name,
                rec->conn.address, rec->conn.port,
                rec->tpgt);
        memset(&req, 0, sizeof(req));
        req.command = MGMT_IPC_SESSION_LOGIN;
        memcpy(&req.u.session.rec, rec, sizeof(*rec));
 
        /*
         * Need to handle race where iscsid proc is starting up while we are
         * trying to connect. Retry with exponential backoff, start from 50 ms.
         */
        for (msec = 50; msec <= 15000; msec <<= 1) {
-               rc = iscsid_exec_req(&req, &rsp, 0, ISCSID_REQ_TIMEOUT);
+               rc = iscsid_exec_req(&req, &rsp, 0, -1);
Huckle (1087 rep)
Dec 26, 2020, 05:48 AM • Last activity: Dec 27, 2020, 04:48 AM
Showing page 1 of 20 total questions