I am looking for any opensource software which is capable to create virtual tape library. I am aware of two such software like mhvtl and quadstor VTL. I tried both but the problem is that there is no enough documentation available for mhvtl which can help me to install and configure. For quadstor VTL, I am unable to find the download link. Is there any alternate opensource software available to fulfill the requirement? I am using centos 9 and Ubuntu 24.04. Thanks in advance!!

I'm a little confusing about the tape options in Unix EOM (end of mark) EOT (end of tape) PEOT (physical end of tape) As I understand EOM is the the end of last file, but I have also some free tape EOT is the end of tape, no more space available PEOT is the same of EOT, correct?

I am currently trying to back up data onto an LTO-4 tape using mt-st and gnu tar 1.32, but I want to make sure I stop trying to copy things before the tape runs out! LTO-4 nominally has a capacity of 800G or 1.6T compressed. tapeinfo -f /dev/nst0 | grep Comp returns DataCompEnabled: yes DataCompCapable: yes DataDeCompEnabled: yes CompType: 0x1 DeCompType: 0x1 which I think means that compression is enabled ? Then again, I am adding archives to the tape with mt-st -f /dev/nst0 eod ; tar -czf /dev/nst0 directoryname, so I am also compressing that archive with gzip. In short, I don't know how to visualize how much data the archives on the tape are taking up, they are measured in blocks and I don't know how much data a block consists of. I have copied about 200G of data to the tape already and mt-st -f /dev/nst0 eod ; mt-st -f /dev/nst0 status ; echo -e "\n" ; mt-st -f /dev/nst0 tell returns: SCSI 2 tape drive: File number=1, block number=-1, partition=0. Tape block size 0 bytes. Density code 0x46 (LTO-4). Soft error count since last status=0 General status bits on (9010000): EOD ONLINE IM_REP_EN At block 18763534. But tapeinfo -f | grep MaxBlock returns MaxBlock: 16777215. So it appears I am already passed the maximum block? But mt-st -f /dev/nst0 rewind ; tar -tzvf /dev/nst0 does return a list of all of the files I copied into that archive and moves the tape to the end of data, so I shouldn't have ran out of any space. From looking at the mt manual, I cannot find a way to go to the end of the tape without first writing it. Here is the rest given by tapeinfo if that helps: Vendor ID: 'HP ' Product ID: 'Ultrium 4-SCSI ' Revision: 'U57D' Attached Changer API: No SerialNumber: 'HU1104ERC3' MinBlock: 1 MaxBlock: 16777215 SCSI ID: 0 SCSI LUN: 0 Ready: yes BufferedMode: yes Medium Type: Not Loaded Density Code: 0x46 BlockSize: 0 Block Position: 18763534 Partition 0 Remaining Kbytes: 800226 Partition 0 Size in Kbytes: 800226 ActivePartition: 0 EarlyWarningSize: 0 NumPartitions: 0 MaxPartitions: 0

I'm archiving a few directories every night to LTO-7 tape with about 100 or so large (2GB) files in each of them. As a check that the data has been written correctly, I'm verifying that the number of bytes reported written is the same as what should have been written. I'm first looking at the size of the archive by doing a tar dry-run: tar -cP --warning=no-file-changed $OLDEST_DIR | wc -c Then I'm creating the archive with: tar -cvf /dev/nst0 --warning=no-file-changed --totals $OLDEST_DIR If the filesizes match, then I delete the original file. The problem is that the dry-run has to read the entire contents of the files and can take several hours. Ideally, it should use the reported filesizes, apply the necessary padding / aligning, and report back the size rather than thrashing the disk for hours. Using du -s or similar doesn't work because the sizes don't quite match (filesystems treat a directory as 4096 bytes, tar treats it as 0 bytes for example). Alternatively, is there a better way of checking that the file has been correctly written? I can't trust tar's return code, since I'm ignoring certain warnings (to handle some sort of bug with tar/mdraid)

I have large files which are generated on the fly to stdout, one every 24hours. I would like to archive these files progressively on tapes, ideally in a single archive which potentially spans multiple tapes. Tar is very good for managing the tapes, as it has built-in functionalities to append to an archive and to load the next tape. But it is very poor at accepting data from stdin. No matter what I do, it ends up writing a special file (link or named pipe) to the archive, instead of its content. Here is the example command, that I have been trying. The first day, generate a new archive:

ln -s /dev/stdin day1 # or use the --transform option of tar
data_generator | tar -c -h -M -f /dev/nst0 -H posix -F 'mtx -f /dev/sch0 next' day1

the next day, I would like to just change -c to -A and save the new stream into a new file appended to the tar archive, loading a new tape when it becomes necessary.

data_generator | tar -A -h -M -f /dev/nst0 -H posix -F 'mtx -f /dev/sch0 next' day2

As I said, all I find in the archive is a named pipe (with -h) or a symlink (without -h). Some ideas that I have tried and are not good: 1. Using split instead of tar is not viable, because it is too basic. It can only split to pre-defined dimension (not good if I do not start from the beginning of the tape), and it cannot concatenate the different days in an unpackable archive. Tar does not need to know the size of the data nor the tape, it will just switch to a new tape when it gets a write error. 2. I've read the manuals of cpio, star and dar. I do not get the impression that they cope with pipes better than tar. Thank you for any hints. Edit: I'm starting to think that it is impossible with tar, because it needs to know the size of the file before starting to write. In fact, an archive that can be expanded, appending is very tricky if you do write the size before the content.

I need to know if there is a SCSI tape drive connected to my OpenSUSE Linux server. I tried checking the status of /dev/st0 but got an error message: mt: /dev/st0: Input/output error. /proc/scsi/scsi contains the following entries: Attached devices: Host: scsi0 Channel: 00 Id: 00 Lun: 00 Vendor: hp Model: DVD RW AD-7586H Rev: KP03 Type: CD-ROM ANSI SCSI revision: 05 Host: scsi2 Channel: 00 Id: 05 Lun: 00 Vendor: HP Model: C7438A Rev: ZP8B Type: Sequential-Access ANSI SCSI revision: 03 lsscsi output: [0:0:0:0] cd/dvd hp DVD RW AD-7586H KP03 /dev/sr0 [2:0:5:0] tape HP C7438A ZP8B /dev/st0

My company has a dual Dell LTO-7 drive. I've recently discovered that it's possible to automatically back up to two tapes, sequentially: tar -cv --multi-volume --file=/dev/nst0 --file=/dev/nst1 gigantic10TBfile.bin Excellent. Each tape is 6TB native capacity, we assume the gigantic10TBfile.bin doesn't compress, when the tape in /dev/nst0 is full, tar automatically moves on the the tape in /dev/nst1, and I don't have to leave my chair. Now, in reality, it's not a gigantic file, it's thousands of smaller files. I tee the output of tar to a log file to see how things went. In the case of a transfer spanning multiple tapes, I'd like an entry in this log file saying "tape change" or something. Enter --new-volume-script (also known as --info-script or -F, docs [here](https://www.gnu.org/software/tar/manual/html_node/Multi_002dVolume-Archives.html#info_002dscript)) . Yay. Except it's not executed. Let's make a more manageable example: tar -cvlp --multi-volume --new-volume-script="./new_volume_script.sh" --file=tarball1.tar --file=tarball2.tar --tape-length=4M 10MBrandom.bin Running this example reveals that the --new-volume-script is only executed once tarball1.tar and tarball2.tar have been created, and tar "runs out of" destination files. So, my question is, is there an option to execute a script between the writing to tarball1.tar and tarball2.tar?

I'm running mt version 2.13 on OpenSuse 15.5 with a Quantum DLT-4000 tape drive. When I run mt -f /dev/st0 status I get the following output: drive type = 114 drive status = 419430400 sense key error = 0 residue count = 0 file number = 0 block number = 0 How do I interpret the drive type and drive status codes?

I use

PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian

With an own Kernel

Linux 5.4.61 #1 SMP Tue Sep 1 21:08:21 CEST 2020 x86_64 GNU/Linux

Added SCSI support and everything.

root@server:~# lsscsi
[0:0:0:0]    tape    TANDBERG LTO-5 HH         Z629  /dev/st0 
[1:0:0:0]    disk    ATA      ST4000VN008-2DR1 SC60  /dev/sda 
[2:0:0:0]    disk    ATA      ST4000VN008-2DR1 SC60  /dev/sdb 
[3:0:0:0]    disk    ATA      ST4000VN008-2DR1 SC60  /dev/sdc 
[7:0:0:0]    disk    JetFlash Transcend 16GB   1.00  /dev/sdd

So, i see the tapedrive but have a lot of trouble with LTFS. I used this website for getting LTFS:

https://www.tandbergdata.com/de/index.cfm/solutions/ltfs-for-archive/ltfs-downloads-lto56/ 

And tried the own building stuff, the RHEL binaries. But the own building says it want new builded with libtool 2.2.6b but I have 2.4.6 and there is no possebility for autoreconf --force --install Well, the RHEL stuff want icu 50.2, but for Debian there is only 52.x or I must build it on my own, but I don't think it's the correct solution to downgrade software to really old stuff. I tried to get HPE StoreOpen for RHEL, but it's only for enterprise customers ands costs money, so it's no option... The LTFS from GitHub supports only IBM drives, so no option... it's so frustrating :( Now I just want to know: Are there any basic debian/linux tools for managing LTO drives: formatting, writing, reading, mounting, etc.? Or can anyone help me to get it working on Debian 10? Thank you in advance.

My goal is to extract only the files in a list from an lto5 tape using tar. The data was originally written to the tape using tar as well. I am able to accomplish this task from a .tgz file by doing: gunzip "Unless you are running tar with ‘--create’, you cannot use both > --files-from=- and --file=- (-f -) in the same command." Because it is not possible to use the --file and --files-from (-T) options together, I'm wondering if it's even possible to restore specific files from tape this way? I've seen others have had to position the tape at different points to get specific files and am curious if that is the only method?

Outside of [LTFS](https://en.wikipedia.org/wiki/Linear_Tape_File_System) , the [Microsoft Tape Format](https://en.wikipedia.org/wiki/Microsoft_Tape_Format) seems to be the closest to a widely supported format when it comes to older tape format that don't support LTFS (In my case, Ultrium 3 and 4). However, while there are [several ways to read MTF tapes on Linux](https://unix.stackexchange.com/questions/213162/restoring-from-ntbackup-tape) , it seems that nothing creates tapes using it - opting instead of raw use of tar(/cpio/pax) and mt, or "bigger" backup solutions like Bacula/BareOS. Is my understanding correct that tar really only covers one "archive" on a tape, while MTF is meant to cover the tape as a whole? And if yes, is there a utility on Linux that uses MTF as a tape backup format?

I am reading/writing data from/to tapes using Debian and recently ran into an issue where I needed to figure out the size of the stored data stream, i.e. the data actually written to the tape (see [this question](https://unix.stackexchange.com/q/698589)) . In short, using .tar.lzma, this does not seem to be possible without copying the entire archive off the tape first. Are there any alternative container formats to compressed tarballs that: - allow for determining the full size of the archive itself linearly (i.e. without shoe-shining the tape) - determine the size *without* reading the entire archive to disk - allow for linear decompression/access - are Linux-compatible (BSD-compatibility is fine too) I think [7-Zip](https://7-zip.org/) may work in this situation, but isn't it discouraged for archival in *nix environments?

I have a tape with a .tar.lzma archive written to it. How can I determine the size of the .tar.lzma archived _as is_? In other words, I'm looking to find the size of the compressed archive, _not_ the size of the uncompressed .tar it is storing -- ideally without having to copy the archive off the tape to a disk and checking from there. The tape in question (LTO-6) is **not** formatted with any special file system, such as LTFS. My system is using Debian 11 Bullseye on amd64, if it is relevant to mention this.

The Wikipedia LTO article says that every LTO drive can read out the memory chip of a tape via 13.56 MHz NFC. I expect here to find serial numbers, tape properties and usage data. How can I read this data with free and open-source software on a Linux system?

I have a HP 1x8 G2 Autoloader that connected to a SAN switch, on my VMware environment I have an Oracle Linux that should work with the tape. The problem is after connecting tape to VM, medium changer is not detected but tape drive is OK!

$ cat /proc/scsi/scsi 
Attached devices:
Host: scsi2 Channel: 00 Id: 00 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 2.0
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 01 Lun: 00
  Vendor: HP       Model: Ultrium 6-SCSI   Rev: 25MW
  Type:   Sequential-Access                ANSI  SCSI revision: 06
Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: NECVMWar Model: VMware IDE CDR00 Rev: 1.00
  Type:   CD-ROM                           ANSI  SCSI revision: 05

$ lsscsi -g 
[0:0:0:0]    cd/dvd  NECVMWar VMware IDE CDR00 1.00  /dev/sr0   /dev/sg2
[2:0:0:0]    disk    VMware   Virtual disk     2.0   /dev/sda   /dev/sg0
[2:0:1:0]    tape    HP       Ultrium 6-SCSI   25MW  /dev/st0   /dev/sg1

$ ls -l /dev/tape/by-path
lrwxrwxrwx 1 root root  9 Jul 30 08:49 pci-0000:00:10.0-scsi-0:0:1:0 -> ../../st0
lrwxrwxrwx 1 root root 10 Jul 30 08:49 pci-0000:00:10.0-scsi-0:0:1:0-nst -> ../../nst0

$ lsmod | grep scsi
scsi_transport_spi     32768  1 mptspi
mptscsih               36864  1 mptspi
mptbase                94208  2 mptspi,mptscsih

$ lsmod | grep sg 
sg                     36864  0

$ lsmod | grep st
st                     61440

I installed latest version of HPE Array Controller for Centos 8

$ yum list installed 
kmod-hpsa.x86_64                     3.4.20-5.el8_4.elrepo                       @elrepo

and tape Autoloader was in Sequential mode which I changed it to Auto. kernel info:

$ uname -r
5.4.17-2102.202.5.el8uek.x86_64

mtx outputs:

$ mtx -f /dev/nst0 status
mtx: Request Sense: Long Report=yes
mtx: Request Sense: Valid Residual=no
mtx: Request Sense: Error Code=70 (Current)
mtx: Request Sense: Sense Key=Illegal Request
mtx: Request Sense: FileMark=no
mtx: Request Sense: EOM=no
mtx: Request Sense: ILI=no
mtx: Request Sense: Additional Sense Code = 20
mtx: Request Sense: Additional Sense Qualifier = 00
mtx: Request Sense: BPV=no
mtx: Request Sense: Error in CDB=no
mtx: Request Sense: SKSV=no
READ ELEMENT STATUS Command Failed

$ mtx -f /dev/nst0 inquiry
Product Type: Tape Drive
Vendor ID: 'HP      '
Product ID: 'Ultrium 6-SCSI  '
Revision: '25MW'
Attached Changer API: No

I have a situation. I was supposed to run tar -x to recover a tar file from a tape but I pressed "c" and forgot to put the tape on readonly. The file was on fsf position 23, but the tape had 27 files. I know that file 23 is lost but since I pressed ctrl+c to cancel the tar command, can I still hope to somehow recover the other tar files? I tried fsf past file 23 and all I get is input/output error. I was thinking maybe do a dump from file 23 using dd and then get this file and use a signature file recovery tool, like photorec. Is this doable?

I just installed an Exabyte tape drive:

$ lsscsi -g
[11:0:0:0]   process Marvell  91xx Config      1.01  -          /dev/sg0 
[12:0:1:0]   cd/dvd  TOSHIBA  CD-ROM XM-3401TA 0283  /dev/sr0   /dev/sg1 
[12:0:5:0]   tape    EXABYTE  EXB-8500-85Qanx0 046G  /dev/st0   /dev/sg2 
[N:1:4:1]    disk    Samsung SSD 970 EVO 250GB__1               /dev/nvme1n1
[N:0:4:1]    disk    Samsung SSD 970 EVO 250GB__1               /dev/nvme0n1
$   /usr/sbin/tapeinfo -f /dev/sg2  
Product Type: Tape Drive
Vendor ID: 'EXABYTE '
Product ID: 'EXB-8500-85Qanx0'
Revision: '046G'
Attached Changer API: No
SerialNumber: '02541381  '
MinBlock: 1
MaxBlock: 245760
SCSI ID: 5
SCSI LUN: 0
Ready: yes
WriteProtect: yes
BufferedMode: yes
Medium Type: 0x85
Density Code: 0x0
BlockSize: 1024
DataCompEnabled: no
DataCompCapable: no
DataDeCompEnabled: no
CompType: 0x0
DeCompType: 0x0
Block Position: 15750
ActivePartition: 0
EarlyWarningSize: 0

I can read from a tape like this:

$ mt -f /dev/nst0 rewind
$ /usr/sbin/scsitape -f /dev/sg2 read 0  > outfile.bin
$ tar tf outfile.bin
[ a list of files in the tar archive ]

However, I cannot read from the tape with tar in the normal way:

$ mt -f /dev/nst0 rewind
$ tar tf /dev/nst0 
tar: /dev/nst0: Cannot read: Input/output error
tar: At beginning of tape, quitting now
tar: Error is not recoverable: exiting now

In /var/log/messages we see:

Jun 29 13:11:00 horizon kernel: [ 9362.244683] Errata on LSI53C1030 occurredsc->req_bufflen=0x2800, xfer_cnt=0x400,difftransfer=0x0a
Jun 29 13:11:00 horizon kernel: [ 9362.244773] st 12:0:5:0: [st0] Sense Key : Medium Error [current] 
Jun 29 13:11:00 horizon kernel: [ 9362.244790] st 12:0:5:0: [st0] >ASC=0xff >ASCQ=0xff

Reducing the read size (for example with tar -f /dev/nst0 -b 1 --record-size=512 -t) doesn't seem to make a difference. The mechanisms are quite different; scsitape read uses ioctl against /dev/sg2 to read from the tape device, while tar uses read syscalls against /dev/nst0 or /dev/st0 to read. But I don't know why access via /dev/sg2 works while access via /dev/nst0 does not. Can anybody explain this? In case it matters, the mt-st package is not installed and so there is no /etc/stinit.def file. I'm using the mt binary that comes with the cpio package.

I am looking for the simplest (feasible) way to push data away on a tape library. This is for archiving finalized, immutable datasets that only need to be retrieved in case of a catastrophic failure of another storage system out of our reach. If I was operating on the level of a single tape, I would write a simple script to do some bookkeeping on the file positions on the tape, and just append with tar. Unfortunately, I am not dealing with a single tape, but a 48 tapes library (IBM Tape Library System Storage TS3100) and datasets that will mostly span at least 2 tapes (ca. 10TB per dataset) - and I don't really how that thing works, except for having read the man page for mtx. Is it feasible to work with tar -M --new-volume-script ..., and have it note down tape and file number or am I missing something here? Is there an even simpler way, or can anybody recommend a simple backup software that already does what I want?

What does mt -f /dev/st0 erase do exactly with a (LTO-6) tape drive at /dev/st0? It takes hours and seems to write many times. Does it write random data on the tape several times to delete the contents in a secure way? Will it delete the magnetic servo tracks?

I would like to monitor statistics (transfer rate, byte/sec) for each LTO tape drive connected to Linux system. I understand that tapestat command can exactly do this, but this shows the activity of tape driver:/dev/st only. In other words, if we write/read data using SCSI generic driver: /dev/sg, this command shows nothing. Indeed, when reading/writing data with ITDT from IBM that uses the generic driver, I can not see any statistics. Is there any good ways to monitor transfer rate even with SCSI generic driver?