markh794 / mhvtl Goto Github PK

View Code? Open in Web Editor NEW

136.0 136.0 65.0 3.42 MB

Linux based Virtual Tape Library

Home Page: http://sites.google.com/site/linuxvtl2/

License: Other

Makefile 1.12% Shell 4.37% C 87.89% Perl 6.34% PHP 0.01% Roff 0.26%

mhvtl's People

Contributors

Stargazers

Watchers

Forkers

casualfish jonatanblue rajaseelan hankly fencerfan eskatos versity arcapix mondipsen rlugojr fishky jmpq opsroll tmoore-sgi daedalus qiuyi21 rockloveli humin11 boussole mwilck patrickylru hrchu gonzoleeman bmwiedemann boldfield tawaraja zengdabo qrdlvnqww2z1ztmj menglingbo siwen1017 haniffm qikashi jnofi ksw6621 abbbi jeimerson vchaves123 radtek 5l1v3r1 drscriptt xconnection t-kudoh nacmatos franku wrice intraceting dragons-lair dabiged michaelbb31 ginolv cursesun jeffwang0516 victorsow oxsdd nrichtapeark nathanatkinsonwdc davrosyoung melak runxsystem b1ng05

mhvtl's Issues

Data file write failure for LTO-6 drives

I am noticing a lot of errors like this from mhvtl:

Jun  2 18:38:42 lews /usr/bin/vtltape[2249]: ssc_write_6(): ssc_write_6(): 1 block of 524288 bytes (158341) **
Jun  2 18:38:42 lews /usr/bin/vtltape[2249]: retrieve_CDB_data(): retrieving 524288 bytes from kernel
Jun  2 18:38:42 lews /usr/bin/vtltape[2249]: check_restrictions(): returning: writable
Jun  2 18:38:42 lews /usr/bin/vtltape[2249]: writeBlock_lzo(): Compression: Orig 524288, after comp: 488361
Jun  2 18:38:42 lews /usr/bin/vtltape[2249]: write_tape_block(): CRC is 0xfd73edbe
Jun  2 18:38:42 lews /usr/bin/vtltape[2249]: return_sense(): [Key/ASC/ASCQ] [03 0c 00]
Jun  2 18:38:42 lews /usr/bin/vtltape[2249]: ERROR: write_tape_block(): Data file write failure, pos: 163: No message of desired type
Jun  2 18:38:42 lews /usr/bin/vtltape[2249]: write_tape_block(): Truncating data file size: 163

This causes an end of media error for our application and then we lock the drive so it cannot be written to afterward.

What would cause the "Data file write failure"? I looked at the code in vtlcart.c and found this:

        /* Now write out both the data and the header. */
        if (null_media_type) {
                nwrite = disk_blk_size;
        } else
                nwrite = pwrite(datafile, buffer, disk_blk_size, data_offset);
        if (nwrite != disk_blk_size) {
                sam_medium_error(E_WRITE_ERROR, sam_stat);

                MHVTL_ERR("Data file write failure, pos: %" PRId64 ": %s",
                        data_offset, strerror(errno));

                /* Truncate last partital write */
                MHVTL_DBG(1, "Truncating data file size: %"PRId64, data_offset);
                if (ftruncate(datafile, data_offset) < 0) {
                        MHVTL_ERR("Error truncating data: %s", strerror(errno));
                }

                mkEODHeader(blk_number, data_offset);
                return -1;
        }

The pwrite call is returning an unexpected value. This issue seems to occur on multiple different machines. I am using the mhvtl code from the commit on Mar 13th of this year.

The errors are occurring on RHEL8 x86_64 machines.

Thank you,
Peter

Segmentation fault of dump_tape fix

i noticed that dump_tape utility from git head cause Segmentation fault:

[root@dev-5 usr]# gdb --args ./dump_tape -f E01001L4
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-120.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/local/src/2021-04-05/mhvtl/usr/dump_tape...done.
(gdb) r
Starting program: /usr/local/src/2021-04-05/mhvtl/usr/./dump_tape -f E01001L4
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
crc32c using Intel sse4.2 hardware optimization
Media density code: 0x46
Media type code   : 0x08
Media description : Ultrium 4/16T
Tape Capacity     : 524288000 (500 MBytes)
Media type        : Normal data
Media             : read-write
Remaining Tape Capacity : 524288000 (500 MBytes)
Total num of filemarks: 0

Program received signal SIGSEGV, Segmentation fault.
0x00000000004012af in main (argc=<optimized out>, argv=<optimized out>) at dump_tape.c:559
559             while (c_pos->blk_type != B_EOD) {
(gdb) print c_pos
$1 = (struct blk_header *) 0x0
(gdb)

that happens because c_pos variable declared in dump_tape.c code override exported variable from libvtlcart.so:

[root@dev-5 usr]# nm libvtlcart.so | grep c_pos
0000000000214350 D c_pos

solution is very simple:

diff --git a/usr/dump_tape.c b/usr/dump_tape.c
index 039e921..fb6bb66 100644
--- a/usr/dump_tape.c
+++ b/usr/dump_tape.c
@@ -49,8 +49,6 @@ int lib_id;

 extern char home_directory[HOME_DIR_PATH_SZ + 1];

-struct blk_header *c_pos;
-
 static char *progname;

 static void print_mam_info(void)

please apply

Mismatched bettwen devices.conf and library_contents leads to tainted kernel

I have asked for a 4 drive library in library_contents.30, however in my devices.conf file there are only 2 drives defined.
I know I have set the config up incorrectly, but a graceful exit would be preferred!

library_contents.30

head -15 library_contents.30
VERSION: 2

Drive 1:
Drive 2:
Drive 3:
Drive 4:

Picker 1:

MAP 1:
MAP 2:
MAP 3:
MAP 4:

....

...

Library: 30 CHANNEL: 00 TARGET: 08 LUN: 00
 Vendor identification: STK
 Product identification: L80
 Unit serial number: XYZZY_B
 NAA: 30:22:33:44:ab:00:08:00
 Home directory: /opt/mhvtl
 PERSIST: False
 Backoff: 400
# fifo: /var/tmp/mhvtl

Drive: 31 CHANNEL: 00 TARGET: 09 LUN: 00
 Library ID: 30 Slot: 01
 Vendor identification: STK
 Product identification: T10000B
 Unit serial number: XYZZY_B1
 NAA: 30:22:33:44:ab:00:09:00
 Compression: factor 1 enabled 1
 Compression type: lzo
 Backoff: 400
# fifo: /var/tmp/mhvtl

Drive: 32 CHANNEL: 00 TARGET: 10 LUN: 00
 Library ID: 30 Slot: 02
 Vendor identification: STK
 Product identification: T10000B
 Unit serial number: XYZZY_B2
 NAA: 30:22:33:44:ab:00:10:00
 Compression: factor 1 enabled 1
 Compression type: lzo
 Backoff: 400
# fifo: /var/tmp/mhvtl

#Drive: 33 CHANNEL: 00 TARGET: 11 LUN: 00
# Library ID: 30 Slot: 03
# Vendor identification: STK
# Product identification: T10000B
# Unit serial number: XYZZY_B3
# NAA: 30:22:33:44:ab:00:11:00
# Compression: factor 1 enabled 1
# Compression type: lzo
# Backoff: 400
## fifo: /var/tmp/mhvtl
#
#Drive: 34 CHANNEL: 00 TARGET: 12 LUN: 00
# Library ID: 30 Slot: 04
# Vendor identification: STK
# Product identification: T10000B
# Unit serial number: XYZZY_B4
# NAA: 30:22:33:44:ab:00:12:00
# Compression: factor 1 enabled 1
# Compression type: lzo
# Backoff: 400
## fifo: /var/tmp/mhvtl

The following error is printed to dmesg:


[18142.077084] BUG: kernel NULL pointer dereference, address: 00000000000000a8
[18142.082433] #PF: supervisor write access in kernel mode
[18142.086491] #PF: error_code(0x0002) - not-present page
[18142.090601] PGD 0 P4D 0 
[18142.092862] Oops: 0002 [#2] SMP PTI
[18142.095650] CPU: 0 PID: 2449 Comm: kworker/u30:2 Tainted: G      D    OE     5.4.0-88-generic #99-Ubuntu
[18142.101610] Hardware name: Xen HVM domU, BIOS 4.11.amazon 08/24/2006
[18142.106201] Workqueue: scsi_tmf_2 scmd_eh_abort_handler
[18142.110683] RIP: 0010:_raw_spin_lock_irqsave+0x23/0x40
[18142.114547] Code: 0f 1f 80 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 41 54 9c 58 0f 1f 44 00 00 49 89 c4 fa 66 0f 1f 44 00 00 31 c0 ba 01 00 00 00 <3e> 0f b1 17 75 07 4c 89 e0 41 5c 5d c3 89 c6 e8 79 67 61 ff 66 90
[18142.127628] RSP: 0018:ffffae0b80867df8 EFLAGS: 00010046
[18142.131414] RAX: 0000000000000000 RBX: 0000000000000098 RCX: 0000000000000000
[18142.136440] RDX: 0000000000000001 RSI: 0000000000000002 RDI: 00000000000000a8
[18142.141448] RBP: ffffae0b80867e00 R08: 000000000000325f R09: 8080808080808080
[18142.146438] R10: ffff8ddd7e56a7f4 R11: 0000000000000018 R12: 0000000000000202
[18142.151490] R13: ffff8ddd6b0a9120 R14: 0000000000000000 R15: 00000000000000a8
[18142.156320] FS:  0000000000000000(0000) GS:ffff8ddd7e800000(0000) knlGS:0000000000000000
[18142.161685] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18142.165862] CR2: 00000000000000a8 CR3: 0000000034a60003 CR4: 00000000001606f0
[18142.171129] Call Trace:
[18142.173209]  mhvtl_abort+0x50/0x100 [mhvtl]
[18142.176258]  scmd_eh_abort_handler+0x8d/0x220
[18142.179383]  process_one_work+0x1eb/0x3b0
[18142.182437]  worker_thread+0x4d/0x400
[18142.185270]  kthread+0x104/0x140
[18142.187906]  ? process_one_work+0x3b0/0x3b0
[18142.190984]  ? kthread_park+0x90/0x90
[18142.194170]  ret_from_fork+0x35/0x40
[18142.196776] Modules linked in: st ch mhvtl(OE) dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel crypto_simd cryptd glue_helper rapl cirrus
 drm_kms_helper fb_sys_fops psmouse syscopyarea input_leds sysfillrect i2c_piix4 sysimgblt pata_acpi serio_raw floppy mac_hid sch_fq_codel drm ip_tables x_tables autofs4
[18142.220720] CR2: 00000000000000a8
[18142.223398] ---[ end trace abb850ca3c787270 ]---
[18142.226660] RIP: 0010:scsi_remove_device+0x10/0x40
[18142.230107] Code: ef e8 e4 b2 39 00 e9 71 ff ff ff 48 89 df e8 37 b7 ff ff eb 87 0f 1f 44 00 00 0f 1f 44 00 00 55 48 89 e5 41 55 49 89 fd 41 54 <48> 8b 07 4c 8d 60 40 4c 89 e7 e8 01 b8 39 00 4c 89 ef e8 89 fe ff
[18142.243083] RSP: 0018:ffffae0b809cbdf8 EFLAGS: 00010246
[18142.246851] RAX: 000000000000000c RBX: 0000000000000000 RCX: 000000000000f49e
[18142.251665] RDX: ffff8ddd770b0410 RSI: 2279ca348941a586 RDI: 0000000000000000
[18142.256701] RBP: ffffae0b809cbe08 R08: dead000000000100 R09: ffffffff938cbb00
[18142.261571] R10: ffff8ddd67640000 R11: 0000000000000001 R12: ffff8ddd770b0410
[18142.266231] R13: 0000000000000000 R14: ffff8ddd770b0410 R15: dead000000000122
[18142.271295] FS:  0000000000000000(0000) GS:ffff8ddd7e800000(0000) knlGS:0000000000000000
[18142.276551] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18142.281138] CR2: 00000000000000a8 CR3: 0000000034a60003 CR4: 00000000001606f0

Inconsist status after reload

Version: 1.6-2/latest master branch (06e325c)

Variables in make recursion wrong (sometimes)?

I have been staring at this for a couple of months now and could not decide whether it's actually wrong or I've gone mad. I am not a make wizard by any stretch of the imagination, though I'd like to think I have the fundamentals about right.

The Makefiles (the top-level one, I don't think the others are affected), when doing recursion, use a construct like this (at least in the install target):

$(MAKE) -C usr install $(LIBDIR) $(PREFIX) $(DESTDIR) $(SYSTEMD_SERVICE_DIR)

The intent is obvious, but I don't think this means what the intent was - that would have been

$(MAKE) -C usr install LIBDIR=$(LIBDIR) PREFIX=$(PREFIX) DESTDIR=$(DESTDIR) SYSTEMD_SERVICE_DIR=$(SYSTEMD_SERVICE_DIR)

as it is correctly used in a couple of other places.

Since the subdir Makefiles have declarations along the lines of PREFIX ?= /usr/local, until one actually tries to install mhVTL to a different directory (or whatever else the variables control), it does not become very apparent that the Makefile doesn't quite work as intended.

Have I gone mad?

capacity limit not work

# dump_tape -f E01001L4
Media density code: 0x46
Media type code   : 0x08
Media description : Ultrium 4/16T
Tape Capacity     : 524288000 (500 MBytes)
Media type        : Normal data
Media             : read-write
Remaining Tape Capacity : 524288000 (500 MBytes)
Total num of filemarks: 13
Segmentation fault (core dumped)

# du -h /opt/mhvtl/E01001L4/
755M    /opt/mhvtl/E01001L4/

LTFS

Hello, Is it possible to add LTFS support? Thanks

Wrong link to modprobe on Ubuntu

/usr/sbin/modprobe should be changed to /sbin/modprobe in https://github.com/markh794/mhvtl/blob/master/etc/mhvtl-load-modules.service.in#L14,L15

Otherwise the module wont be loaded.

mhvtl crash when using different tape types on the same changer

Mark, I got a crash when trying to write to an LTO-6 tape. The changer was configured to use LTO-6, LTO-7, and LTO-8 drives. The add_library_contents_10() function in generate_device_conf.in had:

    add_library 10 0 0 0 "IBM" "3584"  "2160"  "XYZZY_A"
    #         index channel target LUN S/No Lib# Slot
    add_ibm_ultrium_6_drive 11 0 1 0 "XYZZY_A1" 10 1
    add_ibm_ultrium_6_drive 12 0 2 0 "XYZZY_A2" 10 2
    add_ibm_ultrium_6_drive 13 0 3 0 "XYZZY_A3" 10 3
    add_ibm_ultrium_7_drive 14 0 4 0 "XYZZY_A4" 10 4
    add_ibm_ultrium_7_drive 15 0 5 0 "XYZZY_A5" 10 5
    add_ibm_ultrium_7_drive 16 0 6 0 "XYZZY_A6" 10 6
    add_ibm_ultrium_8_drive 17 0 7 0 "XYZZY_A7" 10 7
    add_ibm_ultrium_8_drive 18 0 8 0 "XYZZY_A8" 10 8
    add_ibm_ultrium_8_drive 19 0 9 0 "XYZZY_A9" 10 9

My test should have been trying to write to an LTO-6 tape since we did not have any LTO-7 or LTO-8 tapes available. I saw this logged to /var/log/messages:

Jan 19 17:03:49 choctaw kernel: vtllibrary[899]: segfault at 14 ip 00007f6d61228079 sp 00007ffcdf4172d0 error 4 in libc-2.17.so[7f6d611db000+1c4000]

and:
Jan 19 17:03:49 choctaw /usr/bin/vtllibrary[899]: dump_element_desc(): ASCII data : T3580-TD8 XYZZY_A8
Jan 19 17:03:49 choctaw /usr/bin/vtllibrary[899]: dump_element_desc(): Debug.... i = 384, len = 48
Jan 19 17:03:49 choctaw /usr/bin/vtllibrary[899]: dump_element_desc(): Element Address : 14368
Jan 19 17:03:49 choctaw /usr/bin/vtllibrary[899]: dump_element_desc(): Status : 0x20
Jan 19 17:03:49 choctaw /usr/bin/vtllibrary[899]: dump_element_desc(): Medium type : 2
Jan 19 17:03:49 choctaw /usr/bin/vtllibrary[899]: dump_element_desc(): Identification Descriptor
Jan 19 17:03:49 choctaw /usr/bin/vtllibrary[899]: dump_element_desc(): Code Set : 0x01
Jan 19 17:03:49 choctaw /usr/bin/vtllibrary[899]: dump_element_desc(): Identifier type : 0x08
Jan 19 17:03:49 choctaw /usr/bin/vtllibrary[899]: dump_element_desc(): Identifier length : 32
Jan 19 17:03:49 choctaw /usr/bin/vtllibrary[899]: dump_element_desc(): ASCII data :
Jan 19 17:03:49 choctaw /usr/bin/vtllibrary[899]: dump_element_desc(): ASCII data :
Jan 19 17:03:49 choctaw /usr/bin/vtllibrary[899]: dump_element_desc(): ASCII data : ULT3580-TD8 XYZZY_A9
Jan 19 17:03:49 choctaw /usr/bin/vtllibrary[899]: decode_element_status(): Element Status Page
Jan 19 17:03:49 choctaw abrt-hook-ccpp: Process 899 (vtllibrary) of user 0 killed by SIGSEGV - dumping core
Jan 19 17:03:49 choctaw systemd: [email protected]: main process exited, code=dumped, status=11/SEGV
Jan 19 17:03:49 choctaw systemd: Unit [email protected] entered failed state.
Jan 19 17:03:49 choctaw systemd: [email protected] failed.

Attached is the /var/log/messages from when the problem occurred. Unfortunately, a complete crash dump was not generated:

Jan 19 17:03:50 choctaw abrt-server: Traceback (most recent call last):
Jan 19 17:03:50 choctaw abrt-server: File "/usr/sbin/sosreport", line 14, in
Jan 19 17:03:50 choctaw abrt-server: from sos.sosreport import main
Jan 19 17:03:50 choctaw abrt-server: ModuleNotFoundError: No module named 'sos'
Jan 19 17:03:50 choctaw abrt-server: 'post-create' on '/var/spool/abrt/ccpp-2022-01-19-17:03:49-899' exited with 1
Jan 19 17:03:50 choctaw abrt-server: Deleting problem directory '/var/spool/abrt/ccpp-2022-01-19-17:03:49-899'
Jan 19 17:03:55 choctaw sm-notify[1523]: Unable to notify rand.clearlake.ibm.com, giving up
Jan 19 17:04:39 choctaw kernel: mhvtl: mhvtl_timer_intr_handler: Unexpected interrupt, indx 0

I thought I would attach the vtl log in case you can figure out what caused the crash.

Thank you,
Peter

mhvtl_crash.tar.gz

Setup procedure

Hi, I just installed the 1.6.4 RPM and started the service, but don't see any devices in lsscsi -g what steps are missing in order to
get devices to be visible in lsscsi?

sudo systemctl status mhvtl.target
● mhvtl.target - mhvtl service allowing to start/stop all [email protected] and [email protected] instances at once
   Loaded: loaded (/usr/lib/systemd/system/mhvtl.target; enabled; vendor preset: disabled)
   Active: active since Wed 2022-06-22 22:07:44 UTC; 20min ago
     Docs: man:man:vtltape(1)
           man:man:vtllibrary(1)

Jun 22 22:07:44 systemd[1]: Reached target mhvtl service allowing to start/stop all [email protected] and [email protected] instances at once.

Kernel module fails to be built on RHEL8.3

Kernel module fails to be built on RHEL8.3.

# LANG=C make
make -C /lib/modules/4.18.0-240.el8.x86_64/build M=/root/mhvtl/kernel modules
make[1]: Entering directory '/usr/src/kernels/4.18.0-240.el8.x86_64'
  CC [M]  /root/mhvtl/kernel/mhvtl.o
In file included from /root/mhvtl/kernel/mhvtl.c:355:
/root/mhvtl/kernel/fetch27.c: In function 'fetch_to_dev_buffer':
/root/mhvtl/kernel/fetch27.c:96:33: error: implicit declaration of function 'scsi_out'; did             you mean 'scsi_ioctl'? [-Werror=implicit-function-declaration]
  struct scsi_data_buffer *sdb = scsi_out(scp);
                                 ^~~~~~~~
                                 scsi_ioctl
/root/mhvtl/kernel/fetch27.c:96:33: warning: initialization of 'struct scsi_data_buffer *'             from 'int' makes pointer from integer without a cast [-Wint-conversion]
/root/mhvtl/kernel/fetch27.c:100:8: error: implicit declaration of function 'scsi_bidi_cmn             '; did you mean 'scsi_ioctl'? [-Werror=implicit-function-declaration]
  if (!(scsi_bidi_cmnd(scp) || scp->sc_data_direction == DMA_TO_DEVICE))
        ^~~~~~~~~~~~~~
        scsi_ioctl
/root/mhvtl/kernel/fetch27.c: In function 'fill_from_user_buffer':
/root/mhvtl/kernel/fetch27.c:116:33: error: implicit declaration of function 'scsi_in'; did             you mean 'scsi_req'? [-Werror=implicit-function-declaration]
  struct scsi_data_buffer *sdb = scsi_in(scp);
                                 ^~~~~~~
                                 scsi_req
/root/mhvtl/kernel/fetch27.c:116:33: warning: initialization of 'struct scsi_data_buffer *'             from 'int' makes pointer from integer without a cast [-Wint-conversion]
/root/mhvtl/kernel/fetch27.c: In function 'fill_from_dev_buffer':
/root/mhvtl/kernel/fetch27.c:139:33: warning: initialization of 'struct scsi_data_buffer *'             from 'int' makes pointer from integer without a cast [-Wint-conversion]
  struct scsi_data_buffer *sdb = scsi_in(scp);
                                 ^~~~~~~
cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:322: /root/mhvtl/kernel/mhvtl.o] Error 1
make[1]: *** [Makefile:1544: _module_/root/mhvtl/kernel] Error 2
make[1]: Leaving directory '/usr/src/kernels/4.18.0-240.el8.x86_64'
make: *** [Makefile:31: default] Error 2
#

Looking into the change log of the kernel packege, some scsi related patches has been backported.

# rpm -q --changelog kernel | awk '/^*/{l=$0;f=1}/1822001/{if(f){print l;f=0}print $0}'
* Tue May 12 2020 Frantisek Hrbata <[email protected]> [4.18.0-198.el8]
- [scsi] scsi: scsi_debug: Fix a recently introduced regression (Ming Lei) [1822001]
- [block] scsi: block: remove bidi support (Ming Lei) [1822001]
- [scsi] scsi: block: remove req->special (Ming Lei) [1822001]
- [scsi] scsi: stop setting up request->special (Ming Lei) [1822001]
- [scsi] scsi: remove bidirectional command support (Ming Lei) [1822001]
- [scsi] scsi: remove the SCSI OSD library (Ming Lei) [1822001]
- [fs] scsi: fs: remove exofs (Ming Lei) [1822001]
- [block] scsi: bsg-lib: handle bidi requests without block layer help (Ming Lei) [1822001]
- [block] scsi: bsg: refactor bsg_ioctl (Ming Lei) [1822001]
# git log -1 -p ae3d56d81507c33024ba7c1eae2ef433aa9bc0d5
commit ae3d56d81507c33024ba7c1eae2ef433aa9bc0d5
Author: Christoph Hellwig <[email protected]>
Date:   Tue Jan 29 09:33:07 2019 +0100


    scsi: remove bidirectional command support

    No real need for bidi support once the OSD code is gone.

    Signed-off-by: Christoph Hellwig <[email protected]>
    Reviewed-by: Jens Axboe <[email protected]>
    Signed-off-by: Martin K. Petersen <[email protected]>
...<snip>...
diff --git a/include/scsi/scsi_cmnd.h b/include/scsi/scsi_cmnd.h
index d85e6be..af6ae3e 100644
--- a/include/scsi/scsi_cmnd.h
+++ b/include/scsi/scsi_cmnd.h
@@ -213,23 +213,6 @@ static inline int scsi_get_resid(struct scsi_cmnd *cmd)
 #define scsi_for_each_sg(cmd, sg, nseg, __i)                   \
        for_each_sg(scsi_sglist(cmd), sg, nseg, __i)

-static inline int scsi_bidi_cmnd(struct scsi_cmnd *cmd)
-{
-       return blk_bidi_rq(cmd->request) &&
-               (cmd->request->next_rq->special != NULL);
-}
-
-static inline struct scsi_data_buffer *scsi_in(struct scsi_cmnd *cmd)
-{
-       return scsi_bidi_cmnd(cmd) ?
-               cmd->request->next_rq->special : &cmd->sdb;
-}
-
-static inline struct scsi_data_buffer *scsi_out(struct scsi_cmnd *cmd)
-{
-       return &cmd->sdb;
-}
-
 static inline int scsi_sg_copy_from_buffer(struct scsi_cmnd *cmd,
                                           void *buf, int buflen)
 {

Although mhvtl has already dealt with these changes in the following commit;

# git log -1 -p e270454c
commit e270454c526fc23fab9c6c27d7ff303b8e496ac1
Author: Mark Harvey <[email protected]>
Date:   Mon Sep 23 08:07:28 2019 +1000

    mhvtl.ko: Fix compile on 5.0+ kernels

    grafted from lib/scattergather.c and drivers/scsi/scsi_debug.c
    Instead of sg_copy_{to|from}_ - I changed the buffer to be __user

    Still not sure if this is 100% legit - but seems to work and nobody
    has corrected me :)

    Signed-off-by: Mark Harvey <[email protected]>

    this original logic use the new code for 5.0+ kernels and so
    unfortunately, RHEL8.2 beta's 4.18-based kernels are out of the
    scope of this commit.

    This patch is a workaround for the 4.8-based kernels.

the current implementation doesn't work well because the following condition doesn't result in fetch50.c due to difference of the base kernel version.

#if LINUX_VERSION_CODE > KERNEL_VERSION(4, 20, 0)
 #include "fetch50.c"
#elif LINUX_VERSION_CODE > KERNEL_VERSION(2, 6, 26)
 #include "fetch27.c"
#elif LINUX_VERSION_CODE == KERNEL_VERSION(2, 6, 26)
 #include "fetch26.c"
#elif LINUX_VERSION_CODE > KERNEL_VERSION(2, 6, 23)
 #include "fetch24.c"
#else
 #include "fetch.c"
#endif

Unfortunately, even after the above the issue is resolved, there is still runtime error when I use mhvtl.

T10k sport cartridges are not implemented.

about scripts\NetBackup

Can you tell me the function about the document scripts\NetBackup?

device-conf-gen: fopen() failure not handled correctly

mhvtl/usr/mhvtl-device-conf-generator.c

Line 181 in 88816cd

if ((fp = fopen(path, "r")) == NULL) {

@gonzoleeman is this really supposed to fall through if the config file cannot be opened?

mhvtl website and GH releases

Hello,

I'm a bit puzzled, I see that mhvtl website from this GitHub repo is https://sites.google.com/site/linuxvtl2/.

But I can find this web site as well which looks not update to date
https://www.mhvtl.com/

Is there a reason to have 2 websites for the same project ?

Also, would you consider having GitHub releases ?

P.S: I'm building mhvtl using Jenkins job(s) triggered by new tag creation, having release-xxx tags would be really appreciated.

Thanks for your feedback

mhvtl in a CI environment

I've tried getting mhvtl working in a CI environment (Github Actions and Travis) for automated testing but without any luck: https://github.com/OskarPersson/mhvtl_ci

Has anyone else been able to get this working?

edit_tape function asks for -b flag, but doesn't accept it as an argument

The edit_tape helpdoc states:

Please supply a barcode (-b barcode)

Usage: edit_tape -l lib -m PCL [-s size] [-t type] [-d density] [-w on|off]
       Where 'size' is in Megabytes
             'lib' is Library number
             'type' is data | clean | WORM
             'PCL' is Physical Cartridge Label (barcode)
             '-w on|off' enable/disable write protect flag
             'density' can be on of the following:

The first line (from line 188 of edit_tape) implies that the cartridge label should be passed with -b flag, but edit_tape does not accept this.

Is the -b flag designed to allow the renaming of virtual tapes, or is this just a typo?

mhvtl 1.5.4 kernel crashes on IBM ppc64 machines

quiznos-build2_mhvtl_kernel_crash_vmcore-dmesg.txt

We have been experiencing frequent mhvtl kernel crashes on our IBM ppc64 systems. Attached is a kernel crash vmcore-dmesg.txt file.

The 'crash vmcore /usr/lib/debug/lib/modules/3.10.0-327.el7.ppc64le/vmlinux' output shows:
KERNEL: /usr/lib/debug/lib/modules/3.10.0-327.el7.ppc64le/vmlinux
DUMPFILE: vmcore [PARTIAL DUMP]
CPUS: 512
DATE: Wed Mar 15 23:32:00 2017
UPTIME: 7 days, 09:33:44
LOAD AVERAGE: 0.00, 0.01, 0.05
TASKS: 5274
NODENAME: quiznos-build2
RELEASE: 3.10.0-327.el7.ppc64le
VERSION: #1 SMP Thu Oct 29 17:31:13 EDT 2015
MACHINE: ppc64le (3026 Mhz)
MEMORY: 37 GB
PANIC: "Unable to handle kernel paging request for data at address 0xffffffffffffff70"
PID: 11978
COMMAND: "vtltape"
TASK: c0000001662e7200 [THREAD_INFO: c000000166378000]
CPU: 501
STATE: TASK_RUNNING (PANIC)

Does anyone have any ideas of what is wrong?

Thank you,
Peter

Cannot compile on 5.11

Wondering what kernel version is compatible currently?

root@dev:~/mhvtl/kernel# make
make -C /lib/modules/5.11.0-22-generic/build M=/root/mhvtl/kernel modules
make[1]: Entering directory '/usr/src/linux-headers-5.11.0-22-generic'
  CC [M]  /root/mhvtl/kernel/mhvtl.o
/root/mhvtl/kernel/mhvtl.c:347:3: error: ‘const struct file_operations’ has no member named ‘ioctl’
  347 |  .ioctl  = vtl_c_ioctl_bkl,
      |   ^~~~~
/root/mhvtl/kernel/mhvtl.c:347:12: error: positional initialization of field in ‘struct’ declared with ‘designated_init’ attribute [-Werror=designated-init]
  347 |  .ioctl  = vtl_c_ioctl_bkl,
      |            ^~~~~~~~~~~~~~~
/root/mhvtl/kernel/mhvtl.c:347:12: note: (near initialization for ‘vtl_fops’)
/root/mhvtl/kernel/mhvtl.c:347:12: error: initialization of ‘loff_t (*)(struct file *, loff_t,  int)’ {aka ‘long long int (*)(struct file *, long long int,  int)’} from incompatible pointer type ‘int (*)(struct inode *, struct file *, unsigned int,  long unsigned int)’ [-Werror=incompatible-pointer-types]
/root/mhvtl/kernel/mhvtl.c:347:12: note: (near initialization for ‘vtl_fops.llseek’)
/root/mhvtl/kernel/mhvtl.c:1695:13: warning: ‘vtl_c_ioctl’ defined but not used [-Wunused-function]
 1695 | static long vtl_c_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
      |             ^~~~~~~~~~~
cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:287: /root/mhvtl/kernel/mhvtl.o] Error 1
make[1]: *** [Makefile:1848: /root/mhvtl/kernel] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.11.0-22-generic'

mhvtl volumes losing their serial number?

On our test and development systems, we use udev rules to identify the mhvtl volumes with a udev rule file like this:

KERNEL=="st*", SUBSYSTEM=="scsi_tape", ENV{ID_SERIAL}=="350223344ab000100", SYMLINK+="abcd/mhvtl_lto-6_01"
KERNEL=="st*", SUBSYSTEM=="scsi_tape", ENV{ID_SERIAL}=="350223344ab000200", SYMLINK+="abcd/mhvtl_lto-6_02"
KERNEL=="st*", SUBSYSTEM=="scsi_tape", ENV{ID_SERIAL}=="350223344ab000300", SYMLINK+="abcd/mhvtl_lto-6_03"
KERNEL=="st*", SUBSYSTEM=="scsi_tape", ENV{ID_SERIAL}=="350223344ab000400", SYMLINK+="abcd/mhvtl_lto-6_04"
KERNEL=="st*", SUBSYSTEM=="scsi_tape", ENV{ID_SERIAL}=="350223344ab000500", SYMLINK+="abcd/mhvtl_lto-6_05"
KERNEL=="st*", SUBSYSTEM=="scsi_tape", ENV{ID_SERIAL}=="350223344ab000600", SYMLINK+="abcd/mhvtl_lto-6_06"
KERNEL=="st*", SUBSYSTEM=="scsi_tape", ENV{ID_SERIAL}=="350223344ab000700", SYMLINK+="abcd/mhvtl_lto-6_07"
KERNEL=="st*", SUBSYSTEM=="scsi_tape", ENV{ID_SERIAL}=="350223344ab000800", SYMLINK+="abcd/mhvtl_lto-6_08"
KERNEL=="st*", SUBSYSTEM=="scsi_tape", ENV{ID_SERIAL}=="350223344ab000900", SYMLINK+="abcd/mhvtl_lto-6_09"
KERNEL=="st*", SUBSYSTEM=="scsi_tape", ENV{ID_SERIAL}=="350223344ab001000", SYMLINK+="abcd/mhvtl_lto-6_10"

where abcd = a short name for our project. After we run udevadm trigger, we can list the tape volumes like this:

# ls -la /dev/abcd | grep mhvtl
However, I am finding that occasionally the mhvtl files listed under /dev/abcd are missing some entries, for example:

# ls -la /dev/abcd | grep mhvtl
lrwxrwxrwx  1 root root    6 Jun 25 03:23 mhvtl_lto-6_02 -> ../st8
lrwxrwxrwx  1 root root    6 Jun 25 03:22 mhvtl_lto-6_03 -> ../st4
lrwxrwxrwx  1 root root    6 Jun 25 03:23 mhvtl_lto-6_04 -> ../st6
lrwxrwxrwx  1 root root    6 Jun 25 03:22 mhvtl_lto-6_06 -> ../st5
lrwxrwxrwx  1 root root    6 Jun 25 03:22 mhvtl_lto-6_07 -> ../st2
lrwxrwxrwx  1 root root    6 Jun 25 03:22 mhvtl_lto-6_08 -> ../st3
lrwxrwxrwx  1 root root    6 Jun 25 03:23 mhvtl_lto-6_09 -> ../st1
lrwxrwxrwx  1 root root    6 Jun 25 03:23 mhvtl_lto-6_10 -> ../st0

Where are the links for mhvtl_lto-6_01 and mhvtl_lto-6_05?

# lsscsi |grep TD6
[2:0:1:0]    tape    IBM      ULT3580-TD6      0107  /dev/st9 
[2:0:2:0]    tape    IBM      ULT3580-TD6      0107  /dev/st8 
[2:0:3:0]    tape    IBM      ULT3580-TD6      0107  /dev/st4 
[2:0:4:0]    tape    IBM      ULT3580-TD6      0107  /dev/st6 
[2:0:5:0]    tape    IBM      ULT3580-TD6      0107  /dev/st7 
[2:0:6:0]    tape    IBM      ULT3580-TD6      0107  /dev/st5 
[2:0:7:0]    tape    IBM      ULT3580-TD6      0107  /dev/st2 
[2:0:8:0]    tape    IBM      ULT3580-TD6      0107  /dev/st3 
[2:0:9:0]    tape    IBM      ULT3580-TD6      0107  /dev/st1 
[2:0:10:0]   tape    IBM      ULT3580-TD6      0107  /dev/st0

# for f in $(lsscsi |grep TD6 | awk '{print $NF}'); do
 echo "Processing $f"; udevadm info --name=$f --query=all |grep "ID_SERIAL="
done
Processing /dev/st9
Processing /dev/st8
E: ID_SERIAL=350223344ab000200
Processing /dev/st4
E: ID_SERIAL=350223344ab000300
Processing /dev/st6
E: ID_SERIAL=350223344ab000400
Processing /dev/st7
Processing /dev/st5
E: ID_SERIAL=350223344ab000600
Processing /dev/st2
E: ID_SERIAL=350223344ab000700
Processing /dev/st3
E: ID_SERIAL=350223344ab000800
Processing /dev/st1
E: ID_SERIAL=350223344ab000900
Processing /dev/st0
E: ID_SERIAL=350223344ab001000

Here is the output from running udevadm info on /dev/st9 which was missing the ID_SERIAL value:

udevadm info ---name=/dev/st9 --query=all
P: /devices/mhvtl_pseudo/adapter0/host2/target2:0:1/2:0:1:0/scsi_tape/st9
N: st9
E: DEVNAME=/dev/st9
E: DEVPATH=/devices/mhvtl_pseudo/adapter0/host2/target2:0:1/2:0:1:0/scsi_tape/st9
E: ID_BUS=scsi
E: ID_SCSI=1
E: MAJOR=9
E: MINOR=9
E: SUBSYSTEM=scsi_tape
E: USEC_INITIALIZED=1615028020641

One without the missing value for ID_SERIAL is shown:

# udevadm info --name=/dev/st8 --query=all
P: /devices/mhvtl_pseudo/adapter0/host2/target2:0:2/2:0:2:0/scsi_tape/st8
N: st8
S: hpss/mhvtl_lto-6_02
S: tape/by-id/scsi-350223344ab000200
E: DEVLINKS=/dev/hpss/mhvtl_lto-6_02 /dev/tape/by-id/scsi-350223344ab000200
E: DEVNAME=/dev/st8
E: DEVPATH=/devices/mhvtl_pseudo/adapter0/host2/target2:0:2/2:0:2:0/scsi_tape/st8
E: ID_BUS=scsi
E: ID_MODEL=ULT3580-TD6
E: ID_MODEL_ENC=ULT3580-TD6\x20\x20\x20\x20\x20
E: ID_REVISION=0107
E: ID_SCSI=1
E: ID_SCSI_SERIAL=XYZZY_A2
E: ID_SERIAL=350223344ab000200
E: ID_SERIAL_SHORT=50223344ab000200
E: ID_TYPE=tape
E: ID_VENDOR=IBM
E: ID_VENDOR_ENC=IBM\x20\x20\x20\x20\x20
E: ID_WWN=0x50223344ab000200
E: ID_WWN_WITH_EXTENSION=0x50223344ab000200
E: MAJOR=9
E: MINOR=8
E: SUBSYSTEM=scsi_tape
E: USEC_INITIALIZED=1615027810565

Our application tries to access the tape volume with one of the udev rules (mhvtl_lto-6_01 or mhvtl_lto-6_05 in the above example.) When the udev rule is lost, our application has failures.

What causes the mhvtl scsi device to lose its information such as the serial number and the model number? Is this a bug in the mhvtl code or an issue somewhere else?

I looked at the /var/log/messages file and the last entries with st9 were:

Jun 25 03:23:02 lews kernel: st 2:0:1:0: Attached scsi tape st9
Jun 25 03:23:02 lews kernel: st 2:0:1:0: st9: try direct i/o: yes (alignment 4 B)

And here are the entries just before our application reported an issue:

Jun 25 03:28:57 lews /usr/bin/vtltape[4034858]: ssc_tur(): Test Unit Ready (2922802) ** : No, No tape loaded
Jun 25 03:28:57 lews /usr/bin/vtltape[4034858]: completeSCSICommand(): s/n: (2922802), sz: 0, sam_status: 2 [02 3a 00]
Jun 25 03:28:57 lews /usr/bin/vtltape[4034858]: CDB (2922803) (delay 805): 12 00 00 00 60 00
Jun 25 03:28:57 lews /usr/bin/vtltape[4034858]: spc_inquiry(): INQUIRY ** (2922803)
Jun 25 03:28:57 lews /usr/bin/vtltape[4034858]: completeSCSICommand(): OP s/n: (2922803), sz: 64, sam_status: 0
Jun 25 03:28:57 lews /usr/bin/vtltape[4034858]: CDB (2922804) (delay 405): 00 00 00 00 00 00
Jun 25 03:28:57 lews /usr/bin/vtltape[4034858]: return_sense(): [Key/ASC/ASCQ] [02 3a 00]
Jun 25 03:28:57 lews /usr/bin/vtltape[4034858]: ssc_tur(): Test Unit Ready (2922804) ** : No, No tape loaded
Jun 25 03:28:57 lews /usr/bin/vtltape[4034858]: completeSCSICommand(): s/n: (2922804), sz: 0, sam_status: 2 [02 3a 00]
Jun 25 03:28:57 lews /usr/bin/vtltape[4034858]: CDB (2922805) (delay 405): 4d 00 51 00 00 00 00 00 0c 00
Jun 25 03:28:57 lews /usr/bin/vtltape[4034858]: ssc_log_sense(): LOG SENSE (2922805) ** :  Unknown code: 0x11
Jun 25 03:28:57 lews /usr/bin/vtltape[4034858]: return_sense(): [Key/ASC/ASCQ] [05 24 00] 0xc0 0002
Jun 25 03:28:57 lews /usr/bin/vtltape[4034858]: completeSCSICommand(): s/n: (2922805), sz: 0, sam_status: 2 [05 24 00]

Thank you,
Peter

wrong denstity values for LTO-8

with #30 LTO-8 support was introduced. However, the LTO-8 emulation uses wrong density values. Looks like the some of the LTO-7 values were just copied for LTO-8, but LTO-8 uses different values for bits-per-mm, tracks and capacity.

Applies to both emulations, ult3580_pm.c and hp_ultrium_pm.c

See IBM Ultrium reference, chapter 5.2.24.1.2 Density information, on page 134.

Tape usage in SCSI log should be cumulative

It is "how many files in the tape" for now.

Spec for ref:
SSC-4 LP 17h: Volume statistics log parameters
"Volume Datasets Written {17h:0002h}: The total number of data sets written to the medium in the volume over the lifetime of the volume. (i.e., Total Datasets Written)"

Verify issues on IBM LTO-7 and LTO-8 that use block protection

Hi, Mark. I tested using mhvtl with block protection and verify 6 for LTO-6, LTO-7, and LTO-8 drives. The tests worked well with LTO-6, but not with LTO-7 and LTO-8. Our application encountered CRC mismatches for the latter. One of my colleagues says that LTO-6 only supported ECMA-319 but that LTO-7 and LTO-8 drives support both crc32c and ECMA-319. I think your code supports Reed Solomon or ECMA-319 but not crc32c. He said that real drives support both ECMA-319 and crc32c. Can you update mhvtl to support crc32c for the CRC check? My colleague says that crc32c allows for significantly faster verification with hardware acceleration.

Thank you,
Peter

successive load/unload cause the drive cannot load cartridge anymore

I have a script to do successive load/unload operations on a same tape drive, e.g.,

# mtx -f /dev/sg17 load  15 0; mtx -f /dev/sg17 unload  15 0; mtx -f /dev/sg17 load  15 0

After doing that, the drive cannot load cartridge anymore until mhvtl is restarted. The error message:

# mtx -f /dev/sg17 load 15 0
Loading media from Storage Element 2 into drive 0...mtx: Request Sense: Long Report=yes
mtx: Request Sense: Valid Residual=yes
mtx: Request Sense: Error Code=70 (Current)
mtx: Request Sense: Sense Key=Hardware Error
mtx: Request Sense: FileMark=no
mtx: Request Sense: EOM=no
mtx: Request Sense: ILI=no
mtx: Request Sense: Residual = 00 00 00 00
mtx: Request Sense: Additional Sense Code = 04
mtx: Request Sense: Additional Sense Qualifier = 03
mtx: Request Sense: BPV=no
mtx: Request Sense: Error in CDB=no
mtx: Request Sense: SKSV=no
MOVE MEDIUM from Element Address 33 to 480 Failed

Except adding some sleep seconds, is there any smart way to prevent the problem?

unload is asynchronous

We ran into a problem with rapid load/unload cycles on a drive. We'd rather quicky hit a 'HARDWARE ERROR' with 'manual intervention required'.

A quick investigation showed syslog interleaving the finishing of the previous unload with the next load. It looks like adding a check_tape_unload and calling it in move_drive2slot here:

mhvtl/usr/smc.c

Line 1253 in 7a78a57

like move_slot2drive does for the load side, making sure we get the 'ejected' message back from vtltape before considering the move complete.

If this solution seems workable, pushing a change is easy enough. Part of filing the issue is to make sure we aren't missing a SCSI command or something else for driving rapid load/unload cycles like this.

A reproducer script, assuming a robot at /dev/sg7.

#!/bin/bash

set -e -o pipefail
set -x

for i in $(seq 0 10); do
        logger "loop $i"
        sg_inq /dev/sg7
        sg_modes --verbose --six /dev/sg7
        logger "loading tape"
        sg_raw -v /dev/sg7 a5 00 00 01 10 00 01 00 00 00 00 00
        logger "unloading tape"
        sg_raw -v /dev/sg7 a5 00 00 01 01 00 10 00 00 00 00 00
        logger "unload finished"
done

Doest it support LTFS feature?

LTO8

Is it possible to add LTO8 support?

Linux 4.15 new timer_setup API breaks mhvtl kernel module compilation

Linux 4.15 introduced a new timer_setup API and removed init_timer.
The current mhvtl kernel module needs to be modified to be compatible with these API changes.

This is affecting fedora 27 and other recent distributions.

mhvtl kernel driver does not work with new 5.14 linux kernel

The kernel driver uses DRIVER_SENSE, but that define has been removed. Actually, the whole sense data usage has changed, so there is not a trivial (simple substitution) solution.

Output looks like:

+ make -C /usr/src/linux-obj/x86_64/default 'EXTRA_CFLAGS=-Iinclude -DMHVTL_DEBUG -DHAVE_UNLOCKED_IOCTL' modules M=/home/abuild/rpmbuild/BUILD/mhvtl-1.63_release+754.ff8861da60c9/obj/default
 make: Entering directory '/usr/src/linux-5.14.0-1-obj/x86_64/default'
  CC [M]  /home/abuild/rpmbuild/BUILD/mhvtl-1.63_release+754.ff8861da60c9/obj/default/mhvtl.o
 /home/abuild/rpmbuild/BUILD/mhvtl-1.63_release+754.ff8861da60c9/obj/default/mhvtl.c:248:18: error: 'DRIVER_SENSE' undeclared here (not in a function)
   248 |                 (DRIVER_SENSE << 24) | SAM_STAT_CHECK_CONDITION;
       |                  ^~~~~~~~~~~~
 make[2]: *** [/usr/src/linux-5.14.0-1/scripts/Makefile.build:272: /home/abuild/rpmbuild/BUILD/mhvtl-1.63_release+754.ff8861da60c9/obj/default/mhvtl.o] Error 1
 make[1]: *** [/usr/src/linux-5.14.0-1/Makefile:1865: /home/abuild/rpmbuild/BUILD/mhvtl-1.63_release+754.ff8861da60c9/obj/default] Error 2
 make: *** [../../../linux-5.14.0-1/Makefile:220: __sub-make] Error 2

Libraries installed at wrong place

Dear Mark,

I just installed mhvtl 1.6 on CentOS 7.6 x86_64. Compilation and installation worked flawlessly, but "ps aux | grep vtl" returned no processes at all. Investigating further, I discovered that /bin/vtlcmd and /bin/vtllibrary could not find libvtlscsi.so and libvtlcart.so (both present in /usr/lib), in turn preventing many VTL related services from launching. Symlinking /usr/lib/libvtlscsi.so and /usr/lib/libvtlcart.so to /lib64 solved the issue though, and everything is working fine now :)

What did I do wrong ?

Best regards,

Samuel, from CINES (Montpellier, FRANCE)
https://www.cines.fr/en/

The arm environment can not compile mhvtl

root@ubuntu18:~/mhvtl/kernel# make
./config.sh
make -C /lib/modules/5.4.0-115-generic/build M=/root/mhvtl/kernel modules
make[1]: Entering directory '/usr/src/linux-headers-5.4.0-115-generic'
  CC [M]  /root/mhvtl/kernel/mhvtl.o
/bin/sh: 1: scripts/basic/fixdep: Exec format error
scripts/Makefile.build:270: recipe for target '/root/mhvtl/kernel/mhvtl.o' failed
make[2]: *** [/root/mhvtl/kernel/mhvtl.o] Error 2
make[2]: *** Deleting file '/root/mhvtl/kernel/mhvtl.o'
Makefile:1762: recipe for target '/root/mhvtl/kernel' failed
make[1]: *** [/root/mhvtl/kernel] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.4.0-115-generic'
Makefile:30: recipe for target 'default' failed
make: *** [default] Error 2
root@ubuntu18:~/mhvtl/kernel# dpkg -l | grep kernel
ii  flash-kernel                           3.98ubuntu11~18.04.2                arm64        utility to make certain embedded devices bootable
ii  kmod                                   24-1ubuntu3.5                       arm64        tools for managing Linux kernel modules
ii  libdrm-common                          2.4.101-2~18.04.1                   all          Userspace interface to kernel DRM services -- common files
ii  libdrm2:arm64                          2.4.101-2~18.04.1                   arm64        Userspace interface to kernel DRM services -- runtime
ii  linux-firmware                         1.173.21                            all          Firmware for Linux kernel drivers
ii  linux-generic                          4.15.0.156.145                      arm64        Complete Generic Linux kernel and headers
ii  linux-headers-4.15.0-156               4.15.0-156.163                      all          Header files related to Linux kernel version 4.15.0
ii  linux-headers-4.15.0-156-generic       4.15.0-156.163                      arm64        Linux kernel headers for version 4.15.0 on ARMv8 SMP
ii  linux-headers-5.4.0-115-generic        5.4.0-115.129~18.04.1               arm64        Linux kernel headers for version 5.4.0 on ARMv8 SMP
ii  linux-headers-generic                  4.15.0.156.145                      arm64        Generic Linux kernel headers
ii  linux-hwe-5.4-headers-5.4.0-115        5.4.0-115.129~18.04.1               all          Header files related to Linux kernel version 5.4.0
ii  linux-image-4.15.0-156-generic         4.15.0-156.163                      arm64        Linux kernel image for version 4.15.0 on ARMv8 SMP
ii  linux-image-5.4.0-115-generic          5.4.0-115.129~18.04.1               arm64        Linux kernel image for version 5.4.0 on ARMv8 SMP
ii  linux-image-generic                    4.15.0.156.145                      arm64        Generic Linux kernel image
ii  linux-modules-4.15.0-156-generic       4.15.0-156.163                      arm64        Linux kernel extra modules for version 4.15.0 on ARMv8 SMP
ii  linux-modules-5.4.0-115-generic        5.4.0-115.129~18.04.1               arm64        Linux kernel extra modules for version 5.4.0 on ARMv8 SMP
ii  linux-modules-extra-4.15.0-156-generic 4.15.0-156.163                      arm64        Linux kernel extra modules for version 4.15.0 on ARMv8 SMP
ii  linux-modules-extra-5.4.0-115-generic  5.4.0-115.129~18.04.1               arm64        Linux kernel extra modules for version 5.4.0 on ARMv8 SMP
ii  rsyslog                                8.32.0-1ubuntu4                     arm64        reliable system and kernel logging daemon
root@ubuntu18:~/mhvtl/kernel# uname -i
aarch64
root@ubuntu18:~/mhvtl/kernel# ^C
root@ubuntu18:~/mhvtl/kernel# lsb
lsblk        lsb_release
root@ubuntu18:~/mhvtl/kernel# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.6 LTS
Release:        18.04
Codename:       bionic

Support for Logical Block Protection

Add support for LBP for Ultrium 7 & 8 and STK T10000

Unable to compile kernel module Ubuntu/Rocky

On newly installed Ubuntu LTS server

root@sp01:/tmp# date
Thu 10 Mar 2022 11:55:42 AM CET

root@sp01:/tmp# cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

root@sp01:/tmp# uname -a
Linux sp01 5.4.0-100-generic #113-Ubuntu SMP Thu Feb 3 18:43:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

root@sp01:/tmp# git clone https://github.com/markh794/mhvtl.git
Cloning into 'mhvtl'...
remote: Enumerating objects: 8006, done.
remote: Counting objects: 100% (590/590), done.
remote: Compressing objects: 100% (387/387), done.
remote: Total 8006 (delta 381), reused 348 (delta 200), pack-reused 7416
Receiving objects: 100% (8006/8006), 3.18 MiB | 1.61 MiB/s, done.
Resolving deltas: 100% (5761/5761), done.

root@sp01:/tmp# cd mhvtl/kernel/

root@sp01:/tmp/mhvtl/kernel# make
make -C /lib/modules/5.4.0-100-generic/build M=/tmp/mhvtl/kernel modules
make[1]: Entering directory '/usr/src/linux-headers-5.4.0-100-generic'
  CC [M]  /tmp/mhvtl/kernel/mhvtl.o
In file included from /tmp/mhvtl/kernel/mhvtl.c:95:
/tmp/mhvtl/kernel/backport.h:63:12: error: static declaration of ‘sysfs_emit’ follows non-static declaration
   63 | static int sysfs_emit(char *buf, const char *fmt, ...)
      |            ^~~~~~~~~~
In file included from ./include/linux/kobject.h:20,
                 from ./include/linux/module.h:17,
                 from /tmp/mhvtl/kernel/mhvtl.c:45:
./include/linux/sysfs.h:325:5: note: previous declaration of ‘sysfs_emit’ was here
  325 | int sysfs_emit(char *buf, const char *fmt, ...);
      |     ^~~~~~~~~~
make[2]: *** [scripts/Makefile.build:270: /tmp/mhvtl/kernel/mhvtl.o] Error 1
make[1]: *** [Makefile:1762: /tmp/mhvtl/kernel] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.4.0-100-generic'
make: *** [Makefile:26: default] Error 2

root@sp01:/tmp/mhvtl/kernel#

Same situation on Rocky Linux 8

[root@sp02 kernel]# make
make -C /lib/modules/4.18.0-348.el8.0.2.x86_64/build M=/tmp/mhvtl/kernel modules
make[1]: Entering directory '/usr/src/kernels/4.18.0-348.12.2.el8_5.x86_64'
  CC [M]  /tmp/mhvtl/kernel/mhvtl.o
In file included from /tmp/mhvtl/kernel/mhvtl.c:95:
/tmp/mhvtl/kernel/backport.h:63:12: error: static declaration of ‘sysfs_emit’ follows non-static declaration
 static int sysfs_emit(char *buf, const char *fmt, ...)
            ^~~~~~~~~~
In file included from ./include/linux/kobject.h:20,
                 from ./include/linux/module.h:17,
                 from /tmp/mhvtl/kernel/mhvtl.c:45:
./include/linux/sysfs.h:336:5: note: previous declaration of ‘sysfs_emit’ was here
 int sysfs_emit(char *buf, const char *fmt, ...);
     ^~~~~~~~~~
make[2]: *** [scripts/Makefile.build:322: /tmp/mhvtl/kernel/mhvtl.o] Error 1
make[1]: *** [Makefile:1571: _module_/tmp/mhvtl/kernel] Error 2
make[1]: Leaving directory '/usr/src/kernels/4.18.0-348.12.2.el8_5.x86_64'
make: *** [Makefile:26: default] Error 2
[root@sp02 kernel]#

kmod-mhvtl and mhvtl-utils packages available from ELRepo

Just to inform you we are adding mhvtl kernel module packages and the mhvtl helper package to ELRepo for RHEL5 and RHEL6 (and derivative distributions). You can find the SPEC files at:

I will update this ticket as soon as we have them pushed to the testing repository.

Update some part of this package?

Hi: I would like to package this project for openSUSE using their build service (OBS). But there are several issues that OBS has with the package that I'd like to fix, such as:

creating the user and group "vtl". Do we really need a completely new/separate group?
(most often the answer is no)
man pages -- several commands need man pages. I'd be glad to add ones not yet
present
setuid processes. Is this really needed? If so, no problem. But, in general, distros
are likely to put such packages through a security screening before allowing them

Are you open to pull requests, or is this package in maintenance-mode only?

And thank you for such a cool project.

Update mhvtl to have support for VERIFY_6 with vbf=1

On our project we would like mhvtl to be updated to offer support for VERIFY_6 with vbf=1. See https://www.ibm.com/support/pages/system/files/inline-files/LTO%20SCSI%20Reference%20(EXTERNAL-18Aug2021)_1.pdf

Thank you,
Peter

dump_tape and preload_tape are exactly the same binary so why build both?

It seems the new dump_tape binary uses exactly the same code as the preload_tape binary. I realize the binary behaves differently using one name or the other, but do we really need to compile both of these? My build service actually complains that these two are identical, so why in the heck aren't I using a link instead of building both?

It would be simple to submit a fix that just links preload_tape to dump_tape, if you would entertain such an optimization. While I was at it, I might combine the two man pages, as well, for bonus points.

resp_spout() - issue with pointer check?

In this line you check if a personality module has an update_encryption_mode() function before calling it:

mhvtl/usr/vtltape.c

Line 1611 in a7d72ee

if (!lu_priv->pm->update_encryption_mode)

It appears that the update_encryption_mode() function will only be called when it points to NULL. Which will no doubt cause a crash? If the pointer is initialized to the proper personality module, it will never be called, no?

带库支持

能模拟ibm 4300 与 4500这两款带库

Fix removal of flush_kernel_dcache_page() in v5.15 linux kernel

Commit:

f358afc52c30 mm: remove flush_kernel_dcache_page

Removed flush_kernel_dcache_page(). It's not clear how it should be replaced. Some of the arch-es replaced it with flush_kernel_dcache_page_addr(page_address(ARG)), but that's not export.

This will cause the mhvtl kernel module build to fail for newer kernels.

Another good time to suggest architecting this in a way that allows upstreaming of your kernel module.

Tape capacity in log page 0x31 cannot be released

Reproduce flow


sg_logs -p 0x31 |grep 'Main partition remaining capacity

mt -f /dev/nst0 rewind && mt -f /dev/nst0 weof 
sg_logs -p 0x31 |grep 'Main partition remaining capacity <= capacity does not released

mt -f /dev/nst0 rewind && mt -f /dev/nst0 wset 
sg_logs -p 0x31 |grep 'Main partition remaining capacity <= capacity does not released

mt -f /dev/nst0 rewind && mt -f /dev/nst0 erase
sg_logs -p 0x31 |grep 'Main partition remaining capacity <= capacity does not released

Installation instructions for Ubuntu Server 20.04.3

I just installed mhvtl on a new Ubuntu server image. We ran into a few speedbumps that I thought I would document for other users.

Kernel Module Instructions

sudo apt-get install sg3-utils lsscsi build-essential
git clone https://github.com/markh794/mhvtl.git
cd mhvtl/kernel
make
sudo make install

Kernel module installation is as per the INSTALL file. No Changes.

User Space instructions:

sudo apt-get install zlib1g-dev
sudo \rm /bin/sh; sudo ln -s /bin/bash /bin/sh
make
sudo make install
sudo systemctl daemon-reload
sudo systemctl enable mhvtl.target
sudo systemctl start mhvtl.target

Note the changes in the User space installation.
Step 1: The zlib-dev package is called zlib1g-dev in ubuntu-land.
Step 2: Ubuntu server now ships dash as the system shell. dash does not support [[ style command line tests and fails with a bunch of errors at the make stage. These errors are added as a post script to aid in users debugging. To fix this you need to change your system shell to bash which is what step 2 does.

@markh794 Not looking for anything to be fixed with this bug report but thought these instructions may help other users. I could add a summary of this to the INSTALL file if that is what is wanted.

sh ./generate_device_conf --force --home-dir=/opt/mhvtl --override-home
./generate_device_conf: 304: [[: not found
./generate_device_conf: 348: [[: not found
./generate_device_conf: 356: [[: not found
./generate_device_conf: 365: [[: not found
===> Generating: ./device.conf ...
./generate_device_conf: 37: 0: not found
./generate_device_conf: 268: 0: not found
./generate_device_conf: 69: 8: not found
./generate_device_conf: 224: 7: not found
./generate_device_conf: 98: 11: not found
./generate_device_conf: 224: 7: not found
./generate_device_conf: 98: 11: not found
./generate_device_conf: 202: 7: not found
./generate_device_conf: 98: 11: not found
./generate_device_conf: 202: 7: not found
./generate_device_conf: 98: 11: not found
./generate_device_conf: 285: 0: not found
./generate_device_conf: 69: 8: not found
./generate_device_conf: 246: 7: not found
./generate_device_conf: 98: 11: not found
./generate_device_conf: 246: 7: not found
./generate_device_conf: 98: 11: not found
./generate_device_conf: 246: 7: not found
./generate_device_conf: 98: 11: not found
./generate_device_conf: 246: 7: not found
./generate_device_conf: 98: 11: not found

Linux 4.15 incompat

make -C /lib/modules/4.15.0-24-generic/build SUBDIRS=/root/src/mhvtl/kernel modules
make[1]: Entering directory '/usr/src/linux-headers-4.15.0-24-generic'
  CC [M]  /root/src/mhvtl/kernel/mhvtl.o
/root/src/mhvtl/kernel/mhvtl.c: In function ‘q_cmd’:
/root/src/mhvtl/kernel/mhvtl.c:513:2: error: implicit declaration of function ‘init_timer’ [-Werror=implicit-function-declaration]
  init_timer(&sqcp->cmnd_timer);

IBM Spectrum Protect ANR8302E I/O error on drive LTO5_1 (\\.\Tape0) with volume NONE (OP=SETAPPENDONLY, Error Number=0, CC=0, rc= 2863, KEY= 05, ASC=24, ASCQ= 00, ...

I've just tried IBM Spectrum Protect Server v8.1.10.0 against

MHVTL 1.6.3-05d4f6f
MHVTL-GUI 1.5.0-cfcd117
TGT 1.0.80-6bd8382

and got this when tried to label or use volumes:

04/09/2021 08:58:51 ANR8799I LABEL LIBVOLUME: Operation for library MHVTL
started as process 3. (SESSION: 4, PROCESS: 3)
04/09/2021 08:58:51 ANR9013W The RESETDRIVES parameter was not specified for
library MHVTL. For this reason, the server cannot access
tape drives that are reserved by another server in the
specified library. (SESSION: 4, PROCESS: 3)
04/09/2021 08:58:52 ANR8439I SCSI library MHVTL is ready for operations.
(SESSION: 4, PROCESS: 3)
04/09/2021 08:58:53 ANR8302E I/O error on drive LTO5_1 (\.\Tape0) with
volume NONE (OP=SETAPPENDONLY, Error Number=0, CC=0, rc
= 2863, KEY= 05, ASC=24, ASCQ= 00,
SENSE=F0.00.05.00.00.00.00.58.00.00.00.00.24.00.00.C9.00-
.01.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.0-
0.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.-
00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00-
.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.0-
0.00.00.00, Description=An undetermined error has
occurred). Refer to the IBM Spectrum Protect
documentation on I/O error code descriptions. (SESSION:
4, PROCESS: 3)
04/09/2021 08:58:53 ANR8996W The server is unable to query or enable
append-only mode on tape drive LTO5_1. (SESSION: 4,
PROCESS: 3)
04/09/2021 08:58:53 ANR8302E I/O error on drive LTO5_1 (\.\Tape0) with
volume NONE (OP=SETAPPENDONLY, Error Number=0, CC=0, rc
= 2863, KEY= 05, ASC=24, ASCQ= 00,
SENSE=F0.00.05.00.00.00.00.58.00.00.00.00.24.00.00.C9.00-
.01.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.0-
0.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.-
00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00-
.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.0-
0.00.00.00, Description=An undetermined error has
occurred). Refer to the IBM Spectrum Protect
documentation on I/O error code descriptions. (SESSION:
4, PROCESS: 3)
04/09/2021 08:58:53 ANR8996W The server is unable to query or enable
append-only mode on tape drive LTO5_1. (SESSION: 4,
PROCESS: 3)
.
.
.

Write smaller file fail silently when the cartridge is full

root@petertc-VirtualBox:/tmp# du  /opt/mhvtl/G03002TA/
535608	/opt/mhvtl/G03002TA/
root@petertc-VirtualBox:/tmp# tar cf /dev/nst4 1K
root@petertc-VirtualBox:/tmp# du  /opt/mhvtl/G03002TA/
535620	/opt/mhvtl/G03002TA/
root@petertc-VirtualBox:/tmp# mt -f /dev/nst4 bsf
root@petertc-VirtualBox:/tmp# tar xf /dev/nst4 
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors

Larger file, e.g., 1M file work as expect, i.e., show No space left on device error msg.

STK library serial number inquiry should not return trailing zero bytes

The change below fixes wrong vpd allocation of 18 bytes insted of 12.

diff --git a/usr/stklxx_pm.c b/usr/stklxx_pm.c
index 3f16e02..559b792 100644
--- a/usr/stklxx_pm.c
+++ b/usr/stklxx_pm.c
@@ -34,7 +34,7 @@ static void update_stk_l_vpd_80(struct lu_phy_attr *lu)
                *lu_vpd = NULL;
        }
 
-       *lu_vpd = alloc_vpd(0x12);
+       *lu_vpd = alloc_vpd(12);
        if (*lu_vpd) {
                d = (*lu_vpd)->data;
                /* d[4 - 15] Serial number of device */

Log Sense TAPE_CAPACITY - definition of datasets

This is a bit pedantic and will most likely not influence any backup software you throw at mhVTL. As an FYI..

Log Sense page 0x30 (Tape Usage) has a field called 'volumeDatasetsWritten'. The routine update_tape_usage() puts the number of filemarks minus one in there. This suggests that Datasets stands for the number of application data sections. That's not correct.

Datasets is defined in ECMA319 as "The smallest complete unit of information written to, or received from, the tape" Each dataset is about 400KB in size on LTO-1. So to be a bit more precise that field should contain the bytes written to disk after compression, divided by 400.000.

Feel free to close right away..

Is it time to tag a new version?

Hi Mark:

Can you tag a new version? Between all the bug fixes, enhancements, and changes for systemd, I'd love to be able update to a new tarball, which comes with tagging a new version.

Thanks.

Behavior of 'mt status' command

Hello. I am not sure how much of this is a "bug" and how much is me expecting the same behavior as my hardware tape library. Here are the issues I am noticing:

The "file number" and "block number" are reported as '-1' even after a tape is loaded. This is corrected if a "rewind" command is issued (they get set to 0).
The "block number" is not updated after a write, it just stays at 0. Using the "tapeinfo" command shows the correct "Block Position".
If a tape is unloaded and another tape is loaded, the next call to "mt status" will show the status from the last tape rather than the new tape. Doing a "mt status" between the unload and the load fixes this.

Logical block protection issues with IBM LTO-7 and LTO-8

Hi, Mark. It seems like the block protection functionality is working well for IBM LTO-6 drives. However, I am having issues when trying to use IBM LTO-7 or IBM LTO-8 drives. I am getting errors like this:

Jan 10 11:43:14 choctaw /usr/bin/vtltape[10349]: ERROR: get_lbp_crc(): CRC32C mismatch - LBP: 0x85f82ccd, calculated: 0xcd2cf885
Jan 10 11:43:14 choctaw /usr/bin/vtltape[10349]: ERROR: writeBlock_lzo(): LBP mis-compare on write : Returning E_LOGICAL_BLOCK_GUARD_FAILED

Our software is detecting an EOM (end of media) and nothing is getting written to the tape. I tried doing a test without logical block protection enabled and I was able to write to an LTO-7 tape. I have attached the vtl log entries from /var/log/messages as well as the generate_device_conf.in and generate_library_contents.in that I used.

lto7_lto8_lbp_failure.tar.gz

markh794 / mhvtl Goto Github PK

mhvtl's People

Contributors

Stargazers

Watchers

Forkers

mhvtl's Issues

Recommend Projects

Recommend Topics

Recommend Org