zmanda/amanda

Amanda Network Backup

amanda's Issues

fix for infinite loop with amflush & inf tapecycle

If the amanda.conf setup has "tapecycle inf tapes", amflush will
get into an infinite loop in perl/Amanda/Amflush.pm::roll_amdump_logs():

my $days = getconf($CNF_TAPECYCLE) + 2;
for (my $i = $days-1; $i >= 1; $i--) {
    my $a = pop @files;
}

The simple patch for this is:

for (my $i = $days-1; @files && $i >= 1; $i--) {

This exists in 3.4.5 and 3.5.1, but not 3.3.4 (the versions I have on hand).

No return in nonvoid functions val_t_to_sv() and val_t_to_print()

During the build of amanda package (Amanda 3.5) for openSUSE Tumbleweed we bumped into the following error:

[ 81s] I: Program returns random data in a function
[ 81s] E: amanda no-return-in-nonvoid-function Amanda/Config.c:1848, 2261

For the current master, it means missing return value in the "fail:" block for the following functions:

static int val_t_to_sv(val_t *val, SV **results)
https://github.com/zmanda/amanda/blob/master/perl/Amanda/Config.swg#L923

static int val_t_to_print(val_t *val, SV **results)
https://github.com/zmanda/amanda/blob/master/perl/Amanda/Config.swg#L1363

Thanks for fixing it.

SSL Authentication

Amanda should have an SSL-based authentication mechanism. SSL is universally supported these days, and the BSD\* are not nearly secure enough for modern applications. The existing SSH protocol is effective, but tricky to set up (as it requires a “manual” login to exchange host keys) and has its own security implications (remote execution of shell commands using an empty passphrase).

Exclude in dumptype using app_amsamba does not work

There is a issue with "exclude" in a dumptype using app_amsamba.
Does not matter what you put in exclude this never match.
I fixed this adding the "r" for tar (smbclient internal command) in app_amsamba.

Now i can use in my dumptype:
exclude "\dir\subdir*.pdf"
exclude append "\dir\subdir\avoid_"
exclude append "\dir\subdir\table?of?values_"

? and * are the valid wildcards, ? is a good idea for spaces.

Change /usr/libexec/amanda/application/amsamba
Replace: $comm .= "X";
With: $comm .= "rX";

Add this line: return if $line =~ /^tar_re_search/;
Like this:
return if $line =~ /^tarmode is now /;
return if $line =~ /^tar_re_search/;
if ($line =~ /dumped (\d+) files and directories/) {

I tried paste a diff file but didint work here.

Wagner Caixeta Rodrigues
wagner {dot} caixeta {in} gmail {dot} com

amdump ignores diskname and uses the diskdevice

If I have something like this in disklist

MACHINE /dir {
tar
exclude "./subdir/*"
}

MACHINE /dir/subdir tar

and run

$ amdump CONF MACHINE ^/dir$
or
$ amdump CONF MACHINE =/dir

It works as expected the only backup made is /dir

But if I have something like this

MACHINE /dir {
tar
exclude "./subdir/*"
}

MACHINE /dir/subdir /dir {
tar
include "./subdir/*"
}

the commands

$ amdump CONF MACHINE ^/dir$
and
$ amdump CONF MACHINE =/dir

Don't work as expected, both backups (/dir and /dir/subdir) are made

From "man disklist"

diskname
The name of the disk (a label). In most case, you set your diskname
to the diskdevice and you don't set the diskdevice. If you want
multiple entries with the same diskdevice, you must set a different
diskname for each entry. It's the diskname that you use on the
commandline for any Amanda command. Look at the example/disklist
file for example.

So from this, if it's the diskname that is used in the command, it
seems that the reality is amdump is using the diskdevice instead!!

more info from Nathan Stratton Treadway [email protected]

My understanding of the inner workings of this part of Amanda is quite
fuzzy, but assuming I have followed the program flow behind "amdump"
correctly, it looks like the code in question is found in
server-src/diskfile.c:match_disklist() , which contains the following
lines:
if ([...] &&
(match_disk(sargv[i], dp->name) ||
(dp->device && match_disk(sargv[i], dp->device)))) {

..so I believe that in fact arguments specified on the amdump command line
are matched against BOTH the "diskname" and "diskdevice" fields, and if
either one matches the argument then that DLE is selected.... (which I
agree doesn't match the wording of the manpage....).

TRANSITION-TO-GLACIER not working as expected

I have

define changer my_s3 {
...
    device_property "TRANSITION-TO-GLACIER" "1"
...
}

and run amadmin amanda-server-archive no-reuse amanda-server-0001 but still 2 days later nothing changes on s3.

What is the command to trigger the transition?

amdump amanda-server-archive doesn't do the trick.

Add -Wshadow to gcc flags?

This option detects situations where a same name is used, either in a function declaration or in a function itself, which shadows an earlier declaration. Examples:

int c;

int f(int c) /* warning here */
{
...
}

int f2(...)
{
    char *tmp;
    if (condition) {
        char *tmp; /* warning here */
    }
}

Unfortunately, there are some warnings triggered by this already, and some of them are not obvious to fix.

Remove DEV_PREFIX in 4.0 final

The title says it all. I don't think there is any point in guessing that in the configure flags.

The point is, not only can configure miss (system configuration changes etc), but in the same vein as RDEV_PREFIX, this can be a security hole. I'd rather the sysadmin specify the devices properly.

How to use chg-manual

Starting with Amanada 3.4.0, it looks like the old Amanda Changer v1 API was removed and replaced with the v2 perl changers.

Does this mean it is not currently possible, or am I missing something?

Workaround

I currently went back to 3.3.9 to get chg-manual working.

amreport BEGIN failure

after first getting a segfault, I recompiled due to perl update in arch linux i get the following now:

Can't use 'defined(@array)' (Maybe you should just omit the defined()?) at /usr/share/perl5/site_perl/Amanda/Report/human.pm line 1203.
Compilation failed in require at (eval 13) line 1.
BEGIN failed--compilation aborted at (eval 13) line 1.

perl -v returns
This is perl 5, version 22, subversion 0 (v5.22.0) built for x86_64-linux-thread-multi

amrecover support for client-side encryption

Per the documentation:

Use amrestore to recover client-encrypted or client-custom-compressed tapes.

Given that amrestore is somewhat painfully low-level, it would be nice if amrecover supported client-side-encrypted dumps.

rpm does not include examples directory

The rpm spec does not appear to include the files in example directory, which I believe are important for amserverconfig to use?

The deb rules does appear to include this.

Reproduce

docker run -it --rm centos:7
yum install -y epel-release
yum install -y \
        ca-certificates mt-st mutt openssh-clients openssh-server gnuplot perl-JSON \
        perl-Encode-Locale gettext mailx libcurl aespipe \
        perl-Data-Dumper perl-XML-Simple \
        https://www.zmanda.com/downloads/community/Amanda/3.5.1/Redhat_Enterprise_7.0/amanda-backup_server-3.5.1-1.rhel7.x86_64.rpm
amserverconfig test2
...
amserverconfig: ERROR: copy dumptypes failed: No such file or directory

I expected to see the same template.d folder I see on debian in /var/lib/amanda so I copy them to /etc/amanda if needed

Workaround

I'll curl them from github to get the files I need.

amanda-3.5.1 : amcheck segfault

Amanda-3.5.1 on latest Gentoo Linux.

I can load and unload tapes via amtape, I can even label tapes.
But amcheck fails as well as amflush or amdump.

$ amcheck -s abt
Amanda Tape Server Host Check
-----------------------------
NOTE: Holding disk '/mnt/amhold/abt': 169 GB disk space available, using 168 GB
ERROR: amcheck-device terminated with signal 11
Server check took 1.438 seconds
(brought to you by Amanda 3.5.1)

# dmesg
amcheck-device[31798]: segfault at 8 ip 00007f23c67c56e6 sp 00007ffc8ed07488 error 4 in libc-2.27.so[7f23c6727000+1be000]

#dmesg after failing amflush
[11729.998836] taper[32221]: segfault at 8 ip 00007f6af16e46e6 sp 00007fff5120d358 error 4 in libc-2.27.so[7f6af1646000+1be000]

amdump Progress bar

When amdump is entered into the console, there is no indication of progress in backup.
Would it be possible to implement a progress bar (such as the one in wget, apt-get, etc.) that can display time remaining and percentage?

amrecover fail with utf-8 filename

all non-ASCII characters are in octal representation (\000) in the index file.
amrecover is run in the user locale, it should be able to display characters in user locale
On recovery, the application is also run in the C locale, but can it accept non-ASCII characters? some can and some can't.
That's why the patch can't be committed.
Also, it can be possible to have filename in a different locale, using the root locale to display them can make it harder.

We should add an amrecover setting that enable the translation of an octal representation to a single bytes.

There is a workaround, octal characters must be preceded by 4 backslash:
amrecover> add \303\275

Configuration parsing using GLib?

http://wiki.zmanda.com/glib-docs/glib/glib-Lexical-Scanner.html

Looks like a good replacement for the existing code, doesn't it?

Stronger SSH peer identity binding

As mentioned in the amanda-auth manual page:

When accepting an incoming conneciton, the SSH daemon gives Amanda information about the remote system in the $SSH_CONNECTION environment variable. Amanda parses this information to determine the remote address, and then performs a similar check to that done by the BSD authentications: the forward and reverse DNS entries for the remote host must match. As such, while SSH authentication can cryptographically ensure that the remote system is recognized (since it had a recognized secret key), its assurances about the remote host's identity are weaker and depend on the integrity of the DNS.

Clearly, this is not ideal, given that SSH provides us with cryptographically strong authentication. I would like to propose the following alternative:

In addition to the check already described, Amanda will also optionally accept (either in an environment variable or as a parameter to a command-line option) a hostname. Amanda will do all the validation she already does, but will also, in addition, absolutely demand that the hostname used in access control checks is equal to the one thus provided.

Of course, sometimes users can control the command line being executed via SSH. In this case, obviously there is no improvement in security since the user can provide whatever hostname they want in this option. There is no degradation, though, because Amanda should still perform all existing checks.

What this means, though, is that a security-conscious administrator has the option of locking down a specific key to a specific fixed command line in the Amanda user’s authorized_keys file, and that fixed command line can include the hostname to whom the key belongs. This prevents that host from impersonating any other host, because their key can only ever execute amandad with its own name passed in.

If this is a good idea, I could try to put together a patch, if that would help, though it might take me a while. I’d also be happy if someone else did it.

stop pushing to 'trunk' on zmanda/amanda

We're using the github IRC notifier now, and it notifies of pushes to any branch. This is OK for 3_1, 3_2, etc., but since pushes to trunk go to both the trunk and master branches on github, we see needlessly duplicated messages. I don't think there's a good reason to push to trunk anymore.

@dwlocks, I think the script that does the pushes is in Buildbot. Can you make this change?

deprecated key derivation used by amcrypt-ossl

With last versions of openssl the warning message

? data encrypt: *** WARNING : deprecated key derivation used.
? data encrypt: Using -iter or -pbkdf2 would be better.

is being produced by amcrypt-ossl.
It can be corrected by adding -pbkdf2 to the amcrypt-ossl calls to openssl:

if [ "$1" = -d ]; then
        # decrypt
        "${OPENSSL}" enc -pbkdf2 -d "-${CIPHER}" -nopad -salt -pass fd:3 3< "${PASSPHRASE}"
else
        # encrypt
        pad | "${OPENSSL}" enc -pbkdf2 -e "-${CIPHER}" -nopad -salt -pass fd:3 3< "${PASSPHRASE}"
fi

numerous rpmlint warnings binary-or-shlib-calls-gethostbyname

rpmlint checks from the opensuse build service are throwing the following warning to port code to getaddrinfo().

[ 329s] amanda.x86_64: I: binary-or-shlib-calls-gethostbyname /usr/sbin/amoldrecover
[ 329s] amanda.x86_64: I: binary-or-shlib-calls-gethostbyname /usr/sbin/amrecover
[ 329s] amanda.x86_64: I: binary-or-shlib-calls-gethostbyname /usr/lib64/amanda/libndmlib-3.3.9.so
[ 329s] The binary calls gethostbyname(). Please port the code to use getaddrinfo().

Cannot write to S3 device from recent Ubuntu; perhaps because of newer libcurl4

Summary

Writing to an S3 device from a recent Ubuntu (tried 19.10 and 20.04) does not work, amanda (version 3.5.1, tried amdump, amlabel and 'amcheck -w') just hangs and never returns. Ubuntu 18.04 (also version 3.5.1) works fine. Downgrading libcurl4 on 19.10 and 20.04 to the version of 18.04 seems to fix the problem (although I haven't tested this in production yet, also because I used a bit of force to downgrade, which I don't want to do in production).

How to reproduce

Install 'amanda-server' on an Ubuntu-20.04 instance (I used a vagrant box based on https://github.com/chef/bento/blob/master/packer_templates/ubuntu/ubuntu-20.04-amd64.json). Configure amanda to use an S3 changer device, and use new virtual tapes. Try to label a tape:

backup@vagrant:~$ amlabel test test-01 slot 1
Reading label...
Found an empty tape.
Writing label 'test-01'...
^C

This does not return (and the virtual tape does not get created). The last lines from amlabel.<timestamp>.debug are:

Wed Jun 03 11:24:48.012207191 2020: pid 14472: thd-0x562400995600: amlabel: Hdr Out: Content-MD5: lYjkchoZihkmrPD9hdJocg==
Wed Jun 03 11:24:48.012210456 2020: pid 14472: thd-0x562400995600: amlabel: Hdr Out: Content-Length: 40
Wed Jun 03 11:24:48.012213590 2020: pid 14472: thd-0x562400995600: amlabel: Hdr Out: Date: Wed, 03 Jun 2020 11:24:50 GMT
Wed Jun 03 11:24:48.012216701 2020: pid 14472: thd-0x562400995600: amlabel: Hdr Out: 

Workaround
Downgrade libcurl4. I did that as follows, but no doubt there are better ways:

• globally replace 'focal' by 'bionic' in /etc/apt/sources.list
• run apt-get update
• run apt-get install libcurl4=7.58.0-2ubuntu3.8:

# apt-get install libcurl4=7.58.0-2ubuntu3.8
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be REMOVED:
  curl pollinate ubuntu-server
The following packages will be DOWNGRADED:
  libcurl4
0 upgraded, 0 newly installed, 1 downgraded, 3 to remove and 1 not upgraded.
...

Now 'amlabel' works just fine:

backup@vagrant:~$ amlabel test test-01 slot 1
Reading label...
Found an empty tape.
Writing label 'test-01'...
Checking label...
Success!

The relevant lines from amlabel.<timestamp>.debug now read:

Wed Jun 03 12:28:14.777182823 2020: pid 15279: thd-0x55e019744600: amlabel: Hdr Out: Content-MD5: lYjkchoZihkmrPD9hdJocg==
Wed Jun 03 12:28:14.777186131 2020: pid 15279: thd-0x55e019744600: amlabel: Hdr Out: Content-Length: 40
Wed Jun 03 12:28:14.777189284 2020: pid 15279: thd-0x55e019744600: amlabel: Hdr Out: Date: Wed, 03 Jun 2020 12:28:15 GMT
Wed Jun 03 12:28:14.777192351 2020: pid 15279: thd-0x55e019744600: amlabel: Hdr Out: Expect: 100-continue
Wed Jun 03 12:28:14.777195436 2020: pid 15279: thd-0x55e019744600: amlabel: Hdr Out: 
Wed Jun 03 12:28:14.817005276 2020: pid 15279: thd-0x55e019744600: amlabel: Hdr In: HTTP/1.1 100 Continue

C and Perl are not sexy.

I have issue with too few issues.

I believe from the core of my being that backup is important. I KNOW that tape backup is cheap while effective and I know that Amanda offers a flexibility that is ABSOLUTELY invaluable. I want Amanda to progress into the 21st century tho, and a lot of that requires EG translations to Python over Perl and...smart bois looking at C so we can write either better C or ASM. Obviously, I don't think machine problems are solved by floating away from the machine. We need some bare-metal. I know there is a commercial UI....but what if it was a user UI? I only ask since I forget what the U in UI stands for...

What do people think? What would you want? What is your use case?

EDIT: user story

When restoring on a client machine, warn if potential ENOSPC

When using, for instance, amfetchdump, to restore from a backup into an arbitrary location, warn/bailout if restoring to the selected device will lead to ENOSPC.

For instance, on IRC, a case has been reported of "gzip exited with return code 1". After some quick investigation from the affected user, it appeared that ENOSPC was the cause. The fact that gzip exited with that particular exit code was unhelpful: the real cause was not found.

While each and every cause cannot be analyzed, I think it is at least possible, before starting to restore a backup to an arbitrary location, to warn the user, if he restores interactively, that restoring this backup to this location can potentially read to ENOSPC, given that:

• the client can know what space is left on the device, and
• the server can know what size is to be written to that device should the restore be ordered.

Yes, there are many corner cases, but a first, not-so-precise implementation can imho be easily achieved.

./configure should fail if SWIG is not installed

Right now it doesn't, as a result the build fails at compile time.

In config/macro_archive/ac_prog_swig.m4, we have:

AC_DEFUN([AC_PROG_SWIG],[
        AC_PATH_PROG([SWIG],[swig])
        if test -z "$SWIG" ; then
                SWIG='echo "Error: SWIG is not installed. You should look at http://www.swig.org" ; false'
        elif test -n "$1" ; then

Hmmyeah. I just think it should fail outright.

automated build tests

Does anyone plan to get automated build tests set up via travis on github?
I use that for a small project and assume this would be very helpful. Maybe I can help here.

amgtar DAR restores do not work with dumps larger than INT_MAX bytes due to use of int type

Restores (amfetchdump, amrecover) that use state-file positioning / direct access recovery and which are from dumps larger than INT_MAX bytes will not be correctly restored by amgtar because the amgtar_restore uses a signed short int type for storing the previous block.

A long int would do fine, or off_t just to be safe...

amrecover does not interpret blocksize correctly

see this thread in the ML :: https://www.mail-archive.com/[email protected]/msg50692.html

amrecover does not interpret the parameters correctly.

$ amrestore  --config abt -b 2097152 /dev/nst0 jupi smb_revision

seems to work now ... at least it starts searching.

I don't know why I have to tell that ... but it seems I have a mismatch:


tapetype says 32 kbytes:

define tapetype LTO-4 {
        comment "Created by amtapetype; compression disabled; 2017-10-31
sgw"
        length 698510208 kbytes
        filemark 0 kbytes
        speed 36696 kps
        blocksize 32 kbytes
}


changer def sets "2 mbytes":

define changer robot {
    tpchanger "chg-robot:/dev/sg1"
    property "tape-device" "0=tape:/dev/nst0"
    device-property "BLOCK_SIZE" "2 mbytes"
    device-property "READ_BLOCK_SIZE" "2 mbytes"
    property "eject-before-unload" "no"
    property "use-slots" "1-24"
    changerfile "/etc/amanda/abt/chg-robot-dev-sg1"
}


storage def pulls in both:


define storage abt {
        tapepool "abt"
        tapetype "LTO-4"
        tpchanger "robot"

[..]

}

Integrity protection for amcrypt*

amcrypt-ossl and amcrypt-ossl-asym encrypt their data, but they don’t, as far as I can tell, append any sort of MAC, signature, or other integrity protection. Therefore an attacker could modify the ciphertext without being immediately detected, and potentially induce a related change to the plaintext. In particular, to quote Wikipedia, regarding CBC mode, which both these scripts use:

Note that a one-bit change to the ciphertext causes complete corruption of the corresponding block of plaintext, and inverts the corresponding bit in the following block of plaintext, but the rest of the blocks remain intact.

So modifying the ciphertext would not merely destroy the entire plaintext in a way that is necessarily obvious. If the backup were of a filesystem containing large files, an attacker could poke at a bit of ciphertext and have a very good chance of corrupting a restored file without causing any immediately noticeable damage to e.g. the containing tar structure (and tar doesn’t appear to checksum its file data contents). As a general principle, integrity checks over encrypted data are A Good Thing™.

Document that exclude lists are included in cleartext in encrypted dumps

In the dump data on tape or vtape, at the start of each file, there is a header with a couple of plain text lines, the DLE definition in XML form, and a note for human consumption on how to restore the dump without using Amanda. This XML block contains the DLE exclude list. The header is not encrypted, even if the dump is otherwise encrypted. It might be nice to note in the documentation somewhere that the exclude list will be included in the dump data in cleartext even if the dump is encrypted, since this is not obvious and filenames could be considered sensitive data in some situations.

compile issues for arch linux

trying to build 3.3.5 or 3.3.6 there are perl related compile errors
I found this patch:
https://www.mail-archive.com/[email protected]/msg1230089.html

which solves the compile issue for me
and I'm not sure if its related but amreport is crashing with a segfault trying to get
a human readable report
and thats with the above patch

perl version for arch at this time is:
This is perl 5, version 20, subversion 0 (v5.20.0) built for x86_64-linux-thread-multi

let me know if there is any more info is needed

Don’t include the exclude list in a dump, or if it is, encrypt it

Right now, the dump details are included at the start of a vtape in cleartext, even if the dump itself is encrypted. The dump details include the exclude list, which reveals filenames. I don’t know why the exclude list needs to be stored at all, but if it does, it would be nice to encrypt it.

amrecover: can't add directories with umlaut(s)

Trying to recover a directory "Projektübersicht".

These commands (inside of amrecover) don't work as it seems not to match:

add P*
add "P*"
add "*sicht"
add *

my workaround is to restore the parent directory, it shows "Projekt\374bersicht"

Zmanda REST-API errror

Hi, I complied and installed amanda on my gnu-linux system.
When I try to run the REST-API-Server, I am getting the following error.

$amanda-rest-server start
The REST-API-PORT must be defined in the global amanda.conf (/usr/local/etc/amanda/amanda.conf) and be larger than 1024

I tried looking for the file in /usr/local/etc/amanda/ but amanda.conf does not exist. I tried adding the REST-API-PORT in the amanda-security.conf but it does not work. I even created a file called amanda.conf in the same path, but the rest server did not start.
Can someone please help me out.

amstatus: bad status on taper SHM-WRITE

From time to time I have this error when I run amstatus

amstatus: bad status on taper SHM-WRITE (dumper): 20 at /usr/lib64/perl5/vendor_perl/Amanda/Status.pm line 929, <$fd> line 443248.

amdump is still running and I can see in the logs that is writing in the Tape

amdump.20191110210001
driver: state time 149906.218 free kps: 10000000 space: 1924739276 taper: writing idle-dumpers: 20 qlen tapeq taper0: 2:0 runq: 0 directq: 0 roomq: 0 wakeup: 0 driver-idle: no-dumpers
driver: interface-state time 149906.218 if default: free 10000000
driver: hdisk-state time 149906.218 hdisk 0: free 1924739276 dumpers 0
driver: result time 149906.218 from taper0: PARTDONE worker0-0 00-00728 B00162L6 657 10485760 "[sec 358.948476 bytes 10737418240 kps 29212.437720 orig-kb 179560674]"

log.20191110210001.0
PART taper "ST:XXX06" "POOL:XXX06" B00162L6 657 xxxxx.xxx.xxxx.xx /dev/vda2 20191110210001 8/-1 3 [sec 358.948476 bytes 10737418240 kps 29212.437720 orig-kb 179560674]

PROPOSED "NEW LOGO"

Hi, I am a graphic designer.

Here I have designed a logo of the "Amanda" pembahruan. I designed it very simple and minimalist, from the logo there is a special symbol of amanda, which has the symbol "a" and the symbol "BackUp". In color I just follow the theme than Amanda, here is the final result of the logo:

I am awaiting feedback from you, and I am also willing to make improvements on request if possible.

I will provide all the original files that can be edited after assessment.

Fix for 3 bugs in amsamba

--- amsamba.original    2013-08-19 00:22:26.532825367 -0300
+++ amsamba     2013-08-19 00:23:41.271825238 -0300
@@ -160,6 +160,7 @@
            if (defined $self->{'subdir'}) {
                $_ =~ s/^\./$self->{'subdir'}/;
            }
+           $_ =~ s/([0-7]{3})/chr oct $1/eg;
            print INC_FILE "$_\n";
        }

@@ -634,7 +635,7 @@
        }
        $comm .= " tar c";
        if ($#{$self->{exclude}} >= 0) {
-           $comm .= "X";
+           $comm .= "rX";
        }
        if ($#{$self->{include}} >= 0) {
            $comm .= "I";
@@ -735,13 +736,13 @@
        debug("stderr: " . $line);
        return if $line =~ /^Domain=/;
        return if $line =~ /^tarmode is now/;
+       return if $line =~ /^tar_re_search set/;
        if ($line =~ /dumped (\d+) files and directories/) {
            $nb_files = $1;
            return;
        }
-
        # message if samba server is configured with 'security = share'
-       return if $line =~ $line =~  /Server not using user level security and no password supplied./;
+      return if $line =~ /Server not using user level security and no password supplied./;
        if ($line =~ /^Total bytes written: (\d*)/) {
            $size = $1;
            return;

+          $_ =~ s/([0-7]{3})/chr oct $1/eg;

Fix a problem when try to restore files in SMB mode with non ascii characters (like áàâãäåāăąèééêëēĕėęěìíîïìĩīĭḩóôõöōŏőùúûüũūŭůäàáâãåæçćĉčöòóôõøüùúûßéèêëýñîìíïş).

-           $comm .= "X";
+            $comm .= "rX";
+        return if $line =~ /^tar_re_search set/;

Fix a problem when try to exclude files in dumptype, it makes possible to use WILD Cards ? and *, need smbclient compiled with HAVE_REGEX_H.

-       return if $line =~ $line =~  /Server not using user level security and no password supplied./;
+      return if $line =~ /Server not using user level security and no password supplied./;

Fix problem like this one:

Unmatched ( in regex; marked by <-- HERE in m/NT_STATUS_OK opening remote file design_grafico_17a_legislatura - deputadosdep_carlosantonio??3CT VISITADOSmosaico.ai ( <-- HERE design_grafico_17a_legislatura - deputadosdep_carlosantonio??3CT VISITADOS)/ at /usr/libexec/amanda/application/amsamba line 743, line 4.]

Slashes need to be escaped to avoid \ to make part of a regex, like \2013 to be understood as octal followed by a "3".

After this fix look the real smbclient line for the error above:
NT_STATUS_OK opening remote file \design_grafico_17a_legislatura - deputados\dep_carlosantonio\2013\CT VISITADOS\mosaico.ai (\design_grafico_17a_legislatura - deputados\dep_carlosantonio\2013\CT VISITADOS)

Disable reverse DNS lookup when auth=ssh

Hi,

There's no security reason to do the reverse DNS lookup when the authentication used is SSH, because the security is based on asymmetric keys and not on the DNS resolution at all.

Please check this thread to get more details about issues with reverse DNS lookup.

In this thread, @martineau propose me a patch that is working perfectly in my case.

I think it can be great to include an option to disable the reverse DNS lookup, or maybe disable it when the auth=SSH.

Documentation unclear on whether chg-multi is deprecated

The wiki page for chg-multi has a big red banner saying chg-multi is deprecated. However, the documentation for the diskflat device, which was added very recently, also claims that chg-multi is the only way to use diskflat. This is confusing. Is chg-multi deprecated? If not, why does the documentation say it is? If so, why was a new feature added in 3.4.4 that requires its use?

master build fails in ndmp-src with undefined references

libtool: link: gcc -DNDMOS_CONST_NDMJOBLIB_REVISION="amanda-4.0.0alpha.git.d12a7613" -g -O2 -fno-strict-aliasing -o .libs/amndmjob amndmjob_main.o ndmjob_args.o ndmjob_job.o ndmjob_rules.o ndmjob_main_util.o amndma_tape_simulator.o -Wl,--export-dynamic -pthread -pthread ./.libs/libndmjob.so /export/home/orion/fedora/amanda/amanda/build/ndmp-src/.libs/libndmlib.so ../common-src/.libs/libamanda.so -lm -lgmodule-2.0 -lgobject-2.0 -lgthread-2.0 -lglib-2.0 -lnsl -lresolv -lcrypto -lcurl -pthread -Wl,-rpath -Wl,/usr/local/lib/amanda
/export/home/orion/fedora/amanda/amanda/build/ndmp-src/.libs/libndmlib.so: undefined reference to xdr_ndmp4_data_listen_reply' /export/home/orion/fedora/amanda/amanda/build/ndmp-src/.libs/libndmlib.so: undefined reference toxdr_ndmp3_config_get_auth_attr_reply'
/export/home/orion/fedora/amanda/amanda/build/ndmp-src/.libs/libndmlib.so: undefined reference to `xdr_ndmp3_config_get_butype_info_reply'
......

remove oldrecover-src/*

The only thing built from this directory is amoldrecover, which is required to communicate with a server older than 2.5.1. I think that era has well and truly passed by now, and we can safely say that a client after 3.3 has no capacity to communicate with a server older than 2.5.1.

The code in this directory is old and crusty, and not tested well. I don't think it's even compiled by buildbot, so it may not even compile anymore.

Can we kill it? Pleease‽

missing-call-to-setgroups-before-setuid /usr/sbin/amoldrecover

RPMLINT checks from the openSUSE Build Service throw the following warning.

amanda.x86_64: W: missing-call-to-setgroups-before-setuid /usr/sbin/amoldrecover
This executable is calling setuid and setgid without setgroups or initgroups.
There is a high probability this mean it didn't relinquish all groups, and
this would be a potential security issue to be fixed. Seek POS36-C on the web
for details about the problem.

remove common-src/columnar.{c,h}

To my knowledge, these are only referenced in amcheck.c, and are never actually used there. Verify that, and then remove the code (woo!)

Force encryption on the client

Doing client-side encryption is nice because it means the server can’t see the client’s data. Ideally, in combination with a locked-down command= parameter in the client’s authorized_keys file, it should be possible for the client to not really trust the server. However, right now, AFAICT, the only place you can choose whether to enable client-side encryption or not is the dumptype, which is controlled by the server. It would be nice to have an option on the client which would reject all dump requests that don’t specify a particular encryption program. Just requiring encryption alone isn’t sufficient, since the server could just ask the client to encrypt but use /bin/cat as the program.

amflock-test is failing due to invalid memory access

Using -fsanitize=address I see:

=================================================================
==23452==ERROR: AddressSanitizer: stack-buffer-overflow on address 0xffffd1e1 at pc 0xf7a82dea bp 0xffffd138 sp 0xffffcd10
READ of size 10 at 0xffffd1e1 thread T0
    #0 0xf7a82de9 in __interceptor_strlen.part.31 (/usr/lib/libasan.so.5+0x92de9)
    #1 0xf7726904 in g_strdup (/usr/lib/libglib-2.0.so.0+0x69904)
    #2 0xf7817bb2 in file_lock_write /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock.c:317
    #3 0x5655760c in inc_counter /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:115
    #4 0x56557fd4 in locking_master /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:206
    #5 0x565593bf in test_inter_proc_locking /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:350
    #6 0x56559750 in run_one_test /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/testutils.c:66
    #7 0x5655a31f in callinfork /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/testutils.c:120
    #8 0x5655a31f in testutils_run_tests /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/testutils.c:272
    #9 0x56556e19 in main /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:374
    #10 0xf7518732 in __libc_start_main (/lib/libc.so.6+0x18732)
    #11 0x56556ebf  (/home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/.libs/amflock-test+0x1ebf)

Address 0xffffd1e1 is located in stack of thread T0 at offset 33 in frame
    #0 0x565574ff in inc_counter /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:104

  This frame has 1 object(s):
    [32, 33) 'new_val' <== Memory access at offset 33 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow (/usr/lib/libasan.so.5+0x92de9) in __interceptor_strlen.part.31
Shadow bytes around the buggy address:
  0x3ffff9e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ffff9f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ffffa00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ffffa10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ffffa20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x3ffffa30: 00 00 00 00 00 00 00 00 f1 f1 f1 f1[01]f2 f2 f2
  0x3ffffa40: 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1
  0x3ffffa50: 01 f2 f2 f2 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ffffa60: f1 f1 f1 f1 00 f2 f2 f2 f2 f2 f2 f2 00 f2 f2 f2
  0x3ffffa70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ffffa80: 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==23452==ABORTING
=================================================================
==23461==ERROR: AddressSanitizer: stack-buffer-overflow on address 0xffffd1a1 at pc 0xf7a82dea bp 0xffffd0f8 sp 0xffffccd0
READ of size 10 at 0xffffd1a1 thread T0
    #0 0xf7a82de9 in __interceptor_strlen.part.31 (/usr/lib/libasan.so.5+0x92de9)
    #1 0xf7726904 in g_strdup (/usr/lib/libglib-2.0.so.0+0x69904)
    #2 0xf7817bb2 in file_lock_write /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock.c:317
    #3 0x5655760c in inc_counter /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:115
    #4 0x56557fd4 in locking_master /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:206
    #5 0x5655907a in test_intra_proc_locking /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:315
    #6 0x56559750 in run_one_test /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/testutils.c:66
    #7 0x5655a31f in callinfork /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/testutils.c:120
    #8 0x5655a31f in testutils_run_tests /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/testutils.c:272
    #9 0x56556e19 in main /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:374
    #10 0xf7518732 in __libc_start_main (/lib/libc.so.6+0x18732)
    #11 0x56556ebf  (/home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/.libs/amflock-test+0x1ebf)

Address 0xffffd1a1 is located in stack of thread T0 at offset 33 in frame
    #0 0x565574ff in inc_counter /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:104

  This frame has 1 object(s):
    [32, 33) 'new_val' <== Memory access at offset 33 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow (/usr/lib/libasan.so.5+0x92de9) in __interceptor_strlen.part.31
Shadow bytes around the buggy address:
  0x3ffff9e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ffff9f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ffffa00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ffffa10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ffffa20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x3ffffa30: f1 f1 f1 f1[01]f2 f2 f2 00 00 00 00 00 00 00 00
  0x3ffffa40: 00 00 00 00 f1 f1 f1 f1 01 f2 f2 f2 00 00 00 00
  0x3ffffa50: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 f2 f2 f2
  0x3ffffa60: f2 f2 f2 f2 00 f2 f2 f2 f2 f2 f2 f2 00 f2 f2 f2
  0x3ffffa70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ffffa80: 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==23461==ABORTING
FAIL amflock-test (exit status: 1)

Problem is that file_lock_write is called with:
char new_val;

which is a character, not terminated by zero. And thus lock->data = g_strdup(data); then fails because it's calling strlen, which is expecting a null terminated string.

Data stuck on holding disk

Amanda 3.3.9’s driver is deciding not to use a new tape even though there is data left to flush. I will attach a level-9 debug output from the taper. I have tried to understand the logic in tape_action and see why this is happening, but I don’t understand it enough to know where the problem is coming from.
driver.20160223003644.txt

Undocumented smbclient du usage

Hello Amanda developers,
Thanks for all your hard work on the Amanda backup system. It really works wonder and is relatively straightforward to setup.

Before I point out a possible problem with amsamba, I would like to emphasize that I am not using a typical setup with Amanda and smbclient. For example, I am using Amanda 3.3.3 which seems to be the current Amanda package for Ubuntu 14.04's package manager. Secondly, I am using smbclient 4.3 which is known to be incompatible with Amanda versions less than 3.3.9 (was it?).

However, that doesn't stop me. I've manually merged necessary changes (from studying github diffs) to amsamba to make 3.3.3 work properly with smbclient 4.x.

I have noticed something odd about amsamba's command_estimate() that prepares smbclient commands "archive 0|1;recurse;du". I agree that du should operate on the current working directory (specified by -D), but it doesn't in smbclient 4.3. It operates directly on the share. More specifically, if you have a Windows shared folder //server/share/subdir ... du operates on //server/share and NOT //server/share/subdir even though -D subdir was explicitly passed and your current working directory in smbclient is subdir. I have verified this by manually executing smbclient and trying du.

Furthermore, the man page documenting smbclient shows that "du" is to be passed a filename and does not appear to mention the behavior of the command in the absence of the filename. Smbclient appears to have never documented the use of "du" in the absence of an argument. Thus, I believe that amsamba is relying on undefined behavior in smbclient.

The fix is simple. You can omit the -D option entirely in command_estimate() and instead change the command to run to:
"archive 0|1;recurse;du \"/subdir\""

Even if there is no subdir, "archive 0|1;recurse;du \"/\"" will still be valid.

And all should work correctly afterward.

NT_STATUS_ACCESS_DENIED causes amdump to fail

When backing up a Windows (an actual Windows server) share //server/share, command_estimate() will fail when smbclient's "du" reports NT_STATUS_ACCESS_DENIED to \System Volume Information and $RECYCLE.BIN\folder which can never be accessed. Currently, it does not appear to be possible to backup entire Windows shares with amsamba.

smbclient 4.x has been known to have a bug where it stops recursing after encountering NT_STATUS_ACCESS_DENIED, but I have manually verified that smbclient 4.3 continues to properly recurse even in the presence of NT_STATUS_ACCESS_DENIED. It reports the total bytes and exits as expected by amsamba. However, amsamba/sendsize seem to treat anything printed to stderr as fatal even though it could get the estimate.

There is no apparent workaround. There is no way to exclude files/folders in recursions performed by smbclient du. There does not appear to be an option to silence such errors in smbclient either.

amsamba needs to discard NT_STATUS_ACCESS_DENIED messages for low level system/file system information like System Volume Information and $RECYCLE.BIN. It might also discard these errors for matching entries in the exclude list/files.

Possible error in amanda.conf include directive documentation

The amanda.conf man page says this, for the include directive:

They must start with "./" and contain no other "/".

However, unwittingly, I put an include directive in one of my DLEs which specifies a more deeply nested subdirectory (i.e. ./foo/bar), and it appears to work just fine. Is the documentation incorrect?

Drop val_t__xxx() macros

They are simply wrappers in order to access the ->v.xxx configuration values, except for two of them, which access other fields, which make them even more confusing.

What's more, theirs being usable as lvalues is touted as a feature in the header file: I call that a bad quirk.

And finally, they are too uncomfortably close in writing to the val_t_to_* function.

I say, drop them.

amreport segmentation fault

on arch linux for a while now I've been seeing a segfault with amreport
see it happening with all my configs on 2 servers
doing a backtrace on it i get the following

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7b20b24 in Perl_pp_formline ()
from /usr/lib/perl5/core_perl/CORE/libperl.so
(gdb) bt
#0 0x00007ffff7b20b24 in Perl_pp_formline ()

from /usr/lib/perl5/core_perl/CORE/libperl.so
#1 0x00007ffff7ae41f6 in Perl_runops_standard ()

from /usr/lib/perl5/core_perl/CORE/libperl.so
#2 0x00007ffff7a7536e in perl_run ()

from /usr/lib/perl5/core_perl/CORE/libperl.so
#3 0x0000000000400e39 in main ()

perl version info is
This is perl 5, version 20, subversion 0 (v5.20.0) built for x86_64-linux-thread-multi

amanda is on 3.3.6
and was compiled with that version of perl also if that matters
let me know if any more info is required