farthinder/proxmox-encrypted-remote-backup

Encrypt and upload Proxmox VM backups to rclone remotes

proxmox-encrypted-remote-backup's Issues

Not working within Proxmox 7 - KeyError: 'TARFILE'

Upgraded to Proxmox 7 and backup is failing with an error: INFO: KeyError: 'TARFILE'

Reason: "Hookscript: The TARFILE environment variable was deprecated in Proxmox VE 6, in favor of TARGET. In Proxmox VE 7, it has been removed entirely and thus, it is not exported to the hookscript anymore." (https://pve.proxmox.com/wiki/Roadmap)

Solution: Change ProxmoxEventHandler.py line #153 from tarfile = os.environ["TARFILE"] to tarfile = os.environ["TARGET"]

Now its working again

Script stopped working due to changed phase name

Reference: https://forum.proxmox.com/threads/info-backup-exit-code-255-vzdump-mit-hook-script.107163/#post-460678

A new backup phase 'job-init' was added to pve-manager and causes the Encrypted-Remote-Backup to stop with an error due to the new unkown phase. It worked again after adding the new phase and changing the phase 'job-start' in ProxmoxEventHandler.py. DUMPDIR is not defined in phase 'job-init' except when --dumpdir is used directly

if self.args.phase == 'job-init':

            storeid = os.environ["STOREID"]

            self.jobinfo[self.args.phase].append(
                {
                    'storeid': storeid
                }
            )

            logging.debug("Environment Variables:" + str(self.jobinfo[self.args.phase]))

            if os.path.isfile(self.jobFilePath):
                logging.error("Existing job file found: " + self.jobFilePath)
                sys.exit(1)

            with open(self.jobFilePath, 'w') as outfile:
                logging.info("Creating new job file:" + self.jobFilePath)
                json.dump(self.jobinfo, outfile)

            return self.args.phase

        elif self.args.phase == 'job-start':

            dumpdir = os.environ["DUMPDIR"]
            storeid = os.environ["STOREID"]

            logging.debug("Environment Variables:" + str(self.jobinfo[self.args.phase]))

            if os.path.isfile(self.jobFilePath) is False:
                logging.error("Existing job file not found: " + self.jobFilePath)
                sys.exit(1)

            with open(self.jobFilePath) as json_file:
                logging.info("Parsing existing job file:" + self.jobFilePath)
                self.jobinfo = json.load(json_file)


            self.jobinfo[self.args.phase] = {
                    'dumpdir': dumpdir,
                    'storeid': storeid
                }


            logging.debug("Environment Variables:" + str(self.jobinfo[self.args.phase]))

            with open(self.jobFilePath, 'w') as outfile:
                logging.info("Updating existing job file:" + self.jobFilePath)
                json.dump(self.jobinfo, outfile)

            return self.args.phase

TypeError: 'NoneType' object is not subscriptable

When I start the backup it will show the following error:

INFO: Traceback (most recent call last):
INFO:   File "/opt/scripting/encryptedRemoteBackup/ProxmoxEncryptedBackup.py", line 118, in <module>
INFO:     outputFilename = re.search("vzdump-qemu-(.*)\.\w*?$", index["tarfile"])[1]
INFO: TypeError: 'NoneType' object is not subscriptable

How do I solve the problem?

Full Log:

INFO: starting new backup job: vzdump 104 --compress lzo --script /opt/scripting/encryptedRemoteBackup/ProxmoxEncryptedBackup.py --quiet 1 --storage LocalBackup --all 0 --mailnotification failure --mode snapshot --node homerserver
INFO: CryptBackup - INFO - Encrypted BackupScript called
INFO: CryptBackup - INFO - Job File:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Phase:job-start
INFO: CryptBackup - INFO - Creating new job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: Starting Backup of VM 104 (lxc)
INFO: Backup started at 2020-04-25 22:43:46
INFO: status = running
INFO: CT Name: Pihole
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: CryptBackup - INFO - Encrypted BackupScript called
INFO: CryptBackup - INFO - Job File:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Phase:backup-start
INFO: CryptBackup - INFO - Parsing existing job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Updating existing job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Encrypted BackupScript called
INFO: CryptBackup - INFO - Job File:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Phase:pre-stop
INFO: CryptBackup - INFO - Parsing existing job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Updating existing job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: create storage snapshot 'vzdump'
  Logical volume "snap_vm-104-disk-0_vzdump" created.
INFO: CryptBackup - INFO - Encrypted BackupScript called
INFO: CryptBackup - INFO - Job File:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Phase:pre-restart
INFO: CryptBackup - INFO - Parsing existing job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Updating existing job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Encrypted BackupScript called
INFO: CryptBackup - INFO - Job File:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Phase:post-restart
INFO: CryptBackup - INFO - Parsing existing job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Updating existing job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: creating archive '/mnt/pve/LocalBackup/dump/vzdump-lxc-104-2020_04_25-22_43_46.tar.lzo'
INFO: Total bytes written: 1659904000 (1.6GiB, 140MiB/s)
INFO: archive file size: 638MB
INFO: delete old backup '/mnt/pve/LocalBackup/dump/vzdump-lxc-104-2020_04_25-22_42_30.tar.lzo'
INFO: CryptBackup - INFO - Encrypted BackupScript called
INFO: CryptBackup - INFO - Job File:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Phase:backup-end
INFO: CryptBackup - INFO - Parsing existing job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Updating existing job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: remove vzdump snapshot
  Logical volume "snap_vm-104-disk-0_vzdump" successfully removed
INFO: Finished Backup of VM 104 (00:00:13)
INFO: Backup finished at 2020-04-25 22:43:59
INFO: CryptBackup - INFO - Encrypted BackupScript called
INFO: CryptBackup - INFO - Job File:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Phase:log-end
INFO: CryptBackup - INFO - Parsing existing job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Updating existing job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Encrypted BackupScript called
INFO: CryptBackup - INFO - Job File:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Phase:job-end
INFO: CryptBackup - INFO - Parsing existing job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Updating existing job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - The backup job has ended successfully, will prepare to encrypt
INFO: Traceback (most recent call last):
INFO:   File "/opt/scripting/encryptedRemoteBackup/ProxmoxEncryptedBackup.py", line 118, in <module>
INFO:     outputFilename = re.search("vzdump-qemu-(.*)\.\w*?$", index["tarfile"])[1]
INFO: TypeError: 'NoneType' object is not subscriptable
INFO: CryptBackup - INFO - Encrypted BackupScript called
INFO: CryptBackup - INFO - Job File:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Phase:job-abort
INFO: CryptBackup - INFO - Parsing existing job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
INFO: CryptBackup - INFO - Updating job file:/opt/scripting/encryptedRemoteBackup/jobs/31315.job
ERROR: Backup job failed - command '/opt/scripting/encryptedRemoteBackup/ProxmoxEncryptedBackup.py job-end' failed: exit code 1
TASK ERROR: command '/opt/scripting/encryptedRemoteBackup/ProxmoxEncryptedBackup.py job-end' failed: exit code 1

Look in to using vzdump --stdout

This might allow for skipping the .vma and .log file and go straight to .enc.

Potential issues are

1. task log output
2. dealing with the fact it´s two files given to stdout

https://pve.proxmox.com/pve-docs/vzdump.1.html

[Feature-Request] Delete old backups

Hey,
it would be possible to add a function that deletes all backups on the remote storage that are older than 2 weeks for example.