danthem/undelete-btrfs

A script that attempts to undelete files in a BTRFS file system. Make sure you read the README.

undelete-btrfs

A tool for automating the generation of path regex for BTRFS restore as well as attempt the restore for you in 3 levels. The longer a file has existed prior to being deleted, the more likely it is to be recovered. This means that the script may not always work well in "test"-environments where you just create a file and then instantly try to recover it, but it should work decently on a 'real' system.

You may also end up recovering an older version of the file. The script will try to recover the most recent version but there's no guarantee the most recent recoverable version is the most recent version of the file.

Script has been tested and confirmed working with btrfs-progs version: v5.19.1 (script should also work on older versions)

Syntax

Syntax: ./undelete.sh <source dev> <recovery destination>

Example: sudo ./undelete.sh /dev/sda1 /mnt/

Note: <source dev> cannot be mounted while you run this script.

I just deleted an important file - what do I do?

• Stop what you're doing
• Umount the FS where the file was located
• Look for up-to-date backups and restore them if possible
• If no backups, attempt to run this script
  • The filesystem needs to be unmounted during this activity, if it's the root path of your OS you'll need to boot from a live USB
    • If using a live USB remember to recover to persistent storage (like an external drive or mounted network share/export) and not to the root FS of the live OS as this lives in RAM and is lost on reboot.

How to use it

When launching the script you will be asked to provide the path to the file/directory you're looking to recover. When entering the path you need to exclude the normal mountpoint for the BTRFS volume, you need to imagine that you're writing relative path from the root of the BTRFS volume itself. Here are some examples of what it should look like:

Recovery of a file:

Actual path to file: /data/Documents/bills/electric.pdf

How to write it in script: /Documents/bulls/electric.pdf

Recovery of a directory:

Actual path to directory: /data/Pictures/2017/Iceland/

How to write it in the script: /Pictures/2017/Iceland/

⚠ Pay attention to the fact that for directory recoveries the path entered must end with a slash ("/"). This tells the script that we're dealing with a directory instead of a file.

Recovery of certain extensions:

On whole volume, regardless of path: .*/.*.pdf

Within a certain directory: /documents/finance/.*.pdf

Recovery of everything:

All files on whole volume: .*

All files Within a directory: /documents/finance/.*

What is actually going on?

Well, there are comments in the code, have a read through that and see if you can make sense of it. But to keep it short and simple: The script automatically generates the somewhat awkward regex syntax required by btfs restore and then attempts three different 'depths' of recovery... It will go through them one by one, if data is found at any level the script will prompt you if the data found is what you're looking for or if you want to look deeper.

Depth?

As you run the script you will see different "depth" levels. The depth level determines how deep we dig for the data, the deeper we go the slower the recovery but also the chance for recovery increases. There are three levels of depth in the undelete-btrfs script: 0, 1 and 2.

Depth 0:

A simple btrfs restore with the regex built from the path provided.

Depth 1:

Find alternative roots using btrfs-find-roots, script loops through every root one by one looking for data matching the generated regex frm path.

Depth 2:

Same as above but the btrfs-find-roots is run with the -a flag which generates a lot more roots. This will is the slowest recovery level, it may take a long time to complete.

Regarding depth level 2: It's somewhat common that btrfs restore segfaults on roots found here, this will flood your terminal with (core-dumped)-messages but the recovery should continue as expected. Remember that depth 2 is the deepest we go and this may take a long time to complete.

Current/known limitations

• If you try to recover a directory make sure to end the path with a slash (/), otherwise you might get a match on dryrun but no files restored during recovery.
• Path and file names are CASE SENSITIVE
  • You can change this behavior by modifying script and adding -c to any btrfs restore-command

Important note

There is no native undelete feature in BTRFS. This script utilizes btrfs restore as well as btrfs-find-root to attempt recovery of deleted files on a given path. There are absolutely no guarantees it will work to recover the file. The best undeletion tool is to restore from backup. The longer a file has existed on your system, the more likely a successful recovery will be.

undelete-btrfs's Issues

btrfs-restore sometimes waits for user input

I noticed while trying to restore some huge files with lots of extents that btrfs restore would sometimes wait for user input if it looped too many times while searching the filesystem tree (or whatever it does). By sending all of btrfs restores output to /dev/null, this feedback is lost and the script apparently hangs—I had to terminate a run because the CPU usage of btrfs restore had sat at 0 for about half an hour.

feature: check if <recovery destination> exists before attempting recovery

Would be lovely to have the script verify whether the <recovery destination> exists.

This could happen either:

• before the file search commences, or
• before the recovery attempt commences

If <recovery destination> does not exist:

• offer an error message prior to attempting data recovery, or
• offer to create <recovery destination>

Currently, if <recovery destination> doesn't exist, the user only discovers the problem and requirement once the recovery process has been engaged, and throws find errors:

"Look one level deeper" jumps directly to level 2

It seems to go level 1 and then directly to level 2.

No data found on LUKS encrypted drive

No matter what I enter as the input, the program is not able to find anything. The unlocked partition (whole disk) is at /dev/mapper/luks_drive . Is this a normal limitation?

No data, of any kind, found

I'm trying to recover the directories I deleted by mistake with the rm command in the /home/user/.config directory.
When I launch the script from a bootable drive of Fedora, it can't find any data from any directory, not even the fstab file.
I've already check for right parsing of the device (which is /dev/sdb4).
Thanks.

Changes to btrfs restore log verbosity impacting script

Due to a recent change to btrfs progs we need to increase verbosity to get the output which the undelete script is looking for.

See PR #16 for details and fix.

Thanks very much

Only to say an huge thanks for doing that script saving files from users.
You deserve much bless in your path.
Have an nice week.

Detect if <dev> is mounted or not

Implement quick check to see if is mounted or not. Script can only run on umounted devices so pointless to try on one that's mounted.

Files not restored

Just recently, I had an incident where I deleted import files. I use BTRFS, so first thing I did was boot a live USB. The files appeared in BTRFS undelete (and I gave it the OK to restore), but when I mounted the drive to look for the files, they were nowhere to be found. Now BTRFS undelete can not even find them anymore to restore. My files are probably gone for good now, but I am curious why it did not save the files, I checked the live usb’s file system and the drive that the files were being restored from.

Need review / testing with newer versions of btrfs-progs

I wrote this back when 4.15 was the latest version of btrfs-progs, btrfs-progs 5.2 is the latest version at the time of writing this.

Need to test and ensure script still works well and explore if there are any new features that can improve chances of recovery.

Could not find any files

I'm using a flat btrfs layout, that is the top volume is at /dev/nvme0n1p1 and several subvolumes inside mounted to different directories, e.g:

Subvol Label	Mountpoint
@	/
@home	/home

I booted into a live CD and run the script by sudo ./undelete-btrfs /dev/nvme0n1p1 /tmp, attempted to specify the file path as /etc/fstab and /@/etc/fstab respectively, but neither of them found the missing file.

Doesn't Generate the Regex?

Hey @danthem , regardless of what i enter for path or directory it just returns: Regex generated: '^/(|$'

This doesn't seem right?

I have tried another script and it did recover the file, but my issue is that I overwrote the file in question, so would it be possible with your script working to recover an older revision of the same file?

Smarter /tmp storage

By default this script stores some temporary data in /tmp , this is not always ideal and should be changed or at least customizable. I'm thinking about storing the files in $dst instead but need to make sure to ignore them when script checks if any data found.

A bonus feature would be to keep the temporary files and if they're detected, query user if they want to use the same roots or re-generate, but this also requires some kind of tracking of depth level.

(this issue was inspired by @d0m1n1qu3 who made a fork due to /tmp running out of space)

Please allow pass btrfs restore options or set -xm as default

Please allow pass btrfs restore options or set -xm as default

-x|--xattr

get extended attributes

-m|--metadata

restore owner, mode and times for files and directories

Thanks!

I don't see a way to contact you on here, but just wanted to say thanks for saving my bacon 🥓.

The script got me out of a major hole at 1am last night!

You should consider adding a donate button...

Is this possible?: Convert /dev/sdd[1-5] Ext4 => /dev/sdd [btrfs - whole disk] - without losing data in sdd5 (old /home)

I have been using Fedora since Core 1 and still have the old HDs and I have kept using ext[234] FSs just for simplicity / consistency up to the current time (F39) - but now I want to experiment with btrfs and I was thinking I could go through the exercise of converting an old SATA boot drive where I am still using /dev/sdd5 (the old /home partition) as one of a few backup partitions / drives for current live data from my workstation and some small servers.

At first I was thinking I could delete ext4 parts 1-4, replace them with a new btrfs partition and then maybe somehow use btrfs-convert to integrate the remaining ext4 part 5 into the new part 1? . . but what I really want to do is create a new btrfs using the whole of the disk (GPT) - but somehow avoid having to spend a LONG time copying back about 4TB (to the 8TB drive) - the only way I could see that possibility working is to somehow do a recovery on the rest of the disk after the newly-created btrfs only takes up part of the beginning of the disk but leaves the old ext4 dirs and files recoverable somehow from near the beginning of the disk to just past halfway on the disk . .

This is just an interesting idea for experimenting with - it is not the end of the world if it is not possible or if it might be possible but difficult and I crash the drive in the experiment . . but I thought this script might offer a solution?