cblichmann/btrfscue

Recover files from damaged BTRFS filesystems

btrfscue

btrfscue is an advanced data recovery tool for the BTRFS filesystem. Despite being a state of the art filesystem, at the time when I started writing this (Q2 2011), BTRFS did not have a stable fsck tool that is capable of restoring a filesystem to a mountable state after a power failure or system crash. Recently, this situation has somewhat improved with the btrfs restore command. Unlike this official tool, btrfscue is designed to be able to restore data from disk images that were obtained from faulty storage devices or if all superblocks were overwritten inadvertently.

Being a recovery tool, btrfscue works best on disk images and will write recovered data to a directory. It can thus be used to convert BTRFS filesystems to any other filesystem supported by the host OS. It will also recover recently deleted files and directories and aid in BTRFS filesystem forensics.

Table of Contents

• btrfscue
  • Development State
  • Requirements
  • Recommended Tools
  • How to Build
    • Build using Make
  • Packages
  • Usage
  • Copyright/License

Development State

As the version number 0.5 implies, this software is pretty much in alpha state. In fact, the repository you're looking at now is a complete rewrite of an earlier attempt that was written in C++ as early as 2011 (so don't let the copyright years fool you :)).

This works:

• Heuristic detection of filesystem identifiers
• Dump meta data to file
• Listing of files and directories in the metadata
• FUSE-mounting a "rescue" view of the metadata

This definitely does not work:

• Actually restoring files bigger than the filesystem block size
• Running on big-endian machines
• BTRFS RAID levels, multi-device FS. These are planned for later.

Requirements

• Go 1.18 or higher
• Git version 1.7 or later
• Optional: CDBS (to build the Debian packages)
• Optional: GNU Make

Recommended Tools

• btrfs-tools to inspect the faulty filesystem before attempting data recovery
• ddrescue to create disk images of faulty drives (download from http://www.gnu.org/software/ddrescue/ddrescue.html or, on Debian, install the package gddrescue)

How to Build

General way to build and install via go install:

go install blichmann.eu/code/btrfscue/cmd/btrfscue@latest

Build using Make

To build from a specific revision/branch/tag, not using go install:

mkdir -p btrfscue && cd btrfscue
git clone --recurse-submodules https://github.com/cblichmann/btrfscue.git .
# Optional: checkout a specific rev./branch/tag using i.e. git checkout
make

You may want to create a symlink to the binary somewhere in your path.

Packages

At the moment, only building Debian packages is supported. Just run make deb to build.

Usage

btrfscue command-line syntax is generally as follows:

btrfscue SUBCOMMAND OPTION...

Data recovery with btrfscue is divided in stages:

1. If you suspect physical damage, use a tool like ddrescue to dump the contents of the damaged filesystem to another disk. Otherwise, the standard dd utility will do just fine. The following steps assume the disk image is named DISKIMAGE. If you don't have enough physical storage space, btrfscue will also directly work with the device file. However, THIS IS NOT RECOMMENDED IN CASE OF SUSPECTED PHYSICAL DAMAGE. Although btrfscue never writes to the device, it may stress the drive too much and may render further recovery attempts impossible. This is even true of damaged SSDs since the flash controller may decide at any time to shutdown the device for good.

2. Build a list of possible ids to help identify the filesystem id for the filesystem that is to be restored by applying a heuristic. This will output a list of filesystem ids along with the number of times the respective id was found while sampling the disk image.

   btrfscue identify DISKIMAGE
   

3. Save metadata for later analysis. This may take a long time to finish as the whole image is being scanned. You need to specify the filesystem to look for by using the --id parameter with a filesystem id FSID.

   btrfscue recon --id FSID --metadata metadata.db DISKIMAGE
   

4. Inspect the metadata dump to help decide what to restore later.

   btrfscue --metadata metadata.db ls /
   ...
   

   Alternatively, if you're on Linux or macOS, you can FUSE-mount a "rescue" of the filesystem metadata:

   btrfscue --metadata metadata.db mount MOUNTPOINT
   

   Explore the metadata from another shell. Type CTRL+C to unmount.

5. Restore the actual data. This is work-in-progress. You can use the mount command to copy files that are no bigger than the filesystem block size.

Copyright/License

btrfscue version 0.6 Copyright (c)2011-2023 Christian Blichmann [email protected]

btrfscue is licensed under a two-clause BSD license, see the LICENSE file for details.

btrfscue's Issues

Help for using btrfscue to undelete some files

Could you please send me a detailed user manual or example for how to undelete some recently deleted files ？

Cannot install using `go get`

go get appears to be unsupported in the latest go. Here's a transcript of my attempt to install btrfscue:

$ go version
go version go1.20.4 linux/amd64
$ go get blichmann.eu/code/btrfscue
go: go.mod file not found in current directory or any parent directory.
        'go get' is no longer supported outside a module.
        To build and install a command, use 'go install' with a version,
        like 'go install example.com/cmd@latest'
        For more information, see https://golang.org/doc/go-get-install-deprecation
        or run 'go help get' or 'go help install'.
$ go install blichmann.eu/code/btrfscue
go: 'go install' requires a version when current directory is not in a module
        Try 'go install blichmann.eu/code/btrfscue@latest' to install the latest version
$ go install blichmann.eu/code/btrfscue@latest
go: downloading blichmann.eu/code/btrfscue v0.0.0-20220830194117-277691d03bb9
go: blichmann.eu/code/btrfscue@latest: module blichmann.eu/code/btrfscue@latest found (v0.0.0-20220830194117-277691d03bb9), but does not contain package blichmann.eu/code/btrfscue

Could we update the readme with working installation instructions for go install?

btrfscue should perform cross-validation of fs entities

The index should store a confidence value of sorts. Confidence should increase in cases where we can validate an fs entity with another.
One example is that every file extent has a corresponding extent item. If both where found during recon, chances are that the entity is valid. There are other implicit redundancies that are in the on-disk format for performance reasons.

Implement handling of non-inline file extents

Currently, only inline file data can be recovered, which is usually very small (< 4KiB). This obviously greatly limits the tool's usefulness.
Implement full support for file extents, block groups and chunks. For the simple, non-RAID case, this should be a matter of implementing the physicalToLogical() and logicalToPhysical() functions.

Implement decompression

Extent data can be optionally stored in zlib and LZO formats (per the on-disk format.
Implement support for those formats:

• zlib: the compress/zlib already supports this
• LZO: A native Go implementation is here: https://github.com/rasky/go-lzo, although not updated in the last 3 years

6 GB metadata.db but no files? :-(

root@rescue /mnt/omg # ls -lh
total 6.2G
-r--r--r-- 1 root root 6.2G Nov  9 14:09 metadata
drwxr-xr-x 1 root root  224 Sep 13 16:49 rescue
root@rescue /mnt/omg # ls rescue/
root@rescue /mnt/omg # 

Am I screwed? I'm trying to recover file names. Photorec has recovered 150 GB of data but it has no file names...

Support multiple devices (BTRFS RAID modes)

Extend the commands to support multiple devices in the various BTRFS RAID modes.

• identify - should be a simple matter of scanning the additional devices and adding the metdata to the index
• recon - similar, but we need to store on which device we found a filesystem object
• recover/mount - need to im implement complete chunk handling (see #4)

recon command crashes on slice out of range

Hi @cblichmann !

I am trying to use your tool to restore a "lost cause" BTRFS partition.
I'm getting this crash while running the command recon on the image:

# identify
root@lubuntu:/tmp# /root/go/bin/btrfscue identify /mnt/wd/Backup/dell_dd/sda5_dd.img 
 2012 / 2013 [==========================================================================================]  99.95% 13s
fsid                                 count entropy  block size
91dea238-022f-4014-8863-f96ec0c657f9 42    3.875000 16384
00000000-0000-00b2-0500-000000000000 7     0.668564 0
d2b16853-89e0-4324-bd66-015325e8d48e 5     3.875000 16384
# recon
root@lubuntu:/tmp# /root/go/bin/btrfscue recon --id 91dea238-022f-4014-8863-f96ec0c657f9 --metadata metadata.91dea238-022f-4014-8863-f96ec0c657f9.1.2.db /mnt/wd/Backup/dell_dd/sda5_dd.img 
 36.27 MB / 76.80 GB [>----------------------------------------------------------------------------------]   0.05% 1s
panic: runtime error: slice bounds out of range

goroutine 1 [running]:
blichmann.eu/code/btrfscue/btrfs.Leaf.Data(0xc4200a4000, 0x1000, 0x1000, 0x0, 0x1000, 0x2444000, 0x1000)
	/root/go/src/blichmann.eu/code/btrfscue/btrfs/leaf.go:123 +0xc6
main.(*reconCommand).Run(0xc42000a3c0, 0xc42000e140, 0x1, 0x1)
	/root/go/src/blichmann.eu/code/btrfscue/recon.go:120 +0x75e
blichmann.eu/code/btrfscue/subcommand.(*CommandSet).Run(0xc420052050, 0xc42000e140, 0x1, 0x1)
	/root/go/src/blichmann.eu/code/btrfscue/subcommand/subcommand.go:147 +0x52
blichmann.eu/code/btrfscue/subcommand.Run()
	/root/go/src/blichmann.eu/code/btrfscue/subcommand/subcommand.go:173 +0x48
main.main()
	/root/go/src/blichmann.eu/code/btrfscue/btrfscue.go:122 +0x18c

I'm not sure why this happens...
I will try to make the slice functionality (Data extraction) safer, but I don't think it's the cause of the real issue here?

LMK if there is more data I can provide.
Thanks :)

btrfscue: no device file given, only inline file data will be visible

btrfscue: no device file given, only inline file data will be visible

I type the command: btrfscue --metadata <metadata.db> mount <mountpoint>

How I can specify the device file, for read the file data?

recon crash

Show TravisCI build status prominently in README.md

btrfscue recon panic

btrfscue recon --id 50127310-d15c-49ca-8cdd-8798ea0fda2e --metadata sdc.db /dev/sdc

2000393699328 / 2000398925824 [------------------------------------------------------------------>] 100.00% 88018369 p/s
panic: runtime error: slice bounds out of range [1791164519:16384]

goroutine 1 [running]:
blichmann.eu/code/btrfscue/pkg/btrfs.Leaf.Data({0xc000190000?, 0x19?, 0x3415?}, 0xc000190065?)
/root/tmp/btrfscue/pkg/btrfs/leaf.go:128 +0xda
blichmann.eu/code/btrfscue/cmd.doScanFS({0x7fffc1366cd3?, 0x0?}, {0x0?, 0x0?}, {{0x50, 0x12, 0x73, 0x10, 0xd1, 0x5c, ...}, ...})
/root/tmp/btrfscue/cmd/recon.go:133 +0x885
blichmann.eu/code/btrfscue/cmd.init.5.func1(0xc00012d180?, {0xc000176320?, 0x5?, 0x5?})
/root/tmp/btrfscue/cmd/recon.go:61 +0xcf
github.com/spf13/cobra.(*Command).execute(0xc00012d180, {0xc0001762d0, 0x5, 0x5})
/root/go/pkg/mod/github.com/spf13/[email protected]/command.go:876 +0x67b
github.com/spf13/cobra.(*Command).ExecuteC(0x86e080)
/root/go/pkg/mod/github.com/spf13/[email protected]/command.go:990 +0x3bd
github.com/spf13/cobra.(*Command).Execute(...)
/root/go/pkg/mod/github.com/spf13/[email protected]/command.go:918
blichmann.eu/code/btrfscue/cmd.Execute()
/root/tmp/btrfscue/cmd/root.go:78 +0x25
main.main()
/root/tmp/btrfscue/cmd/btrfscue/btrfscue.go:38 +0x3b

Linux 6.1.5-229.current