cargo-bins/reflink-copy

Cross-platform COW reflink copy of files; Fork for Binstall

reflink-copy

Cross-platform(!) COW reflink copy of files

Some file systems implement COW (copy on write) functionality in order to speed up file copies. On a high level, the new file does not actually get copied, but shares the same on-disk data with the source file. As soon as one of the files is modified, the actual copying is done by the underlying OS.

This library supports Linux, Android, OSX, ios and Windows. As soon as other OS support the functionality, support will be added. For implementation details, visit the docs.

reflink-copy's Issues

Reenter syscall on EINTR

On Unix, syscall could return EINTR due to signal delivery.

reflink does not work on new Windows DevDrives

DevDrives were announced earlier this year for Windows and are fairly widely rolled out at this point (at least for Insiders). DevDrives use ReFS under the hood, so they should ostensibly be supported by this library, but when I try and do an actual reflink, I get an Access is denied. (os error -2147024891) from reflink-copy. This error happens even when running as administrator. I can confirm that the origin file and the target location are both on the same disk, as well.

I'm slowly getting through conversations internally at Microsoft with the DevDrive team to see if it's something on our end, but I suspect this might be something about how reflink-copy is doing things? Do you have tests verifying that it works?

I don't know much about the Windows API but I'm happy to ping folks internally about stuff, and I can test patches pretty easily if someone doesn't have a Windows machine with DevDrive to test on.

Implement copy_file_range as alternative to ioctl(FICLONE)

reflink functionality got added to zfs in version 2.2.0. ioctl(FICLONE) is supported when reflinking inside a dataset, but it is possible to reflink between datasets using copy_file_range. copy_file_range will fall back to normal copy if reflink is not possible.

Is it possible to add an option to use copy_file_range when reflinking?

openzfs/zfs#405 (comment)

On btrfs in Fedora 39, reflink_src_is_symlink fails

I’m looking at adding a rust-reflink-copy package to Fedora as a dependency for https://github.com/astral-sh/uv. By default, /tmp is a tmpfs in Fedora Workstation, and the tests pass; however, if /tmp is on a btrfs partition (as it is on the builders for Fedora packages), or if I set the TMPDIR environment variable so that temporary files are created on a btrfs partition with default mount options, the reflink_src_is_symlink test fails.

$ gh repo clone cargo-bins/reflink-copy
$ cd reflink-copy
$ pwd
/home/ben/src/reflink-copy
$ mount | grep 'on /home '
/dev/nvme0n1p3 on /home type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,discard=async,space_cache,subvolid=256,subvol=/home)
$ mkdir _tmp
$ TMPDIR="${PWD}/_tmp" cargo test
[…]
failures:

---- reflink_src_is_symlink stdout ----
thread 'reflink_src_is_symlink' panicked at tests/reflink.rs:64:48:
called `Result::unwrap_err()` on an `Ok` value: ()
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[…]
$ TMPDIR="${PWD}/_tmp" RUST_BACKTRACE=1 cargo test
[…]
test reflink_src_is_symlink ... FAILED

failures:

---- reflink_src_is_symlink stdout ----
thread 'reflink_src_is_symlink' panicked at tests/reflink.rs:64:48:
called `Result::unwrap_err()` on an `Ok` value: ()
stack backtrace:
   0: rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: core::result::Result<T,E>::unwrap_err
             at /builddir/build/BUILD/rustc-1.77.0-src/library/core/src/result.rs:1167:22
   4: reflink::reflink_src_is_symlink
             at ./tests/reflink.rs:64:15
   5: reflink::reflink_src_is_symlink::{{closure}}
             at ./tests/reflink.rs:56:28
   6: core::ops::function::FnOnce::call_once
             at /builddir/build/BUILD/rustc-1.77.0-src/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
[…]

It isn’t immediately obvious to me what is going wrong here, but (with a little guidance) I would be happy to help dig into it.

fix the aarch64-apple-darwin build

It failed with process not run when testing, which I have no idea how to fix it.

function import `reflink` is private because fn reflink_not_supported is not public

I guess the intention was that on a unsupported OS such as illumos this crate should also work but return an error.

Unfortunately thats broken because the function is not public.

error[E0603]: function import `reflink` is private
  --> /var/lib/abuild/.cargo/registry/src/index.crates.io-6f17d22bba15001f/reflink-copy-0.1.12/src/sys/mod.rs:10:29
   |
10 |         pub use self::unix::reflink;
   |                             ^^^^^^^ private function import
   |
note: the function import `reflink` is defined here...
  --> /var/lib/abuild/.cargo/registry/src/index.crates.io-6f17d22bba15001f/reflink-copy-0.1.12/src/sys/unix/mod.rs:11:13
   |
11 |         use super::reflink_not_supported as reflink;
   |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: ...and refers to the function `reflink_not_supported` which is defined here
  --> /var/lib/abuild/.cargo/registry/src/index.crates.io-6f17d22bba15001f/reflink-copy-0.1.12/src/sys/mod.rs:20:1
   |
20 | fn reflink_not_supported(_from: &Path, _to: &Path) -> std::io::Result<()> {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For more information about this error, try `rustc --explain E0603`.

Enable dependabot pls

@passcod The dependabot for reflink-copy is not enabled despite having a dependabot configuration.

Compile error on android

I am trying to use this crate for a project that is meant to work on all major platforms. I am getting a compile error on android, specifically when cross compiling to aarch64-linux-android or armv7-linux-androideabi.

Any idea what that could be?

error[E0432]: unresolved import `ioctl_sys`
 --> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/reflink-copy-0.1.6/src/sys/unix/linux.rs:3:5
  |
3 | use ioctl_sys::iow;
  |     ^^^^^^^^^ use of undeclared crate or module `ioctl_sys`

error: cannot determine resolution for the macro `iow`
 --> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/reflink-copy-0.1.6/src/sys/unix/linux.rs:9:22
  |
9 | const FICLONE: u32 = iow!(0x94, 9, C_INT_SIZE);
  |                      ^^^
  |
  = note: import resolution is stuck, try simplifying macro imports

For more information about this error, try `rustc --explain E0432`.
error: could not compile `reflink-copy` (lib) due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
Error: Process completed with exit code 101.

https://github.com/n0-computer/iroh/actions/runs/5973066749/job/16204641443?pr=1405