alexcrichton/tar-rs

Tar file reading/writing for Rust

tar-rs

Documentation

A tar archive reading/writing library for Rust.

# Cargo.toml
[dependencies]
tar = "0.4"

Reading an archive

extern crate tar;

use std::io::prelude::*;
use std::fs::File;
use tar::Archive;

fn main() {
    let file = File::open("foo.tar").unwrap();
    let mut a = Archive::new(file);

    for file in a.entries().unwrap() {
        // Make sure there wasn't an I/O error
        let mut file = file.unwrap();

        // Inspect metadata about the file
        println!("{:?}", file.header().path().unwrap());
        println!("{}", file.header().size().unwrap());

        // files implement the Read trait
        let mut s = String::new();
        file.read_to_string(&mut s).unwrap();
        println!("{}", s);
    }
}

Writing an archive

extern crate tar;

use std::io::prelude::*;
use std::fs::File;
use tar::Builder;

fn main() {
    let file = File::create("foo.tar").unwrap();
    let mut a = Builder::new(file);

    a.append_path("file1.txt").unwrap();
    a.append_file("file2.txt", &mut File::open("file3.txt").unwrap()).unwrap();
}

License

This project is licensed under either of

• Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

tar-rs's Issues

Single entry archives without parent directory?

I see there is an an unpack(...) method on Archive. Any chance of getting a convenient way to extract the contents of single top-level entry archives without the parent directory?

Support linux extended file attributes (like in gnu tar)

Now extended attributes causes error:

thread '

' panicked at 'called Result::unwrap() on an Err value: Error { repr: Custom(Custom { kind: Other, error: TarError { desc: "failed to unpacked ./PaxHeaders.4811/test", io: Error { repr: Custom(Custom { kind: Other, error: StringError("unknown file type 0x78") }) } } }) }', src/libcore/result.rs:746

More about extended attributes: http://linux.die.net/man/5/attr
From gnu tar man:

$ man tar | cat | egrep -A 1 "\-\-xattrs$"
       --xattrs
              Enable extended attributes support.

Environment:

$ grep ^NAME /etc/os-release
NAME="Arch Linux"
$ uname -ro
4.5.5 GNU/Linux
$ tar --version
tar (GNU tar) 1.29
$ rustc -V
rustc 1.9.0

tar-rs = "0.3"
How to reproduce error:

$ cargo new --bin tar_test
$ cd tar_test
$ mkdir test
$ head -c 10 /dev/urandom > test/file
$ setfattr -n user.pax.flags -v "epm" ./test/file
$ tar cf test.tar test --xattrs
$ rm -fr test

$ echo "extern crate tar;
use std::fs::File;
use tar::Archive;

fn main() {
    let file = File::open(\"test.tar\").unwrap();
    let mut a = Archive::new(file);
    a.unpack(\"./\").unwrap();
}" > src/main.rs

$ echo "tar = \"0.3\"" >> Cargo.toml

$ RUST_BACKTRACE=1 cargo run 
     Running `target/debug/tar_test`
thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: Error { repr: Custom(Custom { kind: Other, error: TarError { desc: "failed to unpacked `./PaxHeaders.4811/test`", io: Error { repr: Custom(Custom { kind: Other, error: StringError("unknown file type 0x78") }) } } }) }', src/libcore/result.rs:746
stack backtrace:
   1:      0x39e99e41b20 - std::sys::backtrace::tracing::imp::write::h4c73fcd3363076f5
   2:      0x39e99e445fb - std::panicking::default_hook::_$u7b$$u7b$closure$u7d$$u7d$::h0422dbb3077e6747
   3:      0x39e99e44283 - std::panicking::default_hook::haac48fa641db8fa2
   4:      0x39e99e3b16f - std::sys_common::unwind::begin_unwind_inner::h39d40f52add53ef7
   5:      0x39e99e3bbd8 - std::sys_common::unwind::begin_unwind_fmt::h64c0ff793199cc1b
   6:      0x39e99e41001 - rust_begin_unwind
   7:      0x39e99e7a66f - core::panicking::panic_fmt::h73bf9d7e8e891a73
   8:      0x39e99e27b67 - core::result::unwrap_failed::h8b8227d746f1a946
                        at src/libcore/macros.rs:29
   9:      0x39e99e28194 - _<std..result..Result<T, E>>::unwrap::hb270ae070ca117b6
                        at src/libcore/result.rs:687
  10:      0x39e99e277d9 - tar_test::main::hc9d9cb77fed05a4d
                        at src/main.rs:10
  11:      0x39e99e43ec4 - std::sys_common::unwind::try::try_fn::h09ba69fd13531e58
  12:      0x39e99e40f8b - __rust_try
  13:      0x39e99e4390b - std::rt::lang_start::h5b0863080165c75e
  14:      0x39e99e2a959 - main
  15:      0x33e3bc79740 - __libc_start_main
  16:      0x39e99e275f8 - _start
  17:                0x0 - <unknown>
error: Process didn't exit successfully: `target/debug/tar_test` (exit code: 101)

Doesn't Build with Latest rustc

$ rustc --version
rustc 0.12.0-nightly (b2d4eb186 2014-10-02 15:57:19 +0000)
$ cargo --version
cargo 0.0.1-pre-nightly (a2bbb2a 2014-10-02 20:07:47 +0000)
$ cargo build --verbose
   Compiling tar v0.0.1 (file:///Users/Max/code/rust/tar-rs)
     Running `rustc /Users/Max/code/rust/tar-rs/src/lib.rs --crate-name tar --crate-ty
pe lib -g -C metadata=5f625c2c378c3f7a -C extra-filename=-5f625c2c378c3f7a --out-dir /
Users/Max/code/rust/tar-rs/target --dep-info /Users/Max/code/rust/tar-rs/target/.finge
rprint/tar-5f625c2c378c3f7a/dep-lib-tar -L /Users/Max/code/rust/tar-rs/target -L /User
s/Max/code/rust/tar-rs/target/deps`                                                  
/Users/Max/code/rust/tar-rs/src/lib.rs:139:31: 139:72 error: type `&[u8]` does not imp
lement any method in scope named `slice_to`                                          
/Users/Max/code/rust/tar-rs/src/lib.rs:139             let prefix = path.slice_to(cmp:
:min(path.len(), prefixlen));                                                        
                                                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                         
/Users/Max/code/rust/tar-rs/src/lib.rs:140:54: 140:56 error: the type of this value mu
st be known in this context                                                          
/Users/Max/code/rust/tar-rs/src/lib.rs:140             let pos = match prefix.iter().r
position(|&b| b == b'/' || b == b'\\') {                                             

          ^~                                                                         
/Users/Max/code/rust/tar-rs/src/lib.rs:148:50: 148:69 error: type `&[u8]` does not imp
lement any method in scope named `slice_from`                                        
/Users/Max/code/rust/tar-rs/src/lib.rs:148             bytes::copy_memory(header.name,
 path.slice_from(pos + 1));                                                          

      ^~~~~~~~~~~~~~~~~~~                                                            
/Users/Max/code/rust/tar-rs/src/lib.rs:149:52: 149:65 error: type `&[u8]` does not imp
lement any method in scope named `slice_to`                                          
/Users/Max/code/rust/tar-rs/src/lib.rs:149             bytes::copy_memory(header.prefi
x, path.slice_to(pos));                                                              

        ^~~~~~~~~~~~~                                                                
/Users/Max/code/rust/tar-rs/src/lib.rs:179:19: 179:32 error: type `&[u8, .. 512]` does
 not implement any method in scope named `slice_to`                                  
/Users/Max/code/rust/tar-rs/src/lib.rs:179             bytes.slice_to(148).iter().map(
|i| *i as uint).sum() +                                                              
                                                             ^~~~~~~~~~~~~
/Users/Max/code/rust/tar-rs/src/lib.rs:179:48: 179:50 error: the type of this value mu
st be known in this context                                                          
/Users/Max/code/rust/tar-rs/src/lib.rs:179             bytes.slice_to(148).iter().map(
|i| *i as uint).sum() +                                                              

    ^~                                                                               
/Users/Max/code/rust/tar-rs/src/lib.rs:180:23: 180:38 error: type `&[u8, .. 512]` does
 not implement any method in scope named `slice_from`                                
/Users/Max/code/rust/tar-rs/src/lib.rs:180                 bytes.slice_from(156).iter(
).map(|i| *i as uint).sum() +                                                        
                                                                 ^~~~~~~~~~~~~~~
/Users/Max/code/rust/tar-rs/src/lib.rs:180:54: 180:56 error: the type of this value mu
st be known in this context                                                          
/Users/Max/code/rust/tar-rs/src/lib.rs:180                 bytes.slice_from(156).iter(
).map(|i| *i as uint).sum() +                                                        

          ^~                                                                         
/Users/Max/code/rust/tar-rs/src/lib.rs:192:32: 192:59 error: type `[<generic integer #
5>, .. 512]` does not implement any method in scope named `slice_to`                 
/Users/Max/code/rust/tar-rs/src/lib.rs:192             try!(obj.write(buf.slice_to(rem
aining as uint)));                                                                   
                                                                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~                                                                      
/Users/Max/code/rust/tar-rs/src/lib.rs:252:25: 252:38 error: type `[u8, .. 512]` does 
not implement any method in scope named `slice_to`                                   
/Users/Max/code/rust/tar-rs/src/lib.rs:252         let sum = chunk.slice_to(148).iter(
).map(|i| *i as uint).sum() +                                                        
                                                                   ^~~~~~~~~~~~~
/Users/Max/code/rust/tar-rs/src/lib.rs:252:54: 252:56 error: the type of this value must be known in this context 
/Users/Max/code/rust/tar-rs/src/lib.rs:252         let sum = chunk.slice_to(148).iter().map(|i| *i as uint).sum() +                                                        

          ^~                                                                         
/Users/Max/code/rust/tar-rs/src/lib.rs:253:25: 253:40 error: type `[u8, .. 512]` does 
not implement any method in scope named `slice_from`                                 
/Users/Max/code/rust/tar-rs/src/lib.rs:253                   chunk.slice_from(156).ite
r().map(|i| *i as uint).sum() +                                                      
                                                                   ^~~~~~~~~~~~~~~
/Users/Max/code/rust/tar-rs/src/lib.rs:253:56: 253:58 error: the type of this value mu
st be known in this context                                                          
/Users/Max/code/rust/tar-rs/src/lib.rs:253                   chunk.slice_from(156).ite
r().map(|i| *i as uint).sum() +                                                      

            ^~                                                                       
/Users/Max/code/rust/tar-rs/src/lib.rs:289:20: 289:31 error: type `[u8, .. 6]` does no
t implement any method in scope named `slice_to`                                     
/Users/Max/code/rust/tar-rs/src/lib.rs:289         self.ustar.slice_to(5) == b"ustar"
                                                              ^~~~~~~~~~~
/Users/Max/code/rust/tar-rs/src/lib.rs:413:47: 413:64 error: type `&mut [u8]` does not
 implement any method in scope named `slice_to_mut`                                  
/Users/Max/code/rust/tar-rs/src/lib.rs:413         let amt = try!(self.archive.read(in
to.slice_to_mut(amt)));                                                              

   ^~~~~~~~~~~~~~~~~                                                                 
/Users/Max/code/rust/tar-rs/src/lib.rs:459:26: 459:37 error: type `&'a [u8]` does not 
implement any method in scope named `slice_to`                                       
/Users/Max/code/rust/tar-rs/src/lib.rs:459         Some(i) => slice.slice_to(i),
                                                                    ^~~~~~~~~~~
error: aborting due to 16 previous errors
Could not compile `tar`.

Caused by:
  Process didn't exit successfully: `rustc /Users/Max/code/rust/tar-rs/src/lib.rs --cr
ate-name tar --crate-type lib -g -C metadata=5f625c2c378c3f7a -C extra-filename=-5f625
c2c378c3f7a --out-dir /Users/Max/code/rust/tar-rs/target --dep-info /Users/Max/code/ru
st/tar-rs/target/.fingerprint/tar-5f625c2c378c3f7a/dep-lib-tar -L /Users/Max/code/rust
/tar-rs/target -L /Users/Max/code/rust/tar-rs/target/deps` (status=101)              

tar-rs doesn't recover the mtime of the unpacked files

Current implementation of tar-rs does recover the file permission of the unpacked file but not for the mtime. mtime doesn't matter for most cases but sometime it does.

target:

tar-rs/src/lib.rs

Line 858 in 68c7581

return Ok(());

    fn unpack2(&mut self, dst: &Path) -> io::Result<()> {
        try!(io::copy(self, &mut try!(fs::File::create(dst))));

        // HERE will be the good position to restore mtime of the unpacked file.
        try!(set_perms(dst, try!(self.header().mode())));
        return Ok(());

        #[cfg(unix)]
        fn set_perms(dst: &Path, mode: u32) -> io::Result<()> {
            use std::os::unix::raw;

Unfortunately, the function set_file_times is unstable yet. So we may have to embed the function body of set_file_times into tar-rs just like you did with the cargo.

Option A: Use utimes for unix, and SetFileTime for windows.

👍 : I'll reuse the patch which you made before.
👎 : Additional dependencies (winapi and kernel32-sys) will be added to tar-rs for windows.

Option B: Use utime for both unix and windows.

👍 : No additional dependency.
👎 : utime is obsoleted by POSIX.1-2008.

I'm going to PR this. Do you agree with this plan? Which option do you prefer?

update docs to use version "0.4"

The docs state to add tar to your Cargo like this:

# Cargo.toml
[dependencies]
tar = "0.3"

However, 0.3 is missing several important things (like the Builder object)

Any plans on release?

I hope to release a crate that uses tar in few days. And I need symlink support, which has been added to tar a month ago. And it is not in released version yet. Is there any blockers to publishing a release?

Thanks!

7z shows extra directories in tar-rs archive

I created an archive with two files using append_path. Running 7z l gives me:

Type = tar
Physical Size = 27136
Headers Size = 2048
Code Page = UTF-8

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2016-07-12 23:59:56 .....          232          512  12741255114/Cargo.toml
2016-07-13 00:00:10 .....        24496        24576  12740666100/Cargo.lock
------------------- ----- ------------ ------------  ------------------------
2016-07-13 00:00:10              24728        25088  2 files

Doing the same on an archive created with tar (ignore the different entry order):

Type = tar
Physical Size = 30720
Headers Size = 5632
Code Page = UTF-8

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2016-07-13 00:00:10 .....        24496        24576  Cargo.lock
2016-07-12 23:59:56 .....          232          512  Cargo.toml
------------------- ----- ------------ ------------  ------------------------
2016-07-13 00:00:10              24728        25088  2 files

tar extracts the archive just fine, but it can be a hassle for 7z users, especially on Windows. I haven't tried other archiving programs.

Add support for checking stats for tar::File

Add stat() and lstat() methods for returning std::io::FileStat values for flexible checking in tar::Files.

Unable to parse Cargo.toml in rust 1.5

There seems to be issues parsing line 27 of Cargo.toml in rust 1.5. I upgraded to 1.10 and it works fine there, so not sure how much that matters. Just figured I should let you know. 😺

Allow tar::File to set the file path

I'm porting some code that uses libarchive to Rust and there is a need to replicate the function archive_set_entry_pathname(), which sets the archive entry path to the specified string in C. The entry is like the tar::File while iterating, except I think it's a pointer to the current header of the archive iterator. Being able to change the tar::File's path should accomplish this.

Please refer to the following man pages for more information:
man archive_entry
man archive_entry_paths

Symlink targets are unnecessarily checked

I think it's fine to write any symlink to the archive, i.e. an absolute path or a relative path with ...

In my case, I'm trying to pack a symlink at a/b/c which points to a/x/c. With the current code, I can put neither relative nor absolute link to point to the location. The error I get:

paths in archives must not have `..`

New release?

There are few useful changes in master but not on crates.io. A new release would be cool.

Thanks in advance.

Lifetime requirements backwards incompatibility

extern crate tar;

use std::fs;
use tar::Archive;

fn main() {
    let tnames = vec!["a", "b", "c"];

    let ars: Vec<Archive<fs::File>> = tnames.iter().map(|tname| {
        let file = fs::File::open(tname).unwrap();
        Archive::new(file)
    }).collect();
    let mut arfiless: Vec<Vec<tar::File<fs::File>>> = ars.iter().zip(tnames).map(|(ar, tname)| {
        let arfiles: Vec<_> = ar.files().unwrap().map(|res| res.unwrap()).collect();
        arfiles
    }).collect();

    let mut arheadmaps: Vec<_> =
        arfiless.iter_mut().map(|arfiles| xx(arfiles)).collect();
}

fn xx<'a, 'b: 'a>(arfiles: &'a mut Vec<tar::File<'a, fs::File>>) {

}

$ cargo build
    Updating registry `https://github.com/rust-lang/crates.io-index`
   Compiling tar v0.3.2 (file:///space/aidanhs/rust/tar-rs)
   Compiling dayer v0.1.1 (file:///space/aidanhs/rust/dayer)
src/main.rs:19:9: 19:17 error: `arfiless` does not live long enough
src/main.rs:19         arfiless.iter_mut().map(|arfiles| xx(arfiles)).collect();
                       ^~~~~~~~
src/main.rs:12:18: 20:2 note: reference must be valid for the block suffix following statement 1 at 12:17...
src/main.rs:12     }).collect();
src/main.rs:13     let mut arfiless: Vec<Vec<tar::File<fs::File>>> = ars.iter().zip(tnames).map(|(ar, tname)| {
src/main.rs:14         let arfiles: Vec<_> = ar.files().unwrap().map(|res| res.unwrap()).collect();
src/main.rs:15         arfiles
src/main.rs:16     }).collect();
src/main.rs:17 
               ...
src/main.rs:16:18: 20:2 note: ...but borrowed value is only valid for the block suffix following statement 2 at 16:17
src/main.rs:16     }).collect();
src/main.rs:17 
src/main.rs:18     let mut arheadmaps: Vec<_> =
src/main.rs:19         arfiless.iter_mut().map(|arfiles| xx(arfiles)).collect();
src/main.rs:20 }
error: aborting due to previous error
Could not compile `dayer`.

To learn more, run the command again with --verbose.

This breaks after 71b6039

Easily fixable by adding changing 'a, fs::File to 'b, fs::File on xx, but it worked before.

7z can't extract archives with long paths

    a.append_path("a directory with a reasonably long and mostly uninteresting name/").unwrap();
    a.append_file("a directory with a reasonably long and mostly uninteresting name/this is not a file name0123456789.txt", &mut File::open("file.txt").unwrap()).unwrap();

 $ 7z l foo.tar

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs Intel(R) Celeron(R) CPU  J1900  @ 1.99GHz (30678),ASM)

Scanning the drive for archives:
1 file, 3072 bytes (3 KiB)

Listing archive: foo.tar

--
Path = foo.tar
Type = tar
ERRORS:
Headers Error
WARNINGS:
There are data after the end of archive
Physical Size = 512
Tail Size = 2560
Headers Size = 512
Code Page = UTF-8

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2016-07-19 22:31:39 D....            0            0  a directory with a reasonably long and mostly uninteresting name
------------------- ----- ------------ ------------  ------------------------
2016-07-19 22:31:39                  0            0  0 files, 1 folders

Warnings: 1

Errors: 1

Dump of the said archive:

00000000  61 20 64 69 72 65 63 74  6f 72 79 20 77 69 74 68  |a directory with|
00000010  20 61 20 72 65 61 73 6f  6e 61 62 6c 79 20 6c 6f  | a reasonably lo|
00000020  6e 67 20 61 6e 64 20 6d  6f 73 74 6c 79 20 75 6e  |ng and mostly un|
00000030  69 6e 74 65 72 65 73 74  69 6e 67 20 6e 61 6d 65  |interesting name|
00000040  2f 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |/...............|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000060  00 00 00 00 30 30 30 30  37 35 35 00 30 30 30 31  |....0000755.0001|
00000070  37 35 30 00 30 30 30 31  37 35 30 00 30 30 30 30  |750.0001750.0000|
00000080  30 30 30 30 30 30 30 00  31 32 37 34 33 35 30 30  |0000000.12743500|
00000090  30 33 33 00 30 30 32 33  34 35 30 20 35 00 00 00  |033.0023450 5...|
000000a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000100  00 75 73 74 61 72 20 20  00 00 00 00 00 00 00 00  |.ustar  ........|
00000110  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000140  00 00 00 00 00 00 00 00  00 30 30 30 30 30 30 30  |.........0000000|
00000150  00 30 30 30 30 30 30 30  00 00 00 00 00 00 00 00  |.0000000........|
00000160  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000200  40 4c 6f 6e 67 4c 69 6e  6b 00 00 00 00 00 00 00  |@LongLink.......|
00000210  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000270  00 00 00 00 00 00 00 00  00 00 00 00 30 30 30 30  |............0000|
00000280  30 30 30 30 31 34 37 00  00 00 00 00 00 00 00 00  |0000147.........|
00000290  00 00 00 00 30 30 30 34  34 36 35 20 4c 00 00 00  |....0004465 L...|
000002a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000300  00 75 73 74 61 72 20 20  00 00 00 00 00 00 00 00  |.ustar  ........|
00000310  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000400  61 20 64 69 72 65 63 74  6f 72 79 20 77 69 74 68  |a directory with|
00000410  20 61 20 72 65 61 73 6f  6e 61 62 6c 79 20 6c 6f  | a reasonably lo|
00000420  6e 67 20 61 6e 64 20 6d  6f 73 74 6c 79 20 75 6e  |ng and mostly un|
00000430  69 6e 74 65 72 65 73 74  69 6e 67 20 6e 61 6d 65  |interesting name|
00000440  2f 74 68 69 73 20 69 73  20 6e 6f 74 20 61 20 66  |/this is not a f|
00000450  69 6c 65 20 6e 61 6d 65  30 31 32 33 34 35 36 37  |ile name01234567|
00000460  38 39 2e 74 78 74 00 00  00 00 00 00 00 00 00 00  |89.txt..........|
00000470  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000600  61 20 64 69 72 65 63 74  6f 72 79 20 77 69 74 68  |a directory with|
00000610  20 61 20 72 65 61 73 6f  6e 61 62 6c 79 20 6c 6f  | a reasonably lo|
00000620  6e 67 20 61 6e 64 20 6d  6f 73 74 6c 79 20 75 6e  |ng and mostly un|
00000630  69 6e 74 65 72 65 73 74  69 6e 67 20 6e 61 6d 65  |interesting name|
00000640  2f 74 68 69 73 20 69 73  20 6e 6f 74 20 61 20 66  |/this is not a f|
00000650  69 6c 65 20 6e 61 6d 65  30 31 32 33 34 35 36 37  |ile name01234567|
00000660  38 39 2e 74 30 30 30 30  36 34 34 00 30 30 30 31  |89.t0000644.0001|
00000670  37 35 30 00 30 30 30 31  37 35 30 00 30 30 30 30  |750.0001750.0000|
00000680  30 30 30 30 30 30 30 00  31 32 37 34 33 34 37 37  |0000000.12743477|
00000690  35 35 31 00 30 30 33 31  30 31 30 20 30 00 00 00  |551.0031010 0...|
000006a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000700  00 75 73 74 61 72 20 20  00 00 00 00 00 00 00 00  |.ustar  ........|
00000710  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000740  00 00 00 00 00 00 00 00  00 30 30 30 30 30 30 30  |.........0000000|
00000750  00 30 30 30 30 30 30 30  00 00 00 00 00 00 00 00  |.0000000........|
00000760  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000c00

Dump of an archive created with tar -- which 7z can open:

00000000  61 20 64 69 72 65 63 74  6f 72 79 20 77 69 74 68  |a directory with|
00000010  20 61 20 72 65 61 73 6f  6e 61 62 6c 79 20 6c 6f  | a reasonably lo|
00000020  6e 67 20 61 6e 64 20 6d  6f 73 74 6c 79 20 75 6e  |ng and mostly un|
00000030  69 6e 74 65 72 65 73 74  69 6e 67 20 6e 61 6d 65  |interesting name|
00000040  2f 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |/...............|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000060  00 00 00 00 30 30 30 30  37 35 35 00 30 30 30 31  |....0000755.0001|
00000070  37 35 30 00 30 30 30 31  37 35 30 00 30 30 30 30  |750.0001750.0000|
00000080  30 30 30 30 30 30 30 00  31 32 37 34 33 35 30 30  |0000000.12743500|
00000090  30 33 33 00 30 32 35 37  37 30 00 20 35 00 00 00  |033.025770. 5...|
000000a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000100  00 75 73 74 61 72 20 20  00 XX XX XX XX XX XX XX  |.ustar  .XXXXXXX|
00000110  XX XX 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |XX..............|
00000120  00 00 00 00 00 00 00 00  00 XX XX XX XX XX XX XX  |.........XXXXXXX|
00000130  XX XX 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |XX..............|
00000140  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000200  2e 2f 2e 2f 40 4c 6f 6e  67 4c 69 6e 6b 00 00 00  |././@LongLink...|
00000210  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000260  00 00 00 00 30 30 30 30  36 34 34 00 30 30 30 30  |....0000644.0000|
00000270  30 30 30 00 30 30 30 30  30 30 30 00 30 30 30 30  |000.0000000.0000|
00000280  30 30 30 30 32 34 36 00  30 30 30 30 30 30 30 30  |0000246.00000000|
00000290  30 30 30 00 30 31 31 36  30 35 00 20 4c 00 00 00  |000.011605. L...|
000002a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000300  00 75 73 74 61 72 20 20  00 72 6f 6f 74 00 00 00  |.ustar  .root...|
00000310  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000320  00 00 00 00 00 00 00 00  00 72 6f 6f 74 00 00 00  |.........root...|
00000330  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000400  61 20 64 69 72 65 63 74  6f 72 79 20 77 69 74 68  |a directory with|
00000410  20 61 20 72 65 61 73 6f  6e 61 62 6c 79 20 6c 6f  | a reasonably lo|
00000420  6e 67 20 61 6e 64 20 6d  6f 73 74 6c 79 20 75 6e  |ng and mostly un|
00000430  69 6e 74 65 72 65 73 74  69 6e 67 20 6e 61 6d 65  |interesting name|
00000440  2f 30 31 32 33 34 35 36  37 38 30 31 32 33 34 35  |/012345678012345|
00000450  36 37 38 30 31 32 33 34  35 36 37 38 30 31 32 33  |6780123456780123|
00000460  34 35 36 37 38 30 31 32  33 34 35 36 37 38 30 31  |4567801234567801|
00000470  32 33 34 35 36 37 38 30  31 32 33 34 35 36 37 38  |2345678012345678|
00000480  30 31 32 33 34 35 36 37  38 30 31 32 33 34 35 36  |0123456780123456|
00000490  37 38 39 39 39 39 39 39  39 39 39 30 31 32 33 34  |7899999999901234|
000004a0  35 36 37 38 39 00 00 00  00 00 00 00 00 00 00 00  |56789...........|
000004b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000600  61 20 64 69 72 65 63 74  6f 72 79 20 77 69 74 68  |a directory with|
00000610  20 61 20 72 65 61 73 6f  6e 61 62 6c 79 20 6c 6f  | a reasonably lo|
00000620  6e 67 20 61 6e 64 20 6d  6f 73 74 6c 79 20 75 6e  |ng and mostly un|
00000630  69 6e 74 65 72 65 73 74  69 6e 67 20 6e 61 6d 65  |interesting name|
00000640  2f 30 31 32 33 34 35 36  37 38 30 31 32 33 34 35  |/012345678012345|
00000650  36 37 38 30 31 32 33 34  35 36 37 38 30 31 32 33  |6780123456780123|
00000660  34 35 36 37 30 30 30 30  36 34 34 00 30 30 30 31  |45670000644.0001|
00000670  37 35 30 00 30 30 30 31  37 35 30 00 30 30 30 30  |750.0001750.0000|
00000680  30 30 30 30 30 30 30 00  31 32 37 34 33 35 30 30  |0000000.12743500|
00000690  30 33 33 00 30 33 31 34  31 30 00 20 30 00 00 00  |033.031410. 0...|
000006a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000700  00 75 73 74 61 72 20 20  00 XX XX XX XX XX XX XX  |.ustar  .XXXXXXX|
00000710  XX XX 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |XX..............|
00000720  00 00 00 00 00 00 00 00  00 XX XX XX XX XX XX XX  |.........XXXXXXX|
00000730  XX XX 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |XX..............|
00000740  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002800

Infinite entries iteration on error

There is a regression with the current 0.4 version. If, during the entries iteration an error occurs, the iteration goes on forever, returning always Some(Err(_)).

I've attached a
test file demonstrating the issue.

Minimal snippet to reproduce:

let file = File::open("loop.tar");
let mut tar = Archive::new(file);
for _entry in tar.entries() { }

The error is caused by the last entry in the tar, (which is a non-UTF8 path, but I don't think it matters). With this test:

    let file = File::open("loop.tar").unwrap();
    let mut tar = Archive::new(file);
    for entry in tar.entries().unwrap() {
        match entry {
            Ok(e) => {
                println!("{:?}", e.path());
            }
            Err(e) => {
                println!("{:?}", e);
            }
        }
    }

I can say that the paths are correctly iterated, but at the end, the error is reported forever:

...
Ok("snapshot/two_hardlinked_files1")
Ok("snapshot/two_hardlinked_files2")
Ok("snapshot/\u{62b}\u{fffd}Wb\u{fffd}\u{fffd}]\u{fffd}\u{fffd}\u{15}v*\u{fffd}\u{f}!\u{fffd}>\u{fffd}Y\u{fffd}\u{fffd}\u{fffd}\u{fffd}p\u{fffd}\u{fffd}\u{13}k\u{1d}\u{fffd}\u{fffd}\u{fffd}e\u{fffd}U\u{fffd}\u{fffd}UV\u{fffd}\u{fffd}\u{fffd}4\u{fffd}\u{fffd}X\u{3}\u{fffd}\u{7}s\u{39e}\u{fffd}\u{fffd}4\u{4}\u{fffd}\u{17} \u{fffd}\u{fffd}\u{fffd}\u{fffd}\u{62c}\u{685}\u{fffd}KvC\u{fffd}#\u{fffd}\u{fffd}\u{fffd}\u{277}\u{fffd}_\u{f}\u{fffd}g\u{fffd}B\u{11}<=^\u{fffd}M\u{13}\u{fffd}c\u{fffd}\u{fffd}|*\"/\'^$@#!(){}?+ ~` ")
Error { repr: Custom(Custom { kind: Other, error: StringError("failed to read entire block") }) }
Error { repr: Custom(Custom { kind: Other, error: StringError("failed to read entire block") }) }
...

I've done some preliminary analysis and it seems that some done flag is missing, because next_entry_raw continuously returns an Ok(Error(_)) when it is not able to read data from the header, at archive.rs: 220.

cargo toml parse error

Hi:

When referencing this project in my own Cargo.toml file I get the following errors:

Unable to update https://github.com/alexcrichton/tar-rs

Caused by:
could not parse input TOML
C:\Users\deskv.cargo\git\checkouts\tar-rs-6e41f0ca8f00e811\master\Cargo.toml:
13:18-14:0 control character \r must be escaped
C:\Users\deskv.cargo\git\checkouts\tar-rs-6e41f0ca8f00e811\master\Cargo.toml:
14:77-15:0 control character \r must be escaped
C:\Users\deskv.cargo\git\checkouts\tar-rs-6e41f0ca8f00e811\master\Cargo.toml:
15:74-16:0 control character \r must be escaped
C:\Users\deskv.cargo\git\checkouts\tar-rs-6e41f0ca8f00e811\master\Cargo.toml:
16:73-17:0 control character \r must be escaped
C:\Users\deskv.cargo\git\checkouts\tar-rs-6e41f0ca8f00e811\master\Cargo.toml:
17:75-18:0 control character \r must be escaped

Workaround: When I removed the description from your Cargo.toml (the one downloaded to my local system) the errors went away. Perhaps its a cargo bug?

Thanks.

Adding Hidden "." Directories

I'm having a hard time figuring out how to add hidden directories to the tar archive. The directory structure of the tar that I am creating must contain all files within a "." directory of the tar archive, but I am at a loss as to how to create that directory in the tar. The append_dir method doesn't seem to accept "." as a valid input for the first argument, and the second argument seems to be ignored completely.

The directory structure must be as such:

control.tar
    |--> ./
         |--> control
         |--> md5sums
         |--> etc..

This also failes to create the "." directory, as it complains that there must be at least one component in the path:

    let mut header = TarHeader::new_gnu();
    header.set_size(0);
    header.set_mode(CHMOD_BIN_OR_DIR);
    header.set_path(".").unwrap();
    header.set_entry_type(tar::EntryType::Directory);
    header.set_cksum();
    archive.append(&header, &mut io::empty()).unwrap();

std::fs::canonicalize, realpath(), musl - corner case

Hi,

I've run into a case where I could untar with glibc but not with musl. It maybe not your business as the difference is in the lower level layer, and incompatibilities between the two C library should not be handled here. (But currently I'm handling it in my application code which may be even worse;)

So anyway, fyi:

std::fs::canonicalize uses realpath() and that requires an existing /proc/self with musl but not with glibc, so to replicate the bug I've run into (with 0.4.10 but not with earlier releases) you just have to run unpack() in a musl compiled binary inside a bare chroot:

~/tmp/tar/target/x86_64-unknown-linux-musl/release$ sudo chroot . ./tar
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { repr: Os { code: 2, message: "No such file or directory" } }', /buildslave/rust-buildbot/slave/stable-dist-rustc-musl-linux/build/src/libcore/result.rs:837
note: Run with `RUST_BACKTRACE=1` for a backtrace.

Fix for symlink attack needs to support extracting symlinks pointing outside the root

Thanks for fixing the symlink attack. However, the fix in 85b6399, while addressing the vulnerability, also prevents extracting valid archives that contain a symlink pointing outside the root, even if nothing attempts to write "through" those symlinks. As an example of such a valid tarball, consider extracting the filesystem tarball from a Debian package into a temporary directory. Such a tarball could legitimately contain, for instance, a symlink like /usr/bin/vi -> /etc/alternatives/vi; that shouldn't cause extracting the archive to a temporary directory to fail. It's even OK to have a symlink to a directory outside the root, as long as nothing gets extracted through that symlink.

The fix for busybox (in https://bugs.busybox.net/show_bug.cgi?id=8411) used the approach of writing a "placeholder" for such symlinks, and replacing all of the placeholders with symlinks at the end of the extraction; then, if anything attempts to extract "through" that placeholder, it'll fail.

GNULongLink entries have the wrong name

././@LongLink seems to be the standard name, but tar-rs emits them as @LongLink. I don't have any example of an issue caused by this, however, I expect fixing it will improve compatibility with other applications and libraries.

Expose the entry type in Header

The entry type is now parsed from Header, but the information is kept private. The raw information is available trough the public field link, so the information is already there. I think it's useful to:

• promote EntryType to be public;
• add a entry_type getter in Header.

Support adding a directory recursively

I'd love to have a Builder method to add a directory recursively, to avoid having to walk it by hand.

Archive unpack error

Hi:
$ rustc --version
rustc 0.13.0-nightly (b87619e27 2014-11-02 23:27:10 +0000)
$ cargo --version
cargo 0.0.1-pre-nightly (603f29d 2014-11-01 19:29:48 +0000)

When trying to use the Archive unpack function I get the following error:

G:\Users\deskv\src\rust\dl-snapshot\src\main.rs:144:5: 144:40 error: the trait
std::io::Reader is not implemented for the type &std::path::windows::Path
G:\Users\vjoseph\src\rust\dl-snapshot\src\main.rs:144 a.unpack(&Path::new(do
wnload_path));
^~~~~~~~~~~~~~~~~~~~~~

error: aborting due to previous error


Unrelated but something I found confusing:
https://github.com/alexcrichton/tar-rs/blob/master/src/lib.rs#L156

Does the join function not keep appending to what was there in the previous loop pass?

Thanks.

println! in unpack

It is annoying to get a bunch of stdout output (uncontrollable) upon calling a library function. Does it have any use-case?

tar-rs/src/entry.rs

Line 379 in 28b18c0

println!("{:?} {:?}", src, dst);

Implement IntoIterator for Archive

I found myself wanting to store an Entries iterator in a struct. This however means I have to depend on some Archive reference that must outlive my struct. Wouldn't it be possible to implement IntoIterator for Archive? In this way I can store the iterator only, without external dependencies on an Archive instance.

appending symbolic links to builder

Hi,

I'm trying to basically tar up a directory as is by walking a tree and adding the files therein, but am noticing that the symlinks that exists in the folder are duplicated as an actual full dereferenced copy in the archive. Is there a sane way to add symlinked files? Been grepping the tests and docs, but not really gotten any wiser.

This is what I've got:

    // pipe builder -> encoder -> file
    let file = File::create(&tarball)?;
    let mut encoder = GzEncoder::new(file, Compression::Default); // encoder writes file
    let mut builder = tar::Builder::new(&mut encoder); // tar builder writes to encoder

    let files = WalkDir::new("OUTPUT")
        .min_depth(1)
        .into_iter()
        .filter_map(|e| e.ok())
        .filter(|e| !e.path().is_dir()); // ignore directories (these are created anyway)

    for f in files {
        let pth = f.path().strip_prefix("OUTPUT").unwrap();
        debug!("-> {}", pth.display());
        let mut f = File::open(f.path())?;
        builder.append_file(pth, &mut f)?;
    }

which causes symlinks to be copied.

From the looks of it one can do something with an explicit Header and set_link_names and using builder.append(), when something is a symlink. So have tried doing a match on fs::symlink_metadata(pth), and use builder.append() when it was a link using:

                    let mut h = Header::new_gnu();
                    h.set_path(pth).unwrap();
                    h.set_size(0);
                    let data : &[u8] = &[];
                    let f = fs::read_link(pth)?;
                    h.set_link_name(f)?;
                    builder.append(&h, data)?;

and the builder lets me do this, but this creates a damaged archive that tar tvf complains about. Any help / documentation on adding links would be great. I just want to do the equivalent of tar czvf mytar.tar OUTPUT

compilation failed with rustc 1.0.0-beta

I fail to build tar-rs directly with rustc-1.0.0-beta.

src/lib.rs:11:1: 11:33 error: unstable feature
src/lib.rs:11 #![feature(fs, fs_time, fs_ext)]
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: this feature may not be used in the beta release channel
error: aborting due to previous error

Please note that I don't use cargo for this build (it is a part for bootstrapping cargo under openbsd).

The full command line used for this build is:

rustc --crate-type lib --out-dir ./build --crate-name tar --cfg 'feature="cargo-build"' src/tar-rs/src/lib.rs

Suggestion: publish git tags for releases

While helping out on this Stack Overflow question, it would have been very useful to browse the code at version 0.3.2. Unfortunately, this repo doesn't have any tags present, so we had to use alternate methods.

Additionally, having the tags makes it easier to see code diffs between versions.

Empty symbolic link contents

Reading a symbolic link inside a tar file results in an empty file. The expected behavior is to get the path of the linked file (like the readlink linux command).

PR #40 demonstrate the problem with a new test.

doc: mention of variable `into` for method `unpack`

In the documentation for unpack a variable into is referenced but is not listed in the function nor is it in the source:

"This operation is relatively sensitive in that it will not write files outside of the path specified by into."

What is into in reference to?

Certain file causes panic upon unpack

I tried running this library through afl.rs, and came across a panic:

Here is the file that is fed into stdin (this is base64 encoded, make sure to decode it before):

AAAAAHVzdGFyICAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADAwMDA3NTUAMDAwMTc1
MAAwMDAxNzUwADAwMDAwMDAwMDAwADEyNDQwMDIxNTc0ADAxMDY0NwAgNQAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB1c3RhciAgAG12ZDQyNTMAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAMDY0NwAgNQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=

Code I used:

extern crate tar;

use std::io::{self, Read};
use std::path::Path;


fn main() {
    let mut input = String::new();
    let result = io::stdin().read_to_string(&mut input);
    if result.is_ok() {
        let mut a = tar::Archive::new(input.as_bytes());
        a.unpack(Path::new("/dev/null"));
    }
}

Output

root@afl-rust:~/afl-staging-area# cargo run --verbose < id\:000000\,sig\:04\,src\:000027\,op\:ext_AO\,pos\:268
       Fresh afl-coverage-plugin v0.0.1 (https://github.com/kmcallister/afl.rs#845bdff0)
       Fresh libc v0.1.7
       Fresh gcc v0.3.5
       Fresh afl-coverage v0.0.1 (https://github.com/kmcallister/afl.rs#845bdff0)
       Fresh tar v0.2.11 (file:///root/afl-staging-area)
       Fresh afl-staging-area v0.1.0 (file:///root/afl-staging-area)
     Running `target/debug/afl-staging-area`
thread '<main>' panicked at 'arithmetic operation overflowed', /root/tar-rs/src/lib.rs:167
stack backtrace:
   1:     0x7fac5db00e59 - sys::backtrace::write::hb34cb0734f7a3c97uhs
   2:     0x7fac5db044d1 - panicking::on_panic::h82f65b9161b1f8deGXw
   3:     0x7fac5dafbb62 - rt::unwind::begin_unwind_inner::h9f6dd38aeb9ea42dQCw
   4:     0x7fac5dafbdc7 - rt::unwind::begin_unwind_fmt::h44a1d6134651f778WBw
   5:     0x7fac5db03e26 - rust_begin_unwind
   6:     0x7fac5db35b84 - panicking::panic_fmt::h063af2dc79b71461c0B
   7:     0x7fac5db35604 - panicking::panic::ha74d34b97dbec983JYB
   8:     0x7fac5dadc226 - Archive<R>::unpack::h130850372175687317
                        at /root/tar-rs/src/lib.rs:172
   9:     0x7fac5dad7117 - main::h71d2ed005404877bkaa
                        at src/main.rs:17
  10:     0x7fac5db086b8 - rust_try_inner
  11:     0x7fac5db086a5 - rust_try
  12:     0x7fac5db05d23 - rt::lang_start::he6efc8b28021b078bSw
  13:     0x7fac5daf2be2 - main
  14:     0x7fac5d501a3f - __libc_start_main
  15:     0x7fac5dad6c08 - _start
  16:                0x0 - <unknown>
Process didn't exit successfully: `target/debug/afl-staging-area` (signal: 4)

tar-rs/src/lib.rs

Line 167 in 5830d58

let is_directory = bytes[bytes.len() - 1] == b'/';

is the line in particular it crashes on

Append does not require a mut reference

I noticed this by accident and found it a bit confusing. Although it's perfectly safe (courtesy of the refcell), it is a little surprising that by passing an immutable reference to a function, you could end up with the archive mutated!

Is this a deliberate design decision, and is exactly why RefCell is used?

Upgrading from 0.4.6 to 0.4.8 broke netbsd build

It seems to have introduced a dependency on xattr that doesn't build on netbsd: Stebalien/xattr#7

Archive::files returns Ok on bad input

I could be misunderstanding the Err cases for Archive::files, but I would expect the following to work differently. More details in comments of the code.

extern crate tar;

use std::io::prelude::*;
use std::fs::File;
use tar::Archive;

fn main() {
    // Cargo.lock, obviously not a tar archive.
    let file = File::open("Cargo.lock").unwrap();
    let archive = Archive::new(file);

    // I'd expect this to panic.
    let files = archive.files().unwrap();
    // This shouldn't work.
    println!("{}", files.count());

    // I'd expect this to panic, again.
    let mut files = archive.files().unwrap();
    let f = files.next();
    let mut buf = [];
    // Errors here with 'invalid tar archive'.
    f.unwrap().unwrap().read(&mut buf);
}

Unicode in Ustar

Ustar headers should only contain ASCII according to libarchive https://github.com/libarchive/libarchive/blob/master/libarchive/archive_write_set_format_pax.c#L850 . libarchive punts when building ustar headers for pax archives however, since the the pax archives can contains unicode.

....so should Ustar::set_path call is_ascii?

CC @eternaleye

Unable to build in stable (beta) channel

Per src/lib.rs#L11:

#![feature(fs, fs_time, fs_ext)]

Things that fail without the feature flags:

$ cargo build
   Compiling tar v0.2.9 (file:///Users/dan/Projects/Tracking/tar-rs)
src/lib.rs:177:22: 177:41 error: use of unstable library feature 'fs': a more granual ability to set specific permissions may be exposed on the Permissions structure itself and this method may not always exist
src/lib.rs:177                 try!(fs::set_permissions(&dst, perm));
                                    ^~~~~~~~~~~~~~~~~~~
<std macros>:1:1: 6:48 note: in expansion of try!
src/lib.rs:177:17: 177:55 note: expansion site
src/lib.rs:177:41: 177:41 help: add #![feature(fs)] to the crate attributes to enable
<std macros>:1:1: 6:48 note: in expansion of try!
src/lib.rs:177:17: 177:55 note: expansion site
src/lib.rs:185:18: 185:32 error: use of unstable library feature 'fs_ext': may want a more useful mode abstraction
src/lib.rs:185             perm.set_mode(mode);
                                ^~~~~~~~~~~~~~
src/lib.rs:185:32: 185:32 help: add #![feature(fs_ext)] to the crate attributes to enable
src/lib.rs:307:39: 307:49 error: use of unstable library feature 'fs_time': the return type of u64 is not quite appropriate for this method and may change if the standard library gains a type to represent a moment in time
src/lib.rs:307         octal(&mut header.mtime, stat.modified() / 1000);
                                                     ^~~~~~~~~~
src/lib.rs:307:49: 307:49 help: add #![feature(fs_time)] to the crate attributes to enable
src/lib.rs:360:32: 360:38 error: use of unstable library feature 'fs_ext': may want a more useful mode abstraction
src/lib.rs:360             meta.permissions().mode()
                                              ^~~~~~
src/lib.rs:360:38: 360:38 help: add #![feature(fs_ext)] to the crate attributes to enable
error: aborting due to 4 previous errors
Could not compile `tar`.

Build seems to fail on OS X + Rust Beta

See https://travis-ci.org/dbrgn/tldr-rs/jobs/120345035. Beta on Linux worked.

(If you want to add OS X to your Travis build matrix, you can use this for reference.)

Long file names are not processed correctly

Here is an archive: http://llvm.org/releases/3.8.0/llvm-3.8.0.src.tar.xz

Here is the example output from the library:

$ xzcat llvm-3.8.0.src.tar.xz | cargo run --example list | grep Resources
llvm-3.8.0.src/test/tools/llvm-symbolizer/Inputs/dsym-test-exe-differentname.dSYM/Contents/Resources
llvm-3.8.0.src/test/tools/llvm-symbolizer/Inputs/dsym-test-exe-differentname.dSYM/Contents/Resources
llvm-3.8.0.src/test/tools/llvm-symbolizer/Inputs/dsym-test-exe-differentname.dSYM/Contents/Resources
llvm-3.8.0.src/test/tools/llvm-symbolizer/Inputs/dsym-test-exe.dSYM/Contents/Resources/
llvm-3.8.0.src/test/tools/llvm-symbolizer/Inputs/dsym-test-exe.dSYM/Contents/Resources/DWARF/
llvm-3.8.0.src/test/tools/llvm-symbolizer/Inputs/dsym-test-exe.dSYM/Contents/Resources/DWARF/dsym-te

Similar pipeline using conventional tar:

$ tar -tJf 050a0507-llvm-3.8.0.src.tar.xz | grep Resources                 
llvm-3.8.0.src/test/tools/llvm-symbolizer/Inputs/dsym-test-exe-differentname.dSYM/Contents/Resources/
llvm-3.8.0.src/test/tools/llvm-symbolizer/Inputs/dsym-test-exe-differentname.dSYM/Contents/Resources/DWARF/
llvm-3.8.0.src/test/tools/llvm-symbolizer/Inputs/dsym-test-exe-differentname.dSYM/Contents/Resources/DWARF/dsym-test-exe-second
llvm-3.8.0.src/test/tools/llvm-symbolizer/Inputs/dsym-test-exe.dSYM/Contents/Resources/
llvm-3.8.0.src/test/tools/llvm-symbolizer/Inputs/dsym-test-exe.dSYM/Contents/Resources/DWARF/
llvm-3.8.0.src/test/tools/llvm-symbolizer/Inputs/dsym-test-exe.dSYM/Contents/Resources/DWARF/dsym-test-exe

won't compile with rustc 1.0.0

Was trying to build cargo master and got this error:

/home/hal/.cargo/registry/src/github.com-1ecc6299db9ec823/tar-0.2.11/src/lib.rs:190:27: 190:47 error: mismatched types:
 expected `i32`,
    found `u32`
(expected i32,
    found u32) [E0308]
/home/hal/.cargo/registry/src/github.com-1ecc6299db9ec823/tar-0.2.11/src/lib.rs:190             perm.set_mode(mode as libc::mode_t);
                                                                                                              ^~~~~~~~~~~~~~~~~~~~
/home/hal/.cargo/registry/src/github.com-1ecc6299db9ec823/tar-0.2.11/src/lib.rs:409:25: 409:33 error: type `&std::fs::Metadata` does not implement any method in scope named `as_raw`
/home/hal/.cargo/registry/src/github.com-1ecc6299db9ec823/tar-0.2.11/src/lib.rs:409         let meta = meta.as_raw();
                                                                                                            ^~~~~~~~
error: aborting due to 2 previous errors

UstarHeader::_set_path panicked at 'slice index starts at 135 but ends at 134'

Steps to reproduce, with cargo 0.10.0-nightly (10ddd7d 2016-04-08)

git clone https://github.com/servo/skia
cd skia
git checkout 5d03056b4034c4541c10b990259d99f0f6dae0ef
RUST_BACKTRACE=1 cargo package

Output:

   Packaging servo-skia v0.20130412.6 (file:///home/simon/projects/servo/deps/skia)
thread '<main>' panicked at 'slice index starts at 135 but ends at 134', ../src/libcore/slice.rs:534
stack backtrace:
   1:     0x55b56fb1a710 - std::sys::backtrace::tracing::imp::write::h63529ecac330b5bc
   2:     0x55b56fb236eb - std::panicking::default_hook::_$u7b$$u7b$closure$u7d$$u7d$::h0e99487804ecdaa3
   3:     0x55b56fb2336e - std::panicking::default_hook::h1f0a2403a716c4a6
   4:     0x55b56fb07eef - std::sys_common::unwind::begin_unwind_inner::heae36551373e36bd
   5:     0x55b56fb098a8 - std::sys_common::unwind::begin_unwind_fmt::hfdca93605af5caca
   6:     0x55b56fb19941 - rust_begin_unwind
   7:     0x55b56fb59ddf - core::panicking::panic_fmt::h139963e0c814ece1
   8:     0x55b56fb59fd3 - core::slice::slice_index_order_fail::hfd97ac60d33c912a
   9:     0x55b56f8b8fd7 - tar::header::UstarHeader::_set_path::h17b0206550960dd5
  10:     0x55b56f8b7d66 - tar::header::Header::_set_path::h31d689d2714e1449
  11:     0x55b56f6b1374 - cargo::ops::cargo_package::package::h08cdd994f43d4c04
  12:     0x55b56f5aba5f - cargo::call_main_without_stdin::h31e15b3df12e109b
  13:     0x55b56f570c56 - cargo::execute::hab577e023b3ac5e2
  14:     0x55b56f56b943 - cargo::call_main_without_stdin::hf709651485746f28
  15:     0x55b56f568aa1 - cargo::main::h2b219a79f378c1ef
  16:     0x55b56fb22fe4 - std::sys_common::unwind::try::try_fn::h512e768250918f40
  17:     0x55b56fb198cb - __rust_try
  18:     0x55b56fb22a2b - std::rt::lang_start::h2dd73040e4b52b40
  19:     0x7fe88be9570f - __libc_start_main
  20:     0x55b56f568428 - <unknown>

Note that rustup run beta cargo package runs correctly, with cargo 0.9.0-nightly (8fc3fd8 2016-02-29).

I don’t fully understand the logic of this method yet, but the + 1 in this line of UstarHeader::_set_path in header.rs is suspicious:

            let path = try!(bytes2path(Cow::Borrowed(&bytes[prefixlen + 1..])));

Allow to add `Read` stream to archive with "handmade" `Header`

It would be nice for creating archives with generated content without creating temporary files.

Rework error handling

As discussed on #rust IRC, tar should export TarError, and turn it into an enumerated error type to allow distinguishing various types of errors. This will require bumping the API version.

Interpretation of file modified timestamp is wrong on Windows

https://github.com/alexcrichton/tar-rs/blob/master/src/lib.rs#L308
octal(&mut header.mtime, stat.modified() / 1000);
This line is incorrect on Windows because tar uses Unix epoch time, but the modified time on Windows is relative to January 1, 1601. This results in incorrect file modified times in any crate packaged on Windows, most notably my winapi crates.

Preserve File Permissions (as in GNU tar)

While creating the tar archive not all of the permissions set on the files being appended to the archive are being preserved. Specifically, if the set-user-ID permission bit is set to 1 then this library will disable this bit.

In the GNU tar program, all of the file permissions are being preserved during creation of the archive.

Long paths are truncated

Here is a tar file with a long path: long_path.zip (zipped because github don't like tars).

Listing the content with tar -tf long_path.tar gives:

home/
home/michele/
home/michele/Documenti/
home/michele/Documenti/Development/
home/michele/Documenti/Development/Progetti/
home/michele/Documenti/Development/Progetti/MetaCloudExperiment/
home/michele/Documenti/Development/Progetti/MetaCloudExperiment/Reference/
home/michele/Documenti/Development/Progetti/MetaCloudExperiment/Reference/duplicati/
home/michele/Documenti/Development/Progetti/MetaCloudExperiment/Reference/duplicati/BuildTools/
home/michele/Documenti/Development/Progetti/MetaCloudExperiment/Reference/duplicati/BuildTools/WixIncludeMake/
home/michele/Documenti/Development/Progetti/MetaCloudExperiment/Reference/duplicati/BuildTools/WixIncludeMake/Program.cs

Listing the contents with this snippet:

let file = File::open("long_path.tar").unwrap();
let mut tar = Archive::new(file);
for file in tar.files_mut().unwrap() {
    if let Ok(f) = file {
        println!("{:?}", f.header().path().unwrap())
    }
}

Gives the following result:

"home/michele/"
"home/michele/Documenti/"
"home/michele/Documenti/Development/"
"home/michele/Documenti/Development/Progetti/"
"home/michele/Documenti/Development/Progetti/MetaCloudExperiment/"
"home/michele/Documenti/Development/Progetti/MetaCloudExperiment/Reference/"
"home/michele/Documenti/Development/Progetti/MetaCloudExperiment/Reference/duplicati/"
"home/michele/Documenti/Development/Progetti/MetaCloudExperiment/Reference/duplicati/BuildTools/"
"././@LongLink"
"home/michele/Documenti/Development/Progetti/MetaCloudExperiment/Reference/duplicati/BuildTools/WixIn"
"././@LongLink"
"home/michele/Documenti/Development/Progetti/MetaCloudExperiment/Reference/duplicati/BuildTools/WixIn"

Truncating paths and adding those ././@LongLink entries.

v0.3.1

The filetime feature has been merged into master. Let's package it to the crates.io!

Paths with backslash is replaced with slash

If a tar file contains a file with a backslash '' in its path, it is replaced with a slash '/'. This however is incorrect in my opinion, because:

1. tar format says "separated by slashes", so any tar that assumes '' to be a directory separator is malformed;
2. a file named "fo\o.txt" (with '' intended literally a backslash and not an escape), becomes "fo/o.txt", i.e. a directory and a file within it).

I know that backslashes are problematic under windows. But this behavior is not correct under Linux.

Handle pax_global_header

With the released version (0.3.2) of tar-rs, when unpacking a Github .tar.gz, the directory contains a file called pax_global_header. See backdrop-ops/contrib#55.

So I tried the current master, because I saw that you worked on headers. Now I get the following error:

thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: Error {
repr: Custom(Custom { kind: Other, error: TarError {
desc: "failed to unpacked `foo/pax_global_header`",
io: Error { repr: Custom(Custom { kind: Other,
error: StringError("unknown file type 0x67") }) } } }) }', ../src/libcore/result.rs:738

I'm not sure how to deal with that, is some code missing to handle this header?

tar-rs doesn't build by rust-1.0.0

tar-rs cann't be built on beta rust branch:

src\lib.rs:435:29: 435:46 error: type `&std::fs::Metadata` does not implement any method in scope named `file_attributes`
src\lib.rs:435         let readonly = meta.file_attributes() & libc::FILE_ATTRIBUTE_READONLY;
                                           ^~~~~~~~~~~~~~~~~
src\lib.rs:452:23: 452:34 error: type `&std::fs::Metadata` does not implement any method in scope named `file_type`
src\lib.rs:452         let ft = meta.file_type();
                                     ^~~~~~~~~~~
src\lib.rs:467:27: 467:44 error: type `&std::fs::Metadata` does not implement any method in scope named `last_write_time`
src\lib.rs:467         let mtime = (meta.last_write_time() / (1_000_000_000 / 100)) - 11644473600;
                                         ^~~~~~~~~~~~~~~~~
error: aborting due to 3 previous errors
Could not compile `tar`.

$ rustc --verbose --version
rustc 1.0.0-dev (f873dc5e2 2015-05-05) (built 2015-05-07)
binary: rustc
commit-hash: f873dc5e22e45d901df15a577c016c01cca98af8
commit-date: 2015-05-05
build-date: 2015-05-07
host: x86_64-pc-windows-gnu
release: 1.0.0-dev

`.unpack` on an Archive should handle GNU extensions

https://github.com/alexcrichton/tar-rs/blob/0cc93a7/src/archive.rs#L109

(someone hit this - https://users.rust-lang.org/t/tar-rs-failed-to-iterate-over-archive-when-unpacking-tar/4442)