bestouff/genext2fs

genext2fs - ext2 filesystem generator for embedded systems

genext2fs's Issues

Make it clearer -d/-D args are applied as layers

Genext2fs applies its inputs (-d and -D arguments) as layers. As seen in #15 this may not be obvious. So, make it obvious:

• state it clearly in the manpage, maybe in the helptext
• display each stage as it happens

Fix autotools warnings/errors

Running autoreconf -i produces the following errors (and a few warnings):

aclocal: warning: autoconf input should be named 'configure.ac', not 'configure.in'
automake: warning: autoconf input should be named 'configure.ac', not 'configure.in'
configure.in:10: warning: AM_INIT_AUTOMAKE: two- and three-arguments forms are deprecated.  For more info, see:
configure.in:10: https://www.gnu.org/software/automake/manual/automake.html#Modernize-AM_005fINIT_005fAUTOMAKE-invocation
configure.in:13: installing './compile'
configure.in:10: installing './install-sh'
configure.in:10: installing './missing'
Makefile.am: error: required file './README' not found
Makefile.am: error: required file './ChangeLog' not found
Makefile.am: installing './depcomp'
automake: warning: autoconf input should be named 'configure.ac', not 'configure.in'
parallel-tests: installing './test-driver'
autoreconf: automake failed with exit status: 1

Please support reading tarballs from stdin.

Thanks for accepting the libarchive pull request!

Next I'd like to be able to read tarballs from standard input. I wanted to use this issue to discuss how the interface for that should best look like. The major difference is, that when the data comes from stdin, it cannot be read twice to automatically determine how many inodes to allocate. In principle, the caller can do their own inode computations and then call genext2fs with the right -N argument. But maybe it makes sense for the caller to only supply the size -b and have the number of inodes be chosen by a heuristic if the input comes from stdin? After all, when creating an ext2 filesystem, I also only specify its size and I don't have to specify the number of inodes.

What do you think?

Issue with symlinks where target is exactly 60 characters

When a symlink target path is exactly 60 characters long, the final character is being truncated in the generated image.

For example, here is a symlink in my source directory:

/tmp/genimage/root/etc/ssl/certs/ca-cert-QuoVadis_Root_CA_3_G3.pem -> /usr/share/ca-certificates/mozilla/QuoVadis_Root_CA_3_G3.crt

Looking at the resulting image (mounted at /mnt), the final "t" is missing:

/mnt/etc/ssl/certs/ca-cert-QuoVadis_Root_CA_1_G3.pem -> /usr/share/ca-certificates/mozilla/QuoVadis_Root_CA_1_G3.cr

no support for hardlinks in tarballs

steps to reproduce:

$ mkdir test
$ echo foo > test/foo
$ ln test/foo test/bar
$ tar -C test -cf test.tar .
$ tar tvf test.tar
drwxr-xr-x josch/josch       0 2020-06-24 15:37 ./
-rw-r--r-- josch/josch       4 2020-06-24 15:37 ./bar
hrw-r--r-- josch/josch       0 2020-06-24 15:37 ./foo link to ./bar
$ ./genext2fs -B 1024 -N 17 -b 200 -a test.tar -f -o Linux -q test.img
$ sudo mount test.img /mnt
$ ls -lha /mnt
ls: cannot access '/mnt/foo': Structure needs cleaning
total 23K
drwxr-xr-x  4 root root 1.0K Jan  1  1970 .
drwxr-xr-x 25 root root 4.0K Jun 12 16:28 ..
-rw-------  1 root root    4 Jun 24 15:37 bar
??????????  ? ?    ?       ?            ? foo
drwx------  2 root root  16K Jan  1  1970 lost+found
$ head /mnt/foo                                                                                                                                                                /tmp/genext2fs
head: cannot open '/mnt/foo' for reading: Structure needs cleaning

If hardlinks from tarballs will not be supported or until they are supported, genext2fs should at least throw an error when the tarball contains them.

superblocks not shadow copied

According to the ext2 specification, superblocks and group descriptors are supposed to be copied per block group (or only in certain block groups if the sparse superblock feature is enabled). However, I don't see any copies being made of the superblock at all in this tool. Since it doesn't add in the sparse superblock feature flag it should be making copies for every block group.

wrong filename if tarball entry is exactly 100 characters long

steps to reproduce:

$ mkdir test
$ touch test/0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef01
$ tar -C test -cf test.tar .
$ ./genext2fs -B 1024 -N 17 -b 200 -a - -f -o Linux test.img < test.tar
$ sudo mount test.img /mnt
$ ls -lha /mnt
drwxr-xr-x  3 root  root  1.0K Jan  1  1970 .
drwxr-xr-x 25 root  root  4.0K Jun 12 16:28 ..
-rw-r--r--  1 josch josch    0 Jun 25 13:57 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef010000644
drwx------  2 root  root   16K Jan  1  1970 lost+found

As you can see, the filename has "0000644" appended at the end.

Could you please tag your release commits?

Hello!

Could you please push tags for the 1.4.1 and the recent 1.4.2 commits? That would allow distributions packaging your work to easily pick up when a new version is available.

Thanks!

cheers, josch

genext2fs is painfully slow for multi-GB input

Hi,

I'm now using genext2fs with multi-GB tarballs as input. While this works well it also takes several hours on my machine. So I profiled genext2fs:
gprof.txt

If I read the profiling output correctly, then most time is spent in the function allocate().

Do you have any ideas how to improve the speed by introducing better data structures?

Support for symbolic links in the device table

They don't exactly fit in the format current table format. But I need it to set their uid / gid (the link can be created in the source directory even if user is unprivileged, but ids can not).

I couldn't find a workaround: tried creating the link in the source directory, and adding its path as a regular file / directory /... in the device table file, but genext2fs complains (legitimately) that node types don't match.

The -d, -D and -a options do not allow paths containing a colon

Currently, the argument to -d, -D and -a is split by the first colon. This means that the path to the directory, specfile or tarball must not have a colon in it. This limitation should either be fixed or at least documented somewhere.

For fixing this I thought about replacing the call to strchr by strrchr but that will just shift the problem to the path inside the ext2 filesystem.

Another problem is, that strchr and strrchr operate on bytes and not on characters. This means, that all unicode characters containing 0x3A are also not supported in input paths -- that's a lot of characters.

One way to fix it would be to allow escaping the colon but then parsing becomes hard.

Another way would be to drop the colon syntax in favour of new command line arguments or by allowing the -d, -D and -a options to receive two arguments, so that the shell takes care of the splitting for us.

Support for converting tarballs to ext2

Hi,

would you accept a pull request allowing genext2fs to read a tarball (for example by using libarchive) and turn it into an ext2 image?

Even non-root users can easily create, store and manipulate tarballs containing files, directories and device nodes with arbitrary ownership, permissions, timestamps and other attributes. The interface could be as simple as:

$ genext2fs < rootfs.tar > rootfs.img

What do you think?

How to change owner of root inode

We use genext2fs in a Buildroot (i.e. genimage) environment. We need to change the owner of the root (/) node in order to control the owner of the mountpoint when the created ext2 partition is mounted in the target embedded system.
The ownership of the root directory tree passed via -d is ignored.
So I tried with -D, but a devtable containing something like

/ d 755 1000 1000 - - - - -

is unfortunately rejected with

genext2fs: bad name '/' (contains a slash)

As creation of the ext2 image is part of a (user) build process, we cannot temporarily mount the filesystem to do the required chmod at the mountpoint level, as this would require root permissions. All we can use is a fakeroot environment.

Can you please tell us how to change the root node owner, or could you maybe point us in another direction (maybe some idea for post-processing; unfortunately there is no e2chmod in e2tools - and e2mkdir does nothing for existing directories).

Thank you very much!

Endless loop with --verbose for large images

How to reproduce:

genext2fs -b 1000000 -d testfolder -m 1 -z -U --verbose testimage.ext2

gets stuck in an endless loop. Removing the --verbose flag makes it succeed.

Please add support for extended attributes

Hi,

it would be great if genext2fs would also store the extended attributes of files and directories passed to it.

Thanks!

cheers, josch

The release 1.5.0 tarball still defines 1.4.2 as the version.

Notice that the release genext2fs-1.5.0.tar.gz still defines 1.4.2 for AC_INIT in configure.ac. Should the version be 1.5.0?