tytso / e2fsprogs Goto Github PK
View Code? Open in Web Editor NEWExt2/3/4 file system utilities
Home Page: http://ext4.wiki.kernel.org
e2fsprogs's Introduction
This is the new version (1.47.1) of the second extended file system management programs. From time to time, I release new versions of e2fsprogs, to fix bugs and to make the utilities more robust. You can always find information about the latest version at the the e2fsprogs web page, which is: http://e2fsprogs.sourceforge.net The INSTALL file has instructions on building and installing e2fsprogs. Provisions for building Red Hat RPMs and Debian dpkg files are supplied as well. In case of bugs in these programs, please contact Ted Ts'o at [email protected] or [email protected]. See the e2fsck man page for suggestions of what sort of information to include when submitting bug reports for these programs.
e2fsprogs's People
Contributors
Stargazers
Watchers
Forkers
xiaoqiangnk alhashemi loewis danielnorberg weicao pazdera chitranshi nidoren yanghuicpp guaneryu gwinel crioux azat-linux dmonakhov polyconseil alexander-naumov djwong wangxiaoguang dtfinch koct9i mrvn smartstorage matt-wu b1n0anb cellularmitosis gavin4code zrec8asdf crossbuild nkzxw vitalif joshwget nkk71 chenorange artoria2e5 jameslinus poettering mzsk wisen sgeneix endelight srf1986 dofoak grobmeier drachenfels-gmbh liqingqiya mkatiyar palmer-dabbelt kihwanoh wyllys66 viktor-prutyanov chandanr sensarliar simonjiao mulatinho lightman2 gwli nrclark garymacindoe ng-alt colinianking hngouveia01 neotim huolinliang whissi xuzhenglun vaginessa thqtest123 iflymingchen7 harshadjs abstract0r chn-beta pabs3 guohaoyuan paulyc lvyong1943 sonntex nishuaidai alvingreat moneytech salewski csirosat vladrassokhin flussonic taskset lczerner dkoo2 varphone pbatard littlekign lienze winndows abe149 muralidharb rjotwani uestclab kongy flowdalic hongxu-jia donwulff csurbhie2fsprogs's Issues
ext2fs_rw_bitmaps uses an extra thread on uniprocessor systems
e2fsprogs/lib/ext2fs/rw_bitmaps.c
Lines 543 to 549 in 67f2b54
There's a num_threads == 1 check that short-circuits to not spawning additional threads, but it's before getting the number of CPUs. If sysconf(_SC_NPROCESSORS_CONF) == 1 (a single CPU system) then it will still spawn an additional thread.
But isn't it intended to not spawn more threads if num_threads == 1 as in the first check? Perhaps the check should come after checking the number of CPUs?
By the way, do you think it should avoid threading if fuse2fs is passed the -s (single-threaded) option? (Would this be as trivial as not including EXT2_FLAG_THREADS if -s is passed?)
badblocks stuck forever
One of my disk shows input/output error when I access certain directories, then I want to use badblocks to find out the location of bad blocks, but seems like it is stuck forever after a while, and no timeout option. Can you please suggest me a way to proceed? Thanks
RPM build failure
In my project, i am going to integrate e2fsprogs1.45.5 version to support Project Quota functionality, so i download the source code from sourceforge website, and use "make rpm" to build the corresponding rpm package, but encountered following errors. i also tried the same procedure for 1.43.8 source code, but the same issue was observed. Before "make rpm", i used "./configure" to setup the env, and use "make clean" for clean up in advance.
extracting debug info from /root/rpmbuild/BUILDROOT/e2fsprogs-1.45.4-0.x86_64/lib64/libcom_err.so.2.1
/usr/lib/rpm/sepdebugcrcfix: Updated 28 CRC32s, 0 CRC32s did match.
symlinked /usr/lib/debug/lib64/libcom_err.so.2.1.debug to /usr/lib/debug/lib64/libcom_err.so.2.debug
symlinked /usr/lib/debug/lib64/libe2p.so.2.3.debug to /usr/lib/debug/lib64/libe2p.so.2.debug
symlinked /usr/lib/debug/lib64/libss.so.2.0.debug to /usr/lib/debug/lib64/libss.so.2.debug
symlinked /usr/lib/debug/lib64/libblkid.so.1.0.debug to /usr/lib/debug/lib64/libblkid.so.1.debug
symlinked /usr/lib/debug/lib64/libuuid.so.1.2.debug to /usr/lib/debug/lib64/libuuid.so.1.debug
symlinked /usr/lib/debug/lib64/libext2fs.so.2.4.debug to /usr/lib/debug/lib64/libext2fs.so.2.debug
7388 blocks
- /usr/lib/rpm/check-buildroot
- /usr/lib/rpm/redhat/brp-compress
- /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip
- /usr/lib/rpm/brp-python-bytecompile /usr/bin/python 1
- /usr/lib/rpm/redhat/brp-python-hardlink
- /usr/lib/rpm/redhat/brp-java-repack-jars
Processing files: e2fsprogs-1.45.4-0.x86_64
error: File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.45.4-0.x86_64/sbin/fsck.ext4dev
error: File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.45.4-0.x86_64/sbin/mkfs.ext4dev
error: File not found by glob: /root/rpmbuild/BUILDROOT/e2fsprogs-1.45.4-0.x86_64/usr/share/man/man8/fsck.ext4dev.8*
error: File not found by glob: /root/rpmbuild/BUILDROOT/e2fsprogs-1.45.4-0.x86_64/usr/share/man/man8/mkfs.ext4dev.8*
Executing(%doc): /bin/sh -e /var/tmp/rpm-tmp.6VzOgF - umask 022
- cd /root/rpmbuild/BUILD
- cd e2fsprogs-1.45.4
- DOCDIR=/root/rpmbuild/BUILDROOT/e2fsprogs-1.45.4-0.x86_64/usr/share/doc/e2fsprogs-1.45.4
- export DOCDIR
- /usr/bin/mkdir -p /root/rpmbuild/BUILDROOT/e2fsprogs-1.45.4-0.x86_64/usr/share/doc/e2fsprogs-1.45.4
- cp -pr README /root/rpmbuild/BUILDROOT/e2fsprogs-1.45.4-0.x86_64/usr/share/doc/e2fsprogs-1.45.4
- cp -pr RELEASE-NOTES /root/rpmbuild/BUILDROOT/e2fsprogs-1.45.4-0.x86_64/usr/share/doc/e2fsprogs-1.45.4
- exit 0
RPM build errors:
line 14: prereq is deprecated: Prereq: /sbin/ldconfig
line 39: prereq is deprecated: Prereq: /sbin/install-info
File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.45.4-0.x86_64/sbin/fsck.ext4dev
File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.45.4-0.x86_64/sbin/mkfs.ext4dev
File not found by glob: /root/rpmbuild/BUILDROOT/e2fsprogs-1.45.4-0.x86_64/usr/share/man/man8/fsck.ext4dev.8*
File not found by glob: /root/rpmbuild/BUILDROOT/e2fsprogs-1.45.4-0.x86_64/usr/share/man/man8/mkfs.ext4dev.8*
make: *** [rpm] Error 1
qsort_r not defined in musl
Hello,
The latest version of e2fsprogs uses qsort_r on e2fsck/rehash.c. However, qsort_r is a Glibc specific function and is not supported in Musl C library. Previous versions only used qsort for the same functionality.
e2fsprogs-1.44.4: mke2fs -n overwrites filesystem
I've recently overwritten an ext4 filesystem by following a procedure in the e2fsck man page. Under the '-b' option, it recommends running 'mke2fs -n' to find the locations of the various superblocks. mke2fs -n actually overwrote the disk. Some useful links
The LQ thread:
Next, the offending package: https://drive.google.com/open?id=1zUtXwHQ4ttfq5-8-zXSrHcpM7kpVvWvu
I'm using the e2fsprogs Slackware64 package. ; Tar -xf in a directory recreates the files. Something isn't right with one of the links above, but I'm sure You'll manage. If you look at the thread, further comment is superfluous.
[UBSan] e2fsck: ext_attr.c:1554:3: runtime error: null pointer passed as argument 2, which is declared to never be null
When checking a fuzzed Ext4 image with Undefined behavior sanitizer-enabled build of e2fsck, a NULL is passed to memcpy. Because of length == 0, it does not happened to crash the program on its own in my case but was considered as a violation of non-null requirements by UBSan.
How to reproduce
- Compile e2fsprogs (commit 2d2d00c) with
$ ./configure CC=gcc CFLAGS="-fsanitize=undefined -fno-sanitize-recover=all -g3" LDFLAGS="-fsanitize=undefined -fno-sanitize-recover=all -g3"
$ make
- Unzip the attached reproducer image: ext4-memcpy-null.zip
- Run it:
$ LANG=C gdb -q --args ./e2fsck/e2fsck -fy /tmp/ext4-memcpy-null.img
Reading symbols from ./e2fsck/e2fsck...
(gdb) b __ubsan_handle_nonnull_arg_abort
Breakpoint 1 at 0x131ce0
(gdb) r
Starting program: /hdd/trosinenko/e2fsprogs-clean/e2fsck/e2fsck -fy /tmp/ext4-memcpy-null.img
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
e2fsck 1.46-WIP (09-Oct-2019)
ext2fs_open2: The ext2 superblock is corrupt
/hdd/trosinenko/e2fsprogs-clean/e2fsck/e2fsck: Superblock invalid, trying backup blocks...
/hdd/trosinenko/e2fsprogs-clean/e2fsck/e2fsck: The ext2 superblock is corrupt while trying to open /tmp/ext4-memcpy-null.img
/hdd/trosinenko/e2fsprogs-clean/e2fsck/e2fsck: Trying to load superblock despite errors...
Inode count in superblock is 12336, should be 8192.
Fix? yes
Superblock metadata_csum_seed is not necessary without metadata_csum.Fix? yes
One or more block group descriptor checksums are invalid. Fix? yes
Group descriptor 0 checksum is 0xbbcc, should be 0x0ace. FIXED.
Filesystem has feature flag(s) set, but is a revision 0 filesystem. Fix? yes
Invalid project quota inode 12336. Fix? yes
Pass 1: Checking inodes, blocks, and sizes
Inode 2 missing EXTENT_FL, but is in extents format
Fix? yes
Inode 2 has imagic flag set. Clear? yes
HTREE directory inode 2 has an invalid root node.
Clear HTree index? yes
Inode 2, i_size is 4096, should be 60. Fix? yes
Inode 2, i_blocks is 805306376, should be 0. Fix? yes
Pass 2: Checking directory structure
Entry '..' in ??? (2) has invalid inode #: 127754.
Clear? yes
Directory inode 2, block #0, offset 4: directory corrupted
Salvage? yes
Breakpoint 1, 0x00007ffff75ac7f0 in __ubsan_handle_nonnull_arg_abort () from /usr/lib/x86_64-linux-gnu/libubsan.so.1
(gdb) bt
#0 0x00007ffff75ac7f0 in __ubsan_handle_nonnull_arg_abort () from /usr/lib/x86_64-linux-gnu/libubsan.so.1
#1 0x0000555555783991 in ext2fs_xattr_set (h=0x555555976a10, name=0x55555582c6c6 "system.data", value=0x0, value_len=0) at ext_attr.c:1554
#2 0x00005555557ac8f5 in ext2fs_inline_data_ea_set (data=0x7fffffffd480) at inline_data.c:43
#3 0x00005555557af579 in ext2fs_inline_data_set (fs=0x555555968320, ino=2, inode=0x7fffffffd4a0, buf=0x555555977eb0, size=60) at inline_data.c:582
#4 0x00005555556e48bc in check_dir_block (fs=0x555555968320, db=0x5555559759f0, priv_data=0x7fffffffdae0) at pass2.c:1651
#5 0x00005555556df204 in check_dir_block2 (fs=0x555555968320, db=0x5555559759f0, priv_data=0x7fffffffdae0) at pass2.c:960
#6 0x0000555555772d63 in ext2fs_dblist_iterate3 (dblist=0x555555977e70, func=0x5555556dedd6 <check_dir_block2>, start=0, count=6, priv_data=0x7fffffffdae0) at dblist.c:216
#7 0x0000555555772e16 in ext2fs_dblist_iterate2 (dblist=0x555555977e70, func=0x5555556dedd6 <check_dir_block2>, priv_data=0x7fffffffdae0) at dblist.c:229
#8 0x00005555556d9838 in e2fsck_pass2 (ctx=0x555555964f00) at pass2.c:186
#9 0x000055555569b5b9 in e2fsck_run (ctx=0x555555964f00) at e2fsck.c:253
#10 0x0000555555696bb0 in main (argc=3, argv=0x7fffffffde48) at unix.c:1909
Non-null was violated here.
f_baddotdir test fails on hppa and sparc with 1.46.3
--- f_baddotdir/expect.1 2021-07-27 17:38:14.000000000 +0000
+++ f_baddotdir.1.log 2021-07-28 08:54:18.545029042 +0000
@@ -32,6 +32,9 @@
Directory entry for '.' in ... (19) is big.
Split? yes
+Directory inode 19, block #0, offset 12: directory corrupted
+Salvage? yes
+
Pass 3: Checking directory connectivity
'..' in /a (12) is <The NULL inode> (0), should be / (2).
Fix? yes
@@ -48,7 +51,14 @@
'..' in /f (17) is <The NULL inode> (0), should be / (2).
Fix? yes
+'..' in /g (19) is <The NULL inode> (0), should be / (2).
+Fix? yes
+
+Couldn't fix parent of inode 19: Couldn't find parent directory entry
+
Pass 4: Checking reference counts
+Inode 2 ref count is 10, should be 9. Fix? yes
+
Pass 5: Checking group summary information
Free blocks count wrong for group #0 (69, counted=70).
Fix? yes
@@ -58,5 +68,8 @@
test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+
+test_filesys: ********** WARNING: Filesystem still has errors **********
+
test_filesys: 19/32 files (0.0% non-contiguous), 30/100 blocks
-Exit status is 1
+Exit status is 4
--- f_baddotdir/expect.2 2021-07-27 17:38:14.000000000 +0000
+++ f_baddotdir.2.log 2021-07-28 08:54:19.645029005 +0000
@@ -1,7 +1,20 @@
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
+Directory entry for '.' in ... (19) is big.
+Split? yes
+
+Missing '..' in directory inode 19.
+Fix? yes
+
Pass 3: Checking directory connectivity
+'..' in /g (19) is <The NULL inode> (0), should be / (2).
+Fix? yes
+
Pass 4: Checking reference counts
+Inode 2 ref count is 9, should be 10. Fix? yes
+
Pass 5: Checking group summary information
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
test_filesys: 19/32 files (0.0% non-contiguous), 30/100 blocks
-Exit status is 0
+Exit status is 1
mke2fs permissions
When a new inode is created here:
https://github.com/tytso/e2fsprogs/blob/master/misc/create_inode.c#L125
don't OR the mode with a default value, but assign the original value as intended. With the OR in place all folders get write permissions set for the owner regardless of permissions of the origin folder.
Test:
$ mkdir -p rootfs/test
$ chmod 555 ./rootfs/test
$ sudo mke2fs -t ext4 -d rootfs rootfs.ext4 4096
...
$ sudo debugfs -R "ls -p /" rootfs.ext4 | grep test
/12/040755/1000/1000/test//
inode size for small filesystems and 2038
The default mke2fs.conf set an inode size of 128 for 'small' filesystems (also for floppy and hurd):
Line 19 in 7b8253b
mke2fs regards filesystems smaller than 512MB as 'small', which leads a default inode size of 128 on embedded systems (where such sizes are still relatively common) and boot partitions on PCs. This in turn results in a kernel warning of "ext4 filesystem being remounted at / supports timestamps until 2038 (0x7fffffff)".
Would you take a PR to change the default for 'small', protecting against the 2038 issue on these filesystems?
Fails to build with gettext 0.20
With gettext 0.20:
ERROR: Use of AM_GNU_GETTEXT without [external] argument is no longer supported.
The fix is a trivial change to configure.ac:
-AM_GNU_GETTEXT
+AM_GNU_GETTEXT([external])
e2fsck Error writing block x (Bad file descriptor)
When I want to check fs with e2fsck, it shows message as title. But there's not any obviously message shown.
Firstly, I create a 60TB thick volume on ARM server and take a snapshot on it with e2fsck.
e2fsck 1.43.9 (8-Feb-2018)
Pass 1: Checking inodes, blocks, and sizes
Inode 81855343 has INDEX_FL flag set on filesystem without htree support.
Pass 1: Memory used: 40608k/37792k (7837k/32772k), time: 165.37/150.20/ 1.90
Pass 1: I/O read: 1085MB, write: 0MB, rate: 6.56MB/s
Pass 2: Checking directory structure
Pass 2: Memory used: 40608k/36608k (8325k/32284k), time: 6.40/ 3.16/ 0.21
Pass 2: I/O read: 125MB, write: 0MB, rate: 19.52MB/s
Pass 3: Checking directory connectivity
Peak memory: Memory used: 40608k/36608k (8325k/32284k), time: 217.58/172.47/ 2.24
Pass 3: Memory used: 40608k/36608k (8324k/32285k), time: 0.31/ 0.30/ 0.00
Pass 3: I/O read: 0MB, write: 0MB, rate: 0.00MB/s
Pass 4: Checking reference counts
Pass 4: Memory used: 40608k/36608k (4565k/36044k), time: 114.33/45.76/12.61
Pass 4: I/O read: 0MB, write: 0MB, rate: 0.00MB/s
Pass 5: Checking group summary information
Free inodes count wrong (1006078544, counted=1006078554).
Pass 5: Memory used: 40608k/36608k (2324k/38285k), time: 288.05/15.74/ 4.96
Pass 5: I/O read: 1922MB, write: 0MB, rate: 6.67MB/s
Memory used: 40608k/36608k (2324k/38285k), time: 620.30/234.25/19.81
I/O read: 3161MB, write: 0MB, rate: 5.10MB/s
Error writing block 1 (Bad file descriptor). Ignore error? no
Error writing block 2 (Bad file descriptor). Ignore error? no
Error writing block 3 (Bad file descriptor). Ignore error? no
Error writing block 4 (Bad file descriptor). Ignore error? no
...
Error writing block 3219095552 (Bad file descriptor). Ignore error? no
Error writing block 3219128320 (Bad file descriptor). Ignore error? no
Error writing block 3219161088 (Bad file descriptor). Ignore error? no
Error writing block 3221192704 (Bad file descriptor). Ignore error? no
resize2fs: New size results in too many block group descriptors
can to grow ext4 fs.
sudo resize2fs /dev/nbd0p2
resize2fs 1.46.2 (28-Feb-2021)
resize2fs: New size results in too many block group descriptors.e2fsck report clean
sudo e2fsck /dev/nbd0p2
e2fsck 1.46.2 (28-Feb-2021)
/dev/nbd0p2: clean, 230243/249832576 files, 965174768/1048472576 blocks
sudo e2fsck -fy /dev/nbd0p2
e2fsck 1.46.2 (28-Feb-2021)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/nbd0p2: 230243/249832576 files (0.5% non-contiguous), 965174768/1048472576 blocksfdisk report 1T partition, tried use rest space, still the same.
$ sudo fdisk -l /dev/nbd0
Disk /dev/nbd0: 1.5 TiB, 1649267441664 bytes, 3221225472 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x3c283df9
Device Boot Start End Sectors Size Id Type
/dev/nbd0p1 * 2048 206847 204800 100M 83 Linux
/dev/nbd0p2 206848 2147690495 2147483648 1T 83 Linuxmore details
$ sudo tune2fs -l /dev/nbd0p2
tune2fs 1.46.2 (28-Feb-2021)
Filesystem volume name: <none>
Last mounted on: /
Filesystem UUID: <>
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr dir_index filetype meta_bg extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
Filesystem flags: signed_directory_hash
Default mount options: user_xattr acl
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 249832576
Block count: 1048472576
Reserved block count: 32658598
Free blocks: 83297808
Free inodes: 249602333
First block: 1
Block size: 1024
Fragment size: 1024
Group descriptor size: 64
Blocks per group: 8192
Fragments per group: 8192
Inodes per group: 1952
Inode blocks per group: 244
First meta block group: 257
Flex block group size: 16
Filesystem created: Sun Oct 20 21:58:50 2019
Last mount time: Fri Feb 19 17:00:04 2021
Last write time: Sat Jul 17 20:17:31 2021
Mount count: 0
Maximum mount count: -1
Last checked: Sat Jul 17 20:08:41 2021
Check interval: 0 (<none>)
Lifetime writes: 6813 MB
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal inode: 8
Default directory hash: half_md4
Directory Hash Seed: <>
Journal backup: inode blocks
Checksum type: crc32c
Checksum: 0x198449a0[resize2fs] lots of warnings when expanding very small ext4 fs to 1.5TB
Hi,
I have come along doing the same weirdnesses for the very same reasons as described in #50.
Doing the following commands on ea82add gives a lot of warnings when resizing:
Illegal block number passed to ext2fs_test_block_bitmap #12440 for block bitmap for broken.img
rm -f broken.img
truncate -s 10M broken.img
./misc/mke2fs broken.img -b 1024 -T default
truncate -s +1500G broken.img
./resize/resize2fs broken.img
As stated in #50, it should not be possible to extend the filesystem past 1023GB, but it seems to work (apart from these warnings) and fsck can't find any problem later on. resize2fs only refuses to run when I extend past 2TB.
So I am either expecting that my extension should have also been rejected or these warnings should not exist.
Should not install blkid.h
I don't think it should install /usr/include/blkid/blkid.h it is a very old version and even a privately used library.
This probably applies to all other util-linux libs too (like uuid).
Error messages when mkfs.ext /dev/loop
Hi expert,
During mkfs.ext /dev/loop which losetup with an image placed in tmpfs. I got the following ERROR messages:
[ 183.110770] blk_update_request: I/O error, dev loop6, sector 524160 op 0x9:(WRITE_ZEROES) flags 0x1000800 phys_seg 0 prio class 0
[ 183.123949] blk_update_request: I/O error, dev loop6, sector 522 op 0x9:(WRITE_ZEROES) flags 0x1000800 phys_seg 0 prio class 0
[ 183.137123] blk_update_request: I/O error, dev loop6, sector 16906 op 0x9:(WRITE_ZEROES) flags 0x1000800 phys_seg 0 prio class 0
[ 183.150314] blk_update_request: I/O error, dev loop6, sector 32774 op 0x9:(WRITE_ZEROES) flags 0x1000800 phys_seg 0 prio class 0
[ 183.163551] blk_update_request: I/O error, dev loop6, sector 49674 op 0x9:(WRITE_ZEROES) flags 0x1000800 phys_seg 0 prio class 0
[ 183.176824] blk_update_request: I/O error, dev loop6, sector 65542 op 0x9:(WRITE_ZEROES) flags 0x1000800 phys_seg 0 prio class 0
....
Reasons for this issue is shmem_fallocate doesn't support FL_ZERO_RANGE and some modifies of processing REQ_OP_WRITE_ZEROES with REQ_NOUNMAP were introduced from commit:efcfec57[loop: fix no-unmap write-zeroes request behavior] in v5.2 kernel.
You can see https://lkml.org/lkml/2020/5/13/1213 for more informations.
[debugfs] cmd link hard link count error
I tried to use debugfs to operate my ext4 fs img in CentOS 7(e2fsprogs: 1.42.9, glibc: 2.17), it seems that there is a bug in debugfs -R "link ..." command.
Apparence:
After cmd debugfs -R "link ...", the hard link count is no increase, then $ du would duplicate the same file resulting in the wrong output of statistics.
I read the source code, find that operation will directly write struct ext2_inode member i_links_count without increasing.
I also find the Debian 10(e2fsprogs: 1.44.5, glibc: 2.28) has the same question.
The following is the operation log (hide some useless output), steps show after debugfs -R "link ...", $ df has no change but $ du not. the hard link count is the same as before. However, $ln command will increase the count, and all of the hard link show 2 reference:
root@river-CentOS-7 /h/m/img# mkfs.ext4 ext4.img
mke2fs 1.42.9 (28-Dec-2013)
ext4.img is not a block special device.
Proceed anyway? (y,n) y
Discarding device blocks: done
warning: 257 blocks unused.
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
49248 inodes, 196608 blocks
9830 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=201326592
6 block groups
32768 blocks per group, 32768 fragments per group
8208 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840
Allocating group tables: done
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done
root@river-CentOS-7 /h/m/img# losetup --show -f -P /home/me/img/ext4.img
/dev/loop0
me@river-CentOS-7 ~/img> sudo mount /dev/loop0 /mnt/
me@river-CentOS-7 ~/img> sudo chown -R me:me /mnt
me@river-CentOS-7 ~/img> cd /mnt/
me@river-CentOS-7 /mnt> ls -ali
total 20
2 drwxr-xr-x. 3 me me 4096 Jul 14 23:47 ./
64 dr-xr-xr-x. 18 root root 238 Jul 10 03:21 ../
11 drwx------. 2 me me 16384 Jul 14 23:47 lost+found/
me@river-CentOS-7 /mnt> df --output=source,fstype,itotal,iused,iavail,ipcent,size,used,avail,pcent,file,target -B 4KiB
Filesystem Type Inodes IUsed IFree IUse% 4K-blocks Used Avail Use% File Mounted on
/dev/loop0 ext4 49248 11 49237 1% 189414 386 175266 1% - /mnt
me@river-CentOS-7 /mnt> du --block-size=4KiB -a
4 ./lost+found
5 .
me@river-CentOS-7 /mnt> echo '1234' > test.txt
me@river-CentOS-7 /mnt> df --output=source,fstype,itotal,iused,iavail,ipcent,size,used,avail,pcent,file,target -B 4KiB
Filesystem Type Inodes IUsed IFree IUse% 4K-blocks Used Avail Use% File Mounted on
/dev/loop0 ext4 49248 12 49236 1% 189414 387 175265 1% - /mnt
me@river-CentOS-7 /mnt> du --block-size=4KiB -a
1 ./test.txt
4 ./lost+found
6 .
me@river-CentOS-7 /mnt> sudo -s
Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
root@river-CentOS-7 /mnt# ls -ali
total 24
2 drwxr-xr-x. 3 me me 4096 Jul 14 23:48 ./
64 dr-xr-xr-x. 18 root root 238 Jul 10 03:21 ../
11 drwx------. 2 me me 16384 Jul 14 23:47 lost+found/
12 -rw-r--r--. 1 me me 5 Jul 14 23:48 test.txt
root@river-CentOS-7 /mnt# debugfs -w /dev/loop0 -R 'link test.txt /1.txt'
debugfs 1.42.9 (28-Dec-2013)
root@river-CentOS-7 /mnt# ls -ali
total 28
2 drwxr-xr-x. 3 me me 4096 Jul 14 23:48 ./
64 dr-xr-xr-x. 18 root root 238 Jul 10 03:21 ../
12 -rw-r--r--. 1 me me 5 Jul 14 23:48 1.txt
11 drwx------. 2 me me 16384 Jul 14 23:47 lost+found/
12 -rw-r--r--. 1 me me 5 Jul 14 23:48 test.txt
root@river-CentOS-7 /mnt# du --block-size=4KiB -a
1 ./test.txt
1 ./1.txt
4 ./lost+found
7 .
root@river-CentOS-7 /mnt# df --output=source,fstype,itotal,iused,iavail,ipcent,size,used,avail,pcent,file,target --block-size=4KiB
Filesystem Type Inodes IUsed IFree IUse% 4K-blocks Used Avail Use% File Mounted on
/dev/loop0 ext4 49248 12 49236 1% 189414 387 175265 1% - /mnt
root@river-CentOS-7 /mnt# debugfs -w /dev/loop0 -R 'link test.txt /11.txt'
debugfs 1.42.9 (28-Dec-2013)
root@river-CentOS-7 /mnt# ls -ali
total 32
2 drwxr-xr-x. 3 me me 4096 Jul 14 23:48 ./
64 dr-xr-xr-x. 18 root root 238 Jul 10 03:21 ../
12 -rw-r--r--. 1 me me 5 Jul 14 23:48 11.txt
12 -rw-r--r--. 1 me me 5 Jul 14 23:48 1.txt
11 drwx------. 2 me me 16384 Jul 14 23:47 lost+found/
12 -rw-r--r--. 1 me me 5 Jul 14 23:48 test.txt
root@river-CentOS-7 /mnt# du --block-size=4KiB -a
1 ./test.txt
1 ./1.txt
4 ./lost+found
1 ./11.txt
8 .
root@river-CentOS-7 /mnt# df --output=source,fstype,itotal,iused,iavail,ipcent,size,used,avail,pcent,file,target --block-size=4KiB
Filesystem Type Inodes IUsed IFree IUse% 4K-blocks Used Avail Use% File Mounted on
/dev/loop0 ext4 49248 12 49236 1% 189414 387 175265 1% - /mnt
root@river-CentOS-7 /mnt# debugfs -w /dev/loop0 -R 'link /test.txt /111.txt'
debugfs 1.42.9 (28-Dec-2013)
root@river-CentOS-7 /mnt# ls -ali
total 36
2 drwxr-xr-x. 3 me me 4096 Jul 14 23:48 ./
64 dr-xr-xr-x. 18 root root 238 Jul 10 03:21 ../
12 -rw-r--r--. 1 me me 5 Jul 14 23:48 111.txt
12 -rw-r--r--. 1 me me 5 Jul 14 23:48 11.txt
12 -rw-r--r--. 1 me me 5 Jul 14 23:48 1.txt
11 drwx------. 2 me me 16384 Jul 14 23:47 lost+found/
12 -rw-r--r--. 1 me me 5 Jul 14 23:48 test.txt
root@river-CentOS-7 /mnt# du --block-size=4KiB -a
1 ./test.txt
1 ./111.txt
1 ./1.txt
4 ./lost+found
1 ./11.txt
9 .
root@river-CentOS-7 /mnt# df --output=source,fstype,itotal,iused,iavail,ipcent,size,used,avail,pcent,file,target --block-size=4KiB
Filesystem Type Inodes IUsed IFree IUse% 4K-blocks Used Avail Use% File Mounted on
/dev/loop0 ext4 49248 12 49236 1% 189414 387 175265 1% - /mnt
root@river-CentOS-7 /mnt# ln test.txt ln-cmd-1.txt
root@river-CentOS-7 /mnt# ls -ali
total 40
2 drwxr-xr-x. 3 me me 4096 Jul 14 23:52 ./
64 dr-xr-xr-x. 18 root root 238 Jul 10 03:21 ../
12 -rw-r--r--. 2 me me 5 Jul 14 23:48 111.txt
12 -rw-r--r--. 2 me me 5 Jul 14 23:48 11.txt
12 -rw-r--r--. 2 me me 5 Jul 14 23:48 1.txt
12 -rw-r--r--. 2 me me 5 Jul 14 23:48 ln-cmd-1.txt
11 drwx------. 2 me me 16384 Jul 14 23:47 lost+found/
12 -rw-r--r--. 2 me me 5 Jul 14 23:48 test.txt
root@river-CentOS-7 /mnt# du --block-size=4KiB -a
1 ./test.txt
4 ./lost+found
6 .
root@river-CentOS-7 /mnt# df --output=source,fstype,itotal,iused,iavail,ipcent,size,used,avail,pcent,file,target --block-size=4KiB
Filesystem Type Inodes IUsed IFree IUse% 4K-blocks Used Avail Use% File Mounted on
/dev/loop0 ext4 49248 12 49236 1% 189414 387 175265 1% - /mntDoes fuse2fs support online resizing?
ext4 filesystem support online resizing (increase the space only) through resize2fs.
Does fuse2fs suppport this feature?
[RFE] Switch to fuse3
Hi,
Would it be possible to have e2fsprogs make use of fuse3 instead of fuse (version 2)? Fuse maintainers/developers seem to encourage usage of fuse3 instead of fuse2. I tried to search if this was asked before, seems that in end 2018 it was discussed, see https://lists.debian.org/debian-devel/2018/12/msg00296.html. Various functions have been renamed in fuse3, see https://github.com/libfuse/libfuse/releases/tag/fuse-3.0.0. So although they can both be installed at the same time, a fuse-using application won't be able to compile as-is against fuse3.
The debian-devel mail mentions that e2fsprogs should ideally still compile against old-old-stable. I'm not sure yet if that would be possible if e2fsprogs switches to fuse3. This as I'm not sure old-old-stable Debian has fuse3.
Thanks
Is there an error code description for e2fsck?
I am making a tool for verifying the file system based on e2fsck, but this tool does not need to be as strict as the official one. When the file system cannot be mounted and must be repaired before it can be used, the tool will return an error code. When the file system has only some statistical errors, the tool can still pass the verification.
I tried to modify the source code, but I found that there are more than 300 error codes. I don’t know which error codes do not affect the use of the file system and which error codes affect. Can anyone help me?
Is this the new canonical e2fsprogs development location?
Is this the new canonical e2fsprogs development location?
The README still says the official URL is http://e2fsprogs.sourceforge.net and that bugs should be reported by email. Should they be reported here to GitHub instead? Reporting bugs on a public web site is more desirable than using private email.
The web page https://sourceforge.net/projects/e2fsprogs/support says:
Ext2 Filesystems Utilities says the best way to get help with its software is by visiting http://sourceforge.net/tracker/?func=add&group_id=2406&atid=202406.
However visiting that URL goes to the Support section of the e2fsprogs SourceForge issue tracker which is overrun with spam and in which the Create Ticket functionality has been turned off.
There is no mention of this GitHub repository on the SourceForge project page or web site. If it is official, it should be mentioned there.
Six tests fail often in Nixpkgs package
The following tests often fail with 1.45.2 and 1.45.3. The issue did not occur before.
Tests failed: m_image_mmp m_mmp m_mmp_bad_csum m_mmp_bad_magic t_mmp_1on t_mmp_2off
Related issue on our tracker: NixOS/nixpkgs#65471.
Any idea why it is failing often but not always?
[Question] Can a specific block be overwritten using debugfs
I can see block content with block_dump, is there a way to override it using debugfs?
If not, is there a function in the source code I can use to implement it myself?
e2fsck throwing error "Group descriptors look bad... trying backup blocks"
Hi,
I done resize from ~54TB to ~59TB which got successful.
Later, performing 'e2fsck -fy' thrown error and cleared root inode which led me to data loss. Below is error:
ext2fs_check_desc: Corrupt group descriptor: bad block for inode table
e2fsck: Group descriptors look bad... trying backup blocks...
Superblock has an invalid journal (inode 8).
Clear? yes
*** journal has been deleted ***
Pass 1: Checking inodes, blocks, and sizes
Group 15's inode table at 9274 conflicts with some other fs block.
Relocate? yes
Root inode is not a directory. Clear? yes
Inode 30722 passes checks, but checksum does not match inode. Fix? yes
Relocating group 15's inode table from 9274 to 9658...
Recreate journal? yes
Creating journal (262144 blocks): Inode bitmap checksum does not match bitmap: while trying to create journal
Restarting e2fsck from the beginning...
One or more block group descriptor checksums are invalid. Fix? yes
Group descriptor 15 checksum is 0xb895, should be 0xfaa7. FIXED.
Pass 1: Checking inodes, blocks, and sizes
Root inode is not a directory. Clear? yes
Here is tune2fs output:
tune2fs -l /dev/loop7
tune2fs 1.43.5 (04-Aug-2017)
Filesystem volume name: <none>
Last mounted on: /mnt/managedvolume/ora_IT0794_data_channel8_33ca2aec
Filesystem UUID: f2b26a37-70e6-47c3-88cc-0e0d4f691495
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: ext_attr dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
Filesystem flags: signed_directory_hash
Default mount options: user_xattr acl
Filesystem state: not clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 996694016
Block count: 15947093334
Reserved block count: 797354664
Free blocks: 9329362815
Free inodes: 996693909
First block: 0
Block size: 4096
Fragment size: 4096
Group descriptor size: 64
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 2048
Inode blocks per group: 128
Flex block group size: 16
Filesystem created: Tue Nov 3 08:27:52 2020
Last mount time: Wed Jun 2 20:30:15 2021
Last write time: Wed Jun 2 13:24:42 2021
Mount count: 2
Maximum mount count: -1
Last checked: Wed Jun 2 13:18:28 2021
Check interval: 0 (<none>)
Lifetime writes: 514 TB
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 256
Required extra isize: 32
Desired extra isize: 32
Default directory hash: half_md4
Directory Hash Seed: e7d0b85c-2cb7-46f4-8f9a-6105040b7a8d
Journal backup: inode blocks
Checksum type: crc32c
Checksum: 0x3ee90b49
My question is: Is it completely safe to perform resizes at this huge sizes? And, any pointers to stat debugging at also would be great help based on above error message.
Few more details:
Initially, filesystem is of size ~50TB and have data till ~35TB.
My first resize2fs -f /dev/loop7 from ~50TB to ~54TB went fine, and also, e2fsck didn't thrown any error. Mouting the filesystem as well went successful.
After few hours, I performed second resize, and landed into this issue.
Using fuse2fs on macOS results in "Journal needs recovery"
Hi @tytso,
I'm trying to use fuse2fs on macOS 10.11.6 (El Capitan). I have macFUSE installed (formerly "osxfuse") and I followed the instructions in INSTALL to build, adding export CPPFLAGS=-I/usr/local/include/osxfuse/ before running configure so that fuse.h from macFUSE is found. The build completes successfully.
I receive errors when I try to mount my ext4 volume:
$ mkdir -p ~/mounts/linux
$ cd ~/Code/e2fsprogs/build/misc
$ ./fuse2fs -o ro /dev/disk1s2 ~/mounts/linux/
Mounting read-only.
/dev/disk1s2: Permission denied.
Please run e2fsck -fy /dev/disk1s2
$ sudo ./fuse2fs -o ro /dev/disk1s2 ~/mounts/linux/
Mounting read-only.
Journal needs recovery; running `e2fsck -E journal_only' is required.That partition mounts fine using https://github.com/gerard/ext4fuse (sudo ext4fuse /dev/disk1s2 ~/mounts/linux/ -o allow_other), but that implementation seems to have issues (and is not really maintained) so I thought I would try this project...
support for clearing ext4 "large_dir" feature
[ef2sdroid] Support
Hey, not sure if an issue or not a complete implementation, and i apologise if this has been answered elsewhere but i can barely find any documentation on this.
ef2sdoid is not complete or is not being built?
I'm trying to build it with cygwin but cant seem to find any sources other than yours here
How to define __linux__ automatically?
I tried to enable debug mode in resize/resize2fs.c and I saw one definition:
#ifdef __linux__ /* Kludge for debugging */
#define RESIZE2FS_DEBUG
#endif
How to define the __linux__ automatically? what options when setup configure?
My OS is Ubuntu 16.04.
'make check' tests failure on sparc64 (BE) arch
Hello!
Using latest git on sparc64 vm/ldom:
$ uname -a
Linux ttip 5.12.0-rc3 #199 SMP Mon Mar 15 13:04:24 MSK 2021 sparc64 GNU/Linux
$ git log -1 --oneline
67f2b546 (HEAD -> master, tag: v1.46.2, origin/next, origin/master, origin/maint, origin/HEAD) Update release notes, etc., for the 1.46.2 release
make check returns error on some of the tests :
$ make check
...
t_mmp_2off: disable MMP using tune2fs: ok
369 tests succeeded 4 tests failed
Tests failed: j_recover_csum2_32bit j_recover_csum2_64bit j_recover_fast_commit m_rootdir_acl
make[1]: *** [Makefile:399: test_post] Error 1
tried to bisect some of the failing tests:
$ git bisect log
git bisect start
# bad: [2b0d66bbec7807597673b7e00f2d9ff04439c307] Update release notes, etc., for the 1.46.0 release
git bisect bad 2b0d66bbec7807597673b7e00f2d9ff04439c307
# good: [fa86b5c2aa3f98824e3e717074db13b7802d65b7] Update release notes, etc., for the 1.45.7 release
git bisect good 5403970e44241cec26f98aaa0124b9881b4bbf4f
# good: [a4ce6ecc4979faccd6cec1ba7e7b391d3b097244] ext4: add support for printing the error code associated with an error
git bisect good a4ce6ecc4979faccd6cec1ba7e7b391d3b097244
# bad: [f84fed462a64fa680a8c89d1e248564dda981bf4] e2fsck: remove dead code when recreating the journal
git bisect bad f84fed462a64fa680a8c89d1e248564dda981bf4
# good: [bdcd5f22203fcb5347082ad555a711dbbb2541da] Add configure and build support for the pthreads library
git bisect good bdcd5f22203fcb5347082ad555a711dbbb2541da
# bad: [7ed2b5d0f5a3a650ddf4c0e335ef9881ff782da8] e2fsck: port fc changes from kernel's recovery.c to e2fsck
git bisect bad 7ed2b5d0f5a3a650ddf4c0e335ef9881ff782da8
# good: [e2e58d3128042f0b1aa9783d1e1c28816fa284a2] ext2fs: parallel bitmap loading
git bisect good e2e58d3128042f0b1aa9783d1e1c28816fa284a2
# good: [895e8e33beed102189922418ba611cc7528f45e3] ext2fs: move calculate_summary_stats to ext2fs lib
git bisect good 895e8e33beed102189922418ba611cc7528f45e3
# bad: [2fc929c65ec9a7d316bbaace7acdd32f8d6184aa] e2fsck: add kernel endian-ness conversion macros
git bisect bad 2fc929c65ec9a7d316bbaace7acdd32f8d6184aa
# first bad commit: [2fc929c65ec9a7d316bbaace7acdd32f8d6184aa] e2fsck: add kernel endian-ness conversion macros
"make check" with last 2 bisect commits:
$ git checkout 2fc929c65ec9a7d316bbaace7acdd32f8d6184aa
$ make -j4 check
...
t_mmp_2off: disable MMP using tune2fs: ok
369 tests succeeded 2 tests failed
Tests failed: j_recover_csum2_32bit j_recover_csum2_64bit
make[1]: *** [Makefile:399: test_post] Error 1
$ git checkout 895e8e33beed102189922418ba611cc7528f45e3
$ make -j4 check
...
t_uninit_bg_rm: remove uninit_bg: ok
371 tests succeeded 0 tests failed
make[1]: Leaving directory '/1/mator/e2fsprogs.git/tests'
Lets see what is wrong with j_recover_csum2_32bit test:
$ git desc --long
v1.46.2-0-g67f2b546
$ cd tests
$ ./test_one j_recover_csum2_32bit
j_recover_csum2_32bit: recover 32-bit journal checksum v2: failed
$ ls -la j_recover_csum2_32bit*
-rw-r--r-- 1 mator mator 155 Mar 17 20:13 j_recover_csum2_32bit.1.log
-rw-r--r-- 1 mator mator 155 Mar 17 20:13 j_recover_csum2_32bit.2.log
-rw-r--r-- 1 mator mator 1445 Mar 17 20:13 j_recover_csum2_32bit.failed
$ cat j_recover_csum2_32bit.1.log
test_filesys: recovering journal
Signal (10) SIGBUS si_code=BUS_ADRALN fault addr=0x100001b9cee
../e2fsck/e2fsck(+0x3eff4)[0x1000003eff4]
Exit status is 0
$ mktemp /tmp/e2fsprogs-tmp-j_recover_csum2_32bit.XXXXX
/tmp/e2fsprogs-tmp-j_recover_csum2_32bit.PQNn0
$ bzip2 -dc j_recover_csum2_32bit/image.bz2 > /tmp/e2fsprogs-tmp-j_recover_csum2_32bit.PQNn0
$ ../e2fsck/e2fsck -fy -N test_filesys /tmp/e2fsprogs-tmp-j_recover_csum2_32bit.PQNn0
e2fsck 1.46.2 (28-Feb-2021)
test_filesys: recovering journal
Signal (10) SIGBUS si_code=BUS_ADRALN fault addr=0x100001bc1fe
../e2fsck/e2fsck(+0x3eff4)[0x1000003eff4]
$ gdb -q
(gdb) set args -fy -N test_filesys /tmp/e2fsprogs-tmp-j_recover_csum2_32bit.PQNn0
(gdb) file ../e2fsck/e2fsck
Reading symbols from ../e2fsck/e2fsck...
(gdb) run
Starting program: /1/mator/e2fsprogs.git/e2fsck/e2fsck -fy -N test_filesys /tmp/e2fsprogs-tmp-j_recover_csum2_32bit.PQNn0
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/sparc64-linux-gnu/libthread_db.so.1".
e2fsck 1.46.2 (28-Feb-2021)
test_filesys: recovering journal
Program received signal SIGBUS, Bus error.
0x000001000003a8f4 in jbd2_has_feature_64bit (j=0x100001b2c40) at ../lib/ext2fs/kernel-jbd.h:390
390 JBD2_FEATURE_INCOMPAT_FUNCS(64bit, 64BIT)
(gdb) bt
#0 0x000001000003a8f4 in jbd2_has_feature_64bit (j=0x100001b2c40) at ../lib/ext2fs/kernel-jbd.h:390
#1 read_tag_block (tag=0x100001bc1fe, journal=0x100001b2c40) at recovery.c:381
#2 do_one_pass (journal=0x100001b2c40, info=0x7feffffeccc, pass=<optimized out>) at recovery.c:635
#3 0x000001000003acfc in jbd2_journal_recover (journal=0x100001b2c40) at recovery.c:311
#4 0x00000100000341f8 in recover_ext3_journal (ctx=0x100001ab460) at journal.c:1631
#5 e2fsck_run_ext3_journal (ctx=0x100001ab460) at journal.c:1670
#6 0x0000010000019248 in main (argc=<optimized out>, argv=<optimized out>) at unix.c:1791
(gdb)
Can someone (@harshadjs ?) please look what is wrong here? Thanks.
Please see as well recent build logs on debian buildd servers [1].
PS: ia64 (BE) arch gives the following output on this test run:
$ uname -a
Linux lifshitz 4.15.0 #36 SMP Mon Dec 21 10:36:27 UTC 2020 ia64 GNU/Linux
$ ./test_one j_recover_csum2_32bit
e2fsck(3090213): unaligned access to 0x60000000000268ae, ip=0x4000000000068000
e2fsck(3090213): unaligned access to 0x60000000000268d6, ip=0x4000000000068000
j_recover_csum2_32bit: recover 32-bit journal checksum v2: ok
Generic *print version* parameter for all tools
Hi, would it be possible to add a -v aka --version argument to all tools of e2fsprogs so it's possible to know what's currently installed?
Misleading message when trying to change UUID (^metadata_csum_seed, mounted fs)
# tune2fs -l /dev/sdb1
tune2fs 1.45.6 (20-Mar-2020)
Filesystem volume name: <none>
Last mounted on: <not available>
Filesystem UUID: d01c108f-b24d-4abb-bec1-76dff9137d4d
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
Filesystem flags: signed_directory_hash
Default mount options: user_xattr acl
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 65536
Block count: 262144
Reserved block count: 13107
Free blocks: 249189
Free inodes: 65525
First block: 0
Block size: 4096
Fragment size: 4096
Group descriptor size: 64
Reserved GDT blocks: 127
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 8192
Inode blocks per group: 512
Flex block group size: 16
Filesystem created: Mon Apr 27 17:31:51 2020
Last mount time: Mon Apr 27 17:38:16 2020
Last write time: Mon Apr 27 17:39:12 2020
Mount count: 0
Maximum mount count: -1
Last checked: Mon Apr 27 17:39:12 2020
Check interval: 0 (<none>)
Lifetime writes: 17 MB
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 256
Required extra isize: 32
Desired extra isize: 32
Journal inode: 8
Default directory hash: half_md4
Directory Hash Seed: bc4f58b3-38f1-45ef-bb3a-b52c15f4b2f8
Journal backup: inode blocks
Checksum type: crc32c
Checksum: 0x6a196fae
# e2fsck -f /dev/sdb1
e2fsck 1.45.6 (20-Mar-2020)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/sdb1: 11/65536 files (0.0% non-contiguous), 12955/262144 blocks
# mount /dev/sdb1 /mnt
# tune2fs -U random /dev/sdb1
tune2fs 1.45.6 (20-Mar-2020)
This operation requires a freshly checked filesystem.
Please run e2fsck -f on the filesystem.
Since it does not have metadata_csum_seed, is tune2fs -U random actually asking for an unmounted fs?
(kernel 5.4.32, e2fsprogs 1.45.6)
mkfs.ext3 -d root-directory does not preserve uid/gid > 65535
mkfs.ext3 -d root-directory image_file appears to replicate only the low-order 16 bits of the uid & gid file mode bits of root-directory into the new file system in image_file.
Reproducer here: https://github.com/sylabs/singularity/issues/4878#issuecomment-567213599
mkfs: feature to output filesystem to stdout
I am using mkfs.ext4 to create a filesystem image, kind of like this:
$ mkfs.ext4 filesystem.img 1048576
This creates a file called filesystem.img that is 1048576 blocks in size.
Instead of creating a file, I would like the resulting filesystem to go to stdout so that I can pipe the result in directly to dd and avoid some heavy I/O. Some programs accept - as an argument indicating that the output should go to standard output but alas that has not been implemented here.
I thought I might be able to be clever by passing /dev/stdout as the filename but that didn't work either.
$ mkfs.ext4 /dev/stdout 1048576
mke2fs 1.46.2 (28-Feb-2021)
warning: Unable to get device geometry for /dev/stdout
Warning: could not erase sector 2: Illegal seek
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: cbb2efb4-ad58-424f-b678-5e3b2601d7ca
Superblock backups stored on blocks:
32768, 98304, 163840, 229376
Allocating group tables: done
Warning: could not read block 0: Illegal seek
Warning: could not erase sector 0: Illegal seek
mkfs.ext4: Illegal seek while zeroing block 262128 at end of filesystem
Writing inode tables: done
Creating journal (8192 blocks): mkfs.ext4: Illegal seek
while trying to create journal
The error here might reveal a problem with writing output to stdout. The filesystem is not initialised as a continuous byte stream and requires seeking backwards and forwards - at least this is my assumption. I think it would be acceptable to buffer the whole filesystem in memory and dump the whole thing to stdout when complete.
I have also considered writing the filesystem to tmpfs or ramfs but the environment I'm working with is containerised and creating mounts is not possible.
FSCK Memory leak with mdadm
Hi all, i have an issue while running fsck on a partition created on a soft raid 5
The process have a big leak during the last sequence (step 5) and is using the entire memory on the system (1GB) and is killed by the kernel
It's occurs with version 1.43 and 1.45
The OS is a custom OS built with buildroot but we dont have the issue with the hardware raid on the same platform.
Regards,
Seb
Error while trying to change kbytes_written value
Trying:
sudo debugfs
open -w /dev/mmcblk0p2
set_super_value kbytes_written 1024
Prints error
64-bit field kbytes_written has a second 64-bit field
defined; BUG?!?
Version: debugfs 1.46-WIP (20-Mar-2020)
Can you please take a look whats wrong here?
u_direct_io is failing in 1.46.4
.failed:
root@qemux86-64:/usr/lib/e2fsprogs/ptest/test# cat .failed
--- u_direct_io//expect 2018-03-09 12:34:56.000000000 +0000
+++ .log 2021-08-22 10:40:05.404886893 +0000
@@ -19,8 +19,8 @@
Inode count: 32768
Block count: 32768
Reserved block count: 1638
-Overhead clusters: 5131
-Free blocks: 27631
+Overhead clusters: 6155
+Free blocks: 26607
Free inodes: 32757
First block: 0
Block size: 4096
@@ -29,27 +29,29 @@
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 32768
-Inode blocks per group: 1024
+Inode blocks per group: 2048
Flex block group size: 16
Mount count: 0
Check interval: 15552000 (6 months)
Reserved blocks uid: 0
Reserved blocks gid: 0
First inode: 11
-Inode size: 128
+Inode size: 256
+Required extra isize: 32
+Desired extra isize: 32
Journal inode: 8
Default directory hash: half_md4
Journal backup: inode blocks
Directories: 2
Group 0: block bitmap at 9, inode bitmap at 25, inode table at 41
- 27631 free blocks, 32757 free inodes, 2 used directories
+ 26607 free blocks, 32757 free inodes, 2 used directories
e2fsck -fn -N test_filesys $LOOP
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-test_filesys: 11/32768 files (9.1% non-contiguous), 5137/32768 blocks
+test_filesys: 11/32768 files (9.1% non-contiguous), 6161/32768 blocks
Exit status is 0
e2fsck -fn -N test_filesys $TMPFILE
Pass 1: Checking inodes, blocks, and sizes
@@ -57,5 +59,5 @@
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-test_filesys: 11/32768 files (9.1% non-contiguous), 5137/32768 blocks
+test_filesys: 11/32768 files (9.1% non-contiguous), 6161/32768 blocks
Exit status is 0
.log
root@qemux86-64:/usr/lib/e2fsprogs/ptest/test# cat .log
mke2fs -F -o Linux -t ext4 -O ^metadata_csum,^uninit_bg -D $LOOP
Creating filesystem with 32768 4k blocks and 32768 inodes
Allocating group tables: done
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done
debugfs -D -R stats $LOOP
Filesystem volume name: <none>
Last mounted on: <not available>
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file huge_file dir_nlink extra_isize
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 32768
Block count: 32768
Reserved block count: 1638
Overhead clusters: 6155
Free blocks: 26607
Free inodes: 32757
First block: 0
Block size: 4096
Fragment size: 4096
Reserved GDT blocks: 7
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 32768
Inode blocks per group: 2048
Flex block group size: 16
Mount count: 0
Check interval: 15552000 (6 months)
Reserved blocks uid: 0
Reserved blocks gid: 0
First inode: 11
Inode size: 256
Required extra isize: 32
Desired extra isize: 32
Journal inode: 8
Default directory hash: half_md4
Journal backup: inode blocks
Directories: 2
Group 0: block bitmap at 9, inode bitmap at 25, inode table at 41
26607 free blocks, 32757 free inodes, 2 used directories
e2fsck -fn -N test_filesys $LOOP
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
test_filesys: 11/32768 files (9.1% non-contiguous), 6161/32768 blocks
Exit status is 0
e2fsck -fn -N test_filesys $TMPFILE
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
test_filesys: 11/32768 files (9.1% non-contiguous), 6161/32768 blocks
Exit status is 0
/etc/mke2fs.conf:
root@qemux86-64:/usr/lib/e2fsprogs/ptest/test# cat /etc/mke2fs.conf
[defaults]
base_features = sparse_super,large_file,filetype,resize_inode,dir_index,ext_attr
default_mntopts = acl,user_xattr
enable_periodic_fsck = 0
blocksize = 4096
inode_size = 256
inode_ratio = 16384
[fs_types]
ext3 = {
features = has_journal
}
ext4 = {
features = has_journal,extent,huge_file,flex_bg,metadata_csum,64bit,dir_nlink,extra_isize
}
small = {
blocksize = 1024
inode_ratio = 4096
}
floppy = {
blocksize = 1024
inode_ratio = 8192
}
big = {
inode_ratio = 32768
}
huge = {
inode_ratio = 65536
}
news = {
inode_ratio = 4096
}
largefile = {
inode_ratio = 1048576
blocksize = -1
}
largefile4 = {
inode_ratio = 4194304
blocksize = -1
}
hurd = {
blocksize = 4096
inode_size = 128
warn_y2038_dates = 0
}
There are also messages from the kernel:
root@qemux86-64:/usr/lib/e2fsprogs/ptest/test# ./test_one u_direct_io/
[ 2422.216855] loop0: detected capacity change from 0 to 262144
[ 2422.986613] blk_update_request: operation not supported error, dev loop0, sector 262016 op 0x9:(WRITE_ZEROES) flags 0x800800 phys_seg 0 prio class 0
[ 2423.006788] blk_update_request: operation not supported error, dev loop0, sector 328 op 0x9:(WRITE_ZEROES) flags 0x800800 phys_seg 0 prio class 0
[ 2423.246709] blk_update_request: operation not supported error, dev loop0, sector 16712 op 0x9:(WRITE_ZEROES) flags 0x800800 phys_seg 0 prio class 0
[ 2423.264232] blk_update_request: operation not supported error, dev loop0, sector 120 op 0x9:(WRITE_ZEROES) flags 0x800800 phys_seg 0 prio class 0
[ 2423.268703] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x800800 phys_seg 0 prio class 0
[ 2423.274219] blk_update_request: operation not supported error, dev loop0, sector 16720 op 0x9:(WRITE_ZEROES) flags 0x800800 phys_seg 0 prio class 0
: direct I/O in unix_io: failed
root@qemux86-64:/usr/lib/e2fsprogs/ptest/test# uname -a
Linux qemux86-64 5.13.9-yocto-standard #1 SMP PREEMPT Mon Aug 9 02:11:15 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
readlink -e is not portable and fails with busybox
When calling e2scrub_all on a system using busybox the readlink command fails due to busybox's readlink not supporting the -e option. See here for details.
e2fsprogs/scrub/e2scrub_all.in
Line 86 in 0670fc2
# SERVICE_MODE=1 /sbin/e2scrub_all -A -r
readlink: invalid option -- 'e'
BusyBox v1.31.1 (2020-02-20 16:09:26 MST) multi-call binary.
Usage: readlink [-fnv] FILE
Display the value of a symlink
-f Canonicalize by following all symlinks
-n Don't add newline
-v Verbose
Support for extended attributes and big UID/GID using debugfs rdump
It seems debugfs rdump command lacks the ability to extract extended attributes and big UID/GID automatically.
Example:
# Expected file attributes
drwxrwxr-x. 3 4012201 4012300 u:object_r:system_data_file:s0 4.0K Feb 8 16:58 dummy_file
# Attributes obtained using debugfs rdump
$ debugfs -R "rdump / mount_dir/" image.ext4
drwxrwxr-x 3 14505 14604 ? 4.0K Feb 8 16:58 dummy_file
Note that rdump reads "14505:14604" instead of "4012201:4012300" and also doesn't read extended attributes
Both uid/gid and extended attributes can be obtained using some additional commands:
# Get uid/gid
debugfs -R 'ls -l' image.ext4
# Get extended attributes
debugfs -R 'ea_list dummy_file' image.ext4
, which then require additional parse commands.
Having support for extracting that info with the "debugfs rdump" command would be very useful
e4defrag crash with inline_data
Hello!
I've found that the join_extents function in e4defrag can exit without
updating ext_group_head. When this happens it causes a crash later at line
1564 where orig_group_tmp is dereferenced.
I suspect this is because I've run it on an inline data enabled ext4
filesystem. It happened on short files and as far as I've understood those
don't have any extents when inlined.
With the patch below it can complete without a crash but it's not the right fix.
diff --git a/misc/e4defrag.c b/misc/e4defrag.c
index c6c6f134..557db9fd 100644
--- a/misc/e4defrag.c
+++ b/misc/e4defrag.c
@@ -1522,7 +1522,7 @@ static int file_defrag(const char *file, const struct stat64 *buf,
/* Combine extents to group */
ret = join_extents(orig_list_logical, &orig_group_head);
- if (ret < 0) {
+ if (ret < 0 || orig_group_head == NULL) {
if (mode_flag & DETAIL) {
PRINT_FILE_NAME(file);
PRINT_ERR_MSG_WITH_ERRNO(NGMSG_FILE_EXTENT);
For reproduction here's a base64 encoded test image called "e4defrag_inline_data_crash.img.gz":
H4sICHpRw18CA2IuaW1nAOzdA5AzBxgG4Deobdu2bdu23f4e1LbtDmpzULsd1LZtt2k2ud/23eV5
Zr61d98YaVnAMamrJkslebqSdElSSj9mL6o53QFbPL5br3qVUqtt9X2pMd2O9f60KaVporae5ZM8
dlxyUDV5cYrb79v2tI2+umzTymYX7FF7KgNY7JBDuyyZkTVehmSNvx5Y84RHN5no3KV3//rCfU/6
oZTVM3EKvfdj1Jo9s6eUgVVLySKBsWupH7/+cMIkMyZZsJH/LimnmsK1z88605E59ZUMxsmbLvNm
AAAAAAAAAIB2oVZbrZz/arVaJWPDMTVgrCknmTil8qJJmt3l8qKLNj/DN3cmLB982FFdFtr3sK6H
7l2MSybPuOMtOleXfY7qkkyS5IFKtVT0FeNuqUzU6F507332PXKP/dI+ARMn763+0MsbTDSY/Bfe
qaSTAvm/ZuVF/07dL5W0GJD/g3defXH5B/kH5B+Qf0D+AfkH5B+Qf0D+OzCQ/5YBAADUphugv2UA
aXcAAAAAAAAAAAAAAAAAAIAdt3h8t96VMeS/gzLWfbtGkmrSo77fvSsNlZRTGD/jJZnw51Kq6auU
ZKKMnFOfSmbPo0sv/dfKUzRq1iWm7H38W8Fjk6elzX9/Wtq+Z6al3fN4xrpjjk2yeLU68O1/KZNn
5GyYIVt4m2T23PJb+tFKt3+7n5GWtuh9aWn//JGx7pE1iiAmXQfKfznTpjB+MojHPxMlmTgj5/V7
ivzfdlPafLHQSUsPnP/yVxmdjk1D78d+vSt9jJ9SY/+TakbMJ8+u9FkGYb+1i/1f/szej/+KKtZd
tDOG3HBs87HsIfX19q40lNJm8sb+Z8Tt9eGK12QQpnyy2P/vb/pknRXu613F+ot2v+e/Ol76s/ce
XfYY1ee/uP77rVF5/peZ88lTMghLv1zs/wZXFtd97yrWXbQDAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAQLtRaf4FfsZJMm6S8dr+9XWCAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAMCw+b89OBYAAAAAGORvPYmd1QUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAIJHj92gAABAA
add e2fsdroid compile
hi! android sdk platform tools can provide e2fsdroid,can i compile e2fsdroid binary for arm64 architecture,and i know someone sucessed compiled,but they were unwilling to offer any help,any helps thanks.
Superblock / partition table mismatch - misleading error message formulation
So, the other day I enlarged my Hyper-V Ubuntu guest's virtual disk using Windows' native tools.
Then, in Ubuntu, I used the 'disks' tool to resize my filesystem. It's supposed to take care of the manipulation of the filesystem AND the partition table, but because of a non-reproducible quirk, it did only the former. After rebooting I got the following error message by fsck:
The filesystem size (according to the superblock) is %b blocks
The physical size of the device is %c blocks\n
Either the superblock or the partition table is likely to be corrupt!
(e2fsck/super.c:714, e2fsck/problem.c:140)
This was very misleading because neither is technically corrupt, there is simply a mismatch. The term "physical size" made me think that the virtual disk somehow reported wrong data. Additionally, it's not clear how e2fsck (and other disk/partition/filesystem tools) calculates or gets the size info.
I think it would be much clearer if the message tells you where the information is coming from, something along the line of
The filesystem size (according to the superblock) is %b blocks
The size of /dev/sda1 (according to the partition table) is %c blocks\n
The reported size of at least one of them is likely to be wrong!
(Setting the partition size manually with fdisk finally solved my problem, for anyone's reference who's looking for a solution to this message)
fuse2fs on chromeos for GIT
I'm using fuse2fs on chromeos because no loop devices.
I have a 16GB file on an SD card accessible via 9p. I've made a filesystem within the file with mkfs.ext2.
My fuse.conf has user_allow_other. I'm running fuse2fs as my own user.
The filesystem works for most things I've tried, but not for git clone.
Git clone fails with a Permission Denied.
Cloning into 'spacemacs.d'...
remote: Enumerating objects: 45, done.
remote: Counting objects: 100% (45/45), done.
remote: Compressing objects: 100% (39/39), done.
fatal: Unable to create temporary file '/a/lydia/homes/lydia/disk01/weaves/spacemacs.d/.git/objects/pack/tmp_pack_XXXXXX': Permission denied
fatal: index-pack failed
I tried a few switches: default_permissions and allow other.
I've tried mounting as root with allow_other.
It's always annoying that Git needs to do so much work with the underlying filesystem. I had hoped by using fuse2fs I could use Git on distributions on a large SD card. It seems not, I hope you can do something.
debugfs write command should handle subdirectories
Using v1.46.1, using debugfs to write a file in a subdirectory produces the wrong result without warning. fsck detects the following anomaly:
Entry 'aaa/bbb/motd' in / (2) has illegal characters in its name.
The following is the test program:
% cat test.sh
cat /etc/motd
rm -f test.img
truncate -s $(( 4 * 1024 * 1024 )) test.img
./local/sbin/mke2fs -t ext2 -L BOOT -b 4096 -I 128 -O ^resize_inode test.img
(
echo 'mkdir aaa'
echo 'mkdir aaa/bbb'
echo 'write /etc/motd aaa/bbb/motd'
echo 'quit'
) | ./local/sbin/debugfs -w -f - test.img
./local/sbin/fsck.ext2 -fn -v test.img
% sh -x test.sh
+ rm -f test.img
+ truncate -s 4194304 test.img
+ ./local/sbin/mke2fs -t ext2 -L BOOT -b 4096 -I 128 -O ^resize_inode test.img
mke2fs 1.46.1 (9-Feb-2021)
Discarding device blocks: done
Creating filesystem with 1024 4k blocks and 1024 inodes
Allocating group tables: done
Writing inode tables: done
Writing superblocks and filesystem accounting information: done
+ echo mkdir aaa
+ echo mkdir aaa/bbb
+ echo write /etc/motd aaa/bbb/motd
+ echo quit
+ ./local/sbin/debugfs -w -f - test.img
debugfs 1.46.1 (9-Feb-2021)
debugfs: mkdir aaa
debugfs: mkdir aaa/bbb
debugfs: write /etc/motd aaa/bbb/motd
Allocated inode: 14
debugfs: quit
+ ./local/sbin/fsck.ext2 -fn -v test.img
e2fsck 1.46.1 (9-Feb-2021)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Entry 'aaa/bbb/motd' in / (2) has illegal characters in its name.
Fix? no
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
BOOT: ********** WARNING: Filesystem still has errors **********
14 inodes used (1.37%, out of 1024)
0 non-contiguous files (0.0%)
0 non-contiguous directories (0.0%)
# of inodes with ind/dind/tind blocks: 0/0/0
44 blocks used (4.30%, out of 1024)
0 bad blocks
0 large files
1 regular file
4 directories
0 character device files
0 block device files
0 fifos
0 links
0 symbolic links (0 fast symbolic links)
0 sockets
------------
5 files
Please add a "Quit Safely" command to the e2fsck interactive mode
My request
It would be great if you could please add a "quit safely" command to the e2fsck interactive mode. The "quit safely" command should quit e2fsck, leaving users confident that they didn't cause any extra damage to the filesystem by quitting early.
Rationale
Some users don't know whether or not it is safe to exit e2fsck using Ctrl+C. "Quit Safely" would leave them confident that they can quit safely and go to bed.
"Quit Safely" would be helpful for e2fsck newbies who want to halt the machine, go to bed, and hire an expert the next day.
"Quit Safely" would also be helpful to e2fsck newbies who want to use a console-mode Web browser or IRC client to get help.
"Quit Safely" would also be helpful to e2fsck newbies who:
- accidentally started the tool in fully-interactive mode
- but realize they should have used -y ("Yes to All") mode.
Those users could simply choose "Quit Safely" then rerun e2fsck with the -y option.
Notes
Dosfsck already has a "Quit Safely" command. (Source.)
Thank you for reading this, and for all your work on e2fsck!
e2fsck: pread/read gets EFAULT due to negative `size`
While fuzzing a e2fsck, I have found that 6-byte altering of correct FS image makes e2fsck try to read a huge amount of data. In fact, this seems to be a regular integer sign-extended from 32 to 64 bits.
Version: 4b4f7b3 from master branch
How to reproduce
Unpack the attached files (original ext4 is only for reference and not used for reproducer), then...
$ cmp -l ext4 ext4.dump
1125 153 60
1126 4 60
1359 100 60
1360 0 60
1361 0 60
1362 0 1
$ LANG=C strace -o /tmp/e2fsck.log e2fsck/e2fsck dump.ext4
e2fsck 1.46-WIP (09-Oct-2019)
ext2fs_open2: The ext2 superblock is corrupt
e2fsck/e2fsck: Superblock invalid, trying backup blocks...
e2fsck/e2fsck: The ext2 superblock is corrupt while trying to open dump.ext4
e2fsck/e2fsck: Trying to load superblock despite errors...
e2fsck/e2fsck: The ext2 superblock is corrupt while trying to open dump.ext4
The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem. If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
or
e2fsck -b 32768 <device>
$ grep EFAULT /tmp/e2fsck.log
pread64(3, 0x7fef0d56f010, 18446744071570460672, 4096) = -1 EFAULT (Bad address)
read(3, 0x7fef0d56f010, 18446744071570460672) = -1 EFAULT (Bad address)
$ echo 'obase=16; 18446744071570460672' | bc
FFFFFFFF80801000
$ LANG=C valgrind e2fsck/e2fsck dump.ext4
==5521== Memcheck, a memory error detector
==5521== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==5521== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==5521== Command: e2fsck/e2fsck dump.ext4
==5521==
e2fsck 1.46-WIP (09-Oct-2019)
ext2fs_open2: The ext2 superblock is corrupt
e2fsck/e2fsck: Superblock invalid, trying backup blocks...
e2fsck/e2fsck: The ext2 superblock is corrupt while trying to open dump.ext4
e2fsck/e2fsck: Trying to load superblock despite errors...
==5521== Warning: set address range perms: large range [0x59c9d040, 0x1da49e040) (undefined)
==5521== Syscall param pread64(buf) points to unaddressable byte(s)
==5521== at 0x489CC0A: pread (pread64.c:29)
==5521== by 0x1702CA: pread64 (unistd.h:117)
==5521== by 0x1702CA: raw_read_blk (unix_io.c:186)
==5521== by 0x1709CF: unix_read_blk64.part.0 (unix_io.c:853)
==5521== by 0x164F92: ext2fs_open2.part.0 (openfs.c:427)
==5521== by 0x121C67: try_open_fs (unix.c:1187)
==5521== by 0x11F6FF: main (unix.c:1508)
==5521== Address 0x1da49e040 is 0 bytes after a block of size 6,450,843,648 alloc'd
==5521== at 0x483A7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==5521== by 0x15F60C: ext2fs_get_mem (ext2fs.h:1822)
==5521== by 0x164F20: ext2fs_get_array (ext2fs.h:1845)
==5521== by 0x164F20: ext2fs_open2.part.0 (openfs.c:396)
==5521== by 0x121C67: try_open_fs (unix.c:1187)
==5521== by 0x11F6FF: main (unix.c:1508)
==5521==
==5521== Syscall param read(buf) points to unaddressable byte(s)
==5521== at 0x489C2E2: read (read.c:26)
==5521== by 0x1703F8: read (unistd.h:44)
==5521== by 0x1703F8: raw_read_blk (unix_io.c:211)
==5521== by 0x1709CF: unix_read_blk64.part.0 (unix_io.c:853)
==5521== by 0x164F92: ext2fs_open2.part.0 (openfs.c:427)
==5521== by 0x121C67: try_open_fs (unix.c:1187)
==5521== by 0x11F6FF: main (unix.c:1508)
==5521== Address 0x1da49e040 is 0 bytes after a block of size 6,450,843,648 alloc'd
==5521== at 0x483A7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==5521== by 0x15F60C: ext2fs_get_mem (ext2fs.h:1822)
==5521== by 0x164F20: ext2fs_get_array (ext2fs.h:1845)
==5521== by 0x164F20: ext2fs_open2.part.0 (openfs.c:396)
==5521== by 0x121C67: try_open_fs (unix.c:1187)
==5521== by 0x11F6FF: main (unix.c:1508)
==5521==
e2fsck/e2fsck: The ext2 superblock is corrupt while trying to open dump.ext4
The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem. If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
or
e2fsck -b 32768 <device>
==5521==
==5521== HEAP SUMMARY:
==5521== in use at exit: 6,450,884,485 bytes in 139 blocks
==5521== total heap usage: 290 allocs, 151 frees, 6,452,061,832 bytes allocated
==5521==
==5521== LEAK SUMMARY:
==5521== definitely lost: 0 bytes in 0 blocks
==5521== indirectly lost: 0 bytes in 0 blocks
==5521== possibly lost: 0 bytes in 0 blocks
==5521== still reachable: 6,450,884,485 bytes in 139 blocks
==5521== suppressed: 0 bytes in 0 blocks
==5521== Rerun with --leak-check=full to see details of leaked memory
==5521==
==5521== For lists of detected and suppressed errors, rerun with: -s
==5521== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
RFE: Please consider (optionally) taking BSD file lock on block devices being called on, to avoid races against udev superblock probing
Since a long time udev has been implementing a simple protocol to ensure that invocations of mkfs/mkswap/fdisk tools don't race against udev's device superblock/partition table probing. See this:
https://systemd.io/BLOCK_DEVICE_LOCKING
I'd like to see mke2fs to implement this natively (it's trivial, just optionally do an flock(dev_fd, LOCK_EX) after open()ing the device). This behaviour should be controllable via an env var ideally, and maybe via cmdline argument, and default to off for compat reasons.
I filed a similar bug against util-linux (regarding fdisk and mkswap) here: util-linux/util-linux#921
Why bother with this in mke2fs? Strictly speaking it's not necessary, people can work around this by locking the bock device outside of the tool and invoking mke2fs with the lock already taken. However, I think it's a good thing to let mke2fs do this natively to make it easier for people, require less glue code around it and make tools such as "lslocks" more descriptive. Ideally all the various tools modifying/creating a superblock/partition table, such as mkfs/mkswap/fdisk/… would implement the same logic here, which is why I am filing these bugs.
if you look at how tools such as gparted try to work around the fact that so far mkswap/mkfs/fdisk/… (randomly masking a multitude of system services it maintains in a blacklist that might react too quickly to half-written superblocks) have no concept of locking whatsoever you can get very sad, this is an attempt to move everybody to just use Linux own concept for advisory locks on its own objects to make everybody a good citizen when dealing with block devices and not stepping on each other's toes all the time...
And yes, implementing this will also fix real bugs. Right now in the impementation of the "tmp" option of /etc/crypttab in systemd, we optionally invoke mke2fs after we set up the device. This is currently invoked as it is, racing against udev's device probing, so that udev racily misses the superblock being created. If mke2fs could automatically lock the block device we can make sure that udev's probing will be delayed or restarted after mke2fs did its job and things would just work. And yes, as mentioned we could solve this via other methods but if we'd properly synchronize here we would certainly solve this in the cleanest possible way.
If this is acceptable I might even prep a patch if need be... Please let me know.
debugfs: add warning to manpage about mounted filesystems
As far as I know, debugfs is not safe to use on mounted filesystems. Unfortunately, the man page doesn't say so:
martin.mein-iserv.de ~ # man debugfs | grep -i mount
Display the multiple-mount protection (mmp) field values. If mmp_block is speci‐
Modify the multiple-mount protection (MMP) data so that the MMP field field has
and debugfs also won't print any warnings when opening a mounted filesystem read-write:
martin.mein-iserv.de ~ # debugfs -w /dev/sda2
debugfs 1.44.5 (15-Dec-2018)
debugfs:
IMO this should be documented.
Test: Embedded linux image with e2fsck and return a 2.
Hello,
I don't know if this the right place to put this question, sorry if it is not. I'm looking your regression tests in order to understand how they are performed because i need to create my own embedded linux image. This image when it boots has to perform e2fsck and it has to return a 2. So it means that it should require a boot after this value.
Checking your regression tests, all of them are img file so it means that they are already an image, so i cannot not use it directly. Is there any way that i can corrupt my image in order to be able to get a 2 from e2fsck?
Thanks for your time,
[e2fsprogs-1.45.6] Installation error when MKDIR_P uses config/install-sh
While attempting a cross-compiler build in a busybox environment, the package configuration script deemed the busybox "mkdir -p" not thread safe and therefore selects the config/install-sh script instead for the Makefile MKDIR_P command variable. The e2fsprogs package is built successfully, however during subdirectory installation 'make' is unable to find the script that resides in the config directory of the parent and not in the current subdirectory.
Installing the GNU coreutils mkdir to replace the busybox version is a workaround and demonstrates that the issue resides with the MKDIR_P selection.
[resize2fs] expanding empty ext4 fs to 2TB always leads to corrupt filesystem
Hi,
Having a problem with resize2fs, not sure if it's a bug or a limitation, or something else. When trying to expand an empty 200MB ext4 filesystem that has a 1K block size, to the end of the 2TB disk, resize2fs appears to corrupt it.
Running fsck afterwards shows these errors:
fsck -f /dev/sdb6
fsck from util-linux 2.31.1
e2fsck 1.44.1 (24-Mar-2018)
ext2fs_check_desc: Corrupt group descriptor: bad block for block bitmap
fsck.ext4: Group descriptors look bad... trying backup blocks...
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Block bitmap differences: +(212993--219700) +(229377--236084) -417793 -417803 -417825 -417829 -417835 -417857 -417862 -417867 -(417889--417893) -(417895--417898) -(417900--417901) -417903 -417918 -417937 -418033 -(418036--418037) -418039 -418041 -(418044--418046) -418050 -418059 -418082 -418085 -418091 -418113 -418118 -(418122--418123) -(418145--418154) -(418156--418159) -418174 -(418193--418194) -(418289--418293) -418295 -418299 -418301 -418303 -(418305--418306) -418315 -(418337--418338) -418341 -418347 -418369 -418374 -418380 -418416 -418430 -(418449--418450) -418545 -418547 -(418549--418551) -(418553--418556) ... 107511290 -107511294 -107511296 -107511298 -(107511319--107511321) -107511330 -107511333 -(107511351--107511353) -107511366 -107511371 -(107511383--107511385) -107511408 -107511422 -(107511441--107511442) -107511541 -107511546 -107511550 -(107511552--107511554) -(107511575--107511577) -(107511585--107511586) -107511589 -(107511607--107511609) -107511622 -(107511626--107511627) -(107511639--107511641) -107511664 -107511678 -(107511697--107511698) -(107511793--107511794) -107511798 -107511802 -107511806 -107511808
Fix<y>? yes
myfs: ***** FILE SYSTEM WAS MODIFIED *****
myfs: 3454/244101120 files (0.5% non-contiguous), 31440202/1952801792 blocks
Tried with two different 2TB HDDs and is reproducible every time, with v1.45.6 too, when expanding to 1024GiB or greater.
If doing a resize to only 1023GiB, the problem doesn't seem to appear. With 2K or 4K block sizes the issue is also not visible.
dumpe2fs output:
-
before expand: https://gist.github.com/acostach/af08dddb56e73b0600559a19d0daeeb0
-
after expand: https://gist.github.com/acostach/c622954e17b7676c240eb4c6b6cef387
Any idea what could cause this?
Thank you!
Badblocks not detecting problems on a known to be failing USB memory stick.
TLDR: Running badblocks -sw /dev/disk/by-id/usb-Verbatim_STORE_N_GO_90006B6E8FEC8E73-0\: on a USB memory stick that is known to corrupt data finishes without error.
I have a old 8GB Verbatim Store-n-Go USB that was corrupting data on a ext4 file system. Copying files to the STORE_N_GO with Gnomes file manager did not bring-up error messages in the GUI but copying from the STORE_N_GO did bring up warnings about IO errors while copying some files. Running e2fsck -c did not show problems.
From there I tried the following:
Run badblocks -sw /dev/disk/by-id/usb-Verbatim_STORE_N_GO_90006B6E8FEC8E73-0\: = No errors
Run badblocks -sw -t random /dev/disk/by-id/usb-Verbatim_STORE_N_GO_90006B6E8FEC8E73-0\: = No errors
format with btrfs -m = scrubing warns about errors.
This made me believe the error only shown up on somewhat random access writes as done by a file system writing data to one area and metadata to a another in quick succession.
BUT running
for i in {1..2}; do pv 8GB-bad\ \[4C9D3D1D\].img > /dev/disk/by-id/usb-Verbatim_STORE_N_GO_90006B6E8FEC8E73-0\:0 ; rhash --simple --uppercase /dev/disk/by-id/usb-Verbatim_STORE_N_GO_90006B6E8FEC8E73-0\:0 ; rhash --simple --uppercase /dev/disk/by-id/usb-Verbatim_STORE_N_GO_90006B6E8FEC8E73-0\:0 ; rhash --simple --uppercase /dev/disk/by-id/usb-Verbatim_STORE_N_GO_90006B6E8FEC8E73-0\:0 ; done
7.21GiB 0:34:47 [3.54MiB/s] [=========================================================================================================================================>] 100%
B3D2829A /dev/disk/by-id/usb-Verbatim_STORE_N_GO_90006B6E8FEC8E73-0:0
B3D2829A /dev/disk/by-id/usb-Verbatim_STORE_N_GO_90006B6E8FEC8E73-0:0
B3D2829A /dev/disk/by-id/usb-Verbatim_STORE_N_GO_90006B6E8FEC8E73-0:0
7.21GiB 0:34:47 [3.54MiB/s] [=========================================================================================================================================>] 100%
3CA97365 /dev/disk/by-id/usb-Verbatim_STORE_N_GO_90006B6E8FEC8E73-0:0
3CA97365 /dev/disk/by-id/usb-Verbatim_STORE_N_GO_90006B6E8FEC8E73-0:0
3CA97365 /dev/disk/by-id/usb-Verbatim_STORE_N_GO_90006B6E8FEC8E73-0:0
Shows that sequential writes as I believe is done by badblocks should detect the error. The file 8GB-bad\ [4C9D3D1D].img was created by making a loop device the size of the STORE_N_GO and running badblocks -sw -t random on it to fill it with random data. The resulting CRC32 is 4C9D3D1D.
At this point I am at a loss on what information I need to provided for you to diagnose this false negative.
MY SYSTEM:
OS: Pop!_OS 20.04 LTS x86_64
Kernel: 5.11.0-7620-generic
e2fsprogs: 1.45.5-2ubuntu1
EDIT:
The issue might be related to Buffered I/O vs Direct I/O as badblocks -B (even in read only mode) does correctly detected the problem areas. I am starting to question my memory of the first e2fsck I ran on the STORE_N_GO it is possible that I did not include the '-c' option the first time, as running e2fsck -c now does detect and add badblocks to the bad block inode. IIRC e2fsck uses Buffered I/O.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.