mkfs.fat manpage: Misleading description of the -h option

[dosfstools]

mkfs.fat manpage: Misleading description of the -h option

[fstirlitz]

More specifically, it is indeed the partition offset, but it's counted from the start of the disk only for primary partitions. For extended partitions, it's from the beginning of the cylinder where the partition starts.

Don't look at me like that. It really is that insane.

(2020-01-11: Scratch that, I investigated a little and it seems it's not actually true. Partition offset is always relative to start of disk, even for logical drives. I remember some in-memory data structure being that way, but it doesn't seem to apply to on-disk BPB.)

(2020-01-15: No, wait, scratch that; my first claim was correct, but apparently not for all DOS versions… I suspect in may depend on whether the MBR partition type is set to the 'LBA' or 'CHS' variant. Sigh.)

[pali]

That is for historic reason... and in FAT context, hidden sectors is well-known technical term which @fstirlitz already described.

[pali]

@harrison3000 and @fstirlitz what about following manpage update?

diff --git a/manpages/mkfs.fat.8.in b/manpages/mkfs.fat.8.in
index 7110732..a9cf4e9 100644
--- a/manpages/mkfs.fat.8.in
+++ b/manpages/mkfs.fat.8.in
@@ -119,9 +119,11 @@ disk geometry of \fIDEVICE\fR. They are stored into FAT boot sector. Number
 By default disk geometry is read from \fIDEVICE\fR itself. If is not available
 then LBA-Assist Translation based on total number of sectors is used.
 .IP "\fB\-h\fR \fINUMBER-OF-HIDDEN-SECTORS\fR" 4
-Select the number of hidden sectors in the volume.
+Specify the number of hidden sectors. This number represents sector which is
+beginning of the \fIDEVICE\fR partition and is stored into FAT boot sector.
 Apparently some digital cameras get indigestion if you feed them a CF card
-without such hidden sectors, this option allows you to satisfy them.
+without hidden sectors value in FAT boot sector, this option allows you to
+satisfy them. By default this value is read from the \fIDEVICE\fR partition.
 .IP "\fB\-i\fR \fIVOLUME-ID\fR" 4
 Sets the volume ID of the newly created filesystem; \fIVOLUME-ID\fR is a 32-bit
 hexadecimal number (for example, 2e24ec82).

Or do you have better idea how to describe it in manpage?

[fstirlitz]

Maybe rather like:

Specify the number of so-called "hidden sectors", i.e. the offset of the partition containing the file system (usually relative to disk start), as stored in the FAT boot sector. If the DEVICE is a partition block device, this value is determined automatically by default.

(Not sure if the camera bits are still relevant.)

Although I guess the actual code may need some fix-ups as well, because it doesn't seem to handle extended partitions as I described.

[pali]

(Not sure if the camera bits are still relevant.)

I do not know if this applies for cameras, but I guess there are other (old?) systems which needs to have correct bits in hidden sectors fields.

Although I guess the actual code may need some fix-ups as well, because it doesn't seem to handle extended partitions as I described.

Probably yes. Question is what should be filled when using GPT or any other partition scheme?

[fstirlitz]

I guess you're right. Maybe squeeze in something like

Some systems may fail to read the file system correctly if this field is not set up the way they expect.

between the two sentences.

From what I've read, the field is mostly useful for (particularly simple) boot loaders, and modern systems usually ignore it. I don't think it matters what the field is on GPT-partitioned media; though I guess I'd fill it just like in the primary partition case, in case someone exposes the partition in a 'hybrid MBR' setup.

[fstirlitz]

I stumbled upon something relevant:

number of "hidden" sectors

[…]

Because of the way that MS/PC/DR-DOS device drivers operate, MS/PC/DR-DOS volume format utilities (and thus all other format utilities that emulate them) set this value incorrectly for secondary partitions located within an "extended" partition (in the MBR partitioning scheme). Thus, without external assistance, the boot code in the remainder of the boot block will fail if a secondary partition is booted directly.

Boot manager programs that allow booting from secondary partitions render this external assistance by up fixing this field as they read the volume boot sector in, adding to its value the block address of the secondary MBR that describes the partition.

But it seems this no longer applies to 'LBA' extended partitions created in MS-DOS 7.0+ (i.e. Windows 9x DOS).

[fstirlitz]

Well, I did some investigations, and I've learned enough to consider my other guess confirmed. I generated a number of disk images with partition tables in various permutations of partition types of the extened partition (0x05 or 0x0f), of the logical volume (0x06, 0x0b, 0x0c, 0x0e), EBR alignment and partition offset from EBR. Each image contained one small primary partition and one extended partition containing one logical volume. Then, for each disk image, I launched a number of versions of DOS in virtual machines and told them to format both the primary partition and the logical volume.

I think the table of results for MS-DOS 7.10 (Windows 95 OSR2) is the most illuminating:

EP ID	LV ID	EBR aligned	offset in the hidden sectors field
0x05	0x06	yes	relative to own EBR
0x05	0x06	no	(undefined behaviour)
0x05	0x0b or 0x0c	irrelevant	(partition is invisible)
0x05	0x0e	irrelevant	(undefined behaviour)
0x0f	irrelevant	irrelevant	absolute

EP ID is the type of the extended partition, LV ID is the type of the logical volume within, and 'aligned' refers to whether the extended partitions were cylinder-ailgned. The table for MS-DOS 7.0 (Windows 95 OSR1) is similar, except that logical volume types other than 0x06 are not recognised, while MS-DOS 6.22 doesn't recognise extended partitions of type 0x0f.

(How do I know the offsets are relative to own EBR and not the main extended partition entry? I checked by formatting two logical volumes under MS-DOS 6.22. I assume other MS-DOS versions behave the same.)

I have also tested FreeDOS; its formatting utility always puts relative offsets in extended partitions, ignoring partition IDs. To my knowledge, FreeDOS's kernel bypasses this issue by not using the hidden sectors field anyway; it accesses partitions by computing the offset from the partition table itself.

In summary, it should be enough to check if the partition is contained in an extended partition of type 0x05, and if so, then use the relative 'LBA' offset in the EBR as the hidden sectors value. Otherwise just use the absolute offset from disk start.

[pali]

Great! Thank you for research. Would you provide an update of manpage, so description of -h would not be misleading?

[fstirlitz]

Maybe this:

Specify the number of so-called "hidden sectors", as stored in the FAT boot sector: this number represents the beginning sector of the partition containing the file system. Normally this is an offset (in sectors) relative to the start of the disk, although for MBR logical volumes contained in an extended partition of type 0x05 (a 'non-LBA' extended partition), a quirk in the MS-DOS implementation of FAT requires it to be relative to the partition's immediate containing Extended Boot Record. Boot code and other software handling FAT volumes may also rely on this field being set up correctly; most modern FAT implementations will ignore it.

By default, if the DEVICE is a partition block device, mkfs.fat uses the partition offset relative to disk start. Otherwise, mkfs.fat assumes zero. Use this option to override this behaviour.

[pali]

Thank you! I pushed update to git master branch.