Not readable in Windows 7

[jelchison]

Not readable in Windows 7

[BenKennish]

I also tried to use diskpart to list the volumes connected but it wasn't listed:

C:\Windows\system32
> diskpart

Microsoft DiskPart version 6.1.7601
Copyright (C) 1999-2008 Microsoft Corporation.
On computer: BALOO

DISKPART> list volume

  Volume ###  Ltr  Label        Fs     Type        Size     Status     Info
  ----------  ---  -----------  -----  ----------  -------  ---------  --------
  Volume 0     C   Snake        NTFS   Partition     55 GB  Healthy    System
  Volume 1     D   Solider      NTFS   Partition    111 GB  Healthy    Pagefile

DISKPART>

[BenKennish]

To clarify, the disk is listed in the "Disk Management" window as "Not Initialized" and with the 59.62GB disk space as "Unallocated".

[JElchison]

Hi @BenKennish, thanks for your message. I'm sorry you're having trouble using the script.

This morning before work, I attempted to reproduce your issue, to no avail. I used a 4GB Flash drive previously formatted as UDF, and my Windows 7 laptop detected the UDF drive with no issue.

There was a delay, however. The first time a particular Flash drive is connected to Windows 7, Windows appears to search for and install the drivers that it needs to communicate with this. This only occurs the first time a Flash drive is connected. If Windows failed to identify/install the required drivers on your system, then that may be why Windows never mounts your new UDF Flash drive.

Other differences between your situation and mine:

• I used Ubuntu to run the script. You used CentOS.
• I used a 4GB Flash drive. You used a 64GB Flash drive.

Until I'm able to reproduce your issue, I probably won't be able to help much, unfortuntately. However, I spent some time today brainstorming ways for you to identify what's going wrong.

Here are some things I encourage you to try:

• Check your Internet connection. If Windows needs to reach out to Windows Update to search for a driver, it will likely fail if it cannot reach the Internet.
• Check your Windows 7 log files for any hints
• Ensure that your Windows 7 machine is fully patched. Ensure that Windows Update doesn't have any pending updates to install.
• Ensure that your CentOS machine is fully patched
• Ensure that you're using the latest version of mkudffs(8)
• Try using a different Flash drive altogether
  • Perhaps try using a Flash drive smaller than 64GB
• Try using a different CentOS installation to run the script
  • Perhaps try using an Ubuntu installation to run the script
• Identify what PSP (personal security product) software you're running on Windows 7 (if any). Some PSPs may intercept/block Flash drives that it deems suspicious.
  • Check your PSP logs
• Try using a different Windows 7 installation altogether
• Run badblocks(8) in write mode (-w) to detect whether your Flash drive is starting to go bad. Note that this is a destructive test, and as such will completely blow away everything on the drive (including the UDF formatting).

To answer your last two questions: It sounds like Windows isn't even mounting the UDF Flash drive. Once we get Windows to mount it, it will automatically assign a drive letter.

Feel free to report your findings in the comments below. I'm happy to help where I can.

[BenKennish]

In order to test the USB flash drive for bad blocks but using the OS that is proving problematic, I've used Windows 7 Disk Management to create a single "basic" volume (using MBR) and a single primary partition spanning the whole disk. It is currently (non-quick) formatting using exFAT and is at 40%. I will let you know what happens. If it works, I'm considering just going with exFAT rather than UDF as my PC at home runs Windows 7 and I use a MacBook at work but I would prefer to use UDF and get LInux support too as I am a DevOps guy who works with Linux servers quite a bit.

[BenKennish]

The format completed successfully and running chkdsk from elevated command prompt showed 0 bad blocks. The resulting disk worked fine in both Windows and Mac OS X (Mavericks) but not in CentOS (as expected for exFAT.)

I then attempted to reformat using UDF from CentOS again...

$ blockdev --version
blockdev from util-linux 2.23.2
$ umount --version
umount from util-linux 2.23.2 (libmount 2.23.0: selinux, debug, assert)
$ mkudffs 2>&1 | head -n1
mkudffs 1.0.0b2 for UDF FS 1.0.0-cvs, 2002/02/09

format-udf]$ git pull
Already up-to-date.
format-udf]$ sudo ./format-udf.sh sdb "Gordon"
[sudo] password for ben: 
[+] Testing dependencies...
[+] Looking for drive listing tool...which: no diskutil in (/sbin:/bin:/usr/sbin:/usr/bin)
 using /sbin/blockdev
[+] Looking for unmount tool...which: no diskutil in (/sbin:/bin:/usr/sbin:/usr/bin)
 using /bin/umount
[+] Looking for UDF tool...which: no newfs_udf in (/sbin:/bin:/usr/sbin:/usr/bin)
 using /bin/mkudffs
[+] Validating arguments...
[+] Gathering drive information...
Cruzer Edge     
RO    RA   SSZ   BSZ   StartSec            Size   Device
rw  8192   512  4096          0     64016220160   /dev/sdb
rw  8192   512  4096       2048     64013467648   /dev/sdb1
The above-listed drive (and partitions, if any) will be completely erased.
Type 'yes' if this is what you intend:  yes
[+] Detecting native sector size...
[+] Validating detected sector size...
[+] Unmounting drive...
umount: /dev/sdb: not mounted
[+] Zeroing out any existing partition table on drive...
4096+0 records in
4096+0 records out
2097152 bytes (2.1 MB) copied, 4.49984 s, 466 kB/s
[+] Formatting /dev/sdb ...
start=0, blocks=64, type=RESERVED 
start=64, blocks=12, type=VRS 
start=76, blocks=180, type=USPACE 
start=256, blocks=1, type=ANCHOR 
start=257, blocks=16, type=PVDS 
start=273, blocks=1, type=LVID 
start=274, blocks=125031149, type=PSPACE 
start=125031423, blocks=1, type=ANCHOR 
start=125031424, blocks=239, type=USPACE 
start=125031663, blocks=16, type=RVDS 
start=125031679, blocks=1, type=ANCHOR 
[*] Successfully formatted /dev/sdb: LABEL="Gordon" TYPE="udf" 
Please disconnect/reconnect your drive now.
format-udf]$

The newly formatted disk seems to work in Mac OS X and CentOS but NOT Windows. Upon plugging it in, the device is listed in Device Manager but the drive does not appear in "My Computer". When I load "Disk Management", it prompts me to "Initialize the disk". With nothing to lose, I told it to initialise using MBR but this didn't help. I tried to format using:

mountvol
(took note of the volume id of the disk)
format \\?\Volume\[volume-id] /fs:UDF /q

but it gave an error about the volume not having a drive letter or a mount point.
sigh

[JElchison]

Hi @BenKennish. A few observations...

Did chkdsk perform a destructive block check? If not, then its results may not be as accurate as they could be.

Also, any time you do something that writes a MBR to your Flash drive, that will break the UDF formatting. UDF doesn't use an MBR. For more info, please see http://serverfault.com/questions/35243/what-is-the-partition-id-filesystem-type-for-udf

Since the UDF drive works on OS X, it's unlikely that the format-udf script is at fault. Based on your data, my educated hunch tells me that the problem is one of the following two:

• You're not running the latest version of udftools
• Something is wrong with your Windows machine

I've crossed off the bottom bullet from my list of suggestions above. If you're still interested in digging deeper, there are plenty more diagnostics in that list that you can perform.

Good luck!

[BenKennish]

I performed a format with "quick format" unchecked and was under the impression that this would do a destructive block check (it certainly took a long time) but I'm not 100% sure. The chkdsk command that I ran afterwards was just a basic one to get it to report the count of bad blocks (from the filesystem metadata.)

Just to rule out weird random data, I'm trying a complete zero of the disk:

# blockdev --report
RO    RA   SSZ   BSZ   StartSec            Size   Device
....
rw   256   512  4096          0     64016220160   /dev/sdb
# expr 64016220160 / 32768
1953620
# dd if=/dev/zero of=/dev/sdb bs=32768 count=1953620

When this finishes, I will run the script again and report

I don't think it's a problem with my Windows 7 as I just got a colleague with Windows to read the same disk with the same results. The udftools that I am using is the latest version available for CentOS 7 but I have a Kali (Debian-based) guest that I will try after the zero-ing. Cheers

[JElchison]

If all you care about is getting it to work on Windows, you can always use Windows to format UDF for you. However, I can't guarantee that the resultant drive will work on Linux or OS X.

http://superuser.com/a/727093

[BenKennish]

I may give that a try if it doesn't work after the zero-ing. Ultimately I would like at least Windows and Mac OS X compatibility tho as my home PC is Windows 7 and I have a Mac Book at work. I'll let you know how it goes. Cheers.

[BenKennish]

OK I actually ran

badblocks -c 512 -o ~/badblocks.log -wvs

and left it running overnight but it didn't complete - it was about 60% of the way through the 64gb and with no errors in the .log file the last time I checked and then VirtualBox crashed ;-( In any case, I think we can rule out bad blocks from causing this problem as I doubt Windows is trying to read/write in that part of the disk to detect the filesystem.

In despair, I let Disk Management initialise the disk (using MBT) and then created a single partition that filled the disk. I then attached a drive letter and formatted it using:

format F: /fs:UDF /q

Both Windows and the CentOS guest could recognise the disk and I was looking forward to telling yout that I've fixed the problem but unforuntately I've just arrived at work today and Mac OS X Mavericks tells me that it cannot read the disk. Disk Utility lists the partition (as disk2s1) but says "Format: MS-DOS (FAT-16)"

sigh

[JElchison]

It's not surprising that a Windows-formatted UDF drive isn't readable on other OSes--especially if you let it initialise the disk (using MBR).

I still think your best bet is to use the latest udftools (1.0.0b3) from a Linux distro. Did you ever try it on Kali? Ubuntu would suffice as well.

See the bulleted list above for more ideas. I'd still encourage you to try plugging a UDF-formatted drive into another Waindows machine--perhaps one that isn't at work. It's possible that your corporate group/security policy (or PSP software) is still interfering with what you're trying to do.

[JElchison]

What's most troubling to me yet is that:

• I'm still unable to reproduce your issue
• You're still unable to reproduce my success

If you're willing to invest the time, I'd love it if you could cross off more of my suggestions in the above bulleted list. But I totally understand if you don't have the time.

[BenKennish]

I ran format-udf on the disk earlier so it's now readable to Mac OS X and Linux but I imagine not Windows (when I get home). Could there be something about the SanDisk Flash drive Windows driver that stops UDF on the whole disk (as opposed to as an MBR partition) working? I will have another go this evening with a different Linux distro, proably Kali or Ubuntu, and using my wife's Win 7 laptop. Cheers.

[JElchison]

Yes, that's a definite possibility. And, it's the reason I suggested trying a different Flash drive altogether. :)

Windows Update could be getting in the way as well. If Windows Update is blocked at work (or if it's brokered by your sysadmins), then it's possible that Windows couldn't fetch the driver(s) needed to use your Flash drive properly.

[BenKennish]

OK last night I tried on my PC and my wife's PC with and without anti-virus disabled with the same failure result. At this stage, I'm going to give up and use exFAT on a single MBR partition as that seems to work on Mac OS X and Windows (for my Linux I'll either try and get exFAT working or use a VirtualBox shared folder to share files with a guest.) FYI:

USB Drive: 64GB SanDisk Cruzer Edge

The thing is that I definitely had it working previously but due to filesystem corruption (caused by leaving it outside overnight for 14hrs) I had to reformat and start again. But I really don't think these problems are caused by physical failure. Oh well. Feel free to close this issue if you wish. Thanks for your help

[JElchison]

Thanks for reporting back. For future reference, did you ever attempt using the latest version of udftools (1.0.0b3)? Or Kali? And did you ever try a different Flash drive altogether?

[pali]

@JElchison: What do you mean by "lastest version of udftools"? Lastest "official" version is too old and has couple of bugs. And every Linux distributions include its own patches version of udftools (including mkudffs). So I think there is no "lastest" version, but there are lot of divergented new versions.

Anyway I have similar problem. Windows 7 cannot read formatted UDF 8MB flash disk by Ubuntu 12.04 mkudffs. Everytime when I tried to format I did full disk erase (dd if=/dev/zero of=/dev/disk). I tried also formatting without MBR table (and parititions). Foratting in Windows worked fine and worked also on Linux with newfs_udf tool from BSD UDFClient package (for compilation on linux is needed BSD make -- GNU make will not work).

So I think problem is in mkudffs, is create UDF volume in some conditions somehow incomatible with Windows...

[JElchison]

Hi @pali, my comments earlier in this thread are clear on what I mean by "latest" version. If you're running a version of udftools older than 1.0.0b3, then you need to update--regardless of what distro you're using. If your distro doesn't have an appropriate fork of 1.0.0b3, you should consider updating to the latest-offical package.

I'm sorry to hear that you're having a problem with mkudffs. You're commenting on a GitHub project (format-udf) that is a wrapper for mkudffs. You're welcome to use this forum to exchange ideas concerning your issue, but unfortunately I'm not able to offer support for udftools/mkudffs.

Good luck!

[JElchison]

Hi @BenKennish and @pali,

I've recently encountered some drives that regularly exhibit the problem both of you have experienced. I'm interested in characterizing the problem. Would you be willing to help by adding info about your problem drive(s) to the table on this wiki page?

With my preliminary data, it appears that Windows somehow differentiates block devices according to whether they're a Flash drive or HDD.

Thanks!

[JElchison]

Oh, also: If you can do without some of the POSIX-compatible features, exFAT isn't a bad alternative to UDF. It's natively supported in both Windows and OS X. In Linux, you can enable support with third-party utilities. In Ubuntu, this simply means sudo apt-get install exfat-utils exfat-fuse

[pali]

No, exFAT is even worse as you can imagine! Also it is full of MS patents and without specification. There is no usable Linux support (that fuse stuff is slow and unstable, nothing for real data storage). And exFAT does not solve any FAT32 filesystems problems (except filesize limit per file). So I do not see any reason why to use exFAT.

[JElchison]

@pali: I don't disagree. But, for people who care more about compatibility than performance, do you have a better cross-platform alternative to UDF?

FWIW, exFAT also improves on FAT32 with maximum volume size and allowable characters in directory entries. For a quick summary, see this page.

[pali]

No, there is no other alternative. Windows supports only FAT12, FAT16, FAT32, NTFS, exFAT and UDF. (Plus ISO9660 which it is read-only)

I do not know who can prefer compatiblity instead stable quality... (maybe just some testers who plays with filesystems). Filesystem is for storing data files and it must be stable... I do not know any stable implementation of NTFS or exFAT except one from Microsoft. I saw NTFS filesystem which was damaged by fuse ntfs-3g... For me ntfs and exfat fuse implementation are (very) useful only for data recovery.

So there is only FAT32 and UDF, just choose between those two. I think that Linux and Apple systems have stable implementation of FAT and UDF (or at least more stable as for exFAT or NTFS). But there is no Linux fsck tool for UDF (you can still use one on MS or Apple system).

[JElchison]

Hi @BenKennish and @pali,

I've recently made some updates to format-udf that you may be interested in. Version 1.1.0 of the tool includes a new feature that writes a fake MBR for added UDF compatibility on Windows.

If you're interested in giving it a try, I'm interested in learning whether the new version addresses your issues. Please report back in this thread. thanks!