cryptomator/dokany-nio-adapter

Dokany-based adapter to provide directory contents specified by a java.nio.file.Path (via dokan-java)

Supporting Cryptomator

Cryptomator is provided free of charge as an open-source project despite the high development effort and is therefore dependent on donations. If you are also interested in further development, we offer you the opportunity to support us:

• One-time or recurring donation via Cryptomator's website.
• Become a sponsor via Cryptomator's sponsors website.

Gold Sponsors

Silver Sponsors

Special Shoutout

Continuous integration hosting for ARM64 builds is provided by MacStadium.

---

Introduction

Cryptomator offers multi-platform transparent client-side encryption of your files in the cloud.

Download native binaries of Cryptomator on cryptomator.org or clone and build Cryptomator using Maven (instructions below).

Features

• Works with Dropbox, Google Drive, OneDrive, MEGA, pCloud, ownCloud, Nextcloud and any other cloud storage service which synchronizes with a local directory
• Open Source means: No backdoors, control is better than trust
• Client-side: No accounts, no data shared with any online service
• Totally transparent: Just work on the virtual drive as if it were a USB flash drive
• AES encryption with 256-bit key length
• File names get encrypted
• Folder structure gets obfuscated
• Use as many vaults in your Dropbox as you want, each having individual passwords
• Four thousand commits for the security of your data!! 🎉

Privacy

• 256-bit keys (unlimited strength policy bundled with native binaries)
• Scrypt key derivation
• Cryptographically secure random numbers for salts, IVs and the masterkey of course
• Sensitive data is wiped from the heap asap
• Lightweight: Complexity kills security

Consistency

• Authenticated encryption is used for file content to recognize changed ciphertext before decryption
• I/O operations are transactional and atomic, if the filesystems support it
• Each file contains all information needed for decryption (except for the key of course), no common metadata means no SPOF

Security Architecture

For more information on the security details visit cryptomator.org.

Building

Dependencies

• JDK 21 (e.g. temurin, zulu)
• Maven 3

Run Maven

mvn clean install
# or mvn clean install -Pwin
# or mvn clean install -Pmac
# or mvn clean install -Plinux

This will build all the jars and bundle them together with their OS-specific dependencies under target. This can now be used to build native packages.

License

This project is dual-licensed under the GPLv3 for FOSS projects as well as a commercial license for independent software vendors and resellers. If you want to modify this application under different conditions, feel free to contact our support team.

dokany-nio-adapter's Issues

Deleting read-only files

Commit: 9744f84
OS version: Windows 10 Pro 1809
Dokany version: 1.2.1.2000

If a file is marked as write-protected/read-only, the user can not delete it directly. At first one must unset this file attribute and afterwards the ressource can be deleted.

This stands in contrast to normal windows behaviour where a read-only file can be deleted directly by the owner, as long as the DELETE-flag is set in the ACCESS_MASK.

findFilesWithPattern: wrong search pattern as input

Dokany-nio-adapter version: c3c91de
Dokany version: 1.2.1.2000
OS: Windows 10 Pro 1809
Related bug: cryptomator/cryptomator#837

Description

Some search patterns are translated from the windows world to the dokany-nio-adapter wrongly and thus in the current version there will be no or the wrong search results shown in a directory listing.

Steps to reproduce:

1. Start the ReadWriteMirrorExample with an Folder containing a file named test.txt
2. Open a terminal
3. Change to the mounted directory and enter dir test.*

Results

Expected: test.txt appears in the file listing
Actual: No file is found

Log

11:06:06:036 [Thread-2314] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - Try to open M:\test as Directory.
11:06:06:036 [Thread-2314] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - (340) M:\test opened successful with handle 340.
11:06:06:037 [Thread-2315] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - (340) getFileInformation() is called for M:\test.
11:06:06:037 [Thread-2315] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - (340) File Information successful read from M:\test.
11:06:06:038 [Thread-2316] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - (340) getFileInformation() is called for M:\test.
11:06:06:038 [Thread-2316] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - (340) File Information successful read from M:\test.
11:06:06:039 [Thread-2317] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - (340) findFilesWithPattern() is called for M:\test with search pattern test"*.
11:06:06:039 [Thread-2317] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - (340) Successful searched content in M:\test.
11:06:06:039 [Thread-2318] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - (340) cleanup() is called for M:\test.
11:06:06:040 [Thread-2319] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - (340) closeFile() is called for M:\test.

Add Cryptomator-Icon to Drive

Adding a Cryptomator Icon to unlocked vaults / drives would enhance UX by showing that a vault is unlocked. The feature would also make it easier to distinguish at first glance between a Crypotmator drive and a Windows drive.

Wrong UNC name

Basic Information

Windows: 10 Pro 1903
Adapter: 1.1.10
Dokany: 1.3.1000

Description

Since commit 727f585 also the UNC name of the volume (using the ? notation) mounted with Dokany is given.

This leads to the following (maybe unwanted) feature and a bug.

Feature

Now the volume is also shown under the network device section on a "Dokan1" computer and accessible with a UNC name.

Bug

According to the UNC definition each name starts with \\. But using this notation the volume is not accessible from the network view of the windows file manager. With only one leading \ it works.

Gracefully handle IO errors from the underlying file system

Cryptomator 1.4.2
OS: Windows 10 64-bit build 17134.623
Vault access: Dokany
Vault location: Google Drive File Stream v29.1.78.2103

When running Cryptomator using Dokany and when decrypting a large number of files, cryptomator fails then closes the Dokany session. All further access fails until the vault is manually closed and reopened. This failure occurs on different files at different offsets seemingly at random. It occurs more frequently when more than one vault is open. The vaults are being accessed through Google Drive File Stream and the problem seems to correlate with the load on the internet connection.

17:46:48.937 [Thread-68108] TRACE o.c.frontend.dokany.OpenFile - Reading 471040-475136 (0-1048576)
17:46:48.937 [Thread-68108] TRACE o.c.frontend.dokany.OpenFile - Reading 475136-479232 (0-1048576)
17:46:48.937 [Thread-68108] TRACE o.c.frontend.dokany.OpenFile - Reading 479232-483328 (0-1048576)
17:46:48.937 [Thread-68108] TRACE o.c.frontend.dokany.OpenFile - Reading 483328-487424 (0-1048576)
17:46:48.937 [Thread-68108] TRACE o.c.frontend.dokany.OpenFile - Reading 487424-491520 (0-1048576)
17:47:25.418 [Thread-68108] TRACE o.c.f.dokany.locks.DataRLockImpl - Released read data lock for '/Case/0024/Source7/POSITIONS - Tous les Comptes - Part 2.txt'
17:47:25.418 [Thread-68108] TRACE o.c.f.dokany.locks.PathRLockImpl - Released read path lock for '/Case/0024/Source7/POSITIONS - Tous les Comptes - Part 2.txt'
17:47:25.418 [Thread-68108] TRACE o.c.f.dokany.locks.PathRLockImpl - Released read path lock for '/Case/0024/Source7'
17:47:25.418 [Thread-68108] TRACE o.c.f.dokany.locks.PathRLockImpl - Released read path lock for '/Case/0024'
17:47:25.418 [Thread-68108] TRACE o.c.f.dokany.locks.PathRLockImpl - Released read path lock for '/Case'
17:47:25.418 [Thread-68108] TRACE o.c.f.dokany.locks.PathRLockImpl - Released read path lock for ''
17:47:25.418 [Thread-68108] DEBUG o.c.frontend.dokany.ReadWriteAdapter - (13513) readFile(): IO error while reading file /Case/0024/Source7/POSITIONS - Tous les Comptes - Part 2.txt.
17:47:25.424 [Thread-68108] DEBUG o.c.frontend.dokany.ReadWriteAdapter - Error is:
java.io.IOException: Incorrect function
	at java.base/sun.nio.ch.FileDispatcherImpl.pread0(Native Method)
	at java.base/sun.nio.ch.FileDispatcherImpl.pread(Unknown Source)
	at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
	at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
	at java.base/sun.nio.ch.FileChannelImpl.readInternal(Unknown Source)
	at java.base/sun.nio.ch.FileChannelImpl.read(Unknown Source)
	at org.cryptomator.cryptofs.ChunkLoader.load(ChunkLoader.java:34)
	at org.cryptomator.cryptofs.ChunkCache$1.load(ChunkCache.java:32)
	at org.cryptomator.cryptofs.ChunkCache$1.load(ChunkCache.java:29)
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
	at com.google.common.cache.LocalCache.get(LocalCache.java:3952)
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
	at org.cryptomator.cryptofs.ChunkCache.get(ChunkCache.java:40)
	at org.cryptomator.cryptofs.OpenCryptoFile.read(OpenCryptoFile.java:95)
	at org.cryptomator.cryptofs.CryptoFileChannel.internalRead(CryptoFileChannel.java:184)
	at org.cryptomator.cryptofs.CryptoFileChannel.lambda$read$0(CryptoFileChannel.java:67)
	at org.cryptomator.cryptofs.CryptoFileChannel.blockingIo(CryptoFileChannel.java:283)
	at org.cryptomator.cryptofs.CryptoFileChannel.read(CryptoFileChannel.java:67)
	at org.cryptomator.frontend.dokany.OpenFile.readNext(OpenFile.java:96)
	at org.cryptomator.frontend.dokany.OpenFile.read(OpenFile.java:54)
	at org.cryptomator.frontend.dokany.ReadWriteAdapter.readFile(ReadWriteAdapter.java:324)
	at com.dokany.java.DokanyOperationsProxy$ReadFileProxy.callback(DokanyOperationsProxy.java:72)
	at jdk.internal.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.invokeCallback(CallbackReference.java:520)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.callback(CallbackReference.java:551)

17:47:25.504 [Background Thread 3] TRACE o.c.frontend.dokany.ReadWriteAdapter - unmounted() is called.

Add option to use custom mount options

Adapter version: 1.1.8

Currently a lot of options to mount a volume are hardcoded, see MountFactory,java

For a more flexible approach we should introduce an additional mount method and an option parser.

Creating Files with Office-Applications fail

Basic Info

System: Microsoft Windows 10 Pro (10.0.17134 Build 17134)
Dokany Version: 1.1.0.2000
Adapter Version: 0.1.1

Description

Saving a new file with an Office-application (LibreOffice) in the mirrored directory fails. Editing and saving an already existing one works.

deleteFile(): Try to accquire FileLock throws exception

Commit: d0ab54b
OS: Windows 10 Pro 1809
Dokan: 1.2.2.1000

In the deleteFile() method, the adapter tries to determine if a file can be deleted via accquiring a file lock on the opened file channel. If the channel is read-only this throw an uncatched NonWriteableChannelException.

Some applications create temporary files, save there the current app content and transfer this content to the actual file by opening a read-only file channel to the tmp file. After successful transfer the tmp file (possibly still opened with as read-only) is deleted.

writeFile(): Negative offset when appending to a file

General Info

Commit: 99ff25c
OS: Windows 10 Pro 1809
Dokan: 1.2.2.1000

Description

Sometimes a handle to a file is opened without indicating that the calling process wants to append to the file. In a later writeFile() call the offset will be negative, but DokanyFileInfo.WriteToEndOfFile is set to true.

Currently this flag is not checked and therefore an IllegalArgumentException is thrown due to the attempt of writing to the filechannel with a negative offset.

Steps to reproduce: Mirror the directory of an already existing git repository and make a commit.

Misc

This problem was first described in issue #783 on our cryptomator bugtracker

Relax condition of successful mount

Description

This library considers a mounting operation successful, if the Mounted() method of the Dokany API is called.

This decision was made due to the Dokany implementation, that a mount is in a pending state until Mounted() is called (see https://github.com/dokan-dev/dokany/blob/b5edb0b7a0cc262f61a0b3cdb20a53994352be39/dokan/dokan.c#L292-L345) and until then might be still unexpectedly unmounted.

Unfortunately, due to external system setup, it can take very long time until its execution, or, is never executed at all. To adapt this behaviour, we need to relax the condition on when a mount is considered successful.

findFiles() / findFilesWithPattern: Invalid Memory Access

Basic Info

OS: Windows 10 Pro Ver 1809
Dokany-Version: 1.2.0.1000
Adapter-Version: commit e3d89f5

Description

While browsing through a mounted Volume with a lot of files and folders, sometimes a InvalidMemoryException is thrown during listing the directory content:

15:37:10:292 [Thread-106394] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - (15680) findFilesWithPattern(): found file abc.ESN
15:37:10:292 [Thread-106394] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - (15680) findFilesWithPattern(): found file bbb.ESN
15:37:10:292 [Thread-106394] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - (15680) findFilesWithPattern(): found file ccc.ESN
15:37:10:292 [Thread-106394] ERROR org.cryptomator.frontend.dokany.ReadWriteAdapter - (15680) Error filling Win32FindData with file ccc.ESN. Occurred error is {}
15:37:10:292 [Thread-106394] ERROR org.cryptomator.frontend.dokany.ReadWriteAdapter - (15680) findFilesWithPattern(): Stacktrace:
java.lang.Error: Invalid memory access
	at com.sun.jna.Native.invokeVoid(Native Method)
	at com.sun.jna.Function.invoke(Function.java:414)
	at com.sun.jna.Function.invoke(Function.java:360)
	at com.sun.jna.Function.invoke(Function.java:314)
	at com.sun.jna.CallbackReference$NativeFunctionHandler.invoke(CallbackReference.java:679)
	at com.sun.proxy.$Proxy2.fillWin32FindData(Unknown Source)
	at org.cryptomator.frontend.dokany.ReadWriteAdapter.lambda$findFilesWithPattern$1(ReadWriteAdapter.java:542)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
	at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:430)
	at org.cryptomator.frontend.dokany.ReadWriteAdapter.findFilesWithPattern(ReadWriteAdapter.java:538)
	at com.dokany.java.DokanyOperationsProxy$FindFilesWithPatternProxy.callback(DokanyOperationsProxy.java:112)
	at jdk.internal.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.invokeCallback(CallbackReference.java:520)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.callback(CallbackReference.java:551)

I could caused this by rapidly switching between directories with ~400 files each.

The code where it is thrown is:

dokany-nio-adapter/src/main/java/org/cryptomator/frontend/dokany/ReadWriteAdapter.java

Lines 538 to 549 in e3d89f5

	}).forEach(file -> {
	try {
	if (file != null) {
	LOG.trace("({}) findFilesWithPattern(): found file {}", dokanyFileInfo.Context, file.getFileName());
	rawFillFindData.fillWin32FindData(file, dokanyFileInfo);
	}
	} catch (Error e) {
	//TODO: invalid memory access can happen, which is an Java.Lang.Error
	LOG.error("({}) Error filling Win32FindData with file {}. Occurred error is {}", dokanyFileInfo.Context, file.getFileName());
	LOG.error("(" + dokanyFileInfo.Context + ") findFilesWithPattern(): Stacktrace:", e);
	}
	});

Migrate to Dokany 2.x

Dokany release the next major version 2.x.

The version includes performance improvements, but is not compatible with the 1.x version.

A migration path is provided in https://github.com/dokan-dev/dokany/wiki/Update-Dokan-1.1.0-application-to-Dokany-2.0.0.

Experiments take place in the branch https://github.com/cryptomator/dokany-nio-adapter/tree/feature/dokan-2.0

setFileAttributes(): Attributes cannot be unset

Basic Info

OS: Windows 10 Pro Ver 1809
Dokany-Version: 1.2.0.1000
Adapter-Version: commit e8451d8

Description

In the (private) setFileAttributes() method the caller can set the supported attributes.
The method generates for a given integer mask of attributes a set of FileAttribute objects and the calls for each the FileUtil.setAttribute()-method.

dokany-nio-adapter/src/main/java/org/cryptomator/frontend/dokany/ReadWriteAdapter.java

Lines 580 to 582 in e8451d8

	for (FileAttribute attr : DokanyUtils.enumSetFromInt(rawAttributes, FileAttribute.values())) {
	FileUtil.setAttribute(attrView, attr);
	}

In FileUtil.setAttribute() the value of supported Attributes is set to true.

dokany-nio-adapter/src/main/java/org/cryptomator/frontend/dokany/FileUtil.java

Lines 60 to 77 in e8451d8

	public static void setAttribute(DosFileAttributeView attrView, FileAttribute attr) throws IOException {
	switch (attr) {
	case ARCHIVE:
	attrView.setArchive(true);
	break;
	case HIDDEN:
	attrView.setHidden(true);
	break;
	case READONLY:
	attrView.setReadOnly(true);
	break;
	case SYSTEM:
	attrView.setSystem(true);
	break;
	default:
	LOG.debug("Windows file attribute {} is currently not supported and thus will be ignored", attr.name());
	}
	}

Thus, it is not possible to remove a fileattribute / set its value to zero.

Error handling: Convert uncaught exceptions into generic error code

Basic Info

Commit: b902618
OS: Windows 10 Enterprise N 1809
Dokany: 1.2.2.1000

Description

Currently in every method only exceptions of the type IOException are catched and any other is thrown up to the JNA layer. Since we cannot control in what objects/error codes/what-else these uncaught exceptions are converted, such objects should be caught somewhere and instead the generic NTSTATUS_UNSECCUSFUL error returned.

This also impacts performance because if the error code is returned, the dokan driver gives up on any further file requests with an invalid handle

Add method to API to check if there are open handles

Summary

A method is added to mount object returned by the MounFactory which returns if there are currently any (internal) open handles to filesystem object.

Motivation

From the outside it can be desireable to only close/unmount the filesystem, if currently no operation is running on it/no filechanel open to prevent data loss or unwanted IO results. (see also cryptomator/cryptomator#1228)

Copying files larger than 4gb out of a mirrored directory fails

Version: 0.1.2
Reference: cryptomator/cryptomator#82 (comment)
Copying a file larger than 4GB into a mirrored folder succeeds. But the other direction fails.
E.g. Let be on drive letter K:\ a directory mirrored and in this directory exists a file test.log which has a size of 5GB.
Then trying to copy this file to an NTFS volume with enough size it fails and windows notifies the user with the message "The file 'test.log' is too large for the destination file system." and displays additionally a size of 0 Bytes of test.log

InvalidPathException

Dokany-nio-adapter version: c3c91de
Dokany version: 1.2.1.2000
OS: Windows 10 Pro 1809

Description

Some windows tools (e.g. fsutil) don't check the filename when creating new files. This leads to the throw of an uncatched exception and the wrong or none error code is returned, such that the programs report succes even if the inteded action failed.

Steps to reproduce:

1. Start the ReadWriteMirrorExample with an arbitrary directory
2. Open a terminal
3. Change to the mounted directory
4. enter fsutil.exe file createnew test<.txt 4096

Results

Expected: The file is not created and an error message is displayed "Error: The file name, directory name or volume lable syntax is incorrect"
Actual: The file is not created, but the tool returns a success message "Creation of file test<.txt successful."

Log

15:40:39:846 [Thread-1927] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - (241) File Information successful read from M:\test.
JNA: Callback com.dokany.java.DokanyOperationsProxy$ZwCreateFileProxy@9eb590 threw the following exception:
java.nio.file.InvalidPathException: Illegal char <?> at index 3: yol?
	at java.base/sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
	at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
	at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
	at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92)
	at java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:229)
	at java.base/java.nio.file.Path.resolve(Path.java:515)
	at org.cryptomator.frontend.dokany.ReadWriteAdapter.getRootedPath(ReadWriteAdapter.java:865)
	at org.cryptomator.frontend.dokany.ReadWriteAdapter.zwCreateFile(ReadWriteAdapter.java:89)
	at com.dokany.java.DokanyOperationsProxy$ZwCreateFileProxy.callback(DokanyOperationsProxy.java:56)
	at jdk.internal.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.invokeCallback(CallbackReference.java:520)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.callback(CallbackReference.java:551)
JNA: Callback com.dokany.java.DokanyOperationsProxy$GetFileInformationProxy@307617be threw the following exception:
java.nio.file.InvalidPathException: Illegal char <?> at index 3: yol?
	at java.base/sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
	at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
	at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
	at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92)
	at java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:229)
	at java.base/java.nio.file.Path.resolve(Path.java:515)
	at org.cryptomator.frontend.dokany.ReadWriteAdapter.getRootedPath(ReadWriteAdapter.java:865)
	at org.cryptomator.frontend.dokany.ReadWriteAdapter.getFileInformation(ReadWriteAdapter.java:430)
	at com.dokany.java.DokanyOperationsProxy$GetFileInformationProxy.callback(DokanyOperationsProxy.java:96)
	at jdk.internal.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.invokeCallback(CallbackReference.java:520)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.callback(CallbackReference.java:551)
JNA: Callback com.dokany.java.DokanyOperationsProxy$GetFileInformationProxy@307617be threw the following exception:
java.nio.file.InvalidPathException: Illegal char <?> at index 3: yol?
	at java.base/sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
	at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
	at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
	at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92)
	at java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:229)
	at java.base/java.nio.file.Path.resolve(Path.java:515)
	at org.cryptomator.frontend.dokany.ReadWriteAdapter.getRootedPath(ReadWriteAdapter.java:865)
	at org.cryptomator.frontend.dokany.ReadWriteAdapter.getFileInformation(ReadWriteAdapter.java:430)
	at com.dokany.java.DokanyOperationsProxy$GetFileInformationProxy.callback(DokanyOperationsProxy.java:96)
	at jdk.internal.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.invokeCallback(CallbackReference.java:520)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.callback(CallbackReference.java:551)
15:40:39:862 [Thread-1943] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - (241) cleanup() is called for M:\test.
15:40:39:862 [Thread-1944] TRACE org.cryptomator.frontend.dokany.ReadWriteAdapter - (241) closeFile() is called for M:\test.

Performance: Open FileChannel only when necessary

Dokany-nio-adapter version: 1.1.4

Currently a file channel is always openend when a handle to a file is created, even when the file channel is not used until the closeFile() call.

This can be improved by switching from opening file channels preemptive to only when a real readFile() or writeFile() call is made.

Default mount options are used if none are given

It is not possible to mount a filesystem without any additional mount options. If the mountoption paramter is left out in the additionalOptions string, the defaults are used.

It was possible before.

Migration to JDK 11

Version: 1.1.9

Currently dokany is using Java 9. Since JDK 11 is the current LTS version, we should migrate this library to it.

Google Drive File Stream: Certain applications do not write to file

Dokany-Nio-Adapter: 1.1.14
Dokan: 1.3.1.1000
Underlying filesystem: Google Drive File Stream
OS: Windows 10 Pro 1909

Certain applications are not able to save altered documents when a location of a Google Drive File Stream volume is used.

An example is LibreOffice.

Steps to reproduce:

1. Mirror with the adapter a Google Drive File Stream directory
2. Create a Libre Office document (e.g. with Writer) on the location via the dokan mirror
3. Alter the document and save the modification
4. Close the application
5. Reopen the document

Expected Behaviour

The altered content is shown

Actual Behaviour

The "old" content is shown

Drive letter 'A' not usable as mount point

see dokan-dev/dokany#709

Impossible to mount more filesystems than number of CPU cores

Library version: 1.2.0
Dokany API/Driver version: 140/400

If one tries to mount more filesytems than the number of CPU cores on the system, after this number is reached no more filesystems are mounted.

The reason is that for running the mount processes the ForkJoinPool.commonPool() is used, which uses a parallelism level equal to the number of CPU cores.

Unable to save already exisiting files

Windows Version: Windows 10 Pro 1809
Dokany Version: ecb2d5a

Using the current version of the dokany-nio-adapter, saving already existing files is not possible anymore with specific programs (e.g. altered pdfs with adobe acrobat reader).

It seems the regression happend in commit 2896a5c. The change was that with this commit a filechannel is not opened always with the Write option but it depends on the parameters of the zwCreateFile() function.

Windows tool XCOPY: File metadata preservation issue

System Setup

OS: Windows 10 Pro N 2004
Adapter-Version: 1.1.15
Dokany: 1.4.0

Description

When copying files to the vault with the Windows tool xcopy, it seems that some metadata is not copied along. This holds only for certain file types (e.g. .txt or .msi) while others ( .jpg, .pdf) are not affected.

Steps to reproduce

1. Mirror a directory to a mountpoint (e.g. E:).
2. Create a temporary-folder with some example data
3. Use a Windows Command prompt to run:
   XCOPY temporary-folder E:\Test /E /D /I /Y /V
4. Run the same command a second time:
   XCOPY temporary-folder E:\Test /E /D /I /Y /V

Expected result

The second time no file is copied

Actual result

Some files are copied again.

Returning correct NTSTATUS

Commit: 83f66ec
OS version: Windows 10 Pro 1809
Dokan version: 1.2.2.1000

In the dokan mirror example the NTSTATUS code FILE_IS_A_DIRECTORY is returned when in corresponding the zwCreateFile()-call the ressource is a directory but indicated as anything but a directory:

  if (fileAttr != INVALID_FILE_ATTRIBUTES
    && fileAttr & FILE_ATTRIBUTE_DIRECTORY) {
      if (!(CreateOptions & FILE_NON_DIRECTORY_FILE)) {
        DokanFileInfo->IsDirectory = TRUE;
        // Needed by FindFirstFile to list files in it
        // TODO: use ReOpenFile in MirrorFindFiles to set share read temporary
        ShareAccess |= FILE_SHARE_READ;
      } else { // FILE_NON_DIRECTORY_FILE - Cannot open a dir as a file
        DbgPrint(L"\tCannot open a dir as a file\n");
        return STATUS_FILE_IS_A_DIRECTORY;
      }
  }

We should also follow this convention, currently we return a generic failure code.

Unable to stream video files with VLC

Library Version: 1.2.0
Dokany API/Driver version: 140/400

Playing video files from a dokany mounted filesystem does not work. It worked in previous versions.

Very slow writing speed / Always synchronised writing

Basic Info

Commit: cc16727
Dokan Version: 1.3.1000

Description

Using the mirror test class shows that the maximum writing speed of the adpater decreased by almost 95% (200Mb/s to meager 10 Mb/s). This should be fixed ASAP.

Explanation

The reason for this performance drop lies in commit afbef5d , where the adapter were enhanced by setting the SYNC open option properly. But this is not true.

First, according to the Microsft documentation of the access mask SYNCHRONIZE, this has nothing to do with the Java StandardOpenOption.SYNC. The first one is for waiting until an (IO) operation is finished while the latter one states that changes must not be chached and directly written to the file system. Actually there is no access mask for wrinting directly to the file.
Second the createOptions FILE_SYNCHRONOUS_IOALERT and FILE_SYNCHRONOUS_NON_IOALERT are also flags regarding to wait and notifed when an operation is succesful. Only FILE_WRITE_THROUGH and FILE_NO_INTERMEDIATE_BUFFERING are similar to StandardOpenOption.SYNC.

Add "normal" Windows Explorer behaviour

Environment

OS: Windows 10 Pro 1803
Adapter: 0.1.2

Description

Currently rather than asking the user, if he want to proceed with his action on a mirrored directory, the adapter follows a default behaviour if the user wanted action is invalid. As an example:

1 Create in a empty folder the file test.txt
1 Mirror this folder
1 Create in the mount point of the mirrored folder another file called test.txt

Expected: The file explorer asks if you want to rename your file and aborts otherwise.
Actual: A file with the default name (New Textfile.txt) is created

Stability: Crash of dokany mounted volume

dokany-nio-adapter: 8a977f4
OS: Windows 10 Enterprise N Ver. 1809
Dokany: 1.2.1.2000
Related issue: cryptomator/cryptomator#761, cryptomator/cryptomator#791

Description

There are several reports about a crashes of the provided volume (not responding, no access, etc) when there is high load put onto the volume it.

Steps To Reproduce

TODO

Logs

TODO

FindFilesWithPattern: Wrong pattern matching

With b3fa4a5 the function findFilesWithPattern was reactivated.

Now the pattern matching does not work as expected.

This relates to #19 . We need a DFA to implement it correctly or deactivate it again.

Shutdown hook never invoked

NativeMethods.DokanMain is a blocking method, therefore the following code doesn't get invoked until the drive is unmounted:

dokany-nio-adapter/src/main/java/com/dokany/java/DokanyDriver.java

Lines 61 to 74 in e4e24a9

	try {
	int mountStatus = NativeMethods.DokanMain(deviceOptions, new DokanyOperationsProxy(fileSystem));
	if (mountStatus < 0) {
	throw new IllegalStateException(MountError.fromInt(mountStatus).getDescription());
	}
	Runtime.getRuntime().addShutdownHook(new Thread() {
	@Override
	public void run() {
	shutdown();
	}
	});
	} catch (UnsatisfiedLinkError err) {

The shutdown hook is therefore not only completely useless but even leads to errors, if the drives get unmounted (by a different, working shutdown hook) during program termination:

java.lang.IllegalStateException: Shutdown in progress
	at java.base/java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:66)
	at java.base/java.lang.Runtime.addShutdownHook(Runtime.java:215)
	at com.dokany.java.DokanyDriver.start(DokanyDriver.java:68)

Provisional symlink support

Symlinks are not yet supported by Dokany (blocked by dokan-dev/dokany#343).

Therefore we must make sure, any symlinks found in the underlying file system are ignored during directory listing. Still it should be impossible to create a node with the name of an existing (but unlisted) symlink.

Encode filenames as NFC

Internally we only want NFC filenames. I.e. whenever a file/directory is created or renamed, the name should be normalized using Normalizer.normalize(filename, Normalizer.Form.NFC).

The problem is documented in this thread and can be checked with the following two seemingly identical file names:

NFD-encoded: ü.txt
NFC-encoded: ü.txt

setFileTime(): Don't change FileTime if the given value is the start of the windows epoch

Basic Info

commit: c9139c7
Windows version: Windows 10 Enterprise 1809
Dokany version: 1.2.2.1000

Description

Setting the file time with setFileTime() one can choose the point of the Windows epoch start ( 1601-01-01T00:00:00Z). This indicates that the file time should not be changed, but if this is given into java Files.setTimes() function, the actual file time is changed to now.

Update to JNA 5.x

We should upgrade our code to the latest API to ensure long-term updates.

Windows CMD access the wrong folder

Adapter version: 1.1.15

Description

If one mounts a case sensitive filesystem with the dokany-nio-adapter and navigates through it with the windows cmd shell, the case sensitivity is partially ignored.

For more info, see dokan-dev/dokany#904

Symbolic Link Support

Basic Info

commit: 141a052
Windows version: 10 Enterprise 1809
Dokan version: 1.2.2.1000

Description

The adapter does not support symbolic links. Creating one with the command line tool mklink results in the error message The device does not support symbolic links..

Add support for arbitrary file-paths on NTFS volumes

Version: 0.1.1
On NTFS volumes dokany supports also empty directorys as mount points. CUrrently we only support mounting at drive letters.

Unmount happens when switching user

For the issue in Cryptomator see: cryptomator/cryptomator#708

If the current user is mounting a Filesystem with the adapter and then switches the user account, the filesystem is unmounted. It would be desirable if the volume stays mounted, but can not be accessed by other users.

Use Multithreading

Version: 0.1.1
Currently dokany uses only one thread for handling filesystem requests. This is due to its beta status and to track down basic (and non-multithreading-related) errors:

dokany-nio-adapter/src/main/java/org/cryptomator/frontend/dokany/MountFactory.java

Line 26 in 7f5104f

private static final short THREAD_COUNT = 1;

To increase performance, we should switch in a not-to-distant-future release to multithreading.

Too Long File Names Error Code Support

In windows exists a specific error code if the underlying storage device does not support long file names and the library user tries to create such one.

Currently only a generic failure error code is returned. It is desireable to return the windows specific error code such that applications using this library can react properly.

Delegate revealing a drive to the API consumer

This issue documents the decision made in project https://github.com/orgs/cryptomator/projects/1 in the dokany-nio-adapter repository.

Observe status of mounted filesystem

A dokany mount can be unmounted by external events. We should give downstream libraries the possibility to react to such events by either returning an observable/registerable object when the mount() method is called and let them register an observer.

Remark

This is an upstream issue for cryptomator/cryptomator#1508

Handling of `FILE_NO_INTERMEDIATE_BUFFERING`

The zwCreateFileFunction() is the first function called for any file/directory operation. You can hand over several flags with the parameters, one of them is FILE_NO_INTERMEDIATE_BUFFERING part of the CreateOptions.

The windows documentation describes the purpose of this flag the following:

The file cannot be cached or buffered in a driver's internal buffers. [...]

Currently, when this flag is set, we handle write operations to the opened file as direct IO, leading to a very low write speed.

But this might be overcautiously. Taking this article about buffering from Microsoft into consideration, we do not pay any attention on buffer alignment, this is done by the JVM/System again.

Therefore we should thoroughly test ignoring this flag and then decide what to do.

Invalid size of cipher text file causes abort of directory listing

High level issue is described in cryptomator/cryptomator#673
For the low level causing see cryptomator/cryptofs#29.

We should handle IllegalArgumentExceptions in the methods collecting file meta data, such that valid cipher text files are shown and a message about the not shown/invalid cipher text files is logged.

Problems with write protected files

Basic Info

OS: Windows 10 Pro Ver 1803
Dokany-Version: 1.2.0.1000
Adapter-Version: commit 5df4f08

Description

Write protected files causing "AccessDeniedExceptions" because when opening a file channel the open option WRITE is given into the function. Furthermore they can not be deleted.

Upgrade Guava, fix CVE-2018-10237

We need to upgrade Guava to 25.0 in order to fix CVE-2018-10237.