nahkd123/inking

Java to OpenTabletDriver (OTD) bridge.

Inking

Reading pen tablet inputs from Java. Actually no, it should be "Java bridge to pen tablet drivers".

Build Inking

Instructions on building Inking, including building the natives, can be found in BUILDING.md.

Components

Inking API

The main API component. Inking API by itself is not that useful; you need an implementation.

Inking Implementation (Inking OpenTabletDriver bridge)

Note: As of 06/04/2024, there's only inking-otd implementation, which is the OpenTabletDriver bridge.

Inking implementations like inking-otd or inking-windows (Windows Ink) implements the TabletDriver interface. You can listen for packets, which are states reported from the tablet (pen states, tablet button states, etc).

Inking Manager

Inking Manager provides a simple tablet configuration and filtering system. Configurations are serialized (and deserialized) with Mojang's DataFixerUpper Codec. Filters are connected in a chain, such that the first filter in the list applied first, and then second, the third and so on.

Inking Mananger also provide 2 basic filters: AreaMappingTabletFilter and PressureMappingTabletFilter.

Inking Internals (inking-internal-java21)

Inking internals code that shouldn't be used by your application.

Inking Maven Plugin

The Maven plugin that have 1 single Mojo: strip-preview-flags, which, as the name already said, strips preview flags on all .class files that was included from --enable-preview. This allows Inking to be used on Java 21 JRE without having to explicitly enable preview features.

Warning: This behaviour might be changed in the future.

Inking Samples (inking-sample-otd)

Inking usage through samples. Made for those that want to learn how to use Inking from samples.

License

MIT License. See LICENSE for notices.

inking's Issues

inking-otd: Java 21 (was Java 22)

Move inking-otd out of Java 21 with preview features and make it requires Java 22 or later. Other Inking components will remain at Java 17 for now.

CI: Improve binaries building procedure

This only affect inking-otd, but I thought it would be better to setup a proper procedure for building native binaries.

Currently, binaries are build during pre-package phase. The upcoming change requires maintainers to run a script everytime code for native libraries changed.

For CI, binaries will be built and packaged into artifacts, and those artifacts will be pushed to one single worker to build the rest of Inking.

The workflow will looks like this:

• native-libraries workers: Build just native libraries only
  • win-x64: win-x64 and win-arm64
  • linux-x64: linux-x64 and linux-arm64
  • osx is not included at this moment as I don't have a Mac to test out OTD
• java-library worker: Build Java code after received natives from native-libraries workers
  • Download artifacts
  • Package with native libraries
  • (Optional) Upload artifacts to Maven repository (I do not want to use GitHub Maven repository, because everyone must need to have GitHub account in order to build, which is kinda suck).

inking-manager: Redesign filtering system

Current system is prone to high memory allocation. I believe it's because of allocation of Consumer to pass packets generated from current filter to next filter for every packet emitted. Combine that with my 240Hz tablet and it would fill up the memory, causing GC to trigger more frequent.

The new Packet Filter interface will be something like this:

public interface PacketFilter {
    public void initialize(FilterHost host, Consumer<Packet> pusher);
    public void onPacket(Packet packet);
    public Codec<? extends PacketFilter> getCodec();
}

• initialize(): Called when initializing filters chain, which is when all filters are loaded or the filters ordering changed.
• onPacket(): Called when a packet is received from either the input device or from previous filter in the filters chain. This will always be called in input thread.
• getCodec(): Get DFU codec for filter's configuration.

Additionally, a new AbstractPacketFilter will be added to include additional methods:

public interface IAbstractPacketFilter extends PacketFilter {
    @Override
    default void initialize(FilterHost host, Consumer<Packet> pusher) {} // implemented in abstract class

    public void push(Packet packet);
    public FilterHost getHost();
}

• push(): Push the packet to next filter in the chain or to application if there is no other downstream filters.
• getHost(): Get the filters host, which allows filters to get information about the current tablet and application's info (window size, selected brush, etc.).

inking-linux: libinput integration

See https://wayland.freedesktop.org/libinput/doc/latest/tablet-support.html

Don't know if X11 works but like, does anyone still using X11 nowadays?

inking-otd: Find a way to move tablet configurations out of libraries

OpenTabletDriver.Configurations.DeviceConfigurationProvider constructor get all configurations from resources inside the OpenTabletDriver.Configurations assembly. If rd.xml can be used to remove all .json files in that assembly when using NativeAOT then I want to:

• Move all configurations into a separate location, preferably right inside inking-otd.
• Changes the initialize function (from .NET) to accepts the path to configurations directory.
• When initializing driver (from JVM), the initializer will copies all configurations to a directory, then call the initialize function from native code with the path to configurations directory.
• The function from .NET will have to use reflection (if possible?) to add configurations one by one.

DeviceConfigurationProvider.TabletConfigurations is a read-only property, and DeviceConfigurationProvider.Deserialize() is a private method, so reflection is a must here.

"But why?" - Currently, the configurations are copied to all 6 natives (windows, linux and osx with x64 and arm64 as architectures). Unifying everything into a single set of configurations would reduces storage space a lot.

inking-api: Move back to Java 17?

Note: This does not means inking-otd will be moving back to Java 17 (at least not now). It will stays in Java 21 or maybe updated to Java 22 in the future.

Inking API right now targets Java 21 simply because inking-otd use preview features (Virtual threads + Foreign Functions & Memory API). Moving Inking API back to Java 17 still allows inking-otd to work while also enables different driver implementations that doesn't need features from newer version.

Java 22

Java 22 GA release will be dropped on March 19th. Please reminds me to update to Java 22. Or make a PR. Should be a simple pom.xml change.

inking-otd: Add checksum/native lib version detection

Very important! We need a way to update native libraries.

inking-windows: Windows Ink integration

• Might depends on #2
• C++ binding that interacts with rtscom.h
• Find a way to cross-compile from Linux to Windows (might helps with CI; optional)
• Add API for checking if the tablet device is "virtual". Virtual device always reports pen position in pixels, rather than tablet's hardware position like OpenTabletDriver (OTD).

Windows Ink "driver" can be initialized like this:

long hWnd = 0L; // replace with window's handle
TabletDriver winink = new WindowsInkDriver(hWnd);

I don't want to touch FFAPI from Java 21 for this, because 1. there is JNI header for C++ and 2. user needs to install Java 21 and enable --enable-preview flag OR they can just install Java 22. (FYI: FFAPI was used for OTD because there is no good JNI interface for C# at the time of when I was writing this)