ddvk / remarkable2-framebuffer Goto Github PK

View Code? Open in Web Editor NEW

267.0 24.0 22.0 11.84 MB

remarkable2 framebuffer reversing

License: MIT License

C++ 2.34% QMake 0.24% Shell 0.06% Python 0.23% C 97.12%

remarkable2-framebuffer's Introduction

remarkable2-framebuffer

This repo contains code for drawing to the rM2's framebuffer.

Status

quality: beta

rm2fb can open the framebuffer and draw to it. rm2fb-server exposes a simple API for other processes to draw to the framebuffer using shared mem and message queues. rm2fb-client is a shim that creates a fake framebuffer device for apps to use, allowing rM1 apps to seamlessly draw to the display of the rM2.

Installation

Before installing, be sure that your xochitl version is supported: https://github.com/ddvk/remarkable2-framebuffer/blob/master/src/shared/config.cpp.

rm2fb is available as a package in toltec which sets up the server and client parts for you. Otherwise, the instructions below can be used for manual installation.

Set up build environment

set up remarkable toolchain

Building

Source the remarkable toolchain (example below) and run the following (replacing ${GITROOT} with the directory you have checked the repository out to)

source /usr/local/oecore-x86_64/environment-setup-cortexa9hf-neon-oe-linux-gnueabi
cd ${GITROOT}
qmake
make

Framebuffer Server

build src/server/librm2fb_server.so.1.0.1. Copy it to your remarkable and run:

LD_PRELOAD=/path/to/librm2fb_server.so.1.0.1 /usr/bin/xochitl

Framebuffer Client Shim

build src/client/librm2fb_client.so.1.0.1. Copy it to your remarkable and run: LD_PRELOAD=/path/to/librm2fb_client.so.1.0.1 <rm1app> to run your app.

The client intercepts interactions with /dev/fb0 and forwards them to the rm2fb server.

Configuration

To do their job, both the client and the server need to know the address at which a number of functions reside in the Xochitl binary. These addresses change from one release to the next.

Xochitl function name	Xochitl function role	Notes
`update`	Sends an update request. This function is replaced by the client shim to talk to our server instead.	Uses the string `Unable to complete update: invalid waveform (`. Prior to version 2.9, the signature of this function was `void ()(void, int, int, int, int, int, int)`. Starting from version 2.9, it changed to `void ()(void, QRect&, int, int)`.
`create`	Start the threads that handle update requests. This function is replaced by the client shim with a no-op to avoid conflicting with the server.	Uses the string `Unable to start generator thread\n`.
`wait`	Waits until the update-handling threads have started. This function is replaced by the client shim with a no-op to avoid conflicting with the server.	Calls `usleep(1000)`.
`shutdown`	Stops the update-handling threads. This function is replaced by the client shim with a no-op to avoid conflicting with the server.	Uses the string `Shutting down...`.
`getInstance`	Retrieves the instance of the singleton SWTCON class. This function is used by the server to interact with the screen.	Calls a function that itself calls `create` and `wait`.
`notify`	Called when the framebuffer has been updated, used for the Qt signal/slot connections needed for ScreenShare to work

The client and the server both ship the hardcoded addresses for these functions for various releases. If you get a message saying Missing address for function […], it means that the release you’re running is not yet supported. Please report this in this dedicated thread.

You can manually locate the addresses of the functions listed above by looking at the disassembly and then add a configuration entry to make remarkable2-framebuffer work with your release. In addition to the hardcoded configuration entries, the client and the server will look for addresses in configuration files in the following locations:

/usr/share/rm2fb.conf
/opt/share/rm2fb.conf
/etc/rm2fb.conf
/opt/etc/rm2fb.conf (best option for Toltec users)
rm2fb.conf (relative to the current working directory, best option for manual installs)

Demo

Contributing

Please look at the open github issues and add a new one for any work you are planning on doing so people can see ongoing progress.

Things that can use help:

writing documentation
testing apps and packages
understanding the waveforms used by SWTCON
writing our own implementation of SWTCON
adding support for new OS releases

FAQ

will this brick my rM2?

probably not, but no guarantees

use github issues or ping on the discord in one of the homebrew developer channels. if you mention this repo, someone will probably respond

remarkable2-framebuffer's People

Contributors

Stargazers

Watchers

remarkable2-framebuffer's Issues

Stuck on Starting Screen - Toltec and RM2FB Install Issues

Hi all,

I had a go with Toltec and then I installed "opkg install rm2fb-client"
Then I installed draft but that didn't work.

I have uninstalled everying using "toltecctl uninstall" but....

My RM2 is stuck on the starting screen.
I can still SSH into it and when I type "xochitl" it the software works fine, but it says "Codex is still running" in a banner on the bottom of the screen.
On a restart, it gets stuck on starting
On factory reset, it gets stuck on starting but when I ssh and type "xochitl", it starts back up and works fine (and I took note of the new SSH password!). This seems positive!

My laptop battery is dying, but that is all the info I have now.

Any ideas?

Many thanks!

implement swtfb for libremarkable

[toltec rm2fb 1.0.1-1] 2.6 Feedback

I just tried new rm2fb toltec pack and this works greatly. GJ !

I just want to inform that the new pinch to zoom its kinda laggy and a bit inaccurate. The input itself is right but the screen doesn't show exactly where I am when in moving the 2 fingers during the gesture.
I hope that i express my self, is easier to watch lol

I guess is the limit of framebuffer server/client solution.

p.s. Thanks for the great work all of you do for community!

implement swtfb for qsgepaper shim

roadmap for rm2fb v1

here is tentative plan for rm2fb:

Phase 1: (investigation)

figure out how to draw to screen
figure out where xochitl keeps its memory

Phase 2: (compatibility)

get a simple app for rM1 working on rM2
find out what needs to be done for all apps to upgrade

Phase 3: (single apps)

Phase 4: (distribution)

implement / adapt a launcher to launch apps with the rm2fb client shim
add ability to screenshot xochitl
add ability to screenshot rm2fb
add toltec packages for rm2fb

Phase 5: (beyond)

rm2fb protocol v2: support windowed / split screen applications
implement SWTCON that uses waveforms to draw to screen
swap current rm2fb server to use open source SWTCON
look into FUSE filesystem for /dev/fb0

After installig rm2fb screen frozen

Hello,

I just installed toltec after that everything is fine.
If i install rm2fb - client through toltec the screen gets frozen but i still have acces through ssh.
if i reboot the rm2 gets stucked in "loading".
if i remove rm2fb / toltec everything works again.
Is this a problem with the 2.10.0 Beta software version?

Auto-update address definitions

@Eeems suggested on Discord to make rm2fb automatically fetch the address definitions from a remote server when they are missing. This would make adapting rm2fb to new OS releases easier, since we’d just need to update the remote file and we wouldn’t need to publish new releases/package versions.

koreader touch input doesn't work (stylus input is fine)

After setting up koreader on remarkable2, I have a problem with a touch input. It doesn't work. Operating using stylus works without any problems, but the touching using fingers does not. Is it a issue of the current remarkable2-framebuffer set-up or the koreader itself?

remarkable bricked, unclear why

I have, in that order :

Enabled wifi from xochitl for the first time since unboxing it back in november,
Connected to it using ssh (through usb, with ssh keys and after writing down the password)
Used that internet connection to install toltec, using the three commands in their readme
added opkg to my PATH ? not sure why I had to do that but I had to add /opt/bin to $PATH
I have installed rm2fb from toltec (opkg install rm2fb)
Have installed koreader manually using the latest release at the time.
started the koreader service
it runs well, I'm happy to see it works
I then disabled xochtil, enabled koreader, since I want koreader at startup not xochitl
After using koreader for about 10 minutes it crashed down.
10 . The remarkable does start but does not start xochitl or koreader at startup (not so surprised, could be a lot of things)
I cannot ssh into it, it looks like sshd does not start at all. (This i did not expect !)

If anyone knows why the remarkable won't let me ssh into it I am genuinely interested, I guess I knew I was playing with fire but even looking back now I have no idea how i could lose ssh access doing what I did.

Note : I am not expecting to get it to work again, or complaining, or asking for refund or whatever. I am just sharing my experience, proving myself my own stupidity once again and asking if by any chance anyone knows what could have happened there.

test rm2fb toltec package

toltec-dev/toltec#143 is the PR for the rm2fb package

the rm2fb package can be found at https://toltec-dev.org/stable/rm2fb_1.0.0-1_armv7-3.2.ipk.

to test the package:

install with opkg install rm2fb_1.0.0-1_armv7-3.2.ipk
verify xochitl restarted
verify that /dev/shm/swtfb.01 now exists
test out launchers like draft, oxide or remux
install other apps from toltec: plato, retris & chessMarkable are not yet rm2 compatible
report any issues on this task

if you run into problems: ssh in with the USB cable, remove rm2fb with opkg remove rm2fb and restart xochitl systemctl restart xochitl. also disable / uninstall other launchers

debugging tips:

journalctl -u rm2fb and journalctl -u <service> will show logs
if the rm2fb server is running, remarkable-shutdown will show up in the process list
to run an app from command line, use rm2fb-client <app> - it will add the LD_PRELOAD client shim

~~for testing rmkit apps:~~
* use toltec's testing repo instead of stable repo or download binaries from https://build.rmkit.dev/master/latest/
update jan 04 2021: rm2fb is in toltec stable repository

proposal for layering / compositing

One feature that might be nice is layering. This task is for outlining how layers could work (how many layers should there be, what is the cost of layers, what are some potential uses, etc)

demo.gif is a dead-end git-lfs pointer

In README.md, there is a link to a file in the repo's ./docs/ directory, which looks as if it's supposed to be a 17.6 MB animated GIF, but turns out to be a 133-byte manifest file when cloning the repo locally or previewing it in github:

As far as I can tell, this seems like an issue relating to Git LFS – perhaps due to missing dependencies in the system from which the file was originally committed/pushed. This discussion may shed some light on cause and/or solution.

rm2fb is not working on 2.6 (rm2)

rm2fb is not working on 2.6 (rm2). blank screen

Screensharing in 2.9 doesn't work

First discussed here, ddvk/remarkable-hacks#344
The updates don't seem to work with rm2fb_client even with the unmodified xochitl binary ...

add a CI pipeline to generate and publish the binaries

[release] bump assets to 1.0.1

How can I tell which rm2fb version is on Toltec? It shows that it has version 1.0.0-7, and I'm just wondering when I can tell that this has been updated to the newest version v0.0.3. I'm looking forward to the new PR that it includes as well as just future reference as newer versions are released. Thank you!

Bricked Device after Toltec install

Not entirely sure what happened, but I managed to brick my device installing with toltec. I installed over usb ssh and as soon as the install completed my screen blanked; I was unable to start xochitl via systemctl because the process was exiting (w/ status 255 IIRC) but I didn't get to dig any deeper than that. After uninstalling I was still unable to start xochitl but after a restart things went back to normal.

Since the blanked screen didn't interrupt my ssh access the first time, I decided to try installing again and rebooting to see if it would work. Unfortunately after doing this, my device won't show up on the network at all (and the screen is still blank). The startup screen shows briefly after restarting but otherwise nothing shows up.

Not sure if it's related but I had just updated to 2.8 before setting up toltec and running the install.

I probably won't be able to give many more details on this for a while since I just ordered the parts for https://github.com/ddvk/remarkable2-recovery from alibaba, but can circle back when those arrive.

LD_PRELOAD doesn't work with some binaries from toltec

reMarkable: /opt/lib/ LD_PRELOAD=/opt/lib/librm2fb_client.so.1.0.0 grep ERROR: ld.so: object '/opt/lib/librm2fb_client.so.1.0.0' from LD_PRELOAD cannot be preloaded (internal error): ignored. Usage: grep [OPTION]... PATTERNS [FILE]... Try 'grep --help' for more information.

reMarkable: /opt/lib/ file which grep /opt/bin/grep: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /opt/lib/ld-linux.so.3, for GNU/Linux 3.2.0, stripped

LD_PRELOAD works fine with the busybox grep:
reMarkable: /opt/lib/ LD_PRELOAD=/opt/lib/librm2fb_client.so.1.0.0 busybox grep BusyBox v1.30.1 (2020-11-04 10:42:32 UTC) multi-call binary. Usage: grep [-HhnlLoqvsriwFE] [-m N] [-A/B/C N] PATTERN/-e PATTERN.../-f FILE [FILE]...

reMarkable: /opt/lib/ file /bin/busybox.nosuid /bin/busybox.nosuid: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=e9c75f6d853dde1bbf974fdf11daa3493f4eadde, stripped

I suspect it's a matter of the interpreter. If it's ld-linux-armhf.so.3, it works fine, if it's /opt/lib/ld-linux.so.3 it shows error.

Client does not work with remarkable.net

When using Remarkable.net error messages can be seen in stdout and the display doesn’t update. The error messages can be tracked back to display refresh code in remarkable.net and specifically the C# ioctl call into libc returning -1.

I debugged this at great length with a debug build of the r2fb client and enabled function level debug logging going to stdout.

The client library was preloaded correctly into dotnet and the function tracer output could be seen. However when the main application (my DLL instead of the dotnet interpreter binary) was running it appeared that the function hooks were not being called at all.

I suspect that the existing open/close/ioctl hooks don’t work at all with dotnet applications.

As a workaround I preloaded the binary but also invoked functions directly in the client library instead of libc. At which point everything worked as expected.

I will put a PR into remarkable.net to call the client libraries directly but long term it might be better to have an API (socket, tcp) to use instead as when the client version number changes remarkable.net will break.

Tiny refresh glitch

I don't know enough low-level details to diagnose whether this is a KOReader issue, an rm2fb issue, or some combination of both.
KOReader v2021.01
rm2fb v0.0.2

Sometimes KOReader info boxes pop open only half-formed. I can consistently reproduce it with the "Network Info" message, showing here:

https://imgur.com/a/Mq1fWOl

After the half-mangled message appears, I can swipe to move it a little bit, causing it to cleanly refresh (also shown in the brief clip).

Not a huge deal for my day-to-day usage, although I did have it happen once with a confirmation dialog, so I couldn't read the "yes/no" option buttons until I carefully moved the dialog a bit.

document how to port apps from rM1

For an rM1 app to work on the rM2, they need to do the following:

install rm2fb server and client libraries
run rm2fb server (see README.md)
run the rm1 app with LD_PRELOAD=/path/to/client.so <app>

The rM1 app will now be able to draw to screen.

NOTE: the other two main pieces of compat are the battery location and handling inputs, see Eeems-Org/oxide#48 for discussion of those two.

implement swtfb for rmkit

refresh rate

If this project succeeds, can I use it as a second monitor with a decent refresh rate (like Dasung monitor)

implement wait for update ioctl

in mxcfb on rM1, using ioctl there is ability to do:

send_upate { update_marker: X };
wait_for_update { update_marker: X};

in rm2fb, there is no longer the "wait for update" ability due to two things:

there's only single way communication from client -> server
the framebuffer API has a command, "waitForLastUpdate()" but it does not support supplying a marker ID (i think). translating "send_update; wait_for_update ID" into "send_update; waitForLastUpdate()" requires knowing when to call waitForLastUpate() after drawing an update, but the wait_for_update can come at any arbitrary time and we process the incoming events in order.

this task is to give the ability for rM2 apps to wait for a previous draw to finish.

Segmentation fault with version 0.0.6

I just upgraded my Remarkable to version 2.9.0.210 and remarkable2-framebuffer to version 0.0.6 which is supposed to be compatible, but I get:

308 Segmentation fault

However, with the previous version emarkable2-framebuffer 0.0.5 I have no problems with version 2.9.0.210 of my Remarkable.

peer into xochitl memory and expose it as shared memory

design server / client IPC for sharing framebuffer

background

The rM2's framebuffer is not well understood, but we know it requires a software thread (SWTCON) to drive it. This thread runs with high priority and is responsible for using data from a memory location to drive the contents of the display.

We are able to get access to the framebuffer update APIs by using LD_PRELOAD while loading the remarkable-shutdown binary and hardcoding their memory addresses. The update API is similar to the way the rM1 updates: it takes a rectangle and a few arguments (like waveform_mode) and updates the display.

We propose using a client/server model for interacting with the framebuffer, instead of each applications starting its own SWTCON. (supernote a6 uses a similar design: only one process interacts with the framebuffer and the rest use IPC to communicate with it.)

proposal

have a server process that drives the SWTCON. It opens a shared memory location (/dev/shm/swtfb) and listens over IPC for clients to invoke the update API.
client process can write to the shared memory location and then invoke the IPC API to tell the server which region to update and any additional flags.

Underlying technology:

use shm_open (POSIX) shared memory
use msgrcv/msgsnd (SysV) message queue

remarks

advantages of using a server/client model

the LD_PRELOAD method is fragile and is awkward to launch, isolating that technique to one place will make it easier to manage
easier for different languages to interface with - they only need to make IPC calls
with a client/server API for the framebuffer, we are opening up the path to supporting split applications.
with shared memory, the framebuffer can be screenshotted easily. if each application uses its own SWTCON, they have to expose their framebuffer's memory for screenshotting

potential pitfalls

the API across IPC needs to be stable or upgradeable so that rm2fb doesn't have mismatches with client versions
we are creating extra headache by having to create and maintain a server
we might not achieve low latency drawing if we use IPC
what's the point of server/client? what will you do when SDK comes out?

open questions

do all clients share the framebuffer memory or does each client request its own?
do we deal with input sharing now or later?
what does API look like?

prototype

see master branch

usage document for rm2fb

rm2fb is a server/client model for communicating with the framebuffer on the rM2.

building and running

source rM toolchain
run qmake in top level directory
run make in top level directory
server binary is now in src/server/
copy server binary to rM2 and run with LD_PRELOAD=librm2fb.so remarkable-shutdown

how rm2fb works

server

the server of rm2fb (librm2fb.so) injects itself into the remarkable-shutdown binary using LD_PRELOAD and starts its own server. librm2fb then creates a piece of shared memory (/dev/shm/swtfb*) and starts a message queue. librm2fb then uses shared memory and messages coming over the message queue to tell SWTCON what to draw to screen. the SWTCON is the software thread that is driving the display.

clients

clients interact with the server through IPC. instead of opening /dev/fb0 as their framebuffer, they use /dev/shm/swtfb* as their framebuffer's memory. instead of sending the mxcfb_update ioctl, specifying where to redraw on screen, they send the same struct through a message queue to the librm2fb server.

how to write an IPC client for rm2fb

relevant system APIS: shm_open, msgget, msgsnd

see ipc.cpp for the reference implementation

use shm_open to open /swtfb.01 as the framebuffer and mmap it into memory. it
has a fixed size of 1404x1872xsizeof(uint16_t) and expects rgb565 content
(like the original rM1 does)
open the Sys V message queue with ID 0x2257c using msgget for writing as queue_fd
define an envelope struct that contains two members: long mtype and mxcfb_update_data. mtype must be non-zero

struct swtfb_update_data {
  long mtype;
  mxcfb_update_data update;
}

instead of calling ioctl(fb_fd, MXCFB_SEND_UPDATE, update_data), use:

swtfb_update_data swtfb_update = swtfb_update_data { 1, update_data };
msgsnd(queue_fd, &swtfb_update, sizeof(swtfb_update), 0)

EDIT: 11/6/20: swtfb_update_data is defined to be an envelope around mxcfb_update_data instead of being an augmented mxcfb_update_data

add way to listen for dirty regions

external clients should be able to listen for dirty region updates. this task is to add some IPC mechanism or otherwise to the server that lets a client subscribe to receive dirty region updates. this will allow VNC client to get their speed upgrades.

one question is: do we allow multiple clients to subscribe? or is it ok to have just a FIFO (for example) and require duplexing to happen somewhere downstream.

Does not work with my version of Xochitl

This issue is for when rm2fb's server process fails to find the signature it needs.

Please respond with which binary was being used and what its md5sum is, thank you!

NOTE: this issue is specifically for if rm2fb server or client prints out an error referencing issue #18. The error message means that the signatures we use are no longer valid in the version of xochitl being used. Please open separate threads for other issues.

Make client side MIT licensed

It would be nice if the sample code for the client side of this was MIT licensed. Otherwise, people won't be able to use this project without changing their project to GPL license, or resorting to annoying hacks to keep the binaries separate.

White screen after boot - RM2 2.8.0.98

Just tried to install rm2fb through toltec before installing other apps and a launcher but I got a white screen just after install. White screen also after the boot screen.

I managed to get the UI back by SSHing back into the Rm2 and removing rm2fb.

Is 2.8.0.98 suported yet ? What did I do wrong ?

Last okpk prompt was :

Powering off doesn't fully update the screen to poweroff.png

Describe the bug:
When I turn off my reMarkable, the screen keeps the last screen. Uninstalling rm2fb fixes the bug.

Steps to reproduce the behavior:

1. Turn on reMarkable2
2. Turn it off.

Expected behavior
The screen should be updated to show the poweroff.png before actually powering off,

Screenshots
reMarkable in powered off state:
Pictures of bug

Version Information:

Device: [reMarkable 2]
OS: [2.5.1.45]
rm2fb Version [1.0.0-7]

make locate_signature more robust

when using argv[0] and location_signature, it should be expanded to the full path, so that invocations like LD_PRELOAD=client.so xochitl don't crash

implement LD_PRELOAD shim for rM1 apps

as per #11 (comment), we can try making a shim that intercepts system calls for apps built for the rM1

if done right, this would mean the other integration (rmkit, libremarkable, koreader, qt's eprenderer) work is not necessary.

Waveform file with correct FPL_LOT and TFT_VID not found with latest toltek testing build

I've just got a r2 and I am trying to set up koreader.

I installed the latest r2fb via toltek
opkg install display

Installed koreader from testing
opkg install koreader

Then I wrote a script to switch between xochitl and koreader (modelled on an old one). My plan was to get this working and then use genie to do the switch.

#!/bin/sh

# if koreader is running and we get swiped then we should
# kill koreader
if [ ! -z "`pidof koreader`"]; then
    echo "koreader is running, killing"
    killall koreader
    exit 0
fi

# otherwise stop the ui
systemctl stop xochitl

if [ ! -z "`pidof remarkable-shutdown`"]; then
    killall remarkable-shutdown
fi

LD_PRELOAD=/opt/lib/librm2fb_server.so.1.0.1 /usr/bin/remarkable-shutdown &
sleep 2

echo "Starting koreader"
LD_PRELOAD=/opt/lib/librm2fb_server.so.1.0.1
koreader

echo "Koreader has quit"

killall remarkable-shutdown
systemctl reset-failed xochitl
systemctl start xochitl

Script works pretty well but koreader has lots of graphical problems and isn't really usable. In stdout I see:

Error decoding barcode: EUFA8RBH9W3V00DSSAT -1.45!
Waveform file with correct FPL_LOT and TFT_VID not found, using fallback: /usr/share/remarkable/320_R299_AFC421_ED103TC2U2_VB3300-KCD_TC.wbf
Reading waveforms from /usr/share/remarkable/320_R299_AFC421_ED103TC2U2_VB3300-KCD_TC.wbf

And occasionally the following repeated.

Pan failed: Invalid argument

I'm assuming r2fb doesn't quite understand what waveforms it needs, picks one which then isn't that great.

merge outstanding updates after a slow update

currently, rm2fb runs its updates in a loop and its possible for a series of large updates to clog the SWTCON up. for example, if 10 full screen updates are placed in the queue, it would likely cause 1+ second of delay.

we've previously explored putting large updates into a separate thread, but that led to strange issues and had to be undone.

this task is to enable update merging. for example: let's say there are X items in the queue after a large update. instead of running each item individually, we can merge the remaining updates for each waveform mode and consolidate their dirty rectangles.

there are many ways of handling merging, so part of this task is to also research what merge styles existed in mxcfb, as well as to understand what merging is built in to SWTCON.

[Question] Reading the framebuffer

Is there a way, on a remarkable 2, to read the framebuffer when xochitl is running (in order to stream it to another device)?

Refresh in Oxide after app is close doesn't work correctly

Environment setup:
remarkable2-framebuffer: 706f4cb..
oxide: af869c6... (with adding LD_PRELOAD=/home/root/fb/librm2fb_client.so before application launch)

LD_PRELOAD=/home/root/fb/librm2fb_server.so /usr/bin/remarkable-shutdown &
systemctl start tarnish

Steps:

Start "Process Manager"
Press "Back"

Actual:
"Process Manager" elements are still visible.

Expected:
Oxide should look like after the launch.

I'm attaching video.

The code responsible for the redraw:
https://github.com/Eeems/oxide/blob/af869c69304a9d8debf21dc705783cc9f1c7fdef/applications/system-service/application.h#L162

IMG_0993.mov.zip

progress on porting frameworks and apps

Some frameworks in use around the rM1 community for building apps:

framework	lang	status	apps
libremarkable	rust	finished	retris, chessmarkable, plato, whiteboard
remarkable.NET	C#	not started
rmkit	cpp	finished	harmony, mines, remux
qtsgepaper	cpp+qt	finished	calculator, draft, oxide, wikipedia, keywriter, chess2
koreader base	cpp+lua	finished	koreader

Apps that have been tested:

retris
plato
koreader
mines
chess2
remux
oxide
calculator
draft
retris
chessmarkable
plato
whiteboard

Apps to test:

anything remaining in toltec

investigate fast drawing

things to look into:

is it a parameter in sendUpdate
is it a member on EPFramebuffer (or canvas)
is it a setting used to dictate speed of vsync/flip thread (or type of flipping)
???

better waveform translation from mxcfb & implement WAVEFORM_MODE_AUTO

I saw that for rM2, people switch from WAVEFORM_MODE_AUTO to WAVEFORM_MODE_DU - this is the correct thing to do, as WAVEFORM_MODE_AUTO is not implemented in rm2fb and is upgraded to GC16 (or GL16) and calls clearGhosting().

when using rm2, there is a different set of waveforms available: https://github.com/ddvk/remarkable2-framebuffer/blob/master/src/server/main.cpp#L51 and a translation happens from mxcfb_update -> the rM2 SWTCON API.

it's probably worth re-visiting and tuning the translation layer in rm2fb to get better behavior and maybe add a manual loop that picks the waveform mode to use if AUTO is specified and the region is small enough.

we don't actually have documentation on the rM2 modes and how many colors they support / their update timings for different region sizes. by using a calibration image, we can learn more about the modes exposed to us. (what does INIT support, for example)

(cc @mrichards42, @LinusCDE)

package rm2fb for toltec

let's package rm2fb for toltec.

things to do:

add library files
add binaries for client/server runners
add service file for rm2fb server
add LD_PRELOAD to launchers' service files
add LD_PRELOAD to xochitl service file

integrate against xochitl

one major piece for rm2fb is figuring out how to only have one SWTCON running on the system. currently, xochitl and rm2fb each have their own SWTCON which can lead to problems if they are both active at the same time.

this task is to figure out how to use xochitl's SWTCON for rendering or to make xochitl use rm2fb's SWTCON.

this is currently blocking multi-tasking launchers from working correctly since xochitl has to be stopped/started whenever rm2fb is running.

rm2fb possibly not working on 2.8x

rm2fb installed via opgk does not work on 2.8 Beta. I opened therefore a issue here. So that we maybe can be ahead.

Journalctl only returns xochitl.service: Start request repeated too quickly. Trying to find out more and edit/update the thread

rm2fb waveforms don't support qtswikipedia

from @dps: remarkable-wikipedia doesn't show links correctly.

https://rmkit.dev/rm2fb/qtwikipedia <--- test binary with built in page
https://rmkit.dev/rm2fb/qtwikipedia.png <--- what is in the buffer

but what shows up on screen is that the link is invisible.

i tried using waveforms 0, 1, 2, 3 and none of them showed the link for me

launcher support for rm2fb

This task is to add the ability for launchers to use the LD_PRELOAD shim when launching applications.

Launchers:

draft: single task launcher
oxide: multi task launcher
remux: multi task launcher

The main issue with rm2fb and launchers is interacting with xochitl, since xochitl will spawn its own SWTCON thread.

Note that launchers will have to use rm2fb LD_PRELOAD shim to display to screen