anko/hudkit

transparent fullscreen on-top click-through WebKit web view, for making cool desktop HUDs

hudkit's Issues

Can I use this to display a website as my desktop.

i.e. use case, I would like to run this at the back of all windows? i.e. just above the desktop?

If it's a simple change could you point me in the right direction?

On WSL2: Cannot clickthrough example hud

expected behavior: i should be able to click on stuff through the hud

actual behavior: clicking on things underneath the hud doesn't function. I can click-drag to highlight text displayed on the HUD, however. after a few minutes, or so, a window for "Web Inspector" appears, although it doesn't appear to be clickable, after which the HUD transparent window can be click-dragged off of the screen. there's no associated terminal output, as far as i can tell.

i suspect it may either be an issue with wsl, or an issue with my package manager, tea (tea.xyz), which sometimes messes with PATHs and such.

steps to reproduce: ./run.sh as per the readme, window key to show taskbar, left click to browser window.

environment: Ubuntu 23.04 via WSL2

Overlay extends off-monitor

After following the instructions on the readme.md and running ./example/run.sh, the overlay shows up but extends incorrectly offscreen. Here is a description of my setup as seen in the attached picture:
Monitor 1 is marked as not primary, it is rotated vertically at 1440x2560. It is to the right of monitor 2.
Monitor 2 is marked as primary, it is 3400x1440. It is to the left of monitor 1.
The first overlay that identifies monitor 2 appears on monitor 1, and since the width on monitor 1 of 1440 is less than 3440, the overlay is cut off. I do not see any overlay for monitor 2, but presumably it is extended even further to the right of monitor 1.
I also cannot click on windows behind the overlays.

Continuous integration testing

Ideally, we'd have some sort of automatic test in Travis that starts Xvfb and compton inside that and does some basic pixel reading so I can stop worrying if any of this works after it leaves my PC. 😱

• Does Travis have Xvfb?

  → Yep.

• Is it possible to even run compton in Xvfb?

  →Yes (with +extension COMPOSITE);

  Xvfb +extension COMPOSITE :99 &
  DISPLAY=:99 compton --backend=xrender
  

• If we do run compton in Xvfb, can we run hudkit in that?

  →Yea;

  cd example
  DISPLAY=:99 ./run.sh
  

• Can we pick a colour out of the Xvfb with something?

  → Yas;

  DISPLAY=:99 xwd -root -silent \
    | convert xwd:- -depth 8 -crop "1x1+0+0" txt:- | grep -om1 '#\w\+'
  

• Does Travis have xwd and convert?

  → Yarr; x11-apps contains xwd and imagemagick contains convert, which are both whitelisted.

• Can we even get a compositor on Travis?

  • Does it have a standalone compositor like compton or xcompmgr?

    → ❌ Nope.

  • What if we overkill it by starting a DE with a built-in compositor?

    → ✔️ Yah. It has Metacity!? wat 😲 ok cool

• Does it actually work?

  → ^{exasperated whisper} Noooo.

  → ^{triumphant whisper} Yeeeeehssss.

shell subprocessing + keyboard support? questions

This is an interesting repository as I recently did web development and is into r/unixporn, but I have some questions assuming if this project is still active.

Is there any API or a workaround for subprocessing for getting external information? (active workspace number, filesystem utilizing, dbus utilizing, or powering or rebooting from a powermenu window made with this)

I'm planning to add keyboard support for my powermenu window, but apparently, I saw your pull request about keyboard support, can I utilize it?

At first glance, I felt like this API is a bit lacking, because I feel like it's specifically made for HUDs or status bars to show time, computer performance, etc., unless if this is something made for fun.

Using i3 as my window manager.

"Frame load interrupted" error when loading pages directly from filesystem, with the `file://` protocol

Loading anything with the file:// protocol using webkit2gtk-2.34.0 looks to be borked, which is breaking the runnable example.

Workaround: Start a webserver in the example directory with python -m http.server --bind localhost and load hudkit "http://localhost:8000/page.html" instead.

Hudkit going behind windows?

The title explains the issue pretty clearly.

When opening hudkit with hudkit https://localhost:5173 with a test red side bar, it displays above all windows.

When opening a new window, it goes behind the windows, like the following:

Maybe it has todo with my window manager rules? I dont know, maybe add a rule that puts hudkit at the top?

Thanks.

Research brain-dump: Possible implementation paths for handling input events while also passing them through

As pointed out in #3, Hudkit currently passes through all input events. This is by design, but that's because the design is not ambitious enough. 🔥

If desktop input events could be duplicated to the Hudkit page in addition to the passthrough, that would open up many interactive "applications"—which is to say I really wanna resurrect this ancient hypergurl.com mouse-trailing clock that I thought was cool when I was 12, and I was so disappointed that the page said "This script works on webpages only and cannot be installed to desktops". Now, 15 years of programming later, something is flickering at the end of this tunnel. Maybe.

---

Notes for possible implementation paths:

• Can this be done in GDK/GTK only?

  • Is it possible to send synthetic events?
    → ✔️ Yes. gdk_event_new(GDK_KEY_PRESS), gtk_main_do_event. Gist w/ makefile here.

  • Is it possible to capture all input events globally?
    → ❔ Probably not. GDK appears not to consider this sort of generality.

  • Is it possible to cause GdkEvents to be sent to the desktop?
    → ❔ Maybe. The above example Gist only ever sends events to itself, even if I focus a different window before the on_timeout triggers. But maybe there's a low-level alternative to gtk_main_do_event.

• Can this be done with a combination of X11 and GDK/GTK?

  • Is it possible to capture all input events globally at X11 level (keylogger style) such that they're still delivered to the focused application?
    → ✔️ Yes. I have written such an X11 program.

  • Is it possible to convert XEvents to GdkEvents?
    → ✅ Ostensibly yes; GDK does it internally like this. The function isn't exported to the public interface, but we could pull in GTK sources; they're big (13MiB zipped), but LGPL-licensed. Need to figure out the right compiler incantations, and hopefully not having to compile whole damn GTK with us because nooope.

  • Is it possible to interleave a GTK and X11 event loop, or otherwise coordinate?
    → ✅ Again yeah in principle; GDK must interleave them internally. And could always fork and do it via socket.

  • Is it possible to send a GdkEvent converted from an XEvent directly to a window?
    → ❔ Unsure. I thought gdk_event_put might do that, but the doc's wording of "Appends a copy of the given event onto the front of the event queue for event->any.window’s display" sounds to me like the event will be plopped into the top level queue for this GdkDisplay, before input mask filtering, and then it's just pointless, because it'll be pruned anyhow. But maybe it's worth a try.

If anyone reading knows anything even a little bit related to this, please link!

multi-display bug and feature request

If I launch the command from the 2nd of two monitors it'll take the overall width of the combined displays and initiate the HUD on the second monitor. This results in the first monitor not having the HUD and the second one displaying a HUD that appears stretched.

feature request) add the ability to only display HUD on default monitor when multiple are present via a command line flag

bug report) HUD incorrectly estimates size and position when not initialized from the first of a multiple display setup.

Add support for Wayland (if needed)

I'm looking to use a Wayland-based DE/compositor in the future over X11, and I'm what barriers there would be to using HUDKit on Wayland. Has this been tested?

For context, IINACT recommends HUDKit for overlays on Linux, and I want to make sure this works as well without having to go and use an X11-based environment.

The JS API is callback-based; async/await would be prettier

async/await is conciser, and works with try/catch/finally.

• Do versions of webkit2gtk available on various distros support it?

  (say, Ubuntu, Arch, Fedora)

  → ✔️ Yep. At time of speaking, everybody seems to be on webkit2gtk 2.30.x or 2.31.x, which look like they all support async/await.

• Does webkit have an implementation for the top-level await TC39 proposal yet?

  Not strictly necessary, but it would be real pretty for documentation purposes not to have to wrap everything (async () => { ... })(), considering a lot of our JS API makes sense to call at the top-level of the file.

  → ❌ Nope. Not yet anyway. Track https://bugs.webkit.org/show_bug.cgi?id=202484.

So: This can be done. I'll get around to it eventually.

Hot cool demonstration GIF in the readme

...like https://github.com/progrium/topframe has. 🔥

To demonstrate that parts of the window can be made clickable and that it can really render on top of everything, a neat demo would be an application launcher: Clickable icon in the bottom right, which opens up a large clickable menu for starting programs, with gratuitous RGB flashbang animations to hammer the point home.

---

I'd just screencap my own hudkit setup, but I'm too minimalist for my own good. This is the top right of my screen:

It looks nice, but it's nowhere near explodey enough to demonstrate hudkit's feature-set fully. 😅

Feature request: Recognise keyboard button presses globally, and pass them to addEventListener in page JS

hudkit doesn't seem to recognize when buttons are pressed on the keyboard. Here's a simple test page that should be displaying the keycodes.

<html>
<head></head>
<style>
body {
	background-color: #000;
	font-size: 30px;
	font-weight: bold;
	color: #fff;
}
</style>
<body>
<p id='keyCode'></p>
<script>
document.addEventListener('keydown', function(e) {
	document.getElementById("keyCode").innerHTML = e.keyCode;
});
</script>
</body>
</html>

multi-monitor support seems to not work

running the example page, sometimes it'll open itself like this:

sometimes like this:

and other times the window just won't be visible at all.

i'm using i3wm on ubuntu 18.04; i have two 1080p monitors side-by-side:

$ xrandr
Screen 0: minimum 8 x 8, current 3840 x 1080, maximum 32767 x 32767
DP-0 disconnected (normal left inverted right x axis y axis)
DP-1 disconnected (normal left inverted right x axis y axis)
eDP-1-1 connected primary 1920x1080+0+0 (normal left inverted right x axis y axis) 344mm x 194mm
   1920x1080     60.00*+  59.97    59.96    59.93    48.00  
   1680x1050     59.95    59.88  
   1600x1024     60.17  
   1400x1050     59.98  
   1600x900      59.99    59.94    59.95    59.82  
   1280x1024     60.02  
   1440x900      59.89  
   1400x900      59.96    59.88  
   1280x960      60.00  
   1440x810      60.00    59.97  
   1368x768      59.88    59.85  
   1360x768      59.80    59.96  
   1280x800      59.99    59.97    59.81    59.91  
   1152x864      60.00  
   1280x720      60.00    59.99    59.86    59.74  
   1024x768      60.04    60.00  
   960x720       60.00  
   928x696       60.05  
   896x672       60.01  
   1024x576      59.95    59.96    59.90    59.82  
   960x600       59.93    60.00  
   960x540       59.96    59.99    59.63    59.82  
   800x600       60.00    60.32    56.25  
   840x525       60.01    59.88  
   864x486       59.92    59.57  
   800x512       60.17  
   700x525       59.98  
   800x450       59.95    59.82  
   640x512       60.02  
   720x450       59.89  
   700x450       59.96    59.88  
   640x480       60.00    59.94  
   720x405       59.51    58.99  
   684x384       59.88    59.85  
   680x384       59.80    59.96  
   640x400       59.88    59.98  
   576x432       60.06  
   640x360       59.86    59.83    59.84    59.32  
   512x384       60.00  
   512x288       60.00    59.92  
   480x270       59.63    59.82  
   400x300       60.32    56.34  
   432x243       59.92    59.57  
   320x240       60.05  
   360x202       59.51    59.13  
   320x180       59.84    59.32  
DP-1-1 disconnected (normal left inverted right x axis y axis)
HDMI-1-1 disconnected (normal left inverted right x axis y axis)
HDMI-1-2 connected 1920x1080+1920+0 (normal left inverted right x axis y axis) 477mm x 268mm
   1920x1080     60.00*+  59.94  
   1680x1050     59.88  
   1280x1024     75.02    60.02  
   1440x900      59.90  
   1280x960      60.00  
   1280x720      60.00    59.94  
   1024x768      75.03    70.07    60.00  
   832x624       74.55  
   800x600       72.19    75.00    60.32    56.25  
   720x480       60.00    59.94  
   640x480       75.00    72.81    66.67    60.00    59.94  
   720x400       70.08  

presumably this is something weird about my setup, given it sounds like it works for other people...

Compiling with Nix toolchain

Hello, I've been trying to get hudkit compiled on NixOS using the nix toolchain.

More specifically, I've tried the following with no success:

❯ nix-shell --packages pkg-config gtk3 webkitgtk --run make
gcc -std=c11 main.c -o hudkit `pkg-config --cflags --libs gtk+-3.0 webkit2gtk-4.0`
main.c: In function ‘main’:
main.c:450:32: warning: implicit declaration of function ‘strtok_r’; did you mean ‘strtok’? [-Wimplicit-function-declaration]
  450 |             for (char *entry = strtok_r(
      |                                ^~~~~~~~
      |                                strtok
main.c:450:32: warning: initialization of ‘char *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
main.c:450:13: error: declaration of non-variable ‘strtok_r’ in ‘for’ loop initial declaration
  450 |             for (char *entry = strtok_r(
      |             ^~~
main.c:453:27: warning: assignment to ‘char *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
  453 |                     entry = strtok_r(NULL, ",", &strtok_savepoint)) {
      |                           ^
main.c:711:12: error: variable ‘usr1_action’ has initializer but incomplete type
  711 |     struct sigaction usr1_action = {
      |            ^~~~~~~~~
main.c:712:10: error: ‘struct sigaction’ has no member named ‘sa_handler’
  712 |         .sa_handler = on_signal_sigusr1
      |          ^~~~~~~~~~
main.c:712:23: warning: excess elements in struct initializer
  712 |         .sa_handler = on_signal_sigusr1
      |                       ^~~~~~~~~~~~~~~~~
main.c:712:23: note: (near initialization for ‘usr1_action’)
main.c:711:22: error: storage size of ‘usr1_action’ isn’t known
  711 |     struct sigaction usr1_action = {
      |                      ^~~~~~~~~~~
main.c:714:5: warning: implicit declaration of function ‘sigaction’ [-Wimplicit-function-declaration]
  714 |     sigaction(SIGUSR1, &usr1_action, NULL);
      |     ^~~~~~~~~
make: *** [makefile:2: hudkit] Error 1

Do you have any tips on how I might get through this?

Feedback on using with Linux Mint (mentioning clang in dependencies is necessary?)

If you build on another distro, I'm interested in how it went.

Went grrrreat!

Linux Mint x64 20.1 w/5.8.0-50 kernel.

Dependencies were installed with following commands successfully:

sudo apt-get install libgtk-3-dev
sudo apt-get install libwebkit2gtk-4.0

Make was failing with

clang -std=c11 main.c -o hudkit `pkg-config --cflags --libs gtk+-3.0 webkit2gtk-4.0`
/bin/sh: 1: clang: not found
make: *** [makefile:2: hudkit] Error 127

Solved with
sudo apt-get install clang
(clang probably should be listed in Dependencies)

run.sh sample script output:

Works brilliantly!