galister/wlxoverlay

A simple OpenVR overlay for Wayland and X11 desktops

Superceded by WlxOverlay-S

For the newer version that supports both SteamVR and Monado, see WlxOverlay S.

WlxOverlay

A lightweight OpenVR overlay for Wayland and X11 desktops, inspired by XSOverlay.

Primarily made this because I couldn't find a proper desktop overlay for Linux.

Formerly known as X11Overlay.

Features:

• Access your screens from within OpenVR / SteamVR
• Works with a huge variety of setups, including tiling window managers.
• Mouse pointer that supports left/right/middle click
• Customizable keyboard with 2-hand typing
• Watch panel that shows:
  • Local time + 2 customizable time zones
  • Battery states of SteamVR controllers + all connected trackers
  • Volume rocker (customizable)
  • Toggles for screens / keyboard
• Notifications system with support for VRCX and Dbus (Desktop) notifications

Getting Started

Check out the Wiki here!

Join the discussion!

• Discord: https://discord.gg/gHwJ2vwSWV
• Matrix Space: #linux-vr-adventures:matrix.org

Known Issues

• Dragging curved displays very close may make them disappear. Long click the toggle on the watch to force respawn.
• Wayland: It's possible that your compositor does not implement some of the required protocols. Please create a ticket to let us know in that case.
• Wayland: screencopy can crash, especially with multiple screens up. this is being investigated, recommend wrapping in a restart loop for practical use in the meantime.

Works Used

• freesound, CC0 sound effects (find the sounds by searching for their number)
• FreeTypeSharp, MIT License
• Godot Engine, MIT License
• Liberation Fonts, SIL Open Font License v1.1
• Newtonsoft.Json, MIT License
• OBS Studio, GPLv2 License
• OpenVR SDK, BSD-3-Clause license
• OVRSharp, MIT License
• Silk.NET, MIT License
• SixLabors.ImageSharp, Apache v2 License
• Tmds.DBus, MIT License
• Tdms.LibC, MIT License
• bendahl/uinput, MIT License
• WaylandSharp, MIT License
• YamlDotNet, MIT License

wlxoverlay's Issues

DBus Notifications only work once (at least on X11)

I noticed that DBus Notifications only fire once when using X11.
After the first notification arrives it seems that the DBus Listener crashes and doesn't work anymore.
Did some debugging and it seems that you will have to connect to DBus explicitly to make the Observer/Rule matcher work continuously:

I tried to do that quickly in the NotificaionManager.RegisterDbusAsync() but it doesn't seem to work:

...
    private async Task RegisterDbusAsync()
    {
        var dbus = Connection.Session;
+       await dbus.ConnectAsync();

        var rule = new MatchRule
        {
            Member = "Notify",
            Interface = "org.freedesktop.Notifications",
            Path = "/org/freedesktop/Notifications",
            Type = MessageType.MethodCall,
            Eavesdrop = true
        };
...

Hybrid GPU frame copy

What's the status of peeking at swapchains of mismatching memory tiling right now?

Typically if you don't use linear buffer or translate one tiling to another peeking at windows across different GPUs such as a hybrid setup where your VR is running on your laptop dGPU and the desktop session is running on your iGPU results in completely scrambled visuals.

Is anything done in the project to address this?

`EGL_NO_DISPLAY` returned when `WAYLAND_DISPLAY` is unset upon running

OS: Arch Linux 6.4.3-zen1-2-zen
DE: KDE Plasma (Wayland)
GPU: Nvidia GTX 1070 running on latest nvidia-dkms drivers (535.54.03-1)

Running WlxOverlay with WAYLAND_DISPLAY unset causes eglGetDisplay to return EGL_NO_DISPLAY. I get the same error even if I override WAYLAND_DISPLAY to wayland-0 (output of echo $WAYLAND_DISPLAY for me) in the config.yml file.

Here's the output when I try to run it:

WlxOverlay v1.3.1
OpenVR Version: 1.26.6
IVROverlay_025: pass
IVRCompositor_027: pass
IVRInput_010: pass
Loaded 10 input actions for /actions/default
export_inputs will not be used.
HMD running @ 90 Hz
Wayland detected.
eglGetPlatformDisplayEXT returned EGL_NO_DISPLAY!
eglGetDisplay returned EGL_NO_DISPLAY!
Unhandled exception. System.ApplicationException: Could not get EGL display!
   at WlxOverlay.Desktop.EGL.Initialize() in /home/runner/work/WlxOverlay/WlxOverlay/Desktop/EGL.cs:line 44
   at WlxOverlay.Desktop.Wayland.WaylandInterface.TryInitialize() in /home/runner/work/WlxOverlay/WlxOverlay/Desktop/Wayland/WaylandInterface.cs:line 53
   at Program.<Main>$(String[] args) in /home/runner/work/WlxOverlay/WlxOverlay/Program.cs:line 80
   at Program.<Main>(String[] args)

I see that the master branch may have a fix for this already, however, I'm unable to compile it due to StereoKit being missing from the /lib directory.

Oversized pointer in WlxOverlay v1.4.0-rc3

As of WlxOverlay v1.4.0-rc3, the circle used as a cursor is huge (especially compared to earlier versions). Is this intentional? If so, could there be a config option implemented to bring the smaller cursor back, or an option to customize the size of the cursor? If not, which I imagine is the case because it is incredibly difficult to use, I can send images of what is happening and provide any other information necessary.

Pointer is not visible from the back

The pointer line is one-sided, making it difficult to identify when it's changed color for resizing overlays.

This is using OpenVR (I see that OVR and OXR pointer code is separate).

Resetting the playspace mover offest via the wristwatch sets the offset to the wrong location.

When I use the playspace mover's "Reset Offset" button, I get moved under the floor and to the west of the real playspace. fixing the floor does not reset it to the proper position either. Restarting SteamVR doesn't fix the incorrect offset, the only way to fix it is to redo room setup. I've tested this on both stable and beta SteamVR with the same result (SteamVR beta is shown in the video.) Here's a video of what's happening (the polygons I create are in the same spot on the edge of my desk IRL): Video

Sorry for the compression, I used telegram to send the video file from my vr computer to my laptop.

Unavailable cursor mode 2

Running in Wayland on the Nobara distro (Fedora 38) and using SteamVR

WlxOverlay v1.4.2
/home/eccles/.local/share/Steam/steamapps/common/SteamVR/bin/vrwebhelper/linux64/vrwebhelper.sh: line 17: STEAMVR_VRENV: unbound variable
/home/eccles/.local/share/Steam/steamapps/common/SteamVR/bin/linux64/vrmonitor: error while loading shared libraries: libsteam_api.so: cannot open shared object file: No such file or directory
OpenVR Version: 2.3.1
IVROverlay_025: pass
IVRCompositor_027: pass
IVRInput_010: pass
Loaded 10 input actions for /actions/default
export_inputs will not be used.
HMD running @ 90 Hz
Wayland detected.
!!! Patched GLFW from https://github.com/Admicos/minecraft-wayland
!!! If any issues with the window, or some issues with rendering, occur, first try with the built-in GLFW, and if that solves the issue, report there first.
!!! Use outside Minecraft is untested, and things might break.
EGL 1.5 initialized.
Using PipeWire capture.
PipeWire version: 1.0.0
Using DMA-Buf with supported formats: DRM_FORMAT_ARGB8888, DRM_FORMAT_ABGR8888, DRM_FORMAT_XRGB8888, DRM_FORMAT_XBGR8888
Unhandled exception. Tmds.DBus.Protocol.DBusException: org.freedesktop.portal.Error.InvalidArgument: Unavailable cursor mode 2
   at Tmds.DBus.Protocol.DBusConnection.MyValueTaskSource`1.System.Threading.Tasks.Sources.IValueTaskSource<T>.GetResult(Int16 token) in /home/runner/work/WlxOverlay/WlxOverlay/Protocols/DBus/DBusConnection.cs:line 39
   at Tmds.DBus.Protocol.DBusConnection.CallMethodAsync[T](MessageBuffer message, MessageValueReader`1 valueReader, Object state) in /home/runner/work/WlxOverlay/WlxOverlay/Protocols/DBus/DBusConnection.cs:line 561
   at Tmds.DBus.Protocol.Connection.CallMethodAsync[T](MessageBuffer message, MessageValueReader`1 reader, Object readerState) in /home/runner/work/WlxOverlay/WlxOverlay/Protocols/DBus/Connection.cs:line 226
   at WlxOverlay.Desktop.Wayland.XdgScreenData.SelectSourcesAsync() in /home/runner/work/WlxOverlay/WlxOverlay/Desktop/Wayland/XdgScreenCastHandler.cs:line 191
   at WlxOverlay.Desktop.Wayland.XdgScreenData.InitDbusAsync() in /home/runner/work/WlxOverlay/WlxOverlay/Desktop/Wayland/XdgScreenCastHandler.cs:line 77
   at WlxOverlay.Desktop.Wayland.XdgScreenCastHandler.PromptUserAsync(WaylandOutput output) in /home/runner/work/WlxOverlay/WlxOverlay/Desktop/Wayland/XdgScreenCastHandler.cs:line 14
   at WlxOverlay.Core.Subsystem.WaylandSubsystem.CreatePipeWireScreensAsync() in /home/runner/work/WlxOverlay/WlxOverlay/Core/Subsystem/WaylandSubsystem.cs:line 162
   at WlxOverlay.Core.Subsystem.WaylandSubsystem.CreateScreensAsync() in /home/runner/work/WlxOverlay/WlxOverlay/Core/Subsystem/WaylandSubsystem.cs:line 141
   at Program.<Main>$(String[] args) in /home/runner/work/WlxOverlay/WlxOverlay/Program.cs:line 72
   at Program.<Main>(String[] args)
Aborted (core dumped)

"PlatformError: Wayland: Failed to connect to display" fatal error on X11

WlxOverlay correctly detects that the system is using X11, but it seems like it still tries to connect to a Wayland display. I've attached the applicable logs below, I can provide any more information that is needed upon request.

WlxOverlay v1.3.1
OpenVR Version: 1.26.5
IVROverlay_025: pass
IVRCompositor_027: pass
IVRInput_010: pass
Loaded 10 input actions for /actions/default
export_inputs will not be used.
HMD running @ 120.019 Hz
X11 desktop detected.
Using paplay for audio output.
Listening for notifications @ 127.0.0.1:42069
Unhandled exception. Silk.NET.GLFW.GlfwException: GLFW Init failed, PlatformError: Wayland: Failed to connect to display
   at Silk.NET.GLFW.GlfwProvider.GetGlfw()
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at System.Lazy`1.get_Value()
   at Silk.NET.Windowing.Glfw.GlfwWindow..ctor(WindowOptions optionsCache, GlfwWindow parent, GlfwMonitor monitor)
   at Silk.NET.Windowing.Glfw.GlfwPlatform.CreateWindow(WindowOptions options)
   at Silk.NET.Windowing.Window.Create(WindowOptions options)
   at WlxOverlay.GFX.OpenGL.GlGraphicsEngine.StartEventLoop() in /home/runner/work/WlxOverlay/WlxOverlay/GFX/OpenGL/GlGraphicsEngine.cs:line 38
   at Program.<Main>$(String[] args) in /home/runner/work/WlxOverlay/WlxOverlay/Program.cs:line 120
   at Program.<Main>(String[] args)
fish: Job 1, './WlxOverlay-*.AppImage' terminated by signal SIGABRT (Abort)

Spacedrag is "glitchy"

When attempting to use spacedrag, the more I move my hand that is dragging, the worse the effect, it's like there is the initial drag offset that is then being put through a random vec3 function until I release the spacedrag button.

Cursor on wrong desktop.

Description

With more than one display, the cursor will hover on a different display until you re-arrange display cables.

Side Note(s)

I don't think this is a wlxoverlay specific issue, but this could just be something to keep in mind.

System Info

OS: Arch Linux x86_64
Kernel: 6.4.12-arch1-1
DE: Plasma 5.27.7
WM: kwin

The video menu does nothing.

As of WlxOverlay v1.4.0-rc5, the video menu doesn't do anything on my system. When I select the buttons in the menu, the colors and brightness do not change at all. I've checked that they do not change on the SteamVR overlay, WlxOverlay, and multiple games (VRChat, Beat Saber, H3VR, Synth Riders, Electronauts, and a Test Project in Godot).

Using X11
Arch Linux
SteamVR 1.26.7
WlxOverlay 1.4.0-rc5

I will test on beta SteamVR soon and report if it works properly or not here.

Thank you,
A VRContrib Contributor

Feature Request: Integration with SteamVR Overlay/Plugin management for autostarting

Thanks for creating an overlay replacement as a workaround for the sad state of affairs SteamVR on Linux leaves us with!

I was curious if you'd be be able to to get it to register with and be eligible for autostarting via SteamVR's Overlay/Plugin manager as per some of the vague instructions I stumbled upon here.

https://steamcommunity.com/app/250820/discussions/7/1750147265264506783/

You basically need to add an application manifest via vr::VRApplications()->AddApplicationManifest() and set your application to auto launch vr::VRApplications()->SetApplicationAutoLaunch().

Pic of the Overlay/Plugin manager for reference:

Unhandled Exception Key in Use

Followed instructions as in Setup Instructions How to Run, received following error:

./WlxOverlay-v1.4.1-x86_64.AppImage
WlxOverlay v1.4.1
OpenVR Version: 1.27.5
IVROverlay_025: pass
IVRCompositor_027: pass
IVRInput_010: pass
Loaded 10 input actions for /actions/default
export_inputs will not be used.
HMD running @ 90 Hz
Wayland detected.
EGL 1.5 initialized.
Using desktop capture protocol: zwlr_export_dmabuf_manager_v1
Using paplay for audio output.
Listening for notifications @ 127.0.0.1:42069
GL Context initialized
Unhandled exception. OVRSharp.Exceptions.OpenVRSystemException`1[Valve.VR.EVROverlayError]: Could not initialize overlay: KeyInUse (KeyInUse)
   at OVRSharp.Overlay..ctor(String key, String name, Boolean dashboardOverlay)
   at WlxOverlay.Backend.OVR.OVROverlay.Show() in /home/runner/work/WlxOverlay/WlxOverlay/Backend/OVR/OVROverlay.cs:line 176
   at WlxOverlay.Overlays.BaseOverlay.Show() in /home/runner/work/WlxOverlay/WlxOverlay/Overlays/BaseOverlay.cs:line 98
   at WlxOverlay.Core.MainLoop.Update() in /home/runner/work/WlxOverlay/WlxOverlay/Core/MainLoop.cs:line 58
   at WlxOverlay.GFX.OpenGL.GlGraphicsEngine.OnRender(Double _) in /home/runner/work/WlxOverlay/WlxOverlay/GFX/OpenGL/GlGraphicsEngine.cs:line 120
   at Silk.NET.Windowing.Internals.ViewImplementationBase.DoRender()
   at Silk.NET.Windowing.WindowExtensions.<>c__DisplayClass2_0.<Run>b__0()
   at Silk.NET.Windowing.Internals.ViewImplementationBase.Run(Action onFrame)
   at Silk.NET.Windowing.WindowExtensions.Run(IView view)
   at WlxOverlay.GFX.OpenGL.GlGraphicsEngine.StartEventLoop() in /home/runner/work/WlxOverlay/WlxOverlay/GFX/OpenGL/GlGraphicsEngine.cs:line 45
   at Program.<Main>$(String[] args) in /home/runner/work/WlxOverlay/WlxOverlay/Program.cs:line 81
   at Program.<Main>(String[] args)
Aborted (core dumped)

Hyprland as compositor.

Let me know if you need any more info.

Crash when switching X11 displays

Archlinux + i3, 2 monitors, tested v1.2 and git.

Steps:

1. Show screen 0
2. Show screen 1

Stack trace:

Unhandled exception. OVRSharp.Exceptions.OpenVRSystemException`1[Valve.VR.EVROverlayError]: Could not initialize overlay: KeyInUse (KeyInUse)
   at OVRSharp.Overlay..ctor(String key, String name, Boolean dashboardOverlay)
   at WlxOverlay.Overlays.Simple.BaseOverlay.Show() in /home/runner/work/WlxOverlay/WlxOverlay/Overlays/Simple/BaseOverlay.cs:line 88
   at WlxOverlay.Overlays.Simple.GrabbableOverlay.Show() in /home/runner/work/WlxOverlay/WlxOverlay/Overlays/Simple/GrabbableOverlay.cs:line 53
   at WlxOverlay.Overlays.XorgScreen.Show() in /home/runner/work/WlxOverlay/WlxOverlay/Overlays/XorgScreen.cs:line 46
   at WlxOverlay.Overlays.Simple.BaseOverlay.ToggleVisible() in /home/runner/work/WlxOverlay/WlxOverlay/Overlays/Simple/BaseOverlay.cs:line 55
   at WlxOverlay.Overlays.Watch.<>c__DisplayClass7_2.<.ctor>b__6() in /home/runner/work/WlxOverlay/WlxOverlay/Overlays/Watch.cs:line 176
   at WlxOverlay.UI.Button.OnPointerUp() in /home/runner/work/WlxOverlay/WlxOverlay/UI/Button.cs:line 26
   at WlxOverlay.UI.Canvas.<>c__DisplayClass20_0.<OnPointerDown>b__0() in /home/runner/work/WlxOverlay/WlxOverlay/UI/Canvas.cs:line 132
   at WlxOverlay.Overlays.LaserPointer.AfterInput(Boolean batteryStateUpdated) in /home/runner/work/WlxOverlay/WlxOverlay/Overlays/LaserPointer.cs:line 69
   at WlxOverlay.Core.OverlayManager.Update() in /home/runner/work/WlxOverlay/WlxOverlay/Core/OverlayManager.cs:line 153
   at Silk.NET.Windowing.Internals.ViewImplementationBase.DoRender()
   at Silk.NET.Windowing.Internals.ViewImplementationBase.Run(Action onFrame)
   at Silk.NET.Windowing.WindowExtensions.Run(IView view)
   at WlxOverlay.GFX.OpenGL.GlGraphicsEngine.StartEventLoop() in /home/runner/work/WlxOverlay/WlxOverlay/GFX/OpenGL/GlGraphicsEngine.cs:line 41
   at Program.<Main>$(String[] args) in /home/runner/work/WlxOverlay/WlxOverlay/Program.cs:line 111
   at Program.<Main>(String[] args)
Aborted (core dumped)

Cursor offset when scaling one screen in wayland (KDE Plasma)

I have two monitors connected to my PC, one of which is scaled to 85% using the scale setting of the Display configuration in KDE Plasma.
When Enabling this screen in WlxOverlay the cursor still seems to use the coordinates as if the screen was scaled to 100%, making it impossible to move the cursor to the bottom or right side of the screen.

My best guess (without looking at the code) would be that WlxOverlay uses the resolution to map the cursor position and Plasma expects it to include scaling