rustaudio / baseview Goto Github PK
View Code? Open in Web Editor NEWlow-level window system interface for audio plugin UIs
License: Apache License 2.0
low-level window system interface for audio plugin UIs
License: Apache License 2.0
An X11 window created using Windows::open_parented()
will not receive keyboard input, and as a consequence of that, keyboard input for modifiers, when not hovering over the window. This makes sense as the window is not requesting input focus, but it does mean that weird things happen when you combine this with dragging gestures. When you start a dragging gesture, hold down shift, move your mouse cursor outside of the window, release shift, and then WIndowHandler
won't notice shift being released, and every subsequent interaction would behave as if shift was still pressed. You can cry this out with this example using the patched baseview from #115 and Carla:
NOTE: This example has been updated with the patches from #117, so the issue no longer exists on the latest version.
git clone [email protected]:robbert-vdh/nih-plug.git
cd nih-plug
cargo xtask bundle gain_gui --bundle-vst3 && carla-single vst3 target/gain_gui.vst3
Note that this won't happen in Bitwig, as in Bitwig the host's parent window grabs focus itself and then forwards most keyboard events (except for Space and a couple other keys) to the child window.
Also, because modifiers are not sent as part of the mouse move/click events, the modifiers will be ignored when you Shift+click on a plugin GUI directly after opening the window in Bitwig because the click event arrives before the keyboard even (of it doesn't get sent at all, I'd need to check).
The potential ways to fix this would be to always grab keyboard focus, or to add modifiers to mouse move and click events. The last one is the least invasive, and I think this information is part of those events on every platform.
I'm seeing issues with the rendering of a VST UI in Reaper on a Macbook with a Retina display.
This happens both with imgui-baseview
and with egui-baseview
.
Here's what happens with the Gain example from https://github.com/DGriffin91/imgui_baseview_test_vst2
, for instance (this is Reaper's "FX Chain" UI btw):
The same thing happens with any other imgui or egui example. There is some rendering going on though: changing the clear_color
of the window has effect, for instance. The only way I can fix this is by hard-coding the scale_factor
of the window to 1.0 here. Supplying a WindowScalePolicy::ScaleFactor(1.0)
in the WindowOpenOptions
should do the same thing, but does not seem to have any effect.
If I manually double the window size in the example code here, then I do see the outlines of some UI being rendered, completely off-center and clipped:
Stepping through the baseview
code, I do see that the logical size is the size that is being configured by me and the physical size is double that (since scale_factor
is calculated to be 2.0). I am assuming the issue is not related specifically to either the imgui or egui renderers since they both exhibit the same behaviour. Additionally, running the imgui-baseview
examples by themselves (i.e. outside of the VST host) works fine as well. So I'm a bit stumped on what the root cause could be.
It should be possible for a window handler to change the size of the window it's interacting with. Essentially all commercial plugins have resizable in some way (most modern plugins will either have a drag handle in the bottom right corner or they'll require you to resize the host's window, and older plugins tend to have a dropdown with preset size options). For modern plugin GUIs it would be nice to also be able to do the same thing with baseview.
I ran the test (https://github.com/RustAudio/baseview/blob/master/examples/open_parented.rs)
when hovering over normal window:
in windows 10 WSL => logs Parent: KeyEvent
in normal windows 10 => logs Parent: KeyEvent
when hovering over parented window:
in windows 10 WSL => logs CHILD: KeyEvent
in normal windows 10 => logs PARENT: KeyEvent
because of this key events are also broken in nih-plug on windows 10. robbert-vdh/nih-plug#154
I suggest MIT or MIT/Apache 2.0 dual license.
Hello, I've talked about this issue on the discord before but I thought I'd raise a proper issue with my findings.
For some reason, when the Windows scaling isn't 100% (under setttings -> display -> make everything bigger), my baseview/imgui-rs VST changes the scaling in Ableton and is served a too small window.
In FL Studio, the window is also too small but at least the scaling doesn't change. Fl Studio doesn't seem to ever scale up VSTs.
A VST that causes the issue can be found here: https://github.com/Fredemus/svf_filter
Before adding the vst, windows scaling set to 125%
After adding the vst, windows scaling set to 125%. Note that Serum and the channels in Ableton got smaller.
Interestingly, after adding the VST, then changing scaling to 100%, then back to 125% seems to solve the issue.
Edit: Forgot to mention, this problem didn't exist before this commit: ae2f28c
I thought it would make sense to add an issue for macOS event support, so that there is a location for a structured discussion of how to implement it. There is an analogous implementation in antonok's vst_window crate, which could be used as an inspiration.
In my understanding, the NsView created in Window::open needs to be subclassed with an event handling delegate, which stores a pointer to the WindowHandler trait implementation (?) so that its methods call be called in the event handling delegate's extern "C"-declared methods. Such an implementation would differ from the one in vst_window in that it would handle events synchronously instead of sending them over a channel that is polled over elsewhere.
Opening a gui plugin that uses baseview on macOS causes the entire DAW (FL Studio, Reaper) to be very slow to respond especially when resizing windows and panels.
After much debugging I finally came along this line in src/baseview/src/macos/view.rs
let _: () = msg_send![class!(NSEvent).setMouseCoalescingEnabled: NO];
Because mouseCoalescingEnabled is a class property on NSEvent
changing it affect the entire process i.e. the host DAW. So from that point forward every mouse event is non coalesced and the number of events is just too great for the various DAWs' GUIs to keep up when moving sliders / resizing windows etc.
I tried removing the line which fixes the slowdown, and the plugin GUI interaction even feels smoother to me without.
I'm not sure why that line was there to begin with though, so If it is indeed required, it would be good to at least set the initial value back once the view is cleaned up.
A few users are still on Windows 7 and 8. Currently, plug-ins which use baseview don't even scan on these platforms because GetDpiForWindow
and SetProcessDpiAwarenessContext
are not available there. I will make a PR which uses lazy loading of those functions.
Here https://github.com/RustAudio/baseview/blob/master/src/window.rs#L12 FreeBSD support should be added.
@@ -11,6 +11,8 @@
use crate::win as platform;
#[cfg(target_os = "linux")]
use crate::x11 as platform;
+#[cfg(target_os = "freebsd")]
+use crate::x11 as platform;
pub trait WindowHandler {
fn on_frame(&mut self);
When embedding a baseview window inside another window using the open_parented()
function on macOS, the first click on the window only focuses the window and doesn't trigger a mouse button down event for the window handler. Additionally, after this first click the user needs to wiggle their mouse around a bit because up until that point no mouse movement event will have been sent to the window. The view overrides the acceptsFirstMouse
method, but it never seems to be fired. And the tracking areas should allow the plugin to receive mouse movement updates when the user hovers over an unfocused baseview window but that also doesn't seem to work.
I tested this on macOS Catalina with the NIH-plug plugins and OctaSine, and also with the NIH-plug standalones which embed a baseview window inside of another blocking baseview window.
This is a compilation of discussions from the discord chat from @ilmai et al.
Changes required in baseview to capture focus, pass it to client gui libraries (ie Vizia), then return focus to host if client requests it to bubble up.
Per @geom3trik in plugin-gui:
โ Vizia has a
cx.focus()
method which is called when a view wants keyboard focus. Presumably this function could then call the appropriate baseview function to capture the input focus, and then when the focus is released back to the window baseview could release it to the host?โ
Ilmai explained:
So if you pass a mouse event to vizia and vizia asks for text input focus based on that, you shouldn't also be passing those keyboard events to the host because otherwise pressing the spacebar for example would play/pause the timeline in addition to entering a whitespace in a textbox.
On Linux, I am building OctaSine as a VST2 plugin.
When loading the plugin within REAPER, it panics and causes REAPER to crash.
This is the backtrace gathered from the core dump:
#0 0x00007f86c9a51d22 in raise () at /usr/lib/libc.so.6
#1 0x00007f86c9a3b862 in abort () at /usr/lib/libc.so.6
#2 0x00007f868f14b127 in std::sys::unix::abort_internal () at library/std/src/sys/unix/mod.rs:259
#3 0x00007f868f146860 in std::panicking::rust_panic () at library/std/src/panicking.rs:656
#4 0x00007f868f14673b in std::panicking::rust_panic_with_hook () at library/std/src/panicking.rs:624
#5 0x00007f868f146150 in std::panicking::begin_panic_handler::{closure#0} () at library/std/src/panicking.rs:502
#6 0x00007f868f144764 in std::sys_common::backtrace::__rust_end_short_backtrace<std::panicking::begin_panic_handler::{closure#0}, !> ()
at library/std/src/sys_common/backtrace.rs:139
#7 0x00007f868f1460b9 in std::panicking::begin_panic_handler () at library/std/src/panicking.rs:498
#8 0x00007f868e54b661 in core::panicking::panic_fmt () at library/core/src/panicking.rs:107
#9 0x00007f868e54b753 in core::result::unwrap_failed () at library/core/src/result.rs:1613
#10 0x00007f868e6089bb in core::result::Result<core::result::Result<baseview::x11::window::SendableRwh, ()>, std::sync::mpsc::RecvError>::unwrap<core::result::Result<baseview::x11::window::SendableRwh, ()>, std::sync::mpsc::RecvError> (self=...)
at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/core/src/result.rs:1295
#11 0x00007f868e6409c8 in baseview::x11::window::Window::open_parented<octasine::gui::ParentWindow, iced_baseview::window::IcedWindow<octasine::gui::interface::OctaSineIcedApplication<alloc::sync::Arc<octasine::SyncState>>>, iced_baseview::window::{impl#4}::open_parented::{closure#0}> (parent=0x7ffe27e9e3e0, options=..., build=...)
at baseview-9d6c750431f4479e/004065e/src/x11/window.rs:131
#12 0x00007f868e6359e1 in baseview::window::Window::open_parented<octasine::gui::ParentWindow, iced_baseview::window::IcedWindow<octasine::gui::interface::OctaSineIcedApplication<alloc::sync::Arc<octasine::SyncState>>>, iced_baseview::window::{impl#4}::open_parented::{closure#0}>
(parent=0x7ffe27e9e3e0, options=..., build=...) at baseview-9d6c750431f4479e/004065e/src/window.rs:67
#13 0x00007f868e67cb42 in iced_baseview::window::IcedWindow<octasine::gui::interface::OctaSineIcedApplication<alloc::sync::Arc<octasine::SyncState>>>::open_parented<octasine::gui::interface::OctaSineIcedApplication<alloc::sync::Arc<octasine::SyncState>>, octasine::gui::ParentWindow>
(parent=0x7ffe27e9e3e0, settings=...) at iced_baseview-7c672318e3bd7e89/b4d8edb/src/window.rs:355
#14 0x00007f868e652571 in octasine::gui::Gui<alloc::sync::Arc<octasine::SyncState>>::open_parented<alloc::sync::Arc<octasine::SyncState>>
(parent=..., sync_handle=...) at octasine/src/gui/mod.rs:59
#15 0x00007f868e65276b in octasine::gui::{impl#1}::open<alloc::sync::Arc<octasine::SyncState>> (self=0x2420750, parent=0x6400757)
at octasine/src/gui/mod.rs:87
Unusually, this only occurs when using the plugin within REAPER, whereas it works fine if I load the plugin with Carla.
Additionally, I compiled OctaSine as a Windows DLL and tested that using yabridge in the same REAPER installation, which worked flawlessly.
I feel like open_parented()
/open_as_if_parented()
/open_blocking()
should return a Result<WindowHandle/()>
. Right now there's no way to handle window creation failures, and baseview will just panic if anything goes wrong. That's especially important when creating OpenGL contexts. If baseview would return an error, you would be able to try a couple different settings until you find something that works on the current system.
This is a test.
Here I compiled a short TODO list for baseview:
Right now the keyboard events assume a standard QWERTY layout.
A good starting point could be https://github.com/Smithay/wayland-rs/blob/master/wayland-client/examples/simple_window.rs
We should do this similarily to how it is done in druid.
Cocoa documentation is here. xcb info about keycodes here, iced keyboard event, druid implementation, useful crate
Snippet to extract string from key event:
let characters = unsafe {
let ns_string = NSEvent::characters(event);
let start = NSString::UTF8String(ns_string);
let len = NSString::len(ns_string);
let slice = ::std::slice::from_raw_parts(
start as *const u8,
len
);
::std::str::from_utf8_unchecked(slice)
};
x11rb
dependency instead of xcb
.raw_window_handle
.Currently there's no way to change the mouse cursor icon from within a WindowHandler
. Cursor icons are useful to indicate that UI elements are clickable or that you can enter text to them, but without the ability to set them plugins using baseview also tend to look very buggy in hosts like REAPER where the cursor icon may be stuck on some random non-pointer cursor when interacting with the GUI.
Baseview would not even need to allow you let you use any random cursor icon for this to be useful, just some common cursor types like a pointer, a grab handle, a caret, a crosshair, and a disabled icon would likely be sufficient.
Hello, I'm planning to contribute to the project by providing a basic OpenGL example.
https://github.com/pingpongcat/base_window
I've decided not to interfere with the existing example and keep it super simple, but I'm wondering if that's the right direction.
I'd love to hear your opinion before sending any merge request. I'm using glow crate for OpenGL stuff.
Currently, if a user creates a wgpu surface to attach to the editor window, the window will fail to free itself since it uses retainCount
to approximate when the window is ready to be freed:
Lines 160 to 170 in b371263
I'd like to brainstorm ways to work around this. I don't fully grok the circular dependency issue yet, as I haven't repro'd it, so I want to keep that in mind.
@glowcoil do you happen to have an example of a circular dep, or hints towards how to reproduce the issue you saw?
the NSView's keyDown/keyUp methods are never getting called in Reaper when there's keyboard input with the plugin window in focus. neither turning on "Send all keyboard input to plugin" or opening the UI in its own window changed the situation.
curiously, the events are still consumed on some level as Reaper is also not receiving keyboard events when the plugin window is in focus (space doesn't toggle playback, for example).
i'm running macOS 13.5.2 and Reaper 6.43 and using baseview with nih_plug
When adding a new instance of a VST plugin to a DAW, occasionally this causes the DAW to crash on a SEGFAULT, this was narrowed down to happen somewhere within baseview, or one of the dependencies of baseview (possibly glowcoil/raw-gl-context). Notably this seems only to happen after some (or all) instances of the plugin have been closed, and the plugin is added again.
Debugging with gdb revealed the following stacktrace:
#0 0x00007ffc74374298 in rust_eh_personality ()
from C:\Users\ <omitted full path> \ <name_of_plugin.dll>
#1 0x00007ffcca8d0b54 in ntdll!KiUserCallbackDispatcher () from C:\WINDOWS\SYSTEM32\ntdll.dll
#2 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
release
or debug
profile.I noticed that scroll events seem to be ignored when using baseview on Mac OS. I can reproduce this problem with my VST plugin in Cubase on Mac OS.
Therefore I had a look at the code and it seems to me that scroll events are not handled at all in src/macos/view.rs
.
I created the pull request #110 that adds this functionality there. If you agree that this functionality was missing before, feel free to use my code, or use it as a starting point if you like.
Tested on two systems. I get this crash inside baseview when closing the plugin window of my plugin using Vizia:
17:39:21 [ERROR] thread 'unnamed' panicked at 'called `Result::unwrap()` on an `Err` value: XLibError { error_code: 9, error_message: "BadDrawable (invalid Pixmap or Window parameter)", minor_code: 3, request_code: 130, type: 0, resource_id: 8388612, serial: 2718 }': /home/jussi/.cargo/git/checkouts/baseview-9d6c750431f4479e/7001c25/src/gl/x11.rs:246
0: nih_plug::wrapper::util::log_panics::{{closure}}::{{closure}}
at /home/jussi/.cargo/git/checkouts/nih-plug-a2d2dc277b128e13/ce2eab8/src/wrapper/util.rs:128:29
nih_plug::util::permit_alloc
at /home/jussi/.cargo/git/checkouts/nih-plug-a2d2dc277b128e13/ce2eab8/src/util.rs:25:5
1: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/alloc/src/boxed.rs:2002:9
std::panicking::rust_panic_with_hook
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/panicking.rs:692:13
2: std::panicking::begin_panic_handler::{{closure}}
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/panicking.rs:579:13
3: std::sys_common::backtrace::__rust_end_short_backtrace
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/sys_common/backtrace.rs:137:18
4: rust_begin_unwind
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/panicking.rs:575:5
5: core::panicking::panic_fmt
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/core/src/panicking.rs:64:14
6: core::result::unwrap_failed
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/core/src/result.rs:1790:5
7: baseview::gl::x11::GlContext::swap_buffers::{{closure}}
baseview::gl::x11::errors::XErrorHandler::handle::{{closure}}::{{closure}}
at /home/jussi/.cargo/git/checkouts/baseview-9d6c750431f4479e/7001c25/src/gl/x11/errors.rs:73:17
<core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/core/src/panic/unwind_safe.rs:271:9
std::panicking::try::do_call
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/panicking.rs:483:40
std::panicking::try
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/panicking.rs:447:19
std::panic::catch_unwind
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/panic.rs:140:14
baseview::gl::x11::errors::XErrorHandler::handle::{{closure}}
at /home/jussi/.cargo/git/checkouts/baseview-9d6c750431f4479e/7001c25/src/gl/x11/errors.rs:71:32
std::thread::local::LocalKey<T>::try_with
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/thread/local.rs:446:16
std::thread::local::LocalKey<T>::with
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/thread/local.rs:422:9
8: baseview::gl::x11::errors::XErrorHandler::handle
at /home/jussi/.cargo/git/checkouts/baseview-9d6c750431f4479e/7001c25/src/gl/x11/errors.rs:66:9
baseview::gl::x11::GlContext::swap_buffers
at /home/jussi/.cargo/git/checkouts/baseview-9d6c750431f4479e/7001c25/src/gl/x11.rs:242:9
baseview::gl::GlContext::swap_buffers
at /home/jussi/.cargo/git/checkouts/baseview-9d6c750431f4479e/7001c25/src/gl/mod.rs:107:9
9: <vizia_baseview::window::ViziaWindow as baseview::window::WindowHandler>::on_frame
at /home/jussi/.cargo/git/checkouts/vizia-629668219022e428/39e295a/crates/vizia_baseview/src/window.rs:221:9
10: baseview::x11::window::Window::run_event_loop
at /home/jussi/.cargo/git/checkouts/baseview-9d6c750431f4479e/7001c25/src/x11/window.rs:478:17
11: baseview::x11::window::Window::window_thread
at /home/jussi/.cargo/git/checkouts/baseview-9d6c750431f4479e/7001c25/src/x11/window.rs:365:9
12: baseview::x11::window::Window::open_parented::{{closure}}
at /home/jussi/.cargo/git/checkouts/baseview-9d6c750431f4479e/7001c25/src/x11/window.rs:140:13
std::sys_common::backtrace::__rust_begin_short_backtrace
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/sys_common/backtrace.rs:121:18
13: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/thread/mod.rs:558:17
<core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/core/src/panic/unwind_safe.rs:271:9
std::panicking::try::do_call
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/panicking.rs:483:40
std::panicking::try
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/panicking.rs:447:19
std::panic::catch_unwind
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/panic.rs:140:14
std::thread::Builder::spawn_unchecked_::{{closure}}
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/thread/mod.rs:557:30
core::ops::function::FnOnce::call_once{{vtable.shim}}
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/core/src/ops/function.rs:250:5
14: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/alloc/src/boxed.rs:1988:9
<alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/alloc/src/boxed.rs:1988:9
std::sys::unix::thread::Thread::new::thread_start
at /rustc/8460ca823e8367a30dda430efda790588b8c84d3/library/std/src/sys/unix/thread.rs:108:17
15: start_thread
at ./nptl/pthread_create.c:442:8
16: clone3
at ./misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
Is an update to raw-window-handle
possible?
It would allow integration with parented webview windows via wry >=v0.36.0.
The latest I can push in this direction currently is 0.35.2
, at that stage wry still accepts HasRawWindowHandle
trait.
I notice that keeping support for HasRawWindowHandle
in newer versions requires some work, mainly:
raw_window_handle returns Result:
window.rs
unsafe impl<'a> HasRawWindowHandle for Window<'a> {
fn raw_window_handle(&self) -> Result<RawWindowHandle, HandleError> {
self.window.raw_window_handle()
}
}
---WindowHandle accepts NonZero isize:
win/window.rs
unsafe impl HasRawWindowHandle for WindowHandle {
fn raw_window_handle(&self) -> Result<RawWindowHandle, HandleError> {
self.hwnd
.ok_or(HandleError::Unavailable)
.and_then(|hwnd| NonZeroIsize::new(hwnd as isize).ok_or(HandleError::Unavailable))
.map(|hwnd| RawWindowHandle::Win32(Win32WindowHandle::new(hwnd)))
}
}
This is where it occurred to me that these changes might not simply be a change of internal types, it can affect library to library operations, and it can unveil some hidden complexity I might not be aware of.
I am willing to continue working on a fork and making a PR for this, but I am not quite sure if this is the right approach.
On top of keeping support for the HasRawWindowHandle
, there is also a HasWindowHandle
trait now.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.