Multi-window support

[iced-rs]

Multi-window support

[qthree]

Obligatory link to the mutli-viewports branch of imgui: ocornut/imgui#1542

[ChrisCochrun]

I'd like to maybe use iced possibly, but this would stop me before even trying. My entire application is based off multiple windows. It's a presentation app so it needs another window to house slides and videos while the main window controls it on the projector

[Songtronix]

Well Airshipper will need to soon display a new window for (profile) settings so multi-window support might come in handy! Maybe even showing the game output in a different window might be interesting to experiment with.

[vorner]

Well, inter-process communication is usually quite a PITA to make work, so I'd say it's usually possible but not the first choice.

I could imagine a lot of application with multiple windows that make sense. A CAD application with multiple different views to the same drawing so you can spread them onto multiple screens (where you don't want to synchronize the content of the one drawing between processes), GIMP handles multiple documents + shared tool bars (I'm not saying GIMP is the best GUI design, but if iced aims to be general purpose, it should be possible to do too). These probably would lose some abilities on the web ‒ eg. the CAD application would support only one view on the web, while could do multiple in desktop version.

Another thing that's common to desktop is pop-up dialogs. I can imagine the „Menu -> Edit -> Settings“ being a separate process, but it does seem inconvenient to do.

[maxjoehnk]

1. There could be an api e.g. supports_multi_window() which allows the user to decide how to handle cross platform issues. This would be the simplest option and we could add a crate on top of iced which emulates multiple windows for the web (even though I'm not a fan of emulated multiple windows)
2. Multiple windows are definitely a good choice. When you have multiple screens or multiple desktops you may want to split the application into multiple windows to use the space you have.
   E.g. for music production I may have the mixer open on a second screen or multiple projects in my IDE or multiple browser windows. There are many use cases where a single window architecture just doesn't cut it.
   Regarding dialogs and menus I'm not sure what the better option is. In IDEA the git merge window is technically a dialog but is something I want as a new window so I can move it to another screen but smaller dialogs like confirmation messages don't need to be a new window.

[hecrj]

Implemented in #1964.

[qthree]

I think that makers of custom engines should worry themselves about window managing and iced only needs to support handling of events and rendering of multiple viewports. I mean, iced should not even care if window is virtual or native, it only needs to support some abstract window and let engine makers handle the rest.

[maroider]

1. What @maxjoehnk proposes here seems like a reasonable approach while layers (#30) remain unimplemented.
2. I think multiple windows have their place, but I can understand if you disagree. I remember listening to an interview of Blender's original author, Ton Roosendaal, where I think he said something to the effect of: "Blender's flat interface design was a good idea that stood the test of time. Other 3D modelling applications would hide settings in dialogs and sub-dialogs and so forth, which slows the user down." Mind you, even Blender creates extra native windows in certain parts of the application (the "preferences" window comes to mind).

[ckaran]

My use case is kind of weird, so bear with me. I want to create something like a magnifying glass or X-ray window that can be dragged over the main window to display additional information. There are several components to this though:

• The user can compose multiple different auxiliary windows over one another, so I need to know what the Z order of all of the windows are to ensure the right info is being displayed. E.g., if the X-ray window is partly laid over the magnifying window, then you want part of the X-ray view to show the main window, and another part to show the X-ray of the magnified window. You do NOT want the X-ray view to show only the main window's contents. Also note that the Z-order is important for operations that are not commutative, so you can't just put together a set and call it a day.
• The update rate of the current locations of all of the windows needs to be at least as fast as the refresh rate of the screen. If this isn't true, then the auxiliary windows' contents will be laggy (I'm ignoring the time it takes to calculate and redraw the contents, that's not your problem)
• I would like it if the windows could be translucent/transparent so that in those cases where I just need to overlay some information, I can, rather than having to recalculate the contents of the windows that are below. In the ideal case I could set the window's background to be entirely transparent, but still draw entirely opaque objects into the window's foreground, kind of like how if you laid a piece of plastic over a sheet of paper and then drew on the plastic with an opaque marker.

So, what am I using this capability for? Quite a number of potential use cases, some of which are below:

• BIM/CAD rendering. An X-ray window that lets you 'see' the wiring or plumbing within a wall. Layer over that a measurement window that you expand over an entire wall, and you can select a point on your wall, and a point on a wire, and get a measurement from an outside corner of a building to a point along the wall where the wire/plumbing is.
• As a way of displaying information like fluid flow, or temperature in a simulation when you're interested in information in one area, while still seeing the big picture.
• Anything where you want to see specific information, while still seeing the big picture.

I'm sure others can think of additional uses, but those are the ones that I'm thinking about.

[Songtronix]

Is there an ETA for multiple windows or someone already working on it?

[hecrj]

@Songtronix Not really. I know some folks were experimenting with running two Iced applications in different processes and communicating them using a Subscription.

I will most likely tackle this after #32. It always helps me to know more about different use cases, so feel free to share your own!

[krooq]

I know some folks were experimenting with running two Iced applications in different processes and communicating them using a Subscription.

I was thinking about this concept the other day. I wondered if it is really necessary to have multiple windows for a single "app". Especially since it's not clear what this means for web.

What do you guys think the use cases are for needing multiple windows that cant be achieved with inter-process comms?

[maroider]

@krooq In regards to the web:

If the proposed Application-like trait (which I'll call NativeWindow here) is the interface which gets implemented, then I'd imagine that you'd also have a another trait (which might (not) be called MaybeNativeWindow) which is mostly the same as NativeWindow, but it would be drawn as a "window widget" as described in #254 on the web back-end (and potentially other back-ends which don't let you create multiple windows, if any).

[krooq]

I have 2 thoughts/concerns here:

1. How does multi window change the cross platform nature of the iced ecosystem. As @maroider mentions, one solution is to have a widget that emulates some window behaviour but still only 1 window. @vorner IPC is extra work and its slow so I agree there are definitely cases where process with IPC approach isn't ideal. Which brings me to my second thought..
2. Were multiple windows ever a good choice in any app? What advantage does an app get from having multiple windows?
   e.g. nothing about a dialog means it needs to be in a separate window.
   Apps can simply have one window and control what is drawn inside it. Even an operating system could do this. The flip side is you don't get the OS/window manager behaviours, but do you need them?

[krooq]

@maxjoehnk I was thinking that maybe multiple windows could be emulated inside a single window that spans multiple desktops with transparent areas. But I'm not sure if all of the native APIs allow for areas of event pass through on different windows. Pretty sure it's possible on X.

All that a native window gives you is a rendering context, events and some native window manager stuff like resize and decoration.
How one handles the events and rendering is up to you/iced.

This is why I'm not convinced that multiple windows per application is an ideal solution.
Lots of modern apps actually only have a single window so that they can target the browser as well. A good example is VSCode, AFAIK it is single window, it spools up a new instance for additional workspaces and that model works just fine. In fact I didn't notice that VSCode was single window until just now.

Another advantage to being single window is that when you switch focus, your entire application is brought to front by the window manager. It's like the OS is giving you a box to put your application in and we want to rip a hole in the box and duct tape on another one. That could certainly be fun but I'm not sure it's advisable.

[maxjoehnk]

@krooq I'm not sure how portable that would be, e.g. whether OSX and Windows allow for such a use case. Also iced would need to handle all edge cases with weird screen layouts.
What about hotkeys for window arrangement? I would not be able to assign this application to workspace 2 running on my secondary monitor or iced would need to read all the different window manager configurations to get the hotkeys for whatever I'm running.
This is not a problem for vs code because everthing is in this single window but I'm also not able to have the same project open on two screens without two instances running side by side.
So I'm unable to open a tool window in window 1 when I've focused window 2.

The focus switch sounds quite nice but I would not want this kind of behavior for every application. I like to have multiple browser windows open on multiple workspaces on multiple screens and do not want my wm to bring all the different windows to the front when I focus the window with some documentation for the code I'm working on.
I don't think your analogy is fitting as these are tools the operating system gives you out of the box, even on android (not sure about iOS) you're able to do this.

[krooq]

I'm not sure how portable that would be, e.g. whether OSX and Windows allow for such a use case. Also iced would need to handle all edge cases with weird screen layouts.
What about hotkeys for window arrangement? I would not be able to assign this application to workspace 2 running on my secondary monitor or iced would need to read all the different window manager configurations to get the hotkeys for whatever I'm running.

This is all true. I don't think the transparent window idea is the right solution.

This is not a problem for vs code because everything is in this single window but I'm also not able to have the same project open on two screens without two instances running side by side.
So I'm unable to open a tool window in window 1 when I've focused window 2

But the question is why is another window needed for the tools?
VSCode gets along fine without it.
Some portability will be sacrificed to get that second window, I think its easy to allow features only on some platforms but as soon as you do, suddenly you're not really all that cross platform any more.

I don't think your analogy is fitting as these are tools the operating system gives you out of the box, even on android (not sure about iOS) you're able to do this.

Yes true. However the job of the OS is to let you do whatever you want safely and securely.
I think the job of Iced is to help you do what you really want in a smart way and take that thing wherever you need it.

Don't get me wrong, I still want multi window. I think the concept of multiple windows has great merit and has stood the test of time. I just know how much of a pain they can be across platforms.
As an example, I'm using Java at work for a systems application. We have lots of windows in our application, we even have 3 different windows stacked on top of each other, one for JavaFX, one for a separate OpenGL surface and one for a sharing events. We do this for performance and library limitation reasons. We also have our own window manager written in C since we have requirements about frame coloring when FX or GL graphics are obscured by windows.
We develop on Windows and target Unix and man is it a pain do anything window related. Even things you think would be the same are a little off for some reason.

[maxjoehnk]

But the question is why is another window needed for the tools?
VSCode gets along fine without it.
The second window is not for tools but for having code open on multiple screens but in the same project. So both windows have code but you may have a terminal in window 1 but are currently typing in window 2.
The portability issue is a real problem but I feel like this should be decided by the application and not the by the library it's depending on.

That sounds like a giant pita.

Maybe we can add some kind of extension system where you have traits for specific non cross platform features (e.g. multiple windows or maybe something like dialogs). This could allow native multi window support as well as "emulated" multi window support with the same implementation.

[vorner]

I believe applications should act in a way consistent with the platform they run on. For desktop, that means asking questions as separate modal windows (which you can still drag outside of their parent window to uncover what's below it) or configuration dialog. That certainly is different on non-desktop platforms ‒ modal windows on eg. Android just aren't a thing.

Having multiple top-level windows is also kind of power-feature of desktop that can't really be expected in other contexts (I don't know if single application could open multiple browser tabs and/or windows and act as one application; it certainly doesn't make sense in mobile devices where the surface is severely limited).

So there are two possible ways. One way is somehow trying to find a common ground ‒ which might be good enough for some applications, but not for everything. Once you start drawing modal windows inside the top level windows on desktop, users will get the not-quite-native almost-but-not-really-great feeling.

The other one is somehow mapping the UX intentions to the native expressive means. So I could imagine having some kind of modal question flow expressed in code that gets mapped to a separate window on desktop, replaces the application window in android and draws a window-inside-a-window on the web. I could imagine having some kind of top-level window/viewport thing that you can have multiple ones on desktop and only one on other platforms (and get a panic if you try to create a second) or only one visible at a time on other platforms (and you get some kind of tabs or have to manually switch them).

I also think that you could have almost the same code for the platforms in the end application. So you'd conditionally not show the button to create another top-level window, or conditionally have code for the tabs, but all the bulk of code for drawing into the multiple (same) windows or computations, etc, could stay the same.

Of course the hard part would be actually finding all the conceptual flows and their mappings to native tools.

I think at least these three are there:

• Top-level window (with allowing multiple ones on desktop; maybe limiting to just one or just one visible elsewhere)
• Tooling/non-modal dialog (developer and/or user could choose what to do with it on desktop ‒ like, separate window or dock to the side of the top-level window; it would always get docked on the web; maybe could be hidden by that arrow thingie; roll-up on mobile)
• Modal question/dialog (window on desktop, window-in-window on web, some kind of overlay thing on mobile)

[krooq]

This is all really good discussion.

Maybe we can add some kind of extension system where you have traits for specific non cross platform features (e.g. multiple windows or maybe something like dialogs). This could allow native multi window support as well as "emulated" multi window support with the same implementation.

I like this idea. It should be very clear that to the user how certain features will behave on certain platforms. I'd love to see Iced being used for simple small desktop applications over basic WinAPI or other toolkits, many of these tools are little desktop widgets which may want multi window.

Of course the hard part would be actually finding all the conceptual flows and their mappings to native tools.

I agree, I think this will be very difficult.
WebAssembly has a long way to go still and I don't think we have the scope to solve all of it's complexities. Pretty sure wgpu doesn't support running WASM in a native context e.g. using wasmtime or similar rather than a browser.

I believe we should guide users to think about alternatives to multi-window for their functional use cases as currently multi-window is quite strongly tied to the desktop platform and behavior is generally ambiguous on web and mobile.
For these use cases it should be clear to the user that an alternative strategy will be required if they want some kind of multi-window behavior across platforms. Whatever alternatives are available in iced should be documented clearly on window creation points.

[heyrict]

I would personally prefer an API with e.g.

enum DialogStrategy {
    Auto, // e.g. `Emulated` in web and `Float` in native platform.
    Float, // multi-window one
    Emulated // inside-window one
}

that can be passed to struct Dialog. Though it may be not a good choice to stuff the code for both strategy in the same place, users don't need to call different APIs for different platforms.

---

Actually, I don't think it is a good choice to have the "emulated window" implementation for non-native platforms (esp. the web). In the web, a "dialog" can be as simple as a <div> box element drawn over the content. The title itself is a <div> box inside the dialog, which can have customized styles and colors (even diamond-shaped dialog is fine) that won't be possible in a native dialog. Users should have their own dialog implementation with Layer if they want to target web.

[alvinhochun]

How would multi-window play along with integrating iced_native from within a custom engine? I imagine it would need to be an approach like the above Dear ImGui example, but to get the fancy seamless virtual window and top-level native window switchover would be extremely tricky, so the initial implementation would start out with virtual windows only. And if you consider games that typically may run fullscreen, additional top-level native windows are of no use. Besides, it still needs to support a top-level widget tree that is not a part of any virtual windows, to act as an overlaid HUD, for on-screen controls and to allow for the virtual windows to be docked to the edge of the viewport.

However, that goes against the model of other use cases for traditional desktop application where every "window" is a top-level native window, except for maybe dockable tool windows, but those are still different because they either stay as a top-level native window or get docked inside the main window, and there is almost never the need to become a virtual window.

[obsgolem]

Just a comment on a usecase for this, I would like to wrap plotters in a matlab/matplotlib style api using plotters-iced as a backend. For this to work I would need to be able to support multiple windows, just as those two do.

[ChrisCochrun]

Is anyone aware of the rough ideas one would need to implement this?