johanhelsing/matchbox

Painless peer-to-peer WebRTC networking for rust wasm (and native!)

Painless peer-to-peer WebRTC networking for rust's native and wasm applications.

The goal of the Matchbox project is to enable udp-like, unordered, unreliable p2p connections in web browsers or native to facilitate low-latency multiplayer games.

Matchbox supports both unreliable and reliable data channels, with configurable ordering guarantees and variable packet retransmits.

• Tutorial for usage with Bevy and GGRS

The Matchbox project contains:

• matchbox_socket: A socket abstraction for Wasm or Native, with:
  • ggrs: A feature providing a ggrs compatible socket.
• matchbox_signaling: A signaling server library, with ready to use examples
• matchbox_server: A ready to use full-mesh signalling server
• bevy_matchbox: A matchbox_socket integration for the Bevy game engine

  bevy	bevy_matchbox
  0.13	0.9, main
  0.12	0.8
  0.11	0.7
  0.10	0.6
  < 0.9	Unsupported

Examples

• simple: A simple communication loop using matchbox_socket
• bevy_ggrs: An example browser game, using bevy and bevy_ggrs
  • Live 2-player demo: https://helsing.studio/box_game/
  • Live 4-player demo: https://helsing.studio/box_game/?players=4

How it works

WebRTC allows direct connections between peers, but in order to establish those connections, some kind of signaling service is needed. matchbox_server is such a service. Once the connections are established, however, data will flow directly between peers, and no traffic will go through the signaling server.

The signaling service needs to run somewhere all clients can reach it over http or https connections. In production, this usually means the public internet.

When a client wants to join a p2p (mesh) network, it connects to the signaling service. The signaling server then notifies the peers that have already connected about the new peer (sends a NewPeer event).

Peers then negotiate a connection through the signaling server. The initiator sends an "offer" and the recipient responds with an "answer." Once peers have enough information relayed, a RTCPeerConnection is established for each peer, which comes with one or more data channels.

All of this, however, is hidden from rust application code. All you will need to do on the client side, is:

• Create a new socket, and give it a signaling server url
• .await the message loop future that processes new messages.
  • If you are using Bevy, this is done automatically by bevy_matchbox (see the bevy_ggrs example).
  • Otherwise, if you are using WASM, wasm-bindgen-futures can help (see the simple example).
  • Alternatively, the future can be polled manually, i.e. once per frame.

You can hook into the lifecycle of your socket through the socket's API, such as connection state changes. Similarly, you can send packets to peers using the socket through a simple, non-blocking method.

Showcase

Projects using Matchbox:

• NES Bundler - Transform your NES game into a single executable targeting your favorite OS!
• Cargo Space (in development) - A coop 2D space game about building and flying a ship together
• Extreme Bevy - Simple 2-player arcade shooter
• Matchbox demo
• A Janitors Nightmare - 2-player jam game
• Lavagna - collaborative blackboard for online meetings

Contributing

PRs welcome!

If you have questions or suggestions, feel free to make an issue. There's also a Discord channel if you want to get in touch.

Thanks

• A huge thanks to Ernest Wong for his Dango Tribute experiment! matchbox_socket is heavily inspired its wasm-bindgen server_socket and Matchbox would probably not exist without it.

License

All code in this repository dual-licensed under either:

• MIT License or http://opensource.org/licenses/MIT
• Apache License, Version 2.0 or http://www.apache.org/licenses/LICENSE-2.0

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

matchbox's Issues

Matchbox is broken on Firefox

I tried following @johanhelsing's blog post series for extreme-bevy and got some success, however after a few seconds my clients randomly disconnect after a short lag spike, while my Chromium peer tells me the following:

Failed to send: JsValue(InvalidStateError: 
Failed to execute 'send' on 'RTCDataChannel':
RTCDataChannel.readyState is not 'open'

coming from here:

matchbox/matchbox_socket/src/webrtc_socket/wasm/message_loop.rs

Line 107 in 3f86e86

error!("Failed to send: {err:?}");

After some testing and consulting @gschup on Discord, I found out that this happens every time at least one of the two peers uses Firefox.

I tested the following configurations, all resulting in the error:

• Firefox versions:
  • stable 100.0
  • nightly 102.0a1
• setup:
  • two peers on same machine
  • two peers on two machines
• operating systems
  • Windows 10
  • macOS Monterey
• games
  • https://github.com/janhohenheim/extreme-bevy
    • My Cargo.toml has all git dependencies pinned to specific revisions, so this should be very reproducible
    • networking.rs
  • https://helsing.studio/box_game/ (live version)
  • https://gschup.github.io/bevy_ggrs_demo/ (live version)

untested:

• Firefox versions:
  • older than 100.0
• operating systems
  • linux
• setup
  • more than two peers

Maintain peer list without developer maintenance

disonnected_peers and accept_new_connections are something you have to remember to call to maintain the peers list.

This could be handled a different way which is a lower cost for the developer.

Add CI for wasm

Our CI workflow should make sure the crate builds for wasm as well (currently it's only native).

cannot run bevy_ggrs example

In Windows 10 PowerShell I run:

PS C:\Users\MBS\matchbox\matchbox-main\examples\bevy_ggrs> cargo run --matchbox ws://127.0.0.1:3536 --players 2 --room test
error: Found argument '--matchbox' which wasn't expected, or isn't valid in this context

If you tried to supply '--matchbox' as a value rather than a flag, use '-- --matchbox'

Usage: cargo.exe run [OPTIONS] [args]...

For more information try '--help'

Any ideas?

Modernize API Initiative

Johan,

Thanks for leading the process in WebRTC. I kno I want to point out some initial potential for matchbox to provide a richer user experience and this issue is designed to frame that initiative.

I want to get your feedback and ask what you may agree or disagree on, and then we can stub out the issues separately for tracking.

To ensure the future success of matchbox, here are some initiatives I've thought of:

• #![deny(clippy::unwrap_used)] and #![deny(clippy::expect_used)] There's 46 unwrap()s and 81 expect()s. Granted, most are in demos, several are in the library. Instead, we should return an error type.
• Eagerly return Results. In many cases (such as a peer disconnect #81) we either drop control flow, return early, unwrap, etc. There's no consistency. Some methods like try_send don't return a result, which is unintuitive.
• Rename room_url to session_url or just url
• Do not provide a default WebRtcSocketConfig room url. That seems too specific to default, and could be a footgun for new users.
• Provide fallback/more stun servers. I think right now it's just using one (stun:stun.l.google.com:19302)
• Rename matchbox_server to matchbox_demo_server since it has tight coupling logic to the demo, i.e. (next routing)
• BYOS - Bring your own state machine. Perhaps add the ability to replace the signalling/messaging loops with your own state machine with a trait like MessageLoopStateMachine, with methods on_error, on_message, etc...

Browsers complain about using 5 or more stun servers

WebRTC: Using more than two STUN/TURN servers slows down discovery
WebRTC: Using five or more STUN/TURN servers causes problems

I'm not sure what kind of problems that would be, but perhaps we should go down to 4?

imports granularity = crate

https://rust-lang.github.io/rustfmt/?version=v1.5.1&search=#imports_granularity

Try to decouple signalling from socket message loop

Ideally, I would like matchbox socket to be less coupled to matchbox server (so you could use a custom signalling or matchmaking mechanism).

How to deal with `.expect` and/or `.unwrap`?

A question was raised in #83 about denying use of .expect and .unwrap. Pretty sure this is a no from me, but I thought I'd move the discussion out to a separate issue:

For instance, there are a couple of places where .unwrap() is used intentionally, because the result or option can never be none/err.

If returning Option or Result is inappropriate, and there is logical "proof" what you're doing is safe, we should opt for a // SAFETY: ... note attached to unsafe { action_unchecked(...) }. Denying all unwrap/expect seems to me like a good goal to have as a north-star objective.

wasm-bindgen is generated and not really an idiomatic rust API, and that sometimes makes it very hard not to use expect. There are cases when fields are options, but are documented not to be null in MDN, in which case, I don't want to have a cascading set of errors through the internal API just telling the various parts that your browser is violating the spec or the javascript engine is broken.

In those cases, it's better to panic, and if we get a bug report about a misbehaving browser, we can add the necessary error handling to deal with it.

Other cases are places where we use channels for communicating that something happened in an event callback that will only be called once, though due to how ownership of callbacks are handled, it's hard to convince the rust compiler of that.

Perhaps there are helpers in projects like gloo that would make some of this doeable, though. It's something to investigate.

I think the best thing to do would be to go through the .expect's and .unwraps on a case by case basis, and remove them if we can, propagate Err if it's something that could actually happen without something else (i.e. browser not following spec) being horribly broken, and in the rest of the cases, make sure the .expect message and/or comment say why it should not happen and why we can't remove it.

I'm, unsure whether using SAFETY is the right thing here, I've normally just seen this when something is using unsafe rust, which is not what this is about? Pretty sure we could add #![forbid(unsafe_code) and pass.

Update bevy_ggrs example for Bevy 0.10

Proper error handling

Proper error handling, everything is littered with expect/unwrap at the moment.

If a client disconnects, TOO BAD.

Needs proper mechanisms for handling failures, reconnecting etc.

Handle returning to ice gathering stage

Currently we stop anything related to ice once the data channel has opened, however It's possible to return to the ice gathering state, and we should probably handle that.

Ref this comment:

matchbox/matchbox_socket/src/webrtc_socket/wasm/message_loop.rs

Line 255 in 97c2ea1

// TODO: we should support getting new ICE candidates even after connecting,

https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/iceGatheringState

Handle binary data

Currently there's no handler for binary data, but it could be added quite trivially with bincode. I think this is an easy-win.

matchbox/matchbox_socket/src/webrtc_socket/native/signalling_loop.rs

Line 42 in c9d10cb

warn!("ignoring unexpected non-text message from signalling server: {:?}", message)

Deduplicate and refactor code

Adding complex new features like #64 is made more difficult because of mainly two issues in the code structure.

1. There are two versions of the WebRTC socket and all its components for two compile target (groups) - one for native and one for wasm. All work needs to be done twice and even if both code versions resemble each other in many ways there are also many differences

2. Most of the code lives in one huge file (per target) which is not too easy to understand - the code structure as it is now doesn't quite fit in my head and I find myself looping over and over through the code trying to understand how one thing affects the other and where the data is flowing. For example even after spending hours with it it's still not clear for me what is the main difference between handshake_offer and handshake_accept or the handling of PeerEvent::NewPeer and PeerEvent::Signal and why they are so similar and if the code can be easily deduplicated or not. Or what exactly is the exact purpose of all the different senders and receivers.

Ideally before starting some major refactoring work we can get rid of the separate code for wasm. Unfortunately webrtc-rs doesn't yet support wasm and it's not even planned but they opened an issue for that topic with a help wanted label suggesting that they are open to it:
webrtc-rs/webrtc#351

I would be motivated to have a go at it if you also would be looking forward to use this later in matchbox and if you agree that this could dramatically reduce or even eliminate our wasm-specific code.

Please let me know what you think

Ice gathering times out after 30 seconds in chromium on linux

TURN support

When STUN is not enough, try TURN. This might be very close to working, we just need to set up a test turn server and add API for overriding the ice server config in the WebRTC socket.

`ggrs-socket` feature does not work with Bevy 0.9

The bevy_ggrs main branch has been fixed to work with Bevy 0.9, but needs currently unreleased changes in ggrs.

This means the ggrs-socket feature in this repo depends on an older ggrs version than bevy_ggrs.

After the next version of ggrs is released, we should update the dependency and the matchbox_demo example.

If you encounter this, a possible workaround is to apply a patch for ggrs:

Cargo.toml:

[dependencies]
bevy = "0.9"
matchbox_socket = {version = "0.4", features = ["ggrs-socket"]}
bevy_ggrs = {git = "https://github.com/gschup/bevy_ggrs"}

[patch.crates-io]
ggrs = {git = "https://github.com/gschup/ggrs"}

Add keep-alive/ping request to prevent idle websocket (signalling) connections from timing out

When matchbox_server is hosted behind a https reverse-proxy such as nginx, it is likely configured to have an idle timeout set for websockets, which means that if players wait for a connection longer than the timeout, or are connected longer than the timeout, the socket will disconnect, which currently leads to a panic.

One fix for this would be to simply disable the idle timeout for websockets on your proxy.

A more user-friendly approach is probably to add some sort of ping or keepalive request to the signalling protocol and just call it at regular intervals lower than the lowest common idle timeout (around 1 minute?).

Change the underlying type of `PeerId`?

One thing I've been thinking about more or less from the start of the project, is whether it would be better to use a more well-suited type for PeerId. Some options:

• u64
• uuid::Uuid. This is what we currently use on the singalling server side to generate, then we convert into string after.
• Cow<str>?

Thoughts?

Personally I'm leaning towards u64 (or a newtype for it rather), simply because it's so... simple, should fit well with any kind of serialization/signalling server. And also very obviously cheap.

bevy_ggrs_example errors when run with `wasm32-unknown-unknown` target

When bevy_ggrs_example is run via cargo run -p bevy_ggrs_example --target wasm32-unknown-unknown, the following error message is produced

Finished dev [unoptimized + debuginfo] target(s) in 24.69s
 Running `target/wasm32-unknown-unknown/debug/bevy_ggrs_example.wasm`
target/wasm32-unknown-unknown/debug/bevy_ggrs_example.wasm: 1: Syntax error: end of file unexpected

With WebRtcNonBlockingSocket, there is no way to get your own peer id

Getting your own peer id can be useful in order to seed rng for instance: johanhelsing/johanhelsing.github.io#11 (comment)

It is possible to get the peer id on the WebRtcSocket::id(), however WebRtcNonBlockingSocket wraps it opaquely, which means it's inaccessible to API users.

So this means that we should either:

1. Just implement an id wrapper method on the non-blocking socket as well.
2. Make WebRtcNonBlockingSocket::socket public
3. At this point, the WebRtcSocket<String> interface is so similar to WebRtcNonBlockingSocket that we might not even need a wrapper type for it. We could perhaps just implement the needed ggrs trait directly on WebRtcSocket instead.

refreshing web demo page crashes native one

• how to repo

1. run matchbox server locally:
   cargo run --release --bin matchbox_server
2. run matchbox demo（wasm） locally:
   cargo run --target wasm32-unknown-unknown
3. try to play with demo in browser environment:
   http://127.0.0.1:1334/
4. try to play with demo in native environmnt:
   .\target\release\matchbox_demo.exe
5. refresh web page and it crashes the native one immediately

• error message

thread 'IO Task Pool (3)' panicked at 'called `Result::unwrap()` on an `Err` value: ErrClosedPipe', matchbox_socket\src\webrtc_socket\native\message_loop.rs:530:51
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at 'send_to failed: TrySendError { kind: Disconnected }', matchbox_socket\src\webrtc_socket\socket.rs:282:14

Make a conscious choice about default ICE server configuration

In matchbox 0.3, we defaulted to stun:stun.johanhelsing.studio:3478, that seemed to work fine for me on:

@heartlabs reported some problems with stun requests timing out in Chromium. So we switched to "stun:stun.l.google.com:19302", which seemed to work fine.]

I did some tests on various platforms recently, and although everything seems to work fine with Google's stun server on windows, I no longer get connections in Firefox on linux.

server	FF (win)	FF (linux)	Chrome (win)	Chromium (linux)
stun:stun.l.google.com:19302	ok	no connection	ok	binding timeout (30s) then connects
stun:stun.johanhelsing.studio:3478	ok	ok	ok	binding timeout (30s) then connects

@heartlabs what os and browser + version did you experience the timeouts with?

document what KeepAlive does

These packets being sent seemingly do nothing. Was there a purpose originally intended here?

Create a common library for common code

It would likely make sense to have a common library for code which is shared between the server and the socket. For example, the PeerEvent enum

Make connection to signalling server more robust

Haven't had issues with this yet, but we should probably try reconnecting to the signalling server if the connection is lost.

Connectivity issues in the browser

I am having a few issues with getting p2p connections between browser tabs. In Firefox, I get the error "WebRTC: ICE failed, add a TURN server and see about:webrtc for more details" around 80% of the time (some browser logs below).

On Chrome it works every time but takes pretty long (30+ seconds!). In your video on twitter the connection seems to open quickly in firefox, so I am wondering if this changed for you, or if this is an issue with my code/browser or network.

Interesting parts from firefox webrtc logs in order (but with holes)

...
(id=23622320129 url=http://localhost:4000/)): Skipping STUN server because of address type mis-match
/build/firefox-fXSvPi/firefox-94.0+build3/dom/media/webrtc/transport/third_party/nICEr/src/net/nr_socket_multi_tcp.c:175 function nr_socket_multi_tcp_create_stun_server_socket skipping UDP STUN server(addr:IP4:0.0.0.0:3478/UDP)
/build/firefox-fXSvPi/firefox-94.0+build3/dom/media/webrtc/transport/third_party/nICEr/src/net/nr_socket_multi_tcp.c:175 function nr_socket_multi_tcp_create_stun_server_socket skipping UDP STUN server(addr:IP6:[::]:3478/UDP)
/build/firefox-fXSvPi/firefox-94.0+build3/dom/media/webrtc/transport/third_party/nICEr/src/net/nr_socket_multi_tcp.c:623 function nr_socket_multi_tcp_listen failed with error 3
(id=23622320129 url=http://localhost:4000/)): failed to create passive TCP host candidate: 3

block above two times
...
(id=23622320129 url=http://localhost:4000/))/CAND-PAIR(kMAM): Pairing candidate IP4:10.16.212.74:46958/UDP

... 11 seconds later
Responding with error 400: ICE Failure

ICE(PC:{7ba9e854-55ab-487c-a4bc-76d7fb530c25} <time> (id=27917287425 url=http://localhost:4000/)): Message does not correspond to any registered stun ctx

STUN-CLIENT(kMAM|IP4:10.16.212.74:46958/UDP|IP4:10.16.212.74:44090/UDP(host(IP4:10.16.212.74:46958/UDP)|candidate:0 1 UDP 2122252543 10.16.212.74 44090 typ host)): Received response; processing

STUN-CLIENT(kMAM|IP4:10.16.212.74:46958/UDP|IP4:10.16.212.74:44090/UDP(host(IP4:10.16.212.74:46958/UDP)|candidate:0 1 UDP 2122252543 10.16.212.74 44090 typ host)): nr_stun_process_error_response failed

STUN-CLIENT(kMAM|IP4:10.16.212.74:46958/UDP|IP4:10.16.212.74:44090/UDP(host(IP4:10.16.212.74:46958/UDP)|candidate:0 1 UDP 2122252543 10.16.212.74 44090 typ host)): Error processing response: Retry may be possible, stun error code 400.

... block above 7 times

STUN-CLIENT(kMAM|IP4:10.16.212.74:46958/UDP|IP4:10.16.212.74:44090/UDP(host(IP4:10.16.212.74:46958/UDP)|candidate:0 1 UDP 2122252543 10.16.212.74 44090 typ host)): Timed out

ICE-PEER(PC:{0a2c1cc7-e126-48d3-b5e1-9daf5d6f401f} <time> (id=23622320129 url=http://localhost:4000/):default)/CAND-PAIR(kMAM): setting pair to state FAILED: kMAM|IP4:10.16.212.74:46958/UDP|IP4:10.16.212.74:44090/UDP(host(IP4:10.16.212.74:46958/UDP)|candidate:0 1 UDP 2122252543 10.16.212.74 44090 typ host)

ICE-PEER(PC:{0a2c1cc7-e126-48d3-b5e1-9daf5d6f401f} <time> (id=23622320129 url=http://localhost:4000/):default)/STREAM(PC:{0a2c1cc7-e126-48d3-b5e1-9daf5d6f401f} <time> (id=23622320129 url=http://localhost:4000/) transport-id=transport_0 - 535cef18:724fdd891babdd9d2d8480c9af275f58)/COMP(1): All pairs are failed, and grace period has elapsed. Marking component as failed.

Server: Add additional types of scoping for next_n rooms

The next_n rooms are really useful if you want to connect to other players as quickly as possible.

However it would be nice if they allowed the clients to match on some criteria. in addition to the total number of players. e.g.

• Are they playing the same game? (would allow to reuse the same server for multiple games)
• Game version
• Specific map
• Game mode
• etc.

I think it would make most sense to make this generic so it could support criteria and use cases I haven't thought of.

In its simplest form, this could just be a single string key decided by the game: e.g.

wss://match.myserver.com/next_4?key=thegame_v1.0_deathmatch

Not sure if key is the most fitting name here. scope, namespace? Or have it as a path arg? Open to suggestions.

demo: switch to bevyengine/bevy

We're currently using my own fork: https://github.com/johanhelsing/bevy, which is not ideal, we should just use plain bevy main branch.

There are some problems in the way, however. bevy_webgl2 does not compile with latest bevy, and bevy_ggrs depends on patches not present in bevy 0.5.

One option is to simply wait for bevy 0.6 and bevy_webgl to update.

Another is to fix the compatibility issues in bevy_webgl and use a patched version of that. Perhaps even ask if it could be hosted in the official bevy_webgl2 repo.

[dev profile] Panic in webrtc after 8-10 minutes

I am using matchbox_socket 0.5 with bevy_ggrs 0.11, using the ggrs_socket feature on a native build.

I am able to reliably cause the following panic after ~30000 frames of gameplay (frame count is over some rollbacks, so it was likely even more simulated frames, I can get an exact count later). This occurs in both my private game and in this example repository: https://github.com/cscorley/bevy_ggrs_rapier_example, which I think can be used to reproduce.

I believe it is hitting this limit https://github.com/webrtc-rs/webrtc/blob/master/sctp/src/queue/reassembly_queue.rs#L120 I could not find any existing issue open that seemed related to this in the webrtc repository.

I think this is likely an upstream issue ultimately, but thought I would start tracking it here since the socket seems responsible for the data channel handling (e.g., is there an option to clear/renew?). I can collect more information here in ~24 hours. I'm not certain how to setup a small reproducible example for the webrtc repo yet.

thread 'async-compat/tokio-1' panicked at 'attempt to add with overflow', /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/webrtc-sctp-0.7.0/src/queue/reassembly_queue.rs:270:17
stack backtrace:
   0: rust_begin_unwind
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:575:5
   1: core::panicking::panic_fmt
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/panicking.rs:65:14
   2: core::panicking::panic
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/panicking.rs:115:5
   3: webrtc_sctp::queue::reassembly_queue::ReassemblyQueue::read
   4: webrtc_sctp::stream::Stream::read_sctp::{{closure}}
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/webrtc-sctp-0.7.0/src/stream/mod.rs:199:17
   5: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/future/mod.rs:91:19
   6: webrtc_data::data_channel::DataChannel::read_data_channel::{{closure}}
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/webrtc-data-0.6.0/src/data_channel/mod.rs:171:64
   7: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/future/mod.rs:91:19
   8: webrtc::data_channel::RTCDataChannel::read_loop::{{closure}}::{{closure}}
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/macros/select.rs:517:49
   9: <tokio::future::poll_fn::PollFn<F> as core::future::future::Future>::poll
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/future/poll_fn.rs:58:9
  10: webrtc::data_channel::RTCDataChannel::read_loop::{{closure}}
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/webrtc-0.6.0/src/data_channel/mod.rs:309:34
  11: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/future/mod.rs:91:19
  12: webrtc::data_channel::RTCDataChannel::handle_open::{{closure}}::{{closure}}
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/webrtc-0.6.0/src/data_channel/mod.rs:288:17
  13: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/future/mod.rs:91:19
  14: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/core.rs:223:17
  15: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/loom/std/unsafe_cell.rs:14:9
  16: tokio::runtime::task::core::Core<T,S>::poll
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/core.rs:212:13
  17: tokio::runtime::task::harness::poll_future::{{closure}}
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/harness.rs:481:19
  18: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/panic/unwind_safe.rs:271:9
  19: std::panicking::try::do_call
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:483:40
  20: std::panicking::try
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:447:19
  21: std::panic::catch_unwind
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panic.rs:137:14
  22: tokio::runtime::task::harness::poll_future
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/harness.rs:469:18
  23: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/harness.rs:198:27
  24: tokio::runtime::task::harness::Harness<T,S>::poll
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/harness.rs:152:15
  25: tokio::runtime::task::LocalNotified<S>::run
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/mod.rs:459:9
  26: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/scheduler/current_thread.rs:584:25
  27: tokio::runtime::coop::with_budget
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/coop.rs:102:5
  28: tokio::runtime::coop::budget
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/coop.rs:68:5
  29: tokio::runtime::scheduler::current_thread::Context::run_task::{{closure}}
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/scheduler/current_thread.rs:285:29
  30: tokio::runtime::scheduler::current_thread::Context::enter
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/scheduler/current_thread.rs:350:19
  31: tokio::runtime::scheduler::current_thread::Context::run_task
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/scheduler/current_thread.rs:285:9
  32: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/scheduler/current_thread.rs:583:34
  33: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/scheduler/current_thread.rs:615:57
  34: tokio::macros::scoped_tls::ScopedKey<T>::set
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/macros/scoped_tls.rs:61:9
  35: tokio::runtime::scheduler::current_thread::CoreGuard::enter
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/scheduler/current_thread.rs:615:27
  36: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/scheduler/current_thread.rs:530:19
  37: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/scheduler/current_thread.rs:154:24
  38: tokio::runtime::runtime::Runtime::block_on
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.23.0/src/runtime/runtime.rs:282:47
  39: async_compat::TOKIO1::{{closure}}::{{closure}}
             at /home/cscorley/.cargo/registry/src/github.com-1ecc6299db9ec823/async-compat-0.2.1/src/lib.rs:426:24
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Supporting native clients

If this library were to support native builds (not WASM) to be clients, it would be incredibly useful.

Clarify role and roadmap for signalling server(s)

This is a pretty broad issue, continued from the discussion in #83, that issue interleaves several different issues, so here is the relevant parts quoted:

Rename matchbox_server to matchbox_demo_server since it has tight coupling logic to the demo, i.e. (next routing)

I think there is value in having a "default" server that people can just install and get up and running quickly. While I also feel like the next rooms are bordering on scope creep, it is a feature that is used a lot by early prototype games.

This is an area I've given thought to previously, I initially tried to abstract away the matchmaking part of this, allowing users to quickly write their own room finding logic and run it within the server, however I got stuck on creating a Warp Filter with a specified input type, see: seanmonstar/warp#72. Perhaps if #82 were merged it would be more easily done

What I did in my previous (unfinished) game was to run both matchbox_server and the matchmaking/lobby service behind a reverse proxy. Then the matchmaking service just generated a unique room id and told the appropriate clients it was time to connect through the matchbox_server endpoint.
I guess there are benefits of keeping it all in one process, though.

A possible solution could be to just stuff everything in a demo or examples folder. That would certainly reduce noise on the root level.

But otherwise, I think trying to provide a "one size fits all" general purpose server might not be realistic. If you want to go that route, make a proper CLI binary separate from this project.

Also some history in #4

Some initial thoughts

I think the output from this issue is:

• direction for development of matchbox_server and potential server api in immediate future is clear
• we have a tentative plan for the more distant future
• update readme/crate docs as appropriate

matchbox_server and project structure

About moving matchbox_server to a separate repo: In the immediate future, this is a very clear "no" from me. As it is, we're still adding new requests and events (i.e. addition of PeerLeft in #84); the API is clearly not done. I'm not saying never, but right now, I think the extra overhead of managing dependent PRs across multiple repos, communicating compatible and incompatible versions far outweighs the drawbacks of using a monorepo.

About moving things within the repo, and the name matchbox_server being potentially misleading: This is open for discussion. At the very least, it sounds like we should add a readme to the matchbox_server folder.

Current role of matchbox_server

Currently, matchbox_server is the signalling server. You could easily build your own, compatible, server, since the signalling protocol is so simple, but you'd have to keep up with any changes we make across breaking (0.x) releases.

I'm currently running it unmodified for all my own matchbox-related projects.

Gather requirements for why people don't want to use the existing server (as-is)

As seen in #4, and as @garryod's #83 (comment), there is some interest in being able to run parts of the signalling server yourself in your own server process. I'd like to better understand why. In my own projects, the feature set of matchbox_server has been sufficient and more or less unchanged since 0.1. I run matchmaking as a separate process, and when I have a group of peers I'd like to match, I just hand them a randomly generated id, and they connect to wss:://matchbox_server.example.com/<session_uuid>.

Perhaps this approach should be use by the demo as well? write a tiny "next" machmaking service and remove the feature from matchbox_server?

To me, this feels clean: the scope of matchbox_server is very limited, little room for bugs, easy to reason about or port to new stack, everyone is running the same thing (well-tested).

So what I'd like to know:

1. Are there things you'd like to add to the signalling endpoint itself? Authentication, validate claims etc.?
2. Are you bothered by running more than one process?
3. Other things?

This is not me saying "you're doing it wrong", I'd simply like to get a better idea about where this is coming from before making a decision.

PeerId acknowledgement or retransmit

Should we consider the connection until the user validates they've received their PeerId?

Currently we ignore the failure if it exists.

If the packet is lost, the peer has no way to remedy this, to receive their PeerId after they're connected.

Regression: Panic when the socket is dropped

Use https://github.com/gschup/bevy_ggrs_demo to join a game and then go back to the menu

(after game is finished or during waiting for an opponent). When doing this, the matchbox message loop unwraps None at src\webrtc_socket\native\message_loop.rs:132:39, making it impossible to retry connecting (will just do nothing then).

I think this used to work in 0.1

The demo website show 502 Bad Gateway

Add API for handling peer disconnections

Right now, peers are added to WebRtcSocket::peers when WebRtcSocket::accept_new_connections is called, and after that they can be accessed through WebRtcSocket::connected_peers.

However, WebRtcSocket::peers is never updated if peers disconnect. So connected_peers is actually kind of a lie.

We should add some way of letting the user know when peers disconnect, and make sure what's returned through connected_peers is correct.

Game crashes after a short time by triggering an invalid state

Hello,

I found out that after a short playtime of approximately 10 seconds the game decided to crash. Something that you must know is that I was in the train with a bad internet connection. This is also thanks to try the game with an instable connection that I found out that the game can establish a 2 player link without calling external services (a TRUN/STUN server somewhere?).

Update to webrtc-rs 0.6

Demo: Game pauses if the browser tab is hidden

If you hide a browser window, either by switching tab or minimizing the window. The demo pauses, and the other peers can no longer play.

I'm not sure if this is a bug in bevy_webgl, bevy, winit or whether it's simply not possible to avoid in web browsers. But it would be a good idea to check why it's happening.

Hardcoded ICE server URLs

Hi,
Firstly - thanks for creating matchbox - I love it :)

For me it takes ~30 seconds to establish a connection between peers using your library with wasm in a chromium browser and I tried to dig deeper until I found a nice log of the WebRTC implementation within the browser via chrome://webrtc-internals/.

I think I found the issue in these logs: Right after a ~30 seconds break in the log messages:

icecandidateerror
url: stun:stun.johanhelsing.studio:3478 
address: [0:0:0:x:x:x:x:x] 
port: 57086 
host_candidate: [0:0:0:x:x:x:x:x]:57086 
error_text: STUN binding request timed out. 
error_code: 701

The url surprised me as I didn't configure it anywhere and then I found it quickly in the client code right here:

matchbox/matchbox_socket/src/webrtc_socket/wasm/message_loop.rs

Line 287 in 0671bf1

"stun:stun.johanhelsing.studio:3478".to_string(),

As the request times out anyway probably it would work just as well if it was disabled entirely. However I suppose this part of the code originally made sense so I would like to suggest to make these URLs configurable.

I am quite new to Rust but if you like I could tackle this issue. Just let me know and I will try if I can get you a working PR

Don't panic when connecting to the signalling server fails

https://github.com/johanhelsing/matchbox/blob/main/matchbox_socket/src/webrtc_socket/native/signalling_loop.rs#L16

Same for wasm implementation.

We should not expect here, but rather propagate an error somehow, though exactly how is unclear.

related: #5

No notification when wasm ice connection state changes to Disconnected

Currently when the RtcIceConnectionState changes to we merely log the change, we should probably act on this the same way we act on changes to the RTCPeerConnectionState, sending a PeerState::Disconnected for Disconnected, Failed or Closed

Regression: Some peers can no longer connect

I suspect that there are cases where only non-negotiated channels are possible (should confirm this). If so, this is most likely a regression from #47 .

At least I had problems connecting with a friend today with an app built on the main branch. The old examples work, but my app using main receives ice candidates, but the data channel never opens. Unfortunately I lost the logs, will have to get new ones.

Add support for native to web cross-play

There are some small incompatibilities between the native and wasm implementation that makes connections fail.

The native version implements ICE candidate trickling, while the wasm version does not. (#16)

Simplify topology to users

Different topologies are needed sometimes. Perhaps one needs a global auction house trade negotiator (server<->client), a small matchmaking game of tag (many<->many), or a connect-4 game (one-to-one).

Currently matchbox's examples have many-to-many topology geared examples. I started looking into the code today trying to make a server-client example with matchbox and found it pretty difficult to bend it that way. Actually I gave up because I don't think Matchbox can bend that way currently.

I went looking for ways this is solved today.

Study: wasm-peers

wasm-peers fragments for the specific topology you need.

It's not all that complicated, either.

• one-to-many, many-to-many, and one-to-one are features that bring in the respective handlers, protocols, and callbacks
• protocol contains the signal message defintions for each of these topologies
• And they provide a one-to-many, many-to-many, and one-to-one signalling server

How they do it - High level

Protocols change, depending on the topology used.
For example, if a peer joins, messages to the signalling server could be different:

• many-to-many, one-to-one: SessionJoin(SessionId)
• one-to-many: SessionJoin(SessionId, IsHost)

In the case of one-to-many (client-server), the first peer to claim host IsHost (bool) = true to the signalling server is registered as host. If ever there are >= 2 peers, and one is host, a SessionReady is broadcasted in response to the SessionJoin.

---

I think what wasm-peers does makes sense. We should at least hack a solution to allow a client-server architecture and present it as an example. After that, I'm unsure.

Accessor to return peer connection state

I was able to make a client-server signalling server. When I rugpull (disconnect) the server from the clients, the signalling server still persists, so I have no indication of anything wrong happening (the socket's message loop future doesn't return).

I could make a connected bool or something for clients, listen for new connections (the host) and disconnections (host leaves), but I'm just wondering, is there a way to react on these events?

// Connected to the signalling server
2023-02-26T17:31:58.780432Z  INFO [controlled]: Setting new connection state: Connected    
2023-02-26T17:31:58.780533Z  INFO ICE connection state changed: connected    
2023-02-26T17:31:58.794609Z  INFO peer connection state changed: connected    

// Received host from signalling server
2023-02-26T17:31:58.805397Z  INFO Found a peer "6e405500-9939-48ee-9f2c-0a375bfd5635"    

// Host tells the client hello
2023-02-26T17:31:58.909887Z  INFO Received from "6e405500-9939-48ee-9f2c-0a375bfd5635": "hello client!"    

// Host disconnects
2023-02-26T17:32:25.209918Z  INFO Disconnected peers: ["6e405500-9939-48ee-9f2c-0a375bfd5635"]    
2023-02-26T19:48:57.007260Z  INFO [controlled]: Setting new connection state: Disconnected    
2023-02-26T19:48:57.007538Z  INFO ICE connection state changed: disconnected    
2023-02-26T19:48:57.007561Z  INFO peer connection state changed: disconnected    
2023-02-26T19:49:22.072352Z  WARN [controlled]: Failed to close candidate udp4 host 192.168.100.218:55599: the agent is closed    
2023-02-26T19:49:22.072454Z  INFO [controlled]: Setting new connection state: Failed    
2023-02-26T19:49:22.072689Z  INFO ICE connection state changed: failed    
2023-02-26T19:49:22.072705Z  INFO peer connection state changed: failed 

// Signalling still works! But how do we use this opportunity on the connection state change? Perhaps alert the user they're waiting for a new host?

Could we add something like socket.state()? Would that be useful?

Should peers be allowed to exchange metadata through the signalling server?

@Jobieskii asked this in the matchbox intro blog comments: johanhelsing/johanhelsing.github.io#11

The motivation behind this, is that often the peers want to exchange some extra information that is not necessarily game input. For instance seeds for rng, and chat before and after the game starts.

In the demo, and the extreme_bevy tutorial, this is currently hard to do, as the socket is simply handed to ggrs, and there is no way to use it to send other non-ggrs messages.

We should figure out if this is within scope for matchbox or not.

Technically, it could be done at many levels.

1. Add some sort of broadcast event to the signalling server. This would be really simple to implement, but would increase traffic through the signalling server -> more expensive to run.
2. Make it possible to open different sets of data channels for different parts of the app. One could unreliable and handed to ggrs, the others could perhaps be reliable (and be independent of the "ggrs" channel).
3. Leave to some other service.
4. Maybe it could be added to ggrs instead? Add optional support for rpc-like messages that happen sporadically with variable length?

@gschup What do you think?

Demo should be updated to Bevy 0.7

Needs to wait for a new release of the ggrs and bevy_ggrs crates since our ggrs-socket feature targets a specific version of ggrs.

PeerSignal documentation mishap?

matchbox/matchbox_socket/src/webrtc_socket/messages.rs

Line 22 in c9d10cb

/// Signals go from peer to peer

Is this correct? PeerSignals go from peer<->peer?

Starting with ws:// or ws://

matchbox/matchbox_socket/src/webrtc_socket/mod.rs

Line 47 in 1be86e6

/// This is a websocket url, starting with `ws://` or `ws://` followed by

Not sure what this is trying to say?