Question: Performance characteristics

[supabase]

Question: Performance characteristics

[kiwicopple]

The realtime part will wait until the db has made its changes before sending the update to listeners correct?

That's correct. This repo functions as a CDC, which gives some benefits and some tradeoffs.

Benefits

• The change can be triggered from anywhere. i.e. the transactions don't have to flow through Supabase to bubble up through the realtime server. If the change hits the database, it will be broadcast by this server
• Guaranteed delivery. If this server dies, you can spin it up to pick up where it left off
• Consistency. Because this broadcasts from the bottom-up, only committed transactions will be broadcast. If you broadcast from the middleware, it's possible that you send an event to all listeners which then fails to get saved in the database

Tradeoffs

• Latency/frequency/throughput - as you mention. Because of design (bottom-up events) and because of the tech stack. Saving micro-second state to Postgres is probably excessive and elixir is more well-known for it's concurrency/scalability than throughput.

thousand of updates per second

We're yet to performance test this, but this scale should be fine (with suitably sized servers). Regular postgres->postgres replication can handle this workload, so it should be fine.

Still, for HFT and multi-player you might be better off with C/C++/Rust

[redlock]

Thanks for the response. Looking forward to the benchmarks.

[kiwicopple]

Hey redlock, we're a bit swamped right now but we'd love to do some benchmarks at some stage.

Is there a deepstream benchmark or performance test you can point us towards?

[redlock]

Here is a benchmark they have done a while ago:
https://deepstream.io/info/performance/four-billion-messages-per-hour/

[kiwicopple]

Wow that's a tidy benchmark. We're still iterating on this server so it makes sense to hold the benchmark until we have a stable set of features. Once it's stable then we will loop back on this one.

It's important to note that this repo is for listening to your Postgres changes. You don't send events through this elixir server, it simply picks up the database changes and broadcasts them.

To trigger an event you just need to INSERT/UPDATE/DELETE from postgres. You can do this in any way - we use PostgREST heavily in Supabase, but you could even just update using psql or your favourite database IDE (like Dbeaver) and this will still broadcast the event.

[kiwicopple]

I know this hasn't yet answered your question @redlock - it could be a while before we can close this one. If you're happy to wait, feel free to close it and we'll run the benchmarks before we reach 1.0.

[redlock]

Thanks for the response. Just a quick question. The realtime part will wait until the db has made its changes before sending the update to listeners correct?

This might add a bottleneck here if thousand of updates per second are being sent. Could there be an option to send the update without waiting for the db? For cases where latency is paramount (High frequency trading, multi player gaming, etc)

[kiwicopple]

thanks for checking us out redlock. If I can remember I'll post the benchmarks link here so you get a notification

[kwakwaversal]

@kiwicopple sorry to bump this closed issue, but I was interested in:

Guaranteed delivery. If this server dies, you can spin it up to pick up where it left off

I'm assuming that is to do with piggy backing off the WAL in the same way a disrupted synchronous/streaming replicant needs to know where it was up to.

Do you have an example of this working? Does the server need to run outside of docker and save some state of where it currently is in the WAL and can continue from that point using that saved state when it's restarted?

I've done some basic testing locally but I'm unable to get very far with it. I'm running the latest version of the repo built with the bundled Dockerfile and it's missing the verbose logging that is part of the docker hub latest supabase/realtime image so not sure if it's my issue using the @supabase/realtime-js library or the replica set isn't starting where it last stopped.

[kwakwaversal]

After further digging around I found that it's possible for the server to pick up where it left off if you use an undocumented environment variable called SLOT_NAME (posted an update where setting a name causes the server to crash when it's stopped and started - see #22 (comment)).

Guaranteed delivery. If this server dies, you can spin it up to pick up where it left off

It doesn't actually look like guaranteed delivery is currently possible though for two reasons:

• The server (rightly) doesn't keep a log of the events it's received waiting around in a cache for something to consume them.
• The client connecting to the realtime server using @supabase/realtime-js for example, seems to have a high reconnect time by default. In simple tests I did, the client couldn't reconnect and join a channel fast enough to receive the data from the realtime server when I restarted it.

Is there anything I'm missing that would enable me to reduce the amount of missed events?

[kiwicopple]

Hey @kwakwaversal thanks for taking the time to provide well thought-out comments and dig around the code.

• server doesn't keep a log of the events
• js client takes too long to connect

I think both of these will be mitigated by the improvements we started in #33 and will continue in #48.

We don't intend to turn this into a queuing server, but we will create connectors to kafka/rabbit/sqs etc. This way, if you need guaranteed delivery, the server will send events to persistent servers and you can pluck them off the queue. If the realtime server dies, it can pick up where it left off and start sending the events to the "always-alive" & time-ordered queue/database

I think using a client-side listener for guaranteed delivery might not be the best strategy, but for Supabase we are exploring some persistent/offline client library (like RxDB) which will have elements of guaranteed delivery, syncing with a queue/db of some sort.

Can you elaborate a bit on your particular use-case? It will help with our planning and improvements. Feel free to start a new issue if I haven't correctly addressed your question here

One more thing: we are planning the benchmark test as a requirement to move from alpha to beta. We'll publish the results on our blog once we've run the benchmarks.