Comments (9)
From what I understand, at the moment we are using a temporary replication slot, this means that when the client disconnects (e.g. on crash) the replication slot is dropped.
I then tried to look at the pg_replication_slots
table using a named replication slot
postgres=# select * from pg_replication_slots;
slot_name | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn
-----------+----------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------
abcd | pgoutput | logical | 13117 | postgres | f | t | 2222 | | 644 | 0/1696A58 | 0/1696A58
(1 row)
When I insert a record from the example web application we can see the restart_lsn
/confirmed_flush_lsn
values increment.
postgres=# select * from pg_replication_slots;
slot_name | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn
-----------+----------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------
abcd | pgoutput | logical | 13117 | postgres | f | t | 2233 | | 648 | 0/1698080 | 0/1698080
(1 row)
I then disconnect the realtime application and manually insert a couple of rows in the users table, the *_lsn
values don't change since we are not replicating.
postgres=# select * from pg_replication_slots;
slot_name | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn
-----------+----------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------
abcd | pgoutput | logical | 13117 | postgres | f | f | | | 648 | 0/1698080 | 0/1698080
(1 row)
When I restart the application (using xlog/offset = {"0", "0'}
, i.e. what we have now) we start reading the replication log where we left and correctly push the new records to the channels.
postgres=# select * from pg_replication_slots;
slot_name | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn
-----------+----------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------
abcd | pgoutput | logical | 13117 | postgres | f | t | 2270 | | 650 | 0/16980B8 | 0/1699530
(1 row)
So I think the only real change required is to use named replication slots and not a temporary one, then postgres will do the right thing automatically.
from realtime.
I imagine with this one, we are going to need a database of some sort. Ideally it isn't another whole install and can just be done within the server itself. Something like SQLite. I know that elixir also has strong support for mnesia, but that's beyond my knowlege of whether this is a good use case
from realtime.
Following my own advice to post on GH:
We may not even need to store state. The master server keeps track of the subscribers progress for purging. It may even be possible to ask the master PG "give me everything from the last time I acknowleged you"
from realtime.
that's amazing news. If that's the case, then I believe we can just update this line right?
Looks like it from the comments. And according to the comments we can also leave it at {0, 0}
so that the master can decide what to start sending:
So just to confirm, these are the only tasks: (?)
- update
slot: :temporary
to accept a config var - add a config var that reads from ENV (
slot_name = System.get_env("SLOT_NAME") || :temporary
) - On our servers, add a new ENV var (something like
SLOT_NAME=realtime
)
from realtime.
Yes, that's exactly how I plan to implement it!
from realtime.
Nice one @fracek 👍
from realtime.
@kiwicopple Do I need to add this env var to KPS once we include this latest realtime version?
from realtime.
Ah yes please @awalias . The env var is SLOT_NAME
from realtime.
KPS is shipped and frontend/worker will ship with next deployment
from realtime.
Related Issues (20)
- Docker hub `latest` image tag is very old HOT 3
- What version of realtime is hosted Supabase using? HOT 1
- Add region to all Logger metadatas
- Realtime: Self Hosting - Docker Swarm mode HOT 6
- Docker-compose file appears to be broken HOT 40
- Unable to use null filters like 'parent IS NULL' in supabase realtime. HOT 3
- How is postgres binary data serialized?
- > Does supabase have null filters? HOT 1
- Self hosted : failed to connect: ** (Postgrex.Error) ssl not available HOT 1
- Startup error running migrations: table identifier does not refer to an existing ETS table HOT 8
- Erroneous public.test_tenant file from Supabase Realtime Migrations HOT 5
- Supabase Realtime Subscript Fails With Undefined Error (Possibly: `Auth error: tenant 'realtime-dev' not found` HOT 15
- supabase realtime not working HOT 26
- Seeds broken in latest docker image HOT 2
- horizontal scaling for self-hosted realtime servers, for broadcast and presence features HOT 5
- HTTP 431 & Realtime Subscriptions Failing HOT 11
- Realtime event broadcasting through REST calls does not work HOT 8
- Excessive logs being printed to STD OUT/ERR HOT 1
- [Broken] Custom primary keys (nanoids) breaks realtime listeners
- Supabase realtime in own database - error relation "tenants" does not exist HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from realtime.