Server not correctly mapping 'numeric' types

[supabase]

Server not correctly mapping 'numeric' types

[kiwicopple]

@dragarcia this could be a good first task for elixir tomorrow.

[dragarcia]

Solution

Now that we forked away from PGOutputDecoder, the following were implemented:

1. server/lib/decoder/oid_database.ex
   • Update the mapping on server/lib/decoder/oid_database.ex to cover all base PgSQL data types. This is based on this
   • Change mapping function to parse through the comprehensive map instead of creating multiple functions to map to each data type. If the data_type_id is found, the corresponding data type name is provided. If not found, then the data type id is returned.
2. server/lib/realtime/replication.ex
   • We noticed that for unique data types, they are being broadcasted under the Type flag. We utilised this opportunity under process_message(%Type{} = msg, state) to create a mapping of unique data type IDs and their corresponding data type name.
   • Under process_message(%Relation{} = msg, state), we take the list of columns processed by the function mentioned above by oid_database.ex and go through each column.
   • If the type of the column is found as a key in the mapping of unique data type IDs and its names, then we update the type to be the corresponding name of the data type id.
   • This now ensures that all data types are properly identified and labeled before reaching the client.

How I tested

1. Run monorepo without realtime
2. Run a local version of realtime, ensuring that the database connection is pointed to the database in monorepo
3. Run HQ locally and start making changes
4. Observe the logs on the local version of realtime.
   • I left a log to show that all columns have been labeled properly
   • This is ran in the very last process_message function that is ran before being sent to the client which is:

process_message(
        %Commit{lsn: commit_lsn, end_lsn: end_lsn},
        %State{transaction: {current_txn_lsn, _txn}} = state
      )