Is it possible to get the logs for a specific table?

[rusuly]

Is it possible to get the logs for a specific table?

[rusuly]

Ok, just read the docs, looks like replicate-do-db and replicate-do-table parameters must be passed from the library.
But currently the lib doesn't support it. I will try and see if I can easily add support for filtering

[rusuly]

I think it's a bad idea to change binlog_do_db parameter as it affects what is written in the server binlog

[rusuly]

It looks like replicate-do-db and replicate-do-table parameters are used by the internal implementation of MySQL replica server when it replicates binary stream from the master.
So the only way to filter results(as I see it) is to filter events manually based on the DatabaseName and TableName from the TableMapEvent. I wouldn't recommend changing binlog_do_db because it will affect master-slave replication(the database slave will receive filtered events)

[rusuly]

Hi,
The library doesn't filter results.
You need to configure filtering in the master server config file.
The documentation says that there is replicate-do-table parameter although I've never configured filtering.
In my project I switched to PostgreSQL which has logical replication and allows you to select the tables that you want to replicate

[weirdyang]

Thank you!

Just to clarify:
From the documentation, the table map event is triggered automatically, so I will just need to catch that and map then filter based on the table Id or table name. Is that the correct flow?

[rusuly]

Yes, in my docs I wrote this:

A typical transaction has the following structure.

1. GtidEvent if gtid mode is enabled.
2. One or many TableMapEvent events.
   • One or many WriteRowsEvent events.
   • One or many UpdateRowsEvent events.
   • One or many DeleteRowsEvent events.
3. XidEvent indicating commit of the transaction.

So you will receive TableMapEvent event first and then one or many change events associated with it.
You can then skip the events. Just wait for another TableMapEvent.

You can write something like this:

var filterDatabaseName = "MyDatabase";
var filterTableName = "MyTable";
var ignoredTables = new HashSet<long>();

if (binlogEvent is TableMapEvent tableMap)
{
    if(tableMap.DatabaseName != filterDatabaseName || tableMap.TableName != filterTableName)
    {
        ignoredTables.Add(tableMap.TableId);
    }
}
else if (binlogEvent is WriteRowsEvent writeRows)
{
    if(!ignoredTables.Contains(writeRows.TableId))
    {
        await HandleWriteRowsEvent(writeRows);
    }
}
else if (binlogEvent is UpdateRowsEvent updateRows)
{
    if(!ignoredTables.Contains(updateRows.TableId))
    {
        await HandleUpdateRowsEvent(updateRows);
    }
}
else if (binlogEvent is DeleteRowsEvent deleteRows)
{
    if(!ignoredTables.Contains(deleteRows.TableId))
    {
        await HandleDeleteRowsEvent(deleteRows);
    }
}

In PostgreSQL the filtering is done easier. You just create a slot on the master server and tell it to send your code only the changes you want while the database replication continues to work properly from a different event stream.

[weirdyang]

Thanks! I assumed incorrectly it was an indentation mistake. Appreciate the quick response and thanks again!

…

On Fri, 7 May 2021 at 1:49 PM, Ruslan Ulianets ***@***.***> wrote: Yes, in my docs I wrote this: A typical transaction has the following structure. 1. GtidEvent if gtid mode is enabled. 2. One or many TableMapEvent events. - One or many WriteRowsEvent events. - One or many UpdateRowsEvent events. - One or many DeleteRowsEvent events. 3. XidEvent indicating commit of the transaction. So you will receive TableMapEvent event first and then one or many change events associated with it. You can then skip the events. Just wait for another TableMapEvent. You can write something like this: var filterDatabaseName = "MyDatabase";var filterTableName = "MyTable";var ignoredTables = new HashSet<long>(); if (binlogEvent is TableMapEvent tableMap) { if(tableMap.DatabaseName != filterDatabaseName || tableMap.TableName != filterTableName) { ignoredTables.Add(tableMap.TableId); } }else if (binlogEvent is WriteRowsEvent writeRows) { if(!ignoredTables.Contains(writeRows.TableId)) { await HandleWriteRowsEvent(writeRows); } }else if (binlogEvent is UpdateRowsEvent updateRows) { if(!ignoredTables.Contains(updateRows.TableId)) { await HandleUpdateRowsEvent(updateRows); } }else if (binlogEvent is DeleteRowsEvent deleteRows) { if(!ignoredTables.Contains(deleteRows.TableId)) { await HandleDeleteRowsEvent(deleteRows); } } In PostgreSQL the filtering is done easier. You just create a slot on the master server and tell it to send your code only the changes you want while the database replication continues to work properly from a different event stream — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#19 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGDEEVUOZAHXL2EBGSFYH4LTMN5NTANCNFSM44HZGTJA> .