OpenLogReplicator Vs StreamClient

[bersler]

OpenLogReplicator Vs StreamClient

[bersler]

Please compile the code in debug mode and provide the line code that is causing stacktrace, for example:
addr2line -a 0x42fa40 -e ../OpenLogReplicator

[bersler]

Please also provide config file (OpenLogReplicator.json) and full output log when run.

[LoveChina-3000]

log:
[root@izvy205enekylv05zcrhm3z OpenLogReplicator0.9.9]# ./src/OpenLogReplicator
2021-07-06 14:31:15 [INFO] OpenLogReplicator v.0.9.9-beta (C) 2018-2021 by Adam Leszczynski ([email protected]), see LICENSE file for licensing information
2021-07-06 14:31:15 [INFO] adding source: S1
2021-07-06 14:31:15 [INFO] adding target: K1
2021-07-06 14:31:15 [INFO] connecting to Oracle instance of ORCL to //localhost:1521/orcl.test
2021-07-06 14:31:15 [INFO] writer is starting: ZeroMQ:tcp://0.0.0.0:50000
2021-07-06 14:31:15 [INFO] version: Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production, context: orcl, resetlogs: 1071055209, activation: 1598859717, con_id: 0, con_name:
2021-07-06 14:31:15 [INFO] loading character mapping for ZHS16GBK
2021-07-06 14:31:15 [INFO] loading character mapping for AL16UTF16
2021-07-06 14:31:18 [INFO] Oracle Analyzer for ORCL in online mode is starting from scn: 0
2021-07-06 14:31:18 [INFO] starting sequence not found - starting with new batch with seq: 1631
2021-07-06 14:31:18 [INFO] reading schema for ORCL for scn: 205415275
2021-07-06 14:31:18 [INFO] loading character mapping for ZHS16GBK
2021-07-06 14:31:18 [INFO] loading character mapping for AL16UTF16
2021-07-06 14:31:18 [INFO] - found: SYS.CCOL$ (dataobj: 29, obj: 32, cols: 11), part of cluster
2021-07-06 14:31:18 [INFO] - found: SYS.CDEF$ (dataobj: 29, obj: 31, cols: 21), part of cluster
2021-07-06 14:31:18 [INFO] - found: SYS.COL$ (dataobj: 2, obj: 21, cols: 24), part of cluster
2021-07-06 14:31:18 [INFO] - found: SYS.DEFERRED_STG$ (dataobj: 132, obj: 132, cols: 22)
2021-07-06 14:31:18 [INFO] - found: SYS.ECOL$ (dataobj: 123, obj: 123, cols: 3)
2021-07-06 14:31:18 [INFO] - found: SYS.OBJ$ (dataobj: 18, obj: 18, cols: 21)
2021-07-06 14:31:18 [INFO] - found: SYS.SEG$ (dataobj: 8, obj: 14, cols: 20), part of cluster
2021-07-06 14:31:18 [INFO] - found: SYS.TAB$ (dataobj: 2, obj: 4, cols: 37), part of cluster
2021-07-06 14:31:18 [INFO] - found: SYS.TABPART$ (dataobj: 576, obj: 576, cols: 26)
2021-07-06 14:31:18 [INFO] - found: SYS.TABCOMPART$ (dataobj: 598, obj: 598, cols: 35)
2021-07-06 14:31:18 [INFO] - found: SYS.TABSUBPART$ (dataobj: 588, obj: 588, cols: 26)
2021-07-06 14:31:18 [INFO] - found: SYS.USER$ (dataobj: 10, obj: 22, cols: 25), part of cluster
2021-07-06 14:31:18 [INFO] - found: SYNC.MEMBER (dataobj: 84977, obj: 84977, cols: 4)
2021-07-06 14:31:18 [INFO] - found: SYNC.AC85 (dataobj: 79853, obj: 79853, cols: 5)
2021-07-06 14:31:18 [INFO] - found: SYNC.USER1 (dataobj: 84975, obj: 84975, cols: 3)
2021-07-06 14:31:18 [INFO] - found: SYNC.LC30 (dataobj: 84646, obj: 84646, cols: 95)
2021-07-06 14:31:18 [INFO] last confirmed scn: 205425705, starting sequence: 1631, offset: 0
2021-07-06 14:31:18 [INFO] found redo log version: 0x0b200000
2021-07-06 14:31:18 [INFO] processing redo log: group: 3 scn: 205290092 to 0 seq: 1631 path: /data/oracle/oradata/orcl/redo06.log offset: 1024
2021-07-06 14:31:18 [INFO] streaming to client
msgS:
2021-07-06 14:31:18 [INFO] LWN: at: 2 length: 6 chk: 0
ASAN:SIGSEGV

==11669== ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x0000004aae72 sp 0x7f27916a6ae0 bp 0x7f27916a6e10 T1)
AddressSanitizer can not provide additional info.
#0 0x4aae71 (/root/OpenLogReplicator_All/OpenLogReplicator0.9.9/src/OpenLogReplicator+0x4aae71)
#1 0x5d7c71 (/root/OpenLogReplicator_All/OpenLogReplicator0.9.9/src/OpenLogReplicator+0x5d7c71)
#2 0x4d739e (/root/OpenLogReplicator_All/OpenLogReplicator0.9.9/src/OpenLogReplicator+0x4d739e)
#3 0x45a634 (/root/OpenLogReplicator_All/OpenLogReplicator0.9.9/src/OpenLogReplicator+0x45a634)
#4 0x583724 (/root/OpenLogReplicator_All/OpenLogReplicator0.9.9/src/OpenLogReplicator+0x583724)
#5 0x7f28239dda87 (/usr/lib64/libasan.so.0.0.0+0x19a87)
#6 0x7f281e17cea4 (/usr/lib64/libpthread-2.17.so+0x7ea4)
#7 0x7f281ce679fc (/usr/lib64/libc-2.17.so+0xfe9fc)
Thread T1 created by T0 here:
#0 0x7f28239cec2a (/usr/lib64/libasan.so.0.0.0+0xac2a)
#1 0x43f234 (/root/OpenLogReplicator_All/OpenLogReplicator0.9.9/src/OpenLogReplicator+0x43f234)
#2 0x7f281cd8b554 (/usr/lib64/libc-2.17.so+0x22554)
==11669== ABORTING

[LoveChina-3000]

json:

{
"version": "0.9.9-beta",
"sources": [
{
"alias": "S1",
"name": "ORCL",
"reader": {
"type": "online",
"user": "SYNC",
"password": "SYNC",
"server": "//localhost:1521/orcl.test"
},
"format": {
"type": "protobuf"
},
"memory-min-mb": 2048,
"memory-max-mb": 4096,
"tables": [
{"owner": "SYNC", "table": "MEMBER"},
{"owner": "SYNC", "table": "AC85"},
{"owner": "SYNC", "table": "USER1"},
{"owner": "SYNC", "table": "LC30"}
]
}
],
"targets": [
{
"alias": "K1",
"source": "S1",
"writer": {
"type": "zeromq",
"uri": "tcp://0.0.0.0:50000"
}
}
]
}

[bersler]

can you please convert stacktrace to line numbers?
addr2line -a 0x4aae71 -e ./OpenLogReplicator
addr2line -a 0x5d7c71 -e ./OpenLogReplicator
addr2line -a 0x4d739e -e ./OpenLogReplicator
addr2line -a 0x45a634 -e ./OpenLogReplicator
addr2line -a 0x583724 -e ./OpenLogReplicator

[LoveChina-3000]

[root@izvy205enekylv05zcrhm3z OpenLogReplicator0.9.9]# addr2line -a 0x4aae71 -e ./src/OpenLogReplicator
0x00000000004aae71
/root/OpenLogReplicator_All/OpenLogReplicator0.9.9/src/OutputBuffer.h:208
[root@izvy205enekylv05zcrhm3z OpenLogReplicator0.9.9]# addr2line -a 0x5d7c71 -e ./src/OpenLogReplicator
0x00000000005d7c71
addr2line -a 0x4d739e -e ./src/OpenLogReplicator/root/OpenLogReplicator_All/OpenLogReplicator0.9.9/src/OutputBufferProtobuf.cpp:741
[root@izvy205enekylv05zcrhm3z OpenLogReplicator0.9.9]# addr2line -a 0x4d739e -e ./src/OpenLogReplicator
0x00000000004d739e
addr2line -a 0x45a634 -e ./src/OpenLogReplicator/root/OpenLogReplicator_All/OpenLogReplicator0.9.9/src/RedoLog.cpp:1131
[root@izvy205enekylv05zcrhm3z OpenLogReplicator0.9.9]# addr2line -a 0x45a634 -e ./src/OpenLogReplicator
0x000000000045a634
addr2line -a 0x583724 -e ./src/OpenLogReplicator/root/OpenLogReplicator_All/OpenLogReplicator0.9.9/src/OracleAnalyzer.cpp:582
[root@izvy205enekylv05zcrhm3z OpenLogReplicator0.9.9]# addr2line -a 0x583724 -e ./src/OpenLogReplicator
0x0000000000583724
/root/OpenLogReplicator_All/OpenLogReplicator0.9.9/src/Thread.cpp:38

[LoveChina-3000]

The problem looks like that the zeromq mode does not call the (void outputBufferBegin(uint32_t dictId)) function in src/OutputBuffer.h. In my opinion, I can only find this place.

[LoveChina-3000]

I change the OpenLogReplicator.json ,"type": "protobuf" to "type": "json".Now I can run for a while

[bersler]

Right, there is a bug in protobuf code, I am now testing the fix. I will include a fix in next code push to github. Thank you for reporting, @yetengfei .

[bersler]

fixed by last committed code