Giter Site home page Giter Site logo

cardano-foundation / cf-ledger-sync Goto Github PK

View Code? Open in Web Editor NEW
16.0 16.0 1.0 2.12 MB

An open-source Java application designed to facilitate access to indexed Cardano blockchain data

License: Apache License 2.0

Dockerfile 0.05% Java 99.77% Shell 0.05% Smarty 0.13%

cf-ledger-sync's People

Contributors

amb1guousraccoon avatar fabianbormann avatar nemo83 avatar rcmorano avatar satran004 avatar sotatek-huyle3 avatar sotatek-huyle3a avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cf-ledger-sync's Issues

Json error while converting Datum to json

Branch: develop

com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
 at [Source: (String)""; line: 1, column: 0]
        at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
        at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4916)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4818)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3772)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3740)
        at org.cardanofoundation.ledgersync.common.util.JsonUtil.getPrettyJson(JsonUtil.java:37)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.DatumServiceImpl.lambda$handleTransactionOutput$6(DatumServiceImpl.java:141)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.DatumServiceImpl.handleTransactionOutput(DatumServiceImpl.java:131)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.DatumServiceImpl.lambda$handleDatum$0(DatumServiceImpl.java:50)
        at java.base/java.lang.Iterable.forEach(Iterable.java:75)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.DatumServiceImpl.handleDatum(DatumServiceImpl.java:45)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TransactionServiceImpl.prepareAndHandleTxs(TransactionServiceImpl.java:134)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.block.BlockSyncServiceImpl.handleBlockSync(BlockSyncServiceImpl.java:78)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.block.BlockSyncServiceImpl.startBlockSyncing(BlockSyncServiceImpl.java:50)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)

Streamer - Support for multiple serialization formats

The current events in the streamer app are in JSON format, which can be quite large in some cases.

We should consider adding support for different serialization formats for event data, such as AVRO and protobuf, allowing developers to choose their target format.

We need to check whether it's feasible to implement this without making any major changes to the source objects.

[Mainnet] The exception is thrown when handling Tx Input

When synchronizing with the mainnet, the following error occurs, It seems the data may not have been fully saved previously.

2023-10-01 15:45:40.452  INFO 1 --- [ntLoopGroup-4-1] o.c.l.e.listeners.BlockEventListener     : Block  number 6538842, slot_no 46115813, hash a127cae7db121119afca2a6c4ff517c177861fc53bfd36f7368b5aadce091c21
java.lang.IllegalStateException: java.lang.IllegalStateException: Tx in 80aa254b951c57b9cc3cac7fd87da8021839ee47d120c0e7ae228cebe81c6754, index 0, of tx c313dd7f7f084d3a70c692fb08db109d4637261ccb6d8b38edbeacdb42f626d7 has no tx_out before
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
	at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:562)
	at java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:591)
	at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:689)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:765)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TxInServiceImpl.handleTxIns(TxInServiceImpl.java:56)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TransactionServiceImpl.handleTxContents(TransactionServiceImpl.java:193)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TransactionServiceImpl.handleTxs(TransactionServiceImpl.java:146)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TransactionServiceImpl.prepareAndHandleTxs(TransactionServiceImpl.java:135)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.block.BlockSyncServiceImpl.handleBlockSync(BlockSyncServiceImpl.java:78)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.block.BlockSyncServiceImpl.startBlockSyncing(BlockSyncServiceImpl.java:50)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
	at io.micrometer.core.aop.TimedAspect.processWithTimer(TimedAspect.java:221)
	at io.micrometer.core.aop.TimedAspect.perform(TimedAspect.java:196)
	at io.micrometer.core.aop.TimedAspect.timedMethod(TimedAspect.java:188)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:637)
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:627)
	at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:71)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:702)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.block.BlockSyncServiceImpl$$SpringCGLIB$$0.startBlockSyncing(<generated>)
	at org.cardanofoundation.ledgersync.explorerconsumer.listeners.BlockEventListener.handleAggregateBlock(BlockEventListener.java:160)
	at org.cardanofoundation.ledgersync.explorerconsumer.listeners.BlockEventListener.handleBlockEvent(BlockEventListener.java:56)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:702)
	at org.cardanofoundation.ledgersync.explorerconsumer.listeners.BlockEventListener$$SpringCGLIB$$0.handleBlockEvent(<generated>)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:343)
	at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:228)
	at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:165)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:413)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:383)
	at com.bloxbean.cardano.yaci.store.core.service.BlockFetchService.onBlock(BlockFetchService.java:110)
	at com.bloxbean.cardano.yaci.helper.listener.BlockFetchAgentListenerAdapter.blockFound(BlockFetchAgentListenerAdapter.java:62)
	at com.bloxbean.cardano.yaci.core.protocol.blockfetch.BlockfetchAgent.lambda$onReceiveBlocks$4(BlockfetchAgent.java:134)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
	at com.bloxbean.cardano.yaci.core.protocol.blockfetch.BlockfetchAgent.onReceiveBlocks(BlockfetchAgent.java:134)
	at com.bloxbean.cardano.yaci.core.protocol.blockfetch.BlockfetchAgent.processResponse(BlockfetchAgent.java:83)
	at com.bloxbean.cardano.yaci.core.protocol.Agent.receiveResponse(Agent.java:44)
	at com.bloxbean.cardano.yaci.core.network.handlers.MiniProtoClientInboundHandler.channelRead(MiniProtoClientInboundHandler.java:39)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:333)
	at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:349)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.IllegalStateException: Tx in 80aa254b951c57b9cc3cac7fd87da8021839ee47d120c0e7ae228cebe81c6754, index 0, of tx c313dd7f7f084d3a70c692fb08db109d4637261ccb6d8b38edbeacdb42f626d7 has no tx_out before
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TxInServiceImpl.lambda$getTxInSum$4(TxInServiceImpl.java:157)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1707)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.reduce(ReferencePipeline.java:657)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TxInServiceImpl.getTxInSum(TxInServiceImpl.java:164)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TxInServiceImpl.calculateShelleyDeposit(TxInServiceImpl.java:124)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TxInServiceImpl.lambda$handleTxIns$0(TxInServiceImpl.java:69)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
	at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
2023-10-01 15:45:40.659 ERROR 1 --- [ntLoopGroup-4-1] o.c.l.e.listeners.BlockEventListener     : java.lang.IllegalStateException: Tx in 80aa254b951c57b9cc3cac7fd87da8021839ee47d120c0e7ae228cebe81c6754, index 0, of tx c313dd7f7f084d3a70c692fb08db109d4637261ccb6d8b38edbeacdb42f626d7 has no tx_out before

java.lang.ArrayIndexOutOfBoundsException in scheduler

This exception's thrown in scheduler during mainnet syncing, but after restarting, there is no exception

2023-12-12 02:56:24.288 ERROR 1 --- [TaskScheduler-1] o.s.s.s.TaskUtils$LoggingErrorHandler    : Unexpected error occurred in scheduled task

java.lang.ArrayIndexOutOfBoundsException: null
	at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
	at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:542)
	at java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:567)
	at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:670)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:765)
	at org.cardanofoundation.ledgersync.scheduler.service.impl.PoolOfflineDataStoringServiceImpl.saveFailOfflineData(PoolOfflineDataStoringServiceImpl.java:133)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
	at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703)
	at org.cardanofoundation.ledgersync.scheduler.service.impl.PoolOfflineDataStoringServiceImpl$$SpringCGLIB$$0.saveFailOfflineData(<generated>)
	at org.cardanofoundation.ledgersync.scheduler.jobs.PoolOfflineDataScheduler.fetchPoolOffline(PoolOfflineDataScheduler.java:35)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
	at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703)
	at org.cardanofoundation.ledgersync.scheduler.jobs.PoolOfflineDataScheduler$$SpringCGLIB$$0.fetchPoolOffline(<generated>)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84)
	at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 0
	at java.base/java.util.ArrayList.add(ArrayList.java:484)
	at java.base/java.util.ArrayList.add(ArrayList.java:496)
	at org.cardanofoundation.ledgersync.scheduler.service.impl.PoolOfflineDataStoringServiceImpl.lambda$saveFailOfflineData$2(PoolOfflineDataStoringServiceImpl.java:156)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1858)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
	at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

Move Ledger Sync related scheduler code to ledger-sync repo

  1. Create a scheduler module under components folder
  2. Keep only ledger-sync related schedule jobs in this component(For example: pool metadata etc.)
  3. Remove any additional dependencies like Kafka, Redis etc.
  4. Create a top-level module "scheduler-app" with the dependency to scheduler component.
    The scheduler application can be run independently
  5. Alternatively, scheduler component can be added to "ledger-sync" main application to run as a single app.

Update the logic for handling epoch params

The previous implementation loaded genesis data for all epoch params in the Alonzo Era. However, it resulted in incorrect values for some fields of epoch_param table when faced with a new Protocol Param Update.

SQL Error during mainnet sync at random block

The following error occurs occasionally during mainnet sync. After a restart, it continues to sync without any issues. However, it fails again after some time, at a random block.

2023-09-20 20:59:52.676  INFO 475426 --- [ntLoopGroup-4-1] o.c.l.e.listeners.BlockEventListener     : Block  number 5389194, slot_no 22706645, hash c43074a085b04a46a461a66fb52fdcf8d3538174273addc05a2d756a0ba557a1
2023-09-20 20:59:53.223  INFO 475426 --- [ntLoopGroup-4-1] o.c.l.e.listeners.BlockEventListener     : Block  number 5389216, slot_no 22707036, hash 4dc96976391c97332245e16018b61f14a852fd9ab6512f59326b4b227ee9017f
2023-09-20 20:59:53.569  INFO 475426 --- [ntLoopGroup-4-1] c.b.c.y.s.core.service.CursorService     : # of blocks written: 5389400
2023-09-20 20:59:53.569  INFO 475426 --- [ntLoopGroup-4-1] c.b.c.y.s.core.service.CursorService     : Block No: 5389222  , Era: Allegra
org.springframework.dao.InvalidDataAccessApiUsageException: Cannot create binding for parameter reference [org.hibernate.query.sqm.tree.expression.ValueBindJpaCriteriaParameter@33cb0b52] - reference is not a parameter of this query
        at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:371)
        at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:234)
        at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:550)
        at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:61)
        at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:242)
        at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:152)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:702)
        at org.cardanofoundation.ledgersync.explorerconsumer.repository.impl.CustomTxOutRepositoryImpl$$SpringCGLIB$$0.findTxOutsByTxHashInAndTxIndexIn(<generated>)
        at jdk.internal.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.data.repository.core.support.RepositoryMethodInvoker$RepositoryFragmentMethodInvoker.lambda$new$0(RepositoryMethodInvoker.java:288)
        at org.springframework.data.repository.core.support.RepositoryMethodInvoker.doInvoke(RepositoryMethodInvoker.java:136)
        at org.springframework.data.repository.core.support.RepositoryMethodInvoker.invoke(RepositoryMethodInvoker.java:120)
        at org.springframework.data.repository.core.support.RepositoryComposition$RepositoryFragments.invoke(RepositoryComposition.java:516)
        at org.springframework.data.repository.core.support.RepositoryComposition.invoke(RepositoryComposition.java:285)
        at org.springframework.data.repository.core.support.RepositoryFactorySupport$ImplementationMethodExecutionInterceptor.invoke(RepositoryFactorySupport.java:628)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
        at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.doInvoke(QueryExecutorMethodInterceptor.java:168)
        at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.invoke(QueryExecutorMethodInterceptor.java:143)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
        at org.springframework.data.projection.DefaultMethodInvokingMethodInterceptor.invoke(DefaultMethodInvokingMethodInterceptor.java:77)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
        at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
        at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:163)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:218)
        at jdk.proxy2/jdk.proxy2.$Proxy190.findTxOutsByTxHashInAndTxIndexIn(Unknown Source)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TxOutServiceImpl.lambda$getTxOutCanUseByAggregatedTxIns$2(TxOutServiceImpl.java:74)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
        at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
        at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:686)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
        at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:765)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TxOutServiceImpl.getTxOutCanUseByAggregatedTxIns(TxOutServiceImpl.java:73)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TxInServiceImpl.getTxOutFromTxInsMap(TxInServiceImpl.java:102)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TxInServiceImpl.handleTxIns(TxInServiceImpl.java:52)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TransactionServiceImpl.handleTxContents(TransactionServiceImpl.java:193)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TransactionServiceImpl.handleTxs(TransactionServiceImpl.java:146)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TransactionServiceImpl.prepareAndHandleTxs(TransactionServiceImpl.java:135)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.block.BlockSyncServiceImpl.handleBlockSync(BlockSyncServiceImpl.java:78)
        at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.block.BlockSyncServiceImpl.startBlockSyncing(BlockSyncServiceImpl.java:50)
        at jdk.internal.reflect.GeneratedMethodAccessor98.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)

...
...
Caused by: java.lang.IllegalArgumentException: Cannot create binding for parameter reference [org.hibernate.query.sqm.tree.expression.ValueBindJpaCriteriaParameter@33cb0b52] - reference is not a parameter of this query
        at org.hibernate.query.internal.QueryParameterBindingsImpl.makeBinding(QueryParameterBindingsImpl.java:82)
        at org.hibernate.query.internal.QueryParameterBindingsImpl.getBinding(QueryParameterBindingsImpl.java:111)
        at org.hibernate.query.sqm.internal.QuerySqmImpl.<init>(QuerySqmImpl.java:254)
        at org.hibernate.internal.AbstractSharedSessionContract.createCriteriaQuery(AbstractSharedSessionContract.java:1337)
        at org.hibernate.internal.AbstractSharedSessionContract.createQuery(AbstractSharedSessionContract.java:1298)
        at org.hibernate.internal.AbstractSharedSessionContract.createQuery(AbstractSharedSessionContract.java:126)
        at jdk.internal.reflect.GeneratedMethodAccessor87.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:311)
        at jdk.proxy2/jdk.proxy2.$Proxy179.createQuery(Unknown Source)
        at org.cardanofoundation.ledgersync.explorerconsumer.repository.impl.CustomTxOutRepositoryImpl.findTxOutsByTxHashInAndTxIndexIn(CustomTxOutRepositoryImpl.java:45)
        at jdk.internal.reflect.GeneratedMethodAccessor108.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
        at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137)
        ... 145 more
2023-09-20 20:59:53.699 ERROR 475426 --- [ntLoopGroup-4-1] o.c.l.e.listeners.BlockEventListener     : Cannot create binding for parameter reference [org.hibernate.query.sqm.tree.expression.ValueBindJpaCriteriaParameter@33cb0b52] - reference is not a parameter of this query
2023-09-20 20:59:53.725  INFO 475426 --- [ionShutdownHook] c.b.c.yaci.core.network.NodeClient       : Shutdown connection !!!
2023-09-20 20:59:53.725  INFO 475426 --- [ionShutdownHook] c.b.cardano.yaci.core.network.Session    : Disposing the session !!!
2023-09-20 20:59:53.726  INFO 475426 --- [ntLoopGroup-2-1] c.b.c.yaci.core.network.NodeClient       : Connection closed !!!
2023-09-20 20:59:53.731  INFO 475426 --- [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2023-09-20 20:59:53.732  INFO 475426 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : explorer-consumer-pool - Shutdown initiated...
2023-09-20 20:59:53.739  INFO 475426 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : explorer-consumer-pool - Shutdown completed.

                              

[Mainnet] Error occurs when handling epoch param

The following error occurs after restarting service (mainnet sync) when handling epoch param for epoch: 208

2023-10-01 16:13:19.340  INFO 1 --- [ntLoopGroup-4-1] o.c.l.e.listeners.BlockEventListener     : Block  number 4490511, slot_no 4492800, hash aa83acbf5904c0edfe4d79b3689d3d00fcfc553cf360fd2229b98d464c28e9de
2023-10-01 16:13:19.403  INFO 1 --- [ntLoopGroup-4-1] o.c.l.e.s.impl.EpochParamServiceImpl     : Handling epoch param for epoch: 208
org.springframework.dao.DataIntegrityViolationException: not-null property references a null or transient value : org.cardanofoundation.explorer.consumercommon.entity.EpochParam.decentralisation
	at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:286)
	at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:232)
	at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:550)
	at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:61)
	at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:242)
	at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:152)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:163)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:218)
	at jdk.proxy2/jdk.proxy2.$Proxy229.save(Unknown Source)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.EpochParamServiceImpl.handleEpochParam(EpochParamServiceImpl.java:125)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.EpochParamServiceImpl.handleEpochParams(EpochParamServiceImpl.java:68)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:702)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.EpochParamServiceImpl$$SpringCGLIB$$0.handleEpochParams(<generated>)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.block.BlockSyncServiceImpl.handleBlockSync(BlockSyncServiceImpl.java:84)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.block.BlockSyncServiceImpl.startBlockSyncing(BlockSyncServiceImpl.java:50)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
	at io.micrometer.core.aop.TimedAspect.processWithTimer(TimedAspect.java:221)
	at io.micrometer.core.aop.TimedAspect.perform(TimedAspect.java:196)
	at io.micrometer.core.aop.TimedAspect.timedMethod(TimedAspect.java:188)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:637)
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:627)
	at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:71)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:702)
	at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.block.BlockSyncServiceImpl$$SpringCGLIB$$0.startBlockSyncing(<generated>)
	at org.cardanofoundation.ledgersync.explorerconsumer.listeners.BlockEventListener.handleAggregateBlock(BlockEventListener.java:160)
	at org.cardanofoundation.ledgersync.explorerconsumer.listeners.BlockEventListener.handleBlockEvent(BlockEventListener.java:56)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:702)
	at org.cardanofoundation.ledgersync.explorerconsumer.listeners.BlockEventListener$$SpringCGLIB$$0.handleBlockEvent(<generated>)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:343)
	at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:228)
	at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:165)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:413)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:383)
	at com.bloxbean.cardano.yaci.store.core.service.BlockFetchService.onBlock(BlockFetchService.java:110)
	at com.bloxbean.cardano.yaci.helper.listener.BlockFetchAgentListenerAdapter.blockFound(BlockFetchAgentListenerAdapter.java:62)
	at com.bloxbean.cardano.yaci.core.protocol.blockfetch.BlockfetchAgent.lambda$onReceiveBlocks$4(BlockfetchAgent.java:134)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
	at com.bloxbean.cardano.yaci.core.protocol.blockfetch.BlockfetchAgent.onReceiveBlocks(BlockfetchAgent.java:134)
	at com.bloxbean.cardano.yaci.core.protocol.blockfetch.BlockfetchAgent.processResponse(BlockfetchAgent.java:83)
	at com.bloxbean.cardano.yaci.core.protocol.Agent.receiveResponse(Agent.java:44)
	at com.bloxbean.cardano.yaci.core.network.handlers.MiniProtoClientInboundHandler.channelRead(MiniProtoClientInboundHandler.java:39)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:333)
	at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:349)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.hibernate.PropertyValueException: not-null property references a null or transient value : org.cardanofoundation.explorer.consumercommon.entity.EpochParam.decentralisation
	at org.hibernate.engine.internal.Nullability.checkNullability(Nullability.java:111)
	at org.hibernate.engine.internal.Nullability.checkNullability(Nullability.java:55)
	at org.hibernate.action.internal.AbstractEntityInsertAction.nullifyTransientReferencesIfNotAlready(AbstractEntityInsertAction.java:123)
	at org.hibernate.action.internal.AbstractEntityInsertAction.makeEntityManaged(AbstractEntityInsertAction.java:132)
	at org.hibernate.engine.spi.ActionQueue.addResolvedEntityInsertAction(ActionQueue.java:290)
	at org.hibernate.engine.spi.ActionQueue.addInsertAction(ActionQueue.java:264)
	at org.hibernate.engine.spi.ActionQueue.addAction(ActionQueue.java:251)
	at org.hibernate.event.internal.AbstractSaveEventListener.addInsertAction(AbstractSaveEventListener.java:353)
	at org.hibernate.event.internal.AbstractSaveEventListener.performSaveOrReplicate(AbstractSaveEventListener.java:286)
	at org.hibernate.event.internal.AbstractSaveEventListener.performSave(AbstractSaveEventListener.java:192)
	at org.hibernate.event.internal.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:134)
	at org.hibernate.event.internal.DefaultPersistEventListener.entityIsTransient(DefaultPersistEventListener.java:184)
	at org.hibernate.event.internal.DefaultPersistEventListener.onPersist(DefaultPersistEventListener.java:129)
	at org.hibernate.event.internal.DefaultPersistEventListener.onPersist(DefaultPersistEventListener.java:53)
	at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:107)
	at org.hibernate.internal.SessionImpl.firePersist(SessionImpl.java:737)
	at org.hibernate.internal.SessionImpl.persist(SessionImpl.java:721)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:311)
	at jdk.proxy2/jdk.proxy2.$Proxy183.persist(Unknown Source)
	at org.springframework.data.jpa.repository.support.SimpleJpaRepository.save(SimpleJpaRepository.java:613)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.data.repository.core.support.RepositoryMethodInvoker$RepositoryFragmentMethodInvoker.lambda$new$0(RepositoryMethodInvoker.java:288)
	at org.springframework.data.repository.core.support.RepositoryMethodInvoker.doInvoke(RepositoryMethodInvoker.java:136)
	at org.springframework.data.repository.core.support.RepositoryMethodInvoker.invoke(RepositoryMethodInvoker.java:120)
	at org.springframework.data.repository.core.support.RepositoryComposition$RepositoryFragments.invoke(RepositoryComposition.java:516)
	at org.springframework.data.repository.core.support.RepositoryComposition.invoke(RepositoryComposition.java:285)
	at org.springframework.data.repository.core.support.RepositoryFactorySupport$ImplementationMethodExecutionInterceptor.invoke(RepositoryFactorySupport.java:628)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.doInvoke(QueryExecutorMethodInterceptor.java:168)
	at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.invoke(QueryExecutorMethodInterceptor.java:143)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.data.projection.DefaultMethodInvokingMethodInterceptor.invoke(DefaultMethodInvokingMethodInterceptor.java:77)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137)
	... 119 more
2023-10-01 16:13:19.418 ERROR 1 --- [ntLoopGroup-4-1] o.c.l.e.listeners.BlockEventListener     : not-null property references a null or transient value : org.cardanofoundation.explorer.consumercommon.entity.EpochParam.decentralisation

Error while two ledger sync instances pointing to two schemas in same db instance

Steps to reproduce:

  1. Create a new schema cexplorer_one
  2. Configure this schema in application.properties and run Ledger Sync
  3. Stop Ledger Sync
  4. Create another schema cexplorer_two
  5. Confingure this schema in application.properties
  6. Start Ledger Sync

Throws following error

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Migration V1_1_21__create_missing_index.sql failed
--------------------------------------------------
SQL State  : 42704
Error Code : 0
Message    : ERROR: operator class "gin_trgm_ops" does not exist for access method "gin"
Location   : db/migration/consumer/V1_1_21__create_missing_index.sql (/Users/satya/work/cardano-foundation/cf-ledger-sync/application/build/resources/main/db/migration/consumer/V1_1_21__create_missing_index.sql)
Line       : 8
Statement  : CREATE INDEX IF NOT EXISTS name_view_gin_lower ON multi_asset USING gin (lower(name_view) gin_trgm_ops)

	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1770)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:598)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:520)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:325)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:312)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
	at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1158)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:608)
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146)
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:737)
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:439)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1309)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1298)
	at org.cardanofoundation.ledgersync.explorerconsumer.ExplorerConsumerApplication.main(ExplorerConsumerApplication.java:23)
Caused by: org.flywaydb.core.internal.command.DbMigrate$FlywayMigrateException: Migration V1_1_21__create_missing_index.sql failed
--------------------------------------------------
SQL State  : 42704
Error Code : 0
Message    : ERROR: operator class "gin_trgm_ops" does not exist for access method "gin"
Location   : db/migration/consumer/V1_1_21__create_missing_index.sql (/Users/satya/work/cardano-foundation/cf-ledger-sync/application/build/resources/main/db/migration/consumer/V1_1_21__create_missing_index.sql)
Line       : 8
Statement  : CREATE INDEX IF NOT EXISTS name_view_gin_lower ON multi_asset USING gin (lower(name_view) gin_trgm_ops)

	at org.flywaydb.core.internal.command.DbMigrate.doMigrateGroup(DbMigrate.java:384)
	at org.flywaydb.core.internal.command.DbMigrate.lambda$applyMigrations$1(DbMigrate.java:274)
	at org.flywaydb.core.internal.jdbc.TransactionalExecutionTemplate.execute(TransactionalExecutionTemplate.java:55)
	at org.flywaydb.core.internal.command.DbMigrate.applyMigrations(DbMigrate.java:273)
	at org.flywaydb.core.internal.command.DbMigrate.migrateGroup(DbMigrate.java:246)
	at org.flywaydb.core.internal.command.DbMigrate.migrateAll(DbMigrate.java:139)
	at org.flywaydb.core.internal.database.postgresql.PostgreSQLAdvisoryLockTemplate.execute(PostgreSQLAdvisoryLockTemplate.java:73)
	at org.flywaydb.core.internal.database.postgresql.PostgreSQLAdvisoryLockTemplate.lambda$execute$0(PostgreSQLAdvisoryLockTemplate.java:56)
	at org.flywaydb.core.internal.jdbc.TransactionalExecutionTemplate.execute(TransactionalExecutionTemplate.java:55)
	at org.flywaydb.core.internal.database.postgresql.PostgreSQLAdvisoryLockTemplate.execute(PostgreSQLAdvisoryLockTemplate.java:56)
	at org.flywaydb.core.internal.database.postgresql.PostgreSQLConnection.lock(PostgreSQLConnection.java:96)
	at org.flywaydb.core.internal.schemahistory.JdbcTableSchemaHistory.lock(JdbcTableSchemaHistory.java:144)
	at org.flywaydb.core.internal.command.DbMigrate.migrate(DbMigrate.java:96)
	at org.flywaydb.core.Flyway.lambda$migrate$0(Flyway.java:188)
	at org.flywaydb.core.FlywayExecutor.execute(FlywayExecutor.java:196)
	at org.flywaydb.core.Flyway.migrate(Flyway.java:140)
	at org.springframework.boot.autoconfigure.flyway.FlywayMigrationInitializer.afterPropertiesSet(FlywayMigrationInitializer.java:66)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1817)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1766)
	... 18 common frames omitted
Caused by: org.flywaydb.core.internal.sqlscript.FlywaySqlScriptException: Migration V1_1_21__create_missing_index.sql failed
--------------------------------------------------
SQL State  : 42704
Error Code : 0
Message    : ERROR: operator class "gin_trgm_ops" does not exist for access method "gin"
Location   : db/migration/consumer/V1_1_21__create_missing_index.sql (/Users/satya/work/cardano-foundation/cf-ledger-sync/application/build/resources/main/db/migration/consumer/V1_1_21__create_missing_index.sql)
Line       : 8
Statement  : CREATE INDEX IF NOT EXISTS name_view_gin_lower ON multi_asset USING gin (lower(name_view) gin_trgm_ops)

	at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.handleException(DefaultSqlScriptExecutor.java:275)
	at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.executeStatement(DefaultSqlScriptExecutor.java:222)
	at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.execute(DefaultSqlScriptExecutor.java:126)
	at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.executeOnce(SqlMigrationExecutor.java:68)
	at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.lambda$execute$0(SqlMigrationExecutor.java:57)
	at org.flywaydb.core.internal.database.DefaultExecutionStrategy.execute(DefaultExecutionStrategy.java:27)
	at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.execute(SqlMigrationExecutor.java:56)
	at org.flywaydb.core.internal.command.DbMigrate.doMigrateGroup(DbMigrate.java:376)
	... 36 common frames omitted
Caused by: org.postgresql.util.PSQLException: ERROR: operator class "gin_trgm_ops" does not exist for access method "gin"
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2713)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2401)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:368)
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:498)
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:415)
	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:335)
	at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:321)
	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:297)
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:292)
	at org.flywaydb.core.internal.jdbc.JdbcTemplate.executeStatement(JdbcTemplate.java:201)
	at org.flywaydb.core.internal.sqlscript.ParsedSqlStatement.execute(ParsedSqlStatement.java:95)
	at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.executeStatement(DefaultSqlScriptExecutor.java:210)
	... 42 common frames omitted

Use postgresql testcontainer to generate source code

To fix #18 , we try using JOOQ. When using jOOQ, the recommended approach is to use jOOQ code generation. To generate jOOQ code, we need a connection to the database. Testcontainer could be a solution for creating a database instance, which would support the process of generating jOOQ code.

Error occurs when handling redeemer data in preprod sync

While syncing preprod, branch: develop, this error occurs:

Caused by: java.lang.IllegalStateException: org.hibernate.TransientPropertyValueException: Not-null property references a transient value - transient instance must be saved before current operation : org.cardanofoundation.explorer.consumercommon.entity.Redeemer.redeemerData -> org.cardanofoundation.explorer.consumercommon.entity.RedeemerData
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:152)
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:162)
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:168)
at org.hibernate.internal.SessionImpl.firePersist(SessionImpl.java:772)
at org.hibernate.internal.SessionImpl.persist(SessionImpl.java:739)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:311)
at jdk.proxy2/jdk.proxy2.$Proxy175.persist(Unknown Source)
at org.springframework.data.jpa.repository.support.SimpleJpaRepository.save(SimpleJpaRepository.java:618)
at org.springframework.data.jpa.repository.support.SimpleJpaRepository.saveAll(SimpleJpaRepository.java:644)
at org.springframework.data.jpa.repository.support.SimpleJpaRepository.saveAll(SimpleJpaRepository.java:95)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.springframework.data.repository.core.support.RepositoryMethodInvoker$RepositoryFragmentMethodInvoker.lambda$new$0(RepositoryMethodInvoker.java:288)
at org.springframework.data.repository.core.support.RepositoryMethodInvoker.doInvoke(RepositoryMethodInvoker.java:136)
at org.springframework.data.repository.core.support.RepositoryMethodInvoker.invoke(RepositoryMethodInvoker.java:120)
at org.springframework.data.repository.core.support.RepositoryComposition$RepositoryFragments.invoke(RepositoryComposition.java:516)
at org.springframework.data.repository.core.support.RepositoryComposition.invoke(RepositoryComposition.java:285)
at org.springframework.data.repository.core.support.RepositoryFactorySupport$ImplementationMethodExecutionInterceptor.invoke(RepositoryFactorySupport.java:628)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.doInvoke(QueryExecutorMethodInterceptor.java:168)
at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.invoke(QueryExecutorMethodInterceptor.java:143)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.data.projection.DefaultMethodInvokingMethodInterceptor.invoke(DefaultMethodInvokingMethodInterceptor.java:72)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137)
... 112 more
Caused by: org.hibernate.TransientPropertyValueException: Not-null property references a transient value - transient instance must be saved before current operation : org.cardanofoundation.explorer.consumercommon.entity.Redeemer.redeemerData -> org.cardanofoundation.explorer.consumercommon.entity.RedeemerData
at org.hibernate.action.internal.UnresolvedEntityInsertActions.checkNoUnresolvedActionsAfterOperation(UnresolvedEntityInsertActions.java:123)
at org.hibernate.engine.spi.ActionQueue.checkNoUnresolvedActionsAfterOperation(ActionQueue.java:462)
at org.hibernate.internal.SessionImpl.checkNoUnresolvedActionsAfterOperation(SessionImpl.java:595)
at org.hibernate.internal.SessionImpl.firePersist(SessionImpl.java:769)
... 139 more

JOOQ Sync error in preview

It appears there are two errors now, but I was hoping to replicate the first error (which happened 3 tiimes in preview) but could not as I encountered the new error below. The first error happened consistently on block number 110449 related to cbor parse error. Unfortunately, the error below prevented me from capturing the cbor parse error.

For some background, I am runnig Postgres 15 on Ubuntu 23.0

Distributor ID: Ubuntu
Description: Ubuntu 23.04
Release: 23.04
Codename: lunar

The cardano node version is:

cardano-node 8.1.2 - linux-x86_64 - ghc-8.10
git rev d2d90b48c5577b4412d5c9c9968b55f8ab4b9767

Is there a recommended cardano-node version to use with a given version of cf-ledger-sync? No mention of this is made in the README.

I encountered the following error while attempting to sync Preview:

2023-11-12 18:48:18.254 INFO 901515 --- [ntLoopGroup-5-4] o.c.l.e.listeners.BlockEventListener : Block number 99361, slot_no 2116933, hash 152a8f68ccc0a30b800b92424830753ef56c3ce84c95d1a0f27ac6904f6b4e5d
2023-11-12 18:48:18.844 INFO 901515 --- [ntLoopGroup-5-4] o.c.l.e.listeners.BlockEventListener : Block number 99369, slot_no 2117085, hash af7eeab291e462d2e43d00c75bda0dc591899787133a8de8a1147111804fba6a
org.springframework.jdbc.BadSqlGrammarException: jOOQ; bad SQL grammar [select address_token_balance.id, address_token_balance.address_id, address_token_balance.balance, address_token_balance.ident, address_token_balance.stake_address_id, address.id, address.address, address.address_has_script, address.balance, address.tx_count, address.stake_address_id, address.verified_contract, multi_asset.id, multi_asset.fingerprint, multi_asset.name, multi_asset.policy, multi_asset.supply, multi_asset.tx_count, multi_asset.time, multi_asset.total_volume, multi_asset.name_view from address_token_balance join address on address_token_balance.address_id = address.id join multi_asset on address_token_balance.ident = multi_asset.id where ((address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?))]
at org.jooq_3.18.2.POSTGRES.debug(Unknown Source)
at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:246)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73)
at org.springframework.boot.autoconfigure.jooq.JooqExceptionTranslator.translate(JooqExceptionTranslator.java:94)
at org.springframework.boot.autoconfigure.jooq.JooqExceptionTranslator.handle(JooqExceptionTranslator.java:81)
at org.springframework.boot.autoconfigure.jooq.JooqExceptionTranslator.exception(JooqExceptionTranslator.java:55)
at org.jooq.impl.ExecuteListeners.exception(ExecuteListeners.java:304)
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:372)
at org.jooq.impl.AbstractResultQuery.fetch(AbstractResultQuery.java:290)
at org.jooq.impl.SelectImpl.fetch(SelectImpl.java:2838)
at org.cardanofoundation.ledgersync.explorerconsumer.repository.custom.CustomAddressTokenBalanceRepository.findAllByAddressFingerprintPairIn(CustomAddressTokenBalanceRepository.java:55)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703)
at org.cardanofoundation.ledgersync.explorerconsumer.repository.custom.CustomAddressTokenBalanceRepository$$SpringCGLIB$$0.findAllByAddressFingerprintPairIn()
at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.AddressBalanceServiceImpl.lambda$getAddressTokenBalanceMap$11(AddressBalanceServiceImpl.java:376)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
Caused by: org.postgresql.util.PSQLException: ERROR: relation "address_token_balance" does not exist
Position: 505
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2713)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2401)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:368)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:498)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:415)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:190)
at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:177)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java)
at org.jooq.tools.jdbc.DefaultPreparedStatement.execute(DefaultPreparedStatement.java:219)
at org.jooq.impl.Tools.executeStatementAndGetFirstResultSet(Tools.java:4754)
at org.jooq.impl.AbstractResultQuery.execute(AbstractResultQuery.java:236)
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:357)
... 25 more
2023-11-12 18:48:19.689 ERROR 901515 --- [ntLoopGroup-5-4] o.c.l.e.listeners.BlockEventListener : jOOQ; bad SQL grammar [select address_token_balance.id, address_token_balance.address_id, address_token_balance.balance, address_token_balance.ident, address_token_balance.stake_address_id, address.id, address.address, address.address_has_script, address.balance, address.tx_count, address.stake_address_id, address.verified_contract, multi_asset.id, multi_asset.fingerprint, multi_asset.name, multi_asset.policy, multi_asset.supply, multi_asset.tx_count, multi_asset.time, multi_asset.total_volume, multi_asset.name_view from address_token_balance join address on address_token_balance.address_id = address.id join multi_asset on address_token_balance.ident = multi_asset.id where ((address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?) or (address.address = ? and multi_asset.fingerprint = ?))]
2023-11-12 18:48:19.712 INFO 901515 --- [ionShutdownHook] c.b.c.yaci.core.network.NodeClient : Shutdown connection !!!
2023-11-12 18:48:19.712 INFO 901515 --- [ionShutdownHook] c.b.cardano.yaci.core.network.Session : Disposing the session !!!
2023-11-12 18:48:19.712 INFO 901515 --- [ntLoopGroup-3-1] c.b.c.yaci.core.network.NodeClient : Connection closed !!!
2023-11-12 18:48:19.739 INFO 901515 --- [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2023-11-12 18:48:19.743 INFO 901515 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource : explorer-consumer-pool - Shutdown initiated...
2023-11-12 18:48:19.748 INFO 901515 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource : explorer-consumer-pool - Shutdown completed.

SQL error during mainnet sync

ERROR: value too long for type character varying(64)

2023-09-15 13:06:22.080  INFO 76719 --- [ntLoopGroup-7-1] o.c.l.e.listeners.BlockEventListener     : Block already exists. Skipping block no 157182, hash 08ecdb54a80c81073ddee790f3cf4f8d4ff4422eb369b41cd5e0e18c13de6bbe
2023-09-15 13:06:22.109  INFO 76719 --- [ntLoopGroup-7-1] o.c.l.e.listeners.BlockEventListener     : Block  number 5561508, slot_no 26250031, hash 52fd6283bc5a1e6b78707a1534abfb3aa2499ca2c108741d5db6c8c7d99353ae
2023-09-15 13:06:22.300  INFO 76719 --- [ntLoopGroup-7-1] o.h.e.j.b.internal.AbstractBatchImpl     : HHH000010: On release of batch it still contained JDBC statements
2023-09-15 13:06:22.300 ERROR 76719 --- [ntLoopGroup-7-1] o.h.e.jdbc.batch.internal.BatchingBatch  : HHH000315: Exception executing batch [java.sql.BatchUpdateException: Batch entry 0 insert into cexplorer.param_proposal (coins_per_utxo_size, collateral_percent, cost_model_id, decentralisation, entropy, epoch_no, influence, key, key_deposit, max_bh_size, max_block_ex_mem, max_block_ex_steps, max_block_size, max_collateral_inputs, max_epoch, max_tx_ex_mem, max_tx_ex_steps, max_tx_size, max_val_size, min_fee_a, min_fee_b, min_pool_cost, min_utxo_value, monetary_expand_rate, optimal_pool_count, pool_deposit, price_mem, price_step, protocol_major, protocol_minor, registered_tx_id, treasury_growth_rate, id) values (NULL, NULL, NULL, NULL, '[1, d982e06fd33e7440b43cefad529b7ecafbaa255e38178ad4189a37e4ce9bf1fa]', 258, NULL, '268cfc0b89e910ead22e0ade91493d8212f53f3e2164b2e4bef0819b', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 5875270, NULL, 46) was aborted: ERROR: value too long for type character varying(64)  Call getNextException to see other errors in the batch.], SQL: insert into cexplorer.param_proposal (coins_per_utxo_size, collateral_percent, cost_model_id, decentralisation, entropy, epoch_no, influence, key, key_deposit, max_bh_size, max_block_ex_mem, max_block_ex_steps, max_block_size, max_collateral_inputs, max_epoch, max_tx_ex_mem, max_tx_ex_steps, max_tx_size, max_val_size, min_fee_a, min_fee_b, min_pool_cost, min_utxo_value, monetary_expand_rate, optimal_pool_count, pool_deposit, price_mem, price_step, protocol_major, protocol_minor, registered_tx_id, treasury_growth_rate, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
2023-09-15 13:06:22.300  WARN 76719 --- [ntLoopGroup-7-1] o.h.engine.jdbc.spi.SqlExceptionHelper   : SQL Error: 0, SQLState: 22001
2023-09-15 13:06:22.301 ERROR 76719 --- [ntLoopGroup-7-1] o.h.engine.jdbc.spi.SqlExceptionHelper   : Batch entry 0 insert into cexplorer.param_proposal (coins_per_utxo_size, collateral_percent, cost_model_id, decentralisation, entropy, epoch_no, influence, key, key_deposit, max_bh_size, max_block_ex_mem, max_block_ex_steps, max_block_size, max_collateral_inputs, max_epoch, max_tx_ex_mem, max_tx_ex_steps, max_tx_size, max_val_size, min_fee_a, min_fee_b, min_pool_cost, min_utxo_value, monetary_expand_rate, optimal_pool_count, pool_deposit, price_mem, price_step, protocol_major, protocol_minor, registered_tx_id, treasury_growth_rate, id) values (NULL, NULL, NULL, NULL, '[1, d982e06fd33e7440b43cefad529b7ecafbaa255e38178ad4189a37e4ce9bf1fa]', 258, NULL, '268cfc0b89e910ead22e0ade91493d8212f53f3e2164b2e4bef0819b', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 5875270, NULL, 46) was aborted: ERROR: value too long for type character varying(64)  Call getNextException to see other errors in the batch.
2023-09-15 13:06:22.301 ERROR 76719 --- [ntLoopGroup-7-1] o.h.engine.jdbc.spi.SqlExceptionHelper   : ERROR: value too long for type character varying(64)
2023-09-15 13:06:22.308 ERROR 76719 --- [ntLoopGroup-7-1] o.c.l.e.listeners.BlockEventListener     : could not execute batch; SQL [insert into cexplorer.param_proposal (coins_per_utxo_size, collateral_percent, cost_model_id, decentralisation, entropy, epoch_no, influence, key, key_deposit, max_bh_size, max_block_ex_mem, max_block_ex_steps, max_block_size, max_collateral_inputs, max_epoch, max_tx_ex_mem, max_tx_ex_steps, max_tx_size, max_val_size, min_fee_a, min_fee_b, min_pool_cost, min_utxo_value, monetary_expand_rate, optimal_pool_count, pool_deposit, price_mem, price_step, protocol_major, protocol_minor, registered_tx_id, treasury_growth_rate, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]

cbor de-serialization error

@Sotatek-HuyLe3a, here is the cbor de-serialization error i mentioned in the first issue. The env is the same. I encountered this after checking out and running issue_62 branch like you recommended. Network is preview as before:

2023-11-13 18:50:51.028 INFO 26183 --- [ntLoopGroup-5-2] c.b.c.y.s.core.service.CursorService : # of blocks written: 11000
2023-11-13 18:50:51.028 INFO 26183 --- [ntLoopGroup-5-2] c.b.c.y.s.core.service.CursorService : Block No: 110370 , Era: Babbage
2023-11-13 18:50:51.033 INFO 26183 --- [ntLoopGroup-5-2] o.c.l.e.listeners.BlockEventListener : Block number 110371, slot_no 2338497, hash 663e1a1987904650d863b706ca509a7920ab248b22c8f5c990bf686bf63b9929
2023-11-13 18:50:51.545 INFO 26183 --- [ntLoopGroup-5-2] o.c.l.e.listeners.BlockEventListener : Block number 110377, slot_no 2338641, hash 741784040b5886b8b2ed489f8e90ec7674ea2473d84079b53443fbe81b734f03
2023-11-13 18:50:52.141 INFO 26183 --- [ntLoopGroup-5-2] o.c.l.e.listeners.BlockEventListener : Block number 110385, slot_no 2338831, hash 380be731e556bfa3c25798c81301cf904c71126d272a98a02938541f66917346
2023-11-13 18:50:52.704 INFO 26183 --- [ntLoopGroup-5-2] o.c.l.e.listeners.BlockEventListener : Block number 110393, slot_no 2339135, hash a8216d463b1e579099bfd47802f0ffa099dd6d777b4cf1307722edaa401bac89
2023-11-13 18:50:53.292 INFO 26183 --- [ntLoopGroup-5-2] o.c.l.e.listeners.BlockEventListener : Block number 110401, slot_no 2339301, hash 785459563cf44c62cde00e814b7f13b69e7cb0a7013f5cf764df1b610019f70a
2023-11-13 18:50:53.855 INFO 26183 --- [ntLoopGroup-5-2] o.c.l.e.listeners.BlockEventListener : Block number 110409, slot_no 2339561, hash c05c4393081a80429bd0592d9232cc31432d38a16d1f9414e311a69132a0208c
2023-11-13 18:50:54.398 INFO 26183 --- [ntLoopGroup-5-2] o.c.l.e.listeners.BlockEventListener : Block number 110416, slot_no 2339692, hash 245f64cc76a98c37408fe9b470c65608de01bd7ba2770fc7433f0c0f7523af77
2023-11-13 18:50:54.907 INFO 26183 --- [ntLoopGroup-5-2] o.c.l.e.listeners.BlockEventListener : Block number 110420, slot_no 2339839, hash 008ce12593069433c23ea1074ab859f1ce5c25a5552652831200c9a5972c04a6
2023-11-13 18:50:55.424 INFO 26183 --- [ntLoopGroup-5-2] o.c.l.e.listeners.BlockEventListener : Block number 110429, slot_no 2340012, hash d51d93f6de8570a40e72cb511d2e92c65d31f7e3547988653131aa135364e5fe
2023-11-13 18:50:55.962 INFO 26183 --- [ntLoopGroup-5-2] o.c.l.e.listeners.BlockEventListener : Block number 110445, slot_no 2340203, hash 324504cc8e34d908b598fca2fecc903ce194bc736c4ea33bc94db3c8130f2f2f
com.bloxbean.cardano.client.exception.CborRuntimeException: Cbor de-serialization error
at com.bloxbean.cardano.client.common.cbor.CborSerializationUtil.deserialize(CborSerializationUtil.java:187)
at org.cardanofoundation.ledgersync.explorerconsumer.util.RedeemerWrapper.(RedeemerWrapper.java:18)
at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.RedeemerServiceImpl.lambda$handleRedeemers$3(RedeemerServiceImpl.java:66)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708)
at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.RedeemerServiceImpl.handleRedeemers(RedeemerServiceImpl.java:67)
at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TransactionServiceImpl.handleTxContents(TransactionServiceImpl.java:190)
at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TransactionServiceImpl.handleTxs(TransactionServiceImpl.java:146)
at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.TransactionServiceImpl.prepareAndHandleTxs(TransactionServiceImpl.java:135)
at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.block.BlockSyncServiceImpl.handleBlockSync(BlockSyncServiceImpl.java:78)
at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.block.BlockSyncServiceImpl.startBlockSyncing(BlockSyncServiceImpl.java:50)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
at io.micrometer.core.aop.TimedAspect.processWithTimer(TimedAspect.java:227)
at io.micrometer.core.aop.TimedAspect.perform(TimedAspect.java:202)
at io.micrometer.core.aop.TimedAspect.timedMethod(TimedAspect.java:194)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:637)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:627)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:71)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703)
at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.block.BlockSyncServiceImpl$$SpringCGLIB$$0.startBlockSyncing()
at org.cardanofoundation.ledgersync.explorerconsumer.listeners.BlockEventListener.handleAggregateBlock(BlockEventListener.java:159)
at org.cardanofoundation.ledgersync.explorerconsumer.listeners.BlockEventListener.handleBlockEvent(BlockEventListener.java:56)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703)
at org.cardanofoundation.ledgersync.explorerconsumer.listeners.BlockEventListener$$SpringCGLIB$$0.handleBlockEvent()
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:348)
at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:233)
at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:165)
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:174)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:167)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:145)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:437)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:383)
at com.bloxbean.cardano.yaci.store.core.service.BlockFetchService.onBlock(BlockFetchService.java:110)
at com.bloxbean.cardano.yaci.helper.listener.BlockFetchAgentListenerAdapter.blockFound(BlockFetchAgentListenerAdapter.java:62)
at com.bloxbean.cardano.yaci.core.protocol.blockfetch.BlockfetchAgent.lambda$onReceiveBlocks$4(BlockfetchAgent.java:134)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708)
at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
at com.bloxbean.cardano.yaci.core.protocol.blockfetch.BlockfetchAgent.onReceiveBlocks(BlockfetchAgent.java:134)
at com.bloxbean.cardano.yaci.core.protocol.blockfetch.BlockfetchAgent.processResponse(BlockfetchAgent.java:83)
at com.bloxbean.cardano.yaci.core.protocol.Agent.receiveResponse(Agent.java:44)
at com.bloxbean.cardano.yaci.core.network.handlers.MiniProtoClientInboundHandler.channelRead(MiniProtoClientInboundHandler.java:39)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:333)
at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:349)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: co.nstant.in.cbor.CborException: Unexpected end of stream
at co.nstant.in.cbor.decoder.MapDecoder.decodeInfinitiveLength(MapDecoder.java:38)
at co.nstant.in.cbor.decoder.MapDecoder.decode(MapDecoder.java:21)
at co.nstant.in.cbor.CborDecoder.decodeNext(CborDecoder.java:134)
at co.nstant.in.cbor.decoder.ArrayDecoder.decodeInfinitiveLength(ArrayDecoder.java:33)
at co.nstant.in.cbor.decoder.ArrayDecoder.decode(ArrayDecoder.java:21)
at co.nstant.in.cbor.CborDecoder.decodeNext(CborDecoder.java:130)
at co.nstant.in.cbor.CborDecoder.decodeNext(CborDecoder.java:145)
at co.nstant.in.cbor.decoder.ArrayDecoder.decodeFixedLength(ArrayDecoder.java:50)
at co.nstant.in.cbor.decoder.ArrayDecoder.decode(ArrayDecoder.java:23)
at co.nstant.in.cbor.CborDecoder.decodeNext(CborDecoder.java:130)
at co.nstant.in.cbor.CborDecoder.decode(CborDecoder.java:90)
at co.nstant.in.cbor.CborDecoder.decode(CborDecoder.java:78)
at com.bloxbean.cardano.client.common.cbor.CborSerializationUtil.deserialize(CborSerializationUtil.java:185)
... 102 more
2023-11-13 18:50:56.190 ERROR 26183 --- [ntLoopGroup-5-2] o.c.l.e.listeners.BlockEventListener : Cbor de-serialization error
2023-11-13 18:50:56.207 INFO 26183 --- [ionShutdownHook] c.b.c.yaci.core.network.NodeClient : Shutdown connection !!!
2023-11-13 18:50:56.207 INFO 26183 --- [ionShutdownHook] c.b.cardano.yaci.core.network.Session : Disposing the session !!!
2023-11-13 18:50:56.208 INFO 26183 --- [ntLoopGroup-3-1] c.b.c.yaci.core.network.NodeClient : Connection closed !!!
2023-11-13 18:50:56.232 INFO 26183 --- [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2023-11-13 18:50:56.236 INFO 26183 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource : explorer-consumer-pool - Shutdown initiated...
2023-11-13 18:50:56.241 INFO 26183 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource : explorer-consumer-pool - Shutdown completed.

Thanks

Startup Error after updating from repo

Hello, I wasn't sure where to put this as I am sure you have been asked this before. I just updated my local branch from main the sync errored out because of a new column in one of the tables. Is there a recommeded way to make sure the db is up to date? I assumed this was handled by the sync process somehow. In this case, I thought some migration will run to bring the db up to date. Am i missing something? I got the following error below:

java.nio.channels.ClosedChannelException: null
at io.netty.handler.ssl.SslHandler.channelInactive(SslHandler.java:1150)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:305)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:301)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)
at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)
at io.netty.channel.AbstractChannel$AbstractUnsafe$7.run(AbstractChannel.java:813)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:413)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Suppressed: io.netty.handler.ssl.StacklessSSLHandshakeException: Connection closed while SSL/TLS handshake was in progress
at io.netty.handler.ssl.SslHandler.channelInactive(Unknown Source)

2023-12-02 10:56:51.955 INFO 134956 --- [ntLoopGroup-5-1] o.j.i.D.logVersionSupport : Version : Database version is supported by dialect POSTGRES: 15.4 (Ubuntu 15.4-2.pgdg22.04+1)
2023-12-02 10:56:54.740 INFO 134956 --- [ntLoopGroup-5-1] org.hibernate.orm.jdbc.batch : HHH100503: On release of batch it still contained JDBC statements
2023-12-02 10:56:54.742 ERROR 134956 --- [ntLoopGroup-5-1] org.hibernate.orm.jdbc.batch : HHH100501: Exception executing batch [java.sql.BatchUpdateException: Batch entry 0 insert into cexplorer.unconsume_tx_in (redeemer_id,tx_in_id,tx_out_id,tx_out_index,id) values (NULL,3665246,3665241,0,1502011) was aborted: ERROR: column "redeemer_id" of relation "unconsume_tx_in" does not exist
Position: 40 Call getNextException to see other errors in the batch.], SQL: insert into cexplorer.unconsume_tx_in (redeemer_id,tx_in_id,tx_out_id,tx_out_index,id) values (?,?,?,?,?)
2023-12-02 10:56:54.744 WARN 134956 --- [ntLoopGroup-5-1] o.h.engine.jdbc.spi.SqlExceptionHelper : SQL Error: 0, SQLState: 42703
2023-12-02 10:56:54.744 ERROR 134956 --- [ntLoopGroup-5-1] o.h.engine.jdbc.spi.SqlExceptionHelper : Batch entry 0 insert into cexplorer.unconsume_tx_in (redeemer_id,tx_in_id,tx_out_id,tx_out_index,id) values (NULL,3665246,3665241,0,1502011) was aborted: ERROR: column "redeemer_id" of relation "unconsume_tx_in" does not exist
Position: 40 Call getNextException to see other errors in the batch.
2023-12-02 10:56:54.744 ERROR 134956 --- [ntLoopGroup-5-1] o.h.engine.jdbc.spi.SqlExceptionHelper : ERROR: column "redeemer_id" of relation "unconsume_tx_in" does not exist
Position: 40
org.springframework.dao.InvalidDataAccessResourceUsageException: could not execute batch [Batch entry 0 insert into cexplorer.unconsume_tx_in (redeemer_id,tx_in_id,tx_out_id,tx_out_index,id) values (NULL,3665246,3665241,0,1502011) was aborted: ERROR: column "redeemer_id" of relation "unconsume_tx_in" does not exist
Position: 40 Call getNextException to see other errors in the batch.] [insert into cexplorer.unconsume_tx_in (redeemer_id,tx_in_id,tx_out_id,tx_out_index,id) values (?,?,?,?,?)]; SQL [insert into cexplorer.unconsume_tx_in (redeemer_id,tx_in_id,tx_out_id,tx_out_index,id) values (?,?,?,?,?)]
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:256)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:229)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:550)
at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:61)
at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:243)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:152)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:135)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:244)
at jdk.proxy2/jdk.proxy2.$Proxy230.findEpochNoByMaxSlotAndEpochNoMoreThanLastEpochParam(Unknown Source)
at org.cardanofoundation.ledgersync.explorerconsumer.service.impl.EpochParamServiceImpl.handleEpochParams(EpochParamServiceImpl.java:

Thanks

Verify if a protocol param proposal is signed by at least 5 genesis keys before moving to epoch_param

Currently, LS captures all protocol parameter updates in the param_proposal table. However, a protocol parameter update is only confirmed when it is signed by at least 5 out of 7 genesis delegation keys. A protocol update can only be converted to an epoch parameter (epoch_param) in the specified epoch if it is confirmed.

Therefore, this check should be added during the epoch transition. (EpochParamServiceImpl.java)

Remove remaining network specific hardcoded values

There is still some network-specific code in ledger-sync, primarily for genesis data such as epoch length, Shelley start time, etc. Most of the remaining values can be sourced from the genesis file.

Check ConsumerConstant.java

private static final Map<Integer, Integer> shelleyEpochLength = Map.ofEntries(
            Map.entry(Constant.MAINNET, FIVE_DAYS),
            Map.entry(Constant.TESTNET, FIVE_DAYS),
            Map.entry(Constant.PREPROD_TESTNET, FIVE_DAYS),
            Map.entry(Constant.PREVIEW_TESTNET, ONE_DAYS)
    );
    
 private static final Map<Integer, Long> mByronKnownTime = Map.ofEntries(
            Map.entry(Constant.MAINNET, MAINNET_BYRON_KNOWN_TIME),
            Map.entry(Constant.TESTNET, TESTNET_BYRON_KNOWN_TIME),
            Map.entry(Constant.PREPROD_TESTNET, PREPROD_BYRON_KNOWN_TIME)
    );
    

Config jOOQ schema globally

We are currently using setSchema({schema_name]).execute() to set the schema dynamically (SET SEARCH_PATH in PostgreSql). This approach, however, is not reliable and has led to occasional errors, specifically when the schema setting fails, resulting in next queries encountering issues.

It would be safer and more stable to render the schema for each query using withRenderSchema and withRenderMapping provided by jOOQ, and config this globally.

Frequent disconnection error for Sanchonet

Recently, it has been observed that there are frequent disconnections during the sync of Sanchonet.
Though it automatically reconnects, we need to find the root cause.

Taking a significant amount of time to resume consuming a new block after reconnecting to the node

During mainnet syncing, after reconnecting to the node, sometimes it takes a significant amount of time to resume consuming a new block.

2023-12-13 02:55:10.338  INFO 1 --- [ntLoopGroup-5-4] c.b.c.yaci.core.network.NodeClient       : Connection closed !!!
2023-12-13 02:55:10.338  INFO 1 --- [ntLoopGroup-5-4] c.b.cardano.yaci.core.network.Session    : Disposing the session !!!
2023-12-13 02:55:10.338  WARN 1 --- [ntLoopGroup-5-4] c.b.c.yaci.core.network.NodeClient       : Trying to reconnect !!!
2023-12-13 02:55:10.555  INFO 1 --- [ntLoopGroup-5-5] c.b.cardano.yaci.core.network.Session    : Connection established
2023-12-13 02:55:10.555  INFO 1 --- [ntLoopGroup-5-5] c.b.c.yaci.core.network.NodeClient       : Connected !!!
2023-12-13 02:55:11.407  INFO 1 --- [ntLoopGroup-5-5] c.b.c.y.c.p.handshake.HandshakeAgent     : Handshake Ok!!! AcceptVersion(versionNumber=11, versionData=N2NVersionData(initiatorAndResponderDiffusionMode=false, peerSharing=0, query=false))

Ledger sync speed

Testing ledger sync as backend for Gamechanger wallet. Syncing Preprod was pretty fast. But now syncing mainnet takes a long time. Sync started at 13 november at 16:00. Now at slot 56.882.115.
Is this the way it should go? Or is there a snapshot like for db-sync?

System
memory 128GiB System memory
processor AMD Ryzen 9 7900X3D 12-Core Processor
storage Corsair MP700 2TB

Verify data correctness

  • Decide data reconciliation strategy to verify ledger sync data with another source
  • Create scripts/tools to automate this process (If possible)

Genesis file parsing fails for alonzo-genesis.json for Sanchonet

Default Alonzo genesis file (Sanchonet): https://book.world.dev.cardano.org/environments/sanchonet/alonzo-genesis.json

The parsing fails for the following reasons:

  1. For the Plutus V1 cost model, the existing code expects a map of operations and costs. However, the genesis file mentioned above contains an array of costs.

It would be good to handle both types in the parsing logic.

Sanchonet:

"costModels": {
        "PlutusV1": [
            197209,
            0,
            1,
            1,
            ...
            ]
    }

Other networks:

 "costModels": {
    "PlutusV1": {
      "sha2_256-memory-arguments": 4,
      "equalsString-cpu-arguments-constant": 1000,
      "cekDelayCost-exBudgetMemory": 100,
      "lessThanEqualsByteString-cpu-arguments-intercept": 103599,
      "divideInteger-memory-arguments-minimum": 1,
      "appendByteString-cpu-arguments-slope": 621,
      ...
      ]
 }    
  1. Difference in "executionPrices" attribute.

Sanchonet :

"executionPrices": {
        "prMem": 5.77e-2,
        "prSteps": 7.21e-5
 },

Other networks:

"executionPrices": {
    "prSteps":
    {
      "numerator" :   721,
      "denominator" : 10000000
    },
    "prMem":
    {
      "numerator" :   577,
      "denominator" : 10000
    }
 }

Workaround: Copy the existing alonzo-genesis.json file from other networks to the sanchonet folder, as the values are same.

Handle to consume data by batch

Currently LS are consuming one by one. Therefore, to improve the consumption process, we will implement consume in batches

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.