Schema registry aborts after startup

[embeddedkafka]

Schema registry aborts after startup

[francescopellegrini]

Hi @the-ram,
could you provide a failing test?

[the-ram]

Hello @francescopellegrini

Here you go : https://github.com/the-ram/kafka-refdata-sink-vertx-it

sbt clean test

This should fail it!. I am able to get a local cp-schema-registry to run and the test to pass.

[francescopellegrini]

When I run the tests, I get a different error:

org.apache.avro.AvroRuntimeException: Not a valid schema field: unit

At L83 you're trying to set value for a field which doesn't exist (latency and bandwidth are smol only fields).

Also, smol implements SpecificRecord, while in the test you're creating a GenericRecord...

EDIT: please refrain from importing SNAPSHOT versions of this library.

[the-ram]

Hello @francescopellegrini , sorry for that mistake. I have been trying to strip out as many specifics from my code and in the process picked up a random schema on the internet :). Apologies for that , I have since your comment
a) Removed the snapshot version
b) Attached the full trace that i obtained. (logs.txt)

For now i think Generic/Specific record does not seem to be an issue , i just want to get it through kafka to the other parts of it and then probably move to a generic record anyway

[francescopellegrini]

Hi again,
I found a series of issues with the test, but the main one is that the withRunningKafka closure returns a Future which requires the Schema Registry up and running to be successful.

You either have to wait for its result before exiting the closure (using Await.result) or to start/stop EmbeddedKafka using BeforeAndAfterAll methods.

Other issues that I found were:

• the built Avro records miss some fields, so the serializers throw NPE.
• the expected result is wrong, for the promise handler returns the topic being written to.

[the-ram]

Hello @francescopellegrini , the closure was the thing. withKafkaRunning was to be waited on just like you mentioned. As for tests , those were mistakes when i was trying to produce a reproducible test. I added the tests with BeforeAndAfter all and control the lifecycle manually!

[the-ram]

the block needed to be waited on , the closure withRunningKafka returns a future and needs to be waited on. Await.result was good . Better even to avoid time variability is to probably control using Before/After methods!