xtrmstep/kafker

CLI tool to manage snapshots of Kafka topics

kafker's Issues

After finishing of events' extract the tool is hanging up. It requires Ctrl+C to exit.

Error when calling `kafker extract -?`

When calling the command kafker extract -? it throws an error: An error has occurred: Unable to cast object of type 'OffsetKindValidator' to type 'McMaster.Extensions.CommandLineUtils.Validation.AttributeValidator'.

Rename command 'create-template' to 'create'

The command create-template should be renamed to a simple one, just create.

Make the attribute for topic name be specified with same name (letter) in all commands

There are different name to specify name of a topic. For extract it's -t. For creating of a template it's -n. This is confusing. We need to use same option name to specify the topic across all commands.

Emit doesn't recognize INT values in CSV file

For the case when a value clearly integer, the emit command converts everything to string representation with symbol ". For example, operator_id is integer:

event.metric_code;event.operator_customer;event.operator_id;event.customer_id;event.metric_value
customer_single_bet_stake;20060_23405493;20060;23405493;7.252267499999999

But in JSON in Kafka it will be represented as a string: "operator_id":"20060"

It should preserve the integer type and generate it as follows: "operator_id":20060

Remove artificial delay in the algorithm of extracting events

It makes to slow reading of events when reading thousands of them.

When called from random location cannot find some location

PS C:\WINDOWS\system32> kafka-topic-extractor.exe list
Unhandled exception. System.ArgumentNullException: Value cannot be null. (Parameter 'path')
at System.IO.Path.GetFullPath(String path)
at KafkaTopicExtractor.Commands.ListCommand.InvokeAsync() in C:\projects\kafka-topic-exporter\src\KafkaTopicExtractor\Commands\ListCommand.cs:line 26
at KafkaTopicExtractor.Program.<>c__DisplayClass0_0.<

b__7>d.MoveNext() in C:\projects\kafka-topic-exporter\src\KafkaTopicExtractor\Program.cs:line 72
--- End of stack trace from previous location where exception was thrown ---
at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync(String[] args, CancellationToken cancellationToken)
at KafkaTopicExtractor.Program.Main(String[] args) in C:\projects\kafka-topic-exporter\src\KafkaTopicExtractor\Program.cs:line 97
at KafkaTopicExtractor.Program.(String[] args)

When extracting without configuration file, the snapshot file has only timestamp in its name

For example _20201121_111636.dat

It should use topic name and be like this: topic_20201121_111636.dat