dogacel/kotlinx-protobuf-gen

Generate kotlinx serializable data classes from protobuf

kotlinx-protobuf-gen's Issues

Support custom Well-known types

This feature contains two parts:

• Provide popular well-known Kotlin types to substitute well-known protobuf types in generated code such as Timestamp to kotlinx.datetime.Instant, Duration to kotlin.time.Duration or IntWrapper to Int?.
• [Optional] Users should be able to override certain types by the type of their choice. I.e. a user can define a convention to use UUID for certain wrapper type or a specific field.

Make sure this is defined as an option in code generator.

Make sure serialization / deserialization works with the generated types.

Write a detailed documentation with examples and intention

• Explain why we need this library, what are the advantages
• Showcase the generated code in-detail
• Present real-life use cases hopefully when exists
• Consider releasing a github pages page to show documentation. Documentation page can be written in mkdocs or docusaurus.

Some generated repeated fields do not work correctly

Those types include:

• fixedXX
• sfixedXX
• uintXX
• sintXX

Following up with kotlinx.serialization team here.

Most likely solution is gonna be determined by the serialization lib. we should contribute to main library.

Run conformance tests and report

Run the full set of conformance tests and report or document it somewhere.

Those tests will ensure we are almost 100% compatible with the protobuf wire format. Optionally protobuf json compatability can be tested as well.

More details on conformance tests here.

Note

Unfortunately kotlinx.serialization also does not have conformance tests built-in. Considering it is an experimental API, we can support both our code and their code at the same time by contributing to the library. I have made an attempt to contribute by opening this PR: Kotlin/kotlinx.serialization#2404. This is a good starting point, it is very easy to see what kotlinx.serialization library is lacking by running those kinds of tests.

Add in-code comments to generated code

We should show copy comments in protobuf files to generated code.

Source Code Info can be found inside the file descriptor proto.

Example:

// A test message
message TestMessage {
  // The id
  int32 id = 1;
  string name = 2; // A name
}

should be converted to

/**
 * A test message
 *
 * @param id The id
 * @param name A name
**/
@Serializable
data class TestMessage ( ... )

Setup release pipeline to maven central

Getting a namespace seems though. Need to figure out if we can get our own namespace in github such as com.github.dogacel for the first release.

Releasing a version is very important as we can get some early adopters this way.

Maps with `fixed` keys can't be serialized

Generated map fields with fixedXX and sfixedXX keys can't be serialized.

We should follow up with `kotlinx.serializationz team. I already made some attempts:

• Kotlin/kotlinx.serialization#2401
• Kotlin/kotlinx.serialization#2404

gRPC wire protocol support

Currently services and methods are generated. But there is no way to call or serve those actual methods.

There are several caveats,

1. Don't write your own client library. There are lots of good options and we don't want our users to depend on our version which is definitely less reliable than something like Netty or grpc-kotlin.
2. Make your code inter-operable. Investigate libraries for kotlin js, jvm and native. Make sure generated stubs can be plugged in into any of those libraries.

My service of preference is Armeria for testing.

Easiest way to develop such a feature would be starting with Unary only methods.

Negative enum values do not work

Opened a pr to kotlinx.serialization:

• Kotlin/kotlinx.serialization#2400

Change project structure to fully automate code generation and tests

Currently code generation is done manually by running a file in tests. We should generate the code to a separate sub-project automatically during CI.

We should create tests for the actual generators, not just for the generated code. Kover shows 0% coverage because of it.

[FEATURE] - Issue Duplicated Enum Value

prefer to protocolbuffers/protobuf#5425
Do you have any options to solve this issue when we gen the kotlin file ?

Support Protobuf JSON

Make sure well-known-types are also handled correctly. Full details here

For example JSON encoding for a Timestamp is not { "seconds": 10, "nanos": 0 } but rather an ISO string such as "2020-01-01T12:00:00Z".

Put the necessary annotations to the generated code.

Supporting both snake case and json_name option is optional for this issue.

Two identical messages containing `bytes` are not considered equal

Either consider using something other than ByteArray or override the necessary methods in generated data class.

Data classes methods equals and hashCode should be overriden when a ByteArray property exists.

Overriding those methods will cause a huge equals function with lots of boilerplate. This should be fine considering all code is just auto-generated.

[FEATURE] Prettify enum names

Is your feature request related to a problem? Please describe.
Enum names are always prefixed by ENUM_NAME_ and this is not very intuitive for kotlin code. This is a limitation to support C++ generated code.

Describe the solution you'd like
Add an option to automatically remove ENUM_NAME_ prefix from generated enums.

Describe alternatives you've considered
None

Additional context

I think we can include an option to remove the ENUM_NAME_ prefix from the actual enum names in the generated code. Such as

enum State {
  STATE_UNKNOWN = 0
  STATE_KNOWN = 1
}

will be converted to

@Serializable
public enum class State {
    @ProtoNumber(number = 0)
    UNKNOWN,
    @ProtoNumber(number = 1)
    KNOWN,
}

And this can be toggled via an option

    generateProtoTasks {
        all().forEach {
            it.plugins {
                id("kotlinx-protobuf-gen") {
                    option("remove_enum_prefix") // <-- this
                }
            }
        }
    }

Based on conversation with @tamnguyenhuy from #14

[BUG] Message without any properties is not compilable

Describe the bug
A proto message without any properties creates a code that is not compilable.

To Reproduce

syntax = "proto3";

message Hello {
  enum Type {
    UNKNOWN = 0;
    KNOWN = 1;
  }
  message SubHello {
    message HelloExtend {
      optional Type type = 1;
    }
    optional SubHello subHello = 1;
  }
}

Expected behavior
A data class shouldn't be generated, instead it should be just class.

Additional context
Data classes require at least 1 property.

Support message and field options

We should respect options such as deprecated or json_name.

Before starting, we need to decide on the list of options we want to support initially. Generated code can be morph based on those options.