tiziano88/elm-protobuf

protobuf plugin for elm

elm-protobuf

Join #elm-protobuf on Slack

Experimental protobuf plugin generating elm code to decode/encode Proto3/JSON from proto definitions.

The plugin itself is written in Go, and it requires the base protoc protobuf compiler to be installed on the system.

For a sample generated output file, see Repeated.elm.

Supported features

• double/float fields
• int32/int64/uint32/uint64/sint32/sint64/fixed32/fixed64/sfixed32/sfixed64 fields
• bool fields
• string fields
• bytes fields
• message fields
• enum fields
• imports
• nested types
• Any type
• Timestamp type
• Duration type
• Struct type
• wrapper types
• FieldMask type
• ListValue type
• Value type
• NullValue type
• oneof
• map
• packages
• options

How to install

Release

The simplest way to install the plugin is to download a pre-compiled binary from https://github.com/tiziano88/elm-protobuf/releases , then unpack it and copy or move the protoc-gen-elm binary somewhere in your $PATH.

From source

• Make sure that you have a Go environment correctly set up, and that $GOPATH/bin is included in your $PATH. See https://golang.org/doc/install for info.

• Install a recent protoc compiler version from https://github.com/google/protobuf (it must have support for proto3 format).

• Obtain the protoc-gen-elm binary using go get:

  go get github.com/tiziano88/elm-protobuf/protoc-gen-elm
  

How to run

Run the protoc compiler specifying the --elm_out flag:

protoc --elm_out=. *.proto

protoc will automatically detect the protoc-gen-elm binary from your $PATH and use it to generate the output elm code.

Then, in your project, add a dependency on the runtime library:

elm install tiziano88/elm-protobuf

References

https://developers.google.com/protocol-buffers/

https://developers.google.com/protocol-buffers/docs/proto3#json

https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.compiler.plugin.pb

https://github.com/google/protobuf/wiki/Third-Party-Add-ons

elm-protobuf's Issues

support gRPC services

this could generate typed client code for grpc services

LIst of items not decoding

First thanks for the elm protobuf compiler, its fantastic and lets me roll my own websocket/json-rpc with protobuf+elm on one side and protobuf+go on the other. The compiled elm has been trouble-free until I tried to decode a list of objects (a "Schedule").

the js console shows elm received some json of apparently the right format

response: { id = "90de-18f43f9f486f", method = "ScheduleListResponse", object = { Ok = True, Schedules = { 0 = { Id = "schedules-877d-76499fe253ae", AccountId = "accounts-b8b0-6d34a035a65e", AlgorithmId = "noop", Status = "active" } } } }

after calling scheduleListResponseDecoder response.object, the decoded object, always has an empty array for Schedules.

ScheduleListResponseMsg { ok = True, message = "", schedules = [] }

the ScheduleListResponse protbuf is defined as

import "proto/schedule.proto";

message ScheduleListResponse {
  bool Ok = 1;
  string Message = 2;
  repeated Schedule Schedules = 3;
}

and the Schedule is

message Schedule {
  string Id = 1;
  string AccountId = 2;
  string AlgorithmId = 3;
  string Status = 4;
}

the Elm for ScheduleListResponse also looks good

type alias ScheduleListResponse =
    { ok : Bool -- 1
    , message : String -- 2
    , schedules : List Schedule -- 3
    }


scheduleListResponseDecoder : JD.Decoder ScheduleListResponse
scheduleListResponseDecoder =
    JD.lazy <| \_ -> decode ScheduleListResponse
        |> required "Ok" JD.bool False
        |> required "Message" JD.string ""
        |> repeated "Schedules" scheduleDecoder

any ideas what might be wrong?

name collisions

Here's some protobuf for which the elm protobuf generator results in elm code that fails to compile. For one, the MATCH enum value and the match oneof field collide. And after fixing that, I get collisions between for example the Change message type alias and the change oneof field.

I can get around this for now by renaming things (and I realize the README checklist lists oneof as unsupported), but seems worth addressing at some point? And I hope a "real-world" test case might be good to have around anyway.

syntax = "proto3";

enum ClaimType {
  MATCH = 0;
  NOMATCH = 1;
}

message Claim {
  ClaimType type = 1;
  repeated uint32 cards = 2;
}

message Event {
  message Join {
    string name = 1;
  }
  message Claimed {
    string name = 1;
    ClaimType type = 2;
    enum Result {
      CORRECT = 0;
      WRONG = 1;
      LATE = 2;
    }
    Result result = 3;
  }   
  oneof event_oneof {
    Join join = 1;
    Claimed claimed = 2;
  }
}

message Change {
  message Position {
    uint32 x = 1;
    uint32 y = 2;
  }
  message Deal {
    message Place {
      Position position = 1;
      uint32 card = 2;
    }
    repeated Place places = 1;
  }
  message Match {
    repeated Position positions = 1;
  }
  message Move {
    message MoveOne {
      Position from = 1;
      Position to = 2;
    }
    repeated MoveOne moves = 1;
  }
  oneof change_oneof {
    Deal deal = 1;
    Match match = 2;
    Move move = 3;
  }
}

message Update {
  oneof update_oneof {
    Change change = 1;
    Event event = 2;
  }
}

Support for proto2 syntax?

I have an existing set of pb files that I'd be interested in using with elm, but they're written with proto2, and it looks like migrating to proto3 might require changes to the code interacting with them. Is there a chance we can support proto2 for this?

Map support

Are there any steps planned on supporting map in the near future?

We are currently looking into GRPC and its integration with Elm for Prometheus Alertmanager. Map support would be a crucial feature for us. (See PR)

Elm 0.19 upgrade not working

When installing this package, now get the following:

└[~/g/s/g/v/t/front]> elm install tiziano88/elm-protobuf
-- OLD DEPENDENCIES --------------------------------------------------- elm.json

The following packages do not work with Elm 0.19.0 right now:

    tiziano88/elm-protobuf

This may be because it is not upgraded yet. It may be because a better solution
came along, so there was no need to upgrade it. Etc. Try asking around on Slack
to learn more about the topic.

Note: Whatever the case, please be kind to the relevant package authors! Having
friendly interactions with users is great motivation, and conversely, getting
berated by strangers on the internet sucks your soul dry. Furthermore, package
authors are humans with families, friends, jobs, vacations, responsibilities,
goals, etc. They face obstacles outside of their technical work you will never
know about, so please assume the best and try to be patient and supportive!

Is this because the version number in the elm.json was not changed from 2.1.0? Or does it need to be republished?

infix functions in generated decoders

Given infix functions are problematic in Elm at the moment (they cause conflicts if any 2 libraries define the same function), and may be going away except for in core libraries in 0.19, would it be possible to replace the (<$>) and (<*>) in the generated decoders?

(Sorry if this is already done, I have not looked at the implementation, just the expected output in the tests.)

Update Functions

I am wondering your thoughts on creating chainable updateX methods for each message. This is something I often do in elm for types I create.

There is a history of such methods on proto objects:

• Java's protobuf Builder objects have chainable setX methods.
• C++/JavaScript/PHP defaults to having setX methods, though they are unchainable.
• Python/Ruby allow direct access with mutation
• Objective-C makes the properties readwrite

Example:

message Model {
  string string_field = 1;
  int64 long_field = 2;
}

Example Proto Output (only new functions shown):

updateStringField : String -> Model -> Model
updateStringField newValue record =
  { record | stringField = newValue }

updateLongField : Int -> Model -> Model
updateLongField newValue record =
  { record | longField = newValue }

Example Usage:

update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
  case msg of
    SomeMsg someStr someLong ->
      ( model
          |> updateStringField someStr
          |> updateLongField someLong
      , Cmd.none )

I would gladly offer to help with this if you're interested in the idea

Incorrect code generated with type names

When compiling a single proto file that imports another, the referenced types don't seem to be correct:

sub.proto:

syntax = "proto3";

package Foo;

message Baz2 {
	int32 field_a = 1;
}

main.proto:

syntax = "proto3";

import "sub.proto";

package Foo;

message Baz {
	Baz2 field_a = 1;
}

Compiled with: protoc --elm_out=. main.proto sub.proto

Produces incorrect code:

-- Sub.elm
type alias Baz2 =
    { fieldA : Int -- 1
    }

-- Main.elm
type alias Baz =
    { fieldA : Maybe Foo_Baz2 -- 1
    }

Compiler error for reference:

I cannot find a `Foo_Baz2` type:

16|     { fieldA : Maybe Foo_Baz2 -- 1

It seems as though the package name is incorrectly being added to the name of the type.
As a workaround, if I comment out the package name from both proto files it seems to generate correct code.

I don't know if this falls into the unsupported 'Packages' listing in the main README.md, since it didn't list details as to why it wasn't supported.

Interestingly, this is very similar to a bug I just reported in the other Elm protoc project: eriktim/protoc-gen-elm#7

Update tags to support Go modules

There may be a workaround, but I'm having a lot of trouble trying to install latest due to the current tags lacking the "v" prefix.

enum as int32

I'm using protoc with elm and go to pass data structures back and forth over websocket/json. Its been going great until I tried using enums. To my delight, the elm module uses the string value of each enum option. Unfortunately, the go (and crystal) code generators use the int32 form of an enum option. So {"status":3} does not decode on the elm side, and {"status":"ready"} does not decode on the go side Is there a way to get the generated elm code to use the int? I'd rather use the string val name but it appears int is more canonical, and possibly a win for space efficiency.