grpc-ecosystem/grpc-gateway

gRPC to JSON proxy generator following the gRPC HTTP spec

grpc-gateway's Issues

gRPC gateway Bazel build rules

I'm using the normal bazel go rules (http://bazel.io/docs/be/go.html) and I've started looking into creating a build rule for Bazel so the output from grpc-gateway can be depended upon directly. I have a sneaking suspicion that you're using Bazel in some of your projects. Do you have any tricks/tips for getting everything set up in Bazel? Would you be willing to put a sample in the examples folder of how you're doing this?

Emit API definition in Swagger schema format

Swagger provides a good way to generate API references and client libraries of the REST API.
So we should have a way to generate swagger schema from .proto file to leverage with Swagger.

Is PascalFromSnake the right conversion to be doing?

As far as I can tell, a protobuf name like uri becomes URI when generated. PascalFromSnake will convert uri into Uri, which is not compatible, causing populateQueryParameter to fail to do its job.

What's the right way to solve this mismatch?

Should PascalFromSnake be replaced with something that uses Go style casing that keeps acronym case matching (i.e. either uri or URI, never Uri)? That seems impossible to do in the general case, unless you hardcode a dictionary of all words that are considered acronyms. Although I wonder how the protobuf generator does this conversion.

The alternative is to have protobuf generator use pascal casing instead of Go style casing. This seems more friendly towards automatic code generation since it doesn't require hardcoded special cases (an acronym dictionary).

Any thoughts? How is this problem generally solved?

Runtime panic with CloseNotify

The integration test sometimes fails with a runtime panic.

e.g. https://travis-ci.org/gengo/grpc-gateway/jobs/111913217

$ env GLOG_logtostderr=1 go test -v github.com/gengo/grpc-gateway/...
=== RUN   TestIntegration
E0226 02:27:10.407340    5251 a_bit_of_everything.go:67] nested:<name:"hoge" amount:10 > nested:<name:"fuga" amount:20 > float_value:1.5 double_value:2.5 int64_value:4294967296 uint64_value:9223372036854775807 int32_value:-2147483648 fixed64_value:9223372036854775807 fixed32_value:4294967295 bool_value:true string_value:"strprefix/foo" uint32_value:4294967295 sfixed32_value:2147483647 sfixed64_value:-4611686018427387904 sint32_value:2147483647 sint64_value:4611686018427387903 
E0226 02:27:10.407661    5251 a_bit_of_everything.go:67] nested:<name:"hoge" amount:10 > nested:<name:"fuga" amount:20 > float_value:1.5 double_value:2.5 int64_value:4294967296 uint64_value:9223372036854775807 int32_value:-2147483648 fixed64_value:9223372036854775807 fixed32_value:4294967295 bool_value:true string_value:"strprefix/bar" uint32_value:4294967295 sfixed32_value:2147483647 sfixed64_value:-4611686018427387904 sint32_value:2147483647 sint64_value:4611686018427387903 
E0226 02:27:10.407774    5251 a_bit_of_everything.go:67] nested:<name:"hoge" amount:10 > nested:<name:"fuga" amount:20 > float_value:1.5 double_value:2.5 int64_value:4294967296 uint64_value:9223372036854775807 int32_value:-2147483648 fixed64_value:9223372036854775807 fixed32_value:4294967295 bool_value:true string_value:"strprefix/baz" uint32_value:4294967295 sfixed32_value:2147483647 sfixed64_value:-4611686018427387904 sint32_value:2147483647 sint64_value:4611686018427387903 
E0226 02:27:10.407873    5251 a_bit_of_everything.go:67] nested:<name:"hoge" amount:10 > nested:<name:"fuga" amount:20 > float_value:1.5 double_value:2.5 int64_value:4294967296 uint64_value:9223372036854775807 int32_value:-2147483648 fixed64_value:9223372036854775807 fixed32_value:4294967295 bool_value:true string_value:"strprefix/qux" uint32_value:4294967295 sfixed32_value:2147483647 sfixed64_value:-4611686018427387904 sint32_value:2147483647 sint64_value:4611686018427387903 
E0226 02:27:10.407969    5251 a_bit_of_everything.go:67] nested:<name:"hoge" amount:10 > nested:<name:"fuga" amount:20 > float_value:1.5 double_value:2.5 int64_value:4294967296 uint64_value:9223372036854775807 int32_value:-2147483648 fixed64_value:9223372036854775807 fixed32_value:4294967295 bool_value:true string_value:"strprefix/quux" uint32_value:4294967295 sfixed32_value:2147483647 sfixed64_value:-4611686018427387904 sint32_value:2147483647 sint64_value:4611686018427387903 
panic: net/http: CloseNotify called after ServeHTTP finished
goroutine 89 [running]:
panic(0x87f8e0, 0xc82036c150)
    /home/travis/.gimme/versions/go/src/runtime/panic.go:483 +0x3f3
net/http.(*response).CloseNotify(0xc820360270, 0x0)
    /home/travis/.gimme/versions/go/src/net/http/server.go:1535 +0x9d
github.com/gengo/grpc-gateway/examples/examplepb.RegisterABitOfEverythingServiceHandler.func9.1(0x7fe8ab5313a0, 0xc820360270, 0xc82036c070)
    /home/travis/gopath/src/github.com/gengo/grpc-gateway/examples/examplepb/a_bit_of_everything.pb.gw.go:658 +0x27
created by github.com/gengo/grpc-gateway/examples/examplepb.RegisterABitOfEverythingServiceHandler.func9
    /home/travis/gopath/src/github.com/gengo/grpc-gateway/examples/examplepb/a_bit_of_everything.pb.gw.go:660 +0x132
FAIL    github.com/gengo/grpc-gateway/examples  0.149s
?       github.com/gengo/grpc-gateway/examples/examplepb    [no test files]
?       github.com/gengo/grpc-gateway/examples/server   [no test files]
?       github.com/gengo/grpc-gateway/examples/sub  [no test files]
?       github.com/gengo/grpc-gateway/protoc-gen-grpc-gateway   [no test files]

Add HTTP error code, error status to responseStreamChunk Error

When using unitary-to-streaming requests, it is hard to reason about HTTP error codes based on a chunk that contains error: it only contains the text of the error response.

Instead, it would be useful for the error field of a chunk to be structured:

• grpc_code containing the code of the RPC error (to avoid the pain of extracting it from the error string)
• http_code that is from HTTPStatusFromCode
• message containing the current string of the error
• http_status containing a textual representation of the grpc_code or http_code.

google.api.http options in multi-line format not supported

  rpc Echo(EchoRequest) returns (EchoReply) {
    // (1) works
    //option (google.api.http) = {
    //  post: "/v1/echo/echo"
    //  body: "*"
    //};
    // (2) doesn't work
    option (google.api.http).body = "*";
    option (google.api.http).post = "/v1/echo/echo";
  }

It seems like grpc-gateway compiler only supports aggregate TextFormat syntax. When using (2), compiler fails with error:

$ protoc -I. -I/usr/local/include -I../third_party/googleapis --grpc-gateway_out=logtostderr=true:. echo.proto
E0530 22:16:47.262160    6265 services.go:106] No pattern specified in google.api.HttpRule: Echo

According to the protobuf developers guide, both syntax are correct.
https://developers.google.com/protocol-buffers/docs/proto#options

echo service example does not compile

➜ go build
# github.com/gengo/grpc-gateway/examples/examplepb
examplepb/echo_service.pb.go:114: cannot use _EchoService_Echo_Handler (type func(interface {}, context.Context, grpc.Codec, []byte) (interface {}, error)) as type grpc.methodHandler in field value
examplepb/echo_service.pb.go:118: cannot use _EchoService_EchoBody_Handler (type func(interface {}, context.Context, grpc.Codec, []byte) (interface {}, error)) as type grpc.methodHandler in field value

REST API without proxying

Hello. I think it is a little wasteful to parse JSON then create a protobuf message and again parse it, rather than just parse the JSON into protobuf structure. Services could perform a little slower when parsing big amounts of JSON but I think overall cost would be reduced. Please tell me what do You think about this idea.

Support oneof types of fields

Extracted from #5.

Currently oneof fields are not supported. They should be allowed in request body and query string as well as other types.

• This feature requires #79 because "encoding/json".Marshal returns error for oneof fields
• No plan to allow them in path parameters because path parameters are "mandatory" parameters. So they don't fit to oneof very well.

Method parameter in query string

undefined: proto.SizeVarint

Getting the following error when attempting to run go get github.com/gengo/grpc-gateway/protoc-gen-grpc-gateway

Format Errors to JSON

Would it make sense to have some way to format the errors received from the RPC call into JSON? Currently, this function is called when an error occurs. A client of the HTTP server would be expecting JSON in the response though.

The response code is available which could indicate the error type but passing along an error message in JSON would be useful for the client.

Thoughts?

Logging

Hey,

Old topic, simpler idea: I don't want to re-invent the wheel, but can we do something similar to https://github.com/grpc/grpc-go/tree/master/grpclog for gprc-gateway? Using glog means everyone is locked into glog's very opinionated way of storing output, and it means in a lot of cases (including mine) that I just completely lose grpc-gateway logs.

The only change in the code would be to get rid of v-level logging, but I mean this is minor as I see it within the code.

Missing default values in JSON output?

Is this intentional, or would it perhaps make more sense if default values were added to the output JSON?

Support for Any types

I am using Any type as described in golang/protobuf#60. But GRPC gateway will transform this into a base64 encoded string from a byte[]. How can Any type be converted into a JSON object given the byte[] is JSON data?

Better grpc error strings

Errors currently look similar to this:

{
  "error": "rpc error: code = 16 desc = \"Unauthenticated\""
}

Wouldn't it be better to set error just to the description?

Example:

{
  "error": "Unauthenticated"
}

This can easily be achieved by replacing err.Error() with grpc.ErrorDesc(err) on this line:
https://github.com/gengo/grpc-gateway/blob/master/runtime/errors.go#L77

I can send a pull request if wished.

No pattern specified in google.api.HttpRule

Hi,

I'm starting using grpc-gateway and this is probably a newbie question.

When running this I'm getting the following error. Probably missing something...

~/github/grpc-gateway (master)$ protoc -I/usr/local/include -I.  -I$GOPATH/src  -I$GOPATH/src/github.com/gengo/grpc-gateway/third_party/googleapis  --grpc-gateway_out=logtostderr=true:.  examples/examplepb/echo_service.proto
E1209 15:00:22.665743   18512 services.go:107] No pattern specified in google.api.HttpRule: Echo

Thanks,
Paulo Grácio

allow use of jsonpb for marshaling

github.com/golang/protobuf/jsonpb is a more compliant protobuf -> JSON package than encoding/json, and is the only way github.com/golang/protobuf plans to support JSON encoding in the future.

See this comment

The newly added jsonpb package (see 67cbcad) is the official vehicle for JSON support...

It would be great to be able to using this package for marshaling. It might even be wise to switch to it entirely.

URL usage of nested messages causes nil pointer in proto3

Take this as an example (in proto3):

message DeploymentId {
  string name = 1;
}

message FlagId {
  DeploymentId deployment = 1;
  string name = 2;
}

message Flag {
  FlagId id = 1;
  string value = 2;
  string description = 3;
}

service Deployments {
  rpc GetFlag (FlagId) returns (Flag) {
    option (google.api.http) = {
      get: "/deployment/v1/{deployment.name}/flag/{name}"
    };
  }
}

When I call the service through GET /deployment/v1/mydeployment/flag/myflag I get a nil pointer dereference:

2015/07/26 18:58:31 http: panic serving [::1]:59983: runtime error: invalid memory address or nil pointer dereference
goroutine 18 [running]:
net/http.func·011()
    /usr/local/Cellar/go/1.4/libexec/src/net/http/server.go:1130 +0xbb
github.com/mwitkow-io/grpc-experiment/proto/deployments.request_Deployments_GetFlag0(0xb5fb88, 0xc2080eda80, 0xb5fc00, 0xc2080363a8, 0xc208032410, 0xc20811a240, 0x0, 0x0, 0x0, 0x0)
    /Users/michal/code/mygo/src/github.com/mwitkow-io/grpc-experiment/proto/deployments/deployments.pb.gw.go:46 +0x84b
github.com/mwitkow-io/grpc-experiment/proto/deployments.func·003(0xb60068, 0xc208042780, 0xc208032410, 0xc20811a240)

the Gateway-generated code in question:

func request_Deployments_GetFlag_0(ctx context.Context, client DeploymentsClient, req *http.Request, pathParams map[string]string) (msg proto.Message, err error) {
    var protoReq FlagId

    var val string
    var ok bool

    val, ok = pathParams["deployment.name"]
    if !ok {
        return nil, grpc.Errorf(codes.InvalidArgument, "missing parameter %s", "deployment.name")
    }
    protoReq.Deployment.Name, err = runtime.String(val)
    if err != nil {
        return nil, err
    }

    val, ok = pathParams["name"]
    if !ok {
        return nil, grpc.Errorf(codes.InvalidArgument, "missing parameter %s", "name")
    }
    protoReq.Name, err = runtime.String(val)
    if err != nil {
        return nil, err
    }

    if err := runtime.PopulateQueryParameters(&protoReq, req.URL.Query(), filter_Deployments_GetFlag_0); err != nil {
        return nil, grpc.Errorf(codes.InvalidArgument, "%v", err)
    }

    return client.GetFlag(ctx, &protoReq)
}

The problem seems to be that the Deployment field of FlagId is not initialized and is a nil reference:

type FlagId struct {
    Deployment *DeploymentId `protobuf:"bytes,1,opt,name=deployment" json:"deployment,omitempty"`
    Name       string        `protobuf:"bytes,2,opt,name=name" json:"name,omitempty"`
}

I found some complex expressions like {bucket_name=buckets/*} inparse_test.go`, but I don't think these would help here. @yugui is this a bug or am I just using grpc gateway wrong?

panic: net/http: CloseNotify called after ServeHTTP finished

I made a simple modification to the echo example in which I use a message that is type bytes:

message BytesMessage {
    bytes id =1;
}

Changed the proto for EchoBody to use BytesMessage:

    rpc EchoBody(BytesMessage) returns (BytesMessage) {
        option (google.api.http) = {
            post: "/v1/example/echo_body"
            body: "*"
        };
    }

Modified echo.go:EchoBody to be:

func (s *echoServer) EchoBody(ctx context.Context, msg *examples.BytesMessage) (*examples.BytesMessage, error) {

And then when I send invalid information:

➜   curl -X POST -k http://localhost:8080/v1/example/echo_body -H "Content-Type: text/plain" -d '{"id": "notbase64"}'`
{"error":"illegal base64 data at input byte 8","code":3}%                                                                                                                                  ➜  

It panics:

➜  ./examples
panic: net/http: CloseNotify called after ServeHTTP finished

goroutine 7 [running]:
panic(0x7efa00, 0xc82000bd00)
    /usr/lib/go/src/runtime/panic.go:464 +0x3e6
net/http.(*response).CloseNotify(0xc8201101a0, 0x0)
    /usr/lib/go/src/net/http/server.go:1535 +0x9d
github.com/gengo/grpc-gateway/examples/examplepb.RegisterEchoServiceHandler.func2.1(0x7fdb33c0ae98, 0xc8201101a0, 0xc82000bbc0)
    /home/diogo/go/src/github.com/gengo/grpc-gateway/examples/examplepb/echo_service.pb.gw.go:127 +0x27
created by github.com/gengo/grpc-gateway/examples/examplepb.RegisterEchoServiceHandler.func2
    /home/diogo/go/src/github.com/gengo/grpc-gateway/examples/examplepb/echo_service.pb.gw.go:129 +0x132

cannot find package "google/api"

Upstream protobuf package changed and the generated code now fails:

⚛ ~/go/src/github.com/gengo/grpc-gateway(master)$ git pull
Current branch master is up to date.
⚛ ~/go/src/github.com/gengo/grpc-gateway(master)$ git clean -xdf .
⚛ ~/go/src/github.com/gengo/grpc-gateway(master)$ make examples
go get github.com/golang/protobuf/protoc-gen-go
go build -o bin/protoc-gen-go github.com/golang/protobuf/protoc-gen-go
protoc -I /usr/local/bin//../include -I. -Ithird_party/googleapis --plugin=bin/protoc-gen-go --go_out=Mgoogle/protobuf/descriptor.proto=github.com/golang/protobuf/protoc-gen-go/descriptor,Mexamples/sub/message.proto=github.com/gengo/grpc-gateway/examples/sub,plugins=grpc:. examples/examplepb/echo_service.proto examples/examplepb/a_bit_of_everything.proto examples/examplepb/flow_combination.proto
protoc -I /usr/local/bin//../include  -Ithird_party/googleapis --plugin=bin/protoc-gen-go --go_out=Mgoogle/protobuf/descriptor.proto=github.com/golang/protobuf/protoc-gen-go/descriptor,Mexamples/sub/message.proto=github.com/gengo/grpc-gateway/examples/sub:third_party/googleapis third_party/googleapis/google/api/annotations.proto third_party/googleapis/google/api/http.proto
go build -o bin/protoc-gen-grpc-gateway github.com/gengo/grpc-gateway/protoc-gen-grpc-gateway
protoc -I /usr/local/bin//../include -I. -Ithird_party/googleapis --plugin=bin/protoc-gen-grpc-gateway --grpc-gateway_out=logtostderr=true,Mgoogle/protobuf/descriptor.proto=github.com/golang/protobuf/protoc-gen-go/descriptor,Mexamples/sub/message.proto=github.com/gengo/grpc-gateway/examples/sub:. examples/examplepb/echo_service.proto examples/examplepb/a_bit_of_everything.proto examples/examplepb/flow_combination.proto
protoc -I /usr/local/bin//../include -I. --plugin=bin/protoc-gen-go --go_out=Mgoogle/protobuf/descriptor.proto=github.com/golang/protobuf/protoc-gen-go/descriptor,Mexamples/sub/message.proto=github.com/gengo/grpc-gateway/examples/sub,plugins=grpc:. examples/sub/message.proto
⚛ ~/go/src/github.com/gengo/grpc-gateway(master)$ go build ./...
examples/examplepb/a_bit_of_everything.pb.go:10:8: cannot find package "google/api" in any of:
    /usr/local/Cellar/go/1.5.1/libexec/src/google/api (from $GOROOT)
    /Users/tmc/go/src/google/api (from $GOPATH)

Reverse the code gen idea

I would like it is this translates a grpc into a RESTful JSON api.
This is the opposite of what the code currently does, accordion to the docs.

The reason I am wanting this is because its more bottom up. I typically build my service api, and generate the service proxy from it, and then get the web site to use that.

If there is any confusion please ask me

Support for multi-segment elements

Hi,

I interpret https://github.com/googleapis/googleapis/blob/master/google/api/http.proto#L191 as it should be OK to for example write:

        rpc Lookup(IdMessage) returns (ABitOfEverything) {
                option (google.api.http) = {
                       get: "/v1/example/a_bit_of_everything/{uuid=**}"
                };
        }

.. and in this case make the the URL http://localhost:8080/v1/example/a_bit_of_everything/foo/bar valid (uuid would then become "foo/bar").

Changing to {uuid=*} and comparing the generated protobuf code with just {uuid} results in no diff, excellent! Changing to {uuid=**} provides this diff:

-       pattern_ABitOfEverythingService_Lookup_0 = runtime.MustPattern(runtime.NewPattern(1, []int{2, 0, 2, 1, 2, 2, 1, 0, 4, 1, 5, 3}, []string{"v1", "example", "a_bit_of_everything", "uuid"}, ""))
+       pattern_ABitOfEverythingService_Lookup_0 = runtime.MustPattern(runtime.NewPattern(1, []int{2, 0, 2, 1, 2, 2, 3, 0, 4, 1, 5, 3}, []string{"v1", "example", "a_bit_of_everything", "uuid"}, ""))

Which seems to be replacing a OpPush with OpPushM which seems exactly what we want.

Recompiling the example, adding debug output to the gRPC backend server, and trying to access the webpage gives:

http://localhost:8080/v1/example/a_bit_of_everything/foo
{"error":"foo not found","code":5}

but

http://localhost:8080/v1/example/a_bit_of_everything/foo/bar
Not Found

What gives?

Ading grpc-timeout support

First off: gRPC Gateway is awesome. We've been using it in Prod since time immortal.

We're currently adding gRPC timeouts throughout our stack, to make sure that hanging connections don't happen. grpc-timeout is a first class citizen in all gRPC implementations (see http://www.grpc.io/docs/guides/wire.html), and in case of gRPC-Go it is propagated using the context.Deadline in the gRPC client.

We tries enabling deadliens by passing a context.WithTimeout through RegisterMyFooServiceHandler, but unfortunately this lead to all our REST-originating gRPC requests to timeout immediately. That's because context.WithTimeout is implemented through context.WithDeadline from time.Now() of the server initialization :(

Unfortunately gRPC ClientConn doesn't support default context timeouts, so we want to work around it as follows:

We'd be happy to send in a patch that would:

• in runtime.AnnotateContext extract a header Grpc-Timeout and set it into the context.WithTimeout
• in absense of the Grpc-Timeout header, it'd use a static variable that controls the timeout runtime.DefaultTimeout = 0 (similar to runtime.HTTPError), which by default would disable the behaviour

What do you think? We're happy to provide this as a PR :)

Is there any plan to developing a C++ version?

Is there any plan to develop a C++ version? :)

Static mapping from proto field names to golang field names

As discussed in #85 (comment), runtime/query.go requires runtime mapping from proto field names to golang field names.
But the mapping rule got complicated in golang/protobuf@2c7bafc.

So it is better to generate a static mapping for each message or query fields with struct tags to keep consistency to the mapping rule.

Values containing slashes in URL template?

From reading https://github.com/google/googleapis/blob/master/google/api/http.proto, it suggests that some parameters may be a part of the URL template, for example:

get: "/v1/messages/{message_id}"
get: "/v1/users/{user_id}/messages/{message_id}"

HTTP	RPC
GET /v1/messages/123456	GetMessage(message_id: "123456")
GET /v1/users/me/messages/123456	GetMessage(user_id: "me" message_id: "123456")

However, if I'm reading this spec right, those values may not contain slashes, can they?

For example, suppose I wanted to define:

get: "/v1/dirs/{dir_path}"

And have the following HTTP requests match:

HTTP	RPC
GET /v1/dirs/foo	GetDirs(dir_path: "foo")
GET /v1/dirs/foo/bar	GetDirs(dir_path: "foo/bar")
GET /v1/dirs/foo/bar/baz	GetDirs(dir_path: "foo/bar/baz")

Only the first will get matched, the other two will not. Is that so?

Is it true that there's no way to have all 3 of the above paths get matched, as long as grpc-gateway follows that google.api.http spec? Is the only alternative to use URL queries, like
GET /v1/dirs?dir_path=foo/bar/baz?

If so, that seems to put a heavy restriction on one's ability to create RESTful APIs using grpc-gateway, since it's often desirable to have some slash separated paths as values.

array of maps in json

Hi,

Not sure if this a grpc-gateway thing or a proto3 thing. But I cannot get grpc-gateway to parse [{"2":"3"}]. The proto3 def that I'm using is this:

message mapper {
    map<string,string> m = 1;
}

message Alerts {
  repeated mapper alert = 1;
}

proto3 language def says maps can't be repeated, but protoc happily generates (Go) code for this, so....?

Dropping the [] and using {"2":"3"} works as expected. Is this a bug or the proto3 limitation?

Breaks with 1.5rc1 due to "internal" package name.

Generation of one swagger file out of multiple protos?

Hi,

First I want to thank you for creating this extremely useful tool!

Now this is probably more on the level of protoc than grpc-gateway but only concerns me regarding a RESTful API.

It is only possible to generate one *.swagger.json file out of one .proto file and not out of multiple files? (same for *.pb.gw.go) I am currently planning new software that should have a very modular structure and will most likely end up with a lot of .proto file. That makes the swagger generator more or less useless for me, doesn't it?

Thank you and kind regards,

Fabian

Infinite loop in generator when package name conflicts

https://github.com/gengo/grpc-gateway/blob/master/protoc-gen-grpc-gateway/gengateway/generator.go#L48

alias := fmt.Sprintf("%s_%d", pkg.Name, i)
if err := reg.ReserveGoPackageAlias(pkg.Name, pkg.Path); err != nil {
    continue
}

reg.ReserveGoPackageAlias(pkg.Name, pkg.Path) should be reg.ReserveGoPackageAlias(alias, pkg.Path) instead.

generated code has a data race

ctx is overwritten while a goroutine could possibly read from it, breaking go test -race. You can reproduce by simply running go test -race ./... from the repository's base directory.
the read is here and the write is here

This can be fixed by using a differently named variable here instead of overwriting ctx and changing its subsequent usage.

Gateway code generation broken when rpc method with a streaming response has an input paramter

For example, for this proto file

syntax = "proto3";

import "google/api/annotations.proto";
package grpc.testing;

option java_package = "grpc.testing";

message Price {
    string symbol = 1;
    int32 price =2;
    // adding this later to existing code
    repeated string additionalSymbols = 3;
    // again added after the fact
    map<string,int32> symbolIds = 4;
}

message Id {
    int32 clientId = 1;
}

service TestService {
    rpc Ping (Price) returns (Price) {
        option (google.api.http) = {
            get: "/v1/test/ping/{symbol}/{price}"
        };
    }

    rpc StreamingPing (Price) returns (stream Price) {
        option(google.api.http) = {
            post: "/v1/test/streamingping/{symbol}"
        };
    }

    rpc PubSubPing (Id) returns (stream Price);

    rpc BiDirPing (stream Id) returns (stream Price);
}

The gateway code generated for the StreamingPing method does not compile. This only happens when the URL has a input param for a method with a streaming response.

Support wildcard HTTP method in CustomHttpPattern

According to the spec, google.api.CustomHttpPattern.kind can be *. In this case the http pattern matches to any http method.

Support this feature in the implementation of grpc-gateway.

Integrate authentication

Also relay credentials too in addition to method invocation itself.

JavaScript Proxy

This is working well for me a the moment.

I am using Polymer Elements in the browser, looking to generate a JavaScript proxy of the restful gateway.

Has anyone done this ? Any advice here ?

go vet issues in template_test.go

I'm not sure what the string should be otherwise I would have made the PR myself, but there are multiple go vet issues in protoc-gen-swagger/genswagger/template_test.go:

https://goreportcard.com/report/github.com/gengo/grpc-gateway#go_vet

Closing the HTTP connection does not cancel the Context

If you have e.g. a streaming endpoint sending results from the server in an infinite stream and close the HTTP connection, the grpc server doesn't know about that and keeps waiting for the client to accept more data. To prevent that, the gprc gateway should cancel the context when the HTTP connection is closed.

Feature Request: Support for non-nullable nested messages.

gogo protobuf generator has an option to generate non-nullable nested messages via an [(gogoproto.nullable) = false] option.

Could support for that be added to grpc-gateway in its populateQueryParameter func?

Since the code change is really small, I will make a PR that resolves this issue, but of course this it will need discussion/review.

Multiple .proto files generates invalid import statements.

passing multiple .proto files that include an import generates invalid import lines.

ex: compile a.proto and b.proto together with a importing b:

import (
proto_0 "b.pb"
)

would be generated.

Encoding with application/x-www-form-urlencoded

Today request/response body is always encoded in JSON.
Should accept application/x-www-form-urlencoded too.

1. Define a new field in the custom option to define a list of acceptable encoding of request body
2. Similar field to define the list of possible response body
   • maybe should prefer the same encoding as the request
3. Make the parameter deserialization/serialization pluggable

Fields aren't named in the same manner as golang/protobuf

For example:

diff --git a/examples/examplepb/echo_service.proto b/examples/examplepb/echo_service.proto
index 5e44b28..f39e711 100644
--- a/examples/examplepb/echo_service.proto
+++ b/examples/examplepb/echo_service.proto
@@ -5,13 +5,13 @@ package gengo.grpc.gateway.examples.examplepb;
 import "google/api/annotations.proto";

 message SimpleMessage {
-       string id = 1;
+       string ID = 1;
 }

 service EchoService {
        rpc Echo(SimpleMessage) returns (SimpleMessage) {
                option (google.api.http) = {
-                       post: "/v1/example/echo/{id}"
+                       post: "/v1/example/echo/{ID}"
                };
        }
        rpc EchoBody(SimpleMessage) returns (SimpleMessage) {

Will fail with

# github.com/gengo/grpc-gateway/examples/examplepb
examples/examplepb/echo_service.pb.gw.go:47: protoReq.Id undefined (type SimpleMessage has no field or method Id, but does have ID)

Simply using the exported CamelCase function fixes this:

package utilities

import "github.com/gogo/protobuf/protoc-gen-gogo/generator"

// PascalFromSnake converts an identifier in snake_case into PascalCase.
func PascalFromSnake(str string) string {
    return generator.CamelCase(str)
}

Header Forwarding from server.

@yugui With the specified header Prefix we can forward header from gateway to the grpc server and can get it within the context. But can't forward header from the server to the client with the gateway.

Grpc server can send header with the context with

grpc.SendHeader(context, header-metadata)

But the header is truncated in the gateway. The forwarded header is not received by the client.
Are there any work progress in it or any other ways to doing it.

CloseNotify race with ServeHTTP

The following request of the provided example server:

curl http://localhost:8080/v1/example/echo_body -d 'foo'

Causes the following panic:

panic: net/http: CloseNotify called after ServeHTTP finished

goroutine 73 [running]:
panic(0x3fcb00, 0xc8201e2840)
    /usr/local/Cellar/go/1.6/libexec/src/runtime/panic.go:464 +0x3e6
net/http.(*response).CloseNotify(0xc8201e0750, 0x0)
    /usr/local/Cellar/go/1.6/libexec/src/net/http/server.go:1535 +0x9d
github.com/gengo/grpc-gateway/examples/examplepb.RegisterEchoServiceHandler.func2.1(0xc8201e2720, 0x98a070, 0xc8201e0750, 0xc8201e2730)
    /Users/cdo/gocode/src/github.com/gengo/grpc-gateway/examples/examplepb/echo_service.pb.gw.go:134 +0x66
created by github.com/gengo/grpc-gateway/examples/examplepb.RegisterEchoServiceHandler.func2
    /Users/cdo/gocode/src/github.com/gengo/grpc-gateway/examples/examplepb/echo_service.pb.gw.go:137 +0x19e

Examples break with 1.5 because of import of "main" examples package

can't load package: $GOPATH/src/github.com/gengo/grpc-gateway/examples/server/main.go:10:2: import "github.com/gengo/grpc-gateway/examples" is a program, not an importable package

Generate main func

protoc-gen-grpc-gateway should optionally generate the main function of the gateway server so that users don't need to write any golang codes unless they want to customize the entrypoint.

Should streaming endpoints produce text/event-stream content?

https://html.spec.whatwg.org/multipage/comms.html#server-sent-events

This would allow newer browsers to use the EventSource APIs cleanly.

NodeJS support

Hello gengo,

I'm very happy that you've addressed a problem which many of gRPC consumers will have. The internal use of gRPC is useful, fast and reliable. But serving RPCs for external use will be difficult.
This is where the RESTful API as hybrid solution with gRPC comes in.

My question is if this works already with NodeJS, I can't figure out how to implement it. If not, are you interested in implementing a solution for NodeJS as proxy.

Kind regards,
mastermix252

Generated .pb.go with services no longer works with latest version of grpc-go.

Support bytes fields in path parameter

Should decode percent encoding to support bytes fields