black-06/grpc-gateway-file

file upload and download in gRPC-Gateway

gRPC-Gateway-file

Are you confused about how to integrate file upload and download into gRPC-Gateway?

Try gRPC-Gateway-file, it is a plugin of the gRPC-Gateway.

It allows you to define upload and download api in the gRPC service proto file.

Feat

• For upload, you no longer have to manually add routes to the mux
• For download, It supports Resume Transfer Protocol.
• Best of all, you can implement them directly in gRPC service.

Usage

0. Get it.

go get -u github.com/black-06/grpc-gateway-file

1. Defining gRPC proto file.
   The result of download api, and the request of upload api, must be "stream google.api.HttpBody"

import "google/api/annotations.proto";
import "google/api/httpbody.proto";

service Service {
  rpc DownloadFile (XXX) returns (stream google.api.HttpBody) {
    option (google.api.http) = { get: "/api/file/download" };
  };

  rpc UploadFile (stream google.api.HttpBody) returns (XXX) {
    option (google.api.http) = { post: "/api/file/upload", body: "*" };
  };
}

2. Generate golang code as usual.
3. Using gRPC-Gateway-file in gRPC-Gateway.

import gatewayfile "github.com/black-06/grpc-gateway-file"

mux := runtime.NewServeMux(
	gatewayfile.WithFileIncomingHeaderMatcher(),
	gatewayfile.WithFileForwardResponseOption(),
	gatewayfile.WithHTTPBodyMarshaler(),
)

4. Done, enjoy it

A more complete example is here

Known issues

HTTPBodyMarshaler will change the Delimiter of all server-stream to empty.

Be careful if you have other server streams api.

More context see grpc-ecosystem/grpc-gateway#2557

grpc-gateway-file's Issues

The audio files are getting distorted afterd download, resulting in different md5sum

When I attempt to download an MP3 audio file, it becomes corrupted, exhibiting distortion noises. Upon checking the md5 checksums of the files, I found them to be different.

Help wanted: "stream terminated by RST_STREAM with error code: PROTOCOL_ERROR"

If deploy example into kubernetes with linkerd, we got "stream terminated by RST_STREAM with error code: PROTOCOL_ERROR" error if we try to download file.

Maybe you know how to fix it?

WithHTTPBodyMarshaler & UnmarshalOptions

Hi! In WithHTTPBodyMarshaler you're using DiscardUnknown=true. This causes grpc_gateway to stop checking the enums. Is there any need for this option?

Problem with processing two or more files

Hello. When processing two or more files I get an error: "parse multipart form failed multipart: NextPart: EOF". Could you please tell me if it is possible to process two or more files? In your MultipartFormHeader method I am not sure about the logic and the possibility of working with multiple files. This library is convenient, but if there is a possibility, please clarify this point. Thank you.

func MultipartFormHeader(server UploadServer, key string) (*multipart.FileHeader, error) {

   form, err := ParseMultipartForm(server)
   if err != nil {
   	return nil, errors.Wrapf(err, "parse multipart form failed")
   }
   if headers := form.File[key]; len(headers) > 0 {
   	return headers[0], nil
   }
   return nil, http.ErrMissingFile

}