arshia001/fsharp.grpccodegenerator

A protoc plugin to enable generation of F# code + supporting libraries

fsharp.grpccodegenerator's Issues

How to achieve `Any.Pack/Unpack` from FSharp?

How do I create a Google.Protobuf.FSharp.WellKnownTypes.Any instance from another message?

In C# we use method static Google.Protobuf.WellKnownTypes.Any.Pack(IMessage) and Google.Protobuf.WellKnownTypes.Any.[Try]Unpack<T>(...)

Also, how to know the message type from the Any.TypeUrl?

Thanks for this libraries! I finally found some FSharp libraries to work with Protobuf/Grpc 🥳

Sample ASP.NET Core App?

I'm trying to create a small sample. However, I'm not able to get the server app to compile with:

/home/ryan/.nuget/packages/grpc-fsharp.tools/0.1.0/build/_protobuf/Google.Protobuf.Tools.targets(62,5): error MSB4062: The "Grpc.Tools.ProtoToolsPlatform" task could not be loaded from the assembly /home/ryan/.nuget/packages/grpc.tools/2.34.0/build/_protobuf/netstandard2.0/Grpc-FSharp.Tools.dll. Could not load file or assembly '/home/ryan/.nuget/packages/grpc.tools/2.34.0/build/_protobuf/netstandard2.0/Grpc-FSharp.Tools.dll'. The system cannot find the file specified. [/home/ryan/Code/fsgrpc/GrpcSample/GrpcSample.fsproj]
/home/ryan/.nuget/packages/grpc-fsharp.tools/0.1.0/build/_protobuf/Google.Protobuf.Tools.targets(62,5): error MSB4062:  Confirm that the <UsingTask> declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [/home/ryan/Code/fsgrpc/GrpcSample/GrpcSample.fsproj]

The client app fails to compile with:

/home/ryan/Code/grpc-fsharp-client-sample/GrpcClientSample/Program.fs(11,9): error FS0039: The value, namespace, type or module 'Greet' is not defined. [/home/ryan/Code/grpc-fsharp-client-sample/GrpcClientSample/GrpcClientSample.fsproj]
/home/ryan/Code/grpc-fsharp-client-sample/GrpcClientSample/Program.fs(14,11): error FS0039: The value, namespace, type or module 'Greet' is not defined. [/home/ryan/Code/grpc-fsharp-client-sample/GrpcClientSample/GrpcClientSample.fsproj]
/home/ryan/Code/grpc-fsharp-client-sample/GrpcClientSample/Program.fs(15,15): error FS0039: The record label 'Name' is not defined. [/home/ryan/Code/grpc-fsharp-client-sample/GrpcClientSample/GrpcClientSample.fsproj]
/home/ryan/Code/grpc-fsharp-client-sample/GrpcClientSample/Program.fs(18,9): error FS0072: Lookup on object of indeterminate type based on information prior to this program point. A type annotation may be needed prior to this program point to constrain the type of the object. This may allow the lookup to be resolved. [/home/ryan/Code/grpc-fsharp-client-sample/GrpcClientSample/GrpcClientSample.fsproj]
/home/ryan/Code/grpc-fsharp-client-sample/GrpcClientSample/Program.fs(20,18): error FS0072: Lookup on object of indeterminate type based on information prior to this program point. A type annotation may be needed prior to this program point to constrain the type of the object. This may allow the lookup to be resolved. [/home/ryan/Code/grpc-fsharp-client-sample/GrpcClientSample/GrpcClientSample.fsproj]

NOTE: I'm compiling on the CLI.

Do you happen to have a small sample to show how to correctly build the app?

Here are my initial attempts:

• https://github.com/panesofglass/grpc-fsharp-sample

Can you point out what I've missed?

Arm64 Linux support

Please could you support Arm64 Linux? (AWS Graviton etc.)

I could set PROTOBUF_PROTOC to apt-installed protoc but it's some extra effort to make sure to set it across multiple environments, multiple contexts (SSH, cron, sudo, machinectl, systemd service, docker, ...)

Not sure if Arm64 Mac is worth an explicit support when almost everyone can use Rosetta binary translator. But it's perhaps more popular than i386 Mac...

.NET 7 support

My life would be a little easier if you could release a binary that works with .NET 7 (or maybe, netstandard2.1, if possible? I'm not so knowledgeable with compatibility and TFM and whatnot...)

Release the protoc plugins artifacts

Hey! I'm really excited to see that you are working on gRPC for F# and I'm curious if you could also release the protoc plugins as release artifacts on the repository?

I'm currently using F# using Bazel instead of MSBuild and I would like to try creating a Bazel rule that could would use the protoc plugins to generate the F# files.

doc update

Hi, I'm learning to use this project, so please correct me if I'm wrong:
Should this example be updated according to this commit?

Generated enum type case names are not capitalized like originals

It seems that

enum MyEnum {
  MySingleEnumCase = 0;
}

would generate

type MyEnum =
| [<global.Google.Protobuf.Reflection.OriginalName("MySingleEnumCase")>] Mysingleenumcase = 0

with only the first letter capitalized.

I am glad to offer whatever assistance I can in finding and repairing this apparent bug. For now, I'll wait for reply since this repository does not seem to see much activity.

WellKnownTypes.Timestamp does not implement IComparable

Google.Protobuf implemented IComparable for WellKnownTypes.Timestamp. Could we do so as well?

Error received when a record type with the same name exists in a different namespace

I am receiving a number of error FS0656: This record contains fields from inconsistent types errors when I have a record type of the same name in another namespace.

namespace rec Grpc.Carrier.V1
// ...
type Carrier = {
    mutable _UnknownFields: global.Google.Protobuf.UnknownFieldSet
    mutable Id: ValueOption<string>
    mutable Name: ValueOption<string>
} with
    [<global.System.Diagnostics.DebuggerNonUserCodeAttribute>]
    member me.Clone() : Carrier = { // error FS0656: This record contains fields from inconsistent types        Carrier._UnknownFields = global.Google.Protobuf.UnknownFieldSet.Clone(me._UnknownFields)
        Carrier.Id = me.Id
        Carrier.Name = me.Name
    }
// ...
module Carrier =
    [<global.System.Diagnostics.DebuggerNonUserCodeAttribute>]
    let internal DefaultValue = { // error FS0656: This record contains fields from inconsistent types
        Carrier._UnknownFields = null
        Carrier.Id = ValueNone
        Carrier.Name = ValueNone
    }
    [<global.System.Diagnostics.DebuggerNonUserCodeAttribute>]
    let empty () = { // error FS0656: This record contains fields from inconsistent types
        Carrier._UnknownFields = null
        Carrier.Id = ValueNone
        Carrier.Name = ValueNone
    }
// ...

I have found that removing the Carrier. from each line inside Clone, DefaultValue and empty removes the error messages.

---

Elsewhere in my code:

namespace Carrier
open ConstrainedType
type Carrier = { Id: Id; Name: string }

and here is the proto file I was trying

syntax = "proto3";
package product.v1;
option csharp_namespace = "Grpc.Carrier.V1";

service PolicyService {
  rpc GetCarrier(GetCarrierRequest) returns (Carrier) {};
}
message GetCarrierRequest {
  string id = 1;
}
message Carrier {
  string id = 1;
  string name = 2;
}

How is a repeated field initialized on a send request?

I have tried the following - and all permutations - without success

message Rectangle {
int32 X = 1;
int32 Y = 2;
int32 Width = 3;
int32 Height =4;
}
message Word {
google.protobuf.Int32Value VocabularyId = 1;
bytes Strokes = 2;
string Text = 3;
string Confidence = 4;
Rectangle BoundingBox =5;
}

message SaveVocabularyRequest {
repeated Word Words =1 ;
}

This fails:

let h = words |> Seq.map InkWordToWord |> Seq.toList

{ Protocol.ProtoBuf.SaveVocabularyRequest.empty() with Words = { Words = {h}} }

Error Message: No Assignment given for field '_UnknownFields' of type 'Protocol.ProtoBuf.SaveVocabularyRequest

Thank you.

Protoc binaries seem to be different versions for different architectures

I got error : This file contains proto3 optional fields, but --experimental_allow_proto3_optional was not set. when trying to build a project using this dependency on a linux Docker container. Looking at the protoc directory, linux_x64 and macos_x86 both seem to be much older than the rest of them.

Can these be updated so the build passes on linux?

Thanks.

Including one proto file into another does not work

Having this project

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <TargetFramework>net5.0</TargetFramework>
        <GenerateDocumentationFile>true</GenerateDocumentationFile>
        <WarnOn>3390;$(WarnOn)</WarnOn>
    </PropertyGroup>

    <ItemGroup>
        <Protobuf Include="proto/unpacker.proto" GrpcServices="Both"/>
        <Protobuf Include="proto/task.proto"/>
    </ItemGroup>
    
    <ItemGroup>
      <PackageReference Include="Grpc-FSharp.Core" Version="0.1.1" />
      <PackageReference Include="Grpc-FSharp.Tools" Version="0.1.1">
        <PrivateAssets>all</PrivateAssets>
        <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      </PackageReference>
    </ItemGroup>
</Project>

I've got the following errors

0>/Users/vaskir/.nuget/packages/grpc-fsharp.tools/0.1.1/tools/macosx_x64/protoc --fsharp_out=obj/Debug/net5.0/proto/unpacker.proto --proto_path=/Users/vaskir/.nuget/packages/grpc-fsharp.tools/0.1.1/build/native/include --proto_path=. --dependency_out=obj/Debug/net5.0/dfe805b7589f203f_unpacker.protodep --error_format=msvs proto/unpacker.proto
0>task.proto: Error  : File not found.
0>proto/unpacker.proto(6,1): Error  : Import "task.proto" was not found or had errors.
0>proto/unpacker.proto(13,3): Error  : "Task" is not defined.
0>proto/unpacker.proto(17,3): Error  : "Task" is not defined.
0>------- Finished building project: Proto. Succeeded: False. Errors: 4. Warnings: 0

The editor shows the errors as well

Project is not conforming to the proto3 spec

I've been diving a bit deeper into the proto3 spec and it seems like wrapping scalars in options is not according to the spec.

Until recently Proto3 did not have any way to track presence for scalar types and therefore it is not possible to know if an int32 is set to 0 or if the value was not set at all since the default value for int32 is 0.

People worked around this by using the wrappers.proto well-known types: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/wrappers.proto
This allows proto generators to have a special case where if a type is wrapped in a well known wrapper type it will be unwraped and e.g. wrapped in an option instead.

Recently the optional keyword was also introduced in proto3 as a way to track presence. The wire protocol represents the optional fields as a single case oneof but generators can unwrap the oneof and represent the field in a way that is idiomatic to their programming language.

I propose that we make the following changes to the generator:

1. Non-optional scalars will no longer be wrapped in option since there is no way to know if a value was set intentionally or is just set to the default value.
2. Unwrap the optional fields single case union and represent them as a option type
3. (Optional) Add support for the well-known wrappers type

What do you think @Arshia001 ?

Implement comparison on UnknownFieldSet

I would like to be able to use Set operations on my generated message types, but this requires comparison. Could we implement comparison on UnknownFieldSet so F# can use structural comparison?

Is it possible to use this tool passing an URL

Hello, I want to use this tool in order to generate F# code from protobuf files. Btw, it would be cool to not need to download this protobuf file locally. Is there a way to do it packaged with this tool?

Generated code does not work with JsonTranscoding

Get the following error when starting the app with JSON Transcoding enabled in .net 7:

warn: Microsoft.AspNetCore.Grpc.JsonTranscoding.Internal.Binding.JsonTranscodingServiceMethodProvider[2]
      Error getting service descriptor for BarServer.BarServerService.
      System.InvalidOperationException: Get not find Descriptor property on FooServiceMethodBinder.
         at Grpc.Shared.ServiceDescriptorHelpers.GetServiceDescriptor(Type serviceReflectionType)
         at Microsoft.AspNetCore.Grpc.JsonTranscoding.Internal.Binding.JsonTranscodingServiceMethodProvider`1.OnServiceMethodDiscovery(ServiceMethodProviderContext`1 context)

This seems like it's related to some missing generated Descriptor properties.

Docs should note dotnet-sdk-2.1 requirement

I has some initial trouble getting started, and at least some of the issues were because I didn't have dotnet-sdk-2.1 installed. After installing that, protoc-gen-fsharp seemed to work without throwing an error. Would you mind adding that to the README?

Plugin fails when proto3 fields are marked as optional

The F# plugin fails when it encounters an proto3 file that contains the optional keyword. This keyword was recently to the proto3 spec. More information about it here: https://github.com/protocolbuffers/protobuf/blob/master/docs/implementing_proto3_presence.md

My guess is that this does not really affect the output of the F# plugin since it it is not outputting nullable fields?

Specify MergeFrom behavior using FieldMask.MergeOptions

Is it currently possible to use FieldMask.MergeOptions to modify the behavior of the MergeFrom members generated? I am particularly interested in ReplaceRepeatedFields.

Works on ionide?

I trying to run https://github.com/panesofglass/grpc-fsharp-sample/ and create a simple using the tutorial, but I have
The value, namespace, type or module 'Greeter' is not defined. F# Compiler(39) on ionide

Seeming error with generated code when using Goole API Annotations

When using a proto file with Goole API Annotations for a gateway I get an error about a function that cannot be applied:

.../Grpc/Health.fs(28,25): error FS0003: This value is not a function and cannot be applied. .../Product/Product.fsproj]

snippet of the generated code:

global.System.Lazy<_>(
            (fun () ->
                global.Google.Protobuf.Reflection.FileDescriptor.FromGeneratedCode(
                    descriptorData,
                    [|
                        global.Google.Api.AnnotationsReflection.Descriptor() // this is like 28, the line that is generating the error
                    |],
                    new global.Google.Protobuf.Reflection.GeneratedClrTypeInfo(
                        null,
                        null,
                        [|
                            new global.Google.Protobuf.Reflection.GeneratedClrTypeInfo(typeof<global.Grpc.Health.V1.HealthCheckRequest>, global.Grpc.Health.V1.HealthCheckRequest.Parser, [| "Service" |], null, null, null, null)
                            new global.Google.Protobuf.Reflection.GeneratedClrTypeInfo(typeof<global.Grpc.Health.V1.HealthCheckResponse>, global.Grpc.Health.V1.HealthCheckResponse.Parser, [| "Status" |], null, [| typeof<global.Grpc.Health.V1.HealthCheckResponse.Types.ServingStatus> |], null, null)
                        |]
                    )
                )
            ),
            true
        )

Seems if I remove the () from global.Google.Api.AnnotationsReflection.Descriptor() that particular error goes away, not sure if it's generating any other problems as I'm not using the dotnet version of the gateway at this time.

I used the following proto file for this example:

syntax = "proto3";

package grpc.health.v1;

import "google/api/annotations.proto";

option csharp_namespace = "Grpc.Health.V1";
option go_package = "google.golang.org/grpc/health/grpc_health_v1";

message HealthCheckRequest {
  string service = 1;
}

message HealthCheckResponse {
  enum ServingStatus {
    UNKNOWN = 0;
    SERVING = 1;
    NOT_SERVING = 2;
    SERVICE_UNKNOWN = 3;  // Used only by the Watch method.
  }
  ServingStatus status = 1;
}

service Health {
  rpc Check(HealthCheckRequest) returns (HealthCheckResponse) {
    option (google.api.http).post = "/health/status";
  };
}

Trying to use the message formatter throws "Unable to format value of type Microsoft.FSharp.Core.FSharpValueOption`1" for optional types

This happens when trying to use the JsonFormatter for types with any optional fields.

Simple proto file:

message TestMessage {
  optional string TestField = 1;
}

Simple test:

let x = { TestMessage.empty() with TestField = ValueSome "hello" }
let string = Google.Protobuf.JsonFormatter.Default.Format(x)

Result:

  Message: 
System.ArgumentException : Unable to format value of type Microsoft.FSharp.Core.FSharpValueOption`1[System.String]

  Stack Trace: 
JsonFormatter.WriteValue(TextWriter writer, Object value)
JsonFormatter.WriteMessageFields(TextWriter writer, IMessage message, Boolean assumeFirstFieldWritten)
JsonFormatter.WriteMessage(TextWriter writer, IMessage message)
JsonFormatter.Format(IMessage message, TextWriter writer)
JsonFormatter.Format(IMessage message)

Find a way to let the tool work without a global tool installation

I'd rather not have the users of this library need to install a global tool. For one thing, it's impossible to make a consistent version of the tool available on the machines of all developers of a project. Then there's the fact that global tool installation complicates project setup and Dockerfiles.

The global tool is needed because a locally installed tool cannot be run as a stand-alone executable, instead needing to be called via the dotnet command line, e.g. dotnet protoc-gen-fsharp instead of protoc-gen-fsharp.

One way to work around this limitation would be to include the plugin with the Tools package. We're already checking the user's OS and calling the correct version of protoc; if we had access to the plugin's executables for each platform, we could pass it in to the compiler with --plugin=protoc-gen-fsharp=....

To do this, we need to compile and publish the plugin for each of the 3 main OS's, and include the published binaries in the final Tools package. However, this is further complicated by the fact that we don't know the user's runtime environment in advance, so we can't decide on one framework to publish for, so we'd have to bundle for multiple runtimes on multiple OS's, which will grow the Tools package's size even further.

We could also forgo the platform-specific binaries, and go with shell scripts instead:

#! /usr/bin/env bash

dotnet_version=$(dotnet --version | extract-version-in-some-way)
dotnet ../$dotnet_version/FSharp.GrpcCodeGenerator.dll

However, this creates an additional dependency on the existence of specific shells on the user's system, which is undesirable.