bufbuild/protoschema-plugins

Protobuf plugins that generate various schemas from protobuf files - JSON Schema, PubSub, etc.

protoschema-plugins

The protoschema-plugins repository contains a collection of Protobuf plugins that generate different types of schema from protobuf files. This includes:

• PubSub
• JSON Schema

PubSub Protobuf Schema

Generates a schema for a given protobuf file that can be used as a PubSub schema in the form of a single self-contained messaged normalized to proto2.

Install the protoc-gen-pubsub plugin directly:

go install github.com/bufbuild/protoschema-plugins/cmd/protoc-gen-pubsub@latest

Or reference it as a Remote Plugin in buf.gen.yaml:

version: v1
plugins:
  - plugin: buf.build/bufbuild/protoschema-pubsub
    out: ./gen

For examples see testdata which contains the generated schema for test case definitions found in proto.

JSON Schema

Generates a JSON Schema for a given protobuf file. This implementation uses the latest JSON Schema Draft 2020-12.

Install the protoc-gen-jsonschema directly:

go install github.com/bufbuild/protoschema-plugins/cmd/protoc-gen-jsonschema@latest

Or reference it as a Remote Plugin in buf.gen.yaml:

version: v1
plugins:
  - plugin: buf.build/bufbuild/protoschema-jsonschema
    out: ./gen

For examples see testdata which contains the generated schema for test case definitions found in proto.

Here is a simple generated schema from the following protobuf:

message Product {
  message Location {
    float lat = 1;
    float long = 2;
  }

  int32 product_id = 1;
  string product_name = 2;
  float price = 3;
  repeated string tags = 4;
  Location location = 5;
}

Results in the following JSON Schema files:

Product.schema.json

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "product_id": {
      "type": "integer"
    },
    "product_name": {
      "type": "string"
    },
    "price": {
      "type": "number"
    },
    "tags": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "location": {
      "type": "object",
      "properties": {
        "lat": {
          "type": "number"
        },
        "long": {
          "type": "number"
        }
      },
      "required": ["lat", "long"]
    }
  },
  "required": ["product_id", "product_name", "price", "tags", "location"]
}

Product.Location.schema.json

{
  "$id": "Product.Location.schema.json",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "additionalProperties": false,
  "properties": {
    "lat": {
      "anyOf": [
        {
          "type": "number"
        },
        {
          "type": "string"
        },
        {
          "enum": ["NaN", "Infinity", "-Infinity"],
          "type": "string"
        }
      ]
    },
    "long": {
      "anyOf": [
        {
          "type": "number"
        },
        {
          "type": "string"
        },
        {
          "enum": ["NaN", "Infinity", "-Infinity"],
          "type": "string"
        }
      ]
    }
  },
  "type": "object"
}

Community

For help and discussion around Protobuf, best practices, and more, join us on Slack.

Status

This project is currently in alpha. The API should be considered unstable and likely to change.

Legal

Offered under the Apache 2 license.

protoschema-plugins's Issues

Overgenerates and doesn't respect paths=source_relative

Testing this next to protoc-gen-bq-schema, there were two initial problems that kept me from moving forward:

1. I expected schemas to be opt-in, and only generated when an option was set
   option (gen_bq_schema.bigquery_opts).table_name = "bar_table";
2. Everything was generated to the same directory, not to the expected destination directories where all my other plugins generate code

buf.yaml

  - name: bq-schema
    out: gen/go
    opt: paths=source_relative

  - name: bigquery # v0.1.2
    out: gen/go
    opt: paths=source_relative

Ability to override additional properties

I may want to make an incomplete schema, allowing me to override additional properties on a message level would be appreciated.

Comparison to protoc-gen-bq-schema

How does this compare to github.com/GoogleCloudPlatform/protoc-gen-bq-schema, both in its current state and end goals?

generate proto from bigquery schema

We're wanting to migrate all our schema interactions to happen proto first, but that's tough because we have lots of BigQuery schemas with no matching proto. It would help us if we could provide the schema json, and use that to create the proto scaffold. In most cases, I think that's a one-time translation, but I think there are also use cases if you don't control a schema, where it would be convenient to keep the proto in sync as the upstream BigQuery schema changes.

Generate JSON schema into a single file

Hi! I'm currently using https://github.com/chrusty/protoc-gen-jsonschema and want to migrate to this plugin instead. I host the generated schema in my own repository and refer to it from https://github.com/SchemaStore/schemastore. My understanding is that if I switch to this plugin, I'll not be able to refer to the schema in my repo anymore (see this).

Would it be possible to add support for putting the generated schema into a single file? As part of this, I suppose the user should be able to chose for which messages the schema is generated (with all the dependent messages pulled in).