bufbuild/modules

Collection of third-party modules managed and synced by Buf.

modules

Description

This repository contains first and third-party modules synced and published to the Buf Schema Registry.

If you'd like a common third-party module to be managed by Buf, open an issue using the Managed Module Request for Buf Schema Registry issue template and our team will follow up.

Managed modules

We currently sync automatically the following modules:

Module	Source Community Repository	Depends on
bufbuild/confluent	https://github.com/bufbuild/confluent-proto
bufbuild/protovalidate	https://github.com/bufbuild/protovalidate
bufbuild/protovalidate-testing	https://github.com/bufbuild/protovalidate	- bufbuild/protovalidate
bufbuild/reflect	https://github.com/bufbuild/reflect
cncf/xds	https://github.com/cncf/xds	- envoyproxy/protoc-gen-validate - google/cel-spec - googleapis/googleapis
envoyproxy/envoy	https://github.com/envoyproxy/envoy	- cncf/xds - envoyproxy/protoc-gen-validate - googleapis/googleapis - opencensus/opencensus - opentelemetry/opentelemetry - prometheus/client-model
envoyproxy/protoc-gen-validate	https://github.com/envoyproxy/protoc-gen-validate
gogo/protobuf	https://github.com/gogo/protobuf
google/cel-spec	https://github.com/google/cel-spec	- googleapis/googleapis
googleapis/googleapis	https://github.com/googleapis/googleapis
googlechrome/lighthouse	https://github.com/GoogleChrome/lighthouse
googlecloudplatform/bq-schema-api	https://github.com/GoogleCloudPlatform/protoc-gen-bq-schema
grpc/grpc	https://github.com/grpc/grpc-proto	- googleapis/googleapis
grpc-ecosystem/grpc-gateway	https://github.com/grpc-ecosystem/grpc-gateway
opencensus/opencensus	https://github.com/census-instrumentation/opencensus-proto
opentelemetry/opentelemetry	https://github.com/open-telemetry/opentelemetry-proto
prometheus/client-model	https://github.com/prometheus/client_model
protocolbuffers/gofeatures	https://github.com/protocolbuffers/protobuf-go	- protocolbuffers/wellknowntypes
protocolbuffers/wellknowntypes	https://github.com/protocolbuffers/protobuf

How we handle dependencies

Dependencies are an essential part of these community modules as they help developers reuse well known Protobuf types, reduce errors and speed up the development process. We do not control the source of these modules, and managing and pinning dependencies to their exact commit can be difficult, especially when multiple module sources and build systems are involved.

To minimize issues with pinned dependencies on these modules, we sync them in the following order. First, we sync standalone modules. After they succeed, we then sync the modules that depend on them, which use the latest pushed dependency commit. As long as the dependencies don’t have any breaking change in the source code, this should be sufficient and stable for upstream modules.

Community

For help and discussion regarding Protobuf managed modules, join us on Slack.

For feature requests, bugs, or technical questions, email us at [email protected].

Legal

Offered under the Apache 2 license.

modules's Issues

Unexpected Fetch references failure

The automatic Fetch references workflow failed, please investigate:
https://github.com/bufbuild/modules/actions/runs/7819224455/attempts/1

Add managed module `bufbuild/reflect`

Mandatory

Where is the source code for the Managed Module?

https://github.com/bufbuild/reflect-proto

Do the proto files declare a package?

Yes, already pushed via buf-push-action to BSR https://buf.build/bufbuild/reflect

Optional

Does this module have other module dependencies/imports?

No

Based on the repository's release process, do you prefer syncing by releases (semver) or by git commit SHA?

git commit SHA

Additionally, is there a sensible initial reference to sync from?

TBD

Add managed module: `googlecloudplatform/bq-schema-api`

Mandatory

Where is the source code for the Managed Module?

https://github.com/GoogleCloudPlatform/protoc-gen-bq-schema

bq_field.proto
bq_table.proto

They also seem to have a ./protos directory (need to double-check if this is duplicated for a reason). This is the generated location for .pb.go (i.e., more like ./gen directory), ignore.

Do the proto files declare a package?

Yes.

package gen_bq_schema;

Optional

Does this module have other module dependencies/imports?

No.

Based on the repository's release process, do you prefer syncing by SemVer releases or by git commit SHA?

git commit SHA. There is no sensible release process or up-to-date tags.

Additionally, is there a sensible initial reference to sync from?

Latest is fine.

https://github.com/GoogleCloudPlatform/protoc-gen-bq-schema/tree/31a7e43419f7c19d79de0bb506798bc602287e80

Do you think this module is widely used by the community, and is not already provided on the BSR by the owners?

Yes, it's been requested a few times via other OSS issues, in the bufbuild/plugins repository and there are a few community modules already published to the BSR.

Unexpected Fetch references failure

The automatic Fetch references workflow failed, please investigate:
https://github.com/bufbuild/modules/actions/runs/7876905087/attempts/1

Add managed module: `envoyproxy/ratelimit`

Not ready to open an issue, but want to chat about your module? Come find us on our Public Slack channel:

https://buf.build/links/slack

Mandatory

Where is the source code for the Managed Module?

• https://github.com/envoyproxy/ratelimit

Do the proto files declare a package?

• Yup

Optional

Does this module have other module dependencies/imports?
Yup, envoy/service/discovery/v3/discovery.proto: https://github.com/envoyproxy/ratelimit/blob/main/api/ratelimit/service/ratelimit/v3/rls_conf_ds.proto#L5C1-L5C53

Based on the repository's release process, do you prefer syncing by SemVer releases or by git commit SHA?

• git commit SHA

Additionally, is there a sensible initial reference to sync from?

• Can we sync from the Github repo? https://github.com/envoyproxy/ratelimit/blob/main/api/ratelimit/config/ratelimit/v3/rls_conf.proto

Along with a sync method, please propose a reference to commence syncing from. Consider commonly used versions and whether there are major breaking changes across versions that segment the user base.

Do you think this module is widely used by the community, and is not already provided on the BSR by the owners?
Yup, the envoyproxy ratelimit library seems relatively popular. It also doesn't need to be updated too often

Unsure? Let's chat! https://buf.build/links/slack

Add bufbuild/confluent managed module

Mandatory

Where is the source code for the Managed Module?
https://github.com/bufbuild/confluent-proto

Do the proto files declare a package?
Yes - buf.confluent.v1.

Optional

Does this module have other module dependencies/imports?

No.

Based on the repository's release process, do you prefer syncing by releases (semver) or by git commit SHA?

Syncing git commit SHA (with labels for releases).

Additionally, is there a sensible initial reference to sync from?
bufbuild/confluent-proto@18c83a3

Do you think this module is widely used by the community, and is not already provided on the BSR by the owners?
Not yet!

Move `envoyproxy/envoy` to sync semver releases

The source repository https://github.com/envoyproxy/envoy does SemVer releases, it can be moved to sync them instead of individual git commits.

We need to wait for the next MINOR release (v1.31.0) to avoid pushing breaking changes to the BSR, since we're currently syncing the main branch, which already includes some new proto fields after the latest MINOR release (v.30.0).

Skip invalid references

Related to #416

There will be scenarios in which we'll find invalid commits in the main branch, or even in some semver releases, that we cannot process because of a failure of syntax or other cause that makes the module unbuildable.

We need a way to support "skippable references" per module, that allows continue managing the module while skipping bad commits.

Add managed module: `buf.build/google/gnostic`

Mandatory

Where is the source code for the Managed Module?

https://github.com/google/gnostic

Specifically the openapiv3 directory, but also maybe the openapiv2 directory.

Do the proto files declare a package?

Yes, but not a great one: openapi.v3. Doesn't line up great with https://github.com/grpc-ecosystem/grpc-gateway/blob/main/protoc-gen-openapiv2/options/openapiv2.proto

Optional

Does this module have other module dependencies/imports?

No.

Based on the repository's release process, do you prefer syncing by SemVer releases or by git commit SHA?

SemVer releases.

Additionally, is there a sensible initial reference to sync from?

0.4.1 - it's the first one where the subdirectory is named openapiv3.

Do you think this module is widely used by the community, and is not already provided on the BSR by the owners?

Likely yes.

Other notes

• Just a proposal.
• Not sure how this fits in with other managed modules that potentially push OpenAPI API definitions.
• Note that gnostic also has an openapi.v2 package in openapiv2 - should probably be pushing this up as well, but note grpc-gateway has definitions as well.

Managed Module request for Buf Schema Registry (grpc health service)

Mandatory

Where is the source code for the Managed Module?

https://github.com/grpc/grpc-proto/tree/master

My current need is only https://github.com/grpc/grpc-proto/blob/master/grpc/health/v1/health.proto but not sure if makes sense to manage the entire thing.

Do the proto files declare a package?

Yes. package grpc.health.v1.

Optional

Does this module have other module dependencies/imports?

Nope.

Based on the repository's release process, do you prefer syncing by releases (semver) or by git commit SHA?

Git commit SHA.

Additionally, is there a sensible initial reference to sync from?

HEAD should be fine. If not, 6565a1ba38af695ace7c3ce6e6ff837ee87d4c10 is the one I'm explicitly using (HEAD~1).

Do you think this module is widely used by the community, and is not already provided on the BSR by the owners?

Unsure how wide-spread it is but I doubt we're the only ones using this service definition for health checking.
Ref docs: https://github.com/grpc/grpc/blob/master/doc/health-checking.md

Add managed module `planetscale/vitess`

Mandatory

Where is the source code for the Managed Module?

https://github.com/planetscale/vitess-types

Do the proto files declare a package?

Yes, example https://github.com/planetscale/vitess-types/blob/main/src/vitess/vttime/v18/vttime.proto#L22

Optional

Does this module have other module dependencies/imports?

Based on the repository's release process, do you prefer syncing by releases (semver) or by git commit SHA?

Additionally, is there a sensible initial reference to sync from?

Do you think this module is widely used by the community, and is not already provided on the BSR by the owners?

Unsure? Let's chat!

add GoogleChrome/lighthouse module

Mandatory

Where is the source code for the Managed Module?

https://github.com/GoogleChrome/lighthouse/blob/main/proto/lighthouse-result.proto

Optional

Does this module have other module dependencies/imports?

import "google/protobuf/struct.proto";
import "google/protobuf/timestamp.proto";
import "google/protobuf/wrappers.proto";

Do you think this module is widely used by the community, and is not already provided on the BSR by the owners?

Not sure how many people are interacting with the Lighthouse proto directly, but in general, demand for headless Chrome via Puppeteer and other methods is growing.

Unexpected Fetch references failure

The automatic Fetch references workflow failed, please investigate:
https://github.com/bufbuild/modules/actions/runs/8900177501/attempts/1

Module cncf/xds is not syncing: new dependency?

See https://github.com/bufbuild/modules/actions/workflows/fetch.yaml failed the last 3 times.

When processing reference cncf/xds:0f5e0d9dbc1270b5eddd1be47fa8b2aea72f9a1f, buf build fails because:

+ buf build
xds/type/v3/cel.proto:7:8:cel/expr/checked.proto: does not exist

That import checked.proto was added here, and then reverted here (although the PR says it will be reintroduced). Following now this issue: cncf/xds#76

In the PR that introduced it, they also add "@dev_cel//proto/cel/expr:checked_proto", to the BUILD bazel rules (which assume it translates to a new dependency that should go in the buf.yaml?)

I suspect that dep comes from here: https://github.com/google/cel-spec, although in that repo I don't see any checked.proto file.

Should we skip this commit? or make sure the dependency is synced before trying to sync cncf/xds again?

I'd appreciate your input here @rodaine @pkwarren

Move `googlecloudplatform/bq-schema-api` to sync semver releases

The source repository https://github.com/GoogleCloudPlatform/protoc-gen-bq-schema does SemVer releases, it can be moved to sync them instead of individual git commits.

We need to wait for the next release to avoid pushing breaking changes to the BSR, since we're currently syncing the main branch, which already includes some new proto fields after the latest release.

Increase the set of synced apis from googleapis/googleapis in the bsr

Mandatory

I am looking to use a wider set of the googleapis/googleapis that are synced in the bsr. Currently the bsr includes a subset of:

https://github.com/googleapis/googleapis/tree/master/google

Specifically resources within cloud/kms/v1 are omitted:
https://github.com/googleapis/googleapis/blob/master/google/cloud/kms/v1/resources.proto#L636

The package in particular is google/cloud/kms/v1. I would like to include some of these resources within our .proto types in order to return them via our gRPC service(s).

For example (PublicKey):

message GetPublicKeyResponse {
  google.cloud.kms.v1.PublicKey public_key = 1;
}

Optional

Does this module have other module dependencies/imports?

Yes - it has quite a few dependencies. The good news is these are all already included in the bsr except one:

• google/api (already included)
• google/longrunning (already included)
• google/rpc (already included)
• google/logging (missing)

Some of these googleapis/googleapis are already synced. The suggestion would be to sync this wider set in the same manner.

I think anyone who wants to include these resources within their services within the wider community would find value in having these available on the bsr.

support full googleapis/googleapis

Mandatory

Where is the source code for the Managed Module?

https://github.com/googleapis/googleapis/tree/master/google

I'm aware of the decision to slim down the googleapis module, which still feels like a reasonable decision for the general use case.
https://docs.buf.build/faq#googleapis-failure

I think there still is enough usage of other Google Cloud / Workspace apis that having a separate module with all the dependencies makes sense.