Giter Site home page Giter Site logo

dhiguero / grpc-proto-manager Goto Github PK

View Code? Open in Web Editor NEW
0.0 2.0 0.0 67 KB

GPM: A tool to orchestrate the generation of gRPC protos on different languages and repos

License: Apache License 2.0

Makefile 4.03% Go 62.86% Dockerfile 9.85% Shell 23.26%
protobuf grpc-protos grpc generator

grpc-proto-manager's Introduction

grpc-proto-manager

Generating gRPC protos can be a critical tasks for development. However, automating this task is usually done in a non-reusable, script-based and pipelined way for CI integration. This project aims to provide an easy-to-use tool to generate a collection of protos assuming a simple structure:

<my_base_path>/myservice -> github.com/my_user/grpc-myservice-<target_language>

Layout structure

The tool relies in a simple directory structure to create the protos.

  1. Independently of whether proto definitions are stored on a mono-repo or are spread accross different repositories, the tool expects a directory with the name of high-level entity that is associated with the protos. For example, if a microservice is named login, or a high-level entity in your system is user the tool assumes you are storing protos definitions (e.g., entities.proto & services.proto) in a directory named login or user respectively.

  2. Target repos must exist on your account. This may be addressed for now, but for now, your administrator should create a repo named grpc-<high_level_entity>-<target_language> that will store the generated code.

  3. To specify the target languages use a file named .protolang inside each directory. Be aware that this has been tested for now for Golang, other languages may not work :).

Can I see a working example?

Yes! For a working example, check the following repositories:

  • gpm-example: Simple monorepo with the layout structure expected by GPM. It contains two protobuf services: ping and agenda.
  • grpc-agenda-go: Repository containing the golang generated code for the agenda service.
  • grpc-ping-go: Repository containing the golang generated code for the ping service.
$ ./bin/darwin/gpm generate /<full_path>/gpm-example
9:03PM INF configuration loaded path=/<full_path>/gpm-example/.gpm.yaml
9:03PM DBG Launching GPM
9:03PM INF app config commit=0465ed57eeb8a3e890c33b1c23ac8467ad1be365 version=v0.0.1
9:03PM INF Paths Project=/<full_path>/gpm-example Temp=/tmp/gpm
9:03PM INF Providers Repository=github generator=docker
9:03PM INF Defaults Language=go
9:03PM INF generated code repository URL=dhiguero
9:03PM INF processing proto directory path=/<full_path>/gpm-example/agenda
9:03PM INF publishing new version newVersion=v0.1.0 repo=grpc-agenda-go
9:03PM INF processing proto directory path=/<full_path>/gpm-example/ping
9:03PM INF publishing new version newVersion=v0.1.0 repo=grpc-ping-go

Installation

Generating protos

There are different methods to execute GPM and generate protobuf stubs from CLI to docker environments.

Using the CLI

To use the CLI, clone the repo and execute:

$ make build
$ ./bin/darwin/gpm generate <your_protorepo_path>

For linux OS, use ./bin/linux/gpm.

Using a docker container

A docker container is also available so it is easier to generate the protos from a local machine:

$ docker run -v <your_protorepo_path>:/defs -v /<your_user_home>/.ssh:/root/.ssh dhiguero/gpm:v0.0.3

By using this approach, the protorepo is mounted on the /defs directory of the docker image which corresponds to the default location for the generation of protos. Currently, ssh keys are required since this method clones repos through SSH and the public key is required for this to work. An alternative using GitHub Personal Access Tokens will be available through the GPM GitHub Action or by executing the CLI passing --personalAccessToken.

Integration with GitHub Actions

In the works :) Stay tunned.

  • Add a new GitHub action provider that makes use of HTTPS clonning
  • Modify the dockerfile if needed to execute from the current working directory independenly of the expected one. GitHub sets its own.
  • The new GitHub provider may need to set the pusher data

Roadmap/TODO

  • Add tests
  • Improve parametrization
  • Brew installation
  • Extract the project to a single GitHub organization :)
  • Improve overall documentation abouth protos and GPM

Inspiration

Organizing and managing a protobuf repository is something that may developers and companies face, some of the resources that inspired this tool are:

License

Copyright 2020 Daniel Higuero.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

grpc-proto-manager's People

Contributors

dhiguero avatar

Watchers

 avatar  avatar

grpc-proto-manager's Issues

Dockerize GPM

The current implementation only supports a dockercmd provider that executes several docker run commands to generate the protos using an open source third party docker image. It will be useful for some scenarios such as CI pipelines to provide a single docker image that embeds both the GPM command and the required gRPC generation helpers.

Pending tasks:

  • Analyze the third party image to determine integration points.
  • Create a base docker file embedding GPM from the previous one.
  • Define a new entrypoint for the image and document require environment variables and mounting paths.
  • Implement new proto generator implementation

Provide a brew formula

To improve the usability of GPM for users wanting to install it as a command, provide a brew formula to easy the installation steps. The current method involves cloning the repo, executing make build, and moving/linking the resulting binaries.

Integrate with GitHub actions for PR

  • Integrate the project with a CI/CD system to check the PR and maintain a minimum level of quality :)
  • Compile code
  • Pass the tests
  • Report coverage
  • Publish docker image when a PR is merged

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.