googlecloudplatform/proto-gen-md-diagrams

A utility for generating Markdown documentation for Protocol Buffers that include Mermaid UML Diagrams.

Proto Gen MD Diagrams

This utility package is a compiled Go program that reads a protobuf source directory and generates Mermaid Diagrams in .md files in each directory, or the output directory with the given tree structure.

NOTE: Only Proto 3 syntax is supported.

This utility was created to ease documentation generation of complex Protobuf libraries to visualize models and services described in a Protocol buffers.

If you find this useful, awesome! If you find a bug, please contribute a patch, or open a bug. Please follow the Contributing guidelines.

Test Input and Output

Build and test

Using Native Go

go build && go test ./...
./proto-gen-md-diagrams -d ./pkg/proto/data/test/

Using Bazel

Since Bazel is CI/CD workflow program it compiles for all targets.

bazel build //... && bazel test //...

# Linux 
bazel-bin/proto-gen-md-diagrams-linux-x86_64

# OS X X64
proto-gen-md-diagrams-osx-x86_64

# OS X Apple Silicon
bazel-bin/proto-gen-md-diagrams-osx-arm64

# Windows
bazel-bin/proto-gen-md-diagrams-win-x86_64

Input File	Output File
Location Protobuf	Location Markdown
Location Service Protobuf	Location Service Markdown

Building

Go Lang

cd proto-gen-md-diagrams
// Build
go build && go test ./...

Use and Options

./proto-gen-md-diagrams -h

Usage of ./proto-gen-md-diagrams:
  -d string
        The directoryFlag to read. (default ".")
  -debugFlag
        Enable debugging
  -o string
        Specifies the outputFlag directoryFlag, if not specified, the processor will write markdown in the proto directories. (default ".")
  -r    Read recursively. (default true)
  -v    Enable Visualization (default true)
  -w    Enable writing output (default true)
  -md   Enable pure MD output (default false)

  
./proto-gen-md-diagrams -d test/protos

Quick Example

Protobuf Input

// A physical location that can be described with either an address
// or a set of geo coordinates.
message PhysicalLocation {
  // A postal address for the physical location.
  message Address {
    // Address type is used to identify the type of address.
    enum AddressType {
      RESIDENTIAL = 0; // A residential address
      BUSINESS = 1; // A business address
    }
    // First line of the address
    string line1 = 1;
    // Second line of the address
    string line2 = 2;
    // Third line of the address
    string line3 = 3;
    // The city or township
    string city = 4;
    // The state or province
    string state = 5;
    // The postal code
    string zipcode = 6;
    // The type of address
    AddressType type = 7;
    // Reserved for future use
    reserved 8 to 20;
  }
  // The timestamp the record was created
  google.protobuf.Timestamp created = 1;
  // The mailing address of the location
  Address address = 2;
  // Longitude degrees
  int32 longitude_degrees = 3 [json_name = 'lng_d'];
  // Longitude Minutes
  int32 longitude_minutes = 4 [json_name = 'lng_m'];
  // Longitude Seconds
  int32 longitude_seconds = 5 [json_name = 'lng_s'];
  // Longitude Degrees
  int32 latitude_degrees = 6  [json_name = 'lat_d'];
  // Latitude Minutes
  int32 latitude_minutes = 7  [json_name = 'lat_m'];
  // Latitude Seconds
  int32 latitude_seconds = 8  [json_name = 'lat_s'];
  // Latitude Direction Code
  string latitude_direction_code = 9  [json_name = 'lat_dir_code'];
  // Altitude in Meters
  double altitude_meters = 10  [json_name = 'alt_m'];
  // Additional Meta Data
  map<string, string> meta = 11;
  // Names for the location
  repeated string names = 12 [json_name = 'names'];
}

Markdown Output

Diagram

classDiagram
direction LR

%% A physical location that can be described with either an address or a set of geo coordinates.
class PhysicalLocation {
  + Address address
  + double altitude_meters
  + google.protobuf.Timestamp created
  + int32 latitude_degrees
  + string latitude_direction_code
  + int32 latitude_minutes
  + int32 latitude_seconds
  + int32 longitude_degrees
  + int32 longitude_minutes
  + int32 longitude_seconds
  + Map<string,  string> meta
  + List<string> names
}
PhysicalLocation --> `Address`
PhysicalLocation --> `google.protobuf.Timestamp`
PhysicalLocation --o `Address`

%% A postal address for the physical location.
class Address {
  + string line1
  + string line2
  + string line3
  + string city
  + string state
  + string zipcode
  + AddressType type
}
Address --> `AddressType`
Address --o `AddressType`
%% Address type is used to identify the type of address.
class AddressType{
  <<enumeration>>
  RESIDENTIAL
  BUSINESS
}

Loading

Description

FQN: test.location.PhysicalLocation

A physical location that can be described with either an address or a set of geo coordinates.

Field	Ordinal	Type	Label	Description
address	2	Address		The mailing address of the location
altitude_meters	10	double		Altitude in Meters
created	1	google.protobuf.Timestamp		The timestamp the record was created
latitude_degrees	6	int32		Longitude Degrees
latitude_direction_code	9	string		Latitude Direction Code
latitude_minutes	7	int32		Latitude Minutes
latitude_seconds	8	int32		Latitude Seconds
longitude_degrees	3	int32		Longitude degrees
longitude_minutes	4	int32		Longitude Minutes
longitude_seconds	5	int32		Longitude Seconds
meta	11	string, string	Map	Additional Meta Data
names	12	string	Repeated	Names for the location

proto-gen-md-diagrams's Issues

mermaid diagram was not generated correctly for some enum or with extensions

Issue 1:
Example of protobuf:

syntax = "proto3";

package foo;

option java_multiple_files = true;
option java_outer_classname = "FooProto";
option java_package = "foo";

message MessageA {
  // a
  enum ENUM_A {
    ENUM_A_UNSPECIFIED = 0;
    ENUM_A_1 = 1;
    ENUM_A_2 = 2;
  }
  // Type of the appliance.
  ENUM_A enum_a = 1;
  enum ENUM_B {
    ENUM_B_UNSPECIFIED = 0;
    ENUM_B_1 = 1;
    ENUM_B_2 = 2;
  }
  // b
  ENUM_B enum_b = 2;
  // bar
  oneof some {
    string t = 3;
    string tt = 4;
  }
  optional string bar = 5;
}

The generated diagram is this:

classDiagram
direction LR

%% 

class MessageA {
  + ENUM_A enum_a
  + ENUM_B enum_b
}
MessageA --> `ENUM_A`
MessageA --> `ENUM_B`
MessageA --o `ENUM_A`
%% a

class ENUM_A{
  <<enumeration>>
  ENUM_A_UNSPECIFIED
  ENUM_A_1
  ENUM_A_2
}MessageA --o `ENUM_B`
%% 

class ENUM_B{
  <<enumeration>>
  ENUM_B_UNSPECIFIED
  ENUM_B_1
  ENUM_B_2
}

Loading

Notice "MessageA --o ENUM_A" and "MessageA --o ENUM_B" that breaks the schema

Issue 2:
if the message includes extensions, the whole message is ignored.
Example:

syntax = "proto3";

package foo;

import "google/protobuf/descriptor.proto";

option java_multiple_files = true;
option java_outer_classname = "OptionProto";
option java_package = "foo";

message CustomOption {
  bool boo = 1;
}

extend google.protobuf.MessageOptions {
  optional CustomOption custom = 60000;
}

message MessageC {
  // ---------------- START State Definition ------------------- //
  option (foo.custom) = {
    boo: true
  };

  string a = 1;
}

The generated diagram for MessageC is like:

classDiagram
direction LR

%% 

class MessageC {
}

Loading

`-o` option is ignored

First of all, thanks a lot for this repo, that is exactly what we needed!

At the moment, it seems like the option -o, which should specify an output folder, is currently ignored by the script.

Sub-message outside of the message itself diagram generation issue

Hello, thank you for this initiative !

It more a question than an issue, if I generate, for example, the following protbuf diagram:

message Email {
  Address from = 1;
  Address to = 2;
  string subject = 3;
  string content = 4;
  message Address {
    string email = 1;
    string name = 2;
  }
}

I get this, and it is pretty cool:

But if I put the Address message outside of the Email message:

message Email {
  Address from = 1;
  Address to = 2;
  string subject = 3;
  string content = 4;
}
message Address {
  string email = 1;
  string name = 2;
}

I get this:

I loose the Address details.

We have a lot of commons messages, so we often are in this case.
Do you plan to manage this case? If no is there another way to do it or do you think it could be not too difficult to make this change?

Bzlmod Support

Wondering if there were any plans to also support Bzlmod workflows for this project?

binary distro

Do you think it would be a good add on a ci to create binary builds for different os as release artefacts?

comments are always wrapped in html

Due to that we have to use html syntax in the comments to use code blocks and link formatting, for example, rather than markdown syntax (which is a bit a counter sense given that we are generating markdown documentation, and that go documentation supports markdown syntax as far as i understand)