jazzycamel/micropython-uprotobuf

A lightweight implementation of Google's Protocol Buffers (protobuf) for micropython

micropython-uprotobuf

Project Status

This project is very much a work in progress and, as such, is incomplete. Currently protocol specifications can be compiled via the plugin and messages can be parsed.

Things To Do

• Handle repeats and groups

Linux Usage

It is assumed that Google Protobuf has been installed and the compiler (protoc) is on the $PATH. It is also assumed that a version of Python (preferably 3) is also available and the protobuf module has been installed (pip install protobuf).

Assuming you have a protocol specification file available (in this case called test1.proto) containing something like the following:

syntax = "proto2";

package test1;

message test1 {
  enum test1enum {
    ValueA=1;
    ValueB=2;
    ValueC=3;
  }

  required int32 a = 1;
  required string b = 2;
  required int64 c = 4;
  required float d = 5;
  required double e = 6;
  required test1enum f = 7 [default=ValueA];
  optional bool g = 8;
  repeated fixed32 h = 9;
  required sint32 i = 10;
  required sfixed64 j = 11;
}

then a micropython compatible module can be created using the uprotobuf plugin as follows:

$ git clone https://github.com/jazzycamel/micropython-uprotobuf.git
$ cd micropython-uprotobuf
$ chmod +x uprotobuf_plugin.py
$ protoc test1.proto --plugin=protoc-gen-custom=uprotobuf_plugin.py --custom_out=. test1.proto

Note: on Windows use

> protoc test1.proto --plugin=protoc-gen-custom=uprotobuf_plugin.bat --custom_out=. --proto_path=your\path\to\test1.proto test1.proto

This will generate a python module named test1_upb2.py, which can be imported and used by micropython, containing a class named Test1Message. This class can currently be used to parse binary messages as follows:

from test1_upb2 import Test1Message

message=Test1Message()
ok=message.parse("\x08\x96\x01")

Windows Usage

It is again assumed, as in "Linux Usage", that you will have installed protoc and placed it on the PATH, installed the protobuf Python module, and created the 'test1.proto' file.

A micropython compatible module can be created using the uprotobuf plugin as follows:

1. Download the code and change into the correct directory:

   git clone https://github.com/jazzycamel/micropython-uprotobuf.git
   cd micropython-uprotobuf

2. Create a file in this directory called 'uprotobuf_plugin.bat' with the following contents:

   @py <path_to_uprotobuf_plugin.py> 

   For example:

   @py C:\Users\me\Desktop\buf\uprotobuf_plugin.py

   Replace @py with the @ sign and then however you run the python command in CMD. This might be python or a filepath to the python executable.

3. Run the command

   protoc test1.proto --custom_out=. --plugin=protoc-gen-custom=<path_to_bat_file> test1.proto

   For example:

   protoc test1.proto --custom_out=. --plugin=protoc-gen-custom=C:\Users\me\Desktop\buf\micropython-uprotobuf\uprotobuf_plugin.bat test1.proto

Why do we need to do this? Modern versions of Protoc on Windows won't accept .py files for the plugin files but they will accept a batch script which triggers the Python file.

Command Explanation

--plugin=protoc-gen-custom=<path_to_plugin> means the protobuf generated code will be generated by a plugin with name "custom", which can be run by running <path to plugin>. The output of the plugin to generate the code is saved at the file indicated with --<plugin>_out where <plugin> is the name of the plugin, in this case custom.

Note

This plugin was written for protobuf version 2. Version 3 has not been tested and may or may not work.

Client/Server Example

In the test directory there are client and server scripts that demonstrate micropython encoding a message via micropython-uprotobuf, transmitting that message via TCP and then decoding the message via the standard Google protobuf python implementation.

First, run the server as follows:

$ cd tests/server
$ python3 server.py 

Then run the client as follows:

$ cd tests/client
$ micropython client.py

Both server and client use a protocol specification that can be found in tests/proto/tests.proto. There is also a script named generate.sh that is used to generate the python and micropython modules used by the example scripts.