cloudwu/pbc

A protocol buffers library for C

PBC

PBC is a google protocol buffers library for C without code generation.

Quick Example

package tutorial;

message Person {
  required string name = 1;
  required int32 id = 2;        // Unique ID number for this person.
  optional string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    required string number = 1;
    optional PhoneType type = 2 [default = HOME];
  }

  repeated PhoneNumber phone = 4;
}

struct pbc_rmessage * m = pbc_rmessage_new(env, "tutorial.Person", slice);
printf("name = %s\n", pbc_rmessage_string(m , "name" , 0 , NULL));
printf("id = %d\n", pbc_rmessage_integer(m , "id" , 0 , NULL));
printf("email = %s\n", pbc_rmessage_string(m , "email" , 0 , NULL));

int phone_n = pbc_rmessage_size(m, "phone");
int i;

for (i=0;i<phone_n;i++) {
	struct pbc_rmessage * p = pbc_rmessage_message(m , "phone", i);
	printf("\tnumber[%d] = %s\n",i,pbc_rmessage_string(p , "number", i ,NULL));
	printf("\ttype[%d] = %s\n",i,pbc_rmessage_string(p, "type", i, NULL));
}

pbc_rmessage_delete(m);

Message API

You can use wmessage for encoding , and rmessage for decoding.

See test/addressbook.c for details.

Pattern API

If you need better performance , you can use pbc_pattern_xxx api .

See test/pattern.c for details.

Pattern api is faster and less memory used because it can access data in native C struct.

Extension

PBC support extension in a very simple way . PBC add a specific prefix to every extension field name.

Service

Not supported

Enum

With message API , you can use both string and integer as enum type . They must be integer in Pattern API.

Lua bindings

cd binding/lua && make or cd binding/lua53 && make

See https://github.com/cloudwu/pbc/tree/master/binding/lua/README.md

Building pbc - Using vcpkg

You can download and install pbc using the vcpkg dependency manager:

git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
./vcpkg install pbc

The pbc port in vcpkg is kept up to date by Microsoft team members and community contributors. If the version is out of date, please create an issue or pull request on the vcpkg repository.

Question ?

• Send me email : http://www.codingnow.com/2000/gmail.gif
• My Blog : http://blog.codingnow.com
• Design : http://blog.codingnow.com/2011/12/protocol_buffers_for_c.html (in Chinese)
• Build for Visual Studio 2012 : https://github.com/miaodadao/pbc

pbc's Issues

Compile error

gcc -O2 -Wall --shared -o libprotobuf.so -I../.. -I/usr/local/include -L/usr/local/bin -L../../build pbc-lua.c ../../build/libpbc.a -llua
pbc-lua.c: In function ‘_pattern_pack’:
pbc-lua.c:790: warning: unused variable ‘b’
/usr/bin/ld: /tmp/ccA2yIgd.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/tmp/ccA2yIgd.o: could not read symbols: Bad value

I already modified /pbc-master/Makefile:

FROM
CFLAGS = -O2
TO
CFLAGS = -O2 -fPIC

是不是不支持import proto message

在使用import时报错，查了下似乎没有import机制，确定下，是否不支持import。

There no lua api to overwrite a pb file at runtime.

I want to support upgrade my application at runtime, so i need to register a new pb file at runtime, currently, it won't overwrite the old one.
Can you show me what's the best way to implement this? I'm happy to send a patch.

使用pbc中的lua decode api，产生了unicode乱码问题

云风您好，我在使用你的pbc库时出现了decode失败的问题，我的使用环境是这样的：

1. proto文件：
   package Cookie;
   message CookieValue {
   message Segments {
   optional int32 s_type = 1;
   optional int32 s_id = 2;
   optional int32 s_ct = 3;
   optional int32 s_ttl = 4;
   }
   repeated Segments seg = 1;

   //last bid info
   message Pbid{
   optional int32 p_id = 1;
   optional int32 p_value = 2;
   }
   repeated Pbid pbid = 2;

   //frequency daily
   message Daily {
   optional int32 daily_id = 1;
   message DailyImp {
   optional int32 d_id = 1;
   optional int32 d_value = 2;
   }
   repeated DailyImp daily_imp = 2;
   }
   repeated Daily daily = 3;

   //frequency total
   message Total {
   optional int32 t_id = 1;
   optional int32 t_value = 2;
   }
   repeated Total total = 4;

   message Imp {
   optional int32 imp_c_id = 1;
   optional int32 win = 2;
   optional int32 bid = 3;
   optional int32 imp_time = 4;
   }
   repeated Imp imp = 5;
   message Clk {
   optional int32 clk_id = 1;
   optional int32 clk_time = 2;
   }
   repeated Clk clk = 6;
   }

2. lua代码：
   local protobuf = require "protobuf"
   local cjson = require "cjson"

addr = io.open("../../build/cookie.pb","rb")
buffer = addr:read "*a"
addr:close()
protobuf.register(buffer)

local CookieValue={
seg={
{s_type=1,s_ct=1415384160,s_ttl=1415384160,s_id=100001}
},
total={
{t_value=1,t_id=100188}
},
pbid={
{p_value=3,p_id=100188},
{p_value=3,p_id=100189},
{p_value=3,p_id=100190}
},
daily={
{daily_id=1125,daily_imp={{d_value=1,d_id=100188}}},
{daily_id=1126,daily_imp={{d_value=1,d_id=100118}}}
},
imp={
{win=2,bid=3,imp_time=8160,imp_c_id=100188},
{win=2,bid=3,imp_time=8155,imp_c_id=100182},
{win=2,bid=3,imp_time=8157,imp_c_id=100181},
}
}

local buffer = protobuf.encode("Cookie.CookieValue", CookieValue)
print('buffer: ',buffer)
print('the len of buffer: ',#buffer)
local t = protobuf.decode("Cookie.CookieValue", buffer)
print('cjson(t): ',cjson.encode(t))

for k,v in pairs(t) do
if type(k) == "string" then
print(k,v)
end
end
for k,v in pairs(t.daily[1]) do
print(k,v)
end

for k,v in pairs(t.pbid[1]) do
print(k,v,type(k),type(v))
end

1. 打印出来的结果：
   buffer: �܎����ݎ����ގ����
   �܎����
   ����"܎���
   � �������������*
   ���� �܎�*
   ���� �֎�*
   ���� �Վ�
   the len of buffer: 117
   cjson(t): {"pbid":[["Cookie.CookieValue.Pbid","\b܎\u0006\u0010\u0003"],["Cookie.CookieValue.Pbid","\bݎ\u0006\u0010\u0003"],["Cookie.CookieValue.Pbid","\bގ\u0006\u0010\u0003"]],"daily":[["Cookie.CookieValue.Daily","\b�\b\u0012\u0006\b܎\u0006\u0010\u0001"],["Cookie.CookieValue.Daily","\b�\b\u0012\u0006\b�u0006\u0010\u0001"]],"total":[["Cookie.CookieValue.Total","\b܎\u0006\u0010\u0001"]],"seg":[["Cookie.CookieValue.Segments"," ����\u0005\u0010��\u0006\b\u0001\u0018����\u0005"]],"imp":[["Cookie.CookieValue.Imp","\u0010\u0002\u0018\u0003 �?\b܎\u0006"],["Cookie.CookieValue.Imp","\u0010\u0002\u0018\u0003 �?\b֎\u0006"],["Cookie.CookieValue.Imp","\u0010\u0002\u0018\u0003 �?\bՎ\u0006"]]}
   pbid table: 0x4005ed38
   daily table: 0x4005ef30
   total table: 0x4005f030
   seg table: 0x4005e708
   imp table: 0x4005e7b8
   1 Cookie.CookieValue.Daily
   2 �܎���
   1 Cookie.CookieValue.Pbid number string
   2 ܎��� number string

---

1. 问题
   从上面可以看到，生成的二进制（使用encode 接口）长度和google 官方生成的二进制长度是一致的的。但是使用decode接口时出现了unicode乱码问题。
   编译生成pbc.a和protobuf.so时我均采用了luajit编译，并且添加了-fPIC选项，如下：
   gcc -O2 -Wall --shared -fPIC -o protobuf.so -I../.. -I/usr/local/include/luajit-2.0 -L/usr/local/bin -L../../build pbc-lua.c -lpbc -lluajit-5.1
   不知对这个问题有没有影响，因为我们对性能要求较高，所以才这么编译了。。谢谢。

run test.lua 产生Bus error: 10错误

运行几个lua测试文件（test.lua、test2.lua、textparser.lua），结果都是Bus error: 10错误。

通过打日志发现，是protobuf.encode或者protobuf.decode触发的，

再往里跟发现是pbc-lua.c文件里的
static void
new_array(lua_State *L, int id, const char *key)函数
里面的lua_setfield(L, -6 , key);这一行触发的，
水平比较差，没有思路，求大神指点。
-------------------环境
系统：mac osx 10.10
lua版本：5.1 5.14 5.3都试过
protobuff版本： 3.0.0-alpha-3

热更新pb

我们在项目中用了pbc来和客户端交互，在个别情况下，策划可能需要对协议做一些热更新（比如在客户端多显示一个XX值）
pbc有办法将register到环境里的数据删掉重新register一遍么？或者直接更新的方法

about 'optional' field .

the message like this:
message Role{
optional int32 fieldA = 1;
}

i got message from server, but server didn't packed 'fieldA' everytime .
so how did know if 'fieldA' exist by lua code ?

线程安全?

这个接口果然比较好用，比nanopb, protobuf-c好用多了！有个问题，这个是不是线程安全的？

message嵌套

你好！我想问下message嵌套问题，代码跟上一位的一致，encode时一直有错。你所说的打开lua 5.2的宏是可以解决这个问题吗？是的话这个宏在哪呢？

pbc_wmessage_integer/pbc_rmessage_integer对64bit整数的支持

hi, 云风，我尝试测试一下pbc对int64的支持，我修改了test/test.proto如下：

message at {
        optional int32 aa = 1;
        optional int32 bb = 2;
        optional string cc = 3;
        optional int64 dd = 4;
}

相应的test/test.c测试test函数改为：

            struct pbc_wmessage* w_msg = pbc_wmessage_new(env, "at");
            struct pbc_rmessage* r_msg = NULL;
            struct pbc_slice sl;
            char buffer[1024];
            sl.buffer = buffer, sl.len = 1024;

            uint32_t ilow = 0xabcdef12, ihigh = 0xabcdef12;
            printf("writing:\tlow=%x, high=%x\n", ilow, ihigh);
            pbc_wmessage_integer(w_msg, "dd", ilow, ihigh);
            pbc_wmessage_buffer(w_msg, &sl);

            r_msg = pbc_rmessage_new(env, "at", &sl);
            uint64_t  d = 0ull;
            uint32_t low = 0, high = 0;

            low = pbc_rmessage_integer(r_msg, "dd", 0, &high);
            printf("read:\tlow=%x, high=%x\n", low, high);
            d = ((uint64_t)high)<<32 | (uint64_t)low;
            printf("dd=%llx\n", d);

            pbc_rmessage_delete(r_msg);
            pbc_wmessage_delete(w_msg);

运行的输出为：

writing:    low=abcdef12, high=abcdef12
read:   low=abcdef12, high=fbcdef12
dd=fbcdef12abcdef12

可以看到dd字段的输入高低32位都是0xabcdef12，而从msg中读取出来的dd字段的打印为dd=fbcdef12abcdef12，请问下这个是个bug?还是目前不支持int64?

Arm 平台WP8 编译pbc报错

#ifndef _LUA_PBC_EXTRA_H_
#define _LUA_PBC_EXTRA_H_

#if defined(_USRDLL)
#define LUA_EXTENSIONS_DLL _decispec(dllexport)

#else
#define LUA_EXTENSIONS_DLL
#endif

#ifdef __cplusplus
extern "C"{
#endif

    #include "lauxlib.h"
    int LUA_EXTENSIONS_DLL luaopen_protobuf_c(lua_State *L);

#ifdef __cplusplus
}
#endif
#endif // _LUA_PBC_EXTRA_H_

在链接时找不到符号 luaopen_protobuf_c

请问如何破解？

当proto定义空字段的message时，parser报错

message foo {
}

parser报错

segment fault after I compile the pbc source into nginx (whole compile)

==13968== Invalid write of size 4
==13968== at 0x4249F4: _pbcA_open (array.c:19)
==13968== by 0x428732: pbc_pattern_unpack (pattern.c:835)
==13968== by 0x429472: register_internal (bootstrap.c:275)
==13968== by 0x42980C: _pbcB_init (bootstrap.c:302)
==13968== by 0x425D46: pbc_new (proto.c:32)

note: when I use pbc as a library and link into nginx, it is not segment fault.

5.1.5版本的Lua绑定代码中,调用luaopen_protobuf_c后栈没有平衡,有table在栈顶

调用luaopen_protobuf_c之后,lua_gettop会返回1,建议加上一个lua_pop的调用,以符合
Lua手册中说的好的编程实践.

Note that the code above is "balanced": at its end, the stack is back to its original configuration. This is considered good programming practice.

double or float type problems

proto 文件中定义有 double 或者 float类型的
在lua中进行赋值的时候 设置int类型 或者 double/float浮点类型 进行encode通过http发送到java后台进行解析的时候解析不出来
我进行在lua中把encode的数据 在decode回来进行打印发现浮点类型加长了好多位 我不知道是不是这样导致java端解析不出数据
求解

Proto file defined in double or float type
In lua assignment set type int or double/float float to encode sent via HTTP to a Java background when parsing resolution not to come out
I conducted in lua to encode the data found in floating-point decode back for printing type lengthened a lot I don't know whether that is in Java can't parse the data
For help

pbc lua 无法遍历嵌套对象等

环境： luajit 2.0.2, windows

1.无法遍历嵌套对象
2.无法对decode的结果encode
3.如果optional字段不编进去，decode得到table却不是nil

message A {
required int64 id = 1;
}
message B{
required int64 id = 1;
optional A a = 2;

}

local deData = {
id = 1,
a = {id = 1}
}
local enData = protobuf.encode("B", deData)
local deData = protobuf.decode("B", enData)

for k, v in pairs(deData.a) do
print(k, v) --无法遍历
end

protobuf.encode("B", deData) --出错

---

local deData = {
id = 1
}
local enData = protobuf.encode("B", deData)
local deData = protobuf.decode("B", enData)

print(deData.a) --不是nil

testparser.lua crashed on Mac OSX 64bit

使用lua5.1运行testparser.lua脚本：
lua ./testparser.lua
出现 segmentation fault，其他的test脚本以及c的test程序均可正常运行。使用gdb环境执行，得到的输出和调用栈如下：

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000000
0x00007fff887f0403 in strncmp ()

(gdb) bt
#0  0x00007fff887f0403 in strncmp ()
#1  0x000000010020f50b in pbc_wmessage_string (m=0x10011fd80, key=0x100101e48 "type", v=0x10011bef8 "TYPE_STRING", len=11) at wmessage.c:289
#2  0x0000000100201bb9 in _wmessage_string (L=0x100100810) at pbc-lua.c:251
#3  0x000000010004b854 in luaD_precall ()
#4  0x0000000100056091 in luaV_execute ()
#5  0x000000010004b8ff in luaD_call ()
#6  0x000000010004ad45 in luaD_rawrunprotected ()
#7  0x000000010004adb9 in luaD_pcall ()
#8  0x0000000100045623 in lua_pcall ()
#9  0x0000000100001386 in docall ()
#10 0x0000000100001e5b in handle_script ()
#11 0x0000000100002457 in pmain ()
#12 0x000000010004b854 in luaD_precall ()
#13 0x000000010004b8ee in luaD_call ()
#14 0x000000010004ad45 in luaD_rawrunprotected ()
#15 0x000000010004adb9 in luaD_pcall ()
#16 0x000000010004547b in lua_cpcall ()
#17 0x000000010000256f in main ()
(gdb) 

其直接崩溃原因是传给strncmp的字符串指针为0x0

我做了一些简单的分析，发现在pbc_wmessage_string这一层m的type是google.protobuf.FieldDescriptorProto，field的id是5，type是14，我查看了google.protobuf.FieldDescriptorProto的定义，id为5的字段的确叫做type_name但是，类型是string，不应该是14。

我没有完整仔细的阅读pbc的全部代码，可能分析并不正确，如果你需要什么额外的信息，请告诉我 :)

BTW:
我仔细看了一下pbc的lua binding代码，有一个疑问，就是，如果用户在decode对象中获取并持有一个sub message的对象，那么其父对象被gc之后，这个子对象应该就会变成野指针吧？例如在test.lua中的一段代码加以演绎：

decode = protobuf.decode("tutorial.Person" , code)

local myphone = decode.phone
decode = nil
print(myphone[1].number

error building lua binding

[david@host1 lua{master}]$ C_INCLUDE_PATH=/usr/include/lua5.1     make  
gcc -O2 -Wall --shared -o protobuf.dll -I../.. -I/usr/local/include -L/usr/local/bin -L../../build pbc-lua.c -lpbc -llua52
pbc-lua.c: In function ‘_env_enum_id’:
pbc-lua.c:80:38: warning: implicit declaration of function ‘_pbcM_sp_query’ [-Wimplicit-function-declaration]
     struct _enum *enum_map = (struct _enum *)_pbcM_sp_query(env->enums, enum_type);
                                      ^
pbc-lua.c:80:64: error: dereferencing pointer to incomplete type
     struct _enum *enum_map = (struct _enum *)_pbcM_sp_query(env->enums, enum_type);
                                                                ^
pbc-lua.c:86:5: warning: implicit declaration of function ‘_pbcM_si_query’ [-Wimplicit-function-declaration]
     int err = _pbcM_si_query(enum_map->name, enum_name, &enum_id);
     ^
pbc-lua.c:86:38: error: dereferencing pointer to incomplete type
     int err = _pbcM_si_query(enum_map->name, enum_name, &enum_id);
                                      ^
pbc-lua.c: In function ‘_pattern_pack’:
pbc-lua.c:809:14: warning: unused variable ‘b’ [-Wunused-variable]
  luaL_Buffer b;
              ^
make: *** [all] Error 1

Any pointers?

请问optional的message怎么判断是否是默认值呢？

比如
message a
{
optional b;
}
这个b

lua/pbc-lua.c windows下编译问题

由于windows下编译不通过，查看原因。

https://github.com/cloudwu/pbc/blob/master/binding/lua/pbc-lua.c

#ifndef _MSC_VER
#include <alloca.h>
#else
#define alloca _alloca
#endif

是否应该改成

#ifndef WINVER
#include <alloca.h>
#else
#define alloca _alloca
#endif

在用MinGW编译pbc与slua合进dll时发生的错误。

test/addressbook.c内存泄漏

内存泄漏发生的位置是test_wmessage函数内对phone的操作， repeated字段。测试了一下，

struct pbc_wmessage * phone = pbc_wmessage_message(msg , "phone");
pbc_wmessage_string(phone , "number", "87654321" , -1);
调用一次没有问题， 接下来， 再次调用pbc_wmessage_string就会出现内存泄漏

==10504== 8 bytes in 1 blocks are definitely lost in loss record 1 of 1
==10504== at 0x402BE68: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==10504== by 0x8048871: _pbcM_malloc (in /home/van9ogh/Desktop/mr/tmp/pbc/test/a.out)
==10504== by 0x8051E5A: pbc_wmessage_string (in /home/van9ogh/Desktop/mr/tmp/pbc/test/a.out)
==10504== by 0x8052DEC: test_wmessage (in /home/van9ogh/Desktop/mr/tmp/pbc/test/a.out)
==10504== by 0x8052F18: main (in /home/van9ogh/Desktop/mr/tmp/pbc/test/a.out)
==10504==
==10504== LEAK SUMMARY:
==10504== definitely lost: 8 bytes in 1 blocks
==10504== indirectly lost: 0 bytes in 0 blocks
==10504== possibly lost: 0 bytes in 0 blocks
==10504== still reachable: 0 bytes in 0 blocks
==10504== suppressed: 0 bytes in 0 blocks

optional string pack cause core dump

pattern.c line 752

if (len <= 0) {
return strcmp(pf->defv->s.str, slice->buffer) == 0;
}

when len ==0

would cause core dump

wmessage.c

hi, cloudwu,

thanks for your code of pbc. Maybe I found a bug in the wmessage.c , when a field is LABEL_OPTIONAL , the output of protobuf bytes may lost the _bitField information，because you don't write this field to output bytes when it has a default value. Just to see the code below:

if (f->label == LABEL_OPTIONAL) {
if (f->type == PTYPE_ENUM) {
if (strncmp(v , f->default_v->e.name, len) == 0 && f->default_v->e.name[len] =='\0') {
return 0;
}
} else if (f->type == PTYPE_STRING) {
if (len == f->default_v->s.len &&
strcmp(v, f->default_v->s.str) == 0) {
return 0;
}
}
}

then, when a user decode this bytes, call the function to check whether this field has been set, got the wrong result.

I'v port it to windows

hi cloud and all,

because of some reason, I port pbc to windows(vs2010), there is no big problem in the process, if anyone want one, mail me. if cloud wanna me to commit it to git, it's my presure.

vincent

pbc在arm平台上会不会有字节序的问题？编译器是arm-none-linux-gnueabi-gcc

打算引入pbc了， cloudwu出品必属精品， ;-)

项目跑在基于linux的ARM设备上， 貌似交叉编译没什么问题， 还没跑， 不知云风哥protobuf解析会不会有字节序的问题？ 还是protoc工具本身的事情 :-)

import别的proto文件后注册失败问题

我们的项目使用lua来注册proto，项目自定义的proto文件需要引入descriptor.proto，当用下面这种方式的时候：
import "descriptor.proto"
生成的pb文件可以注册成功，但是在Java上使用时会因为命名空间的检查导致失败；
如果我们使用：
import "google/protobuf/descriptor.proto"
做引用，生成的pb文件总是注册失败。请问有什么方法可以解决么？

当没有定义package时，嵌套定义的message注册错误

测试代码

// foo.proto

message Foo {
    message Bar {
        required uint32 value = 1;
    };
    required Bar bar = 1;
};

    // foo.c
    struct pbc_wmessage * foo = pbc_wmessage_new(env, "Foo");
    struct pbc_wmessage * bar = pbc_wmessage_message(foo, "bar");

    pbc_wmessage_integer(bar, "value", 1, 0);      // 这里会core掉，因为bar->type == NULL

看了一下代码，问题应该时出在 src/register.c 文件中几次对_register_message的调用时候，参数prefix_sz 填的是 prefix_sz + name_sz + 1
而这里如果prefix_sz = 0 的话，事实上不应该+1，结果导致嵌套message的名字错误

pbc don't implement forward-compatibility allowed by official implementation.

According to official documentation, https://developers.google.com/protocol-buffers/docs/cpptutorial, official implementation allow compatibility when add/delete optional fields. But pbc don't implement this.

test_optional.lua :

pb = require "protobuf"

addr = io.open("optional.pb","rb")
buffer = addr:read "*a"
addr:close()

pb.register(buffer)
local buf = pb.encode('Message1', {x=1,y=2,w=3,h=4})
local o = assert(pb.decode('Message2', buf), 'decode failed')
print(o.x)

optional.proto :

message Message1 {
    required int32 x = 1;
    required int32 y = 2;
    optional int32 w = 3;
    optional int32 h = 4;
}

message Message2 {
    required int32 x = 1;
    required int32 y = 2;
}

The output is expected to be 1, but gets a decode error.

pbc无法解析C++ protobuf序列化后的字符串

C++：

tutorial::Person person;

  person.set_name("Alice");
  person.set_id(123)
 std::string stringbuffer =person:SerializeAsString();

字符传给lua
Lua:

local file = "person.pb";
protobuf.register_file(file)

result = protobuf.decode("tutorial.Person", stringbuffer)

result一直是false

not support import another .proto?

lua: ./protobuf.lua:545: register fail
stack traceback:
[C]: in function '_env_register'
./protobuf.lua:545: in function 'register'
test2.lua:6: in main chunk
[C]: ?

lua中怎么样取出enum的值

在lua中，test.proto
message A{
enum {
B = 1;
};
};

pb:register 'test.pb'后

在lua怎么拿到 B的值

在怎么样取出enum的值

在lua中，test.proto
message A{
enum {
B = 1;
};
};

pb:register 'test.pb'后

在lua怎么拿到 B的值

off-by-one reading in pbc_wmessage_string() line 268,303 ?

当一个字段同时为enum和optional时，如果enum值刚好为0，对端(flash)接受到为null值。

encode immediately after decode failed

在message A中包含了另外一个message B
在Deocde A的时候不会立即对B进行解析，需要通过访问B中的内容来获取B的完整解析
现在的应用场景是从数据库中取出pb结构，只更改某一个字段，其它不变，然后需要存回数据库，结果encode不支持这种属于中间状态的结构进行存储
不知道是基于什么样的考虑没有提供这样的一个encode方法?

代码https://gist.github.com/wang2191195/7348669

dump.c

in function dump_value
pbc_rmessage_integer(m, key, i, NULL); i should be idx ...

it seemed lost some api like “has_xxx”

查了下protobuf.lua,没找到类似has_xx的api，这让使用者无法区分table中的某个字段是缺省值还是根本就没有（用nil?)

Using lua protobuf.so for iOS

I've been trying to make your protobuf working for iOS. At first I couldn't compile on Mac OS X but figured it out by adding dynamic_lookup into CFLAGS. Think you could add this as an option for Mac OS X build

But when I loaded using protobuf.lua in my iOS game (running quick-cocos2dx which is lua) I got this error:

[LUA ERROR] error loading module 'protobuf.c' from file '/usr/local/lib/lua/5.1/protobuf.so':
dlopen(/usr/local/lib/lua/5.1/protobuf.so, 2): no suitable image found. Did find:
/usr/local/lib/lua/5.1/protobuf.so: mach-o, but not built for iOS simulator

Any ideas?

import "MessageType.proto";报错.\parser.lua:53: syntax error at [Scene.proto] (8)

testLogin.lua 文件内容如下：
require "protobuf"

t = parser.register("MessageType.proto","../../test/aa")
t = parser.register("Scene.proto","../../test/aa")

MessageType.proto 文件内容如下：
package com.proto.msg;
option java_package = "com.proto.msg.common";
option java_outer_classname = "MessageType";

enum CGMessageType{
CG_START_MOVE = 851;
}

Scene.proto 文件内容如下：
package com.proto.msg.scene;
option java_package = "com.proto.msg.scene";
option java_outer_classname = "Scene";
import "MessageType.proto";

message CGStartMove {
required int32 x = 1 ;
required int32 y = 2 ;
}

我尝试了按顺序注册，但还是报错。如果打成一个pb文件的话，实战中某些pb文件会过大，不利于客户端动态更新

protobuf.unpack 无法解析消息中嵌套的对象数组，但protobuf.decode可以成功

将您的test.lua做了些微改动之后test.lua代码如下：
require "protobuf"

addr = io.open("../../build/addressbook.pb","rb")
buffer = addr:read "*a"
addr:close()
protobuf.register(buffer)

addressbook = {
name = "Alice",
id = 12345,
phone = {
{ number = "1301234567" },
{ number = "87654321", type = "WORK" },
}
}

code = protobuf.encode("tutorial.Person", addressbook)

decode = protobuf.decode("tutorial.Person" , code)

print(decode.name)
print(decode.id)
for _,v in ipairs(decode.phone) do
print("\t"..v.number, v.type)
end

print("=========");
phonebuf = protobuf.pack("tutorial.Person.PhoneNumber number","87654321");
buffer = protobuf.pack("tutorial.Person name id phone", "Alice", 123, { phonebuf });
name,id,phone = protobuf.unpack("tutorial.Person name id phone", buffer);
print(name)
print(id)
for _,v in pairs(phone) do
print(v.number, v.type)
end

输出结果如下：
Alice
12345
1301234567 HOME

87654321 WORK

Alice
123
nil nil

不知道为什么87654321 WORK会在这个网页上变粗体加大

有个optional int32 相关的 bug

optional int32 的字段如果赋值为0, encode 的时候好像没有把这个字段encode 进去， decode的时候也没有应字段的值

与protobuf的兼容问题

发现编码的结果跟protobuf不一致，生成的编码protobuf不能正常解析。

对于消息：
message test{
required string id = 1;
}

赋值：
test.id = "1"

pbc编码结果：
0d 0a 02 31 20

protobuf编码结果：
0a 01 31

hi, 这个库大概需要占用多少内存呢？

能否用到嵌入式系统中，嵌入式通信这一块是个问题

luajit decode method use void * buffer get from ffi

the buffer type is : cdata<void *>, when call decode method from luajit, I get an error message : protobuf.lua:502: userdata 5 is nil.

Example of Lua binding with nested message?

Hi, Is it possible to include short example of using the lib from Lua for encoding / decoding nested messages? I mean when the message i have to encode has a field of type another message?
Thanks,
Plamen

makefile has a little problem

define PROTO_temp
TAR := $(BUILD)/$(notdir $(basename $(1)))
PROTO := $(PROTO) $$(TAR).pb
$$(TAR).pb : | $(BUILD)
$$(TAR).pb : test/$(1)
protoc -o$$@ $$&lt; ==&gt;build/pbc -o $$@ $$<
endef

编译binding/lua时报错

环境是mac osx Yosemite 10.10.4，openresty附带的luajit-2.1。

···
gcc -O2 -fPIC -Wall -shared -o protobuf.so -I../.. -I/usr/local/openresty/luajit/include/luajit-2.1 -L../../build pbc-lua.c -lpbc
Undefined symbols for architecture x86_64:
"_luaL_checkinteger", referenced from:
__rmessage_new in pbc-lua-cc4638.o
__rmessage_integer in pbc-lua-cc4638.o
__rmessage_int32 in pbc-lua-cc4638.o
__rmessage_int64 in pbc-lua-cc4638.o
__rmessage_int52 in pbc-lua-cc4638.o
__rmessage_uint52 in pbc-lua-cc4638.o
__rmessage_real in pbc-lua-cc4638.o
...
"_luaL_checklstring", referenced from:
__env_register in pbc-lua-cc4638.o
__env_type in pbc-lua-cc4638.o
__rmessage_new in pbc-lua-cc4638.o
__rmessage_integer in pbc-lua-cc4638.o
__rmessage_int32 in pbc-lua-cc4638.o
__rmessage_int64 in pbc-lua-cc4638.o
__rmessage_int52 in pbc-lua-cc4638.o
...
"_luaL_checknumber", referenced from:
__wmessage_real in pbc-lua-cc4638.o
__wmessage_int52 in pbc-lua-cc4638.o
__wmessage_uint52 in pbc-lua-cc4638.o
__pattern_pack in pbc-lua-cc4638.o
"_luaL_checktype", referenced from:
__decode in pbc-lua-cc4638.o
"_luaL_error", referenced from:
__env_register in pbc-lua-cc4638.o
__env_type in pbc-lua-cc4638.o
__rmessage_new in pbc-lua-cc4638.o
__rmessage_delete in pbc-lua-cc4638.o
__rmessage_integer in pbc-lua-cc4638.o
__rmessage_int32 in pbc-lua-cc4638.o
__rmessage_int64 in pbc-lua-cc4638.o
...
"_luaL_register", referenced from:
_luaopen_protobuf_c in pbc-lua-cc4638.o
"_lua_call", referenced from:
_push_value in pbc-lua-cc4638.o
"_lua_checkstack", referenced from:
__pattern_unpack in pbc-lua-cc4638.o
"_lua_createtable", referenced from:
__pattern_unpack in pbc-lua-cc4638.o
__decode in pbc-lua-cc4638.o
__gc in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
"_lua_insert", referenced from:
_decode_cb in pbc-lua-cc4638.o
"_lua_isstring", referenced from:
__rmessage_new in pbc-lua-cc4638.o
__pattern_unpack in pbc-lua-cc4638.o
"_lua_isuserdata", referenced from:
__pattern_unpack in pbc-lua-cc4638.o
"_lua_newuserdata", referenced from:
__gc in pbc-lua-cc4638.o
"_lua_objlen", referenced from:
__pattern_pack in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
"_lua_pushboolean", referenced from:
__env_type in pbc-lua-cc4638.o
__pattern_unpack in pbc-lua-cc4638.o
__decode in pbc-lua-cc4638.o
_push_value in pbc-lua-cc4638.o
"_lua_pushcclosure", referenced from:
__gc in pbc-lua-cc4638.o
"_lua_pushinteger", referenced from:
__env_type in pbc-lua-cc4638.o
__rmessage_integer in pbc-lua-cc4638.o
__rmessage_size in pbc-lua-cc4638.o
__wmessage_buffer in pbc-lua-cc4638.o
__pattern_size in pbc-lua-cc4638.o
__pattern_unpack in pbc-lua-cc4638.o
__env_enum_id in pbc-lua-cc4638.o
...
"_lua_pushlightuserdata", referenced from:
__env_new in pbc-lua-cc4638.o
__rmessage_new in pbc-lua-cc4638.o
__rmessage_int32 in pbc-lua-cc4638.o
__rmessage_message in pbc-lua-cc4638.o
__wmessage_new in pbc-lua-cc4638.o
__wmessage_message in pbc-lua-cc4638.o
__wmessage_buffer in pbc-lua-cc4638.o
...
"_lua_pushlstring", referenced from:
__rmessage_int64 in pbc-lua-cc4638.o
__rmessage_string in pbc-lua-cc4638.o
__wmessage_buffer_string in pbc-lua-cc4638.o
__pattern_unpack in pbc-lua-cc4638.o
__pattern_pack in pbc-lua-cc4638.o
_push_value in pbc-lua-cc4638.o
"_lua_pushnumber", referenced from:
__rmessage_int52 in pbc-lua-cc4638.o
__rmessage_uint52 in pbc-lua-cc4638.o
__rmessage_real in pbc-lua-cc4638.o
__pattern_unpack in pbc-lua-cc4638.o
_push_value in pbc-lua-cc4638.o
"_lua_pushstring", referenced from:
__env_type in pbc-lua-cc4638.o
__last_error in pbc-lua-cc4638.o
_push_value in pbc-lua-cc4638.o
"_lua_pushvalue", referenced from:
__decode in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
_push_value in pbc-lua-cc4638.o
"_lua_rawgeti", referenced from:
__pattern_pack in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
"_lua_rawseti", referenced from:
__pattern_unpack in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
"_lua_setfield", referenced from:
__gc in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
"_lua_setmetatable", referenced from:
__gc in pbc-lua-cc4638.o
"_lua_settop", referenced from:
__env_type in pbc-lua-cc4638.o
__pattern_pack in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
"_lua_toboolean", referenced from:
__pattern_pack in pbc-lua-cc4638.o
"_lua_tointeger", referenced from:
__rmessage_string in pbc-lua-cc4638.o
__rmessage_message in pbc-lua-cc4638.o
__pattern_unpack in pbc-lua-cc4638.o
__pattern_pack in pbc-lua-cc4638.o
"_lua_tolstring", referenced from:
__rmessage_new in pbc-lua-cc4638.o
__wmessage_int64 in pbc-lua-cc4638.o
__pattern_unpack in pbc-lua-cc4638.o
__pattern_pack in pbc-lua-cc4638.o
"_lua_touserdata", referenced from:
__env_register in pbc-lua-cc4638.o
__env_type in pbc-lua-cc4638.o
__rmessage_new in pbc-lua-cc4638.o
__rmessage_delete in pbc-lua-cc4638.o
__rmessage_integer in pbc-lua-cc4638.o
__rmessage_int32 in pbc-lua-cc4638.o
__rmessage_int64 in pbc-lua-cc4638.o
...
"_lua_type", referenced from:
__env_type in pbc-lua-cc4638.o
__wmessage_int32 in pbc-lua-cc4638.o
__wmessage_int64 in pbc-lua-cc4638.o
__pattern_pack in pbc-lua-cc4638.o
__decode in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [protobuf.so] Error 1
···

Compiling problem on Linux

Cannot compile on ubuntu 11.10 (x86_64).
Propose following fix:

diff --git a/Makefile b/Makefile
index c71ed25..5b59879 100644
--- a/Makefile
+++ b/Makefile
@@ -39,7 +39,7 @@ define BUILD_temp
$$(TAR).o : | $(BUILD_O)
-include $$(TAR).d
$$(TAR).o : src/$(1)

•   $(CC) $(CFLAGS) -c -Isrc -I. -o $$@ -MMD $$<
  

•   $(CC) $(CFLAGS) -c -Isrc -I. -fPIC -o $$@ -MMD $$<
  

  endef

  $(foreach s,$(LIBSRCS),$(eval $(call BUILD_temp,$(s))))
  @@ -55,7 +55,7 @@ define TEST_temp
  $$(TAR)$$(EXE) : | $(BUILD)
  $$(TAR)$$(EXE) : | $(LIBNAME)
  $$(TAR)$$(EXE) : test/$(1)

•   cd $(BUILD) && $(CC) $(CFLAGS) -I.. -L. -lpbc -o $$(notdir $$@) ../$$<
  

•   cd $(BUILD) && $(CC) $(CFLAGS) -I.. -o $$(notdir $$@) ../$$<  -L. -lpbc
  

  endef

  $(foreach s,$(TESTSRCS),$(eval $(call TEST_temp,$(s))))

Don't support nested message.

The decode api only decode one layer.

For example, a message like this:

message Detail {
  ...
}
message List {
  repeated Detail details;
}

I want to decode a list, i have to manually decode details with a loop:

local list = protobuf.decode('List', buffer)
for _, buf in list.details or {} do
    local detail = protobuf.decode(unpack(buf))
end

Is there an api to decode it recursively, or is it possible to implement this using a lua function.

repeate字段无数据，解包修改后，会影响后续所有此字段无数据的包。

message EnterMapNotice
{
required bool is_login = 1;
repeated limit.Datum limit_dt = 2;
}

local enData = protobuf.encode("ic.scene.EnterMapNotice", {is_login = true})
local deData1 = protobuf.decode("ic.scene.EnterMapNotice", enData)
table.insert(deData1.limit_dt, {aaa = 1})

local enData = protobuf.encode("ic.scene.EnterMapNotice", {is_login = true})
local deData2 = protobuf.decode("ic.scene.EnterMapNotice", enData)
print(deData2.limit_dt, deData2.limit_dt[1], deData2.limit_dt[1].aaa)
---这里deData2.limit_dt[1]应该没数据的，被打印出来了

对repeate字段，如果无数据，进行修改后。
会影响后续所有此字段无数据的包。

应该是都指向空的默认table，而非生成新的table的原因。

不管是否可修改，留给其他用户提个醒。