Giter Site home page Giter Site logo

tableauio / tableau Goto Github PK

View Code? Open in Web Editor NEW
28.0 4.0 6.0 1004 KB

A modern configuration converter based on Protobuf (proto3).

Home Page: https://tableauio.github.io

License: MIT License

Go 99.80% Shell 0.20%
excel xml csv json protobuf golang configuration converter parser go

tableau's Introduction

Tableau

Modern Configuration Converter

Release Status Testing Status Code Coverage GitHub release (latest SemVer including pre-releases) go.dev GitHub

Tableau

A modern configuration converter based on Protobuf (proto3).

Features

  • Convert Excel/CSV/XML to JSON/Text/Bin, JSON is the first-class citizen of exporting targets.
  • Use Protobuf to define the structure of Excel/CSV/XML.
  • Use Golang to develop the conversion engine.
  • Support multiple programming languages, thanks to Protobuf (proto3).

Concepts

  • Importer: Excel/CSV/XML importer.
  • IR: Intermediate Representation.
  • Filter: filter the IR.
  • Exporter: JSON, Text, and Bin.
  • Protoconf: a dialect of Protobuf (proto3) extended with tableau options, also a conversion spec with expressive, elegant syntax.

Design

See official document: Design.

Contribution

Requirements

Protobuf

Goto Protocol Buffers v21.12, choose and download the correct platform of protoc, then install by README.

protoc-gen-go

Install: go install google.golang.org/protobuf/cmd/[email protected]

tableau's People

Contributors

huieric avatar kybxd avatar wenchy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

tableau's Issues

confgen: export deterministic JSON/Text/Bin

Problems

Protobuf map type is a hashmap, so elements are not ordered and non-deterministic.

  • encoding/protojson#Marshal
    Marshal writes the given proto.Message in JSON format using default options. Do not depend on the output being stable. It may change over time across different versions of the program.
  • encoding/prototext#Marshal
    Marshal writes the given proto.Message in textproto format using default options. Do not depend on the output being stable. It may change over time across different versions of the program.
  • encoding/protowire#Marshal

Related issues

Solutions

  • JSON: to gain some degree of output stability, we recommend running the output through a JSON formatter.
  • Text: To obtain some degree of stability, we recommend passing the output of prototext through the txtpbfmt program. The formatter can be directly invoked in Go using parser.Format.
  • Bin: set marshal option Deterministic as true. See MarshalOptions.

protogen: support workbook alias in metasheet @TABLEAU

Currently, each worksheet in a workbook can be specified an alias, but not the workbook name.

Here, we propose the special sign # in Sheet column of metasheet @TABLEAU to represent this wokbook self:

Sheet Alias
# WorkbookAlias
Sheet1 Sheet1Alias

confgen: auto deduce whether the map key is unique

By default, the map field key is treated as unique, but if nesting is like:

  1. map1 in map with same layout, the map1 has no sub-field (map or list) with cardinality
  2. list in map with same layout

Note that layout can be:

  • LAYOUT_VERTICAL
  • LAYOUT_HORIZONTAL

Then the map field key is not unique in order to aggregate sub-field (map or list) with cardinality.

protoconf: add new type: `union`

Theory

In protoconf, union type means the tagged union: a data structure used to hold a value that could take on several different, but fixed, types. Only one of the types can be in use at any one time, and a tag field explicitly indicates which one is in use. More details can be learned from https://en.wikipedia.org/wiki/Tagged_union.

Tagged union in different programming languages:

Implementation

In protobuf, use message to bundle enum and oneof together to implement tagged union. By default, each enum value (>0) is bound to a field with the same tag number of oneof type.

Extend MessageOptions in proto/tableau/protobuf/tableau.proto:

extend google.protobuf.MessageOptions{
  ...
  UnionOptions union= 50001;
}

message UnionOptions {
  string type = 1;   // Tagged field: auto bound to enum type field in message.
  string value = 2;  // Union value: auto bound to oneof type field in message.
}

extend google.protobuf.OneofOptions {
  optional OneofOptions oneof = 50000;
}

message OneofOptions {
  string name = 1; // Alias.
}

Predefined in proto

A union type should be predefined:

message Foo {
  option (tableau.union) = true;
  
  Type type = 9999 [(tableau.field) = { name: "Type" }];
  oneof value {
    option (tableau.oneof) = {field: "Field"};
    Pvp pvp = 1; // struct type bound to enum Type value: TYPE_PVP as their tag id are same to 1.
    Pve pve= 2; // struct type bound to enum Type value: TYPE_PVP as their tag ids are same to 2.
  }

  enum Type {
    TYPE_NIL = 0;
    TYPE_PVP = 1 [(tableau.evalue) = {name:"PVP"}];
    TYPE_PVE = 2 [(tableau.evalue) = {name:"PVE"}];
  }
  message Pvp {
    int32 battle_id = 1;
    uint32 kills = 2;
    int64 damage = 3;
  }
  message Pve {
    Highlight highlight= 1; // incell struct
    repeated int32 heros = 2; // incell list
    map<int32, int64> dungeons = 3; // incell map
    
    message Highlight {
      int32 battle_id = 1;
      int32 boss_id = 2;
      int64 damage = 3;
    }
  }
}

Then worksheet configured as:

FooType FooField1 FooField2 FooField3
map<enum<.Foo.Type>, .Foo> union union union
Foo's type Foo's value field1 Foo's value field2 Foo's value field3
TYPE_PVP 1 10 100
TYPE_PVE 1,100,999 1,2,3 1:10,2:20,3:30
ID FooType FooField1 FooField2 FooField3
map<int32, Task> {.Foo}enum<.Foo.Type> union union union
ID Foo's type Foo's value field1 Foo's value field2 Foo's value field3
1 PVP 1 10 100
2 PVE 1,100,999 1,2,3 1:10,2:20,3:30

confgen: improve perf stats

Calculate the real consuming time (without locking and waiting time) for parsing each sheet.

TODO:

  • field prop refer: lock and wait

union(protogen): reuse same field type for different fields

The concept is like Custom named struct.

Syntax: just after struct type name, use parentheses () to specify struct variable name: VariableType(VariableName).

Example

Name Alias Field1 Field2 Field3
Battle(PvpBattle) PVP BattleId
int32
Kills
uint32
Damage
int64
Battle(PveBattle) PVE BattleId
int32
Kills
uint32
Damage
int64
message ActivityTarget {
  option (tableau.union) = true;

  Type type = 9999 [(tableau.field) = { name: "Type" }];
  oneof value {
    option (tableau.oneof) = {field: "Field"};

    Battle pvp_battle = 1; // Bound to enum value: TYPE_PVP_BATTLE.
    Battle pve_battle = 2; // Bound to enum value: TYPE_PVE_BATTLE.
  }
  enum Type {
    TYPE_INVALID = 0;
    TYPE_PVP_BATTLE = 1 [(tableau.evalue).name = "PVP"];
    TYPE_PVE_BATTLE = 2 [(tableau.evalue).name = "PVE"];
  }
  message Battle {
    // ...
  }
}

feat: support predefined-incell-struct

Refer: https://tableauio.github.io/docs/excel/struct/#in-cell-struct

proto

Predefined struct:

message Property {
  int32 id = 1 [(tableau.field) = {name:"ID"}];
  string name = 2 [(tableau.field) = {name:"Name"}];
  string desc = 3 [(tableau.field) = {name:"Desc"}];
}

excel

ID Prop
map<uint32, Item> {.Property}
Item’s ID Item’s property.
1 1,Orange,A good fruit.
2 2,Apple
3 3

generated

message ItemConf {
  option (tableau.worksheet) = {name:"PredefinedInCellStruct" namerow:1 typerow:2 noterow:3 datarow:4};

  map<uint32, Item> item_map = 1 [(tableau.field) = {key:"ID" layout:LAYOUT_VERTICAL}];
  message Item {
    uint32 id = 1 [(tableau.field) = {name:"ID"}];
    protoconf.Property prop = 2 [(tableau.field) = {name:"Prop" span:SPAN_INNER_CELL}];
  }
}

XML: duplicated field from last node

<?xml version="1.0" encoding="UTF-8"?>
<!--
<@TABLEAU>
    <Item Sheet="ProcConf" Template="true"/>
</@TABLEAU>

<ProcConf>
    <World ID="uint32" Platform="[]<string>" />
    <mesh_tcp_listen_port>int32</mesh_tcp_listen_port>
    <mesh_polaris_udp_name>string</mesh_polaris_udp_name>
    <mesh_polaris_tcp_name>string</mesh_polaris_tcp_name>
    <mesh_ploaris_namespace>string</mesh_ploaris_namespace>    
    <ServerIP>string</ServerIP>
    <GIDInitSeq>int32</GIDInitSeq>
    <Tconnd Addr="[]<string>" />
    <UDP Port="int32" />
    <dns_server_ip>string</dns_server_ip>
    <dns_udp_port>int32</dns_udp_port>
    <cmd_http_listen_port>int32</cmd_http_listen_port>
    <TCPPort>int32</TCPPort>
    
    <Log>
        <OutputType>int32</OutputType>
        <BaseLogPath>string</BaseLogPath>
        <BaseLogPriority>[]<enum<.LogPriority>></BaseLogPriority>
        <BaseLogFileCfg>
            <ShiftType>{.LogFileCfg}int32</ShiftType>
            <MaxFileSize>int32</MaxFileSize>
            <MaxFileNum>int32</MaxFileNum>
        </BaseLogFileCfg>
        <Msg QueueType="[Msg]int32" CatType="int32" Open="bool" BufferedLen="int32" OutputPlayerLog="bool" OutputMain="bool" UseRealTime="bool">
            <Name>
                <Path>string</Path>
                <BaseName>string</BaseName>
            </Name>
            <LogFileCfg>
                <ShiftType>{.LogFileCfg}int32</ShiftType>
                <MaxFileSize>int32</MaxFileSize>
                <MaxFileNum>int32</MaxFileNum>
            </LogFileCfg>
            <LogPriority>[]<enum<.LogPriority>></LogPriority>
            <Whitelist>
                <ID>[]uint64</ID>
            </Whitelist>
        </Msg>
    </Log>
</ProcConf>
-->

<ProcConf>
    <World ID="1" Platform="AQQ,IQQ,AWX,IWX" />    
    <mesh_tcp_listen_port>18182</mesh_tcp_listen_port>
    <mesh_polaris_udp_name>Test-MeshUdp</mesh_polaris_udp_name>
    <mesh_polaris_tcp_name>Test-MeshTcp</mesh_polaris_tcp_name>
    <mesh_ploaris_namespace>Test</mesh_ploaris_namespace>
    <ServerIP>9.134.54.48</ServerIP>
    <GIDInitSeq>9</GIDInitSeq>
    <Tconnd Addr="1.0.15.1" />
    <UDP Port="12869" />
    <dns_server_ip></dns_server_ip>
    <TCPPort>0</TCPPort>
    
<dns_udp_port>18083</dns_udp_port>
    
<cmd_http_listen_port>28081</cmd_http_listen_port>
    
    <Log>
        <OutputType>0</OutputType>
        <BaseLogPath>/data/home/user00/log/gamesvr_1.0.13.1</BaseLogPath>
        <!--默认的日志优先级-->
        <BaseLogPriority>DEBUG,INFO,SYSTEM,WARN,ERROR,FATAL</BaseLogPriority>
        <!--默认的文件格式描述-->
        <BaseLogFileCfg>
            <!-- 0 按文件大小偏移, 1按小时偏移, 2按天偏移-->
            <ShiftType>1</ShiftType>
            <MaxFileSize>102400000</MaxFileSize>
            <MaxFileNum>168</MaxFileNum>
        </BaseLogFileCfg>
        <!--QueueType  管道类型:0:主线程Ctrl, 1:Cfg线程,2:Handle -->
        <!--CatType    日志类型:0:主日志,   2,Player日志, 3,Room日志, 4: pb日志 5:Error日志-->
        <!--BufferedLen   需要使用buff时配置此值-->
        <Msg QueueType="0" CatType="0" Open="1" BufferedLen="8196" OutputPlayerLog="0">
            <Name>
                <!--文件基础名称-->
                <BaseName>svr</BaseName>
            </Name>
        </Msg>
        <Msg QueueType="0" CatType="4" Open="1" BufferedLen="8196" OutputPlayerLog="0" >
            <Name>
                <!--文件基础名称-->
                <Path>pb</Path>
                <BaseName>pb</BaseName>
            </Name>
        </Msg>
        <Msg QueueType="0" CatType="5" Open="1" BufferedLen="8196" OutputPlayerLog="0" OutputMain="1" >
            <Name>
                <!--文件基础名称-->
                <BaseName>svr_err</BaseName>
            </Name>
            <LogFileCfg>
                <!-- 0 按文件大小偏移-->
                <ShiftType>0</ShiftType>
                <MaxFileSize>102400000</MaxFileSize>
                <MaxFileNum>168</MaxFileNum>
            </LogFileCfg>
        </Msg>
        <Msg QueueType="1" CatType="0" Open="1" BufferedLen="0">
            <Name>
                <!--文件基础名称-->
                <Path>cfg</Path>
                <BaseName>cfg</BaseName>
            </Name>
            <LogPriority>INFO,SYSTEM,WARN,ERROR,FATAL</LogPriority>
        </Msg>
        <Msg QueueType="0" CatType="2" Open="1"  BufferedLen="0">
            <!--相对路径:相对于$Home/log/appname.进程号 -->
            <Name>
                <!--目录名称 不配默认在$Home/log/Appname.进程号/ 路径下-->
                <Path>player</Path>
                <!--文件基础名称-->
                <BaseName>player</BaseName>
            </Name>
            <Whitelist>
                <ID>11111</ID>
            </Whitelist>
        </Msg>
    </Log>
</ProcConf>

Generated json is like this:

{
    "meshTcpListenPort":  18182,
    "meshPolarisUdpName":  "Test-MeshUdp",
    "meshPolarisTcpName":  "Test-MeshTcp",
    "meshPloarisNamespace":  "Test",
    "serverIp":  "9.134.54.48",
    "gidInitSeq":  9,
    "dnsServerIp":  "",
    "dnsUdpPort":  18083,
    "cmdHttpListenPort":  28081,
    "tcpPort":  0,
    "world":  {
        "id":  1,
        "platformList":  [
            "AQQ",
            "IQQ",
            "AWX",
            "IWX"
        ]
    },
    "tconnd":  {
        "addrList":  [
            "1.0.15.1"
        ]
    },
    "udp":  {
        "port":  12869
    },
    "log":  {
        "outputType":  0,
        "baseLogPath":  "/data/home/user00/log/gamesvr_1.0.13.1",
        "baseLogPriorityList":  [
            "LOG_PRIORITY_DEBUG",
            "LOG_PRIORITY_INFO",
            "LOG_PRIORITY_SYSTEM",
            "LOG_PRIORITY_WARN",
            "LOG_PRIORITY_ERROR",
            "LOG_PRIORITY_FATAL"
        ],
        "baseLogFileCfg":  {
            "shiftType":  1,
            "maxFileSize":  102400000,
            "maxFileNum":  168
        },
        "msgList":  [
            {
                "queueType":  0,
                "catType":  0,
                "open":  true,
                "bufferedLen":  8196,
                "outputPlayerLog":  false,
                "outputMain":  false,
                "useRealTime":  false,
                "logPriorityList":  [],
                "name":  {
                    "path":  "",
                    "baseName":  "svr"
                },
                "logFileCfg":  null,
                "whitelist":  null
            },
            {
                "queueType":  0,
                "catType":  4,
                "open":  true,
                "bufferedLen":  8196,
                "outputPlayerLog":  false,
                "outputMain":  false,
                "useRealTime":  false,
                "logPriorityList":  [],
                "name":  {
                    "path":  "pb",
                    "baseName":  "pb"
                },
                "logFileCfg":  null,
                "whitelist":  null
            },
            {
                "queueType":  1,
                "catType":  0,
                "open":  true,
                "bufferedLen":  0,
                "outputPlayerLog":  false,
                "outputMain":  false,
                "useRealTime":  false,
                "logPriorityList":  [
                    "LOG_PRIORITY_INFO",
                    "LOG_PRIORITY_SYSTEM",
                    "LOG_PRIORITY_WARN",
                    "LOG_PRIORITY_ERROR",
                    "LOG_PRIORITY_FATAL"
                ],
                "name":  {
                    "path":  "cfg",
                    "baseName":  "cfg"
                },
                "logFileCfg":  null,
                "whitelist":  null
            },
            {
                "queueType":  0,
                "catType":  2,
                "open":  true,
                "bufferedLen":  0,
                "outputPlayerLog":  false,
                "outputMain":  false,
                "useRealTime":  false,
                "logPriorityList":  [
                    "LOG_PRIORITY_INFO",
                    "LOG_PRIORITY_SYSTEM",
                    "LOG_PRIORITY_WARN",
                    "LOG_PRIORITY_ERROR",
                    "LOG_PRIORITY_FATAL"
                ],
                "name":  {
                    "path":  "player",
                    "baseName":  "player"
                },
                "logFileCfg":  null,
                "whitelist":  {
                    "idList":  [
                        "11111"
                    ]
                }
            },
            {
                "queueType":  0,
                "catType":  5,
                "open":  true,
                "bufferedLen":  8196,
                "outputPlayerLog":  false,
                "outputMain":  true,
                "useRealTime":  false,
                "logPriorityList":  [
                    "LOG_PRIORITY_INFO",
                    "LOG_PRIORITY_SYSTEM",
                    "LOG_PRIORITY_WARN",
                    "LOG_PRIORITY_ERROR",
                    "LOG_PRIORITY_FATAL"
                ],
                "name":  {
                    "path":  "",
                    "baseName":  "svr_err"
                },
                "logFileCfg":  {
                    "shiftType":  0,
                    "maxFileSize":  102400000,
                    "maxFileNum":  168
                },
                "whitelist":  {
                    "idList":  [
                        "11111"
                    ]
                }
            }
        ]
    }
}

ecode: add `xerrors.Is` API to check ecode in error

  1. Implement custom xerrors package with advanced features, based on source code from https://pkg.go.dev/errors
  2. Add APIs to xerrors/error.go :
// Is reports whether any error in err's tree matches code.
func Is(err error, code tableaupb.Code) bool {
	return Code(err) == code
}
// Code returns the top-level code wrapped in error in err's tree.
func Code(err error) tableaupb.Code {
	if err == nil {
		return tableaupb.Code_SUCCESS
	}
	for err != nil {
		cause, ok := err.(xcauser)
		if !ok {
			break
		}
		if w, ok := err.(*withCode); ok {
			return w.Code()
		}
		err = cause.Cause()
	}
	return tableaupb.Code_ERR_UNKNOWN
}

Customizing error tests with Is and As methods

See The Go Blog: Working with Errors in Go 1.13

The errors.Is function examines each error in a chain for a match with a target value. By default, an error matches the target if the two are equal. In addition, an error in the chain may declare that it matches a target by implementing an Is method.

As an example, consider this error inspired by the Upspin error package which compares an error against a template, considering only fields which are non-zero in the template:

type Error struct {
    Path string
    User string
}

func (e *Error) Is(target error) bool {
    t, ok := target.(*Error)
    if !ok {
        return false
    }
    return (e.Path == t.Path || t.Path == "") &&
           (e.User == t.User || t.User == "")
}

if errors.Is(err, &Error{User: "someuser"}) {
    // err's User field is "someuser".
}

The errors.As function similarly consults an As method when present.

References

protogen: propose `union` type definition in a sheet

NOTE: this feature need issue #16 to be resolved ahead.

Define a message to parse this sheet in metabook.proto:

// UnionDescriptor represents union type definition in sheet.
message UnionDescriptor {
  option (tableau.worksheet) = {
    namerow: 1
    datarow: 2
  };

  repeated Value values = 1 [(tableau.field) = { layout: LAYOUT_VERTICAL }];
  message Value {
    optional int32 number = 1 [(tableau.field) = { name: "Number" optional: true }];
    // This is message type name, and the corresponding enum value name
    // is generated as: "TYPE_" + strcase.ToScreamingSnake(name).
    string name = 2 [(tableau.field) = { name: "Name" }];
    string alias = 3 [(tableau.field) = { name: "Alias" }];
    repeated string fields = 4 [(tableau.field) = { name: "Field" layout: LAYOUT_HORIZONTAL }];
  }
}

A sheet named Foo (also the message type name) in workbook:

Name Alias Field1 Field2 Field3
PvpBattle PVP BattleId
int32
Kills
uint32
Damage
int64
PveBattle PVE Highlight
{int32 BattleId, int32 BossId, int64 Damage}Highlight
Heros
[]int32
Dungeons
map<int32, int64>

See option Mode in issue #31,, metasheet @TABLEAU in workbook:

Sheet Mode
Foo MODE_UNION_TYPE

TODO

  • support incell struct field can be defined as enum type.
  • use predefined enum type in union type definition: 3-pass conversion

protogen: support explicit struct variable name definition

Design Goal

Used for identifying name prefix of continuous cells in namerow, when the
protogen can't auto-recognize the variable name.

Example:

RewardItemID RewardItemNum
{Item}int32 int32
Item's ID Item's num
1 100

In this example, protogen can not recognize the struct variable name and scalar variable name in sub-scope.

Syntax Rule

Just after the variable type, use parentheses () to explicitly annotate variable name: VariableType(VariableName).

Implicit struct variable name definition

The struct variable name is same as struct variable type.

ItemID ItemNum
{Item}int32 int32
Item's ID Item's num
1 100

Explicit struct variable name definition

The struct variable name is defined after struct variable type as: (VariableName).

Three practical cases are illustrated as below:

Case 1: the struct type name and variable name are not the same

RewardItemID RewardItemNum
{Item(RewardItem)}int32 int32
Reward's ID Reward's num
1 100

Case 2: the predefined struct variable type and variable name are not the same

Predefined Item as:

message Item {
  int32 id = 1 [(tableau.field) = { name: "ID" }];
  int32 num= 2 [(tableau.field) = { name: "Num" }];
}

Used as:

RewardItemID RewardItemNum
{.Item(RewardItem)}int32 int32
Reward's ID Reward's num
1 100

NOTE: in this case, protogen may detect the predefined Item's first field name ID, and
then recognize the struct variable name.

Case 3: reuse the struct type in the same scope

RewardItemID RewardItemNum CostItemID CostItemNum
{Item(RewardItem)}int32 int32 {Item(CostItem)}int32 int32
Reward's ID Reward's num Cost's ID Cost's num
1 100 2 200

refactor: add new `DocSheet` as middle layer for hierarchical file parsing

refer Pandas: tabular data

type Sheet struct {
Name string
MaxRow int
MaxCol int
Rows [][]string // 2D array of strings.
Meta *tableaupb.Metasheet
}

Support two kinds of descriptors:

  • flat formats: XLSX/CSV -> FlatSheet
  • hierachy formats: XML/YAML -> DocSheet

Define FlatSheet and DocSheet

 type baseSheet struct {
    Name string
    Meta * tableaupb.Metasheet
}

type FlatSheet struct {
    baseSheet
    Rows[][] string // 2D array of strings. 
}

type Attribute struct {
    Name string
    Value string
}

type Node struct {
    Name string
    Attributes[] Attribute
    Children []*Node
}

// NOTE: descriptor sheet (name starts with "@") describes the corresponding sheet's structure.
// E.g: "@ItemConf" describes the structure of "ItemConf"
type DocSheet struct {
    baseSheet
    Nodes[] Node
}

Refer:

feat: add support for incell-map with key/value as enum type

Incell-map with key/value as enum type

For example, enum type FruitType in common.proto is defined as:

enum FruitType {
  FRUIT_TYPE_UNKNOWN = 0 [(tableau.evalue).name = "Unknown"];
  FRUIT_TYPE_APPLE   = 1 [(tableau.evalue).name = "Apple"];
  FRUIT_TYPE_ORANGE  = 2 [(tableau.evalue).name = "Orange"];
  FRUIT_TYPE_BANANA  = 3 [(tableau.evalue).name = "Banana"];
}

enum FruitFlavor {
  FRUIT_FLAVOR_UNKNOWN = 0 [(tableau.evalue).name = "Unknown"];
  FRUIT_FLAVOR_FRAGRANT = 1 [(tableau.evalue).name = "Fragrant"];
  FRUIT_FLAVOR_SOUR = 2 [(tableau.evalue).name = "Sour"];
  FRUIT_FLAVOR_SWEET = 3 [(tableau.evalue).name = "Sweet"];
}

A worksheet ItemConf in HelloWorld.xlsx:

Fruit Flavor Item
map<enum<.FruitType>, int64> map<int64, enum<.FruitFlavor>> map<enum<.FruitType>, enum<.FruitFlavor>>
Fruits Flavors Items
Apple:1,Orange:2 1:Fragrant,2:Sweet Apple:Fragrant,Orange:Sour

protogen

// --snip--
import "common.proto";
option (tableau.workbook) = {name:"HelloWorld.xlsx"};

message ItemConf {
  option (tableau.worksheet) = {name:"ItemConf" namerow:1 typerow:2 noterow:3 datarow:4};

  map<int32, Fruit> fruit_map = 1 [(tableau.field) = {name:"Fruit" key:"Key" layout:LAYOUT_INCELL}];
  message Fruit {
    protoconf.FruitType key = 1 [(tableau.field) = {name:"Key"}];
    int64 value = 2 [(tableau.field) = {name:"Value"}];
  }
  map<int64, protoconf.FruitFlavor> flavor_map = 2 [(tableau.field) = {name:"Flavor" layout:LAYOUT_INCELL}];
  map<int32, Item> item_map = 3 [(tableau.field) = {name:"Item" key:"Key" layout:LAYOUT_INCELL}];
  message Item {
    protoconf.FruitType key = 1 [(tableau.field) = {name:"Key"}];
    protoconf.FruitFlavor value = 2 [(tableau.field) = {name:"Value"}];
  }
}

confgen

Need to support the generated proto.

{
    "fruitMap": {
        "1": {
            "key": "FRUIT_TYPE_APPLE",
            "value": "1"
        },
        "3": {
            "key": "FRUIT_TYPE_ORANGE",
            "value": "2"
        }
    },
    "flavorMap": {
        "1": "FRUIT_FLAVOR_FRAGRANT",
        "2": "FRUIT_FLAVOR_SWEET"
    },
    "itemMap": {
        "1": {
            "key": "FRUIT_TYPE_APPLE",
            "value": "FRUIT_FLAVOR_FRAGRANT"
        },
        "3": {
            "key": "FRUIT_TYPE_ORANGE",
            "value": "FRUIT_FLAVOR_SOUR"
        }
    }
}

metabook: add API to return all workbooks’ descriptors

Note: if this workbook has no metasheet @TABLEAU, then auto scan and find the main workbook (with Merger configured).

Tableau Descriptor

add new file descriptor.proto:

message BookDescriptorSet {
  map<string, BookDescriptor> books = 1;
}
// Describes a workbook file.
message BookDescriptor {
  string name = 1;
  WorkbookOptions book = 2;
  map<string, SheetDescriptor> sheets= 3;
}
// Describes a worksheet message.
message SheetDescriptor {
  string name = 1;
  WorksheetOptions sheet = 2;
}

bug: horizonal list of scalar type doesn't support refer check

I add refer check to a horizonal list of uint32
企业微信截图_16893231816335

But this doesn't work, the generated message is as follows (no refer prop generated):

message RuneSuitRecommendConf {
  option (tableau.worksheet) = {name:"Sheet1" namerow:1 typerow:2 noterow:3 datarow:5 nameline:1 typeline:2};

  map<uint32, Suit> suit_map = 1 [(tableau.field) = {key:"SuitID" layout:LAYOUT_VERTICAL}];
  message Suit {
    uint32 suit_id = 1 [(tableau.field) = {name:"SuitID"}];
    repeated uint32 red_rune_list = 2 [(tableau.field) = {name:"RedRune" layout:LAYOUT_HORIZONTAL prop:{fixed:true}}];
    repeated uint32 blue_rune_list = 3 [(tableau.field) = {name:"BlueRune" layout:LAYOUT_HORIZONTAL prop:{fixed:true}}];
    repeated uint32 green_rune_list = 4 [(tableau.field) = {name:"GreenRune" layout:LAYOUT_HORIZONTAL prop:{fixed:true}}];
  }
}

confgen(merger/scatter): auto find the primary workbook if only input a secondary workbook

Problem

If only input a secondary workbook path (without metasheet @TABLEAU), tableauc will not generate conf.

Expectation

tableauc should auto find the primary workbook path if only input a secondary workbook path , and generate conf.

Solution

  1. Scan all worksheet.merger and worksheet.scatter in the source proto files (which are already generated ahead), and build merger reverse index: primary/secondary workbook name -> primary workbook name.
  2. Based on merger reverse index, find corresponding primary workbook of each inputed workbook.
  3. Process the primary workbook.

protogen: propose `struct` type definition in a sheet

NOTE: this feature is related to issue #16.

Define a message to parse this sheet in metabook.proto:

// StructDescriptor represents struct type definition in sheet.
message StructDescriptor {
  option (tableau.worksheet) = {
    namerow: 1
    datarow: 2
  };

  repeated Field fields = 1 [(tableau.field) = { layout: LAYOUT_VERTICAL }];
  message Field {
    string name = 1 [(tableau.field) = { name: "Name" }];
    string type = 2 [(tableau.field) = { name: "Type" }];
  }
}

A sheet named Reward (also the enum type name) in workbook:

Name Type
ID uint32
Num int32
FruitType enum<.FruitType>
Feature []int32
Prop map<int32, string>
Detail {enum<.FruitType> Type, string Name, string Desc}Detail

See option Mode in issue #31, Metasheet @TABLEAU in workbook:

Sheet Mode
FruitType MODE_STRUCT_TYPE

Generated:

message Reward {
  uint32 id = 1 [(tableau.field) = { name: "ID" }];
  int32 num = 2 [(tableau.field) = { name: "Num" }];
  FruitType fruit_type = 3 [(tableau.field) = { name: "FruitType" }];
  repeated int32 feature_list = 4 [(tableau.field) = { name: "Features"span:SPAN_INNER_CELL}];
  map<int32, string> prop_map = 5 [(tableau.field) ={ name: "Props" span:SPAN_INNER_CELL}];
  Detail detail = 6 [(tableau.field) ={ name: "Detail" span:SPAN_INNER_CELL}];
  message Detail {
    protoconf.FruitType type = 1 [(tableau.field) = { name: "Type" }];
    string name = 2 [(tableau.field) = { name: "Name" }];
    string desc = 3 [(tableau.field) = { name: "Desc" }];
  }
}

TODO

tableauc: release the smallest `tableauc` executable

1. go build

In Go, it isn't typical to have a debug version or a release version.

By default, go build combines symbol and debug info with binary files. However, you can remove the symbol and debug info with go build -ldflags "-s -w".

It's not typical to strip symbols--if you get a report of a panic out in the wild, for example, it'd be great to have the symbols there for an informative stacktrace.

see https://stackoverflow.com/questions/29599209/how-to-build-a-release-version-binary-in-go

2. the Ultimate Packer for eXecutables

see https://upx.github.io/

FieldProp: add `join` option to combine sub-struct to main struct

The join concept is borrowed from SQL join:

A JOIN clause is used to combine rows from two or more tables, based on a related column between them.

Sheet Chapter

ID Name SectionID SectionDesc
map<uint32, Chapter> string map<uint32, Section>|{join:"Section.ID"} string
Chapter’s ID Chapter’s name Section’s ID Section’s Desc
1 Chapter1 101 SectionDesc1
1 Chapter1 102 SectionDesc2
1 Chapter1 103 SectionDesc3
2 Chapter2 201 SectionDesc4
2 Chapter2 202 SectionDesc5

Sheet Section

ID Name
map<uint32, Section> string
Section’s ID Section’s Name
101 Name1
102 Name2
103 Name3
201 Name4
202 Name5

feat: add support to generate UE DataTable: `ue-csv` and `ue-json`

References

Design

Extend Metasheet in proto/tableau/protobuf/metabook.proto:

message Metasheet {
  ...
  Mode mode = 17 [(tableau.field) = { name: "Mode" optional: true }];
}

enum Mode {
  MODE_DEFAULT = 0; // Default mode.
  // UE DataTable references:
  //  - https://docs.unrealengine.com/5.1/en-US/data-driven-gameplay-elements-in-unreal-engine/
  //  - https://docs.unrealengine.com/5.1/en-US/BlueprintAPI/EditorScripting/DataTable/
  MODE_UE_CSV = 1; // CSV format of UE DataTable.
  MODE_UE_JSON= 2; // JSON format of UE DataTable.
  MODE_ENUM_TYPE = 3; // Enum type definition in sheet.
  MODE_UNION_TYPE = 4; // Union type definition in sheet.
}

For example, metasheet @TABLEAU in workbook:

Sheet Mode
Skill MODE_UE_CSV
Buff MODE_UE_JSON

protogen: propose `enum` type definition in a sheet

NOTE: this feature is related to issue #16.

Define a message to parse this sheet in metabook.proto:

// EnumDescriptor represents enum type definition in sheet.
message EnumDescriptor {
  option (tableau.worksheet) = {
    namerow: 1
    datarow: 2
  };

  repeated Value values = 1 [(tableau.field) = { layout: LAYOUT_VERTICAL }];
  message Value {
    optional int32 number = 1 [(tableau.field) = { name: "Number" optional: true }];
    string name = 2 [(tableau.field) = { name: "Name" }];
    string alias = 3 [(tableau.field) = { name: "Alias" }];
  }
}

A sheet named ItemType (also the enum type name) in workbook:

Name Alias
ITEM_TYPE_DIAMOND Diamond
ITEM_TYPE_EQUIP Equip
ITEM_TYPE_BOX Box

Or with Number column which specify enum value explicitly:

Number Name Alias
1 ITEM_TYPE_DIAMOND Diamond
2 ITEM_TYPE_EQUIP Equip
3 ITEM_TYPE_BOX Box

See option Mode in issue #31, Metasheet @TABLEAU in workbook:

Sheet Mode
ItemType MODE_ENUM_TYPE

Generated:

// Generated from sheet: ItemType.
enum ItemType {
  ITEM_TYPE_INVALID = 0;
  ITEM_TYPE_DIAMOND = 1 [(tableau.evalue).name = "Diamond"];
  ITEM_TYPE_EQUIP = 2 [(tableau.evalue).name = "Equip"];
  ITEM_TYPE_BOX = 3 [(tableau.evalue).name = "Box"];
}

TODO

protogen

  • extraction: search dirs recursively and extract all workbooks with metatsheet @TABLEAU, and cache them in memory.
  • preprocessor: support inside and cross file references, if metasheet's Mode option is MODE_ENUM_TYPE, then convert this enum type to xproto.TypeInfo for later use.

confgen: implment field property: `refer`

Format: "SheetName(SheetAlias).ColumnName". Ensure this field is in another sheet column's (aka message’s field) value space.
E.g. ItemConf.ID.

TODO

  • Reading referred sheet column from Excel directly.
  • Use sync.RWMutex to cache KV SheetName(SheetAlias).ColumnName -> ValueSet only once.
  • Cover metasheet merger option: merging multiple sheets mechanism.
  • Report ERROR when sheet name not found, give suggestion: add (alias).

feat: support advanced incell message

Design

At some situations, we want to configure any complex message in one cell, which is called advanced incell message. And tableau (confgen) should support two kinds of protobuf serialized formats: text format, and JSON format.

Add new field prop mode to tableau.proto:

message FieldProp {
  ...
  // Specify cell's data form for parsing.
  Form form = 8;
}

enum Form {
  FORM_DEFAULT = 0;  // Default form which confgen parser defines.
  FORM_TEXT = 1;     // Refer: https://developers.google.com/protocol-buffers/docs/text-format-spec
  FORM_JSON = 2;     // Refer: https://developers.google.com/protocol-buffers/docs/proto3#json
}

Example

Predefined message Transform:

message Transform {
  Vector3 position = 1;
  Vector3 rotation = 2;
  Vector3 scale = 3;
}

message Vector3 {
  float x = 1;
  float y = 2;
  float z = 3;
}

1. message text format

refer: https://developers.google.com/protocol-buffers/docs/text-format-spec

Transform
{.Transform}|{form:FORM_TEXT}
Box's transform
position:{x:1 y:2 z:3} rotation:{x:4 y:5 z:6} scale:{x:7 y:8 z:9}

2. message JSON format

refer: https://developers.google.com/protocol-buffers/docs/proto3#json

Transform
{.Transform}|{form:FORM_JSON}
Box's transform
{"position":{"x":1, "y":2, "z":3}, "rotation":{"x":4, "y":5, "z":6}, "scale":{"x":7, "y":8, "z":9}}

XML: simplify config syntax

<?xml version="1.0" encoding="UTF-8" ?>
<!--
<@TABLEAU>
    <Item Sheet="LiteConf" />
    <Item Sheet="LoaderConf" />
</@TABLEAU>
<LiteConf>
    <RulerLite CacheExpire="duration" MaxBatchNum="int32" />
    <GuildLite CacheExpire="duration" MaxBatchNum="int32" />
</LiteConf>
<LoaderConf>
    <Server Name="map<string, Server>">
        <Conf Name="map<string, Conf>"/>
    </Server>
</LoaderConf>
-->
<LiteConf>
    <RulerLite CacheExpire="2h" MaxBatchNum="50" />
    <GuildLite CacheExpire="2h" MaxBatchNum="50" />
</LiteConf>
<LoaderConf>
    <Server Name="gamesvr">
        <Conf Name="ItemConf" />
        <Conf Name="DropConf" />
    </Server>
    <Server Name="mailsvr">
        <Conf Name="ItemConf" />
        <Conf Name="NoticeConf" />
    </Server>
</LoaderConf>

vs

<?xml version="1.0" encoding="UTF-8" ?>
<!--
<@TABLEAU>
    <Item Sheet="LiteConf" />
    <Item Sheet="LoaderConf" />
</@TABLEAU>
-->
<LiteConf>
    <!-- <RulerLite CacheExpire="duration" MaxBatchNum="int32" /> -->
    <RulerLite CacheExpire="2h" MaxBatchNum="50" />
    <!-- <GuildLite CacheExpire="duration" MaxBatchNum="int32" /> -->
    <GuildLite CacheExpire="2h" MaxBatchNum="50" />
</LiteConf>
<LoaderConf>
    <!-- <Server Name="map<string, Server>" /> -->
    <Server Name="gamesvr">
        <!-- <Conf Name="map<string, Conf>"/> -->
        <Conf Name="ItemConf" />
        <Conf Name="DropConf" />
    </Server>
    <Server Name="mailsvr">
        <Conf Name="ItemConf" />
        <Conf Name="NoticeConf" />
    </Server>
</LoaderConf>

all: improve error message

Such as enum value name not defined: xxx -> value:%v not found in enum.XXX?

  • scan all original error and rewrite the error message to make concise.
  • use template to implement i18n error messages.

confgen: check the column name is unique

Example

ID ID Desc
Map<uint32, Chapter> uin32 string
Chapter's ID Section's ID Section's desc
1 11 desc1
2 21 desc2
3 32 desc3

An error should be reported if more than two column names are same.

tableauc(confgen): add `Scatter` option to generate different named workbooks by same schema

Suppose there are many workbooks (each with same structure, and Zone1.xlsx is the main workbook):

  • Zone1.xlsx
  • Zone2.xlsx
  • Zone3.xlsx
  • ...

and main worbook's sheet ZoneConf is:

ID Name Difficulty
map<uint32, Zone> string int32
Zone’s ID Zone’s name Zone’s difficulty
1 Infinity 100
2 Desert 200
3 Snowfield 300

add Scatter option to metasheet, and main workbook's metasheet @tableau is:

Sheet Scatter
ZoneConf Zone*.xlsx
// --snip--
option (tableau.workbook) = {name:"Zone1.xlsx"};

message ZoneConf {
  option (tableau.worksheet) = {name:"ZoneConf " namerow:1 typerow:2 noterow:3 datarow:4 scatter: "Zone*.xlsx"};

  map<uint32, Zone> item_map = 1 [(tableau.field) = {key:"ID" layout:LAYOUT_VERTICAL}];
  message Zone {
    uint32 id = 1 [(tableau.field) = {name:"ID"}];
    string name = 2 [(tableau.field) = {name:"Name"}];
    int32 difficulty = 3 [(tableau.field) = {name:"Difficulty"}];
  }
}

It is supposed to generate different configurations (name pattern is : <BookName>_<SheetName>):

  • Zone1_ZoneConf.json
  • Zone2_ZoneConf.json
  • Zone3_ZoneConf.json
  • ...

confgen: check field value limit by field type

case pref.Int32Kind, pref.Sint32Kind, pref.Sfixed32Kind:
if value == "" {
return DefaultInt32Value, false, nil
}
// val, err := strconv.ParseInt(value, 10, 32)
// Keep compatibility with excel number format.
// maybe:
// - decimal fraction: 1.0
// - scientific notation: 1.0000001e7
val, err := strconv.ParseFloat(value, 64)
return pref.ValueOfInt32(int32(val)), true, xerrors.E2012("int32", value, err)

TODO: check value limit before type conversion.

analysis: generate sheet relation diagram about Merger/Scatter

message Diagram {
    map<string, Book> books = 1; // book name -> Book
    message Book {
        string alias = 1; // alias
        map<string, Sheet> sheets = 2; // sheet name -> Sheet
    }
    message Sheet {
        string alias = 1; // alias
        Primary primary = 2;
    }
    message Primary {
        string book_name = 1;
        string sheet_name = 2;
        RelationType relation_type = 3; 
    }
    enum RelationType {
        RELATION_TYPE_MERGER = 1;
        RELATION_TYPE_SCATTER = 2;
    }
}

confgen: cannot parse well-known datetime field of incell struct in union

Problem

image

  message GainFreeAward {
    google.protobuf.Duration start_time = 1 [(tableau.field) = {name:"StartTime"}];
    google.protobuf.Duration end_time = 2 [(tableau.field) = {name:"EndTime"}];
  }
2023-05-18T15:24:40.888+0800|ERROR|tableauc/main.go:133|main.logError|generate conf file failed: |Reason: strconv.ParseFloat: parsing "11:00:00": invalid syntax
github.com/tableauio/tableau/xerrors.ErrorKV
	/data/home/user00/code/tableau/xerrors/error.go:36
github.com/tableauio/tableau/xerrors.WithStack
	/data/home/user00/code/tableau/xerrors/error.go:16
github.com/tableauio/tableau/internal/xproto.ParseFieldValue
	/data/home/user00/code/tableau/internal/xproto/value.go:103
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseFieldValue
	/data/home/user00/code/tableau/internal/confgen/parser.go:1256
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseIncellStruct
	/data/home/user00/code/tableau/internal/confgen/parser.go:1064
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseUnionValueField
	/data/home/user00/code/tableau/internal/confgen/parser.go:1185
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseUnionField.func1
	/data/home/user00/code/tableau/internal/confgen/parser.go:1137
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseUnionField
	/data/home/user00/code/tableau/internal/confgen/parser.go:1143
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseField
	/data/home/user00/code/tableau/internal/confgen/parser.go:321
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseFieldOptions.func1
	/data/home/user00/code/tableau/internal/confgen/parser.go:296
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseFieldOptions
	/data/home/user00/code/tableau/internal/confgen/parser.go:305
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseListField
	/data/home/user00/code/tableau/internal/confgen/parser.go:760
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseField
	/data/home/user00/code/tableau/internal/confgen/parser.go:318
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseFieldOptions.func1
	/data/home/user00/code/tableau/internal/confgen/parser.go:296
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseFieldOptions
	/data/home/user00/code/tableau/internal/confgen/parser.go:305
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseMapField
	/data/home/user00/code/tableau/internal/confgen/parser.go:368
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseField
	/data/home/user00/code/tableau/internal/confgen/parser.go:316
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseFieldOptions.func1
	/data/home/user00/code/tableau/internal/confgen/parser.go:296
github.com/tableauio/tableau/internal/confgen.(*sheetParser).parseFieldOptions
	/data/home/user00/code/tableau/internal/confgen/parser.go:305
github.com/tableauio/tableau/internal/confgen.(*sheetParser).Parse
	/data/home/user00/code/tableau/internal/confgen/parser.go:269
github.com/tableauio/tableau/internal/confgen.parseMessageFromOneImporter
	/data/home/user00/code/tableau/internal/confgen/parser.go:124
github.com/tableauio/tableau/internal/confgen.ParseMessage.func1
	/data/home/user00/code/tableau/internal/confgen/parser.go:75
golang.org/x/sync/errgroup.(*Group).Go.func1
	/data/home/user00/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:57
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1598
|PBFieldType: incell struct
|PBFieldType: incell struct
|SheetName: SectionConf|DataCellPos: F234|DataCell: 11:00:00|ColumnName: ActivityTargetField1|PBFieldType: union value field
|PBFieldName: activity_target|PBFieldOpts: name:"ActivityTarget"  sep:","  subsep:":"
|cross-cell struct list: failed to parse struct
|PBFieldName: section_list|PBFieldOpts: layout:LAYOUT_VERTICAL  sep:","  subsep:":"
|SheetName: SectionConf|DataCellPos: A234|DataCell: 10004301|ColumnName: ChapterId|PBFieldType: vertical map
|PBFieldName: chapter_map|PBFieldOpts: key:"ChapterId"  layout:LAYOUT_VERTICAL  sep:","  subsep:":"
|SheetName: SectionConf|PBMessage: SectionConf
|Module: confgen|BookName: conf/client/Common/DB/excel/Activity/Activity.xlsx
2023-05-18T15:24:40.889+0800|ERROR|tableauc/main.go:135|main.logError|Debugging: 
	Module: confgen
	BookName: conf/client/Common/DB/excel/Activity/Activity.xlsx
	SheetName: SectionConf
	DataCellPos: F234
	DataCell: 11:00:00
	PBMessage: SectionConf
	PBFieldName: activity_target
	PBFieldType: incell struct
	PBFieldOpts: name:"ActivityTarget"  sep:","  subsep:":"
	ColumnName: ActivityTargetField1
	Reason: strconv.ParseFloat: parsing "11:00:00": invalid syntax


工作簿: conf/client/Common/DB/excel/Activity/Activity.xlsx
表单名: SectionConf
单元格位置: F234
单元格的值: 11:00:00
错误原因: strconv.ParseFloat: parsing "11:00:00": invalid syntax

YAML: support yaml

Syntax v1

Solution 1

---
#@sheet @TABLEAU # doc's first line is sheet name line.
LiteConf:
  Sheet: LiteConf
  OrderedMap: true
LoaderConf:
  Sheet: LoaderConf
 
---
#@sheet LiteConf
RulerLite: #@type LiteInfo
  CacheExpire: 2h #@type duration
  MaxBatchNum: 50 #@type int32
GuildLite: #@type LiteInfo
  CacheExpire: 2h  #@type duration
  MaxBatchNum: 50  #@type int32

---
#@sheet LoaderConf
Servers: #@type map<string, Server>
  gamesvr:
    Name: gamesvr #@type string
    Confs:  #@type map<string, Conf>
      ItemConf: 
        Async: true #@type bool
      DropConf:
        Async: true #@type bool
  mailsvr:
    Name: mailsvr
    Confs: 
      ItemConf: 
        Async: true
      DropConf:
        Async: true

Solution 2

---
"@TABLEAU":
  - Sheet: LiteConf
    OrderedMap: true
  - Sheet: LoaderConf

---
LiteConf:
  RulerLite: #@type RulerLite
    CacheExpire: 2h #@type duration
    MaxBatchNum: 50 #@type int32
  GuildLite: #@type LiteInfo
    CacheExpire: 2h  #@type duration
    MaxBatchNum: 50  #@type uint64

---
LoaderConf:
  Server: #@type map<string, Server>
    gamesvr:
      Name: gamesvr #@type string
      Conf:  #@type map<string, Conf>
        ItemConf:
          Async: true #@type bool
        DropConf:
          Async: true #@type bool
    mailsvr:
      Name: mailsvr
      Confs:
        ItemConf:
          Async: true
        DropConf:
          Async: true

Generated proto

message LiteConf {
  option (tableau.worksheet) = {name:"LiteConf"};

  LiteInfo ruler_lite= 1 [(tableau.field) = {name:"RulerLite"}];
  message LiteInfo {
    google.protobuf.Duration cache_expire = 1 [(tableau.field) = {name:"CacheExpire"}];
    int32 max_batch_num= 2 [(tableau.field) = {name:"MaxBatchNum"}];
  }
  LiteInfo guild_lite= 1 [(tableau.field) = {name:"GuildLite"}];
}

message LoaderConf {
  option (tableau.worksheet) = {name:"LoaderConf"};

  map<string, Server> server_map = 1 [(tableau.field) = {key:"Key"}];
  message Server {
    string key = 1 [(tableau.field) = {name:"Key"}];
    string name = 2 [(tableau.field) = {name:"Name"}];
    map<string, Conf> conf_map= 3 [(tableau.field) = {name:"Confs" key:"Confs"}];
      message Conf {
        string key = 1 [(tableau.field) = {name:"Key"}];
        bool async= 2 [(tableau.field) = {name:"Async"}];
      }
  }
}

union: support horizontal/vertical union list

Predefined in proto

A union type should be predefined:

// Predefined union type.
message Target {
  option (tableau.union) = true;

  Type type = 9999 [(tableau.field) = { name: "Type" }];
  oneof value {
    option (tableau.oneof) = {
      field: "Field"
    };
    Pvp pvp = 1;      // Bound to enum value 1: TYPE_PVP.
    Pve pve = 2;      // Bound to enum value 2: TYPE_PVP.
    Story story = 3;  // Bound to enum value 3: TYPE_STORY.
    Skill skill = 4;  // Bound to enum value 4: TYPE_SKILL.
  }

  enum Type {
    TYPE_NIL = 0;
    TYPE_PVP = 1 [(tableau.evalue) = { name: "PVP" }];
    TYPE_PVE = 2 [(tableau.evalue) = { name: "PVE" }];
    TYPE_STORY = 3 [(tableau.evalue) = { name: "Story" }];
    TYPE_SKILL = 4 [(tableau.evalue) = { name: "Skill" }];
  }
  message Pvp {
    int32 type = 1;                          // scalar
    int64 damage = 2;                        // scalar
    repeated protoconf.FruitType types = 3;  // incell enum list
  }
  message Pve {
    Mission mission = 1;             // incell struct
    repeated int32 heros = 2;        // incell list
    map<int32, int64> dungeons = 3;  // incell map

    message Mission {
      int32 id = 1;
      uint32 level = 2;
      int64 damage = 3;
    }
  }
  message Story {
    protoconf.Item cost = 1;                     // incell predefined struct
    map<int32, protoconf.FruitType> fruits = 2;  // incell map with value as enum type
    map<int32, Flavor> flavors = 3;              // incell map with key as enum type
    message Flavor {
      protoconf.FruitFlavor key = 1 [(tableau.field) = { name: "Key" }];
      int32 value = 2 [(tableau.field) = { name: "Value" }];
    }
  }
  message Skill {
    int32 id = 1;      // scalar
    int64 damage = 2;  // scalar
    // no field tag 3
  }
}

Horizontal union list

ID Target1Type Target1Field1 Target1Field2 Target1Field3 Target2Type Target2Field1 Target2Field2 Target2Field3
map<int32, Task> [.Target]enum<.Target.Type> union union union enum<.Target.Type> union union union
ID Target1's type Target1's value field1 Target1's value field2 Target1's value field3 Target2's type Target2's value field1 Target2's value field2 Target2's value field3
1 PVP 1 10 100 PVE 1,100,999 1,2,3 1:10,2:20,3:30
2 PVE 1,100,999 1,2,3 1:10,2:20,3:30 PVP 1 10 100

TODO

  • support union custom variable name

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.