m-zajac/json2go

Create go type representation from json

json2go

Package json2go provides utilities for creating go type representation from json inputs.

Json2go can be used in various ways:

• CLI tool
• Web page with conversion tool: https://m-zajac.github.io/json2go
• Go package
• VSCode extension: vsc-json2go

Why another conversion tool?

There are few tools for converting json to go types available already. But all of which I tried worked correctly with only basic documents.

The goal of this project is to create types, that are guaranteed to properly unmarshal input data. There are multiple test cases that check marshaling/unmarshaling both ways to ensure generated type is accurate.

Here's the micro acid test if you want to check this or other conversion tools:

[{"date":"2020-10-03T15:04:05Z","text":"txt1","doc":{"x":"x"}},{"date":"2020-10-03T15:05:02Z","_doc":false,"arr":[[1,null,1.23]]},{"bool":true,"doc":{"y":123}}]

And correct output with some comments:

type Document []struct {
        Arr  [][]*float64 `json:"arr,omitempty"` // Should be doubly nested array; should be a pointer type because there's null in values.
        Bool *bool        `json:"bool,omitempty"` // Shouldn't be `bool` because when key is missing you'll get false information.
        Date *time.Time   `json:"date,omitempty"` // Could be also `string` or `*string`.
        Doc  *struct {
                X *string `json:"x,omitempty"` // Should be pointer, because key is not present in all documents.
                Y *int    `json:"y,omitempty"` // Should be pointer, because key is not present in all documents.
        } `json:"doc,omitempty"` // Should be pointer, because key is not present in all documents in array.
        Doc2 *bool   `json:"_doc,omitempty"` // Attribute for "_doc" key (other that for "doc"!). Type - the same as `Bool` attribute.
        Text *string `json:"text,omitempty"` // Could be also `string`.
}

CLI Installation

go get github.com/m-zajac/json2go/...

Usage

Json2go can be used as cli tool or as package.

CLI tools can be used directly to create go type from stdin data (see examples).

Package provides Parser, which can consume multiple jsons and outputs go type fitting all inputs (see examples and documentation). Example usage: read documents from document-oriented database and feed them too parser for go struct.

CLI usage examples

echo '{"x":1,"y":2}' | json2go

---

curl -s https://api.punkapi.com/v2/beers?page=1&per_page=5 | json2go

Check this one :)

---

cat data.json | json2go

Package usage examples

inputs := []string{
	`{"x": 123, "y": "test", "z": false}`,
	`{"a": 123, "x": 12.3, "y": true}`,
}

parser := json2go.NewJSONParser("Document")
for _, in := range inputs {
	parser.FeedBytes([]byte(in))
}

res := parser.String()
fmt.Println(res)

Example outputs

{
    "line": {
        "point1": {
            "x": 12.1,
            "y": 2
        },
        "point2": {
            "x": 12.1,
            "y": 2
        }
    }
}

type Document struct {
        Line struct {
                Point1 Point `json:"point1"`
                Point2 Point `json:"point2"`
        } `json:"line"`
}
type Point struct {
        X float64 `json:"x"`
        Y int     `json:"y"`
}

---

[
    {
        "name": "water",
        "type": "liquid",
        "boiling_point": {
            "units": "C",
            "value": 100
        }
    },
    {
        "name": "oxygen",
        "type": "gas",
        "density": {
            "units": "g/L",
            "value": 1.429
        }
    },
    {
        "name": "carbon monoxide",
        "type": "gas",
        "dangerous": true,
        "boiling_point": {
            "units": "C",
            "value": -191.5
        },
        "density": {
            "units": "kg/m3",
            "value": 789
        }
    }
]

type Document []struct {
        BoilingPoint *UnitsValue `json:"boiling_point,omitempty"`
        Dangerous    *bool       `json:"dangerous,omitempty"`
        Density      *UnitsValue `json:"density,omitempty"`
        Name         string      `json:"name"`
        Type         string      `json:"type"`
}
type UnitsValue struct {
        Units string  `json:"units"`
        Value float64 `json:"value"`
}

Contribution

I'd love your input! Especially reporting bugs and proposing new features.

json2go's Issues

Typo in description and README

formated should be formatted.

Output to type Name struct

How do we use a *JSONParser as a type struct without copying and pasting the .string ??? I.e. define a struct with the output of parser.FeedValue() without having to copy and pee.

Null values in arrays always result in []interface{} type

This json: [1, null, 2, 3] generates type []interface{}. But it should produce type similar to what is generated for this data: [{"x":1}, {"x":null}, {"x":2}, {"x":3}]. Valid result should be: []*int

Incorrect struct type

The struct type is incorrectly created for the newly created datatype. The newly datatype is created as an array struct when one of the extractable substructure is an array at one place and non-array in other.

Input
{"starttime":"2021-03-10T08:04:05Z", "endtime":"2021-03-10T15:04:05Z", "details": [{"state":"working", "duration":25 }, {"state":"idle", "duration":15 }, {"state":"working", "duration":20 }] },
{"starttime":"2021-03-10T08:04:05Z", "endtime":"2021-03-10T15:04:05Z", "totalwork": {"state":"working", "duration":25 } },

Ouput
Expected
type Document struct {
Details []DurationState json:"details,omitempty"
Endtime time.Time json:"endtime"
Starttime time.Time json:"starttime"
Totalwork *DurationState json:"totalwork,omitempty"
}
type DurationState struct {
Duration int json:"duration"
State string json:"state"
}

Actual
type Document struct {
Details []DurationState json:"details,omitempty"
Endtime time.Time json:"endtime"
Starttime time.Time json:"starttime"
Totalwork *DurationState json:"totalwork,omitempty"
}
type DurationState []struct {
Duration int json:"duration"
State string json:"state"
}

Here the DurationState is created as an array struct when it should just be a struct.

I am willing to provide a PR for this issue with fix. Please let me know.

feature request: add time.Time and few other changes

1. Please see if it is possible to support time.Time for dates.
2. Option to not sort golang struct alphabatically basically preserving the json appearance of fields in golang struct.
   Excellent tool btw

Output format differs from what go fmt uses

Add support for recurrent types

Calling parser.String() multiple times results in invalid output

Nested arrays parsing error

package main

import (
	"fmt"

	"github.com/m-zajac/json2go"
)

func main() {
	arrSInp := []string{
		`[[{"a": 2, "b": 3}, {"a": 4, "b": 5}]]`,
		`[[{"a":6, "b": 7}, {"a": 8, "b": 9}]]`,
	}
  
	parser := json2go.NewJSONParser("X")
	for _, in := range arrSInp {
		parser.FeedBytes([]byte(in))
	}
	fmt.Println(parser.String())
}

want:

  type X [][]struct {
          A       int     `json:"a"`
          B       int     `json:"b"`
  }

got:

  type X []struct {
          A       int     `json:"a"`
          B       int     `json:"b"`
  }

Invalid output for some inputs

Examples to check:

[1, 2, 3.1]

{
"_id":123,
"id":123
}

{
"i":"sda",
"i":1
}

{
  "test\"`\n}\n \nfunc init() {\n panic(\"\")\nvar test = `\"": 123
}