datadog/jsonapi

A marshaler/unmarshaler for JSON:API.

jsonapi

Package jsonapi implements a marshaler/unmarshaler for JSON:API v1.0.

Version

This package is in production use at DataDog and should be considered stable and production ready.

We follow semver and are reserving the v1.0.0 release until:

• JSON:API v1.1 is released and we evaluate any breaking changes needed (unlikely)
• Community adoption and feedback are evaluated
• Continued internal use lets us gain more experience with the package

Quickstart

Take a look at the go reference for more examples and detailed usage information.

Marshaling

jsonapi.Marshal

type Article struct {
    ID    string `jsonapi:"primary,articles"`
    Title string `jsonapi:"attribute" json:"title"`
}

a := Article{ID: "1", Title:"Hello World"}

b, err := jsonapi.Marshal(&a)
if err != nil {
    // ...
}

fmt.Println("%s", string(b))
// {
//     "data": {
//         "id": "1",
//         "type": "articles",
//         "attributes": {
//             "title": "Hello World"
//         }
//     }
// }

Unmarshaling

jsonapi.Unmarshal

body := `{"data":{"id":"1","type":"articles","attributes":{"title":"Hello World"}}}`

type Article struct {
    ID    string `jsonapi:"primary,articles"`
    Title string `jsonapi:"attribute" json:"title"`
}

var a Article
if err := jsonapi.Unmarshal([]byte(body), &a); err != nil {
    // ...
}

fmt.Prints("%s, %s", a.ID, a.Title)
// "1", "Hello World"

Reference

The following information is well documented in the go reference. This section is included for a high-level overview of the features available.

Struct Tags

Like encoding/json jsonapi is primarily controlled by the presence of a struct tag jsonapi. The standard json tag is still used for naming and omitempty.

Tag	Usage	Description	Alias
primary	jsonapi:"primary,{type},{omitempty}"	Defines the identification field. Including omitempty allows for empty IDs (used for server-side id generation)	N/A
attribute	jsonapi:"attribute"	Defines an attribute.	attr
relationship	jsonapi:"relationship"	Defines a relationship.	rel
meta	jsonapi:"meta"	Defines a meta object.	N/A

Functional Options

Both jsonapi.Marshal and jsonapi.Unmarshal take functional options.

Option	Supports
jsonapi.MarshalOption	meta, json:api, includes, document links, sparse fieldsets, name validation
jsonapi.UnmarshalOption	meta, document links, name validation

Non-String Identifiers

Identification MUST be represented as a string regardless of the actual type in Go. To support non-string types for the primary field you can implement optional interfaces.

You can implement the following on the parent types (that contain non-string fields):

Context	Interface
Marshal	jsonapi.MarshalIdentifier
Unmarshal	jsonapi.UnmarshalIdentifier

You can implement the following on the field types themselves if they are not already implemented.

Context	Interface
Marshal	fmt.Stringer
Marshal	encoding.TextMarshaler
Unmarshal	encoding.TextUnmarshaler

Order of Operations

Marshaling

1. Use MarshalIdentifier if it is implemented on the parent type
2. Use the value directly if it is a string
3. Use fmt.Stringer if it is implemented
4. Use encoding.TextMarshaler if it is implemented
5. Fail

Unmarshaling

1. Use UnmarshalIdentifier if it is implemented on the parent type
2. Use encoding.TextUnmarshaler if it is implemented
3. Use the value directly if it is a string
4. Fail

Links

Links are supported via two interfaces and the Link type. To include links you must implement one or both of the following interfaces.

Type	Interface
Resource Object Link	Linkable
Resource Object Related Resource Link	LinkableRelation

Alternatives

google/jsonapi

• exposes an API that looks/feels a lot different than encoding/json
• has quite a few bugs w/ complex types in attributes
• doesn't provide easy access to top-level fields like meta
• allows users to generate invalid JSON:API
• not actively maintained

manyminds/api2go/jsonapi

• has required interfaces
• interfaces for includes/relationships are hard to understand and verbose to implement
• allows users to generate invalid JSON:API
• part of a broader api2go framework ecosystem

jsonapi's Issues

Embedded structs are not marshalled

Describe the bug
Feel free to file as a feature request if this is intentional behavior.
The standard json marshaller includes embedded structs but the jsonapi marshaller does not.

Expected behavior
Currently embedded structs are not marshalled: https://go.dev/play/p/hLwiUcAHHcF
However the json marshaller does: https://go.dev/play/p/3cBZDtVDyke

Status field type invalid when try to unmarshal a JSON:API error

Describe the bug
When we are using Datadog/jsonapi to consume an API that use JSON:API standard and try to unmarshal a error payload (using native json.Unmarshal()), it not possible because this package set status as a int field, and JSON:API send status as string by default.

Expected behavior
json.Unmarshal() should handle the string status value and convert to int field correctly.

It would be event better if jsonapi.Unmarshal() handle error payload in some way. But at the time, just allow that the field could be unmarshaled by Go native unmarshal is sufficiente. Package customer could do just a Status check to choose if uses jsonapi.Unmarshal() for resources or json.Unmarshal() for errors.

Links
Example: https://go.dev/play/p/Dwa4pztKo0V

Additional context
I'm using this package to consume a third party API that follows JSON:API. When the HTTP status received is a error status a try to Unmarshal the payload as an Error instead of a Resource.

Support member name validation

Is your feature request related to a problem? Please describe.

The JSON:API v1.0 spec and beyond details hard-requirements for member names in https://jsonapi.org/format/1.0/#document-member-names.

The official site also suggests recommended member name restrictions in https://jsonapi.org/recommendations/#naming

Describe the solution you'd like

• By default, member names are validated against the hard-requirements during marshaling or unmarshaling.
• There exists an option to validate member names against the recommended requirements during marshaling or unmarshaling.
• There exists an option to disable member name validation for unique API compatibility or performance reasons.

Describe alternatives you've considered

N/A. At the very least, default member name validation is required by the spec and important for this library implementation.

Additional context

N/A

Resource Object Uniqueness is Not Enforced

Describe the bug
The uniqueness of resource objects (using type+id pairs) is not being enforced by Marshal or Unmarshal

Expected behavior
An error should be produced whenever two resource objects or resource linkages are referenced in a document, which have the same type+id pair.

Links
https://go.dev/play/p/UoFxKltVVI4

Additional context
N/A

Support models without exported ID/primary field

Is your feature request related to a problem? Please describe.

This is a follow up to #10 . We have a use-case where models are used for create only so there is no primary attribute/we don't want to expose this to end user. With the current client mode implementation we have to define it if we want to use jsonapi. E.g.

type Example {
	ID string `jsonapi:"primary,random_type"`
}

Describe the solution you'd like

I understand the tag in primary attribute is used for the type. So to work with this, it would be nice to support private/anonymous primary key:

type Example {
	id interface{} `jsonapi:"primary,random_type"`
}

or

type Example {
	string `jsonapi:"primary,random_type"`
}

Support relationships in Unmarshal

Is your feature request related to a problem? Please describe.

Right now we don't support unmarshaling fields tagged w/ relationship. This is required by the spec: https://jsonapi.org/format/#crud-creating. As a consequence we also don't support unmarshalling include data referenced by those relationships into the underlying struct fields.

Describe the solution you'd like

When a field is tagged w/ relationship and included in a request body it should be unmarshaled into the given data. If the relationship's resource object is present in the include section, then the data within is used during initialization.

Describe alternatives you've considered

N/A

Additional context

N/A

Unmarshalling single object with relationship with no data attribute throws exception

Describe the bug

Introduced in 0.72 it seems that there is an issue unmarshalling objects (non-array) with relationships that do not contain a data attribute

Here is an example code snippet causing the unmarshalling error:

package main

import (
	"fmt"

	"github.com/DataDog/jsonapi"
)

func main() {
	body := `{"data":{"id":"1","type":"articles","attributes":{"title":"Hello World"},"relationships":{"author":{"links":{"self":"https://localhost/article/1/relationships/author","related":"https://localhost/article/1/author"}}}}}`

	type Author struct {
		ID   string `jsonapi:"primary,authors"`
		Name string `jsonapi:"attribute" json:"name"`
	}

	type Article struct {
		ID     string  `jsonapi:"primary,articles"`
		Title  string  `jsonapi:"attribute" json:"title"`
		Author *Author `jsonapi:"relationship" json:"author"`
	}

	var a Article
	err := jsonapi.Unmarshal([]byte(body), &a)
	if err != nil {
		panic(err)
	}

	fmt.Printf("%+v", &a)
}

The result is an exception:

panic: document is missing a required top-level or relationship-level data member

Expected behavior

Unmarshalling should not throw an exception.

Support Unmarshaling of Errors

Is your feature request related to a problem? Please describe.
Currently our lib does not support Unmarshaling errors because it is not seen in requests to JSON:API servers, but it would be a nice-to-have feature for testing / consuming responses from JSON:API-servers.

Describe the solution you'd like
Unmarshal works for jsonapi.Error

Describe alternatives you've considered
N/A

Additional context
There are other things that this lib does not Unmarshal, like links or relationship metas, that may be interesting to do as well

Unmarshalling relationships with null data bodies throws an exception

Describe the bug

When unmarshalling jsonapi data with relationships including null data bodies an exception is thrown:

body is not a json:api representation of *model.SomeModel

The following is an example relationship that causes this behaviour.

"invoice": {
    "links": {
        "self": "https://api.example.io/v1/invoices/123456789/relationships/invoice",
        "related": "https://api.example.io/v1/invoices/123456789/invoice"
    },
    "data": null
},

Expected behavior

Relationship jsonapi such as above is apparently valid according to the spec and therefore should simply be ignored by the code. This was proved by simply removing the offending jsonapi relationships from the source json which allowed for a problem free and correct unmarshalling of the whole json.

Support for unmarshalling top-level links

Is your feature request related to a problem? Please describe.
The package does not support unmarshalling top-level links, usually utilizes for pagination.

Describe the solution you'd like
UnmarshalOption to export the links from top-level, which has been done with meta. for example:

body := `{"data":{"id":"1","type":"articles","attributes":{"title":"Hello World"}},"links":{"self":"foobar"}}`

type Article struct {
    ID    string `jsonapi:"primary,articles"`
    Title string `jsonapi:"attribute"        json:"title"`
}

var (
    a Article
    l jsonapi.Link
)

err := jsonapi.Unmarshal([]byte(body), &a, jsonapi.UnmarshalLinks(&l))
// l.self = "foobar"

Describe alternatives you've considered
n/a

Support "client mode" serialization

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Right now we require id to be present for all serialized data. This is correct for server-side serialization as it's required for valid JSON:API. However for a client serializing JSON:API to send to a server ID is optional (client vs. server-side ID's).

Describe the solution you'd like
A clear and concise description of what you want to happen.

A way to optionally disable (at least) required ID verification for client serialization.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

User other packages that don't have this validation.

Additional context
Add any other context or screenshots about the feature request here.

There may be more, but ID is the critical one.

Behaviour for omitted fields in Unmarshal differs from json.Unmarshal

Describe the bug
When unmarshalling a omitted github.com/google/uuid UUID a error is returned that is not there on json.Unmarshal

Expected behavior
Unmarshal should not be called for omitted fields

Links
https://go.dev/play/p/tih4SyA3Jca

Options ignored when marshaling nil value

Describe the bug

Marshaling a nil value ignores marshaling options.

Expected behavior

I would expect marshaling to apply marshaling options even with a nil value.

Links

https://go.dev/play/p/dtjasHU3_5T

Additional context

The same bug occurs when marshaling a nil slice.

Support for multiple type in the same relationship

Is your feature request related to a problem? Please describe.

When fetching a resource (say an article) I can get either

{
  "data": {
    "id": "1",
    "type": "articles",
    "attributes": { "title": "A" },
    "relationships": { "author": { "data": { "id": "1", "type": "human" } } }
  }
}

or

{
  "data": {
    "id": "1",
    "type": "articles",
    "attributes": { "title": "A" },
    "relationships": { "author": { "data": { "id": "xorg", "type": "bot" } } }
  }
}

Describe the solution you'd like

I would like to be able to Marshal/Unmarshal such a json api structure.

Describe alternatives you've considered

Tried to find a smart way to use StringIdentifier but not working.
I also tried to define twice the relationship field in my golang struct with the same json author key

type Article struct {
...
HumanAuthor        HumanAuthor `jsonapi:"relationship" json:"author"`
BotAuthor              BotAuthor `jsonapi:"relationship" json:"author"`
}

but the same json field cannot co-exist.

Additional context
Add any other context or screenshots about the feature request here.

Unmarshal does not create non-nil slices for empty array relationship data

Describe the bug
When an input document is only an empty primary data array, or if its a non-empty document with some empty array attribute value, then those arrays are Unmarshaled into a non-nil empty Go slice of the desired type.

However, when relationship resource linkage data is an empty array, it gets Unmarshaled into a nil Go slice of the desired type.

Expected behavior
Empty relationship arrays should be Unmarshaled into non-nil slices for greater consistency.

Links
https://go.dev/play/p/v0PJVJc8k7-

Additional context
N/A

Add Fuzz Tests for Unmarshal

Is your feature request related to a problem? Please describe.

While there is a comprehensive suite of documents used to test Unmarshal, it is still difficult to determine if there are missing edge-cases. The complexity of documents we need to test will only increase as we JSON:API v1.1 support is introduced.

Describe the solution you'd like

• Go fuzz tests are now used for Unmarshal
  • If possible, the fuzz tests should replace many of the existing ones (as the example documents can become the seed corpus, which can be executed directly as normal go tests).
• If any bugs with Unmarshal are discovered while running the tests in fuzz mode, they are filed/fixed, and the generated test case is committed to the repo (this comes for free with the fuzzing tool).

Describe alternatives you've considered

Continuing to manually construct example documents and rely on bug reports for edge-cases is still an important part of the validation process, but doesn't scale quite as well as automatic corpus creation with Go fuzzing.

Additional context

Fuzz testing cannot be easily done for Marshal, as the Go Fuzzing mutator only works on primitive types, not structs. There may be alternate fuzzing solutions or corpus creation methods to explore for Marshal in the future.

Empty array unmarshaling fails

Describe the bug
A clear and concise description of what the bug is.

Unmarshaling empty arrays fails. https://go.dev/play/p/sHO6I_iSoCa

Expected behavior
A clear and concise description of what you expected to happen.

Should get an emtpy output arraay.

Links
If possible, add https://go.dev/play/ links to replicate the issue.

Additional context
Add any other context about the problem here.