r3labs/diff

A library for diffing golang structures

diff's Issues

Differ without state

Hi.

Could we do not use state when diff structs?
https://github.com/r3labs/diff/blob/master/diff_struct.go#L101

net.IP comparable as String.

How do I force differ function so it would compare net.IP as a string and not as a slice?

https://go.dev/play/p/0VMlM5RQ9WO

package main

import (
	"log"
	"net"

	"github.com/davecgh/go-spew/spew"
	"github.com/r3labs/diff/v2"
)

type LoadBalancer struct {
	IP []net.IP
}

func main() {
	x := LoadBalancer{
		IP: []net.IP{
			net.ParseIP("192.0.2.1"),
			net.ParseIP("192.0.2.2"),
		},
	}

	y := LoadBalancer{
		IP: []net.IP{
			net.ParseIP("192.0.2.1"),
			net.ParseIP("192.0.2.3"),
		},
	}

	changelog, err := diff.Diff(x, y)
	if err != nil {
		log.Fatal(err)
	}

	spew.Dump(changelog)
}

(diff.Changelog) (len=1 cap=1) {
 (diff.Change) {
  Type: (string) (len=6) "update",
  Path: ([]string) (len=3 cap=3) {
   (string) (len=2) "IP",
   (string) (len=1) "1",
   (string) (len=2) "15"
  },
  From: (uint8) 2,
  To: (uint8) 3,
  parent: (interface {}) <nil>
 }
}

option for ignoring unexported fields

👋🏻 hey there
I'm using this library as it has been the most consistent way to compare structs that I've found so far, thanks!

one feature that I'm missing is the option to ignore unexported fields from being compared and marked as differences, I know I can strip them by marshal/unmarshal but if that functionality would be provided by the library itself that would be amazing 🙂

I'm ok with implementing it myself and submitting a pr!
I would like to know

• is this something you're ok with adding?
• is there some limitation that I could hit while adding this feature?
• any hints where I should start from?

I haven't checked the codebase yet, that's why I'm asking this 😅

Q: Adding a single struct entry to a map produces multiple changes

Not sure if this is expected, but if so, you could consider this more of a question.

As the title says, when a map has an additional entry (entries are custom struct types), then instead of getting 1 Change when comparing the maps, I get N changes (where N=number of fields in the structs).

I have the following types:

type Product struct {
	ID             int `diff:"id,identifier"`
	Title          string
	Description    string
	BaseMaterials  map[MaterialID]bool
	ExtraMaterials []MaterialID
	Sizes          []SizeID
	Weight         int16
	Prices         map[SizeID]ProductPrice // also a struct type
	Type           TypeID
}

type Catalog struct {
  Products map[int]Product `diff:"products"`
}

And I'm trying to compare two catalogs, of which the one has one key missing from its Catalog.Products map. The following example demonstrates:

// assume we have catalog1 & catalog2, which are identical

// remove a single entry (Product) from catalog2.Products
delete(catalog2.Products, 2)

// and compare
changelog, _ := diff.Diff(catalog1, catalog2)

At this point, the changelog contains N create diffs, where N is the number of fields of the Product. All diffs are of type "from nil to something", since in catalog2 the product is not present at all.

What I actually need, is to get a single, compound diff with the new entry. Is that possible given the current implementation, or do I have to iterate over the changelog and manually inspect the diffs to combine them to a single one?

Thanks in advance.

Error with update change

When I have two slices with two changes (update and create) the changelogs returned is not what it is expected. The example is:

I have two slices like these:
a := []map[string]interface{}{ { "name": "name1", "type": []string{"null", "string"}, }, }

b := []map[string]interface{}{ { "name": "name1", "type": []string{"null", "int"}, }, { "name": "name2", "type": []string{"null", "string"}, }, }

changelog, _ := diff.Diff(a, b)

The changelog is:

[{create [0 type 1] <nil> int} {create [1 name] <nil> name2} {create [1 type] <nil> [null string]}]

But the expected result is:

[{update [0 type 1] string int} {create [1 name] <nil> name2} {create [1 type] <nil> [null string]}]

I saw that in the diff_map.go file there is a function called swapChange that does the change from "update" to "create". Maybe here is the problem.

Thank you very much.

Question on performance

Hello!

I'm diffing some large nested structures and running into really long run times with v3.

Interestingly v1 of this package returns in mere seconds with the correct changelog result.

However, I'm really looking for patching support that comes with v3. I've tried using options like 'FlattenEmbeddedStructs', 'DiscardComplexOrigin', and 'DisableStructValues' individually but nothing gets me to the performance that comes with v1

Is there an option or set of options in v3 to align performance with v1? What could be taking so much more time in v3?

Thanks for the help

Append/Update only

I have a use case where the diff to object is sent in many smaller parts, therefore i need to be able to not remove objects, but update/add - can this be accomplished?

Show entire struct data in From/To even though it's only partially updated

package main

import (
	"fmt"

	"github.com/r3labs/diff/v2"
)

type Data struct {
	ID    int32  `json:"id" diff:"ID"`
	Value string `json:"value" diff:"Value"`
}

type Order struct {
	Items []Data `diff:"Items,ID"`
}

func main() {
	a := Order{
		Items: []Data{Data{ID: 1, Value: "foo"}},
	}

	b := Order{
		Items: []Data{Data{ID: 1, Value: "bar"}, Data{ID: 2, Value: "paper"}},
	}

	changelog, err := diff.Diff(a, b, diff.DisableStructValues())
	if err != nil {
		fmt.Println("ERROR", err)
		return
	}
	fmt.Printf("%+v\n", changelog)

}

Output

[{Type:update Path:[Items 0 Value] From:foo To:bar parent:{ID:1 Value:foo}} {Type:create Path:[Items 1] From:<nil> To:{ID:2 Value:paper} parent:<nil>}]

What I would like is a way so that instead of

{Type:update Path:[Items 0 Value] From:foo To:bar parent:{ID:1 Value:foo}}

I get

{Type:update Path:[Items 0 Value] From:{ID: 1 Value: foo} To:{ID: 1 Value: bar} parent:{ID:1 Value:foo}}

EOL pkg found in go mod

github.com/vmihailenco/msgpack v4.0.4+incompatible is an EOL .

Do you know if we can use the the latest version of this package?

Here is the list of policies vmihailenco/msgpack v4.0.4 violates.
Version/Branch EOL
Severity:Major

Category:Security

Scan Modes:Full

Description
Version or branch requested is EOL.   Please update to a supported version.

Conditions
Component Version Approval StatusEQUALS Deprecated
Component UsageNOT EQUAL TO Dev. Tool / Excluded

Patching slice back to empty slice returns patchLog.Applied() to be false

patchLog.Applied() is false when we run the following code snippet regardless the patch operation works as expected.

package main

import (
	"fmt"

	"github.com/r3labs/diff/v3"
)

type Order struct {
	ID         string     `json:"id"`
	OrderItems OrderItems `json:"orderItems"`
}

type OrderItems struct {
	Items []string `json:"items"`
}

func main() {
	a := Order{
		ID: "1234",
	}

	b := Order{
		ID:         "1234",
		OrderItems: OrderItems{[]string{"1", "2", "4"}},
	}

	d, _ := diff.NewDiffer(diff.TagName("json"))

	changelog, _ := d.Diff(b, a)

	patchLog := d.Patch(changelog, &b)
	fmt.Printf("patchlog applied : %t \nhas errors %t \nerror count %d \n\n", patchLog.Applied(), patchLog.HasErrors(), patchLog.ErrorCount())

	fmt.Printf("values \n a: %#v \n b: %#v", a, b)
}

filtering which fields are inspected

I'm trying to diff two structs returned from a particular API and I want to ignore certain struct fields when computing a diff. I do not have control over the struct declarations, so I cannot add struct tags for this purpose. Instead, I would like to supply a filter function that returns true for fields that the diff should descend into, and false for fields that the diff should ignore.

ApplyDiff / RevertDiff

Hi there,
awesome lib, thank you, but sadly it only fill half our need.
We are currently implementing patch and reverse patch method for our internal need on known structures, but it seems more logical to be handle by the diff lib.
Do you have support, plan to support or the willingness to accept a PR about apply/patch and revert diff on arbitrary structures ?

The go get module is different from the one on github

When I go get the module, lets say options.go is missing a few functions.

unsupported type: MongoDB primitive.ObjectID

Will there be a support for the type primitive.ObjectID for MongoDB? Or is there a way to add types ourselves?
https://pkg.go.dev/go.mongodb.org/mongo-driver

Right now I have to deactivate all of them with diff:"-", that wouldn't cause an issue if I was only using primitive.ObjectID as main ID, but I'm also using that to link collections, so I need it in other area on the struct.

type ObjectID [12]byte

fmt.Print(reflect.ValueOf(obj.ID).Kind()) // array
unsupported type: array

It seems that you support Slice only and not Array

Besides that, it works well. Thanks for this library!

Patch delete for nested map deletes parent

When I try to Patch map using code below, it removes details as well as attributes:

testMap := map[string]interface{}{}
testMap["firstName"] = "John"
testMap["lastName"] = "Michael"
testMap["createdBy"] = "TS"
testMap["details"] = map[string]interface{}{
  "status": "active",
  "attributes": map[string]interface{}{
	  "attrA": "A",
	  "attrB": "B",
  },
}

diff.Patch(diff.Changelog{{
  Type: "delete",
  Path: []string{"details", "attributes"},
  From: []interface{}{
    map[string]interface{}{
      "Key":   "attrA",
      "Value": "A",
    },
    map[string]interface{}{
      "Key":   "attrB",
      "Value": "B",
    },
  },
  To: nil,
  }
}, &testMap)
Ω(testMap["details"]).NotTo(BeNil())

Result:

Expected
    <nil>: nil
not to be nil

Expected to have at the end:

{
  "firstName": "John",
  "lastName": "Michael",
  "createdBy": "TS",
  "details": {
    "status": "active"
  }
}

Diffing struct with private field fails

the example code

package main

import (
	"github.com/r3labs/diff"
	"log"
)

type int64Amount struct {
	A int64
	b int64
}

func main() {
	changelog, err := diff.Diff(int64Amount{A: 1}, int64Amount{A: 2})
	log.Println(changelog, err)

	// will get panic
	changelog, err = diff.Diff(int64Amount{b: 2}, int64Amount{b: 3})
	log.Println(changelog, err)
}

we will get a panic

panic: reflect.Value.Interface: cannot return value obtained from unexported field or method

because the field b is a private field. can not execute 'Interface()' methods.

for example on diffInt

func (d *Differ) diffInt(path []string, a, b reflect.Value) error {
	if a.Kind() == reflect.Invalid {
		d.cl.add(CREATE, path, nil, b.Interface())
		return nil
	}

	if b.Kind() == reflect.Invalid {
		d.cl.add(DELETE, path, a.Interface(), nil)
		return nil
	}

	if a.Kind() != b.Kind() {
		return ErrTypeMismatch
	}

	if a.Int() != b.Int() {
		d.cl.add(UPDATE, path, a.Interface(), b.Interface())
	}

	return nil
}

If we sure know the kind is Int, we can use Int() to get the value
I think do some change is a good idea

         if a.Int() != b.Int() {
		d.cl.add(UPDATE, path, a.Int(), b.Int())
	}

nd in structValues function should use the same filters as d

diff/diff_struct.go

Line 66 in 9e5baa4

func (d *Differ) structValues(t string, path []string, a reflect.Value) error {

In this^ fuction, a new Differ nd is used to deal with each of the fields. This works okay, except when the user of the package defines a filter function.

One workaround is to do this:

func (d *Differ) structValues(t string, path []string, a reflect.Value) error {
	var nd Differ
	nd.Filter = d.Filter

	// ...

	for i := 0; i < a.NumField(); i++ {

		// ...

		if nd.Filter != nil && !nd.Filter(fpath, a.Type(), field) {
			continue
		}

		err := nd.diff(fpath, xf, af, a.Interface())
		if err != nil {
			return err
		}
	}

	// ...
}

Add reflect.Invalid check to diffSlice

Currently, when diffing a slice type and a nil value, the diff() method returns an error "types do not match".
Inspecting your code i found that the diffSlice() method is missing the part at the beginning where you check if a or b are of type reflect.Invalid and thus there is a CREATE/DELETE change happening here.
This should be fairly easy to fix though, as you can just copy that part of logic from any other diff-method.
Thanks in advance! :)

Obscure Tag

Sometimes we want to record that a field has been changed, but not actually show the changes performed. A good example is a password hash, or a network passphrase.

We would be interested in a "obscure" field tag, that would record the change but maybe star out, or null the actual field values. Just returning null for from and to would be easiest I guess?

Obscure wouldn't work with patching, so not sure if it's too niche for you?

I'm happy to make a pull request if you think it would fit in with the library, otherwise we can form and just keep the functionality in our own version.

Happy for feedback!

When Bool type = false, is missing from Type:create

Hi

Test code using go 1.20:

type Test struct {
	ID   int64  `diff:"id,identifier"`
	Bool bool   `diff:"bool"`
	Test string `diff:"test"`
}

a := []Test{{ID: 1, Bool: true, Test: "Bool set to true"}}

b := []Test{{ID: 2, Bool: false, Test: "Bool set to false"}}

changelog, err := diff.Diff(a, b)

if err != nil {
	fmt.Println(err)
	return
}

fmt.Printf("%+v\n", changelog)

Gives:

  {Type:create Path:[2 id] From:<nil> To:2 parent:<nil>} {Type:create Path:[2 test] From:<nil> To:Bool set to false parent:<nil>}]

Field Bool is missing from create. Should also have:

  {Type:create Path:[2 bool] From:<nil> To:false parent:<nil>}

Any help is appreciated!

Thanks.

run this test ,find panic

func TestDiff(t *testing.T) {
	type args struct {
		from interface{}
		to   interface{}
	}
	type Info struct {
		Qq     string
		Wechat string
	}
	type data struct {
		Name   string
		Age    int
		Height int
		Info
		listInt []int
	}

	tests := []struct {
		name    string
		args    args
		wantErr bool
	}{
		{
			name: "lch",
			args: args{
				from: data{
					Name:   "lch",
					Age:    1,
					Height: 1,
					Info: Info{
						Qq:     "qq",
						Wechat: "wechat",
					},
					listInt: []int{1, 2, 3, 4},
				},
				to: data{
					Name:   "lchjczw",
					Age:    1,
					Height: 2,
					Info: Info{
						Qq:     "myqq",
						Wechat: "wechat",
					},
					listInt: []int{1, 2, 3},
				},
			},
		},
	}
	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {

			d, err := diff.NewDiffer(diff.AllowTypeMismatch(true),
				diff.FlattenEmbeddedStructs())
			if err != nil {
				panic(err)
			}

			_, err = d.Diff(tt.args.from, tt.args.to)
			if (err != nil) != tt.wantErr {
				t.Errorf("Diff error = %v, wantErr %v", err, tt.wantErr)
				return
			}

		})
	}
}

TestDiff/mixed-slice-map fails randomly

=== RUN   TestDiff/mixed-slice-map
    diff_test.go:645: 
                Error Trace:    diff_test.go:645
                Error:          Not equal: 
                                expected: []string{"1", "\xa4name"}
                                actual  : []string{"1", "\xa4type"}
                            
                                Diff:
                                --- Expected
                                +++ Actual
                                @@ -2,3 +2,3 @@
                                  (string) (len=1) "1",
                                - (string) (len=5) "\xa4name"
                                + (string) (len=5) "\xa4type"
                                 }
                Test:           TestDiff/mixed-slice-map
    diff_test.go:647: 
                Error Trace:    diff_test.go:647
                Error:          Not equal: 
                                expected: string("name2")
                                actual  : []string([]string{"null", "string"})
                Test:           TestDiff/mixed-slice-map
    diff_test.go:645: 
                Error Trace:    diff_test.go:645
                Error:          Not equal: 
                                expected: []string{"1", "\xa4type"}
                                actual  : []string{"1", "\xa4name"}
                            
                                Diff:
                                --- Expected
                                +++ Actual
                                @@ -2,3 +2,3 @@
                                  (string) (len=1) "1",
                                - (string) (len=5) "\xa4type"
                                + (string) (len=5) "\xa4name"
                                 }
                Test:           TestDiff/mixed-slice-map
    diff_test.go:647: 
                Error Trace:    diff_test.go:647
                Error:          Not equal: 
                                expected: []string([]string{"null", "string"})
                                actual  : string("name2")
                Test:           TestDiff/mixed-slice-map

It can be reproduced by run go test -count=10

`diff:"-"` does not work

Panic

panic: reflect.Value.Interface: cannot return value obtained from unexported field or method [recovered]
	panic: reflect.Value.Interface: cannot return value obtained from unexported field or method

package transformer

import (
	"github.com/davecgh/go-spew/spew"
	"github.com/finsmart-gmbh/api/model"
	"github.com/finsmart-gmbh/pointer"
	"github.com/makeless/makeless-go/model"
	"github.com/r3labs/diff/v2"
	"github.com/stretchr/testify/assert"
	"sync"
	"testing"
)

// dst: database company lead
// src: company lead from company lead transformer
func TestCompanyLeadMergeTransformerTransform(t *testing.T) {
	tests := []struct {
		dst *model.CompanyLead
		src *model.CompanyLead
	}{
		{
			dst: &model.CompanyLead{
				Model: makeless_go_model.Model{Id: 1},
				CompanyLeadHouseholds: []*model.CompanyLeadHousehold{
					{
						Model: makeless_go_model.Model{Id: 2},
						CompanyLeadPersons: []*model.CompanyLeadPerson{
							{
								Model:     makeless_go_model.Model{Id: 3},
								FirstName: pointer.StringPtr("Max"),
								RWMutex:   new(sync.RWMutex),
							},
						},
						RWMutex: new(sync.RWMutex),
					},
				},
				RWMutex: new(sync.RWMutex),
			},
			src: &model.CompanyLead{
				Model: makeless_go_model.Model{Id: 1},
				CompanyLeadHouseholds: []*model.CompanyLeadHousehold{
					{
						Model: makeless_go_model.Model{Id: 2},
						CompanyLeadPersons: []*model.CompanyLeadPerson{
							{
								Model:     makeless_go_model.Model{Id: 3},
								FirstName: pointer.StringPtr("Max"),
								Address: &model.Address{
									Street:  pointer.StringPtr("maxstraße"),
									RWMutex: new(sync.RWMutex),
								},
								RWMutex: new(sync.RWMutex),
							},
							{
								Model:     makeless_go_model.Model{Id: 4},
								FirstName: pointer.StringPtr("Fisch"),
								Address: &model.Address{
									Street:  pointer.StringPtr("maxstraße"),
									RWMutex: new(sync.RWMutex),
								},
								RWMutex: new(sync.RWMutex),
							},
						},
						RWMutex: new(sync.RWMutex),
					},
				},
				RWMutex: new(sync.RWMutex),
			},
		},
	}

	for i, test := range tests {
		changelog, err := diff.Diff(test.dst, test.src)
	}
}

Changelog Serialization - Interface issue

Hi diff team,

I am working on a project that is leveraging the library and there appears to be a problem with interfaces when serializing the change logs. The types get lost in translation.

I have created a simplified reproduction of the issue at hand that exemplifies the issue. In the example there is a type that has a field which is an interface where we are adding different types that qualify for the interface. When we serialize the changelog, then de-serialize the changelog, then apply it as a patch, you will see the error. The important piece is that it does not recognize what type it was from before the change log was serialized.

type Something interface {
	Name() string
}
type A struct {
	First  string
	Second string
}

func (a A) Name() string {
	return a.First + a.Second
}

type B struct {
	First  string
	Second string
}

func (b B) Name() string {
	return b.First + b.Second
}

type Example struct {
	This []Something
}

func TestChangeExample(t *testing.T) {
	before := Example{This: []Something{A{First: "Joe", Second: "Shmo"}}}
	after := Example{This: []Something{A{First: "Joe", Second: "Shmo"}, B{First: "Jane", Second: "Doe"}}}
	differ, err := diff.NewDiffer(diff.ConvertCompatibleTypes())
	if err != nil {
		t.Fatal(err)
	}
	cl, err := differ.Diff(&before, &after)
	if err != nil {
		t.Fatal(err)
	}
	b, err := json.Marshal(&cl)
	if err != nil {
		t.Fatal(err)
	}
	var newCL diff.Changelog
	err = json.Unmarshal(b, &newCL)
	if err != nil {
		t.Fatal(err)
	}

	pl := diff.Patch(newCL, &before)
	for _, p := range pl {
		if p.Errors != nil {
			t.Fatal(p.Errors)
		}
	}
	println("success?")
}

In the Goland IDE inspection and comparison to the before after shows that it does not recognize the same type:

Type lost in serialization

Hi, this might not be a bug, but an issue I´ve ran into and need to work around. Consider the following

type Test struct {
	S *int `json:"s,omitempty"`
}

func TestPointerDiff(t *testing.T) {
	val1 := 1
	val2 := 2

	t1 := Test{S: &val1}
	t2 := Test{S: &val2}

	changelog, err := diff.Diff(t1, t2)
	assert.NoError(t, err)

	js, err := json.Marshal(changelog)
	assert.NoError(t, err)

	assert.NoError(t, json.Unmarshal(js, &changelog))

	patchLog := diff.Patch(changelog, &t1)
	assert.False(t, patchLog.HasErrors())
}

This will fail, since the type of the int is lost when marshaling to JSON.

Any ideas how I could work around this?

no differences reported for maps where values are empty structs

in a case where there is a type defined such as

type Set map[string]struct{}

the result of diffing these types always ends up with an empty changelog, even if there are differences on the keys

after digging a bit through the codebase I've realized that the structValues method won't add any entry to the changelog if the struct has no fields; when diffing maps, for a specific key, if any of the maps doesn't have a value for that key, there is a shortcut that simply returns the output of structValues as either create or delete depending which map is missing the data

this, effectively makes the changelog to be empty for the following case

a := Set{"existing": struct{}}
b := Set{}

Recommended compatibility with json tag

It is recommended to be compatible with json tag or support custom tag

Bug: Nil slice == empty slices when diffing

At the moment, it seems nil/uninitialized slices are == initialized empty list slices.

type testStruct struct {
	A []string
}

func Test_DiffNilList(t *testing.T) {
	p := testStruct{}
	assert.Nil(t, p.A)
	n := testStruct{
		A: []string{},
	}
	assert.NotNil(t, n.A)
	res, err := diff.Diff(p, n)
	if !assert.NoError(t, err) {
		return
	}
	assert.Equal(t, 1, len(res))
}

Patch failing for byte slices when patching onto struct with nil destination

I'm getting some unexpected behaviour when diffing and applying the changelog for byte slices. In this example I am comparing 2 structs that have a byte slice and patching the result onto an empty struct that has a nil byte slice. Is this supported behaviour?

Go Playground

package main

import (
	"fmt"

	"github.com/r3labs/diff/v3"
)

type MyType struct {
	MyField []byte
}

func main() {

	left := &MyType{[]byte{208, 72, 51, 52, 175, 134, 76, 84, 143, 38, 99, 184, 128, 24, 107, 163}}
	right := &MyType{[]byte{91, 102, 170, 173, 254, 105, 66, 81, 177, 175, 32, 173, 173, 165, 129, 192}}

	changelog, err := diff.Diff(left, right)
	if err != nil {
		fmt.Println(err)
	}

	dest := &MyType{}
	_ = diff.Patch(changelog, dest)

	fmt.Println(left.MyField)  // [208 72 51 52 175 134 76 84 143 38 99 184 128 24 107 163]
	fmt.Println(right.MyField) // [91 102 170 173 254 105 66 81 177 175 32 173 173 165 129 192]
	fmt.Println(dest.MyField)  // [32 173 173 165 254 192] ?

	// Patchlog errors:

	// [MyField 0] 208 91  Value index 0 is invalid (cause count 1)
	//  scanning for Value index (cause count 0)

	// [MyField 1] 72 102  Value index 1 is invalid (cause count 1)
	//  scanning for Value index (cause count 0)

	// [MyField 2] 51 170  Value index 2 is invalid (cause count 1)
	//  scanning for Value index (cause count 0)

	// [MyField 3] 52 173  Value index 3 is invalid (cause count 1)
	//  scanning for Value index (cause count 0)

	// [MyField 5] 134 105  Value index 5 is invalid (cause count 1)
	//  scanning for Value index (cause count 0)

	// [MyField 6] 76 66  Value index 6 is invalid (cause count 1)
	//  scanning for Value index (cause count 0)

	// [MyField 7] 84 81  Value index 7 is invalid (cause count 1)
	//  scanning for Value index (cause count 0)

	// [MyField 8] 143 177  Value index 8 is invalid (cause count 1)
	//  scanning for Value index (cause count 0)

	// [MyField 9] 38 <nil>  Value index 9 is invalid (cause count 1)
	//  scanning for Value index (cause count 0)

	// [MyField 10] 99 32  Value index 10 is invalid (cause count 1)
	//  scanning for Value index (cause count 0)

	// [MyField 11] 184 173  Value index 11 is invalid (cause count 1)
	//  scanning for Value index (cause count 0)

	// [MyField 12] 128 173  Value index 12 is invalid (cause count 1)
	//  scanning for Value index (cause count 0)

	// [MyField 13] 24 165  Value index 13 is invalid (cause count 1)
	//  scanning for Value index (cause count 0)

	// [MyField 14] 107 129  Value index 14 is invalid (cause count 1)
	//  scanning for Value index (cause count 0)

	// [MyField 15] 163 192  Value index 15 is invalid (cause count 1)
	//  scanning for Value index (cause count 0)

}

panic when trying to patch struct containing pointer to array of strings with nil

I'm having issues when I have a struct that contains a pointer to an array of string. An example of this problem

package main

import (
    "github.com/r3labs/diff/v2"
)

type MyStruct struct{
    MyArr  *[]string  `diff:"myarr"`
}

func saptr(s []string) *[]string {
	return &s
}

func main() {
    orig := MyStruct{
        MyArr: saptr([]string{"foobar"}),
    } 
    new := MyStruct{
        MyArr: nil,
    }
    
    changelog, err := diff.Diff(orig, new)
    if err != nil {
        panic(err)
    }

    _ = diff.Patch(changelog, &orig)
}

produces the panic error

panic: reflect: call of reflect.Value.Set on zero Value [recovered]
        panic: interface conversion: interface {} is *reflect.ValueError, not string

Please let know know if I'm misusing the library or I've implemented this incorrectly.

Panic when struct has a time.Time field

reproduction:

package main

import (
	"log"
	"time"

	"github.com/r3labs/diff"
)

type Order struct {
	ID   string    `diff:"id"`
	Time time.Time `diff:"time"`
}

func main() {
	a := Order{
		ID:   "1234",
		Time: time.Now(),
	}

	b := Order{
		ID:   "1234",
		Time: time.Now(),
	}

	changelog, err := diff.Diff(a, b)
	log.Println(changelog, err)
}

Expected result: something returned
Actual result: panic: reflect.Value.Interface: cannot return value obtained from unexported field or method

go version go1.11.4 linux/amd64

Diff of slices with the same items

When I am trying to diff two slices with different number of items, but repetitive item values, I've got nil as a Diff() result, while those slices are not the same - they have different number of items.
Min example:

	a := []int{1, 1}
	b := []int{1}
	changelog, err := diff.Diff(a, b)

will produce nil,nil as a result - no changes, no errors, while a has different number of items than b and is different
It looks reasonable to have one change log entry of Type: "delete" in this case. Yes, it is impossible to say what is the index of deleted item - 0 or 1, but any of them (0 or 1) will perfectly do and indicate the difference.
Can you, please, advice on how to handle this correctly?

Patch against map doesn't work

When I try to Patch map using code below

  testMap := map[string]interface{}{}
  testMap["firstName"] = "John"
  testMap["lastName"] = "Michael"
  testMap["createdBy"] = "TS"
  
  patchLog := diff.Patch(diff.Changelog{{
      Type: "update",
      Path: []string{"createdBy"},
      From: "TS",
      To:   "KS",
    }
  }, &testMap)
  Ω(patchLog[0].Errors).To(BeNil())

I get the following error

reflect.Set: value of type string is not assignable to type map[string]interface {}

Bug: embeded structs diffs are not detected

When embeding struct within struct, there are no diffs reported

Example reporduction: master...geritol:diff:bug-reproduction

call of reflect.Value.Type on zero Value

Hi, the following crash

func TestZeroPointerDiff(t *testing.T) {
	type T struct {
		Z *int
	}

	val := 1

	t1 := T{Z: &val}
	t2 := T{Z: nil}

	changelog, err := diff.Diff(t1, t2)
	assert.NoError(t, err)

	d, err := diff.NewDiffer(diff.ConvertCompatibleTypes())
	assert.NoError(t, err)
	patchLog := d.Patch(changelog, &t1)
	assert.False(t, patchLog.HasErrors())
}

Diffing struct with private field *big.Int fails

I have problem when trying to diffing struct decimal from https://github.com/shopspring/decimal . looks like it fails because of private field *big.Int

Example code:

package main

import (
	"fmt"
	"math/big"

	"github.com/r3labs/diff"
)

type number struct {
	value *big.Int
	exp   int32
}

func (n *number) SetValue(x int64) {
	n.value = big.NewInt(x)
}

func main() {
	a := number{}
	b := number{}
	b.SetValue(111)

	diffSliceOrder, _ := diff.NewDiffer(diff.SliceOrdering(true))
	changelog, err := diffSliceOrder.Diff(a, b)

	fmt.Println(err)
	fmt.Println(changelog)
}

we will get a panic

panic: reflect.Value.Interface: cannot return value obtained from unexported field or method

goroutine 1 [running]:
reflect.valueInterface(0x525fc0, 0xc000046280, 0xb6, 0x1, 0x0, 0x0)
        c:/go/src/reflect/value.go:1014 +0x1c3
reflect.Value.Interface(...)
        c:/go/src/reflect/value.go:1003
github.com/r3labs/diff.(*Differ).diffPtr(0xc0000044e0, 0xc0000462b0, 0x1, 0x1, 0x525fc0, 0xc000046270, 0xb6, 0x525fc0, 0xc000046280, 0xb6, ...)
        E:/Go Project/pkg/mod/github.com/r3labs/[email protected]/diff_pointer.go:33 +0x8b9
github.com/r3labs/diff.(*Differ).diff(0xc0000044e0, 0xc0000462b0, 0x1, 0x1, 0x525fc0, 0xc000046270, 0xb6, 0x525fc0, 0xc000046280, 0xb6, ...)
        E:/Go Project/pkg/mod/github.com/r3labs/[email protected]/diff.go:128 +0x875
github.com/r3labs/diff.(*Differ).diffStruct(0xc0000044e0, 0x620358, 0x0, 0x0, 0x50bce0, 0xc000046270, 0x99, 0x50bce0, 0xc000046280, 0x99, ...)
        E:/Go Project/pkg/mod/github.com/r3labs/[email protected]/diff_struct.go:50 +0x460
github.com/r3labs/diff.(*Differ).diff(0xc0000044e0, 0x620358, 0x0, 0x0, 0x50bce0, 0xc000046270, 0x99, 0x50bce0, 0xc000046280, 0x99, ...)
        E:/Go Project/pkg/mod/github.com/r3labs/[email protected]/diff.go:112 +0xdf5
github.com/r3labs/diff.(*Differ).Diff(0xc0000044e0, 0x50bce0, 0xc000046270, 0x50bce0, 0xc000046280, 0x0, 0x461b54, 0x508c80, 0xc000046260, 0xc00008def0)
        E:/Go Project/pkg/mod/github.com/r3labs/[email protected]/diff.go:101 +0x163
main.main()
        E:/Go Apps/raditzlawliet/xxx/xxx/experiment/differ.big.Int/main.go:25 +0x18a
exit status 2

Patching pointer values

Hi, In the following I am unable to patch the struct. To me it seems that the diff holds a string and that the reflection machinery does not consider the target pointer. I am missing something here? Thanks:)

type Test struct {
	S *string `json:"s,omitempty"`
}

func TestPointerDiff(t *testing.T) {
	str1 := "before"
	t1 := Test{S: &str1}
	str2 := "after"
	t2 := Test{S: &str2}

	changelog, err := diff.Diff(t1, t2)
	assert.NoError(t, err)

	patchLog := diff.Patch(changelog, t1)
	assert.False(t, patchLog.HasErrors())
}

Support for adding equal values for a changelog

Hi 👋 ! I'd like to know if it would make sense to add a new type of changelog entry for equal values, so that it's possible to detect that two structs are almost equals and in which fields, simply getting the changelog entries for that new EQUALS type.

My use case is comparing two structs and detecting how much similar they are.

Go language version is very old.

go 1.13

Go language version 1.13 is very old.
It is almost four years old.
Please, update it.

[v2.14.0] panic: reflect.Value.Interface: cannot return value obtained from unexported field or method

When trying to use the github.com/deckarep/golang-set library, running a diff on the sets causes a panic from the reflection package.

Here is a minimum reproducible example:

package main

import (
	"github.com/deckarep/golang-set"
	"github.com/r3labs/diff/v2"
)

func main() {
	a := mapset.NewSet("a", "b", "c")
	b := mapset.NewSet("a", "c")
	_, _ = diff.Diff(a, b)
}

And an example error:

panic: reflect.Value.Interface: cannot return value obtained from unexported field or method

goroutine 1 [running]:
reflect.valueInterface({0x8517c0, 0xc000088350, 0x0}, 0xc0)
	C:/Program Files/Go/src/reflect/value.go:1362 +0xd9
reflect.Value.Interface(...)
	C:/Program Files/Go/src/reflect/value.go:1351
github.com/r3labs/diff/v2.(*Differ).diffMap(0x854a00, {0xc000088340, 0x1, 0x1}, {0x854a00, 0xc0000a63a0, 0x854a00}, {0x854a00, 0xc0000a63c0, 0x1b5})
	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff_map.go:27 +0x59f
github.com/r3labs/diff/v2.(*Differ).diff(0xc0000e0000, {0xc000088340, 0x1, 0x1}, {0x854a00, 0xc0000a63a0, 0xc0000ac058}, {0x854a00, 0xc0000a63c0, 0x1b5}, ...)
	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff.go:182 +0xb30
github.com/r3labs/diff/v2.(*Differ).diffStruct(0xc0000e0000, {0x99a288, 0x0, 0x0}, {0x857020, 0xc0000a63a0, 0x54}, {0x857020, 0xc0000a63c0, 0x199})
	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff_struct.go:62 +0xad3
github.com/r3labs/diff/v2.(*Differ).diff(0xc0000e0000, {0x99a288, 0x0, 0x0}, {0x857020, 0xc0000a63a0, 0x92b2d0}, {0x857020, 0xc0000a63c0, 0x199}, ...)
	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff.go:166 +0xf1e
github.com/r3labs/diff/v2.(*Differ).diffPtr(0xc0000e0000, {0x99a288, 0x0, 0x0}, {0x863960, 0xc0000a63a0, 0x2e1cf6a7000}, {0x863960, 0xc0000a63c0, 0x16}, ...)
	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff_pointer.go:45 +0x5bc
github.com/r3labs/diff/v2.(*Differ).diff(0xc0000e0000, {0x99a288, 0x0, 0x0}, {0x863960, 0xc0000a63a0, 0xc0000cfe01}, {0x863960, 0xc0000a63c0, 0x16}, ...)
	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff.go:184 +0xabd
github.com/r3labs/diff/v2.(*Differ).Diff(0xc0000e0000, {0x863960, 0xc0000a63a0}, {0x863960, 0xc0000a63c0})
	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff.go:129 +0x1b3
github.com/r3labs/diff/v2.Diff({0x863960, 0xc0000a63a0}, {0x863960, 0xc0000a63c0}, {0x0, 0x2e1cf630598, 0x60})
	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff.go:72 +0x75
main.main()
	C:/Users/Vilsol/go/src/awesomeProject/difftest/main.go:11 +0x11b

I am using:

• Go -> 1.17
• github.com/r3labs/diff/v2 -> v2.14.0
• github.com/deckarep/golang-set -> v1.7.1

handling type change

Hi, i ran into an issue where i provide differ with maps where type of one of the values within map has changed - currently this result in an error since isvalid() method doesn't allow this. Would you accept a change which would optionally (through opts) change behaviour for this kind of situation and would just report a change for that key ?

Patching a nil *int?

Howdy!

What am I missing here: I am unable to patch my "blank" struct property.
The last assertion fails with reflect: call of reflect.Value.Type on zero Value.

type PatchStruct struct {
	Variable *int `json:"Variable"`
}

func TestPatch(t *testing.T) {
	changes := []diff.Change{
		{
			Type: "update",
			Path: []string{"Variable"},
			From: nil,
			To:   4,
		},
	}

	var patchMe PatchStruct
	d, err := diff.NewDiffer(diff.ConvertCompatibleTypes())
	assert.NoError(t, err)

	result := d.Patch(changes, &patchMe)
	assert.False(t, result.HasErrors())
}

Slice ordering can't be disabled for []map[string]interface{}

I am trying to compare slice of maps but the results are inconsistent.
As I understand correctly the code below should show the same result for a == b and a == c

package main

import (
	"fmt"
	diff "github.com/r3labs/diff/v2"
)

func main() {
	a := []map[string]interface{}{
		{
			"test": 2,
			"a":    "test",
		},
		{
			"test": 12,
		},
	}
	b := []map[string]interface{}{
		{
			"test": 2,
			"a":    "test1",
		},
		{
			"test": 12,
		},
	}
	c := []map[string]interface{}{
		{
			"test": 12,
		},
		{
			"test": 2,
			"a":    "test1",
		},
	}
	changelog1, _ := diff.Diff(a, b, diff.StructMapKeySupport(), diff.SliceOrdering(false))
	changelog2, _ := diff.Diff(a, c, diff.StructMapKeySupport(), diff.SliceOrdering(false))
	fmt.Println(changelog1)
	fmt.Println(changelog2)
}

That would be awesome to ad an option that allows to have consistent results in this case

Diffing maps with pointer values fails

When diffing a map with pointer values, e.g.:

struct1 := tmstruct{Bar: 1, Foo: "one"}
struct2 := tmstruct{Bar: 2, Foo: "two"}

map1 := map[string]*tmstruct{
  "one": &struct1,
}
map2 := map[string]*tmstruct{
"one": &struct1,
"two": &struct2,
}

...this comes back as ErrTypeMismatch due to comparing nil to a pointer.

I was able to fix this for my use case by modifying diff_pointer.go thusly:

func (cl *Changelog) diffPtr(path []string, a, b reflect.Value) error {
	if a.Kind() == reflect.Invalid {
		cl.add(CREATE, path, nil, b.Interface())
		return nil
	}

	if b.Kind() == reflect.Invalid {
		cl.add(DELETE, path, a.Interface(), nil)
		return nil
	}
	[...]

However, this change breaks the tests mismatched-values-struct-nil and mismatched-values-nil-struct.

Do you have thoughts on how this conflict could be resolved?

For testing my change, I added these cases to the case list in TestDiff:

		{
			"map-string-pointer-create-test",
			map[string]*tmstruct{"one": &struct1},
			map[string]*tmstruct{"one": &struct1, "two": &struct2},
			Changelog{
				Change{Type: CREATE, Path: []string{"two"}, From: nil, To: &struct2},
			},
			nil,
		},
		{
			"map-string-pointer-delete-test",
			map[string]*tmstruct{"one": &struct1, "two": &struct2},
			map[string]*tmstruct{"one": &struct1},
			Changelog{
				Change{Type: DELETE, Path: []string{"two"}, From: &struct2, To: nil},
			},
			nil,
		},

Changelog for slices inconsistent when using pointer values

We want to compare slices of strings.

Following example code:

type TestStruct struct {
	Slice  []string `json:"slice"`
}

type TestData struct {
	TS *TestStruct
}

func main() {
	s1 := TestData{&TestStruct{Slice: []string{"1", "2", "3"}}}
	s2 := TestData{&TestStruct{Slice: []string{"1", "4", "3"}}}

	d, err := diff.NewDiffer(diff.SliceOrdering(false))
	if err != nil {
		panic(err)
	}
	changes, _ := d.Diff(nil, &s2)
	if changes != nil { // for usage and debugger
		fmt.Sprintf("")
	}
	changes2, _ := d.Diff(&s1, &s2)
	if changes2 != nil { // for usage and debugger
		fmt.Sprintf("")
	}
	changes3, _ := d.Diff(&s1, nil)
	if changes3 != nil { // for usage and debugger
		fmt.Sprintf("")
	}
}

With this example there are three different Types of changes visible:
changes contains a create
changes2 contains a update
changes3 contains a delete

The create and delete take the whole interface as the From/To values. So the path is TS.

The update on the other side lists all individual changes by index and the path is TS, Slice, 1.

My expectation would be, that the lists of changes are consistent, what means

• either the create and delete also go down to the index level
• or the update also shows the whole slice diff

maybe an option to make on or the other behaviour optional, would be the best case.

Do I miss something here or is this intended behaviour?

My previous concern is still value:
with the above example, the update (example changes2) has one record in the changes, that says

Type: update
path: TS, Slice, 1
From: 2
To: 4

what actually (from my perspective) is wrong, as here we have an create and delete of values, and not an update.

Note: when changing the type of TestData.TS to a value and remove the pointer, the diff is consistent and all items are listed with changes based on indices.
I personally prefer the diff as a whole, like given in create and delete.

EDIT: after some investigation, more info added

Uint key doesn't add to path

Using diff between maps with uint as key type, returns a path without the key value.

Following code prints:
"[{delete [items ] 2 } {delete [items ] 3 }]"

type Order struct {
	ID    string       `diff:"id"`
	Items map[uint]uint `diff:"items"`
}

func main() {
	a := Order{
		ID: "1234",
		Items: map[uint]uint{10: 1,
			20: 2,
			30: 3,
			40: 4},
	}

	b := Order{
		ID: "1234",
		Items: map[uint]uint{10: 1,
			40: 4},
	}

	changelog, err := diff.Diff(a, b)
	if err != nil {
		fmt.Println("error")
	} else {
		fmt.Println(changelog)
	}

Exclude a field or path from diff

I couldn't find anything in the documentation but it would be really useful to exclude specific fields or paths in the struct from output change log.

unsupported type: array when ID withh uuid.UUID

unsupported type: array when ID withh uuid.UUID

Performance guidance

Hi there,

Lovely library, thanks for making it! We have started using it in Aether (https://aether.app - github.com/nehbit/aether) for the past week, and it's been working splendidly so far.

While I realise the best way to figure out suitability is to test, I would love to know if you have any guidance on the use expectations of this library from a performance context. In essence, what order of magnitude of elements would you be comfortable in diffing in, say, a for loop? 10, 100, 1000, 10000 or more? Can we rely on this as a component in a stream pipeline that handles hundreds of thousands of messages on a continuous basis, for example?

The way we're doing diffing right now is to convert the object to JSON with a fast JSON parser, get the SHA256 hash, and compare. We don't actually need to figure out in most cases what changed, just that the object has changed. Would this library offer a better, faster way to do this? Essentially, a way to stop processing once any difference is found, and return that there is a diff.

I know compared to the Go's own JSON implementation, this is likely faster, because Go's own implementation also relies on reflect. Likely this means your library is faster than it. However, I think the specific JSON library we use does not.

I'd love to know if you have a feel for what order of magnitude you'd feel comfortable using this library in re: performance. In the meanwhile, I'll probably run my own tests and post back with the results.