go-playground/mold

:scissors: Is a general library to help modify or set data within data structures and other objects.

mold's Issues

*string to nil if empty string

Let's say I have this struct:

type User struct {
  Username *string
}

and I need to sanitize it from empty string: I want Username to be nil if is an empty string (maybe because previous modifiers made it empty).

Is it correct to do this with this package?

Add it to awesome-go

Add it to https://github.com/avelino/awesome-go.

URGENT: Embedded structs

What I miss using this instead of https://github.com/leebenson/conform is the check of embedded structs:
https://github.com/leebenson/conform/blob/master/conform.go#L272.

Can we add this?

Maybe optional (default: true) with a tag: mod:"embedded=false".

What do you think?

This is URGENT IMO because I cannot use it having a lot of embedded structs.

Register() on modifiers.New()

If I try to register on modifiers.New() it compiles but doesn't work.

var conform = modifiers.New()

func init() {
	conform.Register("set", transformMyData)
}

func transformMyData(ctx context.Context, t *mold.Transformer, value reflect.Value, param string) error {
	value.SetString("test")
	return nil
}

func Struct(myStruct interface{}) error {
	if err := conform.Struct(context.Background(), myStruct); err != nil {
		return err
	}
	return nil
}

Am I right?

Struct func doesn't work on slice fields

Struct(ctx context.Context, v interface{}) error func doesn't work on slice fields.

Here is the example code (inspired by _examples/full/main.go):

import (
	"context"
	"fmt"
	"log"
	"reflect"

	"github.com/go-playground/mold/v3"
	"github.com/go-playground/mold/v3/modifiers"
)

type Address struct {
	Name  string `mod:"trim"`
	Phone string `mod:"trim"`
}

type User struct {
	Name    string           `mod:"trim"`
	Age     uint8
	Gender  string          `mod:"trim"`
	Email   string             `mod:"trim"`
	Address []Address
}

func main() {
        user := User{
		Name:   "test   ",
		Age:    50,
		Gender: "female",
		Email:  "[email protected]",
		Address: []Address{
			{
				Name:  "test   address    ",
				Phone: " 123 456  ",
			},
			{
				Name:  "test   address    ",
				Phone: " 123 456  ",
			},
		},
	}

        conform  := modifiers.New()
	if err := conform.Struct(context.Background(), &user); err != nil {
		log.Panic(err)
	}
	fmt.Printf("Conformed:%+v\n\n", user)
}

Output:

Conformed:{Name:test Age:50 Gender:female Email:[email protected] Address:[{Name:test   address     Phone: 123 456  } {Name:test   address     Phone: 123 456  }]}

So, Name and Phone fields of the Address struct weren't trimmed.

If I add dive tag to the Address field the Struct func will panic:

type User struct {
        ...
	Address []Address `mod:"dive"`
}

Panic:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x32 pc=0x10c0c19]

goroutine 1 [running]:
github.com/go-playground/mold/v3.(*Transformer).setByField(0xc000021ac0, 0x11cf500, 0xc0000160a0, 0x1176760, 0xc000021b00, 0x199, 0xc00018ae70, 0x0, 0x0, 0x0)
/Users/myuser/go/pkg/mod/github.com/go-playground/mold/[email protected]/mold.go:202 +0x79
github.com/go-playground/mold/v3.(*Transformer).setByField(0xc000021ac0, 0x11cf500, 0xc0000160a0, 0x115af80, 0xc00009adf8, 0x197, 0xc00018ae40, 0x0, 0x0, 0x0)
...

The reason for the panic is that the dive tag could not be used alone, so you'll have to add some tag after it (endkeys tag doesn't work as well).

The workaround for that issue is to register a dummy transform function:

func items(_ context.Context, _ *mold.Transformer, _ reflect.Value, _ string) error {
	return nil
}

...

conform.Register("items", items)

... and add a mod:"dive,items" tag to the slice field:

type User struct {
        ...
	Address []Address `mod:"dive,items"`
}

I think that the issue should be addressed either by fixing this in the lib or by adding that to the documentation.

What do you think?

P.S.
I can help with a PR if you need it.

Thanks.

Add docs for json tags list

Add docs for json tags list

`mod` tag vs `mold` tag?

It appears in the long example that mod tag is used when I would guess it's supposed to be mold?

https://github.com/go-playground/mold/blob/master/_examples/full/main.go#L45

Add support for "dive,keys,…,endkeys,dive"

Use case:

type A struct {
    M map[string]*B `mod:"dive"` // WANTED: `mod:"dive,keys,trim,endkeys,dive"`
}
type B struct {
    S string `mod:"trim"`
}

With "dive" on M it will process "trim" on S, but won't process map keys.
With "dive,keys,trim,endkeys" on M it will process map keys but won't process "trim" on S.
And the WANTED string result in an error "invalid dive tag configuration".

Email, maybe a new one?

What do you think about conform's email?

https://github.com/leebenson/conform/blob/master/conform.go#L31-L51

question: plans to port validator on top of this?

Seems like the Transform is a refactoring of the basic struct traversal logic from validator. Was thinking it would be really useful to have a re-usable struct visitor library like this, to build focussed struct processing packages, like one which sets default values, initializes values from environment variables, or validates.

One thing I see missing from mold, which validator uses, is that the Func doesn't know where it is in the hierarchy, so it isn't possible to include that information in error messages or validation problems.

I'm interested in the idea of building other types of struct processors on top of this core because I use a number of them currently, and they are all slightly different in how they work. For example, whether they allow setting the tag name, what they do with fields with no tags, how many tag names that package uses, how they traverse embedded structs, which types they handle, etc. Would be really nice to have a core traversal logic which encouraged some patterns, like one (configurable) tag per package/processor, consistent handling of untagged fields, etc.

Gin example

Do you have any example how to setup gin with v9 and modifiers/scrubbers?

Coming from other languages this seems like a nice way to handle it without writing a custom validator/processor for every user input. The only thing is, at least for me as a newish guy to golang, I have no clue how to integrate it with gin. The validator-upgrade.go is somewhat understandable, but I'm totally lost on how and where to hook into. Where does gin and the validator do the stuff and how do I inject mold.Transformer and actually execute it, prior to the validator (binding) run?

Can you give any hints or is there any help page I've missed?

Is this a misprint?

Why do we need this line here?

https://github.com/go-playground/mold/blob/master/modifiers/multi.go#L21

fmt.Println(param, v.Type())

Maybe a bug, scrubs emails

mold/scrubbers/string.go

Line 17 in 4214131

scrubbed := fmt.Sprintf("<<scrubbed::email::sha1::%s>>", hashString(input[idx:]))

Maybe this?

scrubbed := fmt.Sprintf("<<scrubbed::email::sha1::%s>>", hashString(input[:idx])) 

Should hash the string before @?

Support for more transformers like "max length" on string etc

I'm coming from a Node.js background. My company is switching a lot of text processing stuff from Node to Go and we're looking for validation and sanitization libraries.

We used to use a library called "schema-inspector"
(https://www.npmjs.com/package/schema-inspector) which broke this into two steps. They have a "sanitize" step where the rules are used to modify the data. Then they have a "validate" step where if the data doesn't fit the rule, it throws an error. We use this for strings with rules like "max length". For example, if you have a rule where the username property has a max length of 5 and this object:

{
  "username": "bobbytables"
}

then after the sanitize step, you'll end up with this object:

{
  "username": "bobby"
}

And then after the validate step, this would pass because the data has been transformed to meet the rule during the sanitization step. Usually situations where the data would pass sanitization but fail validation are when you've got broad rules like "max length on a string" in the sanitize step but more focussed rules like a regexp match on the validate step.

I was wondering if as the library authors, you are willing to accept PRs to add these features? I notice the following source code in the library:

// New returns a modifier with defaults registered
func New() *mold.Transformer {
	mod := mold.New()
	mod.SetTagName("mod")
	mod.Register("trim", TrimSpace)
	mod.Register("ltrim", TrimLeft)
	mod.Register("rtrim", TrimRight)
	mod.Register("tprefix", TrimPrefix)
	mod.Register("tsuffix", TrimSuffix)
	mod.Register("lcase", ToLower)
	mod.Register("ucase", ToUpper)
	mod.Register("snake", SnakeCase)
	return mod
}

This is really simple to understand even as a Go novice. I can see how I would extend this by adding more modules I can register. There could be a module added to handle trimming based on a max length for example. If you're willing to accept PRs, I can direct the work here instead of us rolling our own.

Looking forward to hearing back from you!

Probable bugs

These lines: https://github.com/go-playground/mold/blob/master/mold.go#L286-L296

Warnings by Goland (JetBrains):

Accessing field ct.keys may lead to nil pointer dereference
Accessing field ct.next may lead to nil pointer dereference