bluzky/tarams

Cast and validate external data and request parameters for Elixir and Phoenix

Tarams

Phoenix request params validation library.

Warning: Tarams v1.0.0 APIs is not back compatible

• Tarams
  • Why Tarams
  • Installation
  • Usage
  • Define schema
    • Default value
    • Custom cast function
      • 1. Custom cast function accept value only
      • 2. Custom cast function accept value and current object
      • 3.Custom cast function accept tuple {M, f}
    • Nested schema
  • Transform data
    • Field name alias
    • Convert data
  • Validation
  • Contributors

Why Tarams

- Reduce code boilerplate 
- Shorter schema definition
- Default function which generate value each casting time
- Custom validation functions
- Custom parse functions

Installation

Available in Hex, the package can be installed by adding tarams to your list of dependencies in mix.exs:

def deps do
  [
    {:tarams, "~> 1.0.0"}
  ]
end

Usage

Process order

Cast data -> validate casted data -> transform data

@index_params_schema  %{
    keyword: :string,
    status: [type: :string, required: true],
    group_id: [type: :integer, number: [greater_than: 0]],
    name: [type: :string, from: :another_field]
  }

def index(conn, params) do
    with {:ok, better_params} <- Tarams.cast(params, @index_params_schema) do
        # do anything with your params
    else
        {:error, errors} -> # return params error
    end
end

Define schema

Schema is just a map and it can be nested. Each field is defined as

<field_name>: [<field_spec>, ...]

Or short form

<field_name>: <type>

Field specs is a keyword list thay may include:

• type is required, Tarams support same data type as Ecto. I borrowed code from Ecto
• default: default value or default function
• cast_func: custom cast function
• number, format, length, in, not_in, func, required, each are available validations
• from: use value from another field
• as: alias key you will receive from Tarams.cast if casting is succeeded

Default value

You can define a default value for a field if it's missing from the params.

schema = %{
    status: [type: :string, default: "pending"]
}

Or you can define a default value as a function. This function is evaluated when Tarams.cast gets invoked.

schema = %{
    date: [type: :utc_datetime, default: &Timex.now/0]
}

Custom cast function

You can define your own casting function, tarams provide cast_func option. Your cast_func must follows this spec

1. Custom cast fuction accept value only

fn(any) :: {:ok, any} | {:error, binary} | :error

def my_array_parser(value) do
    if is_binary(value) do
        ids = 
            String.split(value, ",")
            |> Enum.map(&String.to_integer(&1))
        
        {:ok, ids}
    else
        {:error, "Invalid string"
    end
end

schema = %{
    user_id: [type: {:array, :integer}, cast_func: &my_array_parser/1]
}

Tarams.cast(%{user_id: "1,2,3"}, schema)

This is a demo parser function.

2. Custom cast function accept value and current object

data = %{
   name: "tada",
   bold: true
}

schema = %{
    name: [type: :string, cast_func: fn value, data -> 
        {:ok, (if data.bold, do: String.upcase(value), else: value)}
    end]
}

Tarams.cast(data, schema)

# > %{name: "TADA"}

3.Custom cast function accept tuple {M, f}

Your cast function must accept 2 arguments

defmodule MyModule do
    def upcase(value, data) do
        {:ok, (if data.bold, do: String.upcase(value), else: value)}
    end
end

data = %{
   name: "tada",
   bold: true
}

schema = %{
    name: [type: :string, cast_func: {MyModule, :upcase}]
}

Tarams.cast(data, schema)

# > %{name: "TADA"}

Nested schema

With Tarams you can parse and validate nested map and list easily

@my_schema %{
    status: :string,
    pagination: %{
        page: [type: :integer, number: [min: 1]],
        size: [type: :integer, number: [min: 10, max: 100"]]
    }
}

Or nested list schema

@user_schema %{
    name: :string,
    email: [type: :string, required: true]
    addresses: [type: {:array, %{
        street: :string,
        district: :string,
        city: :string
    }}]
}

Validation

Tarams uses Valdi validation library. You can read more about Valdi here Basically it supports following validation

• validate inclusion/exclusion

• validate length for string and enumerable types

• validate number

• validate string format/pattern

• validate custom function

• validate required(not nil) or not

• validate each array item

  product_schema = %{
    sku: [type: :string, required: true, length: [min: 6, max: 20]]
    name: [type: :string, required: true],
    quantity: [type: :integer, number: [min: 0]],
    type: [type: :string, in: ~w(physical digital)],
    expiration_date: [type: :naive_datetime, func: &my_validation_func/1],
    # dynamic required
    width: [type: :integer, required: fn value, data -> data.type == "physical" end],
    # validate each array item
    tags: [type: {:array, :string}, each: [length: [max: 50]]]
  }

Dynamic required

• Can accept function or {module, function} tuple
• Only support 2 arity function

def require_email?(value, data), do: is_nil(email.phone)

....

%{
    phone: :string
    name: [type: :string, required: fn value, data -> true end],
    email: [type: :string, required: {__MODULE__, :require_email?}]
}

Validate array item

Support validate array item with :each option, each accept a list of validators

%{
    values: [type: {:array, :number}, each: [number: [min: 20, max: 50]]]
  }

Transform data

Field name alias

You can set alias name for schema fields

data = %{
   name: "tada"
}

schema = %{
    name: [type: :string, as: :full_name]
}

Tarams.cast(data, schema)

# > %{full_name: "tada"}

Convert data

You can specify a function similar to cast_func to manipulate data after casted. However data object passed to transform function is original data before casting.

data = %{status: 10}

schema = %{
    name: [type: :string, into: fn value -> {:ok, "name: #{value}}" end]
}

Tarams.cast(data, schema)
# > %{name: "name: tada"}

• Transform function can return tuple {:ok, value}, {:error, message} or value directly.

schema = %{
    value: [type: :integer, into: &to_string/1]
}

Contributors

If you find a bug or want to improve something, please send a pull request. Thank you!

tarams's Issues

Inclusion doesn't work with array

iex(14)> Tarams.cast(%{"farm" => ["cow","sheep","goat"]}, %{farm: [type: {:array, :string}, in: ["cow", "sheep", "goat"]]})
{:error, %{farm: ["not be in the inclusion list"]}} 

How can I validate each element of the array?

Drop non required fields from validated map

I realize that non required fields are retuned to me as nil values

 Tarams.cast(%{"id" => "some_id"}, %{
    id: [type: :string, required: true],
    optional: [type: :string]
  })

{:ok, %{id: "some_id", optional: nil}}

There some parameter that I can pass to tarams to remove the non required fields from validated map?

Feature Request - Support for dynamic keys for maps/nested schemas

Currently there is no native way to validate a map of dynamic keys

dynamic_map = %{ id_1: %{ value1: "value1" }, id_2: %{ value1: "value1" }}

BadMapError exception when non map value is passed as value for nested schema

Reproductions

schema = %{ test: %{ nested: %{ schema: :string }}}
Tarams.cast(%{ test: %{ nested: []}}, schema)
Tarams.cast(%{ test: %{ nested: 1}}, schema)
Tarams.cast(%{ test: %{ nested: "nested"}}, schema)

Expected:

{:error, %{test: %{nested: ["is invalid"]}}}

Actual:

** (BadMapError) expected a map, got: []
    (stdlib 4.0.1) :maps.find(:schema, [])
    (tarams 1.7.0) lib/tarams.ex:125: Tarams.get_value/3
    (tarams 1.7.0) lib/tarams.ex:107: Tarams.do_cast/3
    (tarams 1.7.0) lib/tarams.ex:83: Tarams.cast_field/2
    (elixir 1.13.4) lib/enum.ex:1597: anonymous fn/3 in Enum.map/2
    (stdlib 4.0.1) maps.erl:411: :maps.fold_1/3
    (elixir 1.13.4) lib/enum.ex:2408: Enum.map/2
    (tarams 1.7.0) lib/tarams.ex:63: Tarams.cast_data/2
    (tarams 1.7.0) lib/tarams.ex:38: Tarams.cast/2
    (tarams 1.7.0) lib/tarams.ex:112: Tarams.do_cast/3
    (tarams 1.7.0) lib/tarams.ex:83: Tarams.cast_field/2
    (elixir 1.13.4) lib/enum.ex:1597: anonymous fn/3 in Enum.map/2
    (stdlib 4.0.1) maps.erl:411: :maps.fold_1/3
    (elixir 1.13.4) lib/enum.ex:2408: Enum.map/2
    (tarams 1.7.0) lib/tarams.ex:63: Tarams.cast_data/2
    (tarams 1.7.0) lib/tarams.ex:38: Tarams.cast/2
    (tarams 1.7.0) lib/tarams.ex:112: Tarams.do_cast/3
    (tarams 1.7.0) lib/tarams.ex:83: Tarams.cast_field/2
    (elixir 1.13.4) lib/enum.ex:1597: anonymous fn/3 in Enum.map/2
    (stdlib 4.0.1) maps.erl:411: :maps.fold_1/3

The get_value function can probably use a guard for maps and have another function head to return the error. Might need to include :ok and :error so we can handle the result before attempting to cast

defp get_value(data, field_name, default \\ nil) when is_map(data) do
  case Map.fetch(data, field_name) do
    {:ok, value} -> {:ok, value}
    _ ->
      case Map.fetch(data, "#{field_name}") do
        {:ok, value} -> {:ok, value}
        _ -> {:ok, default}
      end
  end
end

defp get_value(data, _field_name, default) when not is_nil(default), do: {:ok, default}
defp get_value(_date, _field_name, _default), do: :error

Then we can update the do_cast function to account for this new source of errors

  defp do_cast(data, field_name, definitions) do
    field_name =
      if definitions[:from] do
        definitions[:from]
      else
        field_name
      end

    apply_cast = fn value ->
      case definitions[:cast_func] do
        nil -> cast_value(value, definitions[:type])
        func -> apply_function(func, value, data)
      end
    end

    with {:ok, value} <- get_value(data, field_name, definitions[:default]),
         {:ok, value} <- apply_cast(value) do
      {:ok, value}
    else
      {:error, "is invalid"}
    end
  end

I can make this change, just wanted to hear some input or maybe some alternative ways around this issue I did not think of

How to collect cast errors and validate error together?

I have the following validation schema (it's work properly)

  @order_params %{
    status: [type: :string, required: true, func: &Order.valid_status/1],
    user_id: [type: Ecto.UUID, required: true, func: &User.exists/1],
    services: [
      type: :array,
      required: true,
      func: &Service.has_one/1,
      cast_func: {__MODULE__, :validate_order_services}
    ]
  }

But when I execute the cast, if i have any cast errors, the validation errors are not returned (I receive just the cast errors), if I don't have any cast errors, the validation errors are showed up.

How can I get all the errors at same time (cast and validations errors)?

I need all the erros at the same time, to assign all errors fields in front app.

How To Make Sure a String Is One of List of Options?

I would like to use Tarams to make sure a parameter string is one of 3 possible strings.

I tried to use a cast function like this:

@params %{
    aggregate: [type: :string, required: true, cast_func: &Enum.member?(["avg","min","max"],&1)],
    country_id: :integer
  }

Or with a named function like this:

  def contains_value(value) do
    if Enum.member?(["avg", "min", "max"], value) do
      {:ok,_}
    else
      {:error, "not in list"}
    end
  end

  @show_params %{
    aggregate: [type: :string, required: true, cast_func: &contains_value/1],
    country_id: :integer
  }

Sadly nothing works. Could you may provide an example how to achieve this?

Feature Request - Support for Ecto.Enum

It would be awesome to have Ecto.Enum as a possible type allowing us to use Ecto.Enum.values() during the conversion phase.