solnic/drops

🛠️ Tools for working with data effectively - data contracts using types, schemas, domain validation rules, type-safe casting, and more.

💖 How to support my Open Source work

• Become my Sponsor

Where to find me

• 📝 My blog with articles and videos
• 🦣 Mastodon
• X x.com/solnic_dev
• 🐙 GitHub

About

I started contributing to Open Source projects in 2008 and over time I created, contributed to and helped with over 100 libraries and frameworks. This page lists the most notable of my contributions.

Elixir Drops 💦

My latest project that brings some of the dry-rb goodies to the Elixir ecosystem.

https://solnic.dev/introducing-elixir-drops

Hanami

Hanami is a Ruby application framework with a modern component-based architecture. I've been part of the core team since 2018, working on its 2.0 release which uses various dry-rb libraries as its foundation as well as rom-rb as the default "model layer".

dry-rb

The organization was created by Andy Holland in 2015. I was thinking about doing a similar thing so I decided to contribute and started working on a couple of gems under this organization. The projects aim to be a modern take on solving common problems. Libraries are small and simple to understand with a great focus on reusability.

My main contributions include:

• dry-validation - validation library with type-safe schemas and rules
• dry-schema - schema DSL that was originally provided by dry-validation
• dry-types - a flexible "type system" for Ruby projects. Currently it's the foundation for other libraries, like rom-rb, dry-validation, hanami or reform
• dry-struct - a virtus-like attributes API for POROs
• dry-logic - composable rule objects
• dry-system - a modern way of organizing Ruby applications using dependency injection as the architectural foundation
• dry-auto_inject - container-agnostic auto-injection abstraction
• dry-events - a simple pub/sub solution
• dry-monitor - a set of abstractions that help with application monitoring

Make sure to check out our official website!

rom-rb

The work on Ruby Object Mapper (rom-rb) initially started as an attempt to build the second major version of the DataMapper project; however, in 2014 I decided to take the project in a different direction and turn it into an FP/OO hybrid toolkit that simplifies working with the data using Ruby language. It consists of a small(ish) core and plenty of adapters and extensions.

My contributions include:

• rom-core - design and implementation of the core APIs, such asROM::Gateway, ROM::Relation, ROM::Command or ROM::Mapper
• rom-repository - design and implementation of the repository component
• rom-changeset - design and implementation of the changeset component

Apart from its main components, I've also created or contributed to:

• rom-sql - the official SQL adapter
• rom-elasticsearch - the official Elasticsearch adapter
• rom-factory - the official data generator and "factory" toolkit

Make sure to check out our official website!

Transproc

Transproc is a Ruby gem which provides an API for functional composition of arbitrary Proc-like objects. It introduced the left-to-right >> function composition operator, which is now considered as a potential addition to Ruby's core.

The gem comes with a ton of built-in data transformation functions, which initially was its main purpose. It is used as the data transformation foundation in rom-rb.

Past projects

These are the projects that I used to work in the past that are now discontinued.

DataMapper

DataMapper was a Ruby ORM that was part of the default stack of the Merb framework. I started helping with the project in late 2008 and eventually joined the core team in 2010. I mostly focused on working on the Property API (which later on was extracted into a separate library called Virtus), on-going maintenance, bug fixing, user support and handling releases.

Virtus

Virtus is a project that started as an extraction of the DataMapper Property API back in 2011. Eventually it has become very popular and made typed struct-like objects in Ruby a thing and inspired people to build their own solutions too. There were also many other gems that started using Virtus under the hood for coercions, like Representable or Grape.

The project has been discontinued in 2019 because I shifted my focus on dry-rb which provides much better solutions to the same kind of problems that virtus tried to solve.

Coercible

Coercible is the coercion backend extracted from Virtus which provides a set of generic coercions for most common data types like numbers, dates etc.

drops's Issues

Support for Time coercions

JSON schema

Hi and tanks for creating this lovley validation library.

It seems like it would fit quite nice for validating JSON responses returned from OpenAI and other LLM's.

LLM's usually require a JSON Schema as input. For this reason it would be quite convenient if it would be possible to auto generate a JSON schema from the Drops schema. Would auto generating of the JSON Schema be something that makes sense to add to the Drops library?

missing alias for type spec in Contract behaviour

When I run Dialyzer against code I've build with Drops.Contract, it gives warnings like this:

lib/module_name.ex:20:callback_arg_type_mismatch
The inferred type for the 2nd argument is not a
supertype of the expected type for the conform/3 callback
in the Drops.Contract behaviour.

Success type:
%Drops.Types.Map{_ => _}

Behaviour callback type:
Types.Map

It looks like we need to alias Drops.Types or fully specify the remote type in the callback spec.

release 0.1.1 doesn't define `conform/2` for contract

I get this warning when I compile with 0.1.1:

warning: function conform/2 required by behaviour Drops.Contract is not implemented (in module ModuleName)
  lib/module_name.ex:1: ModuleName (module)

Goes away with 0.1.0, presumably because that function is defined?

schema(atomize: true) doesn't accept atoms

The schema(atomize: true) does not accept atoms. Is this a bug?

defmodule Contract do
  use Drops.Contract
  
  schema(atomize: true) do
    %{
      required(:name) => string()
    }
  end
end

Contract.conform(%{"name" => "John Doe"}) 
# => {:ok, %{name: "John Doe"}}

Contract.conform(%{name: "John Doe"}) 
# {:error,
#  [
#    %Drops.Validator.Messages.Error.Type{
#      path: [:name],
#      text: "key must be present",
#      meta: [predicate: :has_key?, args: [[:name]]]
#    }
#  ]}

Hex docs

define type as a cast

I'd like to device a custom type that uses a cast, but

use Drops.Type, (cast(string(), caster: Caster) |> string())

doesn't seem to work, giving the error:

    The following arguments were given to Drops.Type.infer_constraints/1:

        # 1
        {:cast, {{:type, {:string, []}}, {:type, {:string, []}}, [caster: Caster]}}

    Attempted function clauses (showing 6 out of 6):

        def infer_constraints([])
        def infer_constraints(map) when is_map(map)
        def infer_constraints(type) when is_atom(type)
        def infer_constraints(predicates) when is_list(predicates)
        def infer_constraints({:type, {type, predicates}}) when length(predicates) > 0
        def infer_constraints({:type, {type, []}})

    (drops 0.2.0) lib/drops/type.ex:251: Drops.Type.infer_constraints/1

Support for aliases / mapping between input keys and output fields

It would be nice to be able to map input keys to different output keys, similar to the key: option of Parameter.

This would allow to easily convert between different field names across boundaries at data ingestion point.

For example, something like this (I'm using two different imaginary options in the example)

defmodule UserPayload do
  use Drops.Contract

  schema do
    %{
      required(:user_id, source_key: "userId") => string(),
      required(:surname, aliases: ["lastName", "familyName"]) => string()
    }
  end
end

Which would allow me to conform the input map (e.g. from Jason.decode)

{
  "userId": "user-123",
  "lastName": "Doe"
}

Into the output map:

%{
  user_id: "user-123",
  surname: "Doe"
}

Support for configurable error messages via i18n

https://github.com/elixir-gettext/gettext could be used as the backend

Enum support

Thanks again for this lovely library 😊

Would you consider adding Enum's to this library? Seems like it is a common enough use case that it is suppored for JSON Schemas.

Not sure what the best way to add enum support would be, but I would be happy to contribute.

One option could be an enum type:

schema do
  %{
    required(:some_enum) => type(:enum, values: ["some_string", 1, :some_atom])
  }

The other option seems like adding a validation in Drops.Predicates.

  %{
    required(:some_enum) => type(:any, values: ["some_string", 1, :some_atom])
  }

Seems like we would need a validation either way. So might make sense to start there? Any maybe add a enum type where the values would be a required validation?

Support for Date coercions

Support multiple types

%{
  required(:id) => type([:string, :integer])
}

Data generator

Union data type

Hey, @solnic. How're you doing? Does this library support union types?

I would like to validate a map with union data type. Something like form with lots of Node where Node = Field | Section. We could think of it like:

defmodule Field do
  ...
end

defmodule Section do
  ...
end

defmodule Node do
  @type t :: Field.t() | Section.t()
end

I see this union type describer as a function named union which accepts array of types.

  union([float(), integer(), string()])

For my specific case I could use it like:

defmodule FormContract do
  use Drops.Contract

  schema do
    %{
      required(:title) => string(:filled?),
      required(:nodes) =>
        list(
          union([
            %{
              required(:__type) => "Field",
              required(:name) => string(:filled?),
              required(:value) => union([integer(), float(), string(:filled?), boolean()])
            },
            %{
              required(:__type) => "Section",
              required(:name) => string(:filled?),
              required(:fields) =>
                list(%{
                  required(:__type) => "Field",
                  required(:name) => string(:filled?),
                  required(:value) => union([integer(), float(), string(:filled?), boolean()])
                })
            }
          ])
        )
    }
  end
end

With your new Custom types it will be way easier to read:

defmodule FormContract do
  use Drops.Contract

  defmodule Field do
    use Drops.Type, %{
      required(:__type) => "Field",
      required(:name) => string(:filled?),
      required(:value) => union([integer(), float(), string(:filled?), boolean()])
    }
  end

  defmodule Section do
    use Drops.Type, %{
      required(:__type) => "Section",
      required(:name) => string(:filled?),
      required(:fields) => list(Field)
    }
  end

  # Node = Field | Section
  defmodule Node do
    use Drops.Type, union([Field, Section])
  end

  schema do
    %{
      required(:title) => string(:filled?),
      required(:nodes) => list(Node)
    }
  end
end

What do you think? I could try to implement it if you don't mind

Update version in Readme

The current version of Elixir Drops indicated in Hex 0.2.0 Feb 01, 2024 does not match what is indicated in the Readme.

Support for custom types

Support for DateTime coercions

Using the default :second unit:

defmodule TestContract do
  use Drops.Contract
  
  schema do
    %{required(:test) => from(:integer) |> type(:date_time)}
  end
end

TestContract.conform(%{test: 1695277470})
# {:ok, %{test: ~U[2023-09-21 06:24:30Z]}}

Providing time unit as an argument for the casting function:

defmodule TestContract do
  use Drops.Contract
  
  schema do
    %{required(:test) => from(:integer, [:millisecond]) |> type(:date_time)}
  end
end

TestContract.conform(%{test: 1695277723355})
# {:ok, %{test: ~U[2023-09-21 06:28:43.355Z]}}

Support for remaining predicates

• empty?
• eql?
• not_eql?
• even?
• odd?
• excludes?
• includes?
• match?
• gt?
• gteq?
• lt?
• lteq?
• max_size?
• min_size?
• size?

In Realease, ProtocolUndefinedError for Drops.Type.Validator and Drops.Types.Primitive

Hi Solnic,

I'm posting this because I'm not sure if I did something wrong, on my local machine everything is fine, but on release, I get and error when calling :

Drops.Type.Validator.validate(Drops.Types.Primitive.new(:string), "")

Drops.Type.Validator.validate(Drops.Types.Primitive.new(:string), "")
** (Protocol.UndefinedError) protocol Drops.Type.Validator not implemented for %Drops.Types.Primitive{primitive: :string, constraints: [predicate: {:type?, [:string]}], opts: []} of type Drops.Types.Primitive (a struct). This protocol is implemented for the following type(s): Drops.Types.Cast, Drops.Types.List, Drops.Types.Map, Drops.Types.Map.Key, Drops.Types.Union, Kernel
    (drops 0.2.0) lib/drops/type/validator.ex:1: Drops.Type.Validator.impl_for!/1
    (drops 0.2.0) lib/drops/type/validator.ex:7: Drops.Type.Validator.validate/2
    iex:33: (file)

Maybe it's not related to your lib but related to the release mode and protocol. I don't know.

Cheers

The in? predicate raises an exception for single-element lists.

The in? predicate raises an exception when the list contains a single element.

defmodule Contract do
  use Drops.Contract

  schema do
    %{
      optional(:side) => string(in?: ["salad"]),
      optional(:meat) => string(in?: ["chicken", "beef"])
    }
  end
end

Contract.conform(%{meat: "chicken"}) # => {:ok, %{meat: "chicken"}}

Contract.conform(%{side: "salad"})

# ** (FunctionClauseError) no function clause matching in Drops.Predicates.in?/2
# 
#     The following arguments were given to Drops.Predicates.in?/2:
# 
#         # 1
#         "salad"
# 
#         # 2
#         "salad"
# 
#     Attempted function clauses (showing 1 out of 1):
# 
#         def in?(list, input) when is_list(list)
# 
#     (drops 0.2.0) lib/drops/predicates.ex:377: Drops.Predicates.in?/2
#     (drops 0.2.0) lib/drops/predicates/helpers.ex:22: Drops.Predicates.Helpers.apply_predicate/2
#     (elixir 1.17.1) lib/enum.ex:2531: Enum."-reduce/3-lists^foldl/2-0-"/3
#     (drops 0.2.0) lib/drops/types/map/key.ex:15: Drops.Types.Map.Key.validate/2
#     (elixir 1.17.1) lib/enum.ex:1703: Enum."-map/2-lists^map/1-1-"/2
#     (elixir 1.17.1) lib/enum.ex:1703: Enum."-map/2-lists^map/1-1-"/2
#     (drops 0.2.0) lib/drops/types/map.ex:63: Drops.Types.Map.Validator.validate/2
#     iex:6: Contract.conform/3

How to handle "numbers"

I tried to use a schema for a API endpoint I made that receives JSON. I this case the request originated from a JS environment like this JSON.stringify({threshold: 1.0}) => "{\"threshold\":1}", this is valid behavior for JSON since there is only a numeric type.

However I cant figure out how to process this data with Drops.

Naturally a schema like this will complain about 1 being an integer.

defmodule MySchema do
  use Drops.Contract

  schema(atomize: true) do
    %{
      required(:threshold) => float()
    }
  end
end

After some trial and error I ended up with this, which works:

defmodule MySchema do
  use Drops.Contract

  schema(atomize: true) do
    %{
      required(:threshold) => cast(:any, caster: __MODULE__) |> float()
    }
  end

  def cast(:any, :float, value, _) when is_number(value) do
    value * 1.0
  end
end

Maybe there should also be a "number" type here?

defmodule MySchema do
  use Drops.Contract

  schema(atomize: true) do
    %{
      required(:threshold) => cast(:number) |> float()
    }
  end
end

UndefinedFunctionError when using map(:string) field type

Contract taken from the README (and slimmed down):

defmodule UserContract do
  use Drops.Contract

  schema do
    %{
      required(:name) => string(),
      optional(:settings) => map(:string),
      required(:address) => maybe(:string)
    }
  end
end

Trying to conform a map with settings results in an unmanaged UndefinedFunctionError exception:

UserContract.conform(%{name: "Jane", settings: %{"theme" => "dark"}, address: "main road 1"})

** (UndefinedFunctionError) function Drops.Predicates.string/1 is undefined or private
    (drops 0.2.0) Drops.Predicates.string(%{"theme" => "dark"})
    (drops 0.2.0) lib/drops/predicates/helpers.ex:22: Drops.Predicates.Helpers.apply_predicate/2
    (elixir 1.17.1) lib/enum.ex:2531: Enum."-reduce/3-lists^foldl/2-0-"/3
    (drops 0.2.0) lib/drops/types/map/key.ex:15: Drops.Types.Map.Key.validate/2
    (elixir 1.17.1) lib/enum.ex:1703: Enum."-map/2-lists^map/1-1-"/2
    (drops 0.2.0) lib/drops/types/map.ex:63: Drops.Types.Map.Validator.validate/2
    #cell:rziepynrh7l5vn3j:2: UserContract.conform/3
    #cell:7x24e72nc24vkawk:1: (file)

No issue if we remove the settings key from the map:

UserContract.conform(%{name: "Jane", address: "main road 1"})
# {:ok, %{name: "Jane", address: "main road 1"}}

Is it a bug with the library, or an incorrect/outdated README/documentation?
If the latter, what's the proper way to have a field that needs to be a map with string values?

cast to a map/struct

I would like to be able to cast to a map/struct, but

cast(string(), Caster) |> map()

doesn't work, as there doesn't seem to be a variant of the map function that takes a cast_spec as first argument.

Module free schema

I was wondering if we just make module free schema, I mean I have lots of endpoints to validate, creating module for every one of them is tedious, it would be simple and easy if we had capability to do all the validation without a module struct dependency with it

validate types/schemas inside other schemas?

Firstly, thanks for this awesome library. Big fan of dry-rb and great to see you in elixir-land!

I'm curious about one thing - it would be ideal to be able to individually create/validate subschemas/types, a little more than pure maps so I can add more checks around the place prior to validating the "master" contract. I have previously used DB-less Ecto quite a bit for this, or just pattern matching on structs at the argument level.

It would be helpful to be able to either call "conform" on a type, or otherwise use a structs-like interface to be able to create/match on types. Either that or make a contract embed-able into another.

Any plans (or recommendations) along these lines?

Thanks once again.

Support for lists

defmodule TestContract do
  use Drops.Contract

  schema do
    %{
      required(:tags) => type(list: [:string, :filled?])
    }
  end
end

Support to declare that a key needs to conform with the given Struct

I want to be able to declare the following contract required(:key) => %SomeStruct{} on a drop.

It doesn't seem possible to achieve this, even when using custom types, or am I missing something?

Full example:

defmodule MyApp.APIContract do
    use Drops.Contract

    alias MyApp.WhateverStruct

    @enforce_keys [:context, :action]

    defstruct @enforce_keys

    schema do
      %{
        required(:context) => %WhateverStruct{},
        required(:action) => string(:filled?),
      }
    end

    def new(attrs) when is_map(attrs) do
      case conform(attrs) do
        {:ok, attrs} ->
          struct(__MODULE__, attrs)

        error ->
          error
      end
    end
  end

How to define a filled list of strings

I'm struggling to define a contract to validate a list of strings where each string must be filled, and the minimum size of the list is 1.

%{tags: ["hello"]]} # valid

%{tags: []}   # invalid
%{tags: [1]}  # invalid

I ended up writing a rule, but is it possible to achieve it without a rule?

defmodule Contract do
  use Drops.Contract
  
  schema do
    %{
      required(:tags) => list(:string, [:filled?])
    }
  end
  
  rule(:tags_filled, %{tags: []}) do
    {:error, "tags must not be empty"}
  end
end