dry-rb/dry-transformer

Data transformation toolkit

dry-transformer's Issues

Simplified creation of pipes

Provide an example in the user documentation for short pipe definition + provide helper function to make it possible.

Dry::Transformer introduced the idea of wrapping transformation into classes, which I really like. The new syntax with nested block of code is much more readable than the original one with function composition .>>. The problem I see in the new syntax, is the fact that you have to define a new class for each complex transformation. I know the rationale for that, but this seems awkward.
I would suggest a simple helper function in the Pipe class, that would make the code more rubish.

Examples

Let's assume we have the following module and a class:

module Transformations
  extend Dry::Transformer::Registry
  import Dry::Transformer::HashTransformations
  import Dry::Transformer::ArrayTransformations
  import Dry::Transformer::Recursion
  import Dry::Transformer::Conditional
  import Dry::Transformer::ClassTransformations
  import Dry::Transformer::ProcTransformations
end

class TPipe < Dry::Transformer::Pipe
  import Transformations
end

The module is defined as an entry point for user-defined, global transformations.
The class is defined as a base class for all user-defined pipes. At present defining a new complex transformation requires the following code:

class Service
  def fetch(params)
    converter.call(make_request(params))
  end

  private
  def converter
    Class.new(TPipe) do
      define! do
        map_array do
          rename_keys data: :content
          unwrap :metadata, [:id, :similarity]
        end
      end
    end.new
  end
end

The anonymous class is used to avoid namespace pollution, but the code seems awkward.

We can make it look better with the following definition of TPipe:

class TPipe < Dry::Transformer::Pipe
  import Transformations

  def self.make!(&block)
    Class.new(TPipe) do
      define!(&block)
    end.new
  end
end

The converter then looks as follows:

class Service
  private
  def converter
    TPipe.make! do
      map_array do
        rename_keys data: :content
        unwrap :metadata, [:id, :similarity]
      end
    end
  end
end

As a result it is easier to define new conversions as methods in the local context.
I suggest make! was defined in the Dry::Transoformer::Pipe, so we won't have to define it.
Alternatively current implementation of define! could be changed to allow the syntax introduced by make!.

[Security] Workflow sync_configs.yml is using vulnerable action actions/checkout

The workflow sync_configs.yml is referencing action actions/checkout using references v1. However this reference is missing the commit a6747255bd19d7a757dbdda8c654a9f84db19839 which may contain fix to the some vulnerability.
The vulnerability fix that is missing by actions version could be related to:
(1) CVE fix
(2) upgrade of vulnerable dependency
(3) fix to secret leak and others.
Please consider to update the reference to the action.

Dead links in docs

https://dry-rb.org/gems/dry-transformer/0.1/built-in-transformations/ links to builtin transformation docs are not valid

Better error reporting

Describe the bug

The error reporting when a function is wrongly composed is not useful.

To Reproduce

Let's take the example from the documentation

require 'dry/transformer'

class Mapper < Dry::Transformer::Pipe
  import Dry::Transformer::ArrayTransformations
  import Dry::Transformer::HashTransformations

  define! do
    map_array do
      symbolize_keys
      rename_keys :user_name, :name # the error is here!
      nest :address, [:city, :street, :zipcode]
    end
  end
end


mapper = Mapper.new

pp mapper.(
  [
    { 'user_name' => 'Jane',
      'city' => 'NYC',
      'street' => 'Street 1',
      'zipcode' => '123'
    }
  ]
)

Running the code gives:

Traceback (most recent call last):
	11: from example.rb:19:in `<main>'
	10: from /home/apohllo/.rvm/gems/ruby-2.7.2/gems/dry-transformer-0.1.1/lib/dry/transformer/pipe.rb:71:in `call'
	 9: from /home/apohllo/.rvm/gems/ruby-2.7.2/gems/dry-transformer-0.1.1/lib/dry/transformer/function.rb:50:in `call'
	 8: from /home/apohllo/.rvm/gems/ruby-2.7.2/gems/dry-transformer-0.1.1/lib/dry/transformer/function.rb:50:in `call'
	 7: from /home/apohllo/.rvm/gems/ruby-2.7.2/gems/dry-transformer-0.1.1/lib/dry/transformer/array.rb:45:in `map_array'
	 6: from /home/apohllo/.rvm/gems/ruby-2.7.2/gems/dry-transformer-0.1.1/lib/dry/transformer/array.rb:45:in `map'
	 5: from /home/apohllo/.rvm/gems/ruby-2.7.2/gems/dry-transformer-0.1.1/lib/dry/transformer/array.rb:45:in `block in map_array'
	 4: from /home/apohllo/.rvm/gems/ruby-2.7.2/gems/dry-transformer-0.1.1/lib/dry/transformer/composite.rb:33:in `call'
	 3: from /home/apohllo/.rvm/gems/ruby-2.7.2/gems/dry-transformer-0.1.1/lib/dry/transformer/composite.rb:33:in `call'
	 2: from /home/apohllo/.rvm/gems/ruby-2.7.2/gems/dry-transformer-0.1.1/lib/dry/transformer/function.rb:50:in `call'
	 1: from /home/apohllo/.rvm/gems/ruby-2.7.2/gems/dry-transformer-0.1.1/lib/dry/transformer/function.rb:50:in `call'
/home/apohllo/.rvm/gems/ruby-2.7.2/gems/dry-transformer-0.1.1/lib/dry/transformer/hash.rb:170:in `rename_keys': wrong number of arguments (given 3, expected 2) (ArgumentError)

Expected behavior

I would like that the stack trace included the line that introduced the error, i.e. the line with the comment. But there is no reference to that line, making debugging of the composed code pretty hard. This stays in contrast with regular ruby code, which would directly indicate the line that has the error.

Inconsistent behavior when passing array schema to a property

Describe the bug

There's an inconsistency that leads to some unexpected behavior when using nested schemas. The problem is best described by the example below.

To Reproduce & Expected behavior

FriendSchema = Dry::Schema.Params do
  optional(:nickname).filled(:string)
end

class UserSchema < Dry::Validation::Contract
  params do
    optional(:name)
    optional(:close_friends).maybe(:array, FriendSchema)
    optional(:friends).maybe do
      array(FriendSchema)
    end
  end
end

UserSchema.new.call({name: "John", friends: []}) # works
UserSchema.new.call({name: "John", close_friends: []}) # expects to work but does not

NoMethodError: undefined method `key?' for []:Array
from /Users/lenart/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/dry-logic-1.1.0/lib/dry/logic/predicates.rb:25:in `key?'

If filled(:string) is omitted from FriendSchema the error goes away but in that case, all params get through.

FriendSchema = Dry::Schema.Params do
  optional(:nickname)
end

# UserSchema same as in above example

UserSchema.new.call({name: "John", close_friends: [{any: 'value'}]})
# expected {name: "John"}
# actual {name: "John", close_friends: [{any: "value"}]}

UserSchema.new.call({name: "John", friends: [{any: 'value'}]})
# returns as expected {name: "John", friends: [{}]}

I'm not sure that the empty object is what I'd expect in the friends array.

---

The same syntax seems to work fine when dealing with hashes (instead of arrays).

FriendSchema = Dry::Schema.Params do
  optional(:nickname).filled(:string)
end

class UserSchema < Dry::Validation::Contract
  params do
    optional(:name)
    optional(:close_friends).maybe(:hash, FriendSchema)
    optional(:friends).maybe do
      hash(FriendSchema)
    end
  end
end

UserSchema.new.call({name: "John", close_friends: {any: 'value'}}) 
# returns as expected {name: "John", close_friends: {}}

UserSchema.new.call({name: "John", friends: {any: 'value'}}) 
# returns as expected {name: "John", friends: {}} 

My environment

• Affects my production application: YES
• Ruby version: 2.6.6
• OS: MacOS Big Sur 11.2.1
• dry-transformer: 0.1.1

other related gems in Gemfile.lock

    dry-configurable (0.12.1)
    dry-container (0.7.2)
    dry-core (0.5.0)
    dry-equalizer (0.3.0)
    dry-inflector (0.2.0)
    dry-initializer (3.0.4)
    dry-logic (1.1.0)
    dry-schema (1.6.1)
    dry-transformer (0.1.1)
    dry-types (1.5.1)
    dry-validation (1.6.0)

Error importing library

Describe the bug

I cannot import and use it.

To Reproduce

osboxes@eduardo-XPS-13-9310:~/w/pricing-update$ cat Gemfile.lock | grep dry
    dry-configurable (0.16.1)
      dry-core (~> 0.6)
    dry-container (0.11.0)
    dry-core (0.9.1)
    dry-inflector (0.3.0)
    dry-initializer (3.1.1)
    dry-logic (1.3.0)
      dry-core (~> 0.9, >= 0.9)
    dry-schema (1.11.3)
      dry-configurable (~> 0.16, >= 0.16)
      dry-core (~> 0.9, >= 0.9)
      dry-initializer (~> 3.0)
      dry-logic (~> 1.3)
      dry-types (~> 1.6)
    dry-transformer (1.0.1)
    dry-types (1.6.1)
      dry-container (~> 0.3)
      dry-core (~> 0.9, >= 0.9)
      dry-inflector (~> 0.1, >= 0.1.2)
      dry-logic (~> 1.3, >= 1.3)
    dry-validation (1.9.0)
      dry-container (~> 0.7, >= 0.7.1)
      dry-core (~> 0.9, >= 0.9)
      dry-initializer (~> 3.0)
      dry-schema (~> 1.11, >= 1.11.0)
      dry-validation (~> 1.0)
  dry-transformer (~> 1.0, >= 1.0.1)
  dry-validation (~> 1.8, >= 1.8.1)
osboxes@eduardo-XPS-13-9310:~/w/pricing-update$ rails c
Loading development environment (Rails 7.0.4)
3.2.1 :001 > require 'dry/transformer/all'
/usr/share/rvm/gems/ruby-3.2.1/gems/zeitwerk-2.6.0/lib/zeitwerk/kernel.rb:35:in `require': cannot load such file -- dry/transformer/all (LoadError)
3.2.1 :002 > 

And

$ cat Gemfile | grep dry
gem 'dry-validation', '~> 1.8', '>= 1.8.1'
gem 'dry-transformer', '~> 1.0', '>= 1.0.1'

And

$ bundle install | grep trans
Using dry-transformer 1.0.1

Expected behavior

It should import and work.

My environment

$ ruby --version
ruby 3.2.1 (2023-02-08 revision 31819e82c8) [x86_64-linux]
$ bundle --version
Bundler version 2.3.22
$ rails --version
Rails 7.0.4

Code examples doesn't work

Code examples in the "Transformation objects" documentation doesn't work:

class MyMapper < Dry::Transformer[Dry::Transformer::Registry]
  define! do
    map_array do
      symbolize_keys
      rename_keys user_name: :name
      nest :address, [:city, :street, :zipcode]
    end
  end
end

mapper = MyMapper.new

Cause:

Traceback (most recent call last):
       23: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/bin/irb:23:in `<main>'
       22: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/bin/irb:23:in `load'
       21: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/irb-1.1.0/exe/irb:11:in `<top (required)>'
       20: from (irb):17
       19: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/pipe/class_interface.rb:82:in `new'
       18: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/pipe/class_interface.rb:82:in `tap'
       17: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/pipe/class_interface.rb:83:in `block in new'
       16: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/pipe/dsl.rb:35:in `call'
       15: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/compiler.rb:17:in `call'
       14: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/compiler.rb:17:in `map'
       13: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/compiler.rb:22:in `visit'
       12: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/compiler.rb:22:in `public_send'
       11: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/compiler.rb:27:in `visit_fn'
       10: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/compiler.rb:27:in `map'
        9: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/compiler.rb:27:in `block in visit_fn'
        8: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/compiler.rb:22:in `visit'
        7: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/compiler.rb:22:in `public_send'
        6: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/compiler.rb:43:in `visit_t'
        5: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/compiler.rb:17:in `call'
        4: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/compiler.rb:17:in `map'
        3: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/compiler.rb:22:in `visit'
        2: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/compiler.rb:22:in `public_send'
        1: from /Users/dzmitry/.rvm/gems/ruby-2.6.3/gems/dry-transformer-0.1.1/lib/dry/transformer/compiler.rb:29:in `visit_fn'
NoMethodError (undefined method `contain?' for Dry::Transformer::Registry:Module)
Did you mean?  constants

Next release drops Ruby 2.7?

According to the readme and the gemspec the next release of this gem appears to target Ruby >= 3, however, the next version of ROM, at version 6 will target Ruby >= 2.7, unless that has changed. Was bumping past 2.7 a mistake?

If the next release drops ruby 2.7, will it be a major version bump to 2, following the spirit of SemVer?

Composite#call should take a variable number of arguments

Take this example:

# example.rb

require "dry/transformer"

module F
  extend Dry::Transformer::Registry

  def self.constantly(n)
    n
  end

  def self.square(n)
    n ** 2
  end
end

def F(*args)
  F[*args]
end

f = F(:constantly, 5) >> F(:square)
f.call

❯ ruby example.rb
Traceback (most recent call last):
	1: from example.rb:20:in `<main>'
/Users/me/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/dry-transformer-0.1.1/lib/dry/transformer/composite.rb:32:in `call': wrong number of arguments (given 0, expected 1) (ArgumentError)

Since all arguments have been applied, f.call should not require any arguments. However, Composite#call always requires one argument: https://github.com/dry-rb/dry-transformer/blob/master/lib/dry/transformer/composite.rb#L32 A simple fix would just to make Composite#call take a variable number of arguments, so that the first function in the chain can be called with as many arguments as it needs.