datamapper/dm-serializer

DataMapper plugin for serializing Resources and Collections

dm-serializer's Issues

Failing specs for to_yaml on 1.9.2-p290

Attempted running specs and noticed #to_yaml is having issues with Psyck on 1.9.2-p290.

TypeError in 'DataMapper::Serializer#to_yaml serializes an array of extended objects'
wrong argument type Symbol (expected String)
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:21:in `scalar'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:21:in `visit_Psych_Nodes_Scalar'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/visitor.rb:6:in `accept'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:32:in `block in visit_Psych_Nodes_Mapping'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:32:in `each'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:32:in `visit_Psych_Nodes_Mapping'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/visitor.rb:7:in `accept'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:26:in `block in visit_Psych_Nodes_Sequence'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:26:in `each'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:26:in `visit_Psych_Nodes_Sequence'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/visitor.rb:8:in `accept'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:16:in `block in visit_Psych_Nodes_Document'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:16:in `each'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:16:in `visit_Psych_Nodes_Document'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/visitor.rb:10:in `accept'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:10:in `block in visit_Psych_Nodes_Stream'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:10:in `each'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:10:in `visit_Psych_Nodes_Stream'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/visitor.rb:11:in `accept'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/nodes/node.rb:36:in `to_yaml'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych.rb:166:in `dump'
/home/hal/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/core_ext.rb:13:in `psych_to_yaml'
/vault/1/code/forks/dm-serializer/spec/spec_helper.rb:17:in `test'
spec/lib/serialization_method_shared_spec.rb:23:in `block (2 levels) in '

ValidationErrors#to_json doesn't get declared if requirements are done in wrong order (e.g. via bundler)

Title is pretty much explanatory. ActiveSupport overrides #to_json, leading to wrong json generated.

http://pastie.org/private/1mgbqwfskokwcs4htkccba

Notice how dm-validations is included after dm-serialize. Bundler doesn't know that dm-validations has some great stuff to be added from dm-serialize and thus can put their requires in any order.

I see some ways to fix this:

1. Add #as_json method to ValidationErrors. It should be there anyways. This way, AS will still override #to_json, but will also properly encode your errors
2. Ask yehuda to use some stable topological sort (I don't know if there is such a thing at all) thus making it possible to manually manage the requirement order
3. Move #to_json / #as_json code to dm-validations instead of dm-serialization -- possibly causes the same problem but reversed

---

Created by mark - 2010-09-11 06:59:36 UTC

Original Lighthouse ticket: http://datamapper.lighthouseapp.com/projects/20609/tickets/1409

Update json dependency

The latest release of this gem depends on the 1.x series of the json gem. That means that dm-serializer cannot be used on ruby 2.4, because support for ruby 2.4 was added in the 2.0 release of json. Please update the dependency so that it would be possible to use this gem on ruby 2.4.

extracting multiple multiple levels

Perhaps not an issue, but a question for something I can't get to work

4 models, each associated to the one above
List
Categories
Items
Item choices

Given aa is a List

bb = aa.to_json(:relationships=>{:categories=>{:include=>[:name],:methods=>[:items]}}) works

but trying to go deeper to Item choices, I've been getting nothing.

This is the closest I've gotten without error, but doesn't actually pull up itemchoices

aa.to_json(:relationships=>{:categories=>{:include=>[:name],:methods=>[:items], :relationships=>{:multiplechoices=>{:include=>[:itemchoices]}}}})

I'm guessing I need to fuddle around a bit more, and wondering if there is some limit or barrier to getting multiple levels, supposing I need to go even more complex, another six levels as an example

Deserialization support

It appears that dm-serializer can convert objects to xml/json/csv but not convert xml/json/csv back into objects. Often any time a program would want to serialize information into a database, it would at one point want to retrieve that same information. Why is dm-serializer part of a database project if it only accomplishes half of what a database does?

It seems more to be a simple serialization utility that can be used for a multitude of purposes... but not quite appropriate for database usage if it can't be used to retrieve information and only to store it. ActiveRecord has proper deserialization, so why can't DataMapper?

Overriding to_json in a class causes associated objects to over-quote and over-escape

Any suggestions for a clearer title for this ticket would be very much appreciated ;)

If you override to_json in a class Foo, and class Bar has n foos, calling @bar.foos.to_json in an instance of Bar will emit bad JSON which has too many quotes and backslash escapes.

I created a toy project that demonstrates this at http://github.com/xxx/dm10-association_to_json .

Maybe there's just a different way to do this now.

---

Created by mpd - 2009-10-03 19:06:22 UTC

Original Lighthouse ticket: http://datamapper.lighthouseapp.com/projects/20609/tickets/1075

YAML.load of a to_yaml Datamapper object returns a hash

Shouldn't it return a Datamapper object instead? This is causing issues with DelayedJob

[dm-serializers] better support for deep object tree

the to_xml can pass in two options :collection_element_name or :element_name whether it is a collection or a single resource. but with this it is very difficult to just change the "storage name". this gist illustrate it: http://gist.github.com/241055

so I think the better way to do define the element name would be using the storage name of resource. the default naming convention for table names are perfect XML element names !!

the patch also includes an issue when I have something like this:

class Child
include DataMapper::Resource

belongs_to :mother, :model => Child
belongs_to :father, :model => Child
end

then child.to_xml(:methods => [:mother, :father]) should produce following xml

currently I get

the last issue I came across was that a property value was frozen and that produces stacktrace in one of the xml serializiers.

so I just want to share my current fork od dm-serializer and if the patches are going to be included then I can add a few more specs for the changes.

---

Created by kristian - 2009-11-23 13:16:18 UTC

Original Lighthouse ticket: http://datamapper.lighthouseapp.com/projects/20609/tickets/1132

dm-serializer redefines the global CSV constant on Ruby 1.8

dm-serializer/to_csv.rb redefines the global CSV constant to be an alias to FasterCSV when running under Ruby 1.8. While I understand the desire to keep the remainder of the code the same under 1.8 and 1.9, this causes two problems:

• If the built-in CSV library has already been required when dm-serializer is loaded, you get a constant redefinition warning.
• If the built-in CSV library is required after dm-serializer (e.g., by another library) you get a superclass mismatch for class Row error.

An alternative: define a method on DataMapper::Serializer that returns the appropriate module to use (CSV or FasterCSV) depending on the ruby version.

Add further options to :exclude

Hi!
It'd be awesome if :exclude were a hash, just like the relationships one but for excluding data from the join
Thanks!

json_pure version requirement conflicts with gemcutter version

Is json_pure 1.2.x strictly required, or can the test in the gemspec be changed to => 1.2.0?

I would like to deploy a datamapper-based Sinatra app to Heroku and so do not want to mess with the gemspec file for dm-serializer.

tporter@eclipse /TROPO/sin_rest[master*]$ rackup config.ru
/home/tporter/.rvm/rubies/ruby-1.8.7-p249/lib/ruby/site_ruby/1.8/rubygems.rb:230:in `activate': can't activate json_pure (> 1.2.0, runtime) for ["dm-serializer-0.10.2"], already activated json_pure-1.4.3 for "gemcutter-0.5.0"

A removed test that should be reviewed.

Ping @dkubb @postmodern @mbj @solnic: ac078f2

I removed it so that Travis would pass but it was one of the tests that was sending to_json to test. Yajl doesn't support to_json without require "yajl/json_gem"... which multi_json doesn't support, plus that test seems like it's testing the output of another lib, either way I only removed it for Travis but left it there so it could be revered easily.

Allow properties to be converted back to their underlying field names

I'm working on the dm-rest-adapter code, which uses dm-serializer. Currently that code does not correctly honour :field options on properties, since it just calls #to_xml on the resource, so I'll have to post-process the result.

Perhaps a :raw => true option, or some such would be useful?

@user.to_json
# =>
{
  "id" : 42,
  "full_name" : "Testy McTesty"
  "date_of_birth" : "1980-10-01"
}

@user.to_json(:raw => true)
# =>
{
  "user_id" : 42,
  "fullname" : "Testy McTesty",
  "dob" : "1980-10-01"
}

cant figure out how to install everything to contribute

Hello,

I wanted to try out an idea for default associations, but I can't figure out how to get everything set up to the point where the existing test suite passes.

I forked the repo, and then ran:

$ ADAPTER=mysql bundle install
$ rake spec

And got:

/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-1.3.2/lib/spec/runner/options.rb:188: Use RbConfig instead of obsolete and deprecated Config.
Profiling enabled.
...F.FF..***................*...FFFFFFFFFFFFFF.FF*FFF.FFFFFFFFFFFFFFFF.FF*FFF.FFFFFFFFFFFFFFFF.FF*FFF.FFF.......*******.....*.

With errors such as the following:

NoMethodError in 'DataMapper::Serializer#to_json should integrate with dm-validations'
undefined method `violations' for #<DataMapper::Validations::OrderedHash:0x000001021def18>
/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/gems/1.9.1/gems/dm-validations-1.2.0/lib/dm-validations/validation_errors.rb:116:in `method_missing'
/Users/ajostrow/Documents/code/ruby-programs/dm-serializer/lib/dm-serializer/to_json.rb:72:in `to_json'
/Users/ajostrow/Documents/code/ruby-programs/dm-serializer/spec/spec_helper.rb:17:in `test'
spec/lib/serialization_method_shared_spec.rb:284:in `block (2 levels) in <top (required)>'
/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-1.3.2/lib/spec/example/example_methods.rb:40:in `instance_eval'
/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-1.3.2/lib/spec/example/example_methods.rb:40:in `block in execute'
/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/1.9.1/timeout.rb:54:in `timeout'
/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-1.3.2/lib/spec/example/example_methods.rb:37:in `execute'
/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-1.3.2/lib/spec/example/example_group_methods.rb:219:in `block in run_examples'
/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-1.3.2/lib/spec/example/example_group_methods.rb:217:in `each'
/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-1.3.2/lib/spec/example/example_group_methods.rb:217:in `run_examples'
/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-1.3.2/lib/spec/example/example_group_methods.rb:103:in `run'
/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-1.3.2/lib/spec/runner/example_group_runner.rb:23:in `block in run'
/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-1.3.2/lib/spec/runner/example_group_runner.rb:22:in `each'
/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-1.3.2/lib/spec/runner/example_group_runner.rb:22:in `run'
/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-1.3.2/lib/spec/runner/options.rb:153:in `run_examples'
/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-1.3.2/lib/spec/runner/command_line.rb:9:in `run'
/usr/local/Cellar/ruby/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-1.3.2/bin/spec:5:in `<main>'

I have a feeling things didn't get installed or included properly. Any suggestions? Would really love the opportunity to try and contribute.

Best, AJ

new gem version

just wonder when the gem version can be updated so that the one line fix in to_jason.rb line #19 can be in:
options = {} if options.nil?

Thanks

Loosen multi_json dependency to ~> 1.0

multi_json 1.2.0 has been released. Is there a reason to remain using the 1.0.x version family of multi_json?

Default Serialization options for Collection

It's possible to override the default serialization for a single record by overriding the to_json method, however, it would be nice to be able to apply the same global overrides to a collection of said resource

Can't install json_pure

Hello,

I can't seem to use dm-serializer because I can't install json_pure. I'm not sure what the issue is, when I execute "gem install json_pure" it simply installs the json gem. When trying to load dm-serializer it gives me the following error:

$ sudo gem install json_pure
Building native extensions.  This could take a while...
Successfully installed json-1.4.3
1 gem installed
Installing ri documentation for json-1.4.3...
Installing RDoc documentation for json-1.4.3...

$ irb
>> require 'dm-core'
=> true
>> require 'dm-serializer'
Gem::LoadError: Could not find RubyGem json_pure (~> 1.4.3)

    from /Library/Ruby/Site/1.8/rubygems.rb:777:in `report_activate_error'
    from /Library/Ruby/Site/1.8/rubygems.rb:211:in `activate'
    from /Library/Ruby/Site/1.8/rubygems.rb:246:in `activate'
    from /Library/Ruby/Site/1.8/rubygems.rb:245:in `each'
    from /Library/Ruby/Site/1.8/rubygems.rb:245:in `activate'
    from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:35:in `require'
    from (irb):2
>>

Any ideas?

To_json(:to_json => false) doesn't serialize complex types

When calling to_json(:to_json=>false), I was expecting a hash with primitive types. However, properties and methods are pushed into the hash as is, w/o to_json serialization. To get the true json hash, I'd have to serialize to json and deserialize the data back:

reloaded_json = JSON.load(JSON.dump( json_representation ))

Here's a spec that captures this behavior: https://gist.github.com/777189 .

---

Created by Alexander Sorokin - 2011-01-13 00:52:08 UTC

Original Lighthouse ticket: http://datamapper.lighthouseapp.com/projects/20609/tickets/1474

NoMethodError: undefined method `encode_json' for Sun, 28 Nov 2010 00:00:00 -0500:DateTime

I have a model defined as follows:

`

class Match
include DataMapper::Resource

storage_names[:default] = 'matches'

property :id, Serial, :field => 'matches_id', :required => true

def self.find_matches
repository.adapter.select <<-SQL
select
m.matches_id,
m.begin_date,
m.windows_time_zone,
team1_id,
team2_id,
t1.name as team1,
t2.name as team2,
m.status,
COALESCE(md.name,'') as match_day_name,
s.name as season_name,
r.name as round_name,
c.name as competition_name,
COALESCE(g.name,'') as group_name,
COALESCE(m.use_gsm,'N') as use_gsm,
COALESCE(m.team1_finalscore,0) as team1_score,
COALESCE(m.team2_finalscore,0) as team2_score
from matches m
inner join round r on (r.round_id = m.round_id)
inner join season s on (s.season_id = r.season_id)
inner join competition c on (c.competition_id = s.competition_id)
inner join team t1 on (t1.team_id = m.team1_id)
inner join team t2 on (t2.team_id = m.team2_id)
left outer join match_day md on (m.match_day_id = md.match_day_id)
left outer join groups g on (r.round_id = g.round_id)
where m.begin_date <= (CURDATE() + interval 1 day)
and m.begin_date >= (CURDATE() - interval 1 day)
order by m.begin_date desc
SQL
end
end

`

Calling Match.find_matches from the console returns an array of
structs like the following:

#<struct matches_id=54235, begin_date=Sun, 28 Nov 2010 00:00:00 -0500, windows_time_zone="Morocco Standard Time", team1_id=937, team2_id=943, team1="Deportes Tolima", team2="La Equidad", status="N", match_day_name="Cuadrangular Semifinal Fecha 3", season_name="Finalizacion 2010", round_name="Cuadrangular Semifinal", competition_name="Liga Postobon - Colombia", group_name="", use_gsm="N", team1_score=0, team2_score=0>

It appears that to_json is choking on the begin_date format. Is there
anything I can do to correct this? I'm using MySQL 5.1 if that helps.

properties_to_serialize doesn't use the query's projected fields

Datamapper allows to restrict the fields to fetch from database via the :fields option, for example :

MyModel.all(:fields => [:id, :name])

However, when serializing such a "restricted" collection, all the fields are serialized, causing new SQL queries to be sent to the database backend.

I find this counter-intuitive :-)

As a quick test, I tried to replace a line in lib/dm-serializer/common.rb :

--- lib/dm-serializer/common.rb.ori 2012-06-11 14:04:32.201075846 +1100
+++ lib/dm-serializer/common.rb 2012-06-11 14:08:03.326296260 +1100
@@ -12,7 +12,7 @@
       only_properties     = Array(options[:only])
       excluded_properties = Array(options[:exclude])

-      model.properties(repository.name).reject do |p|
+      query.fields.reject do |p|
         if only_properties.include? p.name
           false
         else

... and it seems to work for me, both with projected fields and without.

But I'm not sure every DataMapper::Collection has a query.

Would you consider the original behaviour (serialize all model fields) to be expected, or a bug?

Add to_json and as_json to Models

With the move to multi_json to_json and as_json will become unavailable on Models by default, I think it would be a good idea to add those to the base Model so that there is at least backwards compatibility there.

NoMethodError: undefined method `add_domain_type' for Psych:Module

Getting this error since upgrading to ruby 2.2.1p85 (2015-02-26 revision 49769) [x86_64-linux]

! Unable to load application: NoMethodError: undefined method `add_domain_type' for Psych:Module
/home/asd/.gem/ruby/2.2.0/gems/dm-serializer-1.2.2/lib/dm-serializer/to_yaml.rb:15:in `included': undefined method `add_domain_type' for Psych:Module (NoMethodError)
        from /usr/lib/ruby/gems/2.2.0/gems/dm-core-1.2.1/lib/dm-core/model.rb:219:in `include'
        from /usr/lib/ruby/gems/2.2.0/gems/dm-core-1.2.1/lib/dm-core/model.rb:219:in `block in extended'
        from /usr/lib/ruby/gems/2.2.0/gems/dm-core-1.2.1/lib/dm-core/model.rb:219:in `each'
        from /usr/lib/ruby/gems/2.2.0/gems/dm-core-1.2.1/lib/dm-core/model.rb:219:in `extended'
        from /usr/lib/ruby/gems/2.2.0/gems/dm-core-1.2.1/lib/dm-core/resource.rb:71:in `extend'
        from /usr/lib/ruby/gems/2.2.0/gems/dm-core-1.2.1/lib/dm-core/resource.rb:71:in `included'
        from /home/asd/.gem/ruby/2.2.0/gems/dm-noisy-failures-0.2.3/lib/data_mapper/noisy_failures.rb:31:in `call'
        from /home/asd/.gem/ruby/2.2.0/gems/dm-noisy-failures-0.2.3/lib/data_mapper/noisy_failures.rb:31:in `included'

When dm-noisy-failures disabled:

! Unable to load application: NoMethodError: undefined method `add_domain_type' for Psych:Module
/home/asd/.gem/ruby/2.2.0/gems/dm-serializer-1.2.2/lib/dm-serializer/to_yaml.rb:15:in `included': undefined method `add_domain_type' for Psych:Module (NoMethodError)
        from /usr/lib/ruby/gems/2.2.0/gems/dm-core-1.2.1/lib/dm-core/model.rb:219:in `include'
        from /usr/lib/ruby/gems/2.2.0/gems/dm-core-1.2.1/lib/dm-core/model.rb:219:in `block in extended'
        from /usr/lib/ruby/gems/2.2.0/gems/dm-core-1.2.1/lib/dm-core/model.rb:219:in `each'
        from /usr/lib/ruby/gems/2.2.0/gems/dm-core-1.2.1/lib/dm-core/model.rb:219:in `extended'
        from /usr/lib/ruby/gems/2.2.0/gems/dm-core-1.2.1/lib/dm-core/resource.rb:71:in `extend'
        from /usr/lib/ruby/gems/2.2.0/gems/dm-core-1.2.1/lib/dm-core/resource.rb:71:in `included'