mongomapper/plucky

Thin layer over the ruby driver that allows you to quickly grab hold of your data.

MongoMapper

A Ruby Object Mapper for Mongo.

Install

$ gem install mongo_mapper

Documentation

http://mongomapper.com/documentation/

http://rdoc.info/github/mongomapper/mongomapper

Open Commit Policy

Like Rubinius, we're trying out an "open commit policy".

If you've committed one (code) patch that has been accepted and would like to work some more on the project, send an email to Scott Taylor [email protected] along with your commit sha1.

Compatibility

MongoMapper is tested against:

• MRI 2.4 - 3.2
• JRuby (Versions with 3.1 compatibility)

Additionally, MongoMapper is tested against:

• Rails 5.0 - 5.2
• Rails 6.0 - 6.1
• Rails 7.0 - 7.1

Note, if you are using Ruby 3.0+, you'll need Rails 6.0+.

Contributing & Development

$ git clone https://github.com/mongomapper/mongomapper && cd mongomapper
$ bundle install
$ bundle exec rake

• Fork the project.
• Make your feature addition or bug fix. All specs should pass.
• Add specs for your changes. This is important so that it doesn't break in a future version.
• Commit, do not mess with Rakefile, version, or history. If you want to have your own version, that is fine but bump version in a commit by itself in another branch so a maintainer can ignore it when your pull request is merged.
• Send a pull request. Bonus points for topic branches.

How to release

See HOW_TO_RELEASE.md

Problems or Questions?

Hit up the Google group: http://groups.google.com/group/mongomapper

Copyright

Copyright (c) 2009-2023 MongoMapper. See LICENSE for details.

Contributors

MongoMapper/Plucky is:

• John Nunemaker
• Chris Heald
• Scott Taylor

But all open source projects are a team effort and could not happen without everyone who has contributed. See CONTRIBUTORS for the full list. Thank you!

plucky's Issues

plucky converts _ids to ObjectIds when it shouldn't

I've got a complex query. If I use the driver, no problem:

Product.collection.find('aggregated_substances.endpoints' => {"$elemMatch" => {'_id' => 'env_chronic_aquatic_tox_score', 'assessment.key' => 'h'}}).count
=> 12

Plucky barfs on the same expression:

Product.where('aggregated_substances.endpoints' => {"$elemMatch" => {'_id' => 'env_chronic_aquatic_tox_score', 'assessment.key' => 'h'}}).count
BSON::InvalidObjectId: illegal ObjectId format
from /Library/Ruby/Gems/1.8/gems/bson-1.1.5/lib/../lib/bson/types/object_id.rb:1 22:in from_string' from /Library/Ruby/Gems/1.8/gems/plucky-0.3.6/lib/plucky.rb:23:into_object_id'
from /Library/Ruby/Gems/1.8/gems/plucky-0.3.6/lib/plucky/criteria_hash.rb:127:in `normalized_value'
....

But plucky is happy if I give it an ObjectId:

Product.where('aggregated_substances.endpoints' => {"$elemMatch" => {'_id' => BSON::ObjectId.new, 'assessment.key' => 'h'}}).count
=> 0

Plucky is happy with 'something._id' => 'a_string', but barfs when I use '_id' in a hash expression as above.

I should add that Plucky is happy if I pass in a String with a valid
BSON::ObjectId format:

Product.where('aggregated_substances.endpoints' => {"$elemMatch" =>
{'_id' => '4d4c7b610de1fa73af000005', 'assessment.key' => 'h'}}).count
=> 0

To be explicit, since Mongodb does not require ObjectIds for its _ids,
Plucky should not automatically try to transform any _id into an
ObjectId.

Askhat check commit 'deep fetch'

Check it, debug and if everything works great => try it.
What I've done there.
deep_fetch takes two arguments => key and path to it. Also it is free to pass some additional params.
For instance
we have a hash = {1 => {2 => {3 => {4 => 5}}}}
if you would like to get value of 4 than you need:
hash.deep_fetch(4, "1.2.3", :casting => :to_i, :default => :not_a_value)

By the way, I know that rails uses hash with indifferent access and all the keys are also symbols. If that a matter, you could not provide any additional options. :casting => :to_sym => by default.
:default => nil = by default.

group operator

Hey, I can't seem to find it anywhere, and I just wanted to double-check. Does Plucky currently support Mongo's group operator anywhere?

Thanks!

Access to repo?

@cheald I can't seem to push to plucky. Can you add me as a contributor?

Any chance plucky will work with the new (2.x) mongo driver more or less as is?

or does it require major changes?

"can't modify literal regexp" when using (:foo => /bar/) in queries

But (:foo => Regexp.new(/bar/)) works. Would it be possible to support literal regexp or is it up to the user to wrap it in Regexp.new()?

MongoDB Driver Update

Any plans to upgrade the dependency of the mongodb driver to the 1.6 family?

Symbol operators don't work in $or conditions

(Courtesy Dan Morrison)

Breaks:

where :$or => [
    {:begin_on.gte => range_begin, :begin_on.lte => range_end}, # begins in the range
    {:end_on.gte => range_begin,   :end_on.lte => range_end},   # ends in the range
    {:begin_on.lt => range_begin, :end_on.gt => range_end}      # longer than range
  ]

Works:

where :$or => [
    {:begin_on => {:$gte => range_begin, :$lte => range_end}},            # begins in the range
    {:end_on =>   {:$gte => range_begin, :$lte => range_end}},            # ends in the range
    {:begin_on => {:$lt => range_begin}, :end_on => {:$gt => range_end}}  # longer than range
  ]

The top one breaks with this stack trace:

TypeError: keys must be strings or symbols
    from /Users/daniel/.rvm/gems/ruby-1.9.2-p180-patched@somewhere/gems/bson-1.3.1/lib/bson/bson_c.rb:24:in `serialize'
    from /Users/daniel/.rvm/gems/ruby-1.9.2-p180-patched@somewhere/gems/bson-1.3.1/lib/bson/bson_c.rb:24:in `serialize'
    from /Users/daniel/.rvm/gems/ruby-1.9.2-p180-patched@somewhere/gems/mongo-1.3.1/lib/mongo/cursor.rb:425:in `construct_query_message'
    from /Users/daniel/.rvm/gems/ruby-1.9.2-p180-patched@somewhere/gems/mongo-1.3.1/lib/mongo/cursor.rb:405:in `send_initial_query'
    from /Users/daniel/.rvm/gems/ruby-1.9.2-p180-patched@somewhere/gems/mongo-1.3.1/lib/mongo/cursor.rb:371:in `refresh'
    from /Users/daniel/.rvm/gems/ruby-1.9.2-p180-patched@somewhere/gems/mongo-1.3.1/lib/mongo/cursor.rb:87:in `next_document'
    from /Users/daniel/.rvm/gems/ruby-1.9.2-p180-patched@somewhere/gems/mongo-1.3.1/lib/mongo/cursor.rb:248:in `each'
    from /Users/daniel/.rvm/gems/ruby-1.9.2-p180-patched@somewhere/gems/mongo-1.3.1/lib/mongo/cursor.rb:267:in `to_a'
    from /Users/daniel/.rvm/gems/ruby-1.9.2-p180-patched@somewhere/gems/mongo-1.3.1/lib/mongo/cursor.rb:267:in `to_a'
    from /Users/daniel/.rvm/gems/ruby-1.9.2-p180-patched@somewhere/gems/plucky-0.3.8/lib/plucky/query.rb:76:in `all'
    from /Users/daniel/.rvm/gems/ruby-1.9.2-p180-patched@somewhere/gems/mongo_mapper-0.9.1/lib/mongo_mapper/plugins/identity_map.rb:38:in `all'
    from (irb):8

Order of criteria values is not preserved.

I just got bit by this issue here... http://jira.mongodb.org/browse/SERVER-1990

Unfortunately the order of parameters matters when using $near
db.collection.find( { latlng : { $near : [50,50] , $maxDistance : 5 } } )

The following mongomapper/plucky query causes errors...

locations = Location.where(
  :location_type => params[:type], :loc => {'$near' => loc, '$maxDistance' => 25}
).paginate(
  :page => params[:page] || 1, :per_page => params[:per_page] || 1
)

The error is due to the order of $near and $maxdistance
Mongo::OperationFailure: missing geo field (loc) in : { loc: { $maxDistance: 25, $near: [ 41.760067, -88.25218 ] }, location_type: "blah" }

I tried making the loc: parameter an ActiveSupport::OrderedHash but I believe the values are copied out and into a new hash by Plucky.

Support Collection#find without timeout

The Mongo driver requires Collection#find to be called with a block if the :timeout => false option is passed. Plucky currently doesn't support this, making it impossible to do a no-timeout query.

The following patch fixes this, but it does break one test, which expects find_each to return an open, rewound cursor. Fixing this would break that spec, but I think it's probably correct; find_each with a block shouldn't expect to do anything with the cursor after the block terminates in just about any circumstance I can think of.

Is this worth submitting a PR for?

diff --git a/lib/plucky/query.rb b/lib/plucky/query.rb
index 4922e1d..f03d125 100755
--- a/lib/plucky/query.rb
+++ b/lib/plucky/query.rb
@@ -68,14 +68,14 @@ module Plucky

       def find_each(opts={})
         query = clone.amend(opts)
-        cursor = query.cursor

         if block_given?
-          cursor.each { |doc| yield doc }
-          cursor.rewind!
+          query.cursor do |cursor|
+            cursor.each {|doc| yield doc }
+          end
+        else
+          query.cursor
         end
-
-        cursor
       end

       def find_one(opts={})
@@ -230,8 +230,8 @@ module Plucky
       @options.to_hash
     end

-    def cursor
-      @collection.find(criteria_hash, options_hash)
+    def cursor(&block)
+      @collection.find(criteria_hash, options_hash, &block)
     end

   private

Query#find_each should accept a block

See MongoMapper #386.

find_each only returns an enumerator instead of calling any block given to it. It's a vestige from before document transformation was put into the driver. find_each should call a block if given and return an enumerator.

Code: https://github.com/jnunemaker/plucky/blob/master/lib/plucky/query.rb#L54

$nor is treated as a key, not as a query option

When passing a $nor query, MongoMapper (up to 0.10.1) appears to incorrectly add an $in around the $nor's options array, as if it were a key.

$or:

Post.where(:$or => [{:publish_at => 1}, {:publish_at => 2}])
 => #<Plucky::Query $or: [{:publish_at=>1}, {:publish_at=>2}]...>

$nor:

Post.where(:$nor => [{:publish_at => 1}, {:publish_at => 2}])
 => #<Plucky::Query $nor: {"$in"=>[{:publish_at=>1}, {:publish_at=>2}]}...>

From brief investigation, it looks like $nor should be added to $or and $and here:

https://github.com/jnunemaker/plucky/blob/master/lib/plucky/criteria_hash.rb#L123

Thanks !
Milan Iliev

Add hint method

http://groups.google.com/group/mongomapper/browse_thread/thread/b9416cd3c9ea4e15

[FEATURE REQUEST] Make behavior on key conflict in where clause compatible with ActiveRecord

Currently, Plucky and ActiveRecord generate different queries when calling where clause method twice with the same key, i.g. where(a: 1).where(a: 2).

Plucky generates { :a => { :$in => [1, 2] } }, and it behaves like { :$or => [{ :a => 1 }, { :a => 2 }] }.

require "bundler/inline"

gemfile do
  source "https://rubygems.org"
  gem 'plucky'
end

connection = Mongo::Client.new(["127.0.0.1"], :logger => Logger.new("/dev/null"))
db = connection.use("test").database
collection = db['users']

p Plucky::Query.new(collection).where(a: 1).where(a: 2).criteria_hash
#=> {:a=>{:$in=>[1, 2]}}

On the other hand, ActiveRecord generates "users"."a" = 1 AND "users"."a" = 2, and it returns empty set.

require "bundler/inline"

gemfile do
  source "https://rubygems.org"
  gem "activerecord", "6.1.0"
  gem "sqlite3"
end

require "active_record"

# This connection will do for database-independent bug reports.
ActiveRecord::Base.establish_connection(adapter: "sqlite3", database: ":memory:")
ActiveRecord::Base.logger = Logger.new("/dev/null")

ActiveRecord::Schema.define do
  create_table :users, force: true do |t|
    t.integer :a
  end
end

class User < ActiveRecord::Base
end

puts User.where(a: 1).where(a: 2).to_sql
#=> SELECT "users".* FROM "users" WHERE "users"."a" = 1 AND "users"."a" = 2

These behaviors are opposite and can confuse us when building queries with split scoped or methods. So I request to add an option or something like that to make Plucky's behavior compatible with ActiveRecords' one.

New ruby hash syntax and SymbolOperator

Wondering if there's any way to get Plucky's SymbolOperator to work with Ruby's new hash syntax?

For example is there a way to get:

People.where :created_at.lt => Time.now

to use the form:

People.where created_at: Time.now

but somehow apply the .lt operator?

Having the two forms sprinkled throughout a codebase, depending on whether I need an operator or not, is really messy.

unset (version 0.3.6)

Either problem with

1. Me
2. MongoMapper
3. Plucky

resource.unset("disco.beans")

BSON::InvalidObjectId: illegal ObjectId format

its trying to make an ObjectId from "disco.beans"

sort(:order.asc) generates sort(["sort", 1])

Key normalization makes it impossible to sort on a field called order, since it gets translated to sort. I'm looking into a fix.

Ordering on id field not working

Hey guys,

Maybe I'm missing something here but 'asc' and 'desc' should at least return something different when ordering by id, correct?

Test case below...

connection = Mongo::MongoClient.new
db = connection.db('test')
collection = db['users']
collection.remove # clear out the collection

def generate_id_for(time)
  BSON::ObjectId.from_time(time, :unique => true)
end

collection.insert({'_id' => generate_id_for(2.hours.ago), 'name' => 'chris', 'age' => 26, 'name' => 'Chris'})
collection.insert({'_id' => generate_id_for(1.hour.ago), 'name' => 'steve', 'age' => 29, 'name' => 'Steve'})
collection.insert({'_id' => generate_id_for(Time.now), 'name' => 'john',  'age' => 28, 'name' => 'John'})

# initialize query with collection
query = Plucky::Query.new(collection)

puts ""
puts "-----------------------------"
puts 'Querying...'
puts ""
puts "Using 'id asc' seems to increase in order (correct)"
pp query.all(:order => "_id asc").map {|doc| doc['_id'].generation_time}
puts ""
puts "Using 'id desc' seems to also increase in order (incorrect)"
pp query.all(:order => "_id desc").map {|doc| doc['_id'].generation_time}
puts ""
puts "asc and desc should not give the same output above"
puts ""
puts "-----------------------------"

puts 'Same result with _id ...'
puts ""
puts "Using 'id asc' seems to increase in order (correct)"
pp query.all(:order => "_id asc").map {|doc| doc['_id'].generation_time}
puts ""
puts "Using 'id desc' seems to also increase in order (incorrect)"
pp query.all(:order => "_id desc").map {|doc| doc['_id'].generation_time}
puts ""
puts "asc and desc should not give the same output above"
puts ""
puts "-----------------------------"

puts 'Same result with fancy syntax ...'
puts ""
puts "Using 'id asc' seems to increase in order (correct)"
pp query.all(:order => :id.asc).map {|doc| doc['_id'].generation_time}
puts ""
puts "Using 'id desc' seems to also increase in order (incorrect)"
pp query.all(:order => :id.desc).map {|doc| doc['_id'].generation_time}
puts ""
puts "asc and desc should not give the same output above"
puts ""
puts "-----------------------------"

puts 'Same result with order as a method call...'
puts ""
puts "Using 'id asc' seems to increase in order (correct)"
pp query.order(:id.asc).map {|doc| doc['_id'].generation_time}
puts ""
puts "Using 'id desc' seems to also increase in order (incorrect)"
pp query.order(:id.desc).map {|doc| doc['_id'].generation_time}
puts ""
puts "asc and desc should not give the same output above"
puts ""
puts "-----------------------------"

Merging Criteria doesn't work properly with BSON::ObjectId objects

While debugging I found an issue with merging criteria. Here is the simplified version of the problem:

class Resource
  include MongoMapper::Document
end

# the query below will result in a query with conditions  {:_id => nil)
Resource.where(:id.in => [BSON::ObjectId('50ca414e76dc3d10f00008fe' ), BSON::ObjectId('50ca414e76dc3d10f00008ff') ]).where(:id => BSON::ObjectId('50ca414e76dc3d10f00008fe'))

I have tracked the issue to lib/plucky/criteria_hash.rb#60:

value[modifier_key].concat(Array(other_value)).uniq!

The issue is that calling Array() on a BSON::ObjectID does not yield the expected result:

Array(BSON::ObjectId('50ca414e76dc3d10f00008fe')) #=> [80, 202, 65, 78, 118, 220, 61, 16, 240, 0, 8, 254]

I've also noticed that you use that same Array() several places throughout the method. So this bug may be hidden throughout the code. Not sure the best way to address this problem, otherwise I would have submitted a patch.

Thanks