artsy / garner Goto Github PK

View Code? Open in Web Editor NEW

345.0 73.0 25.0 480 KB

A set of Rack middleware and cache helpers that implement various caching strategies.

License: MIT License

Ruby 100.00%

garner's Issues

Document how to set up Rspec for testing

Please add documentation detailing what's needed to be able to run tests with Garner. Thanks!

Stale record retrieved via garnered_find

Trying to track a cache invalidation issue. A garnered_find of a record returns a stale result and no saving/invalidating of the original is clearing anything.

gravity:production> Profile.garnered_find('andrea-meislin-gallery')
fetch: {:strategy=>Garner::Strategies::Binding::Key::BindingIndex, :proxied_binding=>"Garner::Mixins::Mongoid::Identity/klass=Profile,handle=andrea-meislin-gallery"}
fetch: {:strategy=>Garner::Strategies::Binding::Key::BindingIndex, :proxied_binding=>"Profile/id=51120ae9d0c2eb4545004a16"}
fetch: {:binding_keys=>["831181d39281d48a71f76e43"], :context_keys=>{:garnered_find_args=>["andrea-meislin-gallery"]}}
=> #<Profile _id: 54188b287261692d7acf0200, created_at: 2014-09-16 19:10:32 UTC, updated_at: 2014-10-13 16:13:25 UTC, handle: "andrea-meislin-gallery", _slugs: ["andrea-meislin-gallery"], published: true, published_at: 2014-10-13 15:57:55 UTC, published_changed_at: 2014-10-13 15:57:55 UTC, locked_at: nil, locked_until: nil, short_description: nil, bio: "Andrea Meislin Gallery focuses on internationally recognized artists whose work contributes to the diasporic-based dialogue domestically and abroad.", website: "", location: "", default_icon_version: :square, subtype: nil, menu_color_class: nil, private: false, follows_count: 0, owner_type: "PartnerGallery", owner_id: BSON::ObjectId('54188b287261692d7acd0200')>

The proxy record is pointing to a stale record, which has the wrong ID 51120ae9d0c2eb4545004a16 (this profile belongs to a user), while the real record has an ID of 54188b287261692d7acf0200 (this profile belongs to a gallery). At some point the profile belonged to the user, and then it had to be renamed (handle changed), then the partner's profile was renamed to the same handle. The repro code however didn't exhibit this problem, this spec passed:

profile = Fabricate(:profile)
partner = Fabricate(:partner, given_name: 'gallery')
expect(partner.profile.handle).to_not eq 'gallery'
profile.update_attributes!(handle: 'not-gallery')
partner.profile.update_attributes!(handle: 'gallery')
expect(Profile.garnered_find('gallery').id).to eq partner.profile.id

How did we get here?

A save of the profile that is supposed to invalidate the above:

gravity:production> profile = Profile.find('andrea-meislin-gallery')
=> #<Profile _id: 54188b287261692d7acf0200, created_at: 2014-09-16 19:10:32 UTC, updated_at: 2014-10-14 19:02:38 UTC, handle: "andrea-meislin-gallery", _slugs: ["andrea-meislin-gallery"], published: true, published_at: 2014-10-13 15:57:55 UTC, published_changed_at: 2014-10-13 15:57:55 UTC, locked_at: nil, locked_until: nil, short_description: nil, bio: "Andrea Meislin Gallery focuses on internationally recognized artists whose work contributes to the diasporic-based dialogue domestically and abroad.", website: "", location: "", default_icon_version: :square, subtype: nil, menu_color_class: nil, private: false, follows_count: 0, owner_type: "PartnerGallery", owner_id: BSON::ObjectId('54188b287261692d7acd0200')>
gravity:production> profile.save!
write: {:strategy=>Garner::Strategies::Binding::Key::BindingIndex, :proxied_binding=>"Profile/id=54188b287261692d7acf0200"} => String
write: {:strategy=>Garner::Strategies::Binding::Key::BindingIndex, :proxied_binding=>"Profile/id=54188b287261692d7acf0200"} => String
write: {:strategy=>Garner::Strategies::Binding::Key::BindingIndex, :proxied_binding=>"Profile"} => String
write: {:strategy=>Garner::Strategies::Binding::Key::BindingIndex, :proxied_binding=>"Profile"} => String

Had to manually clear it:

Garner.config.cache.delete(:binding_keys=>["831181d39281d48a71f76e43"], :context_keys=>{:garnered_find_args=>["andrea-meislin-gallery"]})

This could be a total red herring. In

garner/lib/garner/cache.rb

Line 16 in 1292b59

Garner.config.cache.delete(compound_key) unless result

result = Garner.config.cache.fetch(compound_key, options_hash) do
    yield
end
Garner.config.cache.delete(compound_key) unless result

Note the fetch here is by compound key with options, but the delete is without the options. Was this intentional?

Does Garner support binding ActiveRecord models?

Just getting started with Garner and I am having trouble binding to ActiveRecord models. I debugged down into the code and it seems that if you bind an ActiveRecord model class such as garner.bind(Patient) ... end, by default the SafeCacheKey strategy will be used, but since the model class wont respond to :cache_key or :updated_at, we return from the apply method of the strategy with no binding_keys which effectively skips the cache.

Am I doing something wrong? I tried figure out what to do with ActiveRecord from the Mongoid samples.

Any help would be greatly appreciated, also sorry if this is not the correct place for questions, I looked for a Google Group and didn't find one.

Can't get it to work with Grape

Any attempt to cache anything (even without bindings and options) end up like this:

15:10:05 web.1    | 2014-06-13 15:10:05 -0700: Rack app error: #<NoMethodError: undefined method `realpath' for nil:NilClass>
15:10:05 web.1    | /home/vagrant/.rvm/gems/ruby-2.1.0/gems/garner-0.4.5/lib/garner/strategies/context/key/caller.rb:21:in `default_root'
15:10:05 web.1    | /home/vagrant/.rvm/gems/ruby-2.1.0/gems/garner-0.4.5/lib/garner/config.rb:99:in `default_caller_root'
15:10:05 web.1    | /home/vagrant/.rvm/gems/ruby-2.1.0/gems/garner-0.4.5/lib/garner/config.rb:106:in `caller_root'
15:10:05 web.1    | /home/vagrant/.rvm/gems/ruby-2.1.0/gems/garner-0.4.5/lib/garner/strategies/context/key/caller.rb:49:in `block in apply'
15:10:05 web.1    | /home/vagrant/.rvm/gems/ruby-2.1.0/gems/garner-0.4.5/lib/garner/strategies/context/key/caller.rb:43:in `each'
15:10:05 web.1    | /home/vagrant/.rvm/gems/ruby-2.1.0/gems/garner-0.4.5/lib/garner/strategies/context/key/caller.rb:43:in `apply'
15:10:05 web.1    | /home/vagrant/.rvm/gems/ruby-2.1.0/gems/garner-0.4.5/lib/garner/mixins/rack.rb:24:in `block in garner'
15:10:05 web.1    | /home/vagrant/.rvm/gems/ruby-2.1.0/gems/garner-0.4.5/lib/garner/mixins/rack.rb:23:in `each'
15:10:05 web.1    | /home/vagrant/.rvm/gems/ruby-2.1.0/gems/garner-0.4.5/lib/garner/mixins/rack.rb:23:in `garner'
15:10:05 web.1    | /vagrant/flipagram-rb-music/app/api/admin/category.rb:44:in `block (2 levels) in <class:Categories>'

Grape is standlone app without rails.

Disambiguate bind(Model, id) and bind(Model, { id: id })

We make this mistake all the time.

# correct
cache_or_304(bind: [Fair, { id: params[:id] }])

vs.

# incorrect
cache_or_304(bind: [Fair, params[:id]])

Fix Mongoid4 dependency

Garner works fine with Mongoid4, fix the Gemfile dependency.

`garnered_find` breaks on case-insensitive `find` with `BindingIndex` strategy

We have overridden Mongoid::Document#find as follows:

class Monger
  class << self
    alias_method :mongoid_find, :find
    def find(*args)
      if args.count == 1 && args[0].is_a?(String)
        mongoid_find(args[0].downcase)
      else
        mongoid(find(*args))
      end
    end
  end
end

Now, when we call garnered_find on this class, we end up returning invalid values all over the damn place:

Garner.configure do |config|
  config.binding_key_strategy = Garner::Strategies::Binding::Key::BindingIndex
  config.binding_invalidation_strategy = Garner::Strategies::Binding::Invalidation::BindingIndex
end

Monger.create!({ :name => "M1" })
# => #<Monger _id: 51d71b6a4baef8e63d000001, created_at: 2013-07-05 19:15:54 UTC, updated_at: 2013-07-05 19:15:54 UTC, name: "M1", _slugs: ["m1"], subdocument: nil>
Monger.garnered_find("M1")
# => #<Monger _id: 51d71b6a4baef8e63d000001, created_at: 2013-07-05 19:15:54 UTC, updated_at: 2013-07-05 19:15:54 UTC, name: "M1", _slugs: ["m1"], subdocument: nil>
Monger.garnered_find("M44")
# => #<Monger _id: 51d71b6a4baef8e63d000001, created_at: 2013-07-05 19:15:54 UTC, updated_at: 2013-07-05 19:15:54 UTC, name: "M1", _slugs: ["m1"], subdocument: nil>
Monger.garnered_find(nil)
# => #<Monger _id: 51d71b6a4baef8e63d000001, created_at: 2013-07-05 19:15:54 UTC, updated_at: 2013-07-05 19:15:54 UTC, name: "M1", _slugs: ["m1"], subdocument: nil>

Thanks to @dblock for reporting this.

SafeCacheKey + Touch doesn't pass spec/integration/mongoid_spec.rb

SafeCacheKey + Touch doesn't pass spec/integration/mongoid_spec.rb.

  1) Mongoid integration using Garner::Strategies::Binding::Key::SafeCacheKey with Garner::Strategies::Binding::Invalidation::Touch end-to-end caching and invalidation binding at the instance level binding via find does not invalidate results for other like-classed objects
     Failure/Error: cached_object_namer.call.should eq 'M1'

       expected: "M1"
            got: "M2"

       (compared using ==)
     # ./spec/integration/mongoid_spec.rb:176:in `block (8 levels) in <top (required)>'
     # ./spec/support/active_record.rb:32:in `block (3 levels) in <top (required)>'
     # ./spec/support/active_record.rb:31:in `block (2 levels) in <top (required)>'

  2) Mongoid integration using Garner::Strategies::Binding::Key::SafeCacheKey with Garner::Strategies::Binding::Invalidation::Touch end-to-end caching and invalidation binding at the instance level binding via find with an embedded document with :invalidate_mongoid_root = true invalidates the root document
     Failure/Error: root_cached_object_namer.call.should eq 'Sockeye'

       expected: "Sockeye"
            got: "Trout"

       (compared using ==)
     # ./spec/integration/mongoid_spec.rb:269:in `block (10 levels) in <top (required)>'
     # ./spec/support/active_record.rb:32:in `block (3 levels) in <top (required)>'
     # ./spec/support/active_record.rb:31:in `block (2 levels) in <top (required)>'

  3) Mongoid integration using Garner::Strategies::Binding::Key::SafeCacheKey with Garner::Strategies::Binding::Invalidation::Touch end-to-end caching and invalidation binding at the instance level binding via identify does not invalidate results for other like-classed objects
     Failure/Error: cached_object_namer.call.should eq 'M1'

       expected: "M1"
            got: "M2"

       (compared using ==)
     # ./spec/integration/mongoid_spec.rb:176:in `block (8 levels) in <top (required)>'
     # ./spec/support/active_record.rb:32:in `block (3 levels) in <top (required)>'
     # ./spec/support/active_record.rb:31:in `block (2 levels) in <top (required)>'

  4) Mongoid integration using Garner::Strategies::Binding::Key::SafeCacheKey with Garner::Strategies::Binding::Invalidation::Touch end-to-end caching and invalidation binding at the instance level binding via identify with an embedded document with :invalidate_mongoid_root = true invalidates the root document
     Failure/Error: root_cached_object_namer.call.should eq 'Sockeye'

       expected: "Sockeye"
            got: "Trout"

       (compared using ==)
     # ./spec/integration/mongoid_spec.rb:269:in `block (10 levels) in <top (required)>'
     # ./spec/support/active_record.rb:32:in `block (3 levels) in <top (required)>'
     # ./spec/support/active_record.rb:31:in `block (2 levels) in <top (required)>'

Add automatic support for Grape entities

I'm trying to cache the output generated from a custom entity:

get '/products/:id' do
  cache do
    product = Product.find params[:id]
    present product, with: Entities::Product
  end
end

But this fails with:

no _dump_data is defined for class StringIO

I guess the stream somehow needs to be converted to a String first?

Mongoid mixin should offer a `garnered_find` class method with automatically appropriate key and binding

module Mongoid
  module Document
    module ClassMethods
      def garnered_find(id, options = {})
        Garner::Cache::ObjectIdentity.cache({ bind: [self, id] }, { caller: nil }) do
          self.find(id)
        end
      end
    end
  end
end

The above implementation comes close, but doesn't properly key off parent relations. Specifically, something like Model.garnered_find("id") works, but something like supermodel.models.garnered_find("id") won't work (it will return the same thing regardless of the supermodel's identity).

support for Sequel

I can see that there is support for ActiveRecord, but what about Sequel, has anyone maintaining this gem given any though to it?

Garner/Grape Entity issue

Hey guys!
I'm having an issue using Grape with cache (Garner) and send the results with grape-entity.

The issue is: the second time I request an object Garner start working, but the call for Grape Entity have a problem, because it sets the options as nil, making grape-entity to throw an error, options is not define.

Versions
Grape 0.6.1
Grape-entity 0.4.0
Garner 0.3.2

Remove cache_as from Garner::Mixins::Mongoid::Document

Instead of requiring client to specify cache_as, always use the class's base collection class. Same for binding or invalidating.

Different application root locations result in different cache keys

The Caller context key strategy employs the calling code's full file path in generating a cache key. Our environment is run from heterogeneous servers at different file paths, so we discovered that one environment would continue to fetch stale cache values after an invalidation occurred in a different environment.

The operative lines are here:

garner/lib/garner/strategies/context/key/caller.rb

Line 21 in 1dd40d9

ruby_context.send(:caller).each do |line|

Example of the different results:

caller[0].split(':')[0]  # => "/app/app/api/api.rb" on Heroku
caller[0].split(':')[0]  # => "/srv/www/myapp/releases/20130612030626/app/api/api.rb" on OpsWorks

Mongoid document proxy binding doesn't work with aliased updated_at field

The current implementation of Garner::Mixins::Mongoid::Document::_latest_by_updated_at currently checks for the existence of the updated_at field. This doesn't cover documents that alias the updated_at field (typically by including include Mongoid::Timestamps::Short). This should consider aliased fields. The latest document query can also be optimized to return a single document. Something along these lines:

def self._latest_by_updated_at
  # Only find the latest if we can order by :updated_at
  return nil unless fields['updated_at'] || aliased_fields['updated_at'] 
  only(:_id, :_type, :updated_at).order_by(updated_at: :desc).limit(1).first
end

May serve to document the proxied binding behavior on model classes in the readme, so devs know to create indexes on the updated_at field if they so choose.

Cheers!

Mongoid garnered_find causes two similar queries

Consider Garner::Mixins::Mongoid::Document.garnered_find.

def self.garnered_find(handle)
  return nil unless (binding = identify(handle))
  identity = Garner::Cache::Identity.new
  identity.bind(binding).key({ :source => :garnered_find }) do
    find(handle)
  end
end

The identity.bind(binding) call will cause a database=garner_test collection=mongers selector={"$or"=>[{"_id"=>"m1"}, {"_slugs"=>"m1"}]} flags=[:slave_ok] limit=1 skip=0 batch_size=nil fields={"_id"=>1, "_type"=>1, "updated_at"=>1} query to generate the cache key for this instance.

Then, if we have a cache miss, the second query is placed: database=garner_test collection=mongers selector={"_slugs"=>{"$in"=>["m1"]}} flags=[:slave_ok] limit=1 skip=0 batch_size=nil fields=nil

In this particular scenario it may be more efficient to fetch the entire record in the first query and reuse it as the result, but I am not sure how to express this in code, therefore avoiding the double find.

Garner::Strategies::Keys::Caller fails when running inside IRB

As an example, we have a method in our codebase

def find_cached_by_slug(slug)
  Garner::Cache::ObjectIdentity.cache(bind: [self, slug]) do
    self.find_by_slug(slug)
  end
end

When calling Model.find_cached_by_slug("slug") from inside an IRB session, we get the following exception:

>> Model.find_cached_by_slug("slug”)
Errno::ENOENT: No such file or directory - /Users/macreery/code/artsy/project/(irb)

"TypeError: singleton can't be dumped" error on Heroku

Everything is working fine in development, but on Heroku, I sometimes get TypeError: singleton can't be dumped. Any idea why?

An example is when visiting a specific location: /api/locations/521d32be1974fcdb2b000103

Here's the Grape+Garner code for that endpoint:

resource "locations" do
  desc "Get the details for a specific location"
  get ':id' do
    garner.options(expires_in: 5.minutes) do
      location = Location.find(params[:id])
      location = present location, with: Entities::Location
      location.as_json
    end
  end
end

Here's the stack trace from New Relic:

…undle/ruby/2.0.0/gems/garner-0.4.4/lib/garner/cache.rb:  12:in `fetch'
…y/2.0.0/gems/garner-0.4.4/lib/garner/cache/identity.rb:  22:in `fetch'
…y/2.0.0/gems/garner-0.4.4/lib/garner/cache/identity.rb:  54:in `options'
                                  /app/app/api/ohana.rb:  41:in `block (2 levels) in <class:API>'
…ndle/ruby/2.0.0/gems/grape-0.5.0/lib/grape/endpoint.rb:  31:in `call'
…ndle/ruby/2.0.0/gems/grape-0.5.0/lib/grape/endpoint.rb:  31:in `block in generate_api_method'
…ndle/ruby/2.0.0/gems/grape-0.5.0/lib/grape/endpoint.rb: 400:in `call'
…ndle/ruby/2.0.0/gems/grape-0.5.0/lib/grape/endpoint.rb: 400:in `run'
…ndle/ruby/2.0.0/gems/grape-0.5.0/lib/grape/endpoint.rb: 164:in `block in call!'
…by/2.0.0/gems/grape-0.5.0/lib/grape/middleware/base.rb:  24:in `call'
…by/2.0.0/gems/grape-0.5.0/lib/grape/middleware/base.rb:  24:in `call!'
…by/2.0.0/gems/grape-0.5.0/lib/grape/middleware/base.rb:  18:in `call'
…by/2.0.0/gems/grape-0.5.0/lib/grape/middleware/base.rb:  24:in `call!'
…by/2.0.0/gems/grape-0.5.0/lib/grape/middleware/base.rb:  18:in `call'
…dor/bundle/ruby/2.0.0/gems/rack-1.4.5/lib/rack/etag.rb:  23:in `call'
…/ruby/2.0.0/gems/rack-1.4.5/lib/rack/conditionalget.rb:  25:in `call'
…y/2.0.0/gems/grape-0.5.0/lib/grape/middleware/error.rb:  26:in `block in call!'
…y/2.0.0/gems/grape-0.5.0/lib/grape/middleware/error.rb:  25:in `catch'
…y/2.0.0/gems/grape-0.5.0/lib/grape/middleware/error.rb:  25:in `call!'
…by/2.0.0/gems/grape-0.5.0/lib/grape/middleware/base.rb:  18:in `call'
…/newrelic-grape-1.3.1/lib/newrelic-grape/instrument.rb:  19:in `block in call!'
…/newrelic-grape-1.3.1/lib/newrelic-grape/instrument.rb:  18:in `call!'
…by/2.0.0/gems/grape-0.5.0/lib/grape/middleware/base.rb:  18:in `call'
…dor/bundle/ruby/2.0.0/gems/rack-1.4.5/lib/rack/head.rb:   9:in `call'
…/bundle/ruby/2.0.0/gems/rack-1.4.5/lib/rack/builder.rb: 134:in `call'
…ndle/ruby/2.0.0/gems/grape-0.5.0/lib/grape/endpoint.rb: 165:in `call!'
…ndle/ruby/2.0.0/gems/grape-0.5.0/lib/grape/endpoint.rb: 155:in `call'
….0.0/gems/rack-mount-0.8.3/lib/rack/mount/route_set.rb: 152:in `block in call'
…ems/rack-mount-0.8.3/lib/rack/mount/code_generation.rb:  96:in `block in recognize'
…ems/rack-mount-0.8.3/lib/rack/mount/code_generation.rb:  82:in `optimized_each'
…ems/rack-mount-0.8.3/lib/rack/mount/code_generation.rb:  95:in `recognize'
….0.0/gems/rack-mount-0.8.3/lib/rack/mount/route_set.rb: 141:in `call'
…or/bundle/ruby/2.0.0/gems/grape-0.5.0/lib/grape/api.rb: 480:in `call'
…or/bundle/ruby/2.0.0/gems/grape-0.5.0/lib/grape/api.rb:  49:in `call!'
…or/bundle/ruby/2.0.0/gems/grape-0.5.0/lib/grape/api.rb:  45:in `call'
…le/ruby/2.0.0/gems/journey-1.0.4/lib/journey/router.rb:  68:in `block in call'
…le/ruby/2.0.0/gems/journey-1.0.4/lib/journey/router.rb:  56:in `each'
…le/ruby/2.0.0/gems/journey-1.0.4/lib/journey/router.rb:  56:in `call'
…undle/ruby/2.0.0/gems/rack-pjax-0.7.0/lib/rack/pjax.rb:  12:in `call'
…goid-3.1.4/lib/rack/mongoid/middleware/identity_map.rb:  34:in `block in call'
…y/2.0.0/gems/mongoid-3.1.4/lib/mongoid/unit_of_work.rb:  39:in `unit_of_work'
…goid-3.1.4/lib/rack/mongoid/middleware/identity_map.rb:  34:in `call'
…undle/ruby/2.0.0/gems/rack-cors-0.2.8/lib/rack/cors.rb:  54:in `call'
…dle/ruby/2.0.0/gems/warden-1.2.3/lib/warden/manager.rb:  35:in `block in call'
…dle/ruby/2.0.0/gems/warden-1.2.3/lib/warden/manager.rb:  34:in `catch'
…dle/ruby/2.0.0/gems/warden-1.2.3/lib/warden/manager.rb:  34:in `call'
…dor/bundle/ruby/2.0.0/gems/rack-1.4.5/lib/rack/etag.rb:  23:in `call'
…/ruby/2.0.0/gems/rack-1.4.5/lib/rack/conditionalget.rb:  25:in `call'
…0.0/gems/remotipart-1.2.1/lib/remotipart/middleware.rb:  27:in `call'
…/2.0.0/gems/rack-1.4.5/lib/rack/session/abstract/id.rb: 210:in `context'
…/2.0.0/gems/rack-1.4.5/lib/rack/session/abstract/id.rb: 205:in `call'
…/ruby/2.0.0/gems/rack-1.4.5/lib/rack/methodoverride.rb:  21:in `call'
…/bundle/ruby/2.0.0/gems/rack-1.4.5/lib/rack/runtime.rb:  17:in `call'
                               /app/lib/api_defender.rb:  47:in `call'
…dor/bundle/ruby/2.0.0/gems/rack-1.4.5/lib/rack/lock.rb:  15:in `call'
…by/2.0.0/gems/rack-cache-1.2/lib/rack/cache/context.rb: 136:in `forward'
…by/2.0.0/gems/rack-cache-1.2/lib/rack/cache/context.rb: 245:in `fetch'
…by/2.0.0/gems/rack-cache-1.2/lib/rack/cache/context.rb: 185:in `lookup'
…by/2.0.0/gems/rack-cache-1.2/lib/rack/cache/context.rb:  66:in `call!'
…by/2.0.0/gems/rack-cache-1.2/lib/rack/cache/context.rb:  51:in `call'
…ruby/2.0.0/gems/rack-timeout-0.0.4/lib/rack/timeout.rb:  16:in `block in call'
       /app/vendor/ruby-2.0.0/lib/ruby/2.0.0/timeout.rb:  66:in `timeout'
…ruby/2.0.0/gems/rack-timeout-0.0.4/lib/rack/timeout.rb:  16:in `call'
…by/2.0.0/gems/unicorn-4.6.3/lib/unicorn/http_server.rb: 552:in `process_client'
…by/2.0.0/gems/unicorn-4.6.3/lib/unicorn/http_server.rb: 632:in `worker_loop'
…by/2.0.0/gems/unicorn-4.6.3/lib/unicorn/http_server.rb: 500:in `spawn_missing_workers'
…by/2.0.0/gems/unicorn-4.6.3/lib/unicorn/http_server.rb: 142:in `start'

Garner should not write new cache entries upon invalidation

This can be more costly than deleting, depending on the cache interface, in cases where the object is not already cached.

BindingIndex and SafeCacheKey key strategies both incorrectly return non-nil cache keys for Model.identify(nil)

Garner::Strategies::Binding::Key::BindingIndex.apply(Model.identify(nil))
# => "2986056a81ed9ec670d172f8"
Garner::Strategies::Binding::Key::SafeCacheKey.apply(Model.identify(nil))
# => "models/516738f913d2c34741000002-20130423234925.0000000000"

Thanks @joeyAghion for catching this.

Q: Logic when both HTTP_IF_MODIFIED_SINCE and HTTP_IF_NONE_MATCH are sent by client

How does Garner return response if it received both HTTP_IF_MODIFIED_SINCE and HTTP_IF_NONE_MATCH? Is there a place in code you could point out to check?

Thanks!

Garner.cache loses namespace options when deleting a record as block yields nil

See #80.

garner/lib/garner/cache.rb

Line 16 in 1292b59

Garner.config.cache.delete(compound_key) unless result

result = Garner.config.cache.fetch(compound_key, options_hash) do
    yield
end
Garner.config.cache.delete(compound_key) unless result

The fetch here is by compound key with options, but the delete is without the options.

File cache store is limited by filename length constraint.

The following error came up in development with long URL cache keys:

File name too long - /Users/gib/Code/artsy/gravity/gib/tmp/cache/proxied_binding%3DGarner%3A%3AMixins%3A%3AMongoid%3A%3AIdentity%2Fklass%3DArtwork%2Chandle%3Dkammy-roulner-when-confronted-with-a-piece-of-art-dot-dot-dot%2Fstrategy%3DGarner%3A%3AStrategies%3A%3ABinding%3A%3AKey%3A%3ABindingIndex20130704-19291-s7dqiv.lock

Thanks for the speedy response and suggestions @fancyremarker!

Mongo garnered_find not respecting chained criteria

#68 with a description and code.

Undefined method [] for Pathname(/path/to/my/app) (problem with Caller key strategy)

I seem to be getting an issue with my caller key strategy. I actually don't want to use caller, so I tried to override it like this:

Garner.configure do |config|
  config.rack_context_key_strategies = [
    Garner::Strategies::Context::Key::RequestGet,
    Garner::Strategies::Context::Key::RequestPost,
    Garner::Strategies::Context::Key::RequestPath
  ]
end

But that failed with no method error on rack_context_key_strategies

Has anyone successfully used a non-caller key strategy? I just want to use this with Grape so I am interested in caching based on the request params and my custom binding.

Support arrays in Garner::Cache::ObjectIdentity::IDENTITY_FIELDS

Notably mongoid-slug now has slugs, which is an array of IDs, all of which must be invalidated.

Default expires_in override global_cache_options

In README

:global_cache_options: A hash of options to be passed on every call to Garner.config.cache, like { :expires_in => 10.minutes }. Defaults to {}

But In

garner/lib/garner/cache/identity.rb

Line 15 in bbe95a4

@options_hash.merge!(expires_in: Garner.config.expires_in)

@options_hash = Garner.config.global_cache_options || {}
@options_hash.merge!(expires_in: Garner.config.expires_in)

The default expires_in is nil, it will override expires_in in global_cache_options.
So possible changes:

Update README to use another key name and add expires_in to full list of Garner.config attributes.

@options_hash[:expires_in] ||= Garner.config.expires_in

Run tests on JRuby

Right now bundle install complains that posix-spawn is required, which needs a C extension.

Add a binding key strategy that lists all compound cache keys associated with a binding

I'm going to credit @joeyAghion with requesting this feature. Some thoughts on implementation to follow.

garnered_find slower than find?

I encountered this surprise in our own application, and was able to reproduce it from a checkout of garner with the minimal models used by specs:

$ GARNER_MEMCACHE_SERVER=localhost bundle exec irb -I lib/
irb(main):001:0> require 'rspec'
=> true
irb(main):002:0> require './spec/support/cache'
=> true
irb(main):003:0> require './spec/support/mongoid'
=> true
irb(main):004:0> Monger.delete_all
=> 100
irb(main):005:0> (1..100).each { |i| Monger.create!(name: "monger ##{i}") }
=> 1..100
irb(main):006:0> ids = Monger.pluck(:_id); ids.size
=> 100
irb(main):007:0> n = 100
=> 100
irb(main):008:0> Benchmark.bm(15) do |x|
irb(main):009:1*   x.report('find') { n.times { ids.each{|id| Monger.find(id) } } }
irb(main):010:1>   x.report('garnered_find') { n.times { ids.each{|id| Monger.garnered_find(id) } } }
irb(main):011:1> end
                      user     system      total        real
find              4.210000   0.440000   4.650000 (  5.451388)
garnered_find     8.770000   0.910000   9.680000 ( 11.849314)

It's possible that this set-up isn't configuring the cache optimally, but I'd expect this to be sufficient to see cache hits and the library's benefits. What am I doing wrong?

garner fails if mongoid not loaded yet

$ bundle exec rspec

/Users/joey/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/ext/module.rb:36:in `const_missing': uninitialized constant Garner::Mixins::Mongoid (NameError)
    from /Users/joey/dev/artsy/gravity/config/initializers/mongoid_document.rb:4:in `<module:Document>'
    from /Users/joey/dev/artsy/gravity/config/initializers/mongoid_document.rb:2:in `<module:Mongoid>'
    from /Users/joey/dev/artsy/gravity/config/initializers/mongoid_document.rb:1:in `<top (required)>'
    from /Users/joey/.rvm/gems/ruby-1.9.2-p290@artsy/gems/railties-3.2.6/lib/rails/engine.rb:588:in `block (2 levels) in <class:Engine>'
    from /Users/joey/.rvm/gems/ruby-1.9.2-p290@artsy/gems/railties-3.2.6/lib/rails/engine.rb:587:in `each'
    from /Users/joey/.rvm/gems/ruby-1.9.2-p290@artsy/gems/railties-3.2.6/lib/rails/engine.rb:587:in `block in <class:Engine>'
    from /Users/joey/.rvm/gems/ruby-1.9.2-p290@artsy/gems/railties-3.2.6/lib/rails/initializable.rb:30:in `instance_exec'
    from /Users/joey/.rvm/gems/ruby-1.9.2-p290@artsy/gems/railties-3.2.6/lib/rails/initializable.rb:30:in `run'
    from /Users/joey/.rvm/gems/ruby-1.9.2-p290@artsy/gems/railties-3.2.6/lib/rails/initializable.rb:55:in `block in run_initializers'
    from /Users/joey/.rvm/gems/ruby-1.9.2-p290@artsy/gems/railties-3.2.6/lib/rails/initializable.rb:54:in `each'
    from /Users/joey/.rvm/gems/ruby-1.9.2-p290@artsy/gems/railties-3.2.6/lib/rails/initializable.rb:54:in `run_initializers'
    from /Users/joey/.rvm/gems/ruby-1.9.2-p290@artsy/gems/railties-3.2.6/lib/rails/application.rb:136:in `initialize!'
    from /Users/joey/dev/artsy/gravity/config/application.rb:46:in `initialize!'
    from /Users/joey/.rvm/gems/ruby-1.9.2-p290@artsy/gems/railties-3.2.6/lib/rails/railtie/configurable.rb:30:in `method_missing'
    from /Users/joey/dev/artsy/gravity/config/environment.rb:48:in `<top (required)>'
    from /Users/joey/.rvm/gems/ruby-1.9.2-p290@artsy/gems/railties-3.2.6/lib/rails/application.rb:103:in `require_environment!'
    from /Users/joey/.rvm/gems/ruby-1.9.2-p290@artsy/gems/railties-3.2.6/lib/rails/commands.rb:40:in `<top (required)>'
    from script/rails:6:in `require'
    from script/rails:6:in `<main>

Release Next

@fancyremarker: Should this be a 0.4.6 or a 0.5.0? There're new features, sounds like a 0.5.0. Feel free to cut a release, I am happy to as well if you don't want to.

Caching POST request and responses

Does Garner support caching post requests? Sorry, I didn't know where else to post this question. Thanks!

Using non-ORM bindings

I'm using Garner in a project (really, a couple projects) where we don't ever need to bind to ActiveRecord or Mongoid objects. Instead, we want to cache certain data (the roles to which a user belongs) and bind it to the OAuth token they present.

Right now, I'm doing this by monkey-patching the AccessToken class in an initializer:

Garner.configure do |config|
  # Other configuration options...
  config.binding_key_strategy = Garner::Strategies::Binding::Key::CacheKey
end

module Fridge
  class AccessToken
    include Garner::Cache::Binding

    def cache_key
      id
    end
  end
end

Then I can do stuff like:

garner.bind(token) { roles.map(&:href) }

What do we think is the appropriate way to "Garner-ify" non-ORM objects? Is re-opening the class the right way to do this? Right now, it's necessary to, at a minimum include Garner::Cache::Binding in any class whose instances will be used as bindings. Is there a better way?

How to cache only anonymous(non-login) request?

I'm using grape for api and devise for authentication.
I want to cache api response only if user-request hasn't login info.( Non-login status ). Then, how to do it?

logged-in request serves normal response without caching, don't expires cache data(don't affect with caching).
not-logged-in request serves response with caching

Inconsistency between Mongoid find and Garner::Mixins::Mongoid::Document.garnered_find

Considering Garner::Mixins::Mongoid::Document.garnered_find I believe the API's for the two 'find' methods should be identical, here is a repro of an example that will cause different results between the two.

Here is the garnered_find method for reference:

def self.garnered_find(handle)
  return nil unless (binding = identify(handle))
  identity = Garner::Cache::Identity.new
  identity.bind(binding).key({ :source => :garnered_find }) do
    find(handle)
  end
end

Assuming an emptied cache:

Artist.garnered_find(['andy-warhol'])  
# returns array of one artist (correct)

Artist.garnered_find('andy-warhol')  
# returns array of one artist, rather than the object itself (incorrect)

Or (assuming an empty cache)

Artist.garnered_find('andy-warhol')  
# returns artist object (correct)

Artist.garnered_find(['andy-warhol'])  
# returns artist object, rather than an array (incorrect)

Contrast with find:

Artist.find(['andy-warhol'])   
# returns array of one artist (correct)

Artist.find('andy-warhol')   
# returns artist object (correct)

Caller context key strategy totally borks if Garner::Cache::Context#garner is monkey-patched

Let's say I do the following

# config/initializers/garner.rb

module Garner
  module Mixins
    module Rack

      alias_method :_garner, :garner
      def garner(&block)
        response, headers = _garner.key({ foo: "bar" }, &block)
      end
    end
  end
end

Now the calling line number will always be config/initializers/garner.rb:9, instead of where the method was originally called from.

Can we be smarter about this?

Where to include Garner::Mixins::Rack ?

Hi,

I'd like to try this gem out, but I can't get it to recognize the mixin in my Rails app. It would be great if the documentation were more specific as to where the include needs to go. Here is my setup:

app/api/api.rb:

module API
  class Root < Grape::API
    prefix 'api'
    format :json
    version 'v1', using: :header, vendor: 'ohanapi'

    mount Ohana::API
    Grape::Endpoint.send :include, LinkHeader
    add_swagger_documentation markdown: true, hide_documentation_path: true,
      hide_format: true, api_version: 'v1'
  end
end

app/api/ohana.rb:

module Ohana
  class API < Grape::API

  resource "/" do
...

I tried including it in both files, and also in the Grape::Endpoint.send line, but I keep getting uninitialized constant Garner::Mixins. What am I doing wrong?

Support ActiveRecord 4

ActiveRecord 4 cache_key is not in nsec format.

Cache keys are now "activists/1-20140717140709836493000". It used to be in the :number format, but now it's in the :nsec format. SafeCacheKey shortcuts apply because that is not in a valid format per SafeCacheKey::VALID_FORMAT.

Find a better way to rewrite how the Mongoid::Identity mixin yields conditions

We set Garner.config.mongoid_identity_fields = [:_id, :_slugs]. In our case, these are two separate ways to find an instance of a Model that are mutually exclusive and follow distinct formats.

Garner's Garner::Mixins::Mongoid::Identity will combine these into an $or. Here're two possible queries:

{"$or"=>[{"_id"=>"jason-bryant-hope-slash-hate"}, {"_slugs"=>"jason-bryant-hope-slash-hate"}]}

{"$or"=>[{"_id"=>"51ca3bec8b3b8132c60002ea"}, {"_slugs"=>"51ca3bec8b3b8132c60002ea"}]}

This is not very efficient and we can make the decision about whether the query is done by slug or by id upfront, so we monkey patch this as follows.

module Garner
  module Mixins
    module Mongoid
      class Identity
        private

        def self.conditions_for(klass, handle)
          # optimize away the query from an $or to a find where appropriate
          # in the case of Gravity _id and _slugs can be used independently
          # and we can never have a non BSON _id and valid BSON IDs are redundant with _id inside _slugs
          conditions = Moped::BSON::ObjectId.legal?(handle) ? { _id: handle } : { _slugs: handle }

          # _type conditions
          selector = klass.where({})
          conditions.merge!(selector.send(:type_selection)) if selector.send(:type_selectable?)

          conditions
        end
      end
    end
  end
end

This is very much custom. But maybe there's a better way to describe this in Garner without writing a completely different mixin?

Caching the Index method

Hi!
In my Orders controller, there's an index method that returns all orders for a specific account.
Is there a way to cache calls like current_account.orders (get all orders that the current_account has), invalidating only when the number of orders for current_account changes?
Thanks!

Does Garner work with Roar?

Hi,
Me again 😄
I'm using Roar as my representer. I have 3 models: Organization, Location, and Services.
Here's my LocationRepresenter, which also includes some properties from the Organization and Service models:

require 'roar/representer/json'
require 'roar/representer/feature/hypermedia'

module LocationRepresenter
  include Roar::Representer::JSON
  include Roar::Representer::Feature::Hypermedia

  property :id
  # more location properties

  property :organization, extend: OrganizationRepresenter, class: Organization

  collection :services, :extend => ServiceRepresenter, :class => Service
end

Here's my Grape API location resource without Garner, which works fine:

...
get do
  locations = Location.includes([:organization, :services]).page(params[:page]).per(params[:per_page])
  locations.extend LocationsRepresenter
end

If I add Garner like this:

get do
  garner.options(expires_in: 15.minutes) do
    locations = Location.includes([:organization, :services]).page(params[:page]).per(params[:per_page])
    locations.extend LocationsRepresenter
  end
end

On the first request, I get the full representation. On the garnered request, I get the non-represented version of locations which is missing the organization and services stuff. I also tried garner.bind(Location) and garner.bind(Location).bind(Organization).bind(Service) and I'm still not getting the full representation back from garner.

If I try the same thing on an individual location:

get ':id' do
  garner.options(expires_in: 15.minutes) do
    location = Location.includes([:organization, :services]).find(params[:id])
    location.extend LocationRepresenter
  end
end

I get can't dump anonymous class #<Module:0x007f875130b3f0>.

I get the same dump error with the bind method:

get ':id' do
  garner.bind(Location) do
    location = Location.includes([:organization, :services]).find(params[:id])
    location.extend LocationRepresenter
  end
end

Any help would be greatly appreciated! Thanks!

Add support for caching both body and headers in Garner::Mixins::Rack

We have monkey patched the Rack mixin to allow caching both headers and body, like this:

module Garner
  module Cache
    class Identity
      TRACKED_HEADERS = %w{ X-Total-Count }

      alias_method :_fetch, :fetch
      def fetch(&block)
        if @key_hash[:tracked_grape_headers]
          result, headers = _fetch do
            result = yield
            headers = @ruby_context.header.slice(*TRACKED_HEADERS) if @ruby_context.respond_to?(:header)
            [result, headers]
          end

          (headers || {}).each do |key, value|
            @ruby_context.header(key, value) if @ruby_context.respond_to?(:header)
          end

          result
        else
          _fetch(&block)
        end
      end

    end
  end
end

require "garner/mixins/rack"

module Garner
  module Mixins
    module Rack

      alias_method :_garner, :garner
      def garner(&block)
        response, headers = _garner.key({ tracked_grape_headers: true }, &block)
      end

    end
  end
end

It should be possible to roll this functionality into Garner. The Identity part should probably become more generic to store data and metadata, while the Rack mixin would just reuse that functionality.

Add cache logging support

Create a feature around cache logging. We want to see what Garner is reading or writing to/from cache with a single switch. Currently this monkey-patch:

module StoreEx
  def read(key, options = nil)
    puts "read: #{key}"
    super
  end

  def read_multi(*names)
    puts "read_multi: #{names}"
    super
  end

  def write(key, value, options = nil)
    puts "write: #{key} => #{value.class}"
    super
  end

  def fetch(name, options = nil)
    puts "fetch: #{name}"
    super
  end
end

Garner.config.cache.extend StoreEx

Garner::Strategies::Keys::Caller broken on Heroku

Create a more robust way to identity the caller location. On Heroku you get this:

LINE: /app/.bundle/gems/ruby/1.9.1/gems/garner-0.1.2/lib/garner/cache/object_identity.rb:94:in `block in key_context'
SET: /app/.bundle/gems/ruby/1.9.1/gems/garner-0.1.2/lib/garner/cache/object_identity.rb:94

This is because the bundle is under app. All the rest of the hackery is not great.

ActiveRecord mixin - incorrect initializer code, and dumping problems

@fancyremarker When I add the code to the initializer,

/Users/yan/dev/reverb/config/initializers/garner.rb:4:in `<module:ActiveRecord>': Base is not a module (TypeError)

Which is because Base is actually a Class. The correct initializer code is:

require "garner/mixins/active_record"

module ActiveRecord
  class Base
    include Garner::Mixins::ActiveRecord::Base
  end
end

Note the require line, and the use of class Base. I can make a pull request for that, but I'm afraid there are other issues.

But then I run into further problems when actually trying to cache that resources:
no _dump_data is defined for class Proc

/Users/yan/.rvm/gems/ruby-1.9.3-p392@reverb/gems/rspec-mocks-2.12.2/lib/rspec/mocks/extensions/marshal.rb:5:in `dump'
/Users/yan/.rvm/gems/ruby-1.9.3-p392@reverb/gems/rspec-mocks-2.12.2/lib/rspec/mocks/extensions/marshal.rb:5:in `dump_with_mocks'
/Users/yan/.rvm/gems/ruby-1.9.3-p392@reverb/gems/activesupport-3.2.12/lib/active_support/cache.rb:561:in `initialize'
/Users/yan/.rvm/gems/ruby-1.9.3-p392@reverb/gems/activesupport-3.2.12/lib/active_support/cache.rb:363:in `new'
/Users/yan/.rvm/gems/ruby-1.9.3-p392@reverb/gems/activesupport-3.2.12/lib/active_support/cache.rb:363:in `block in write'
/Users/yan/.rvm/gems/ruby-1.9.3-p392@reverb/gems/activesupport-3.2.12/lib/active_support/cache.rb:520:in `instrument'
/Users/yan/.rvm/gems/ruby-1.9.3-p392@reverb/gems/activesupport-3.2.12/lib/active_support/cache.rb:362:in `write'
/Users/yan/.rvm/gems/ruby-1.9.3-p392@reverb/gems/activesupport-3.2.12/lib/active_support/cache.rb:299:in `fetch'
/Users/yan/.rvm/gems/ruby-1.9.3-p392@reverb/bundler/gems/garner-b880202c454c/lib/garner/cache.rb:12:in `fetch'
/Users/yan/.rvm/gems/ruby-1.9.3-p392@reverb/bundler/gems/garner-b880202c454c/lib/garner/cache/identity.rb:16:in `fetch'
/Users/yan/.rvm/gems/ruby-1.9.3-p392@reverb/bundler/gems/garner-b880202c454c/lib/garner/cache/identity.rb:24:in `bind'

Abstract all Grape middleware to be (more generically) Rack middleware

This shouldn't be too challenging.

Replace multiple identity support with a canonical id and a lookup

There're many problems with supporting multiple ID fields, especially when a single objects gets cached by any of those ID fields (slug, id). We basically don't support ID changes in that case.

Implement a strategy that does a slug -> id lookup, and only caches objects by ID. Things like slug become external lookup fields that hit the cache as well. It's 2 cache reads, but it's a lot less writes.

artsy / garner Goto Github PK

garner's Issues

Recommend Projects

Recommend Topics

Recommend Org