shopify/limiter

Simple Ruby rate limiting mechanism.

Limiter

This gem implements a simple mechanism to throttle or rate-limit operations in Ruby.

Installation

Add this line to your application's Gemfile:

gem 'ruby-limiter'

And then execute:

$ bundle

Or install it yourself as:

$ gem install ruby-limiter

Usage

Basic Usage

To rate limit calling an instance method, a mixin is provided. Simply specify the method to be limited, and the maximum rate that the method can be called. This rate is (by default) a number of requests per minute.

class Widget
  extend Limiter::Mixin

  # limit the rate we can call tick to 300 times per minute
  # when the rate has been exceeded, a call to tick will block until the rate limit would not be exceeded
  limit_method :tick, rate: 300

  ...
end

To specify the rate in terms of an interval shorter (or longer) than 1 minute, an optional interval parameter can be provided to specify the throttling period in seconds.

class Widget
  extend Limiter::Mixin

  # limit the rate we can call tick to 5 times per second
  # when the rate has been exceeded, a call to tick will block until the rate limit would not be exceeded
  # and the provided block will be executed
  limit_method(:tick, rate: 5, interval: 1) do
    puts 'Limit reached'
  end

  ...
end

Load balancing

By default all calls to the limit_method will be bursted, e.g. as quick as possible, until the rate is exceeded. Then we wait for the remainder of the interval to continue. To even out the burst, an optional balanced parameter can be provided to enable interleaving between the method calls, e.g: interleave = interval / size.

  ...
  limit_method :tick, rate: 60, balanced: true
  ...

For example: with an interval of 60 seconds and a rate of 60:

balanced: false : As quickly as possible we call the method 60 times, then we wait for the remainder of the time.

balanced: true : We interleave each call with 1 second so we call this method every second.

Resetting a rate-limited method

There are times when it may be necessary to reset the rate limiter for a method, for example during testing.

This can be done by calling reset_method_limit! on the class, where "method" is replaced with the name of the method being limited.

Given the example above, the following would reset the rate limit for the tick method during test setup:

class WidgetTest < Minitest::Test
  def setup
    Widget.reset_tick_limit!
  end

  ...
end

Advanced Usage

In cases where the mixin is not appropriate the RateQueue class can be used directly. As in the mixin examples above, the interval parameter is optional (and defaults to 1 minute). It is also possible to provide the block to RateQueue, which will be executed on each limit hit (useful for metrics).

class Widget
  def initialize
    # create a rate-limited queue which allows 10000 operations per hour
    @queue = Limiter::RateQueue.new(10000, interval: 3600) do
      puts "Hit the limit, waiting"
    end
  end

  def tick
    # this operation will block until less than 10000 shift calls have been made within the last hour
    @queue.shift
    # do something
  end
end

Resetting a RateQueue

In some circumstances it may be desirable to reset a rate queue, for example after invoking an API that resets an external rate limit.

This can be done by calling reset on the queue.

  ...
  @queue.reset
  ...

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake test to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/Shopify/limiter.

License

The gem is available as open source under the terms of the MIT License.

limiter's Issues

Mixin needs to support methods with arguments

throttling a method with arguments does not work

for example, given a method with a single argument, the method signature for the throttled override does not and will result in the error...

ArgumentError: wrong number of arguments (given 1, expected 0)

Use `Process.clock_gettime(Process::CLOCK_MONOTONIC)` instead of `Time`?

Hi,

Thank you for open sourcing this!

I recently came across a post that I think might be useful for this gem: Elapsed time with Ruby, the right way. The author writes:

To recap: system clock is constantly floating and it doesn't move only forwards. If your calculation of elapsed time is based on it, you're very likely to run into calculation errors or even outages.

Remember: wall clock is for telling time, monotonic clock is for measuring time. ⏰

This is why you probably don't want to use Time but Process.clock_gettime(Process::CLOCK_MONOTONIC) to calculate the elapsed time.

Cheers!

Limiting based on association

Hey Shopify crew!

Thanks for making such amazing gems.

Use so many of them in my products, can't tell you how much I appreciate it.

Quick q: if I'd like to limit based on an association count, what would be the best way to handle it using this library? Say, I want to limit the intake of events being created per visitor (i.e 100 events per visitor).

Thank you! ⭐

`limit_method` not working on multi threaded calls

Sample code:

class A
  extend Limiter::Mixin
  limit_method :blah, rate: 30, interval: 60
  def blah
    puts 'blah'
  end
end
a=A.new
50.times{ Thread.new{ a.blah } }

Testing

I'm not sure this belongs to issues so please move if there is a more appropriate place for questions of this type.

I'm wondering how one could test the method with the rate limit, for example, if we have limit_method :tick, rate: 300 and we want to ensure that the tick method is not called more frequently, how could that test look like? I'm guessing Timecop could be used but I haven't found solution.
I guess most users of the app have this need and explanation might have its place in README if it would be simple.

Thanks!

How to prevent rate limiting during tests? (e.g. rspec)

I am starting to use limiter in my app. LOVE IT!

However, I find that my rspec tests are getting rate limited between examples. The result is that I now have a suite of flickering tests, based on what order they run and delays between examples.

Is there a way limiter can automatically purge its cache between examples?

Failing that, is there a way to explicitly purge it, such as in a before block for relevant examples?

Question: What use cases have you used this Gem for?

Very interested to hear about when this gem has been used at Shopify and others who've used it. Limiting on the method level is very interesting to me.

Does it block the call and hang up resources until the limit has been lifted?

Rate Limit Violations | Theory

Problem Theory

I recently faced an issue related to the specified rate limit getting exceeded on certain occasions. Upon inspection of the code, I found a loophole which could theoretically make the rate limit reach up to twice the specified amount.

In my use case, I was trying to rate limit API calls in my client to a 3rd party API for which there was a rate limit imposed on server side. For the sake of simplicity within the context, let's assume a rate limit of 2 calls per second.
No matter how hard we try, we cannot record the exact time when the API call is made to the server in @ring array. The recorded time is always behind the actual time because time is recorded before forwarding the method call to super. The difference may become substantial because:

1. If i'm trying to rate limit API calls, I might have a method for facilitating the calls. There might be an extra layer of logic between the method call and the API call inside the method which would account for some extra time. Even if I'm not using a wrapper method I might be using a library to make calls, like Typhoeus or Net::HTTP, which would have a similar problem.
2. If I'm using multiple threads to make the API calls, the threads won't be truly parallel in Ruby due to shared GIL. Consequently, there might be a time gap between when the time gets recorded in @ring and when the method gets called (or when the API gets called) due to thread scheduling.

Based on above two reasonings, below is a tailor made example where the specified rate limit can get exceeded.
Rate limit of 2 calls per second.
Time recorded for first & second method call: 0.00 & 0.01 respectively. But the actual API calls happen at 0.98, 0.99.
After 1 second has elapsed, Time recorded for 3rd & 4th method call: 1.00 & 1.01. This time the actual API calls happen at 1.01, 1.02.
This results in 4 API calls happening in a duration of 0.04 seconds as opposed the expected rate limit of 2 calls per second. This could potentially make the API calls reach up to twice the specified amount in the specified duration.

This is amazing. Thank you!

Redis-backed distributed rate limiter based on this project

Hi, I just wanted to make y'all aware of redrate, a gem based almost entirely on this one that uses Redis instead of instance variables to store the ring, head, and mutex. This allows multiple running instances of the code to share a queue, even across different servers, making it possible to rate-limit access to a shared external resource.

Thanks @sbfaulkner for the interface, code, and (especially) tests that made the port possible!

How to sync rate limit among multiple servers?

Hello,

Upon reviewing the documentation, I noticed there is no mention of synchronizing rate limits across multiple servers. My question pertains to implementing rate limiting within an application running in a Docker container. Specifically, if I were to deploy 10 such containers, all making calls to the same third-party API, how would I ensure that the rate limit is consistently enforced across all these containers? Is there a recommended approach for managing and communicating rate limits among multiple Docker containers to avoid surpassing the API's rate limit constraints?

Thank you for your advice.

wrong gem name in README

Hi, the gem name limiter specified in the README is wrong, the correct one is ruby-limiter. This might cause some confusion, please fix :)