Multipurpose concurrency primitive for asyncio
coroutines.
This throttle is configured with two related, but different parameters:
Throttle(capacity: float, concurrency: int = 0, period: float = 1.0)
capacity
- Sets the rate limit for requests, as capacity
per period
seconds.
concurrency
- The number of jobs that can be executing at a given time.
Usually, servers will set policies on both of these dimensions, and will suspend clients that violate either of them.
The pause
method will lock the throttle for the given number of seconds.
For example, if an API bans your client from accessing resources due to violating their rate-limit, you can tell your code to sleep for a period of time and try again later.
The throttle can be a drop-in replacement for another primitive like asyncio.Semaphore
.
In fact, it's really just an asyncio.Semaphore
(which handles the concurrency
limit) mixed with a token bucket to provide rate limiting.
throttle = Throttle(10, 2) # Two concurrent coros limited to 10qps (total).
# Perform one task when the budget allows
async with throttle:
# Do some (async) thing
# For tasks that should consume more of the budget, you can call the throttle:
async with throttle(5):
# Do some more expensive thing, equivalent to 5 requests
Like other asyncio
primitives, Throttle
is not thread-safe.