express-rate-limit/express-rate-limit

Basic rate-limiting middleware for the Express web server

express-rate-limit

Basic rate-limiting middleware for Express. Use to limit repeated requests to public APIs and/or endpoints such as password reset. Plays nice with express-slow-down and ratelimit-header-parser.

Usage

The full documentation is available on-line.

import { rateLimit } from 'express-rate-limit'

const limiter = rateLimit({
	windowMs: 15 * 60 * 1000, // 15 minutes
	limit: 100, // Limit each IP to 100 requests per `window` (here, per 15 minutes).
	standardHeaders: 'draft-7', // draft-6: `RateLimit-*` headers; draft-7: combined `RateLimit` header
	legacyHeaders: false, // Disable the `X-RateLimit-*` headers.
	// store: ... , // Redis, Memcached, etc. See below.
})

// Apply the rate limiting middleware to all requests.
app.use(limiter)

Data Stores

The rate limiter comes with a built-in memory store, and supports a variety of external data stores.

Configuration

All function options may be async. Click the name for additional info and default values.

Option	Type	Remarks
windowMs	number	How long to remember requests for, in milliseconds.
limit	number | function	How many requests to allow.
message	string | json | function	Response to return after limit is reached.
statusCode	number	HTTP status code after limit is reached (default is 429).
legacyHeaders	boolean	Enable the X-Rate-Limit header.
standardHeaders	'draft-6' | 'draft-7'	Enable the Ratelimit header.
requestPropertyName	string	Add rate limit info to the req object.
skipFailedRequests	boolean	Uncount 4xx/5xx responses.
skipSuccessfulRequests	boolean	Uncount 1xx/2xx/3xx responses.
keyGenerator	function	Identify users (defaults to IP address).
handler	function	Function to run after limit is reached (overrides message and statusCode settings, if set).
skip	function	Return true to bypass the limiter for the given request.
requestWasSuccessful	function	Used by skipFailedRequests and skipSuccessfulRequests.
validate	boolean | object	Enable or disable built-in validation checks.
store	Store	Use a custom store to share hit counts across multiple nodes.

Thank You

Sponsored by Zuplo a fully-managed API Gateway for developers. Add dynamic rate-limiting, authentication and more to any API in minutes. Learn more at zuplo.com

---

Thanks to Mintlify for hosting the documentation at express-rate-limit.mintlify.app

---

Finally, thank you to everyone who's contributed to this project in any way! 🫶

Issues and Contributing

If you encounter a bug or want to see something added/changed, please go ahead and open an issue! If you need help with something, feel free to start a discussion!

If you wish to contribute to the library, thanks! First, please read the contributing guide. Then you can pick up any issue and fix/implement it!

License

MIT © Nathan Friedly, Vedant K

express-rate-limit's Issues

`onLimitReached` is misleading

The name suggests it's triggered when the limit is reached, not for every requests that is send after the limit is reached.

Error: Not Acceptable

It appears you're only supporting a couple of res.format responses during rate-limiting events. I think this could be rectified by adding a default case that sends only the error code.

Error: Not Acceptable
Jun 21 12:37:36 staging-cloud app/web.1: at ServerResponse.res.format (/app/node_modules/express/lib/response.js:675:15)
Jun 21 12:37:36 staging-cloud app/web.1: at Object.handler (/app/node_modules/express-rate-limit/lib/express-rate-limit.js:28:15)
Jun 21 12:37:36 staging-cloud app/web.1: at /app/node_modules/express-rate-limit/lib/express-rate-limit.js:81:30
Jun 21 12:37:36 staging-cloud app/web.1: at MemoryStore.incr (/app/node_modules/express-rate-limit/lib/memory-store.js:12:7)
Jun 21 12:37:36 staging-cloud app/web.1: at rateLimit (/app/node_modules/express-rate-limit/lib/express-rate-limit.js:63:23)
Jun 21 12:37:36 staging-cloud app/web.1: at Layer.handle [as handle_request]

Date response header is malformed

The Date response header added in 598b9b4 is not in a valid format, which is causing breakage downstream for anything else that relies on the Date value in the response headers (e.g. https://github.com/pquerna/cachecontrol).

The offending code starts here:
https://github.com/nfriedly/express-rate-limit/blob/3b5fe600b3f88add876add420e6898b83dda0f3f/lib/express-rate-limit.js#L76

The Date response header should be in the following format, per the spec:

Tue, 15 Nov 1994 08:12:31 GMT

References:

https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.18
https://tools.ietf.org/html/rfc7231#section-7.1.1.1
https://tools.ietf.org/html/rfc7231#section-7.1.1.2

apply express-rate-limit to dynamic routes

I would like to know if this the expected behavior of express-rate-limiter or if i'm not using express-rate limiter correctly.

i have some routes that are dynamically generated like so:

router.post('/foo/:x/:y', limiter,callback function)

If i access /foo/example1/example2 and /foo/example3/example4 it will count for the same limiter.
Is there a way where i can have a limiter for each different dynamically generated route?
The purpose is to have 1 limiter for /foo/example1/example2, another for /foo/example3/example4 and so on, and so on.

I'm using express4

Any help would be appreciated.

Rate limit reset timestamp (X-RateLimit-Reset header)

Would anyone be interested in adding an X-RateLimit-Reset header? I'd quite like this myself, but didn't want to submit the pull request if no one was interested?

It would follow the same convention as GitHub uses, returning a unix timestamp as an integer (in seconds, not ms).

If you're ok with that, I'll submit the PR one I've got it working otherwise will just use for my own purposes

IP Address Exceptions?

It would be really useful to be able to add a specific IP address or range of IP addresses that are exempt from the API limiting. Is this a feature that currently exists? If not how much work would it be to implement it?

IP Address Issue

I have reached the max limit when I hit 5 times to my API server and it has blocked my IP to my back end server. Now If I try to login using a different ISP which will have different public IP or even from a mobile internet, I get blocked even then and get same response. Is there any misconfiguration or which IP does it actually capture?

Run function on blobk IP

I'm using this package. When an IP address has reached the limitation, the system will show an error message that I've set. But I need, when somebody has reached the limitation, a system run a function about that IP. How can I do that?

I mean I`m using "onLimitReached" but now I have a new problem. I wrote a function that when an IP has reached the limitation, the system will block this IP in the firewall. The problem is, when system found an IP that reached the limitation, it blocks some 4 or 5 IP that is sending request after bad IP.

for example, this is the capture of 5 seconds of requests:

1.1.1.1
2.2.2.2
3.3.3.3
4.4.4.4
5.5.5.5
6.6.6.6

And imagine "5.5.5.5" is a bad IP that reached the limitation. The system will block these IPs:
1.1.1.1
2.2.2.2
3.3.3.3
4.4.4.4
5.5.5.5

why??

can i use it in my project which is running in nodejs with pm2 cluster

can i use it in my project which is running in nodejs with pm2 cluster

Why is a store responsible for resetting counts?

I've implemented a custom (Postgres) store, but missed the fact that the store is apparently responsible for resetting counts. Why is it implemented this way? It seems to me that the store should provide an interface to reset counts, but that the core library should be responsible for calling that interface.

At a minimum, it would be nice if this requirement were documented. As it is, my implementation would eventually leave all users rate limited permanently.

keyGenerator

In your docs you mentioned, a default for keyGenerator is req.ip. But what are my other options? I would like to set it to a counter instead of an ipAdress. Is that possible? Thanks!

Rate limit at small windowMs

Hi, I am trying to prevent user from firing a POST API in a very short period of time, say 3000ms. This is my limit config:

const limiter = new RateLimit({
    windowMs: 3000,
    max: 1, 
    delayMs: 0 
});

And my route is something like:
app.get('/test',limiter,(req,res)=>{res.end("hello")})

When I test it by quickly firing the same API, I got the following result:

The later API call did get block, but the previous one goes through despite the setting.

onDelayReached() ?

Similar to how onLimitReached() is implemented, it would be great if there was an option such as onDelayReached() which would trigger when the request delay is started for an IP address.

How long can I store max amount in the memory for?

I have a max setup to 10 but then it expires after an hour. I am using req.ip == to 127.0.0.1 to keep track of keyGenerator.

How can I fix this so it doesn't go over max after certain period of time?

Please add a these values as new parameters if possible

1. If possible please add some thing that returns the no of attempts left. I want to use in a way like to display the no of attempts left. in the message text.

2. Need time remaining counter to display remaining time to reset after blcking.

How to limit original client IP not proxy IP

req.ip gets proxy ip instead of original clients IP
I want to limit client IP not proxy IP.

Since clients IP is sent in req.params.ip field how would I limit on that value?

Fix test failure on Node.js 10.8

Not sure what the root cause is, but the last test seems to be consistently failing on node.js 10.8.0

  1) express-rate-limit node module should allow the error statusCode to be customized:
     Error: socket hang up
      at createHangUpError (_http_client.js:313:15)
      at Socket.socketOnEnd (_http_client.js:416:23)
      at endReadableNT (_stream_readable.js:1081:12)
      at process._tickCallback (internal/process/next_tick.js:63:19)

https://travis-ci.org/nfriedly/express-rate-limit/jobs/411486427

Options request being included in rate limiting

Perhaps some built in detection for rate limiting options request? At the moment it will count it as one whole request despite it being a standard part of the CORS flow.

Block request coming from hostname?

First of all, really good project @nfriedly 🙌

I just released this project heymoji.cool. The problem I'm facing right now is because you can embed heymoji using iframe I can't just block by IP. Otherwise when I hit the limit I can't "vote" in other sites as well.

Trying to find a way where I can detect the requests coming from X and block it by hostname. So if I hit the fire emoji 100 time on a.com the limit is set, however I still can go to b.com and vote there.

Does it make sense? Do you have any suggestion?

Cheers

Remove usage of `new` from docs.

using the new operator suggests you are creating an instance of a class, this is not the case, you are instead returning a new closure, this can lead to a lot of confusion and is generally a very uncommon thing to do..

Feature request: Add skipSuccessfulRequests

Similar to skipFailedRequests, but skipping successful requests.

Would gladly do it, willing to accept a PR?

Would happily support this project as it will be used by @inalambria

if I start my project in cluster mode

if I start my project in cluster mode, is it work?

it base on the global hits

It's not work when windowMs setting to 0.001

• How to limit API access in milliseconds, thanks !

Query: After how much time a user is allowed to hit the API again after being rate limited?

After how much time a user is allowed to hit the API again after being rate limited?

Exclusions? (by IP or User Agent)

Is there any way to specify to ignore certain user-agents or ips?

After I installed this and set it for normal usage, it ended up failing for newrelic and my load tester sites and I'm looking for a way to white-list user agents or IPs to prevent load testing or new relic from being subject to being rate limited.

Deprecated call

MemJS INCREMENT: using deprecated call - arguments have changed

V3 Goals

Goals for v3:

• Move the delay options to a new module that only slows down repeat requests - call it express-rate-limit-slow-down or something like that
  • DONE - see https://github.com/nfriedly/express-slow-down
• Add X-RateLimit-Reset header - #32 - has issues with using an external store. Possible solutions:
  • Only enable the header when the store provides some particular flag/method/value indicating support
    • And then submit a (backwards-compatible) PR to known stores
• Consider removing some of the other options in order to further simplify things.
• Don't automatically switch formats depending on the request headers. Instead just use req.send(options.message)
  • express automatically stringifies objects to json and sets the appropriate application/json content-type. #53
• Make onLimitReached only fire once before an IP resets - #64, #38

Implement a reset function

Can you please add a reset(req, res, next) function for when we want to reset specific IPs.
I use your work to create this but it will be great if I was able to use your library.

Take into account that in order to implement the reset functionality you now need to somehow save the setTimeout() and call clearTimeout()

Feature request: whitelisted IPs

Allow passing custom json to `message`.

Our API has it's own error format and I'd rather not have it dictated by a library and overriding handler entirely seems a little redundant as it forces me to re-implement the headers, statusCode and message behaviour.

Ideally you would check if options.message is an object and respond with it directly.

Retry-After header always default with memory store

I noticed that if you do not have windowMs set while you are using a store that has an expiration, the Retry-After header will always be whatever the default for windowMs is.

I looked into RedisStore and it does not look like you could retrieve expiry from it in the case that windowMs is not set.

This is the code I had when I noticed this.

var limiter = new RateLimit({
  store: new RedisStore({
    expiry: 100,
    client: redisClient
  }),
  max: 100,
  delayMs: 0
});
app.use(limiter);

Clearing all the storage while using Key Generator with custom value.

I have found this after extensive testing. If I try logging from two different IP's and one of them reached the limit. Now I use second IP after some time and complete the limit for that too. Now if the first ones blocking time is completed and it clears the storage, it will also clear the records for the second IP which still has some time to regain access as well!!

Behind cloudflare

Hello,

My server runs behind a proxy ( Cloudflare ), what is the best option to use?

keyGenerator: function(req) {
return req.headers['cf-connecting-ip'] || req.ip;
}

or

app.enable('trust proxy');

Distinguish between initial and others rate-limit attempts

I need to know/distinguish between the first attempt outside the rate limit (current == options.max + 1) and the others (current > options.max + 1).

I see two options:

a) provide current and options to the handler function
b) create a new method (ex: limit) that will be called on the first attempt outside the rate limit

CC @mderazon

Block Request based on user params.

Hi @nfriedly,

Is there any way to block a request per user instead of IP. For ex if I send user id in request headers and I want to block a request on the basis of user id so that all other users connecting from the same ip should not be blocked.

Thanks,
Ranjan

Request: more options when limit is reached

Hey!

rate-limit is working 10/10, but i would like to have a bit more options when someone reaches the limit, instead of displaying a plain message maybe rendering another file or responding with an JSON obj. like

{
"429": "Too Many Requests"
}

or something like this.

~ Jerome

• can i add a label to this "issue"?

What about IP spoofing?

Hey Nathan,

A very nice module you've created. Thanks a tonne for this.

I have a question about IP spoofing. I realize it's quite difficult to spoof your IP these days, and it would be a niche attack vector to consider, but do you have any knowledge or ideas about how IP spoofing could circumvent this module?

Use option `delayMs` in 429 responses

In line 73 we can see that the handler func is being called without using the delayMs (used in line 78).

I think that the handler should also be delayed.

CC @mderazon

Permaban if limit gets hit multiple times?

What's the best way to implement this? Thanks for the library!

Why use this over nginx built-in rate limiting?

Title - just a question. Would one be better than the other in any scenario, or are they generally equivalent?

'EINVAL: invalid argument, read' in first install

When installing i got this

PS F:\project> npm install --save express-rate-limit
npm WARN deprecated [email protected]: Use uuid module instead
npm WARN deprecated [email protected]: ReDoS vulnerability parsing Set-Cookie https://nodesecurity.io/advisories/130
npm WARN [email protected] No description
npm WARN [email protected] No repository field.

npm ERR! code EINVAL
npm ERR! EINVAL: invalid argument, read

npm ERR! A complete log of this run can be found in:

What should i do ?

TypeScript support

I wrote some typings for this lib, figured I would post them here.

declare module 'express-rate-limit' {
  import { RequestHandler, Request, Response } from 'express';
  namespace limit {
    interface Store {
      /**
       * Increments the value in the underlying store for the given key.
       */
      incr(key: string, cb: (err?: Error, hits?: number) => void): void;
      /**
       * Decrements the value in the underlying store for the given key.
       * Used only and must be implemented when skipFailedRequests is true.
       */
      decrement?(key: string): void;
      /**
       * Resets a value with the given key.
       */
      resetKey(key: string): void;
    }

    interface RateLimitHandler extends RequestHandler {
      /**
       * Proxy function that calls `Store.resetKey`
       */
      resetKey(key: string): void;
    }

    interface RateLimitOptions {
      /**
       * how long to keep records of requests in memory.
       * Is not used for actually checking anything if `store` is set.
       * Will be used when `headers` is enabled.
       */
      windowMs?: number;
      /**
       * how many requests to allow through before starting to delay responses
       */
      delayAfter?: number;
      /**
       * base delay applied to the response - multiplied by number of recent hits for the same key.
       */
      delayMs?: number;
      /**
       * max number of recent connections during `window` milliseconds before sending a 429 response
       */
      max?: number;
      /**
       * Message to be send on ratelimit hit.
       * Ignored if `handler` is set.
       * If request is `json`, the message will be send as `{ message: MSGOBJECT }`
       * If request is `html`, the message will not be altered.
       * Default:'Too many requests, please try again later.',
       */
      message?: string | object;
      /**
       * Status code to be send on ratelimit hit.
       * Ignored if `handler` is set.
       * Defaults to 429 (RFC)
       */
      statusCode?: number;
      /**
       * Send custom rate limit header with limit and remaining.
       * Ignored if `handler` is set.
       * Defaults to true.
       */
      headers?: boolean;
      /**
       * Do not count failed requests (status >= 400).
       * Store must implement `decrement` method for this to work.
       * Defaults to false.
       */
      skipFailedRequests?: boolean;
      /**
       * allows to create custom keys.
       * Defaults to request ip.
       */
      keyGenerator?(req: Request, res: Response): string;
      /**
       * allows to skip certain requests.
       * Defaults to skipping none.
       */
      skip?(req: Request, res: Response): boolean;
      /**
       * allows to skip certain requests.
       * Defaults to skipping none.
       */
      skip?(req: Request, res: Response): boolean;
      /**
       * Can be overriden to implement custom response behaviour.
       * See other options to see what this does.
       */
      handler?: RequestHandler;
      /**
       * Called each time the limit is reached.
       * You can use it to debug/log.
       * Defaults to doing nothing.
       */
      onLimitReached?(req: Request, res: Response, opts: RateLimitOptions): void;

      /**
       * Store to use for ratelimiting.
       * Defaults to in memory.
       */
      store?: Store;
    }
  }
  function limit(options?: limit.RateLimitOptions): limit.RateLimitHandler;

  export = limit;
}

Remove and ignore package-lock.json

The file is ignored when publishing to npm, it's useless to keep commiting, diffing, etc. It should be ignored.

How to show the remaining timeout?

I'm using custom HTML that shows the user the number of attempts and the waiting period but would like to show the time remaining until the user can actually use the route again.

const getFile = filename => {
    return fs.readFileSync('./public/' + filename + '.html', 'utf-8')
}

const findAndReplace = (rule, replacement, model) => {
    model = model.replace(/{{+[a-zA-Z0-9_]+=+[a-zA-Z0-9_=:.\/@#&-]+}}/gi, wholeMatch => {
        if ( wholeMatch ) {
            wholeMatch = wholeMatch.replace(/{{/g, '').replace(/}}/g, '')
            let index = wholeMatch.split('=')
            //
            if ( Array.isArray(rule) ) {
                for (let i = 0; i < rule.length; i++) {
                     switch (index[0]) {
                         case rule[i]:
                             return replacement[i]
                         break
                     }
                }
            } else {
                switch (index[0]) {
                    case rule:
                        return replacement
                    break
                }
            }
        } else {
            return ''
        }
    })
    return model
}
const limiter = new RateLimit({
    windowMs: 30 * 60 * 1000,
    max: 3,
    delayMs: 0,
    message: findAndReplace(
        ['limit', 'expire', 'docs'],
        [3, 30, 'https://domain.com/documentation#limiter',
        getFile('limiter')
    )
})

// express route
route.get('/awesomeroute', limiter, (req, res, next) => {
    // do stuff....
})

The output would look something like this:

How could I get the remaining time to show the user instead of using the total time?

Could this be used with Express 4.0's Router?

thanks!

503 Error after about 6 hours uptime on heroku

This is only happening to my endpoints using this module. Here is my config:

 var limiter = new RateLimit({
      store: new RedisStore({
        client: redis(config.get('redisUrl'))
      }),
      handler: function (req, res /*next*/) {
        res.json({error: 'Too many recent requests.'})
      }
    });


    app.post(baseURL + '/test', limiter, function (req, res) {
       res.json({});
    });

After about 5 hours of uptime, these endpoints stop responding:
Uncaught Response with status: 503 OK for URL: http://127.0.0.1/api/v1/test

Any idea why? Is it the redis module?

Rate limit decreasing faster than actual requests

I have an API hosted on https://conv.mridulahuja.com/INR/0.5656. Now, when you make the first request, the response will have header X-RateLimit-Remaining:19 which is correct. When I make the same request again, it shows say, X-RateLimit-Remaining:15, when it should be 18, and keeps on decreasing for example: 15 -> 11 -> 6 -> 0. As a result, it only allows 4-5 requests when I want to allow 20 requests per minute.

My code:

var limiter = new RateLimit({
    windowMs: 1*60*1000,
    max: 20,
    delayMs: 0
});
app.use(limiter);

How to resolve this @nfriedly ??

record hits per IP per route

Perhaps the next feature could be allowing a different record for each route like express-brute?

This means that I can allow a particular IP to access /routeA 50 times in an hour and /routeB 30 times per minute.

Error: The store is not valid.

So everything worked fine on the first run but after shutting the node server down and restarting it this error occured

Any idea how i can resolve this issue ?

// ensure that the store has the incr method
    if (typeof options.store.incr !== 'function' || typeof options.store.resetKey !== 'function') {
        throw new Error('The store is not valid.');
    }

Error: The store is not valid.
    at new RateLimit (/Users/deltam/git/project/node_modules/express-rate-limit/lib/express-rate-limit.js:38:15)
    at Object.<anonymous> (/Users/deltam/git/project/server.js:196:15)
    at Module._compile (module.js:541:32)
    at Object.Module._extensions..js (module.js:550:10)
    at Module.load (module.js:456:32)
    at tryModuleLoad (module.js:415:12)
    at Function.Module._load (module.js:407:3)
    at Function.Module.runMain (module.js:575:10)
    at startup (node.js:159:18)
    at node.js:444:3

Scaling out to a million users causes node to fall over

A timed circular buffer might need to be created where the more entries in the cache the shorter time they are persisted. Another option is a computed hash that uses user time and server time. the same hash is sent to users between a timespan. ( x mins) The server can cache their results under the same hash using computed fingerprint based on their session. This would allow for a set of entries to be flushed at time rather than individual entries with a settimeout. Just a thought. Either way, the issue looks to be memory/resource related.