need support for delayBatch

[diamondio]

need support for delayBatch

[retorquere]

Wouldn't this be solved by having your callback just use setTimeout to trigger the actual action and callback?

[retorquere]

I could see why that would be an issue, but how else do you want to create a delay if no by using settimeout? I mean you could use two queues, the primary having a afterProcessDelay of one minute, and have tasks on that queue put tasks on a secondary queue, but I'm betting better-queue uses (something like) setTimeout for afterProcessDelay, so it really wouldn't help.

Why do you need to have these events held for a minute BTW? Is it to collapse events? I use cancelIfRunning: true for that (or merge could be used similarly). If it's just to pace things out, you could use a paused queue and have a setTimer activate the queue only once a minute

But with these kind of volumes, you may want to look at something like a redis-backed queue. I'd need to know more about your specific use-case.

[dungvu-novus]

You could not use setTimeout for 100K-300K event in flash time (10-15s) in coming. So I try to use queue to reproduce and persistent the data.

[dungvu-novus]

even you could handle the big events, there no memory to delay 1 minute or more time period. That in my stress test, but in the production (~100k per minute) there is not good idea.

[dungvu-novus]

yep, I looking for db-based queue like mongo or redis for persistent big data amounts. But sadly my project is too small and so hard to add new db system :(
I find this lib that use sqllite in 1 file for backed storage, so I try to use this lib, but as your advice, there is no the good ideal. Could you have some ideal for this.
Thanks,