Option to Replace / Not Replace Jobs With Same JID

[seomoz]

Option to Replace / Not Replace Jobs With Same JID

[ssoroka]

is it possible that the qless-core API is already a little crowded? REDIS itself implements many specific (short) methods to do specific things. qless-core seems to have a few methods that do many things, depending on the parameters you send them.

[myronmarston]

Doesn't Qless already support replacing an existing job if you put a new one with the same jid? I've been using that for a while.

Not sure what you mean by "not replacing" a job with the same jid--if you're putting a job with the jid of an existing job, what else would it do?

[ssoroka]

There's two approaches you could take when placing a job and there's already one in queue:

1. Delete the job you're placing (not place it) and leave the existing one
2. Delete the existing job and place the new one.

Both are useful depending on the application.

Eg: cache regeneration would use #1, but list sort order changing would use #2

• Steven Soroka

On 2012-10-08, at 10:42 PM, Myron Marston [email protected] wrote:

Doesn't Qless already support replacing an existing job if you put a new one with the same jid? I've been using that for a while.

Not sure what you mean by "not replacing" a job with the same jid--if you're putting a job with the jid of an existing job, what else would it do?

—
Reply to this email directly or view it on GitHub.

[myronmarston]

Delete the job you're placing (not place it) and leave the existing one

Honestly, I have a hard time seeing how this would ever be desired behavior. It sounds too much like "silent failure". If Qless isn't going to do what you ask (e.g. put the job), then it would seem preferable to raise an error, and the you're code could always rescue it and ignore it.

Can you explain a bit more about your cache regeneration use case? My use case of Qless may just be entirely different.

Also, I feel like #put should have the same semantics as an HTTP PUT (e.g. idempotent create or replace).

[ssoroka]

#1 is common: any throttling code typically does exactly this. The first request goes through and the rest don't until the first is processed. If you're doing some slow task in the background, it's not out of date until the first one you asked for completes, and other requests are redundant and superfluous. Because this has to happen atomicly, a simple check first to see if the record is in the queue isn't good enough.

[phs]

Idempotent enqueuing (1.) is something I'm also interested in. I have an upstream slow, durable dumb queue that I'd like to feed into my qless (actually plines) installation. We're still feeling out if we want a push or a pull connecting them, but a batch push would certainly want idempotency to avoid clobbering jobs in flight.