resque/resque-pool

quickly fork a pool resque workers, saving memory (w/REE) and monitoring their uptime.

resque-pool's Issues

respond to INT or TERM when in the middle of QUIT

if the workers are taking too long to shutdown via QUIT, INT or TERM should behave like normal and allow an immediate manager shutdown (without needing to resort to KILL).

(this was broken a while ago... it might have been fixed and I'm simply forgetting about it. need to test.)

more flexible config file format

To help with certain new features, but still retain backwards compatibility, here's the proposal: https://github.com/nevans/resque-pool/wiki/New-config-file-format

man pages

compatible with gem-man

resque-pool(1)
resque-pool.yml(5)

Add per-host configuration

In a cluster of workers, I usually want different workers handling different queues. It'd be nice if in addition to splitting by environment that I could split by hostname in that environment.

Should HUP pick up new capistrano release?

I'm managing resque-pool with bluepill and deploys with capistrano. The only way I'm able to get the resque workers to pick up the latest release though is to kill the master resque-pool process and start it back up again.

Was HUP intended for this kind of use or do I need to find another approach?

Along the same lines, is it possible to spawn the new pool of workers on the new release prior to quitting the old workers to guarantee zero downtime?

Sending HUP does not reopen log files

Currently using logrotate to rotate the log files for resque-pool but sending HUP does reopen the log files - instead the manager keeps logging to the renamed log file, in this case resque.log.1.

Documentation Request, how is this different from doing COUNT= in resque?

Resque has the COUNT= command to spawn multiple workers. What does resque pool give in addition to that? The ability to customize the workers between queues?

change process group for workers

upstartd sends the signals to the process group, which defeats the purpose of our fancy signal handling.

provide Unix style log formatter

compatible with $stdout/$stderr, because that's how resque rolls its logs.

Most useful is seeing the PIDs in the logfile, and differentiating between master messages and child messages. When run from the CLI, it would default to on, but have a command line option to turn it off.

Lots of Interrupted system call - connect(2) (Errno::EINTR) on startup

I'm not entirely sure what version this started at, but it seems to be a recent thing. When I launch resque-pool on my development machine, it does successfully start, but it has a bit of a bumpy ride to get there. Do you have any idea what's causing this? Would it be something in my redis configuration, or a concurrency issue on the ruby side of things?

chris@chris:~/flippa/rails$ bundle exec resque-pool
resque-pool-manager[8078]: Resque Pool running in development environment
(in /home/chris/flippa/flippa-rails)
resque-pool-worker[8101]: Starting worker chris:8101:competitor_data
resque-pool-worker[8103]: Starting worker chris:8103:operations
resque-pool-worker[8115]: Starting worker chris:8115:reminders
resque-pool-worker[8108]: Starting worker chris:8108:stats
/home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/connection/ruby.rb:26:in `initialize': Interrupted system call - connect(2) (Errno::EINTR)
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/connection/ruby.rb:26:in `new'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/connection/ruby.rb:26:in `block in connect'
    from /home/chris/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/timeout.rb:57:in `timeout'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/connection/ruby.rb:128:in `with_timeout'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/connection/ruby.rb:25:in `connect'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:227:in `establish_connection'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:23:in `connect'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:247:in `ensure_connected'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:137:in `block in process'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:206:in `logging'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:136:in `process'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:46:in `call'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis.rb:421:in `block in smembers'
    from /home/chris/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis.rb:420:in `smembers'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-namespace-1.0.3/lib/redis/namespace.rb:213:in `method_missing'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-1.19.0/lib/resque/worker.rb:27:in `all'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-1.19.0/lib/resque/worker.rb:330:in `prune_dead_workers'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-1.19.0/lib/resque/worker.rb:228:in `startup'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-1.19.0/lib/resque/worker.rb:114:in `work'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:314:in `block in spawn_worker!'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:309:in `fork'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:309:in `spawn_worker!'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:288:in `block in spawn_missing_workers_for'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:287:in `times'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:287:in `spawn_missing_workers_for'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:273:in `block in maintain_worker_count'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:271:in `each'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:271:in `maintain_worker_count'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:191:in `start'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:65:in `run'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool/tasks.rb:17:in `block (2 levels) in <top (required)>'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:636:in `call'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:636:in `block in execute'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:631:in `each'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:631:in `execute'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:597:in `block in invoke_with_call_chain'
    from /home/chris/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:590:in `invoke_with_call_chain'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:583:in `invoke'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool/cli.rb:110:in `start_pool'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool/cli.rb:16:in `run'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/bin/resque-pool:5:in `<top (required)>'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/bin/resque-pool:19:in `load'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/bin/resque-pool:19:in `<main>'
resque-pool-manager[8078]: started manager
resque-pool-manager[8078]: Pool contains worker PIDs: [8101, 8103, 8108, 8115, 8125]
resque-pool-worker[8125]: Starting worker chris:8125:mail_queue
/home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/connection/ruby.rb:26:in `initialize': Interrupted system call - connect(2) (Errno::EINTR)
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/connection/ruby.rb:26:in `new'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/connection/ruby.rb:26:in `block in connect'
    from /home/chris/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/timeout.rb:57:in `timeout'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/connection/ruby.rb:128:in `with_timeout'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/connection/ruby.rb:25:in `connect'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:227:in `establish_connection'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:23:in `connect'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:247:in `ensure_connected'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:137:in `block in process'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:206:in `logging'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:136:in `process'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:46:in `call'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis.rb:421:in `block in smembers'
    from /home/chris/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis.rb:420:in `smembers'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-namespace-1.0.3/lib/redis/namespace.rb:213:in `method_missing'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-1.19.0/lib/resque/worker.rb:27:in `all'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-1.19.0/lib/resque/worker.rb:330:in `prune_dead_workers'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-1.19.0/lib/resque/worker.rb:228:in `startup'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-1.19.0/lib/resque/worker.rb:114:in `work'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:314:in `block in spawn_worker!'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:309:in `fork'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:309:in `spawn_worker!'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:288:in `block in spawn_missing_workers_for'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:287:in `times'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:287:in `spawn_missing_workers_for'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:273:in `block in maintain_worker_count'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:271:in `each'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:271:in `maintain_worker_count'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:191:in `start'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:65:in `run'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool/tasks.rb:17:in `block (2 levels) in <top (required)>'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:636:in `call'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:636:in `block in execute'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:631:in `each'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:631:in `execute'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:597:in `block in invoke_with_call_chain'
    from /home/chris/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:590:in `invoke_with_call_chain'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:583:in `invoke'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool/cli.rb:110:in `start_pool'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool/cli.rb:16:in `run'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/bin/resque-pool:5:in `<top (required)>'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/bin/resque-pool:19:in `load'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/bin/resque-pool:19:in `<main>'
resque-pool-manager[8078]: Reaped resque worker[8101] (status: 1) queues: competitor_data
resque-pool-manager[8078]: Reaped resque worker[8108] (status: 1) queues: stats
resque-pool-worker[8144]: Starting worker chris:8144:competitor_data
resque-pool-worker[8146]: Starting worker chris:8146:stats
/home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/connection/ruby.rb:26:in `initialize': Interrupted system call - connect(2) (Errno::EINTR)
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/connection/ruby.rb:26:in `new'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/connection/ruby.rb:26:in `block in connect'
    from /home/chris/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/timeout.rb:57:in `timeout'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/connection/ruby.rb:128:in `with_timeout'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/connection/ruby.rb:25:in `connect'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:227:in `establish_connection'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:23:in `connect'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:247:in `ensure_connected'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:137:in `block in process'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:206:in `logging'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:136:in `process'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis/client.rb:46:in `call'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis.rb:421:in `block in smembers'
    from /home/chris/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-2.2.2/lib/redis.rb:420:in `smembers'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/redis-namespace-1.0.3/lib/redis/namespace.rb:213:in `method_missing'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-1.19.0/lib/resque/worker.rb:27:in `all'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-1.19.0/lib/resque/worker.rb:330:in `prune_dead_workers'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-1.19.0/lib/resque/worker.rb:228:in `startup'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-1.19.0/lib/resque/worker.rb:114:in `work'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:314:in `block in spawn_worker!'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:309:in `fork'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:309:in `spawn_worker!'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:288:in `block in spawn_missing_workers_for'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:287:in `times'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:287:in `spawn_missing_workers_for'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:273:in `block in maintain_worker_count'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:271:in `each'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:271:in `maintain_worker_count'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:212:in `block in join'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:207:in `loop'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:207:in `join'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool.rb:65:in `run'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool/tasks.rb:17:in `block (2 levels) in <top (required)>'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:636:in `call'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:636:in `block in execute'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:631:in `each'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:631:in `execute'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:597:in `block in invoke_with_call_chain'
    from /home/chris/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:590:in `invoke_with_call_chain'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/rake-0.8.7/lib/rake.rb:583:in `invoke'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool/cli.rb:110:in `start_pool'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/lib/resque/pool/cli.rb:16:in `run'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/gems/resque-pool-0.2.0/bin/resque-pool:5:in `<top (required)>'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/bin/resque-pool:19:in `load'
    from /home/chris/.rvm/gems/ruby-1.9.2-p180/bin/resque-pool:19:in `<main>'
resque-pool-manager[8078]: Reaped resque worker[8144] (status: 1) queues: competitor_data
resque-pool-worker[8162]: Starting worker chris:8162:competitor_data

Here's my resque-pool.yml:

defaults: &defaults
  competitor_data: 1
  operations: 1
  stats: 1
  reminders: 1
  mail_queue: 1

development:
  <<: *defaults

monitor child memory usage

monit style limits, e.g. 2 cycles at 500MB and QUIT, 6 cycles at 600MB and TERM, 8 cycles at 600MB and KILL.

must be settable in the config file (and the WebUI). must be settable per worker type, as well as for default.

will need to extend config file format to support this.

pool manager crashes when delete_worker attempts to delete missing worker

Reported by Spencer Portée from Patch.com:

I think I found a weird bug with r-p. In pool.rb, reap_all_workers, a pid is getting reaped, but the worker is missing. delete_worker returns nil because the pid that threw its signal (not sure which sig yet) maps to a worker that's gone. Perhaps the missing worker crashed or what-have-you; I'm still figuring that out.

Because the worker is missing, the debug statement that prints the status and the queues it was on fails, crashing r-p completely.

reopen logfiles on HUP

HUP signal should cause the logfiles to be reopened. This is most useful in a postrotate script for logrotate.

Remove dependency on Resque

There are several different Resque forks, with different names (for example mongo-resque and resque-mongo).
Luckily, resque-pool is compatible with almost all of them.

Can the dependency on Resque be removed (or made optional somehow)?
That way, all Resque-compatible forks can be used.

(BTW: Thanks for the great work on resque-pool! It's awesome!)

Monitor resque pool workers using god

How to monitor resque pool workers using god.please help.

Thanks

rdoc

There isn't much to it in terms of API, besides Resque::Pool.after_fork and Resque::Pool.run. Still, rdoc is good. :)

increase test coverage!!!

I'd like to get some decent full stack (cucumber?) tests over this. I'm not aware of any good examples of multi-proc testing with cucumber, so I'll have to think about it a bit, and probably have some step definitions making excessive use of ps. Some refactoring will be needed to get better unit level coverage over the code.

zero-downtime restarts

for those time-sensitive mission critical jobs that just can't be delayed by 15 seconds without annoying users (or those apps that take more than 15 seconds to boot up).

Resque-pool does not start properly all workers

My environment:

• OS: Ubuntu 11.04
• Ruby: ruby 1.9.2p180 (2011-02-18 revision 30909) [x86_64-linux]
• Redis server 2.0.1

Gems:

• redis 2.2.1
• resque 1.17.1
• resque-scheduler 1.9.9
• rails 3.0.8
• resque-pool 0.2.0

Run command:

bundle exec resque-pool --daemon --pidfile=tmp/pids/resque-pool.pid --environment=development

Resque pool config:

account_expire_jobs: 2
domain_expire_notifiers: 2
domain_exporters: 2
domain_info_fetchers: 2

ps -ef f | grep [r]esque outputs:

deploy 12455 7623 0 16:39 pts/1 Sl 0:05 | \_ ruby /usr/local/lib/ruby/gems/1.9.1/bin/rake resque:scheduler deploy 7708 1 0 01:43 ? Sl 1:10 /usr/bin/ruby1.9.1 /usr/bin/resque-web config/initializers/resque.rb deploy 12547 1 0 16:42 ? Sl 0:04 resque-pool-master: managing [12552, 12555, 12559, 12562, 12564, 12566, 12568, 12570] deploy 12552 12547 0 16:42 ? Sl 0:00 \_ resque-1.17.1: Waiting for account_expire_jobs deploy 12555 12547 0 16:42 ? Sl 0:00 \_ resque-1.17.1: Waiting for account_expire_jobs deploy 12559 12547 0 16:42 ? Sl 0:00 \_ resque-1.17.1: Waiting for domain_expire_notifiers deploy 12562 12547 0 16:42 ? Sl 0:00 \_ resque-1.17.1: Waiting for domain_expire_notifiers deploy 12564 12547 98 16:42 ? Rl 21:32 \_ resque: Starting deploy 12566 12547 0 16:42 ? Sl 0:00 \_ resque-1.17.1: Waiting for domain_exporters deploy 12568 12547 49 16:42 ? Rl 10:51 \_ resque: Starting deploy 12570 12547 49 16:42 ? Rl 10:51 \_ resque: Starting

rails console:

deploy@s314:~/www/current$ bundle exec rails console
Loading development environment (Rails 3.0.8)
irb(main):001:0> puts Resque::Worker.all
s314:12559:domain_expire_notifiers
s314:12552:account_expire_jobs
s314:12562:domain_expire_notifiers
s314:12566:domain_exporters
s314:12555:account_expire_jobs
=> nil
irb(main):002:0> puts Resque.redis.smembers(:workers)
s314:12559:domain_expire_notifiers
s314:12552:account_expire_jobs
s314:12562:domain_expire_notifiers
s314:12566:domain_exporters
s314:12555:account_expire_jobs
=> nil
irb(main):003:0>

integrate with resque-scheduler

Perhaps a config option to startup resque-scheduler inside our daemon?

Getting a cant modify frozen object error in resque workers

I recently started using resque-pool and I noticed that I get errors like the following:

Can't modify frozen object
/home/ec2-user/.rvm/gems/ruby-1.9.2-p290/gems/json_pure-1.6.1/lib/json/pure/generator.rb:219:in configure' /home/ec2-user/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.0.10/lib/active_support/json/encoding.rb:61:inoptions_for'
/home/ec2-user/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.0.10/lib/active_support/json/encoding.rb:46:in block in encode' /home/ec2-user/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.0.10/lib/active_support/json/encoding.rb:77:incheck_for_circular_references'
/home/ec2-user/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.0.10/lib/active_support/json/encoding.rb:45:in encode' /home/ec2-user/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.0.10/lib/active_support/json/encoding.rb:30:inencode'
/home/ec2-user/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.0.10/lib/active_support/core_ext/object/to_json.rb:20:in to_json' /home/ec2-user/.rvm/gems/ruby-1.9.2-p290/gems/json_pure-1.6.1/lib/json/pure/generator.rb:246:ingenerate'
/home/ec2-user/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/json/common.rb:212:in `generate'

This happens inside my worker logic.

I am using only one resque worker in the resque-pool for simplicity.

I don't face this issue without resque-pool (i.e. when I just use resque rake tasks with one or more workers).

Any idea what can be causing this?

ruby-1.9.2 and SystemTimer

I was having an issue with SystemTimer and found on the googlez that it doesn't play nicely with ruby-1.9.2 so I added this to the resque-pool.gemspec file

s.add_development_dependency "SystemTimer" if RUBY_VERSION =~ /^1\.8/

bundle install was happy after I did that.

Passing signals

Pardon me if this is an obvious question.

In order to kill or restart workers I would think that you would pass the signals like so:

resque-pool QUIT --environment production

But this proves incorrect.

Can you please spell it out for a simple man.

Cheers,

LB

How enable rdebug-ide with rescue-pool ?

Hello.
I want to do remote debug with rubymine.
I have rescue-pool.
Can I use rdebug-ide for debug rescue-pool?
If yes, then give me console command for it, please.

clean up the code

I stole the guts of it from unicorn, and it's still a bit bastardized;

excessive use of vim foldmarkers are a code smell!

At the very least, lib/resque/pool.rb and Resque::Pool should probably give birth to some new classes and files.

ActiveRecord::ConnectionNotEstablished

Using sqlite3 everywhere but production, and everything else is running fine, but I'm seeing this error in development. Any ideas? Seems like this shouldn't happen with a functioning sqlite3 setup (created, migrated, etc).

$ resque-pool --environment development
resque-pool-manager[xxx][77466]: Resque Pool running in development environment
/Users/kcm/.rvm/gems/ruby-1.9.3-p327/gems/activerecord-3.2.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:404:in `retrieve_connection': ActiveRecord::ConnectionNotEstablished (ActiveRecord::ConnectionNotEstablished)
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327/gems/activerecord-3.2.6/lib/active_record/connection_adapters/abstract/connection_specification.rb:168:in `retrieve_connection'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327/gems/activerecord-3.2.6/lib/active_record/connection_adapters/abstract/connection_specification.rb:142:in `connection'
    from /Users/kcm/src/distill/distill/lib/tasks/resque.rake:38:in `block in <top (required)>'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327@global/gems/rake-10.0.2/lib/rake/task.rb:227:in `call'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327@global/gems/rake-10.0.2/lib/rake/task.rb:227:in `block in execute'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327@global/gems/rake-10.0.2/lib/rake/task.rb:222:in `each'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327@global/gems/rake-10.0.2/lib/rake/task.rb:222:in `execute'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327@global/gems/rake-10.0.2/lib/rake/task.rb:166:in `block in invoke_with_call_chain'
    from /Users/kcm/.rvm/rubies/ruby-1.9.3-p327/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327@global/gems/rake-10.0.2/lib/rake/task.rb:159:in `invoke_with_call_chain'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327@global/gems/rake-10.0.2/lib/rake/task.rb:187:in `block in invoke_prerequisites'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327@global/gems/rake-10.0.2/lib/rake/task.rb:185:in `each'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327@global/gems/rake-10.0.2/lib/rake/task.rb:185:in `invoke_prerequisites'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327@global/gems/rake-10.0.2/lib/rake/task.rb:165:in `block in invoke_with_call_chain'
    from /Users/kcm/.rvm/rubies/ruby-1.9.3-p327/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327@global/gems/rake-10.0.2/lib/rake/task.rb:159:in `invoke_with_call_chain'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327@global/gems/rake-10.0.2/lib/rake/task.rb:152:in `invoke'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327/gems/resque-pool-0.3.0/lib/resque/pool/cli.rb:114:in `start_pool'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327/gems/resque-pool-0.3.0/lib/resque/pool/cli.rb:16:in `run'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327/gems/resque-pool-0.3.0/bin/resque-pool:7:in `<top (required)>'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327/bin/resque-pool:19:in `load'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327/bin/resque-pool:19:in `<main>'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327/bin/ruby_noexec_wrapper:14:in `eval'
    from /Users/kcm/.rvm/gems/ruby-1.9.3-p327/bin/ruby_noexec_wrapper:14:in `<main>'

workers fail to startup with redis-error

/home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/redis-3.0.1/lib/redis/client.rb:271:in ensure_connected': Tried to use a connection from a child process without reconnecting. You need to reconnect to Redis after forking. (Redis::InheritedError) from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/redis-3.0.1/lib/redis/client.rb:167:inblock in process'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/redis-3.0.1/lib/redis/client.rb:242:in logging' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/redis-3.0.1/lib/redis/client.rb:166:inprocess'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/redis-3.0.1/lib/redis/client.rb:78:in call' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/redis-3.0.1/lib/redis.rb:714:inblock in get'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/redis-3.0.1/lib/redis.rb:36:in block in synchronize' from /home/rajesh/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/monitor.rb:201:inmon_synchronize'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/redis-3.0.1/lib/redis.rb:36:in synchronize' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/redis-3.0.1/lib/redis.rb:713:inget'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/redis-namespace-1.2.0/lib/redis/namespace.rb:243:in method_missing' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/bundler/gems/resque-9e11b107ccf8/lib/resque/worker.rb:454:injob'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/bundler/gems/resque-9e11b107ccf8/lib/resque/worker.rb:387:in unregister_worker' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/bundler/gems/resque-9e11b107ccf8/lib/resque/worker.rb:159:inensure in work'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/bundler/gems/resque-9e11b107ccf8/lib/resque/worker.rb:159:in work' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/resque-pool-0.3.0/lib/resque/pool.rb:326:inblock in spawn_worker!'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/resque-pool-0.3.0/lib/resque/pool.rb:321:in fork' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/resque-pool-0.3.0/lib/resque/pool.rb:321:inspawn_worker!'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/resque-pool-0.3.0/lib/resque/pool.rb:300:in block in spawn_missing_workers_for' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/resque-pool-0.3.0/lib/resque/pool.rb:299:intimes'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/resque-pool-0.3.0/lib/resque/pool.rb:299:in spawn_missing_workers_for' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/resque-pool-0.3.0/lib/resque/pool.rb:285:inblock in maintain_worker_count'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/resque-pool-0.3.0/lib/resque/pool.rb:283:in each' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/resque-pool-0.3.0/lib/resque/pool.rb:283:inmaintain_worker_count'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/resque-pool-0.3.0/lib/resque/pool.rb:198:in start' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/resque-pool-0.3.0/lib/resque/pool.rb:72:inrun'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/resque-pool-0.3.0/lib/resque/pool/tasks.rb:17:in block (2 levels) in <top (required)>' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/rake-0.9.2.2/lib/rake/task.rb:205:incall'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/rake-0.9.2.2/lib/rake/task.rb:205:in block in execute' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/rake-0.9.2.2/lib/rake/task.rb:200:ineach'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/rake-0.9.2.2/lib/rake/task.rb:200:in execute' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/rake-0.9.2.2/lib/rake/task.rb:158:inblock in invoke_with_call_chain'
from /home/rajesh/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/monitor.rb:201:in mon_synchronize' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/rake-0.9.2.2/lib/rake/task.rb:151:ininvoke_with_call_chain'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/rake-0.9.2.2/lib/rake/task.rb:144:in invoke' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/resque-pool-0.3.0/lib/resque/pool/cli.rb:114:instart_pool'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/resque-pool-0.3.0/lib/resque/pool/cli.rb:16:in run' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/gems/resque-pool-0.3.0/bin/resque-pool:7:in<top (required)>'
from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/bin/resque-pool:19:in load' from /home/rajesh/.rvm/gems/ruby-1.9.2-p180@spaghetti-300/bin/resque-pool:19:in

'

rename to resque-squad

It just works better with the "rescue worker" pun.

change procline for state changes

e.g.: malformed config file, graceful shutdown, etc.

It would be nice to also reflect this in the web UI

start without running queues throws error

When starting for the first time without running workers an error is thrown and stops the whole pool from running:

bundle exec resque-pool
=> calls reaper

gems/resque-pool-0.2.0/lib/resque/pool.rb:243:in `reap_all_workers': You have a nil object when you didn't expect it! (NoMethodError)
The error occurred while evaluating nil.queues

Possible solution, add 'if worker' to reaper message:

log "Reaped resque worker[#{status.pid}] (status: #{status.exitstatus}) queues: #{worker.queues.join(",")}" if worker

I am not sure, but i think last week i had it running local without this error. maybe it is the startup time of the workers which prevents the reaper from seeing them.

integrate with resque-batchworker

integrate with or incorporate resque-batchworker features

syslog support

priorities for queues

In resque we can set priorities for queues like below

$ QUEUES=file_serve,warm_cache rake resque:work
When the worker looks for new jobs, it will first check file_serve. If it finds a job, it'll process it then check file_serve again. It will keep checking file_serve until no more jobs are available. At that point, it will check warm_cache. If it finds a job it'll process it then check file_serve (repeating the whole process).

is it possible to implement this with resque-pool.how i implement this in yml file.Please help

Thanks.

Resque::DirtyExit when sending kill -s QUIT

We're getting multiple Resque::DirtyExit exceptions being thrown when we send a kill -s QUIT to resque-pool.

As far as I understood, resque-pool should forward on the QUIT signal to the resque workers, meaning we should never see a Resque::DirtyExit exception?

We're running resque-1.17.1 and resque-pool-0.2.0.

kill -SIGHUP doesn't always restart workers

It appears that kill -SIGHUP stops restarting the workers eventually and I have to do a WINCH and HUP to start them over again. Is there a place in the log files where I can inspect what is going on or is there another reason the pool-master stops responding to these signals?

resque-pool identifies workers from other applications as orphans

When running resque-pool for multiple apps on the same server, this line fails to identify real orphans:

ps -f axf | grep resque | grep -v grep | grep -v resque-web | grep -v bluepill | grep -v master | awk '{printf("%d %d\n", $2, $3)}'

Resque-pool should probably set its a more unique process title and use that for detection.

SIGWINCH should not be used

SIGWINCH is used to report window change events. This means that, for instance, changes to the size of Terminal.app will cause workers to stop while running rake rescue:pool in development. This is undesirable.

Jobs failing during startup of resque-pool

Hi,

I use resque-pool for managing multiple resque instances and found it very useful because of very fast start up and lesser memory usage. I am facing some issues with resque-pool during start up. First few jobs fail consistently in production environment, ( It never happened in development/testing environment). The errors were mainly Method missing ActiveRecord exceptions. Example: Model.find_by_<column_name> - find_by_<column_name> not found. Same Jobs when i retry after sometime work fine. Any directions on what i could be doing wrong will be very helpful.

Thanks,
Rajesh Shanmugam
Developer,Althea systems,

configure TERM behavior from command line

since TERM is the default, sometimes it's easier to behave differently for TERM than to send a different signal.

Use command line options to configure; e.g. --term-graceful-wait --term-graceful --term-immediate

monitor worker run time

must be settable per worker type, in config file and in web UI. see also issue #18

Polling frequency?

Hi,

Resque support setting the polling frequency with the INTERVAL option.
I am currently using resque pool and start my workers with:

resque-pool --daemon --environment ... --pidfile ...

If I just add the INTERVAL option, will this be honoured also by the workers stared by resque-pool?

Thanks in advance
Vito

options to skip rakefile

By default we load the rakefile and run the resque:pool rake task, because that follows the instructions in the resque README. We can also provide the ability to run Resque::Pool.run directly, from inside the resque-pool daemonizing script.

--init, -i <s>: Preload ruby script to initialize resque-pool
     --no-rake: Run without loading Rakefile or resque:pool task

to be run like so:
resque-pool -d -e production -i config/resque-pool-init.rb --no-rake

and config/resque-pool-init.rb might look like so:
require File.expand_path('environment', File.dirname(FILE))
# setup resque failure backend
# close any sockets or files in pool master
ActiveRecord::Base.connection.disconnect!
# and re-open them in the resque worker parent
Resque::Pool.after_prefork do |job|
ActiveRecord::Base.establish_connection
end

web interface

A new tab for resque-web, allowing you to temporarily override the config, send signals, etc.

recover gracefully from a malformed config file (on startup and HUP)

report malformed config in procline and on web interface

on startup: just startup with no workers configured

on HUP: just keep current config

`resque-pool` is not using initializers

@nevans -

It seems that for whatever reason, resque-pool is not using initializers properly when run using this command:

bundle exec resque-pool --daemon --environment production

• Log output (resque-pool.stderr.log)
• lib/tasks/resque.rake
• config/initializers/resque.rb
• config/initializers/redis.rb
• config/redis.yml

I tried to include all of the information needed. Let me know what else I can do.

`resque-pool -c config.yml` command line option broken

Trollop thinks it's a boolean but it's a string

Don't rely on git in gemspec

Bundler can't load an app using resque-pool if git isn't on the PATH used to invoke bundler. This is a problem for any Mac GUI app (e.g., RubyMine), since Apple uses a default PATH that does not include /usr/local.

Generally speaking, a runtime dependency on git probably should be avoided.

No such file or directory - tmp/pids/resque-pool.pid

/home/vinny/.rvm/gems/ruby-1.9.2-head@logikcull/gems/resque-pool-0.2.0/lib/resque/pool/cli.rb:66:in `initialize': No such file or directory - tmp/pids/resque-pool.pid (Errno::ENOENT)
    from /home/vinny/.rvm/gems/ruby-1.9.2-head@logikcull/gems/resque-pool-0.2.0/lib/resque/pool/cli.rb:66:in `open'
    from /home/vinny/.rvm/gems/ruby-1.9.2-head@logikcull/gems/resque-pool-0.2.0/lib/resque/pool/cli.rb:66:in `manage_pidfile'
    from /home/vinny/.rvm/gems/ruby-1.9.2-head@logikcull/gems/resque-pool-0.2.0/lib/resque/pool/cli.rb:13:in `run'

I am using this on a non-rails project and ran into this problem. I can submit a patch later today for it. There is another issue that I also noted and could affect things. File.exist? only works on directories in 1.9.2. I'm not sure about 1.8.7 and before. To actually check to see if the file exists you have to use File.file?. Anyways, back to the original problem, It looks like the issue is that since I am not using a typical rails project, the tmp/pid/ directory doesn't exist and the subsequent open file goes KABOOM!

Resque-web only showing one worker

Perhaps I don't have something configured correctly, but my resque-web doesn't seem to be able to see all the workers (they're stil working though).

Do I have to let resque-web know in any way that resque-pool is running?

`resque-pool` executable does not honor configured initializers

Unsurprisingly (once I thought it through a little) the resque-pool executable doe not honor configurations set in the Rails environment. Here's my use-case.

# ./config/initializers/resque.rb
config = YAML.load_file(Rails.root + './config/resque.yml')
Resque.redis = config[Rails.env]
Resque.redis.namespace = "project:resque:#{Rails.env}"

# ./config/resque.yml
development: localhost:6379
test: localhost:6379

Naturally, I can start it with rake, but then I'm paying the start-up time for the whole environment, which actually isn't a huge problem, but it does leave rather a nasty hole, where you can bring up workers that ignored your environmental config.

I know there's some discussion about resque-namepsace, and/or whether it's a good idea, or you should use individual databases, or etc – but I think in this case, at least - we should patch something.

I'm proposing that resque-pool become a wrapper for the similarly named rake task, so the environment is always honored.