Giter Site home page Giter Site logo

solid_queue's Introduction

Welcome to Rails

What's Rails?

Rails is a web-application framework that includes everything needed to create database-backed web applications according to the Model-View-Controller (MVC) pattern.

Understanding the MVC pattern is key to understanding Rails. MVC divides your application into three layers: Model, View, and Controller, each with a specific responsibility.

Model layer

The Model layer represents the domain model (such as Account, Product, Person, Post, etc.) and encapsulates the business logic specific to your application. In Rails, database-backed model classes are derived from ActiveRecord::Base. Active Record allows you to present the data from database rows as objects and embellish these data objects with business logic methods. Although most Rails models are backed by a database, models can also be ordinary Ruby classes, or Ruby classes that implement a set of interfaces as provided by the Active Model module.

View layer

The View layer is composed of "templates" that are responsible for providing appropriate representations of your application's resources. Templates can come in a variety of formats, but most view templates are HTML with embedded Ruby code (ERB files). Views are typically rendered to generate a controller response or to generate the body of an email. In Rails, View generation is handled by Action View.

Controller layer

The Controller layer is responsible for handling incoming HTTP requests and providing a suitable response. Usually, this means returning HTML, but Rails controllers can also generate XML, JSON, PDFs, mobile-specific views, and more. Controllers load and manipulate models, and render view templates in order to generate the appropriate HTTP response. In Rails, incoming requests are routed by Action Dispatch to an appropriate controller, and controller classes are derived from ActionController::Base. Action Dispatch and Action Controller are bundled together in Action Pack.

Frameworks and libraries

Active Record, Active Model, Action Pack, and Action View can each be used independently outside Rails.

In addition to that, Rails also comes with:

  • Action Mailer, a library to generate and send emails
  • Action Mailbox, a library to receive emails within a Rails application
  • Active Job, a framework for declaring jobs and making them run on a variety of queuing backends
  • Action Cable, a framework to integrate WebSockets with a Rails application
  • Active Storage, a library to attach cloud and local files to Rails applications
  • Action Text, a library to handle rich text content
  • Active Support, a collection of utility classes and standard library extensions that are useful for Rails, and may also be used independently outside Rails

Getting Started

  1. Install Rails at the command prompt if you haven't yet:

    $ gem install rails
  2. At the command prompt, create a new Rails application:

    $ rails new myapp

    where "myapp" is the application name.

  3. Change directory to myapp and start the web server:

    $ cd myapp
    $ bin/rails server

    Run with --help or -h for options.

  4. Go to http://localhost:3000 and you'll see the Rails bootscreen with your Rails and Ruby versions.

  5. Follow the guidelines to start developing your application. You may find the following resources handy:

Contributing

We encourage you to contribute to Ruby on Rails! Please check out the Contributing to Ruby on Rails guide for guidelines about how to proceed. Join us!

Trying to report a possible security vulnerability in Rails? Please check out our security policy for guidelines about how to proceed.

Everyone interacting in Rails and its sub-projects' codebases, issue trackers, chat rooms, and mailing lists is expected to follow the Rails code of conduct.

License

Ruby on Rails is released under the MIT License.

solid_queue's People

Contributors

adrianthedev avatar allcentury avatar anonychun avatar ansonj avatar bbonamin avatar bensheldon avatar brunoprietog avatar dependabot[bot] avatar djmb avatar excid3 avatar intrepidd avatar intrip avatar jmarsh24 avatar johnmcdowall avatar jonathanhefner avatar jpcamara avatar justinko avatar kevinmirc avatar kiriakosv avatar klenis avatar ksylvest avatar morgoth avatar namolnad avatar nashby avatar neuged avatar nimmolo avatar northeastprince avatar packagethief avatar rosa avatar songjiz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

solid_queue's Issues

Does it scale horizontally?

I am used to Sidekiq where I usually have 1 process with N threads and then I scale horizontally (e.g. I use many Kubernetes pods, each one with a Sidekiq process).

I see that Solid Queue is more like Puma, and it supports multiple threads and multiple forked processes using the config file.

Is it correct / safe to scale Solid Queue horizontally? i.e. run bundle exec rake solid_queue:start on multiple machines at the same time in production

Or Solid Queue supports only vertical scaling on a single machine?

Unreasonable solid queue polling defaults

I have completed a fresh install of solid_queue and added as a plugin to puma. By default it seems to have set
5,Dispatcher,2024-02-06 13:47:45.935556,4,62761,Justins-MacBook-Pro.local,{"polling_interval":1,"batch_size":500},2024-02-06 13:47:45.942486

6,Worker,2024-02-06 13:47:45.943155,4,62762,Justins-MacBook-Pro.local,{"polling_interval":0.1,"queues":"*","thread_pool_size":5},2024-02-06 13:47:45.952185

image

These causes my web logs to go crazy because it's basically a water fall of logs.

web    |   SolidQueue::ScheduledExecution Pluck (5.3ms)  SELECT "solid_queue_scheduled_executions"."job_id" FROM "solid_queue_scheduled_executions" WHERE "solid_queue_scheduled_executions"."scheduled_at" <= $1 ORDER BY "solid_queue_scheduled_executions"."scheduled_at" ASC, "solid_queue_scheduled_executions"."priority" ASC LIMIT $2 FOR UPDATE SKIP LOCKED  [["scheduled_at", "2024-02-06 13:48:23.493518"], ["LIMIT", 500]]
web    |   TRANSACTION (0.6ms)  COMMIT
web    |   SolidQueue::Pause Pluck (1.9ms)  SELECT "solid_queue_pauses"."queue_name" FROM "solid_queue_pauses"
web    |   TRANSACTION (0.9ms)  BEGIN
web    |   SolidQueue::ReadyExecution Pluck (2.9ms)  SELECT "solid_queue_ready_executions"."job_id" FROM "solid_queue_ready_executions" ORDER BY "solid_queue_ready_executions"."priority" ASC, "solid_queue_ready_executions"."job_id" ASC LIMIT $1 FOR UPDATE SKIP LOCKED  [["LIMIT", 5]]
web    |   TRANSACTION (1.4ms)  COMMIT
web    |   SolidQueue::Pause Pluck (1.8ms)  SELECT "solid_queue_pauses"."queue_name" FROM "solid_queue_pauses"
web    |   TRANSACTION (1.2ms)  BEGIN
web    |   SolidQueue::ReadyExecution Pluck (5.9ms)  SELECT "solid_queue_ready_executions"."job_id" FROM "solid_queue_ready_executions" ORDER BY "solid_queue_ready_executions"."priority" ASC, "solid_queue_ready_executions"."job_id" ASC LIMIT $1 FOR UPDATE SKIP LOCKED  [["LIMIT", 5]]
web    |   TRANSACTION (0.6ms)  COMMIT
web    |   SolidQueue::Pause Pluck (1.5ms)  SELECT "solid_queue_pauses"."queue_name" FROM "solid_queue_pauses"
web    |   TRANSACTION (0.4ms)  BEGIN
web    |   SolidQueue::ReadyExecution Pluck (2.1ms)  SELECT "solid_queue_ready_executions"."job_id" FROM "solid_queue_ready_executions" ORDER BY "solid_queue_ready_executions"."priority" ASC, "solid_queue_ready_executions"."job_id" ASC LIMIT $1 FOR UPDATE SKIP LOCKED  [["LIMIT", 5]]
web    |   TRANSACTION (0.6ms)  COMMIT
web    |   SolidQueue::Pause Pluck (0.8ms)  SELECT "solid_queue_pauses"."queue_name" FROM "solid_queue_pauses"
web    |   TRANSACTION (0.3ms)  BEGIN
web    |   SolidQueue::ReadyExecution Pluck (1.7ms)  SELECT "solid_queue_ready_executions"."job_id" FROM "solid_queue_ready_executions" ORDER BY "solid_queue_ready_executions"."priority" ASC, "solid_queue_ready_executions"."job_id" ASC LIMIT $1 FOR UPDATE SKIP LOCKED  [["LIMIT", 5]]
web    |   TRANSACTION (0.3ms)  COMMIT
web    |   SolidQueue::Pause Pluck (3.2ms)  SELECT "solid_queue_pauses"."queue_name" FROM "solid_queue_pauses"
web    |   TRANSACTION (0.7ms)  BEGIN
web    |   SolidQueue::ReadyExecution Pluck (2.2ms)  SELECT "solid_queue_ready_executions"."job_id" FROM "solid_queue_ready_executions" ORDER BY "solid_queue_ready_executions"."priority" ASC, "solid_queue_ready_executions"."job_id" ASC LIMIT $1 FOR UPDATE SKIP LOCKED  [["LIMIT", 5]]
web    |   TRANSACTION (0.7ms)  COMMIT
web    |   SolidQueue::Pause Pluck (2.0ms)  SELECT "solid_queue_pauses"."queue_name" FROM "solid_queue_pauses"
web    |   TRANSACTION (0.6ms)  BEGIN
web    |   SolidQueue::ReadyExecution Pluck (2.2ms)  SELECT "solid_queue_ready_executions"."job_id" FROM "solid_queue_ready_executions" ORDER BY "solid_queue_ready_executions"."priority" ASC, "solid_queue_ready_executions"."job_id" ASC LIMIT $1 FOR UPDATE SKIP LOCKED  [["LIMIT", 5]]
web    |   TRANSACTION (0.5ms)  COMMIT
web    |   SolidQueue::Pause Pluck (0.9ms)  SELECT "solid_queue_pauses"."queue_name" FROM "solid_queue_pauses"
web    |   TRANSACTION (0.4ms)  BEGIN
web    |   SolidQueue::ReadyExecution Pluck (2.0ms)  SELECT "solid_queue_ready_executions"."job_id" FROM "solid_queue_ready_executions" ORDER BY "solid_queue_ready_executions"."priority" ASC, "solid_queue_ready_executions"."job_id" ASC LIMIT $1 FOR UPDATE SKIP LOCKED  [["LIMIT", 5]]
web    |   TRANSACTION (0.4ms)  COMMIT
web    |   SolidQueue::Pause Pluck (1.8ms)  SELECT "solid_queue_pauses"."queue_name" FROM "solid_queue_pauses"
web    |   TRANSACTION (0.6ms)  BEGIN
web    |   SolidQueue::ReadyExecution Pluck (2.1ms)  SELECT "solid_queue_ready_executions"."job_id" FROM "solid_queue_ready_executions" ORDER BY "solid_queue_ready_executions"."priority" ASC, "solid_queue_ready_executions"."job_id" ASC LIMIT $1 FOR UPDATE SKIP LOCKED  [["LIMIT", 5]]

Stop log spam from polling in development

When running in development with :debug log level, the polling triggers a constant stream of log writes, which makes watching the logs for web app debugging more difficult.

A similar issue was seen when loading the Rails console: rails/rails#31688. The fix there simply sets ActiveRecord::Base.verbose_query_logs = false when starting the console.

I'm not sure what the best strategy is here though. I see in the README there are plans to have solid_queue have an in-process mode similar to GoodJob, it's not immediately obvious to me how you could turn down the logs just for one thread. However if you do it in the supervisor, threaded mode will have the same verbosity issue.

I'll attach a fairly ungraceful attempt as a PR and see what can be done to make it fit more with the code style.

Does `solid_queue` depend on the whole Rails stack?

Hi @rosa - we'd love to try out solid_queue in production, but our app does not need to load ActiveStorage, ActionText or ActionMailbox. I think it's pretty common for people to just use some Rails gems.

Would you accept a simple PR #158 adjusting the solid_queue.gemspec so it doesn't require all of rails as a dependency? Including just railties and necessary gems is how they do it in the gemspec for solid_cache too.

solid_queue seems to only need part of the stack. :

# solid_queue/bin/rails

require "rails"
# Pick the frameworks you want:
require "active_model/railtie"
require "active_job/railtie"
require "active_record/railtie"
# require "active_storage/engine"
require "action_controller/railtie"
# require "action_mailer/railtie"
require "action_view/railtie"
# require "action_cable/engine"
require "rails/test_unit/railtie"
require "rails/engine/commands"

Issue with "on_thread_error" override

I am trying to override on_thread_error by adding the following in application.rb.

config.solid_queue.on_thread_error = ->(exception) {
  puts "*" * 100
  puts "exception: #{exception}"

  # do something

  Rails.error.report(exception, source: "solid_queue", severity: :error)
}

I have a sample job to trigger the above

class SendMailJob < ApplicationJob
  def perform
    raise StandardError
  end
end

But the proc doesn't get executed when I run SendMailJob.perform_later in the rails console.

Getting Lost connection to MySQL server during query (Mysql2::Error::ConnectionError)

Env and Versions:

  • Ruby 3.3.0
  • MySQL: 8.2.0 for macos13.5 on arm64 (Homebrew)
  • MySQL Adapter: mysql2 V0.5.5
  • Rails 7.1.2
  • Solid-Queue 0.1.2

Solid_Queue Configurations:

config/solid_queue.yml:

development:
  dispatchers:
    - polling_interval: 1
      batch_size: 500
  workers:
    - queues: "dc_development_*"
      threads: 2
      processes: 1
      polling_interval: 0.1

Steps to reproduce:

  1. Started a supervisor on shell tab:
bundle exec rake solid_queue:start
  1. Started Rails Console on another tab to queue a job (simple mailer job):
Notifications::MailerJob.perform_later(Payment.last, :successfull)

Solid_Queue Log Trace:

[SolidQueue] Starting Dispatcher(pid=69907, hostname=Mohameds-MacBook-Pro-2.local, metadata={:polling_interval=>1, :batch_size=>500})
[SolidQueue] Starting Worker(pid=69908, hostname=Mohameds-MacBook-Pro-2.local, metadata={:polling_interval=>0.1, :queues=>"dc_development_*", :thread_pool_size=>2})
[SolidQueue] Claimed 1 jobs
/Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/mysql2-0.5.5/lib/mysql2/client.rb:151:in `_query': Lost connection to MySQL server during query (Mysql2::Error::ConnectionError)
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/mysql2-0.5.5/lib/mysql2/client.rb:151:in `block in query'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/mysql2-0.5.5/lib/mysql2/client.rb:150:in `handle_interrupt'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/mysql2-0.5.5/lib/mysql2/client.rb:150:in `query'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:100:in `block (2 levels) in raw_execute'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:1028:in `block in with_raw_connection'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.2/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:1000:in `with_raw_connection'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:98:in `block in raw_execute'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.2/lib/active_support/notifications/instrumenter.rb:58:in `instrument'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:1143:in `log'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:97:in `raw_execute'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:233:in `execute_and_free'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:23:in `internal_exec_query'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:630:in `select'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:71:in `select_all'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract/query_cache.rb:114:in `select_all'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:14:in `block in select_all'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:1028:in `block in with_raw_connection'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.2/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:1000:in `with_raw_connection'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:10:in `select_all'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/relation/calculations.rb:286:in `block in pluck'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/relation.rb:1003:in `skip_query_cache_if_necessary'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/relation/calculations.rb:282:in `pluck'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/app/models/solid_queue/queue_selector.rb:52:in `prefixed_names'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/app/models/solid_queue/queue_selector.rb:37:in `eligible_queues'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/app/models/solid_queue/queue_selector.rb:31:in `queue_names'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/app/models/solid_queue/queue_selector.rb:27:in `none?'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/app/models/solid_queue/queue_selector.rb:15:in `scoped_relations'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/app/models/solid_queue/ready_execution.rb:11:in `claim'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/worker.rb:35:in `block in poll'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/processes/poller.rb:16:in `with_polling_volume'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/worker.rb:34:in `poll'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/worker.rb:19:in `run'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/processes/runnable.rb:47:in `block in do_start_loop'
	from <internal:kernel>:187:in `loop'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/processes/runnable.rb:44:in `do_start_loop'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/processes/runnable.rb:39:in `start_loop'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/processes/runnable.rb:13:in `start'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:123:in `block in start_fork'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:122:in `fork'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:122:in `start_fork'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:152:in `replace_fork'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:134:in `block in reap_and_replace_terminated_forks'
	from <internal:kernel>:187:in `loop'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:130:in `reap_and_replace_terminated_forks'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:51:in `block in supervise'
	from <internal:kernel>:187:in `loop'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:47:in `supervise'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:26:in `start'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:14:in `start'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/tasks.rb:4:in `block (2 levels) in <main>'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `block in execute'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `each'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `execute'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:199:in `synchronize'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:199:in `invoke_with_call_chain'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:188:in `invoke'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:182:in `invoke_task'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `block (2 levels) in top_level'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `each'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `block in top_level'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:147:in `run_with_threads'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:132:in `top_level'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:83:in `block in run'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:208:in `standard_exception_handling'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:80:in `run'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/exe/rake:27:in `<top (required)>'
	from /Users/nimir/.rbenv/versions/3.3.0/bin/rake:25:in `load'
	from /Users/nimir/.rbenv/versions/3.3.0/bin/rake:25:in `<top (required)>'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/cli/exec.rb:58:in `load'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/cli/exec.rb:58:in `kernel_load'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/cli/exec.rb:23:in `run'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/cli.rb:451:in `exec'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/vendor/thor/lib/thor.rb:527:in `dispatch'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/cli.rb:34:in `dispatch'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/cli.rb:28:in `start'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/exe/bundle:28:in `block in <top (required)>'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/exe/bundle:20:in `<top (required)>'
	from /Users/nimir/.rbenv/versions/3.3.0/bin/bundle:25:in `load'
	from /Users/nimir/.rbenv/versions/3.3.0/bin/bundle:25:in `<main>'
[SolidQueue] Restarting fork[67633] (status: 1)
[SolidQueue] Starting Worker(pid=67637, hostname=Mohameds-MacBook-Pro-2.local, metadata={:polling_interval=>0.1, :queues=>"dc_development_*", :thread_pool_size=>2})
/Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/mysql2-0.5.5/lib/mysql2/client.rb:151:in `_query': Mysql2::Error::ConnectionError: Lost connection to MySQL server during query (ActiveRecord::ConnectionFailed)
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/mysql2-0.5.5/lib/mysql2/client.rb:151:in `block in query'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/mysql2-0.5.5/lib/mysql2/client.rb:150:in `handle_interrupt'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/mysql2-0.5.5/lib/mysql2/client.rb:150:in `query'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:100:in `block (2 levels) in raw_execute'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:1028:in `block in with_raw_connection'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.2/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:1000:in `with_raw_connection'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:98:in `block in raw_execute'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.2/lib/active_support/notifications/instrumenter.rb:58:in `instrument'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:1143:in `log'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:97:in `raw_execute'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:233:in `execute_and_free'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:23:in `internal_exec_query'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:630:in `select'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:71:in `select_all'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract/query_cache.rb:114:in `select_all'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:14:in `block in select_all'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:1028:in `block in with_raw_connection'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.2/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:1000:in `with_raw_connection'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:10:in `select_all'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/relation/calculations.rb:286:in `block in pluck'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/relation.rb:1003:in `skip_query_cache_if_necessary'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/relation/calculations.rb:282:in `pluck'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/app/models/solid_queue/queue_selector.rb:65:in `paused_queues'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/app/models/solid_queue/queue_selector.rb:31:in `queue_names'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/app/models/solid_queue/queue_selector.rb:27:in `none?'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/app/models/solid_queue/queue_selector.rb:15:in `scoped_relations'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/app/models/solid_queue/ready_execution.rb:11:in `claim'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/worker.rb:35:in `block in poll'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/processes/poller.rb:16:in `with_polling_volume'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/worker.rb:34:in `poll'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/worker.rb:19:in `run'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/processes/runnable.rb:47:in `block in do_start_loop'
	from <internal:kernel>:187:in `loop'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/processes/runnable.rb:44:in `do_start_loop'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/processes/runnable.rb:39:in `start_loop'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/processes/runnable.rb:13:in `start'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:123:in `block in start_fork'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:122:in `fork'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:122:in `start_fork'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:152:in `replace_fork'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:134:in `block in reap_and_replace_terminated_forks'
	from <internal:kernel>:187:in `loop'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:130:in `reap_and_replace_terminated_forks'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:51:in `block in supervise'
	from <internal:kernel>:187:in `loop'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:47:in `supervise'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:26:in `start'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/supervisor.rb:14:in `start'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.1.2/lib/solid_queue/tasks.rb:4:in `block (2 levels) in <main>'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `block in execute'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `each'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `execute'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:199:in `synchronize'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:199:in `invoke_with_call_chain'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:188:in `invoke'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:182:in `invoke_task'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `block (2 levels) in top_level'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `each'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `block in top_level'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:147:in `run_with_threads'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:132:in `top_level'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:83:in `block in run'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:208:in `standard_exception_handling'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:80:in `run'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/exe/rake:27:in `<top (required)>'
	from /Users/nimir/.rbenv/versions/3.3.0/bin/rake:25:in `load'
	from /Users/nimir/.rbenv/versions/3.3.0/bin/rake:25:in `<top (required)>'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/cli/exec.rb:58:in `load'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/cli/exec.rb:58:in `kernel_load'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/cli/exec.rb:23:in `run'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/cli.rb:451:in `exec'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/vendor/thor/lib/thor.rb:527:in `dispatch'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/cli.rb:34:in `dispatch'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/cli.rb:28:in `start'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/exe/bundle:28:in `block in <top (required)>'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
	from /Users/nimir/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/exe/bundle:20:in `<top (required)>'
	from /Users/nimir/.rbenv/versions/3.3.0/bin/bundle:25:in `load'
	from /Users/nimir/.rbenv/versions/3.3.0/bin/bundle:25:in `<main>'

EXC_BAD_ACCESS on MacOS 14.3

Testing this out as a possible resque migration and getting a recurring crash in what seems to be the spawning of the various job threads.

I'm running on Ruby 3.2.2, postgres 15.5.
Laptop is an M1 MBPro

The crash generates an IPS file which I have - but I'm not sure if uploading that here could potentially leak sensitive data so holding off on that.

A snippet of the file:

"exception" : {"codes":"0x0000000000000001, 0x0000000104838ac0","rawCodes":[1,4370696896],"type":"EXC_BAD_ACCESS","signal":"SIGABRT","subtype":"KERN_INVALID_ADDRESS at 0x0000000104838ac0"},

Anyone seeing anything similar?

Support for unique jobs

We need this feature, but I'm still not sure what it'll look like for Solid Queue. We have two use cases for it that couldn't be more different 😅 :

  • Prevent identical jobs from being enqueued together, keeping just one. In this case, when a job starts running, we want to allow other identical jobs to be enqueued right away. The uniqueness constraint would only apply while the jobs are waiting to be run. It wouldn't apply to scheduled jobs, we could have identical jobs scheduled, and if they run at different times, they'd be allowed to do so. Nope! I realised this is not necessarily true for our use case. We could have a restriction that applied to scheduled jobs and jobs waiting to be run, but that would have to be lifted as soon as jobs are ready to run. This restriction could apply to the solid_queue_ready_executions table alone. A new uniqueness_key with a unique index would work for this case.
  • Truly unique jobs: identical jobs are completely prevented from existing in the system, even after a job has already run (for the time jobs are preserved in the system, which depends on clear_finished_jobs_after). This restriction would apply to the solid_queue_jobs table. A uniqueness_key with a unique index would work in this case. I'd like this feature for #104, to prevent multiple jobs being enqueued for the same recurring task at a given time.

I'd like a common way to support both, but that might be tricky as it also needs to be performant. If I end up with two different implementations, they should be different enough not to be confusing. I could also reframe the second case, and instead of making it part of unique jobs, make it part of the implementation for cron jobs. They are different enough to grant that distinction.

After realising that the first case can work with the jobs table too, because all we need is to lift the restriction when a job is moved to ready, I think there's a good path for a common solution 🤔

The program executes two sets, is it normal?

image

Should only one group be started as configured?
A supervisor, a dispatcher, and a worker.

If only one group is needed, is it possible to use the kill command and just stop it?

solid_queue.yml file:

default: &default
  dispatchers:
    - polling_interval: 1
      batch_size: 500
      concurrency_maintenance_interval: 300
  workers:   
    - queues: "chrome_single"
      threads: 1
      processes: 1
      polling_interval: 1


development:
 <<: *default

test:
 <<: *default

production:
 <<: *default

Solid Queue is not retrying job

I am trying to retry a failed job by using retry_on in the job class but it is not working consistently.

class RequestTestimonialJob < ApplicationJob
  queue_as :default
  retry_on StandardError, attempts: 3, priority: 0
  
  def perform(user_id)
    raise StandardError
  end
end

I have also overridden

config.solid_queue.on_thread_error = ->(exception) { Bugsnag.notify(exception) }

but nothing happens. I have tried with and without it. The behaviour seems unpredictable or it is just me.

Also throws this error sometimes,

No live threads left. Deadlock? (fatal)
04:53:43 solid.1 | 6 threads, 6 sleeps current:0x0000000113428570 main thread:0x0000000124004080
04:53:43 solid.1 | * #<Thread:0x0000000122834140 sleep_forever>
04:53:43 solid.1 |    rb_thread_t:0x0000000124004080 native:0x00000001f4bd1e00 int:0
04:53:43 solid.1 |    
04:53:43 solid.1 | * #<Thread:0x00000001234f93d0@DEBUGGER__::SESSION@server /Users/zain/.rvm/gems/ruby-3.0.0@devtree/gems/debug-1.7.2/lib/debug/session.rb:179 sleep_forever>
04:53:43 solid.1 |    rb_thread_t:0x00000001118b04c0 native:0x000000016e587000 int:0
04:53:43 solid.1 |    
04:53:43 solid.1 | * #<Thread:0x00000001234e8968@worker-1 /Users/zain/.rvm/gems/ruby-3.0.0@devtree/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:332 sleep_forever>
04:53:43 solid.1 |    rb_thread_t:0x00000001118b0db0 native:0x000000016e793000 int:0
04:53:43 solid.1 |    
04:53:43 solid.1 | * #<Thread:0x0000000123543d68 /Users/zain/.rvm/gems/ruby-3.0.0@devtree/gems/activerecord-7.0.4.3/lib/active_record/connection_adapters/abstract/connection_pool/reaper.rb:40 sleep_forever>
04:53:43 solid.1 |    rb_thread_t:0x0000000113428570 native:0x000000016e99f000 int:0
04:53:43 solid.1 |    
04:53:43 solid.1 | * #<Thread:0x0000000123b86dd8@worker-1 /Users/zain/.rvm/gems/ruby-3.0.0@devtree/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:332 sleep_forever>
04:53:43 solid.1 |    rb_thread_t:0x0000000113580970 native:0x000000016ebab000 int:0 mutex:0x0000000124004500 cond:1
04:53:43 solid.1 |    
04:53:43 solid.1 | * #<Thread:0x0000000123b61b28@io-worker-1 /Users/zain/.rvm/gems/ruby-3.0.0@devtree/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:332 sleep_forever>
04:53:43 solid.1 |    rb_thread_t:0x0000000114c25ae0 native:0x000000016edb7000 int:0 mutex:0x0000000113580bf0 cond:1

How can I make sure the job is retried after any exception? any help would be appreciated.

Thanks

Code reload in development throws an error

When I try running solid queue locally in development evnironment it works fine.
I start worker by rake solid_queue:start
Then I change something in the code (in ApplicationJob class to be precise) and run some job without restarting the workers, the error is thrown:

/home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/worker.rb:35:in `block in poll': uninitialized constant SolidQueue::ReadyExecution (NameError)

          SolidQueue::ReadyExecution.claim(queues, pool.idle_threads, process.id)
                    ^^^^^^^^^^^^^^^^
Did you mean?  SolidQueue::FailedExecution
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/processes/poller.rb:14:in `block in with_polling_volume'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/activesupport-7.1.2/lib/active_support/logger_silence.rb:18:in `block in silence'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/activesupport-7.1.2/lib/active_support/logger_thread_safe_level.rb:45:in `log_at'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/activesupport-7.1.2/lib/active_support/logger_silence.rb:18:in `silence'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/processes/poller.rb:14:in `with_polling_volume'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/worker.rb:34:in `poll'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/worker.rb:19:in `run'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/processes/runnable.rb:47:in `block in do_start_loop'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/processes/runnable.rb:44:in `loop'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/processes/runnable.rb:44:in `do_start_loop'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/processes/runnable.rb:39:in `start_loop'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/processes/runnable.rb:13:in `start'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/supervisor.rb:123:in `block in start_fork'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/supervisor.rb:122:in `fork'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/supervisor.rb:122:in `start_fork'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/supervisor.rb:72:in `block in start_forks'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/supervisor.rb:72:in `each'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/supervisor.rb:72:in `start_forks'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/supervisor.rb:45:in `supervise'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/supervisor.rb:26:in `start'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/supervisor.rb:14:in `start'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/solid_queue-0.1.1/lib/solid_queue/tasks.rb:4:in `block (2 levels) in <main>'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `block in execute'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `each'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `execute'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:199:in `synchronize'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:199:in `invoke_with_call_chain'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:188:in `invoke'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:182:in `invoke_task'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `block (2 levels) in top_level'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `each'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `block in top_level'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:147:in `run_with_threads'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:132:in `top_level'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:83:in `block in run'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:208:in `standard_exception_handling'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:80:in `run'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rake-13.1.0/exe/rake:27:in `<top (required)>'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/bin/rake:25:in `load'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/bin/rake:25:in `<top (required)>'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/site_ruby/3.2.0/bundler/cli/exec.rb:58:in `load'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/site_ruby/3.2.0/bundler/cli/exec.rb:58:in `kernel_load'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/site_ruby/3.2.0/bundler/cli/exec.rb:23:in `run'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/site_ruby/3.2.0/bundler/cli.rb:451:in `exec'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/site_ruby/3.2.0/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/site_ruby/3.2.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/site_ruby/3.2.0/bundler/vendor/thor/lib/thor.rb:527:in `dispatch'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/site_ruby/3.2.0/bundler/cli.rb:34:in `dispatch'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/site_ruby/3.2.0/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/site_ruby/3.2.0/bundler/cli.rb:28:in `start'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.5.1/exe/bundle:28:in `block in <top (required)>'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/site_ruby/3.2.0/bundler/friendly_errors.rb:117:in `with_friendly_errors'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.5.1/exe/bundle:20:in `<top (required)>'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/bin/bundle:25:in `load'
	from /home/wojtek/.asdf/installs/ruby/3.2.2/bin/bundle:25:in `<main>'
[SolidQueue] Restarting fork[785525] (status: 1)
[SolidQueue] Starting Worker(pid=785589, hostname=arda, metadata={:polling_interval=>1, :queues=>"urgent,default", :thread_pool_size=>3})
[SolidQueue] Dispatched scheduled batch with 1 jobs

The jobs are still processed, however the error is thrown from time to time.

I believe it might be some issue with class reloading in development.
I'm using this config with Rails 7.1.2

development:
  dispatchers:
    - polling_interval: 1
      batch_size: 500
  workers:
    - queues:
        - urgent
        - default
      processes: 1
      threads: 3
      polling_interval: 1

Not compatible with strict loading by default

It looks like some of the queries aren't compatible with a strict_loading_by_default configuration.

With this configuration in application.rb, solid_queue doesn't work.

config.active_record.strict_loading_by_default = true

I will try to find some time to dig into his and open a fix.

SEGV Fault error when starting worker

I'm running into the following SEGV error when trying to boot up the solid_queue worker. I have suspicions that it's related to the mac os version because a colleague of mine has an m2 on 13.6.3 and it boots fine. I'm not sure if this is the place to post it, but this error only comes up with this gem so far. I haven't had any other issues.

rails version 7.1.3
ruby version 3.1.4 (rbenv)
m2 macbook pro
Sonoma 14.2.1

bundle exec rake solid_queue:start
DEPRECATION WARNING: DeprecatedConstantAccessor.deprecate_constant without a deprecator is deprecated (called from require at <internal:/Users/jm_mbp/.rbenv/versions/3.2.2/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:37)
warning: parser/current is loading parser/ruby32, which recognizes 3.2.3-compliant syntax, but you are running 3.2.2.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
[SolidQueue] Starting Dispatcher(pid=10816, hostname=Justins-MacBook-Pro.local, metadata={:polling_interval=>1, :batch_size=>500})
[SolidQueue] Starting Worker(pid=10817, hostname=Justins-MacBook-Pro.local, metadata={:polling_interval=>0.1, :queues=>"*", :thread_pool_size=>5})
/Users/jm_mbp/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/pg-1.5.4/lib/pg/connection.rb:690: [BUG] Segmentation fault at 0x0000000105ef0ace
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin23]

-- Crash Report log information --------------------------------------------
   See Crash Report log file in one of the following locations:
     * ~/Library/Logs/DiagnosticReports
     * /Library/Logs/DiagnosticReports
   for more details.
Don't forget to include the above Crash Report log file in bug reports.

-- Control frame information -----------------------------------------------
c:0116 p:---- s:0700 e:000699 CFUNC  :connect_poll
c:0115 p:0358 s:0696 e:000695 METHOD /Users/jm_mbp/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/pg-1.5.4/lib/pg/connection.rb:690
c:0114 p:0265 s:0683 e:000682 METHOD /Users/jm_mbp/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/pg-1.5.4/lib/pg/connection.rb:824
c:0113 p:0007 s:0672 e:000671 METHOD /Users/jm_mbp/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/pg-1.5.4/lib/pg/connection.rb:759
c:0112 p:0012 s:0666 e:000665 METHOD /Users/jm_mbp/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/pg-1.5.4/lib/pg.rb:63
c:0111 p:0006 s:0660 e:000659 METHOD /Users/jm_mbp/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/postgresq
c:0110 p:0008 s:0654 SEGV received in SEGV handler
e:000653 METHOD /Users/jm_mbp/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/postgresq
c:0109 p:0020 s:0649 e:000648 METHOD /Users/jm_mbp/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/postgresq
c:0108 p:0004 s:0645 e:000644 BLOCK  /Users/jm_mbp/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/abstract_
c:0107 p:0002 s:0639 e:000638 METHOD /Users/jm_mbp/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/activesupport-7.1.3/lib/active_support/concurrency/null_lock.rb:9
c:0106 p:0031 [SolidQueue] Restarting fork[10816] (status: )
[SolidQueue] Starting Dispatcher(pid=10824, hostname=Justins-MacBook-Pro.local, metadata={:polling_interval=>1, :batch_size=>500})
/Users/jm_mbp/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/pg-1.5.4/lib/pg/connection.rb:690: [BUG] Segmentation fault at 0x0000000105ef0ace
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin23]

-- Crash Report log information --------------------------------------------
   See Crash Report log file in one of the following locations:
     * ~/Library/Logs/DiagnosticReports
     * /Library/Logs/DiagnosticReports                           SEGV received in SEGV handler

   for more details.                 [SolidQueue] Restarting fork[10824] (status: )
[SolidQueue] Starting Dispatcher(pid=10825, hostname=Justins-MacBook-Pro.local, metadata={:polling_interval=>1, :batch_size=>500})
/Users/jm_mbp/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/pg-1.5.4/lib/pg/connection.rb:690: [BUG] Segmentation fault at 0x0000000105ef0ace
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin23]

-- Crash Report log information --------------------------------------------
   See Crash Report log file in one of the following locations:
     * ~/Library/Logs/DiagnosticReports
     * /Library/Logs/DiagnosticReports
   for more details.
Don't forget to include the above Crash Report log file in bug reports.

Thread.abort_on_exception = true is causing worker to be restarted even when it's handled in job

So I'm not sure what the intended behaviour is but I think what I'm seeing is very odd.

If you have a simple job like so:

module Contracts
  class UpdateDetailsJob < ApplicationJob
    def perform
      t = Thread.new do
        sleep 2
        raise "error!"
      end
      t.abort_on_exception = true
      t.join
    rescue => ex
      puts "Error caught! - #{ex.message}"
    end
  end
end

I see the following happen:

07:35:51 solid_queue.1      | #<Thread:0x000000012f5354c8 /.../contracts/update_details_job.rb:10 run> terminated with exception (report_on_exception is true):
07:35:51 solid_queue.1      | /.../contracts/update_details_job.rb:12:in `block in perform': error! (RuntimeError)
07:35:51 solid_queue.1      | Error caught! - error!
07:35:51 solid_queue.1      | /.../contracts/update_details_job.rb:12:in `block in perform': error! (RuntimeError)
07:35:51 solid_queue.1      | [SolidQueue] Restarting fork[62533] (status: 1)
07:35:51 solid_queue.1      | [SolidQueue] Starting Worker(pid=62588, hostname=local, metadata={:polling_interval=>0.1, :queues=>"contracts", :thread_pool_size=>5})

Specifically the part about Restarting the fork and the worker immediately after.

Given that I'm handling the exception (and indeed it's outputting the "Error caught"), why is ithe worker thread getting restarted? If that worker is handling multiple threads of work for other jobs, that's not very desirable.

Should also add, if you keep t.abort_on_exception = false then there's no restarting of the worker.

Running out of connections

Summary

I've been getting a couple of connection errors in production. I have about 110-120 free connection on the database, but SolidQueue is reporting it couldn't get a connection.

Here's the error in question:

2024-01-29 14:28:07.568 [optonal_prod] worker.1 [SolidQueue] could not obtain a connection from the pool within 5.000 seconds (waited 5.081 seconds); all pooled connections were in use

Here's my config/database.yml

default: &default
  adapter: postgresql
  encoding: unicode
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  timeout: 5000

development:
  <<: *default
  database: optonal_development
  username: <%= 'postgres' if File.exist?('/.dockerenv') %>
  password:
  host: <%= 'db' if File.exist?('/.dockerenv') %>

test:
  <<: *default
  database: optonal_test
  username: <%= ENV.fetch('POSTGRES_USER', '') %>
  password: <%= ENV.fetch('POSTGRES_PASSWORD', '') %>
  host: <%= ENV.fetch('CI', '').empty? ? '' : 'postgres' %>

production:
  <<: *default
  database: optonal_production

Here's my config/solidqueue.yml

default: &default
   dispatchers:
     - polling_interval: 1
       batch_size: 500
   workers:
     - queues: [ real_time, background ]
       threads: 5
       processes: 1
       polling_interval: 0.1

development:
 <<: *default

test:
 <<: *default

production:
 <<: *default

Proposal

Document which connection pool SolidQueue uses to connect to the database. Document how many connections does SolidQueue need (e.g. threads * processes).

Missing GUI

I don't see any reference to access a GUI interface for this library. Is this just missing from the documentation? I also skimmed through the codebase and didn't see anything regarding a GUI user interface. Perhaps if it's still in the works, it would be great to still create a GUI interface section in the readme and state this it's a work in progress?

migration path not respected when setting up solid_queue on separate database?

Firstly, thank you for all the work on solid queue so far!

Apologies if this is lack of understanding and user-error on my part, but when the docs mention supporting solid queue on a different database than the main app (via connects_to), does that mean that the install/migration commands should automatically pick up on the settings I've defined in my app's config/database.yml for the given connects_to database key, or do I still need to manually tell solid queue which one to use when running those?

Currently, with the config and database.yml settings listed below for a separate :solid_queue db, when I run bin/rails generate solid_queue:install or bin/rails solid_queue:install:migrations, the migration files for the SolidQueue tables are copied to the db/migrate folder, resulting in the tables getting added to the :primary db instead of the :solid_queue one. However, I would have expected the install to copy them to db/solid_queue_migrate, to match the migrations_paths defined in database.yml for the solid_queue db)

# config/application.rb

config.active_job.queue_adapter = :solid_queue
config.solid_queue.connects_to = { database: { writing: :solid_queue, reading: :solid_queue } }
# config/database.yml

default: &default
  adapter: postgresql
  encoding: unicode
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

solid_queue: &solid_queue
  <<: *default
  migrations_paths: db/solid_queue_migrate

development:
  primary:
    <<: *default
    database: queue_demo_development
  solid_queue:
    <<: *solid_queue
    database: queue_demo_solid_queue_development

test:
  primary:
    <<: *default
    database: queue_demo_test

production:
  primary:
    <<: *default
  solid_queue:
    <<: *solid_queue
    url: <%= ENV["SOLID_QUEUE_DATABASE_URL"] %>

If I manually pass in the DATABASE as an arg to the migration command (bin/rails solid_queue:install:migrations DATABASE=solid_queue), then things do work as expected, with files copied to db/solid_queue_migrate and added to the solid_queue db). If this is how it's meant to work, maybe that can be made a little clearer in the docs. Thanks!

Document how to retry jobs

To retry a job, one has to configure it by hand, in example by adding retry_on with some logic.
I believe it is very similar to GoodJob retries, where it's documented on https://github.com/bensheldon/good_job/tree/50199f9bfcaca1c23ae7373efe781c1862dcc0cb#retries

Also a note how to report an error on retry would be very useful. Currently only the final error (the one that no longer is retried) is reported via on_thread_error.

I could try to prepare PR, but I'm very curious how to do it in your production apps

Planned batch support?

First off, congrats on the initial release! I'm really excited to start giving this a try - it includes a very robust feature set out of the gate which is amazing!

The docs mention features coming very soon:

Proper support for perform_all_later, improvements to logging and instrumentation, a better CLI tool, a way to run within an existing process in "async" mode, unique jobs and recurring, cron-like tasks are coming very soon.

Have you had any internal discussions or thoughts on batch support? By batch support I mean like GoodJob batches (https://github.com/bensheldon/good_job?tab=readme-ov-file#batches) and Sidekiq Batches (https://github.com/sidekiq/sidekiq/wiki/Batches). They're a huge benefit for job coordination.

I'd be very interested in contributing something like this, if there were no plans for it!

[idea] implementing rate limiting per queue?

Great release! Looking forward to trying this in tandem with Mission Control in 2024!

I had a question about concurrency and how it might play with uniqueness and rate-limits for things like external APIs..

Whats the recommended way for dealing with external APIs which impose a rate limit (e.g. 5 requests per second) with solid_queue?

The concurrency controls with key is great btw, I'm often concerned about two workers writing to the same record concurrently so this is super helpful :)

User must exist error when trying to sync

I am getting the following error when trying to sync or getting webhook to save subscriptions in the jobs

image

Validation failed: User must exist

My User model looks like this

class User < ApplicationRecord
  pay_customer default_payment_processor: :stripe, stripe_attributes: :stripe_attributes

  has_secure_password

  validates :email, presence: true, uniqueness: true
  validates :password, presence: true

  normalizes :name, with: -> { _1.strip }
  normalizes :email, with: -> { _1.downcase.strip }

  generates_token_for :password_reset, expires_in: 30.minutes do
    password_salt&.last(10)
  end

  generates_token_for :email_confirmation, expires_in: 30.minutes do
    email
  end

  def stripe_attributes(pay_customer)
    {
      metadata: {
        pay_customer_id: pay_customer.id,
        user_id: pay_customer.owner_id
      }
    }
  end

  def has_active_subscription?
    subscriptions.any?(&:active?)
  end
end

Not sure what is the reason as debugging it fails on this line https://github.com/pay-rails/pay/blob/4344796a3986621cf539bb28a7ca3a13d5550cd1/lib/pay/stripe/subscription.rb#L124

The checkout session generated is as such and shown on the pricing page.

class SubscriptionsController < ApplicationController
  before_action :authenticate_user!

  def plans
    current_user.pay_customers
    @pay_per_use = current_user.payment_processor.checkout(
      mode: 'payment',
      line_items: "price_...",
    )

    @monthly_plan = current_user.payment_processor.checkout(
      mode: 'subscription',
      line_items: "price_...",
    )

    @annual_plan = current_user.payment_processor.checkout(
      mode: 'subscription',
      line_items: "price_...",
    )
  end
end

In Development

Hi,

I've added a job in development and I have jobs: bundle exec rake solid_queue:start in my Procfile.dev but it doesn't run it?

What must I be doing wrong?

undefined method `log_writer' for #<Puma::Launcher on using plugin :solid_queue in puma.rb

(gemfile)

ruby "3.0.3"
gem "rails", "~> 7.0.2", ">= 7.0.2.4
gem "solid_queue", require: true || gem "solid_queue" (tried both)

puma.rb

max_threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 }
min_threads_count = ENV.fetch("RAILS_MIN_THREADS") { max_threads_count }
threads min_threads_count, max_threads_count

worker_timeout 3600 if ENV.fetch("RAILS_ENV", "development") == "development"

port ENV.fetch("PORT") { 3000 }

environment ENV.fetch("RAILS_ENV") { "development" }

pidfile ENV.fetch("PIDFILE") { "tmp/pids/server.pid" }

plugin :solid_queue

Error

/home/moavia/.rvm/gems/ruby-3.0.3/gems/solid_queue-0.1.2/lib/puma/plugin/solid_queue.rb:7:in start': undefined method log_writer' for #<Puma::Launcher:0x000055dac33b1ae8 @runner=#<Puma::Single:0x000055dac3512360 @launcher=#<Puma::Launcher:0x000055dac33b1ae8 ...>, @events=#<Puma::Events:0x000055dac2edd710 @Formatter=#Puma::Events::DefaultFormatter:0x000055dac2edc8d8, @stdout=#<IO:>, @stderr=#<IO:>, @debug=false, @error_logger=#<Puma::ErrorLogger:0x000055dac2ed7e28 @ioerr=#<IO:>, @debug=false>, @hooks={}>, @options=#<Puma::UserFileDefaultOptions:0x000055dac2ebf030 @user_options={:early_hints=>true, :environment=>"development"}, @file_options={:config_files=>["config/puma.rb"], :min_threads=>5, :max_threads=>5, :worker_timeout=>3600, :binds=>["tcp://0.0.0.0:3000"], :environment=>"development", :pidfile=>"tmp/pids/server.pid"}, @default_options={:min_threads=>0, :max_threads=>5, :log_requests=>false, :debug=>false, :binds=>["tcp://0.0.0.0:9292"], :workers=>0, :silence_single_worker_warning=>false, :mode=>:http, :worker_check_interval=>5, :worker_timeout=>60, :worker_boot_timeout=>60, :worker_shutdown_timeout=>30, :worker_culling_strategy=>:youngest, :remote_address=>:socket, :tag=>"rails_acb", :environment=>"development", :rackup=>"config.ru", :logger=>#<IO:>, :persistent_timeout=>20, :first_data_timeout=>30, :raise_exception_on_sigterm=>true, :max_fast_inline=>10, :io_selector_backend=>:auto, :mutate_stdout_and_stderr_to_sync_on_write=>true, :preload_app=>false}>, @app=nil, @control=nil, @started_at=2024-03-07 01:56:40.02002962 +0500, @wakeup=nil>, @events=#<Puma::Events:0x000055dac2edd710 @Formatter=#Puma::Events::DefaultFormatter:0x000055dac2edc8d8, @stdout=#<IO:>, @stderr=#<IO:>, @debug=false, @error_logger=#<Puma::ErrorLogger:0x000055dac2ed7e28 @ioerr=#<IO:>, @debug=false>, @hooks={}>, @argv=["-C", "config/puma.rb", "--early-hints", "-e", "development"], @original_argv=["-C", "config/puma.rb", "--early-hints", "-e", "development"], @config=#<Puma::Configuration:0x000055dac2ed4d90 @options=#<Puma::UserFileDefaultOptions:0x000055dac2ebf030 @user_options={:early_hints=>true, :environment=>"development"}, @file_options={:config_files=>["config/puma.rb"], :min_threads=>5, :max_threads=>5, :worker_timeout=>3600, :binds=>["tcp://0.0.0.0:3000"], :environment=>"development", :pidfile=>"tmp/pids/server.pid"}, @default_options={:min_threads=>0, :max_threads=>5, :log_requests=>false, :debug=>false, :binds=>["tcp://0.0.0.0:9292"], :workers=>0, :silence_single_worker_warning=>false, :mode=>:http, :worker_check_interval=>5, :worker_timeout=>60, :worker_boot_timeout=>60, :worker_shutdown_timeout=>30, :worker_culling_strategy=>:youngest, :remote_address=>:socket, :tag=>"rails_acb", :environment=>"development", :rackup=>"config.ru", :logger=>#<IO:>, :persistent_timeout=>20, :first_data_timeout=>30, :raise_exception_on_sigterm=>true, :max_fast_inline=>10, :io_selector_backend=>:auto, :mutate_stdout_and_stderr_to_sync_on_write=>true, :preload_app=>false}>, @plugins=#<Puma::PluginLoader:0x000055dac2ebe720 @instances=[#<#Class:0x000055dac3513a30:0x000055dac35134b8>]>, @user_dsl=#<Puma::DSL:0x000055dac2ebe1a8 @config=#<Puma::Configuration:0x000055dac2ed4d90 ...>, @options={:early_hints=>true

Solid_queue bug in windows

i used rails 7.1 and added solid_queue to my project.
I run rails solid_queue:install:migrations
I created inside de config folder solid_queue.yml then bundle exec rake solid_queue:start and i got this error

image

image

Make YAML Configuration and Rails Config Equivalent

For matt_accessor's that are settable, I am unable to set some of these in config.yml.

For example, silence_polling in development:

development:
  <<: *default
  silence_polling: true

I'm unable to find the exact problem but happy to put in a fix if someone can point me in the right direction. Maybe here, but I don't see how is this line called?

How to make solid_queue work well with fibers?

We make quite a few LLM calls in our background jobs and want to leverage fibers to efficiently use the resources as these jobs spend most of their time waiting for HTTP responses.

I have attempted to make it work using async gem
But it sometimes get into a deadlock (my guess) and stops process the jobs

# config/initializers/solid_queue.rb

require "async"

module SolidQueue
  module AsyncableWorker
    extend ActiveSupport::Concern

    def initialize(**options)
      super
      @async = options.fetch(:async, false)
      @pool = Pool.new(options[:threads], on_idle: -> { wake_up }, async: @async)
    end

    def do_start_loop
      if @async
        Async do
          super
        end
      else
        super
      end
    end
  end

  module AsyncablePool
    extend ActiveSupport::Concern

    def initialize(size, on_idle: nil, async: false)
      super(size, on_idle: on_idle)
      @async = async
    end

    def post(execution)
      if @async
        available_threads.decrement
        Async do
          wrap_in_app_executor do
            execution.perform
          rescue => error
            handle_thread_error(error)
          ensure
            available_threads.increment
            mutex.synchronize { on_idle.try(:call) if idle? }
          end
        end
      else
        super
      end
    end
  end
end

SolidQueue::Pool.class_eval do
  prepend SolidQueue::AsyncablePool
end

SolidQueue::Worker.class_eval do
  prepend SolidQueue::AsyncableWorker
end

NOTE: See the newly introduced async option in workers which enables fiber usage in above initialiser code

#config/solid_queue.yml

# The supervisor forks workers and dispatchers according to the configuration, controls their heartbeats, and sends them signals to stop and start them when needed.
default: &default
  # Dispatchers are in charge of selecting jobs scheduled to run in the future that are due and dispatching them,
  # which is simply moving them from the solid_queue_scheduled_executions table over to the solid_queue_ready_executions table so that workers can pick them up.
  # They also do some maintenance work related to concurrency controls.
  dispatchers:
    - polling_interval: 1 # seconds
      batch_size: 500
      concurrency_maintenance_interval: 600 # seconds before checking if blocked jobs can be unblocked
  # Workers are in charge of picking jobs ready to run from queues and processing them.
  # They work off the solid_queue_ready_executions table.
  workers:
    - queues: [critical, default, lowpriority, oneoff]
      async: <%= ENV['SOLID_QUEUE_DISABLE_ASYNC_WORKERS'] != 'true' %> # whether to use async workers (fibers) or not (threads)
      threads: 5 # thread/fiber pool max size. Also used as batch size when fetching jobs.
      processes: <%= (ENV['SOLID_QUEUE_WORKER_PROCESS_COUNT'] || Concurrent.processor_count).to_i %>
      polling_interval: 0.1 # seconds

And start the process regularly bundle exec rake solid_queue:start


Has anyone else attempted similar or know what the issue could be?

Index size exceeded

Not sure what is happening here, but I started getting this today on my development environment.

PG::ProgramLimitExceeded: ERROR: index row size 3056 exceeds btree version 4 maximum 2704 for index "solid_error_uniqueness_index" DETAIL: Index row references tuple (2,15) in relation "solid_errors". HINT: Values larger than 1/3 of a buffer page cannot be indexed. Consider a function index of an MD5 hash of the value, or use full text indexing.

Failed executions are not logged

I was trying to create a reproducible bug report script for this (which led to #106), but in the meantime, let me just write this up.

Nothing is logged when a job raises an exception. I tested this with a simple job:

class RaisingJob < ApplicationJob
  def perform
    raise StandardError
  end
end

In the logs, all I see is:

20:43:35 queue.1 | [SolidQueue] Claimed 1 jobs

I can see the SolidQueue::FailedExecution object:

SolidQueue::FailedExecution.last
=>
#<SolidQueue::FailedExecution:0x0000000128493a58
 id: 17,
 job_id: 26,
 error:
  {"exception_class"=>"StandardError",
   "message"=>"StandardError",
   "backtrace"=>[...]},
 created_at: Sat, 30 Dec 2023 19:43:35.882895000 UTC +00:00>

deregistering process issues?

After running 'bundle exec rake solid_queue:start' for a while and stopping it with Ctrl-C, I noticed that the record of stopped processes in the 'solid_queue_processes' table was not deleted.

Upon restarting, I received an error message stating 'deregistering process 1 - {}' and the corresponding record still existed in the table.

Removing jobs from the solid_queue_jobs table?

Hi,

Hopefully a simple question - what would happen if you remove a job intentionally that was just picked up for processing?

If you apply a lock on the row before removing it, would that prevent solid_queue from working on it?

How would you kill a specific job?

Scenario would be a long-running job that is taking too long and the user wishes to kill it and not have it restarted.

If I were to send the TERM signal to the supervisor pid, I've noticed it has this weird side effect of restarting everything (not just solid_queue) in my procfile (when using foreman).

I also noticed if I were send a TERM signal to the worker (assuming it was a 1 thread/1 process worker), then the worker would get restarted and pick up the same job again.

I suppose it's possible to modify the Job in the solid_queue_jobs table such that its finished_at is set then send the TERM signal to the worker but that seems hack-ish.

Also, what if the worker has 5 threads and they're all processing jobs that I don't want to kill?

Would appreciate some direction on this, thanks!

Error when running Solid Queue's supervisor together with Puma

When I start up my rails server in development mode after having added the plugin :solid_queue line to my config/puma.rb file I get the following errors in the output when running Puma:

objc[56839]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[56839]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.

This doesn't appear to interfere with Puma serving local web requests, but if I kick off a background job it does not get processed. However, if I remove the plugin config line from the Puma file and then run Solid Queue as a separate process, the Puma error goes away when starting the server and Solid Queue picks up background jobs and processes them correctly.

My local dev environment: M2 MacBook running Ventura.

ActiveRecord::StatementInvalid: SQLite3::BusyException: database is locked

I was trying to create a sample application with the solid_queue gem, and it seems like the single case failing when we use the perform_later.

Here is the attached repo for the same, https://github.com/anoobbava/solid_queue_sample
I have received the error as below,

/home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/sqlite3-1.7.0-x86_64-linux/lib/sqlite3/statement.rb:108:in step': database is locked (SQLite3::BusyException)
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/sqlite3-1.7.0-x86_64-linux/lib/sqlite3/statement.rb:108:in block in each' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/sqlite3-1.7.0-x86_64-linux/lib/sqlite3/statement.rb:107:in loop'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/sqlite3-1.7.0-x86_64-linux/lib/sqlite3/statement.rb:107:in each' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/sqlite3/database_statements.rb:42:in to_a'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/sqlite3/database_statements.rb:42:in block (2 levels) in internal_exec_query' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/abstract_adapter.rb:1028:in block in with_raw_connection'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activesupport-7.1.3/lib/active_support/concurrency/null_lock.rb:9:in synchronize' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/abstract_adapter.rb:1000:in with_raw_connection'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/sqlite3/database_statements.rb:33:in block in internal_exec_query' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activesupport-7.1.3/lib/active_support/notifications/instrumenter.rb:58:in instrument'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/abstract_adapter.rb:1143:in log' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/sqlite3/database_statements.rb:32:in internal_exec_query'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/sqlite3/database_statements.rb:61:in exec_delete' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/abstract/database_statements.rb:208:in delete'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/abstract/query_cache.rb:25:in delete' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/persistence.rb:624:in _delete_record'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/persistence.rb:1198:in _delete_row' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/persistence.rb:1194:in destroy_row'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/counter_cache.rb:197:in destroy_row' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/locking/optimistic.rb:125:in destroy_row'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/persistence.rb:783:in destroy' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/callbacks.rb:423:in block in destroy'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activesupport-7.1.3/lib/active_support/callbacks.rb:110:in run_callbacks' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activesupport-7.1.3/lib/active_support/callbacks.rb:952:in _run_destroy_callbacks'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/callbacks.rb:423:in destroy' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/transactions.rb:305:in block in destroy'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/transactions.rb:365:in block in with_transaction_returning_status' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/abstract/transaction.rb:535:in block in within_new_transaction'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activesupport-7.1.3/lib/active_support/concurrency/null_lock.rb:9:in synchronize' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/abstract/transaction.rb:532:in within_new_transaction'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/abstract/database_statements.rb:344:in transaction' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/transactions.rb:361:in with_transaction_returning_status'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/transactions.rb:305:in destroy' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activerecord-7.1.3/lib/active_record/persistence.rb:797:in destroy!'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/solid_queue-0.2.0/app/models/solid_queue/process.rb:23:in deregister' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/solid_queue-0.2.0/lib/solid_queue/processes/registrable.rb:33:in deregister'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activesupport-7.1.3/lib/active_support/callbacks.rb:403:in block in make_lambda' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activesupport-7.1.3/lib/active_support/callbacks.rb:274:in block in simple'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activesupport-7.1.3/lib/active_support/callbacks.rb:602:in block in invoke_after' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activesupport-7.1.3/lib/active_support/callbacks.rb:602:in each'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activesupport-7.1.3/lib/active_support/callbacks.rb:602:in invoke_after' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activesupport-7.1.3/lib/active_support/callbacks.rb:111:in run_callbacks'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/solid_queue-0.2.0/lib/solid_queue/processes/runnable.rb:52:in do_start_loop' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/solid_queue-0.2.0/lib/solid_queue/processes/runnable.rb:39:in start_loop'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/solid_queue-0.2.0/lib/solid_queue/processes/runnable.rb:13:in start' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/solid_queue-0.2.0/lib/solid_queue/supervisor.rb:123:in block in start_fork'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activesupport-7.1.3/lib/active_support/fork_tracker.rb:20:in block in fork' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/debug-1.9.1/lib/debug/session.rb:2460:in block in fork'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/debug-1.9.1/lib/debug/session.rb:2462:in fork' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/debug-1.9.1/lib/debug/session.rb:2462:in fork'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/activesupport-7.1.3/lib/active_support/fork_tracker.rb:18:in fork' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/solid_queue-0.2.0/lib/solid_queue/supervisor.rb:122:in start_fork'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/solid_queue-0.2.0/lib/solid_queue/supervisor.rb:72:in block in start_forks' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/solid_queue-0.2.0/lib/solid_queue/supervisor.rb:72:in each'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/solid_queue-0.2.0/lib/solid_queue/supervisor.rb:72:in start_forks' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/solid_queue-0.2.0/lib/solid_queue/supervisor.rb:45:in supervise'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/solid_queue-0.2.0/lib/solid_queue/supervisor.rb:26:in start' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/solid_queue-0.2.0/lib/solid_queue/supervisor.rb:14:in start'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/solid_queue-0.2.0/lib/solid_queue/tasks.rb:4:in block (2 levels) in <main>' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/task.rb:281:in block in execute'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/task.rb:281:in each' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/task.rb:281:in execute'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/task.rb:219:in block in invoke_with_call_chain' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/task.rb:199:in synchronize'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/task.rb:199:in invoke_with_call_chain' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/task.rb:188:in invoke'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/application.rb:182:in invoke_task' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/application.rb:138:in block (2 levels) in top_level'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/application.rb:138:in each' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/application.rb:138:in block in top_level'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/application.rb:147:in run_with_threads' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/application.rb:132:in top_level'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/application.rb:83:in block in run' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/application.rb:208:in standard_exception_handling'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/lib/rake/application.rb:80:in run' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/rake-13.1.0/exe/rake:27:in <top (required)>'
from /home/AnoobBava/.rbenv/versions/3.0.6/bin/rake:25:in load' from /home/AnoobBava/.rbenv/versions/3.0.6/bin/rake:25:in <top (required)>'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/3.0.0/bundler/cli/exec.rb:58:in load' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/3.0.0/bundler/cli/exec.rb:58:in kernel_load'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/3.0.0/bundler/cli/exec.rb:23:in run' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/3.0.0/bundler/cli.rb:479:in exec'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/3.0.0/bundler/vendor/thor/lib/thor/command.rb:27:in run' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/3.0.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in invoke_command'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/3.0.0/bundler/vendor/thor/lib/thor.rb:392:in dispatch' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/3.0.0/bundler/cli.rb:31:in dispatch'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/3.0.0/bundler/vendor/thor/lib/thor/base.rb:485:in start' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/3.0.0/bundler/cli.rb:25:in start'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/bundler-2.2.33/libexec/bundle:49:in block in <top (required)>' from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/3.0.0/bundler/friendly_errors.rb:103:in with_friendly_errors'
from /home/AnoobBava/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/bundler-2.2.33/libexec/bundle:37:in <top (required)>' from /home/AnoobBava/.rbenv/versions/3.0.6/bin/bundle:23:in load'
from /home/AnoobBava/.rbenv/versions/3.0.6/bin/bundle:23:in <main>'

image

How to recover in-process yet abandoned jobs?

I have got some load on SolidQueue in my production app.

Our workers work in an environment where the worker can be killed at any time (cloud-type infrastructure). Because of this, over the course of time, we will develop some jobs that show as "in-process" but the worker that was running them has died. So, they have been "in process" for 13 days, etc.

I'm able to query the jobs and find the jobs that are in process but not assigned to any current active worker.

current_worker_ids = SolidQueue::Process.select(:id).where(kind: "Worker").map { |x| x.id }
SolidQueue::Job.joins(:claimed_execution).where(finished_at: nil).where.not(claimed_execution: {process_id: current_worker_ids} .where_assoc_not_exists(:failed_execution)

I've built a method to try to recover the jobs.

    def requeue_abandoned!
      count = 0
      total_to_queue = abandoned_in_progress_jobs_count
      logger.info "Requeuing #{total_to_queue} abandoned jobs"
      abandoned_in_progress_jobs.find_each do |job|
        job.claimed_execution.delete
        schedule = SolidQueue::ScheduledExecution.create_or_find_by!(job_id: job.id)
        schedule.update!(scheduled_at: Time.zone.now)
        logger.info "Requeued #{count} of #{total_to_queue} jobs" if count % 100 == 0
      end
      logger.info "Requeued #{count} of #{total_to_queue} jobs"
      true
    end

As you can see, it deletes the claimed execution. Then, it tries to find the scheduled execution and set its time to now to make it ready.

This seems to work. BUT, it throws a nasty error and 0 of my workers are now working.


2024-02-21 07:28:01.643 | DETAIL:  Key (job_id)=(109233) already exists. |  
-- | -- | --
  |   | 2024-02-21 07:28:01.643 | /usr/local/bundle/ruby/3.2.0/gems/activerecord-7.1.3/lib/active_record/connection_adapters/postgresql_adapter.rb:894:in `exec_params': ERROR:  duplicate key value violates unique constraint "index_solid_queue_claimed_executions_on_job_id" (PG::UniqueViolation)

So, I'm wondering if I don't understand from reading the docs how these executions work. It COULD be that this error is somewhat unrelated to what I did above (which I did for about 5000 jobs). But, I'd guess this was related.

The documentation on the executions and stuff is pretty sparse, and I'm not sure I really "got" how this works. Any documentation help would help me get to the bottom of this.

I'd appreciate it.

Puma plugin :solid_queue throws an error on start

I'm trying to use puma plugin :solid_queue but I get an error

gems/solid_queue-0.1.1/lib/puma/plugin/solid_queue.rb:11:in `block in start': uninitialized constant SolidQueue (NameError)
      SolidQueue::Supervisor.start(mode: :all)
                ^^^^^^^^^^^^

this happens on my MacBook (M1) and in my linux server/container

my solid_queue.yml

production:
  dispatchers:
    - polling_interval: 1
      batch_size: 500
  workers:
    - queues:
      - "*"
      threads: 1
      polling_interval: 2
    - queues:
      - real_time
      - background
      threads: 1
      polling_interval: 0.1
      processes: 1

The full stack

Running 'bundle exec puma -t 5:5 -p ${PORT:-3000} -e ${RACK_ENV:-development}'
[62] Puma starting in cluster mode...
[62] * Puma version: 6.4.0 (ruby 3.2.2-p53) ("The Eagle of Durango")
[62] *  Min threads: 5
[62] *  Max threads: 5
[62] *  Environment: deployment
[62] *   Master PID: 62
[62] *      Workers: 4
[62] *     Restarts: (✔) hot (✔) phased
[62] * Listening on http://0.0.0.0:10000/
[62] Use Ctrl-C to stop
/opt/render/project/.gems/ruby/3.2.0/gems/solid_queue-0.1.1/lib/puma/plugin/solid_queue.rb:11:in `block in start': uninitialized constant SolidQueue (NameError)
      SolidQueue::Supervisor.start(mode: :all)
                ^^^^^^^^^^^^
	from /opt/render/project/.gems/ruby/3.2.0/gems/solid_queue-0.1.1/lib/puma/plugin/solid_queue.rb:9:in `fork'
	from /opt/render/project/.gems/ruby/3.2.0/gems/solid_queue-0.1.1/lib/puma/plugin/solid_queue.rb:9:in `start'
	from /opt/render/project/.gems/ruby/3.2.0/gems/puma-6.4.0/lib/puma/plugin.rb:24:in `block in fire_starts'
	from /opt/render/project/.gems/ruby/3.2.0/gems/puma-6.4.0/lib/puma/plugin.rb:22:in `each'
	from /opt/render/project/.gems/ruby/3.2.0/gems/puma-6.4.0/lib/puma/plugin.rb:22:in `fire_starts'
	from /opt/render/project/.gems/ruby/3.2.0/gems/puma-6.4.0/lib/puma/launcher.rb:188:in `run'
	from /opt/render/project/.gems/ruby/3.2.0/gems/puma-6.4.0/lib/puma/cli.rb:75:in `run'
	from /opt/render/project/.gems/ruby/3.2.0/gems/puma-6.4.0/bin/puma:10:in `<top (required)>'
	from /opt/render/project/.gems/bin/puma:27:in `load'
	from /opt/render/project/.gems/bin/puma:27:in `<top (required)>'
	from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:58:in `load'
	from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:58:in `kernel_load'
	from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:23:in `run'
	from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/cli.rb:492:in `exec'
	from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
	from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
	from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor.rb:527:in `dispatch'
	from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/cli.rb:34:in `dispatch'
	from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
	from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/cli.rb:28:in `start'
	from /opt/render/project/.gems/gems/bundler-2.4.22/exe/bundle:37:in `block in <top (required)>'
	from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
	from /opt/render/project/.gems/gems/bundler-2.4.22/exe/bundle:29:in `<top (required)>'
	from /opt/render/project/.gems/bin/bundle:108:in `load'
	from /opt/render/project/.gems/bin/bundle:108:in `<main>'
[62] Detected Solid Queue has gone away, stopping Puma...
[62] - Gracefully shutting down workers...
[70] Early termination of worker
[68] Early termination of worker
[74] Early termination of worker
[72] Early termination of worker
[62] === puma shutdown: 2023-12-20 22:15:18 +0000 ===
[62] - Goodbye!

Jobs stuck in ready_execution

Summary

I currently have a lot of jobs stuck in solid_queue_ready_execution that are not getting picked up. I'm not sure how I got to this error state, but it might be related to #137

Proposal

Document how I can manually force a worker to pick up a job and process it.

Trouble installing required MySQL dependencies in order to contribute

I don't typically use MySQL, but I wanted to contribute, so when I pulled the app and tried to bundle install its dependencies, I got an error. I had to install MySQL and then run the gem install command below to get it to work. I'm on a Mac, so I used Homebrew to install MySQL. I'm not sure if this is the best way to do it, but it worked for me.

First, install MySQL if you haven't already:

brew install mysql

Then, check the installed version of MySQL:

mysql --version

After that, you can install the mysql2 gem using the specific libraries installed by Homebrew. It's important to specify the version of the mysql2 gem that is required by the solid_queue gem. As of now, version 0.5.4 is required, so I've included that in the command below. Replace <mysql2_gem_version> with the required version (in this case, '0.5.4'), and <mysql_version> with the version you found using the mysql --version command:

gem install mysql2 -v <mysql2_gem_version> -- \
--with-mysql-lib=/opt/homebrew/Cellar/mysql/<mysql_version>/lib \
--with-mysql-dir=/opt/homebrew/Cellar/mysql/<mysql_version> \
--with-mysql-config=/opt/homebrew/Cellar/mysql/<mysql_version>/bin/mysql_config \
--with-mysql-include=/opt/homebrew/Cellar/mysql/<mysql_version>/include

Voila! The mysql2 gem should now be successfully installed. You can verify this by running bundle to ensure a successful installation.

I also encountered an issue with the SQLite native extension. This turned out to be a missing dependency, which was easily resolved by installing pkg-config via Homebrew. I include this here for completeness:

brew install pkg-config

After this my bundle was totally successful and I was ready to contribute. 🎉

Some jobs failing due to ActiveRecord::Deadlocked when trying to create a ScheduledExecution

We are seeing some failed jobs due to hitting a deadlock when solid queue is trying to create the ScheduledExecution for the job. This is usually happening for us on jobs that are having to be retried due to a throttling constraint we are dealing with from an external api. Here is one example, with part of the backtrace. The job attempts to execute, gets the throttling constraint so it tries to schedule a retry, and it looks like it's trying to do the ScheduledExecution.create_or_find_by! on line 40 of app/models/solid_queue/job/schedulable.rb when it hits the deadlock on the insert.

Screenshot 2024-03-01 at 12 19 12 PM

Backtrace:

/var/www/shield/vendor/bundle/ruby/3.2.0/gems/mysql2-0.5.5/lib/mysql2/client.rb:151:in `_query'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/mysql2-0.5.5/lib/mysql2/client.rb:151:in `block in query'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/mysql2-0.5.5/lib/mysql2/client.rb:150:in `handle_interrupt'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/mysql2-0.5.5/lib/mysql2/client.rb:150:in `query'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:100:in `block (2 levels) in raw_execute'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/connection_adapters/abstract_adapter.rb:1028:in `block in with_raw_connection'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.2/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/connection_adapters/abstract_adapter.rb:1000:in `with_raw_connection'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:98:in `block in raw_execute'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.2/lib/active_support/notifications/instrumenter.rb:58:in `instrument'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/connection_adapters/abstract_adapter.rb:1143:in `log'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:97:in `raw_execute'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:233:in `execute_and_free'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/connection_adapters/mysql2/database_statements.rb:23:in `internal_exec_query'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:153:in `exec_insert'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:191:in `insert'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/connection_adapters/abstract/query_cache.rb:25:in `insert'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/persistence.rb:588:in `_insert_record'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/persistence.rb:1252:in `_create_record'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/counter_cache.rb:187:in `_create_record'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/locking/optimistic.rb:84:in `_create_record'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/encryption/encryptable_record.rb:184:in `_create_record'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/attribute_methods/dirty.rb:240:in `_create_record'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/callbacks.rb:445:in `block in _create_record'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:110:in `run_callbacks'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:952:in `_run_create_callbacks'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/callbacks.rb:445:in `_create_record'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/timestamp.rb:114:in `_create_record'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/persistence.rb:1221:in `create_or_update'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/callbacks.rb:441:in `block in create_or_update'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:110:in `run_callbacks'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:952:in `_run_save_callbacks'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/callbacks.rb:441:in `create_or_update'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/timestamp.rb:125:in `create_or_update'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/persistence.rb:751:in `save!'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/validations.rb:55:in `save!'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/transactions.rb:313:in `block in save!'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/transactions.rb:365:in `block in with_transaction_returning_status'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:342:in `transaction'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/transactions.rb:361:in `with_transaction_returning_status'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/transactions.rb:313:in `save!'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/suppressor.rb:56:in `save!'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/persistence.rb:55:in `create!'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/relation.rb:918:in `_create!'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/relation.rb:118:in `block in create!'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/relation.rb:929:in `_scoping'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/relation.rb:467:in `scoping'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/relation.rb:118:in `create!'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/relation.rb:229:in `block in create_or_find_by!'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:535:in `block in within_new_transaction'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.2/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:532:in `within_new_transaction'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:344:in `transaction'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/transactions.rb:212:in `transaction'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/relation/delegation.rb:105:in `transaction'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/relation.rb:229:in `create_or_find_by!'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.2/lib/active_record/querying.rb:23:in `create_or_find_by!'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/solid_queue-0.2.1/app/models/solid_queue/job/schedulable.rb:40:in `schedule'
/var/www/shield/vendor/bundle/ruby/3.2.0/gems/solid_queue-0.2.1/app/models/solid_queue/job/executable.rb:65:in `prepare_for_execution'

Should the dispatcher run on only one server?

I was trying to figure out if having a dispatcher process running on multiple servers would be a problem, and I noticed on the rake tasks there are there are two other tasks (workand dispatch) that boot solid_queue with only those of two processes.

So my question is: If I have multiple servers, am I supposed to use solid_queue:start on all of them, or do I use solid_queue:start on a single one and solid_queue:work on the rest?

Error: no implicit conversion of ActiveSupport::TimeWithZone

env:
ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-linux]
Rails 7.1.2
Mysql 5.7

[ActiveJob] SolidQueue::Job Create (1.7ms) INSERT INTO solid_queue_jobs (queue_name, class_name, arguments, priority, active_job_id, scheduled_at, finished_at, concurrency_key, created_at, updated_at) VALUES ('default', 'FutureInsJob', '{"job_class":"FutureInsJob","job_id":"071e3a57-3b88-4a9e-a363-0f84aef16580","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[],"executions":0,"exception_executions":{},"locale":"en","timezone":"UTC","enqueued_at":"2023-12-22T04:21:08.478Z","scheduled_at":null}', 0, '071e3a57-3b88-4a9e-a363-0f84aef16580', '2023-12-22 04:21:08.478207', NULL, NULL, '2023-12-22 04:21:08.504246', '2023-12-22 04:21:08.504246')

[ActiveJob] SolidQueue::Job Create (1.7ms) INSERT INTO solid_queue_jobs (queue_name, class_name, arguments, priority, active_job_id, scheduled_at, finished_at, concurrency_key, created_at, updated_at) VALUES ('default', 'FutureInsJob', '{"job_class":"FutureInsJob","job_id":"24169f46-9bf2-4d16-91be-ff7ec0b61ad5","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[],"executions":0,"exception_executions":{},"locale":"zh-CN","timezone":"Beijing","enqueued_at":"2023-12-22T12:22:07.791+08:00","scheduled_at":null}', 0, '24169f46-9bf2-4d16-91be-ff7ec0b61ad5', '2023-12-22 12:22:07.791779', NULL, NULL, '2023-12-22 12:22:07.820188', '2023-12-22 12:22:07.820188')

solid_queue_failed_executions error:

{"exception_class":"TypeError","message":"no implicit conversion of ActiveSupport::TimeWithZone into String","backtrace":["/user-path/.rbenv/versions/3.1.2/lib/ruby/3.1.0/time.rb:624:in `xmlschema'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/activejob-7.1.2/lib/active_job/core.rb:163:in `deserialize'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/activejob-7.1.2/lib/active_job/core.rb:66:in `deserialize'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/activejob-7.1.2/lib/active_job/execution.rb:29:in `block in execute'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.1.2/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/activejob-7.1.2/lib/active_job/railtie.rb:67:in `block (4 levels) in \u003cclass:Railtie\u003e'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.1.2/lib/active_support/reloader.rb:77:in `block in wrap'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.1.2/lib/active_support/execution_wrapper.rb:88:in `wrap'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.1.2/lib/active_support/reloader.rb:74:in `wrap'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/activejob-7.1.2/lib/active_job/railtie.rb:66:in `block (3 levels) in \u003cclass:Railtie\u003e'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.1.2/lib/active_support/callbacks.rb:130:in `instance_exec'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.1.2/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.1.2/lib/active_support/callbacks.rb:141:in `run_callbacks'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/activejob-7.1.2/lib/active_job/execution.rb:28:in `execute'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/solid_queue-0.1.1/app/models/solid_queue/claimed_execution.rb:50:in `execute'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/solid_queue-0.1.1/app/models/solid_queue/claimed_execution.rb:29:in `perform'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/solid_queue-0.1.1/lib/solid_queue/pool.rb:23:in `block (2 levels) in post'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.1.2/lib/active_support/execution_wrapper.rb:92:in `wrap'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/solid_queue-0.1.1/lib/solid_queue/app_executor.rb:7:in `wrap_in_app_executor'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/solid_queue-0.1.1/lib/solid_queue/pool.rb:22:in `block in post'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/safe_task_executor.rb:24:in `block in execute'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:48:in `block in synchronize'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:48:in `synchronize'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:48:in `synchronize'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/safe_task_executor.rb:22:in `execute'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/ivar.rb:170:in `safe_execute'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/future.rb:55:in `block in execute'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:352:in `run_task'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:343:in `block (3 levels) in create_worker'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:334:in `loop'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:334:in `block (2 levels) in create_worker'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:333:in `catch'","/user-path/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.2.2/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:333:in `block in create_worker'"]}

activejob-7.1.2/lib/active_job/core.rb deserialize(job_data)

 self.enqueued_at = Time.iso8601(job_data["enqueued_at"]) if job_data["enqueued_at"]                       
{"job_class"=>"TestJob",
 "job_id"=>"06bf063f-2db2-42f5-bcc1-01c6bf22a1f9",
 "provider_job_id"=>nil,
 "queue_name"=>"default",
 "priority"=>nil,
 "arguments"=>[],
 "executions"=>0,
 "exception_executions"=>{},
 "locale"=>"en",
 "timezone"=>"Beijing",
 "enqueued_at"=>Thu, 21 Dec 2023 18:10:02.000000000 CST +08:00,
 "scheduled_at"=>nil}

class name of job_data[:enqueued_at] ActiveSupport::TimeWithZone

SolidQueue::ReadyExecution error

Hi,

When running Solid Queue with all default config I get:

/gems/solid_queue-0.1.2/lib/solid_queue/worker.rb:35:in block in poll': uninitialized constant SolidQueue::ReadyExecution (NameError)
21:27:46 jobs.1 |
21:27:46 jobs.1 | SolidQueue::ReadyExecution.claim(queues, pool.idle_threads, process.id)
21:27:46 jobs.1 | ^^^^^^^^^^^^^^^^
21:27:46 jobs.1 | Did you mean? SolidQueue::FailedExecution`

Any suggestions?

SolidQueue::Job inside active_admin

For those still stuck with active admin :)
The below is a quick means to get the jobs appear inside your backend admin.

# typed: false
# frozen_string_literal: true

# monkey patch to make ActiveAdmin work with SolidQueue::Job
class SolidQueue::Job

  def self.ransackable_attributes(auth_object = nil)
    ["active_job_id",
     "arguments",
     "class_name",
     "concurrency_key",
     "created_at",
     "finished_at",
     "id",
     "id_value",
     "priority",
     "queue_name",
     "scheduled_at",
     "updated_at"]
  end

end

ActiveAdmin.register SolidQueue::Job do
  menu parent: "Data", label: I18n.t("admin.solid_queue.jobs")

  filter :queue_name
  filter :class_name
  filter :created_at
  filter :scheduled_at
  filter :finished_at

  index do
    selectable_column
    column :queue_name
    column :class_name
    column :created_at
    column :scheduled_at
    column :finished_at
    actions defaults: true
  end
end

Comma separated queue names don't work

When specifying multiple queues as one string, ie:

development:
  workers:
    - queues: real_time,background

this does not work. The workers are not taking scheduled jobs (real_time or background) at all.

It works for me, when queues are an array, ie:

development:
  workers:
    - queues:
      - real_time
      - background

Should the documentation be fixed to list queues as array in YAML or maybe the logic to fix comma separated string?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.