envek/after_commit_everywhere

Use ActiveRecord transactional callbacks outside of models, literally everywhere in your application.

after_commit everywhere

Allows to use ActiveRecord transactional callbacks outside of ActiveRecord models, literally everywhere in your application.

Inspired by these articles:

• https://evilmartians.com/chronicles/rails-after_commit-everywhere
• https://blog.arkency.com/2015/10/run-it-in-background-job-after-commit/

Installation

Add this line to your application's Gemfile:

gem 'after_commit_everywhere'

And then execute:

$ bundle

Or install it yourself as:

$ gem install after_commit_everywhere

Usage

Recommended usage is to include it to your base service class or anything:

class ServiceObjectBtw
  include AfterCommitEverywhere

  def call
    ActiveRecord::Base.transaction do
      after_commit { puts "We're all done!" }
    end
  end
end

Or just extend it whenever you need it:

extend AfterCommitEverywhere

ActiveRecord::Base.transaction do
  after_commit { puts "We're all done!" }
end

Or call it directly on module:

AfterCommitEverywhere.after_commit { puts "We're all done!" }

That's it!

But the main benefit is that it works with nested transaction blocks (may be even spread across many files in your codebase):

include AfterCommitEverywhere

ActiveRecord::Base.transaction do
  puts "We're in transaction now"

  ActiveRecord::Base.transaction do
    puts "More transactions"
    after_commit { puts "We're all done!" }
  end

  puts "Still in transaction…"
end

Will output:

We're in transaction now
More transactions
Still in transaction…
We're all done!

Available callbacks

after_commit

Will be executed right after outermost transaction have been successfully committed and data become available to other DBMS clients.

If called outside transaction will execute callback immediately.

before_commit

Will be executed right before outermost transaction will be commited (I can't imagine use case for it but if you can, please open a pull request or issue).

If called outside transaction will execute callback immediately.

Supported only starting from ActiveRecord 5.0.

after_rollback

Will be executed right after transaction in which it have been declared was rolled back (this might be nested savepoint transaction block with requires_new: true).

If called outside transaction will raise an exception!

Please keep in mind ActiveRecord's limitations for rolling back nested transactions. See in_transaction for a workaround to this limitation.

Available helper methods

in_transaction

Makes sure the provided block is running in a transaction.

This method aims to provide clearer intention than a typical ActiveRecord::Base.transaction block - in_transaction only cares that some transaction is present, not that a transaction is nested in any way.

If a transaction is present, it will yield without taking any action. Note that this means ActiveRecord::Rollback errors will not be trapped by in_transaction but will propagate up to the nearest parent transaction block.

If no transaction is present, the provided block will open a new transaction.

class ServiceObjectBtw
  include AfterCommitEverywhere

  def call
    in_transaction do
      an_update
      another_update
      after_commit { puts "We're all done!" }
    end
  end
end

Our service object can run its database operations safely when run in isolation.

ServiceObjectBtw.new.call # This opens a new #transaction block

If it is later called from code already wrapped in a transaction, the existing transaction will be utilized without any nesting:

ActiveRecord::Base.transaction do
  new_update
  next_update
  # This no longer opens a new #transaction block, because one is already present
  ServiceObjectBtw.new.call
end

This can be called directly on the module as well:

AfterCommitEverywhere.in_transaction do
  AfterCommitEverywhere.after_commit { puts "We're all done!" }
end

in_transaction?

Returns true when called inside an open transaction, false otherwise.

def check_for_transaction
  if in_transaction?
    puts "We're in a transaction!"
  else
    puts "We're not in a transaction..."
  end
end

check_for_transaction
# => prints "We're not in a transaction..."

in_transaction do
  check_for_transaction
end
# => prints "We're in a transaction!"

Available callback options

• without_tx allows to change default callback behavior if called without transaction open.

  Available values:

  • :execute to execute callback immediately
  • :warn_and_execute to print warning and execute immediately
  • :raise to raise an exception instead of executing

• prepend puts the callback at the head of the callback chain, instead of at the end.

FAQ

Does it works with transactional_test or DatabaseCleaner

Yes.

Be aware of mental traps

While it is convenient to have after_commit method at a class level to be able to call it from anywhere, take care not to call it on models.

So, DO NOT DO THIS:

class Post < ActiveRecord::Base
  def self.bulk_ops
    find_each do
      after_commit { raise "Some doesn't expect that this screw up everything, but they should" }
    end
  end
end

By calling the class level after_commit method on models, you're effectively adding callback for all Post instances, including future ones.

See #13 for details.

But what if I want to use it inside models anyway?

In class-level methods call AfterCommitEverywhere.after_commit directly:

class Post < ActiveRecord::Base
  def self.bulk_ops
    find_each do
       AfterCommitEverywhere.after_commit { puts "Now it works as expected!" }
    end
  end
end

For usage in instance-level methods include this module to your model class (or right into your ApplicationRecord):

class Post < ActiveRecord::Base
  include AfterCommitEverywhere

  def do_some_stuff
    after_commit { puts "Now it works!" }
  end
end

However, if you do something in models that requires defining such ad-hoc transactional callbacks, it may indicate that your models have too many responsibilities and these methods should be extracted to separate specialized layers (service objects, etc).

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install.

Releasing new versions

1. Bump version number in lib/after_commit_everywhere/version.rb

   In case of pre-releases keep in mind rubygems/rubygems#3086 and check version with command like Gem::Version.new(AfterCommitEverywhere::VERSION).to_s

2. Fill CHANGELOG.md with missing changes, add header with version and date.

3. Make a commit:

   git add lib/after_commit_everywhere/version.rb CHANGELOG.md
   version=$(ruby -r ./lib/after_commit_everywhere/version.rb -e "puts Gem::Version.new(AfterCommitEverywhere::VERSION)")
   git commit --message="${version}: " --edit

4. Create annotated tag:

   git tag v${version} --annotate --message="${version}: " --edit --sign

5. Fill version name into subject line and (optionally) some description (list of changes will be taken from CHANGELOG.md and appended automatically)

6. Push it:

   git push --follow-tags

7. GitHub Actions will create a new release, build and push gem into rubygems.org! You're done!

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/Envek/after_commit_everywhere.

License

The gem is available as open source under the terms of the MIT License.

after_commit_everywhere's Issues

Doesn't work with ActiveRecord 6

0c393f2 adds the code to test AR 6

all the tests fail with errors like

  1) AfterCommitEverywhere#after_commit within transaction executes code only after commit
     Failure/Error:
       ActiveRecord::Base.transaction do
         subject
         expect(handler).not_to have_received(:call)
       end

     NoMethodError:
       undefined method `trigger_transactional_callbacks?' for #<AfterCommitEverywhere::Wrap:0x00007f97cb078748>
     # ./spec/after_commit_everywhere_spec.rb:27:in `block (4 levels) in <top (required)>'

Flaky behavior in Minitest with callbacks and ActiveJob

Before anything else, I want to say thank you for this gem. It is awesome and very helpful in our code base.

Currently, I'm running into some flaky test behavior that I have narrowed down to this gem.

Use case: We have an ActiveRecord class for which do a 1:1 sync to an external service. Whenever we destroy a record, we want to enqueue a job to remove it from an external service.

class ExampleRecord < ApplicationRecord 
  after_destroy :sync_destroy_to_external_service

  protected

    def sync_destroy_to_external_service
       ExternalServiceDeleteJob.perform_when_transaction_commits("some-key")
    end
end

To enable this method, we have the following:

class ExternalServiceDeleteJob < ActiveJob::Base
  extend AfterCommitEverywhere

  def self.perform_when_transaction_commits(*args)
    return perform_later(*args) unless in_transaction?

    after_commit { perform_later(*args) }
  end
end

We test this behavior in a really simple way:

test "enqueues destroy job on destruction" do
  example_record = example_records(:some_fixture)
  example_record.destroy!
  assert_enqueued_with(job: ExternalServiceDeleteJob)
end

Nine out of ten times, this spec passes. However, on the random outlier, we get the following:

No enqueued job found with {:job=>ExternalServiceDeleteJob}

How I am really scratching my head. I suspect this has something to do with how rails test manages transaction.

Have you ever seen something like this before? Any idea on what might be causing it?

Thank you in advance for any guidance! And thank you again for this gem. Happy to contribute to a fix in anyway.

Many vulnerabilities with dependent gems

Several of the dependent gems have open CVE records suggesting to upgrade. For example:
CVE-2023-22799 - globalid
CVE-2022-23516 - loofah

Could a new gem version be pushed with the latest working versions?

It is triggering lots of false positives in security scanning of a docker image we use this gem in.

Thanks

calling methods that trigger callbacks can cause `after_commit` callbacks to run multiple times

If an after_commit is called inside of a transaction and that transaction has calls to model methods that can trigger model callbacks, the block passed to after_commit will be called multiple times. For example,

ActiveRecord::Base.transaction do
  User.create!(email:)
  ev = Event.create!(name: 'created user', source: 'admin')

  after_commit do
    send_welcome_email!
  end

  track_event!(ev)
end

When this transaction runs, the welcome e-mail will be sent multiple times because the block is run repeatedly.

before_commit example

Just creating an issue for giving an example that can be used in the doc for before_commit. Very useful in the case where you want a stripe payment to validate your transaction:

something like that:

ActiveRecord::Base.transaction do 
   amount = 4000
   payment = Payment.create(amount: amount)
   before_commit do
      Stripe::Charge.create(amount: amount,
                                           currency: 'cad',
                                           source: params['payment']['token'])
   end
end

payment will only be created if the stripe payment has successfully been done. If payment fail, stripe would raise an error and transaction would rollback.

I think in this case, it's better to make the stripe payment in before_commit cause in the case of a more complex transaction, you don't want to make the stripe payment inside the transaction, at the beginning, and have another piece of code to fail a bit later. Only make the payment if whole transaction went well (before_commit) and only validate the transaction if stripe payment didn't raise an error.

What happens if code inside after_commit callback fails?

Hey there,

in this example:

  def call
    ActiveRecord::Base.transaction do
      create_user!
      after_commit { some_important_stuff }
    end
  end

If something bad inside some_important_stuff happens and throws an error, then anything that was persisted in create_user! won't be rolled back, since when calling after_commit we already are not inside the transaction anymore, right?

Thank you! :)

Clarify its use within models

Does this gem not work within models or is it just generally not recommended because you can use AR's built-in stuff instead?

For instance-level stuff, I'm thinking of doing something like this (simplified):

class User < ApplicationRecord
  has_many :posts

  def activate
    self.activated_at = Time.now
    save
    after_commit { WebhookJob.perform_later(self) }
  end
end

user = User.find(1)
user.transaction do
  user.activate
  user.posts.make_public!
end  

Is there any problem with this? It could be rewritten to use Rails's after_commit but then that would have to inspect for changes to activated_at which just seems to make it more complicated.

How about in class methods in models? I've seen #13, so I know using after_commit does not do what might be expected here. But is there a way to use it properly?

class User < ApplicationRecord
  has_many :posts

  def self.activate_all
    where(activated_at: nil).find_each do |user|
      user.update(activated_at: Time.now)
      user.posts.make_public!
      # Not this, but maybe something else?
      # after_commit { WebhookJob.perform_later(self) }
    end
  end
end

User.transaction do
  User.activate_all
end

Leaking database connection issue

Thank you so much for this gem. It is a really great thing to have in our code base.

After we added this gem to our project we started getting a lot of these kinds of errors from the Sidekiq:

  could not obtain a connection from the pool within 5.000 seconds (waited 5.000 seconds); all pooled connections were in use

I may assume that this is somehow relates to usage of ActiveRecord::Base.connection instead of ActiveRecord::Base.connection_pool.with_connection { |connection| ... }.

So far we have fixed that issue by wrapping all after_commit { do_stuff } instructions with:

ActiveRecord::Base.connection_pool.with_connection do |conn|
  after_commit(connection: conn) { ...do_stuff... }
end

Probably, it would be nice to have some guard here to check if we have any active connections, if not – we can run the block immediately (absolutely the same check as you have for an opened transaction).

Useful links:

• https://gitlab.com/gitlab-org/gitlab-foss/-/issues/27949

Braking tests

Hello!

Is there a way to configure thig gem to ignore transactions when in tests?
Our DatabaseCleaner is set up to use rollbacks, so obviously the transactions are not committed before the expectations, and the tests fail.

The README.md sais it's working with DBCleaner, but can you include a few lines on how to set it up?

Thanks,

Unexpected behaviour in ActiveJob callbacks

Hello, thanks for all the hard work on this gem! I've been trying to use this gem to prevent ActiveJob jobs from enqueueing within transactions but have run into some unexpected behavior. If my job looks like the following:

class TestJob < ActiveJob::Base
  before_enqueue do |job|
    puts "1"
  end

  around_enqueue do |job, block|
    puts "2"
    block.call
  end

  include AfterCommitEverywhere

  around_enqueue do |job, block|
    puts "3"
    after_commit { block.call }
  end

  around_enqueue do |job, block|
    puts "4"
    block.call
  end

  def perform; end
end

And I try to enqueue it from within a transaction:

ActiveRecord::Base.transaction do
  TestJob.perform_later
end

It outputs 123 1234 instead of 1234 that I'm expecting. It seems like all the callbacks preceding the callback that has after_commit are run twice instead of once.

This is on Rails 6.0.6.1 with after_commit_everywhere (1.1.0).

Am I doing something wrong here or is this a bug in the gem?

Add `perform_after_commit` to ActiveJob jobs?

Since it is a common use to use after_commit around ActiveJob calls (like in #19), would it make sense to add to this gem a perform_after_commit method to all ActiveJob jobs?

I use the following concern in my app:

module AfterCommitableJob
  extend ActiveSupport::Concern

  class_methods do
    def perform_after_commit(...)
      AfterCommitEverywhere.after_commit do
        perform_later(...)
      end
    end
  end
end

Would more people benefit from including this in after_commit_everywhere directly?

Great gem, why not 1.0? :)

Hey :)

I am the author of https://blog.arkency.com/2015/10/run-it-in-background-job-after-commit/ . Thank you for writing this gem, I believe it just works. What's stopping us from releasing version 1.0 😃 ?

Based on: https://rubygems.org/gems/after_commit_everywhere/versions/0.1.5

TOTAL DOWNLOADS: 727,852
FOR THIS VERSION: 152,072

and 0 open issues reported here.

Needs extra caution when using within models

We were looking at some strange bug where one test was weirdly affecting another - jobs enqueued in one test were seemingly trying to run in another.

As it turned out - there was a class method on the model, used for some bulk updates, which were enqueuing async jobs with after_commit {...} block.
Dev intended to run after_commit only inside of that bulk update method, but it actually was defined for the whole model, because the AR model has its own after_commit method.

Extracting that bulk method out from the AR model into a separate service solved the issue.

Probably makes sense to put some warning in the Readme about usage within AR models?