esminc/activejob-google_cloud_tasks-http

Active Job adapter for Google Cloud Tasks HTTP targets

ActiveJob::GoogleCloudTasks::HTTP

ActiveJob::GoogleCloudTasks::HTTP is an ActiveJob adapter for running jobs via Google Cloud Tasks. As the name suggests it only supports HTTP targets.

Installation

Add this line to your application's Gemfile:

gem 'activejob-google_cloud_tasks-http'

And then execute:

$ bundle

Or install it yourself as:

$ gem install activejob-google_cloud_tasks-http

Usage

Setup

Configure an adapter instance and pass it to Active Job:

Rails.application.config.active_job.queue_adapter = ActiveJob::GoogleCloudTasks::HTTP::Adapter.new(
  project: 'a-gcp-project-name',
  location: 'asia-northeast1',
  url: 'https://an-endpoint-to-perform-jobs.a.run.app/_jobs',
  client: Google::Cloud::Tasks.cloud_tasks(version: :v2beta3), # optional
  task_options: { # optional
    oidc_token: {
      service_account_email: '[email protected]'
    }
  }
)

A name passed to queue_as will be used to identify which Cloud Tasks queue will be used by the job:

class GoodJob < ApplicationJob
  queue_as :a_queue_name

  # ...
end

Mount the Rack application to set up an endpoint for performing jobs:

# in config/routes.rb
mount ActiveJob::GoogleCloudTasks::HTTP::Rack.new, at: '/_jobs'

Note that this rack app itself does not have any authentication mechanism.

Testing

Requiring active_job/google_cloud_tasks/http/inline makes the adapter skip enqueueing jobs to Google Cloud Tasks. Once a job is enqueued, it will perform the job immediately.

require 'active_job/google_cloud_tasks/http/inline' unless Rails.env.production?

Error when calling assets:precompile?

When you call assets:precompile, all configs and initializers are loaded. If you load your credentials via environment variables they may not be available and the adapter initialization will cause errors. To solve this, wrap the queue_adapter config in a unless ARGV.include?("assets:precompile") condition. e.g.:

unless ARGV.include?("assets:precompile") # prevents running on assets:precompile
  Rails.application.config.active_job.queue_adapter = ActiveJob::GoogleCloudTasks::HTTP::Adapter.new(
    project: 'my-project',
    location: 'europe-west2',
    url: 'https://www.example.com/jobs',
    client: Google::Cloud::Tasks.cloud_tasks(version: :v2beta3) { |config|
      # this will cause an error if the environment variable does not exist
      config.credentials = JSON.parse(ENV["GOOGLE_CLOUD_PRODUCTION_KEYFILE"])
    }
  )
end

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/esminc/activejob-google_cloud_tasks-http.

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the ActiveJob::GoogleCloudTasks::HTTP project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.

activejob-google_cloud_tasks-http's Issues

Error when building app in Docker container and running assets:precompile

I build my app in a Docker container and use assets:precompile so the app has faster cold starts. When deploying this on production I hit an error because the environment variable I use for Google credentials was not available for the precompile step (assets:precompile runs all config and initializers).

Error in Docker build:

Step #1: Step 17/23 : RUN SECRET_KEY_BASE=1 bundle exec rake assets:precompile
Step #1:  ---> Running in c8abbe14a1b3
Step #1: rake aborted!
Step #1: TypeError: no implicit conversion of nil into String
Step #1: /usr/src/app/config/environments/production.rb:125:in `block in <top (required)>'

relevant production.rb snippet (with some sensitive data obscured):

    config.active_job.queue_adapter = ActiveJob::GoogleCloudTasks::HTTP::Adapter.new(
      project: 'my-project',
      location: 'europe-west2',
      url: 'https://www.example.com/jobs',
      client: Google::Cloud::Tasks.new(
        version: :v2beta3,
        credentials: JSON.parse(ENV["GOOGLE_CLOUD_PRODUCTION_KEYFILE"]) # production.rb:125
      )
    )

The environment variable does not exist yet, nor do I want to include it in the build steps. It's annoying because this is being called by assets:precompile which in no way affects Active Job or Google Cloud Tasks. Ideally it just wouldn't run this at all.

After a few too many hours of frustration I came across a reasonable workaround (via https://stackoverflow.com/a/58017606/3471900):

  unless ARGV.include?("assets:precompile") # wrap the config so it doesn't run when assets:precompile is called
    config.active_job.queue_adapter = ActiveJob::GoogleCloudTasks::HTTP::Adapter.new(
      project: 'my-project',
      location: 'europe-west2',
      url: 'https://www.example.com/jobs',
      client: Google::Cloud::Tasks.new(
        version: :v2beta3,
        credentials: JSON.parse(ENV["GOOGLE_CLOUD_PRODUCTION_KEYFILE"])
      )
    )
  end

Ideally this would be handled natively by the gem, but if not, perhaps adding a section to the readme or wiki would help others and future me :) Quick draft suggestion:

Error when calling assets:precompile?

If you call assets:precompile, all configs and initializers are loaded. If you load your credentials via environment variables they may not be available and the adapter will error. To solve this, wrap the queue_adapter config in a unless ARGV.include?("assets:precompile") condition. Eg.

unless ARGV.include?("assets:precompile") # prevents running on assets:precompile
  Rails.application.config.active_job.queue_adapter = ActiveJob::GoogleCloudTasks::HTTP::Adapter.new(
    project: 'my-project',
    location: 'europe-west2',
    # ...
  )
end

PS. Thank you for the gem! It works great and got me up and running with Cloud Tasks super quickly :)

Getting errors with new API

I'm receiving the following:

ArgumentError (wrong number of arguments (given 3, expected 0; required keywords: project, location, queue))

This is when adapter tries to call queue_path. This is likely due to changes in Google's Ruby libraries.

How to use scheduling?

How can I schedule a job in the near future? Any sample code? Thanks in advance.

Authenticating Rack app

In the README, it states the following:

Note that this rack app itself does not have any authentication mechanism.

I am looking to add a simple HTTP Basic AUTH to the rack app, but can't see an obvious way to allow Google Tasks API to specify these credentials when posting to the jobs URL.

Unable to access standard rails logger from rack app

Hi @hibariya, great gem. Cloud Tasks look like they'll make my async tasks way simpler and cheaper to host!

I'm having trouble with some of my jobs failing however and don't seem to be seeing any output in the normal Rails log. I'm assuming that's because Rack is unable to access the Rails loggers, but I'm new to mounting Rack apps inside Rails ones so I apologize if I'm just confused.

Is there an easy way to expose the Rails logger to your Rack component? Alternately, would you be open to a PR which extracted the logic for running the jobs from your Rack component so I could mix it into a Rails controller?

Thank-you, and please let me know if you need more specific information to give an answer. I'm away from my code right now but can get you code samples later.

Cheers!

NameError: uninitialized constant Google::Protobuf

Just a heads up. This error get's raised when scheduling.