Background Cron Jobs in Rails Using Sidekiq, Redis, and Whenever gems.

Earlier this week, I was tasked to add a feature that sent out notifications when a certain task didn’t check in from a third-party API. I had to come up with a background job that did this heavy lifting in intervals to keep the users informed of certain failures in certain transactions. This is how I solved this challenge and I hope my approach helps you implement your own solution.

NB: This tutorial is based on a fictional scenario that you could easily learn from and implement in your own project. This tutorial could be useful for scenarios like sending out scheduled marketing emails, app administration, etc.

For this, I needed to work with the Whenever gem for running the scheduled checks/jobs in the background and Sidekiq for executing the jobs(also visualizing the queued jobs ) at the same time in form of threads and Redis memory database to store this data in the memory as the app ran.

So let’s dive into it already.

Add the whenever gem to your Gemfile.

gem 'whenever', require: false

then run bundle install to install the gem.

Next we Wheneverize the project by running bundle exec wheneverize . . Read more about whenever gem options from https://github.com/javan/whenever

A schedule.rb the file will be generated in the project Config folder. In this file, you can add your scheduled task runners. In this example, we shall be working with rake tasks. Read more about rake tasks here https://github.com/ruby/rake.

config/schedule.rb

// set where you want to see the logs for jobs at.
set :output, { error: '/log/error.log', standard: '/log/cron.log'}
every 2.hours do
  rake 'shopping_cart_notifier:pending_checkout_clearance'
end

then at this point, let’s go ahead and create the rake tasks because what we have up there is just a placeholder. To be honest you could do this first and add it to the schedule.rb later.

lib/tasks/shopping_cart_notifier.rake

require 'rake'
namespace :shopping_cart_notifier do
  desc 'Sends notification about items in the cart that haven't been paid for and checked out.'
    task pending_checkout_clearance: :environment do
      NotifyPendingCheckoutJob.perform_later
    end
end

Let's go ahead and create the actual function that does the job NotifyPendingCheckoutJob . Now this will require Sidekiq and Redis installed.

Add these gems to your Gemfile and run bundle install.

gem ‘sidekiq’
gem ‘redis’, ‘~> 4.0’
gem ‘letter_opener’ // you may need this to preview emails locally in case its ehat you're using emails as the form of notification.

then go ahead and place this line in the Config/application.rb to set up Sidekiq to work within the application. Add at the end right before the end keyword.

config.active_job.queue_adapter = :sidekiq

Go to the initializers and configure Sidekiq to use Redis. config/initializers/sidekiq.rb

if Rails.env.production?
    redis_host = '127.0.0.1'
    redis_port = 6379
    redis_db = 1
  else
    redis_host = 'localhost'
    redis_port = 6379
    redis_db = 0
  end
  
  Sidekiq.configure_server do |config|
    config.redis = { url: "redis://#{redis_host}:#{redis_port}/#{redis_db}" }
  end
  
  Sidekiq.configure_client do |config|
    config.redis = { url: "redis://#{redis_host}:#{redis_port}/#{redis_db}" }
  end

Then let’s go to the routes and set up a route to view the Sidekiq user interface by adding these two lines.

require 'sidekiq/web'
authenticate :[user(for example the ordinary user)] do
  mount Sidekiq::Web => '/sidekiq'
end

Then lastly, let's configure our app to use the letter opener for testing to check if the letters were sent through. Add these two lines to the config/environments/development.rb file.

config.action_mailer.delivery_method = :letter_opener
config.action_mailer.perform_deliveries = true

Then lets create a job file in app/jobs/notify_pending_checkout_job.rb

class NotifyPendingCheckoutJob < ApplicationJob
    sidekiq_options retry: 5
    queue_as :default
  
    def perform(*_args)
        $pending_checkouts = ShoppingCart.where(status: :not_checked_out).where(['created_at < ?', 2.hours.ago]).all
                          AdminMailer.notify_admin_pending_checkouts($pending_checkouts).deliver! if $pending_checkouts.any? // this is for sending out email notifications
    end
  end

Let’s create the mailer functions now. This can be done in app/mailers/admin_mailer.rb

class AdminMailer<ApplicationMailer
    default from: '[some email]@[some_domain].com'
    layout 'mailer'

    def notify_admin_pending_checkouts(pending_checkout)
        @pending_checkout=pending_checkout
        mail(
        to: '[some admin mail]@domain.com',
        subject: "#{@pending_checkout.count} cart items have not been checked out for the past two hours."
        )
    end
end

Let’s create an html.erb template to display this data to the admin in the tables. Better we do in a table. Create it in app/views/admin_mailer/notify_admin_pending_checkouts.html.erb.

<h1>These users need to be reminded to checkout their cart items. </h1>

// add your display logic here
<table style="width:100%">
    <tr style="text-align:center; ">
        <th>User</th>
        <th>cart Id</th>
    </tr>
    <%= $pending_checkouts.each do |item|%>
        <tr>
            do something
        </tr>
    <% end %>
</table>

To view Sidekiq in the browser, you have to ensure that Redis is installed and running by running redis_server in the terminal and then accessing yourapp.com/sidekiq.. To do this in your app template on a nav link or button, you can also use the Sidekiq path since it's already added to the routes.

<%= link_to sidekiq_web_path,  class: "[your class]", target: "_blank" do%>
 Sidekiq
<% end %>

Looks like we’re done with this. So, let’s test it out locally. Go ahead and create a seeder to create mock data. Alternatively, go ahead and open up the rails console and achieve the same result.

Go to the rake task and change the perform_later to perform_now to be able to see the task in action when you run the rake task in the terminal for instance.

rake 'shopping_cart_notifier:pending_checkout_clearance'

Please remember to revert it after testing. Also, test out the scenario for delivering later by reducing the schedule times in the schedule.rb.

I think that will be all for this post. I will look into writing about using Sidekiq in production managed by Systemd and Redis in my next post.

Read more about Sidekiq: https://github.com/mperham/sidekiq

Thank you for reading.