Save 37% off PRO during our Black Friday Sale! »

Testing Oban Jobs From the Inside Out

Testing Oban Jobs From the Inside Out

Background jobs with Oban are integral to the business side of many applications. Knowing that your jobs function correctly in isolation, and when composed with other jobs, and when integrated with the rest of the system is essential for peace of mind. Fortunately, Oban provides all the tools necessary to test jobs at any level of isolation.

Together we’ll explore how to make optimal use of Oban’s robust testing functionality for unit, integration, and acceptance tests. Come learn how to test Oban from the inside out—from asserting that workers return correct values to assuring every job in a queue is executed as it would be in production.

C4d700e708f2a338a737d0fb80297731?s=128

Parker Selbert

October 14, 2021
Tweet

Transcript

  1. Testing Oban Jobs from the Inside Out

  2. Parker Selbert @sorentwo Dscout / Soren Hiring!!! Not Hiring…

  3. Oban is a persistent background job system with layered components.

    (queues, workers, jobs, plugins, telemetry)
  4. Automated testing is an essential component of any reliable application.

    (at least those that are long-lived and actively developed 😉)
  5. We need to verify that all moving parts are working

    together as we expect.
  6. Database Application Queue Queue Queue Worker Job Job Job Job

    Job Job Job Job Worker Worker Context Controller
  7. Where you focus test effort depends on performance, isolation, and

    boundaries.
  8. Unit Integration Acceptance Recommended Testing Pyramid

  9. Background jobs are always about side effects.

  10. Great tests are async, isolated within transactions.

  11. Testing Setup

  12. Play nicely with Ecto’s SQL.Sandbox for isolated testing.

  13. config :my_app, Oban, repo: MyApp.Repo, plugins: [Oban.Plugins.Pruner, Oban.Plugins.Stager], queues: [

    default: 10, events: 20, emails: 5 ] config.exs
  14. config :my_app, Oban, plugins: false, queues: false test.exs

  15. Use Oban.Testing for unit, integration, and acceptance tests.

  16. defmodule MyApp.Case do use ExUnit.CaseTemplate using do quote do use

    Oban.Testing, repo: MyApp.Repo end end setup do :ok = Ecto.Adapters.SQL.Sandbox.mode(MyApp.Repo, :manual) :ok = Ecto.Adapters.SQL.Sandbox.checkout(MyApp.Repo) end end MyApp.Repo.start_link() ExUnit.start() test_helper.exs
  17. Now we’re ready for some Testing!

  18. Unit Testing

  19. Application Worker Worker Worker Worker Database Unit

  20. @doc """ Construct a job and execute it with a

    worker module. """ @spec perform_job( worker :: Worker.t(), args :: term(), opts :: [Job.option() | {:repo, module()}] ) :: Worker.result() oban/testing.ex
  21. defmodule MyApp.ActivationWorkerTest do use MyApp.Case, async: true test "activating a

    new account" do account = MyApp.Account.create(name: “ElixirConf”, email: “parker@example.com”) {:ok, account} = perform_job(MyApp.ActivationWorker, %{id: account.id}) assert account.active? end end activation_worker_test.exs
  22. defmodule MyApp.ActivationWorkerTest do use MyApp.Case, async: true test "activating a

    new account" do account = MyApp.Account.create(name: “ElixirConf”, email: “parker@example.com") {:ok, account} = perform_job(MyApp.ActivationWorker, %{id: account.id}) assert account.active? end end activation_worker_test.exs
  23. 1) test activating a new account (MyApp.AccountWorkerTest) Expected worker to

    be a module that implements the Oban.Worker behaviour, got: MyApp.AccountWorker code: {:ok, account} = perform_job(MyApp.AccountWorker, %{id: account.id})
  24. defmodule MyApp.ActivationWorker do use Oban.Worker @impl Worker def perform(%Job{args: %{id:

    account_id}}) do MyApp.Account.activate(account_id) end end activation_worker.ex
  25. 1) test activating a new account (MyApp.AccountWorkerTest) ** (FunctionClauseError) no

    function clause matching in MyApp.AccountWorker.perform/1 The following arguments were given to MyApp.AccountWorker.perform/1: # 1 %Oban.Job{args: %{"id" => 1}, attempt: 1, worker: "MyApp.AccountWorker"} code: {:ok, account} = perform_job(MyApp.AccountWorker, %{id: account.id})
  26. defmodule MyApp.ActivationWorker do use Oban.Worker @impl Worker def perform(%Job{args: %{"id"

    => account_id}}) do MyApp.Account.activate(account_id) end end activation_worker.ex
  27. 1) test activating a new account (MyApp.AccountWorkerTest) Expected result to

    be one of - `:ok` - `:discard` - `{:ok, value}` - `{:error, reason}` - `{:discard, reason}` - `{:snooze, duration} Instead received: {:activated, %MyApp.Account{active?: true, id: 1, name: nil}} code: {:ok, account} = perform_job(MyApp.AccountWorker, %{id: account.id})
  28. defmodule MyApp.ActivationWorker do use Oban.Worker @impl Worker def perform(%Job{args: %{id:

    account_id}}) do with {:activated, account} <- MyApp.Account.activate(account_id) do {:ok, account} end end end activation_worker.ex
  29. . Finished in 0.3 seconds (0.2s on load, 0.08s async,

    0.00s sync) 1 test, 0 failures
  30. defmodule MyApp.ActivationWorkerTest do use MyApp.Case, async: true test "activating a

    new account" do account = MyApp.Account.create(name: “ElixirConf”, email: “parker@example.com”) {:ok, account} = perform_job(MyApp.ActivationWorker, %{id: account.id}, priority: 9) assert account.active? end end activation_worker_test.exs
  31. 1) test activating a new account (MyApp.ActivationWorkerTest) Expected args and

    opts to build a valid job, got validation errors: priority: must be less than 4 code: {:ok, account} = perform_job(MyApp.ActivationWorker, %{id: id}, priority: 9)
  32. defmodule MyApp.ActivationWorkerTest do use MyApp.Case, async: true test "activating a

    new account" do account = MyApp.Account.create(name: “ElixirConf”, email: “parker@example.com”) {:ok, account} = perform_job(MyApp.ActivationWorker, %{id: account.id}, priority: 3) assert account.active? end end activation_worker_test.exs
  33. . Finished in 0.3 seconds (0.2s on load, 0.08s async,

    0.00s sync) 1 test, 0 failures
  34. Use perform_job/1 to: • Ensure Worker Behavior • Check String

    Keys in perform/1 • Verify Return Values • Encode/Decode Args to JSON • Validate Job Opts • Exercise a Custom new/1 Callback
  35. Integration Testing

  36. Application Worker Worker Worker Worker Database Integration

  37. @doc """ Assert that a job with particular options has

    been enqueued. """ @spec assert_enqueued(opts :: Keyword.t()) :: true @doc """ Refute that a job with particular options has been enqueued. """ @spec refute_enqueued(opts :: Keyword.t()) :: false oban/testing.ex
  38. defmodule MyApp.AccountTest do use MyApp.Case, async: true test "scheduling activation

    upon sign up" do {:ok, account} = MyApp.Account.sign_up(name: “ElixirConf”, email: “parker@example.com") assert_enqueued worker: MyApp.ActivationWorker, args: %{id: account.id}, queue: :default end end account_test.exs
  39. defmodule MyApp.AccountTest do use MyApp.Case, async: true test "scheduling activation

    upon sign up" do {:ok, account} = MyApp.Account.sign_up(name: “ElixirConf”, email: “parker@example.com") assert_enqueued worker: MyApp.ActivationWorker, args: %{id: account.id}, queue: :default end end account_test.exs
  40. 1) test scheduling activation upon sign up (MyApp.AccountTest) Expected a

    job matching: %{args: %{id: 4}, queue: :default, worker: MyApp.ActivationWorker} to be enqueued in the "public" schema. Instead found: [] code: assert_enqueued worker: MyApp.ActivationWorker,
  41. defmodule MyApp.Account do def sign_up(args) do changeset = changeset(args) with

    {:ok, account} <- MyApp.Repo.insert(changeset) do %{id: account.id} |> MyApp.ActivationWorker.new() |> Oban.insert() {:ok, account} end end account.ex
  42. . Finished in 0.1 seconds (0.1s async, 0.00s sync) 1

    test, 0 failures
  43. def sign_up(args) do changeset = changeset(args) with {:ok, account} <-

    MyApp.Repo.insert(changeset) do Task.start(fn -> Process.sleep(100) %{id: account.id} |> MyApp.ActivationWorker.new() |> Oban.insert() end) {:ok, account} end end account.ex
  44. 1) test scheduling activation upon sign up (MyApp.AccountTest) Expected a

    job matching: %{args: %{id: 4}, queue: :default, worker: MyApp.ActivationWorker} to be enqueued in the "public" schema. Instead found: [] code: assert_enqueued worker: MyApp.ActivationWorker,
  45. test "scheduling activation upon sign up" do {:ok, account} =

    MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") expected_opts = [ worker: MyApp.ActivationWorker, args: %{id: account.id}, queue: :default ] assert_enqueued expected_opts, 150 end account_test.exs
  46. . Finished in 0.1 seconds (0.1s async, 0.00s sync) 1

    test, 0 failures
  47. test "skipping activation on failed sign up" do {:error, _changeset}

    = MyApp.Account.sign_up(name: "", email: "") refute_enqueued [worker: MyApp.ActivationWorker], 150 end account_test.exs
  48. . Finished in 0.1 seconds (0.1s async, 0.00s sync) 2

    tests, 0 failures
  49. Use assert_enqueued/* and refute_enqueued/* to: • Assert Jobs in the

    Database • Refute Jobs in the Database • Query by Worker/Args/Meta/Queue/ScheduledAt/Etc. • Compensate for Async Activity with Timeouts
  50. Acceptance Testing

  51. Application Database Worker Worker Worker Worker Acceptance

  52. @type drain_option :: {:queue, queue_name()} | {:with_limit, pos_integer()} | {:with_recursion,

    boolean()} | {:with_safety, boolean()} | {:with_scheduled, boolean()}
 @type drain_result :: %{ failure: non_neg_integer(), snoozed: non_neg_integer(), success: non_neg_integer() } @doc """ Synchronously execute all available jobs in a queue. """ @spec drain_queue([drain_option()]) :: drain_result() oban.ex
  53. Wait… That’s not in the Oban.Testing module!

  54. test "delivering a follow-up welcome email after sign up" do

    {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") assert %{success: 1} = Oban.drain_queue(queue: :emails) assert_email_sent MyApp.AccountEmail.welcome(account) end account_test.exs
  55. test "delivering a follow-up welcome email after sign up" do

    {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") assert %{success: 1} = Oban.drain_queue(queue: :emails) assert_email_sent MyApp.AccountEmail.welcome(account) end account_test.exs
  56. test "delivering a follow-up welcome email after sign up" do

    {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") assert %{success: 1} = Oban.drain_queue(queue: :emails) assert_email_sent MyApp.AccountEmail.welcome(account) end account_test.exs
  57. 1) test delivering a follow-up welcome email after sign up

    (MyApp.AccountTest) match (=) failed code: assert %{success: 1} = Oban.drain_queue(queue: :emails) left: %{success: 1} right: %{success: 0, failure: 0, snoozed: 0}
  58. def sign_up(args) do changeset = changeset(args) with {:ok, account} <-

    MyApp.Repo.insert(changeset) do Oban.insert_all([ MyApp.ActivationWorker.new(%{id: account.id}), MyApp.WelcomeWorker.new(%{id: account.id}, schedule_in: {1, :hour}) ]) {:ok, account} end end account.ex
  59. defmodule MyApp.WelcomeWorker do use Oban.Worker, queue: :emails @impl Worker def

    perform(%Job{args: %{"id" => account_id}}) do {:ok, account} = MyApp.Repo.get(MyApp.Account, account_id) account |> MyApp.AccountEmail.welcome() |> MyApp.Mailer.deliver() end end welcome_worker.ex
  60. 1) test delivering a follow-up welcome email after sign up

    (MyApp.AccountTest) match (=) failed code: assert %{success: 1} = Oban.drain_queue(queue: :emails) left: %{success: 1} right: %{success: 0, failure: 0, snoozed: 0}
  61. test "delivering a follow-up welcome email after sign up" do

    {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") assert %{success: 1} = Oban.drain_queue(queue: :emails, with_scheduled: true) assert_email_sent MyApp.AccountEmail.welcome(account) end account_test.exs
  62. 1) test delivering a follow-up welcome email after sign up

    (MyApp.AccountTest) match (=) failed code: assert %{success: 1} = Oban.drain_queue(queue: :emails, with_scheduled: true) left: %{success: 1} right: %{success: 0, failure: 1, snoozed: 0}
  63. test "delivering a follow-up welcome email after sign up" do

    {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") assert %{success: 1} = Oban.drain_queue(queue: :emails, with_safety: false, with_scheduled: true) assert_email_sent MyApp.AccountEmail.welcome(account) end account_test.exs
  64. 1) test delivering a follow-up welcome email after sign up

    (MyApp.AccountTest) ** (MatchError) no match of right hand side value: %MyApp.Account{active: false, email: "parker@example.com", id: 16, name: "ElixirConf"} code: Oban.drain_queue(queue: :emails, with_scheduled: true, with_safety: false)
  65. defmodule MyApp.WelcomeWorker do use Oban.Worker, queue: :emails @impl Worker def

    perform(%Job{args: %{"id" => account_id}}) do {:ok, account} = MyApp.Repo.get(MyApp.Account, account_id) account |> MyApp.AccountEmail.welcome() |> MyApp.Mailer.deliver() end end welcome_worker.ex
  66. defmodule MyApp.WelcomeWorker do use Oban.Worker, queue: :emails @impl Worker def

    perform(%Job{args: %{"id" => account_id}}) do account = MyApp.Repo.get(MyApp.Account, account_id) account |> MyApp.AccountEmail.welcome() |> MyApp.Mailer.deliver() end end welcome_worker.ex
  67. . Finished in 0.2 seconds (0.2s async, 0.00s sync) 3

    tests, 0 failures, 0 excluded
  68. test "starting our welcome drip after sign up" do {:ok,

    _account} = MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") assert %{success: 4, failure: 0, snoozed: 0} = Oban.drain_queue(queue: :emails, with_scheduled: true, with_safety: false) end account_test.exs
  69. defmodule MyApp.ActivationWorker do use Oban.Worker @impl Worker def perform(%Job{args: %{"id"

    => account_id} = args}) do with {:activated, account} <- MyApp.Account.activate(account_id) do Oban.insert_all([ MyApp.HelloWorker.new(args, schedule_in: {1, :hour}), MyApp.IntroWorker.new(args, schedule_in: {1, :day}), MyApp.NudgeWorker.new(args, schedule_in: {1, :week}) ]) {:ok, account} end end end activation_worker.ex
  70. 1) test starting our welcome drip after sign up (MyApp.AccountTest)

    match (=) failed code: assert %{success: 4, failure: 0, snoozed: 0} = Oban.drain_queue(queue: :emails, with_scheduled: true, with_safety: false) left: %{failure: 0, snoozed: 0, success: 4} right: %{failure: 0, snoozed: 0, success: 1}
  71. test "starting our welcome drip after sign up" do {:ok,

    _account} = MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") assert %{success: 4, failure: 0, snoozed: 0} = Oban.drain_queue( queue: :emails, with_scheduled: true, with_safety: false, with_recursion: true ) end account_test.exs
  72. . Finished in 0.2 seconds (0.2s async, 0.00s sync) 4

    tests, 0 failures, 0 excluded
  73. Use drain_queue/* to: • Execute All Jobs in a Particular

    Queue • Make Scheduled Jobs Immediately Available • Surface Job Errors and Exceptions • Recursively Execute All Jobs
  74. Inline Testing

  75. If you absolutely must run jobs normally during your tests.

    (for browser testing or live view)
  76. config :my_app, Oban, name: Oban.Disabled, plugins: false, queues: false test.exs

  77. setup do start_supervised_oban!(queues: [default: 5, emails: 5]) :ok end any_test.exs

  78. @doc """ Set up a supvervised version of Oban with

    the provided options. """ def start_supervised_oban!(opts) do opts = opts |> Keyword.put_new(:name, Oban) |> Keyword.put_new(:repo, Repo) |> Keyword.put_new(:plugins, [{Repeater, interval: 25}]) |> Keyword.put_new(:poll_interval, :timer.minutes(10)) |> Keyword.put_new(:shutdown_grace_period, 25) start_supervised!({Oban, opts}, id: opts[:name]) end test_helper.exs
  79. @doc """ Set up a supvervised version of Oban with

    the provided options. """ def start_supervised_oban!(opts) do opts = opts |> Keyword.put_new(:name, Oban) |> Keyword.put_new(:repo, Repo) |> Keyword.put_new(:plugins, [{Repeater, interval: 25}]) |> Keyword.put_new(:poll_interval, :timer.minutes(10)) |> Keyword.put_new(:shutdown_grace_period, 25) start_supervised!({Oban, opts}, id: opts[:name]) end test_helper.exs
  80. @doc """ Set up a supvervised version of Oban with

    the provided options. """ def start_supervised_oban!(opts) do opts = opts |> Keyword.put_new(:name, Oban) |> Keyword.put_new(:repo, Repo) |> Keyword.put_new(:plugins, [{Repeater, interval: 25}]) |> Keyword.put_new(:poll_interval, :timer.minutes(10)) |> Keyword.put_new(:shutdown_grace_period, 25) start_supervised!({Oban, opts}, id: opts[:name]) end test_helper.exs
  81. @doc """ Set up a supvervised version of Oban with

    the provided options. """ def start_supervised_oban!(opts) do opts = opts |> Keyword.put_new(:name, Oban) |> Keyword.put_new(:repo, Repo) |> Keyword.put_new(:plugins, [{Repeater, interval: 25}]) |> Keyword.put_new(:poll_interval, :timer.minutes(10)) |> Keyword.put_new(:shutdown_grace_period, 25) start_supervised!({Oban, opts}, id: opts[:name]) end test_helper.exs
  82. Use start_supervised_oban!/1 to: • Run Jobs Normally in the Background

    • …Without Scheduling • …Without Isolation • …Without Control
  83. Write tests for your Oban jobs! Whether it’s from the

    inside-out or the outside-in, that’s up to you.
  84. 💛 Thanks For Listening