Tweet
Tweet

Slide 1

Slide 1 text

Testing Oban Jobs from the Inside Out

Slide 2

Slide 2 text

Parker Selbert @sorentwo Dscout / Soren Hiring!!! Not Hiring…

Slide 3

Slide 3 text

Oban is a persistent background job system with layered components. (queues, workers, jobs, plugins, telemetry)

Slide 4

Slide 4 text

Automated testing is an essential component of any reliable application. (at least those that are long-lived and actively developed 😉)

Slide 5

Slide 5 text

We need to verify that all moving parts are working together as we expect.

Slide 6

Slide 6 text

Database Application Queue Queue Queue Worker Job Job Job Job Job Job Job Job Worker Worker Context Controller

Slide 7

Slide 7 text

Where you focus test effort depends on performance, isolation, and boundaries.

Slide 8

Slide 8 text

Unit Integration Acceptance Recommended Testing Pyramid

Slide 9

Slide 9 text

Background jobs are always about side effects.

Slide 10

Slide 10 text

Great tests are async, isolated within transactions.

Slide 11

Slide 11 text

Testing Setup

Slide 12

Slide 12 text

Play nicely with Ecto’s SQL.Sandbox for isolated testing.

Slide 13

Slide 13 text

config :my_app, Oban, repo: MyApp.Repo, plugins: [Oban.Plugins.Pruner, Oban.Plugins.Stager], queues: [ default: 10, events: 20, emails: 5 ] config.exs

Slide 14

Slide 14 text

config :my_app, Oban, plugins: false, queues: false test.exs

Slide 15

Slide 15 text

Use Oban.Testing for unit, integration, and acceptance tests.

Slide 16

Slide 16 text

defmodule MyApp.Case do use ExUnit.CaseTemplate using do quote do use Oban.Testing, repo: MyApp.Repo end end setup do :ok = Ecto.Adapters.SQL.Sandbox.mode(MyApp.Repo, :manual) :ok = Ecto.Adapters.SQL.Sandbox.checkout(MyApp.Repo) end end MyApp.Repo.start_link() ExUnit.start() test_helper.exs

Slide 17

Slide 17 text

Now we’re ready for some Testing!

Slide 18

Slide 18 text

Unit Testing

Slide 19

Slide 19 text

Application Worker Worker Worker Worker Database Unit

Slide 20

Slide 20 text

@doc """ Construct a job and execute it with a worker module. """ @spec perform_job( worker :: Worker.t(), args :: term(), opts :: [Job.option() | {:repo, module()}] ) :: Worker.result() oban/testing.ex

Slide 21

Slide 21 text

defmodule MyApp.ActivationWorkerTest do use MyApp.Case, async: true test "activating a new account" do account = MyApp.Account.create(name: “ElixirConf”, email: “[email protected]”) {:ok, account} = perform_job(MyApp.ActivationWorker, %{id: account.id}) assert account.active? end end activation_worker_test.exs

Slide 22

Slide 22 text

defmodule MyApp.ActivationWorkerTest do use MyApp.Case, async: true test "activating a new account" do account = MyApp.Account.create(name: “ElixirConf”, email: “[email protected]") {:ok, account} = perform_job(MyApp.ActivationWorker, %{id: account.id}) assert account.active? end end activation_worker_test.exs

Slide 23

Slide 23 text

1) test activating a new account (MyApp.AccountWorkerTest) Expected worker to be a module that implements the Oban.Worker behaviour, got: MyApp.AccountWorker code: {:ok, account} = perform_job(MyApp.AccountWorker, %{id: account.id})

Slide 24

Slide 24 text

defmodule MyApp.ActivationWorker do use Oban.Worker @impl Worker def perform(%Job{args: %{id: account_id}}) do MyApp.Account.activate(account_id) end end activation_worker.ex

Slide 25

Slide 25 text

1) test activating a new account (MyApp.AccountWorkerTest) ** (FunctionClauseError) no function clause matching in MyApp.AccountWorker.perform/1 The following arguments were given to MyApp.AccountWorker.perform/1: # 1 %Oban.Job{args: %{"id" => 1}, attempt: 1, worker: "MyApp.AccountWorker"} code: {:ok, account} = perform_job(MyApp.AccountWorker, %{id: account.id})

Slide 26

Slide 26 text

defmodule MyApp.ActivationWorker do use Oban.Worker @impl Worker def perform(%Job{args: %{"id" => account_id}}) do MyApp.Account.activate(account_id) end end activation_worker.ex

Slide 27

Slide 27 text

1) test activating a new account (MyApp.AccountWorkerTest) Expected result to be one of - `:ok` - `:discard` - `{:ok, value}` - `{:error, reason}` - `{:discard, reason}` - `{:snooze, duration} Instead received: {:activated, %MyApp.Account{active?: true, id: 1, name: nil}} code: {:ok, account} = perform_job(MyApp.AccountWorker, %{id: account.id})

Slide 28

Slide 28 text

defmodule MyApp.ActivationWorker do use Oban.Worker @impl Worker def perform(%Job{args: %{id: account_id}}) do with {:activated, account} <- MyApp.Account.activate(account_id) do {:ok, account} end end end activation_worker.ex

Slide 29

Slide 29 text

. Finished in 0.3 seconds (0.2s on load, 0.08s async, 0.00s sync) 1 test, 0 failures

Slide 30

Slide 30 text

defmodule MyApp.ActivationWorkerTest do use MyApp.Case, async: true test "activating a new account" do account = MyApp.Account.create(name: “ElixirConf”, email: “[email protected]”) {:ok, account} = perform_job(MyApp.ActivationWorker, %{id: account.id}, priority: 9) assert account.active? end end activation_worker_test.exs

Slide 31

Slide 31 text

1) test activating a new account (MyApp.ActivationWorkerTest) Expected args and opts to build a valid job, got validation errors: priority: must be less than 4 code: {:ok, account} = perform_job(MyApp.ActivationWorker, %{id: id}, priority: 9)

Slide 32

Slide 32 text

defmodule MyApp.ActivationWorkerTest do use MyApp.Case, async: true test "activating a new account" do account = MyApp.Account.create(name: “ElixirConf”, email: “[email protected]”) {:ok, account} = perform_job(MyApp.ActivationWorker, %{id: account.id}, priority: 3) assert account.active? end end activation_worker_test.exs

Slide 33

Slide 33 text

. Finished in 0.3 seconds (0.2s on load, 0.08s async, 0.00s sync) 1 test, 0 failures

Slide 34

Slide 34 text

Use perform_job/1 to: • Ensure Worker Behavior • Check String Keys in perform/1 • Verify Return Values • Encode/Decode Args to JSON • Validate Job Opts • Exercise a Custom new/1 Callback

Slide 35

Slide 35 text

Integration Testing

Slide 36

Slide 36 text

Application Worker Worker Worker Worker Database Integration

Slide 37

Slide 37 text

@doc """ Assert that a job with particular options has been enqueued. """ @spec assert_enqueued(opts :: Keyword.t()) :: true @doc """ Refute that a job with particular options has been enqueued. """ @spec refute_enqueued(opts :: Keyword.t()) :: false oban/testing.ex

Slide 38

Slide 38 text

defmodule MyApp.AccountTest do use MyApp.Case, async: true test "scheduling activation upon sign up" do {:ok, account} = MyApp.Account.sign_up(name: “ElixirConf”, email: “[email protected]") assert_enqueued worker: MyApp.ActivationWorker, args: %{id: account.id}, queue: :default end end account_test.exs

Slide 39

Slide 39 text

defmodule MyApp.AccountTest do use MyApp.Case, async: true test "scheduling activation upon sign up" do {:ok, account} = MyApp.Account.sign_up(name: “ElixirConf”, email: “[email protected]") assert_enqueued worker: MyApp.ActivationWorker, args: %{id: account.id}, queue: :default end end account_test.exs

Slide 40

Slide 40 text

1) test scheduling activation upon sign up (MyApp.AccountTest) Expected a job matching: %{args: %{id: 4}, queue: :default, worker: MyApp.ActivationWorker} to be enqueued in the "public" schema. Instead found: [] code: assert_enqueued worker: MyApp.ActivationWorker,

Slide 41

Slide 41 text

defmodule MyApp.Account do def sign_up(args) do changeset = changeset(args) with {:ok, account} <- MyApp.Repo.insert(changeset) do %{id: account.id} |> MyApp.ActivationWorker.new() |> Oban.insert() {:ok, account} end end account.ex

Slide 42

Slide 42 text

. Finished in 0.1 seconds (0.1s async, 0.00s sync) 1 test, 0 failures

Slide 43

Slide 43 text

def sign_up(args) do changeset = changeset(args) with {:ok, account} <- MyApp.Repo.insert(changeset) do Task.start(fn -> Process.sleep(100) %{id: account.id} |> MyApp.ActivationWorker.new() |> Oban.insert() end) {:ok, account} end end account.ex

Slide 44

Slide 44 text

1) test scheduling activation upon sign up (MyApp.AccountTest) Expected a job matching: %{args: %{id: 4}, queue: :default, worker: MyApp.ActivationWorker} to be enqueued in the "public" schema. Instead found: [] code: assert_enqueued worker: MyApp.ActivationWorker,

Slide 45

Slide 45 text

test "scheduling activation upon sign up" do {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "[email protected]") expected_opts = [ worker: MyApp.ActivationWorker, args: %{id: account.id}, queue: :default ] assert_enqueued expected_opts, 150 end account_test.exs

Slide 46

Slide 46 text

. Finished in 0.1 seconds (0.1s async, 0.00s sync) 1 test, 0 failures

Slide 47

Slide 47 text

test "skipping activation on failed sign up" do {:error, _changeset} = MyApp.Account.sign_up(name: "", email: "") refute_enqueued [worker: MyApp.ActivationWorker], 150 end account_test.exs

Slide 48

Slide 48 text

. Finished in 0.1 seconds (0.1s async, 0.00s sync) 2 tests, 0 failures

Slide 49

Slide 49 text

Use assert_enqueued/* and refute_enqueued/* to: • Assert Jobs in the Database • Refute Jobs in the Database • Query by Worker/Args/Meta/Queue/ScheduledAt/Etc. • Compensate for Async Activity with Timeouts

Slide 50

Slide 50 text

Acceptance Testing

Slide 51

Slide 51 text

Application Database Worker Worker Worker Worker Acceptance

Slide 52

Slide 52 text

@type drain_option :: {:queue, queue_name()} | {:with_limit, pos_integer()} | {:with_recursion, boolean()} | {:with_safety, boolean()} | {:with_scheduled, boolean()}
 @type drain_result :: %{ failure: non_neg_integer(), snoozed: non_neg_integer(), success: non_neg_integer() } @doc """ Synchronously execute all available jobs in a queue. """ @spec drain_queue([drain_option()]) :: drain_result() oban.ex

Slide 53

Slide 53 text

Wait… That’s not in the Oban.Testing module!

Slide 54

Slide 54 text

test "delivering a follow-up welcome email after sign up" do {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "[email protected]") assert %{success: 1} = Oban.drain_queue(queue: :emails) assert_email_sent MyApp.AccountEmail.welcome(account) end account_test.exs

Slide 55

Slide 55 text

test "delivering a follow-up welcome email after sign up" do {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "[email protected]") assert %{success: 1} = Oban.drain_queue(queue: :emails) assert_email_sent MyApp.AccountEmail.welcome(account) end account_test.exs

Slide 56

Slide 56 text

test "delivering a follow-up welcome email after sign up" do {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "[email protected]") assert %{success: 1} = Oban.drain_queue(queue: :emails) assert_email_sent MyApp.AccountEmail.welcome(account) end account_test.exs

Slide 57

Slide 57 text

1) test delivering a follow-up welcome email after sign up (MyApp.AccountTest) match (=) failed code: assert %{success: 1} = Oban.drain_queue(queue: :emails) left: %{success: 1} right: %{success: 0, failure: 0, snoozed: 0}

Slide 58

Slide 58 text

def sign_up(args) do changeset = changeset(args) with {:ok, account} <- MyApp.Repo.insert(changeset) do Oban.insert_all([ MyApp.ActivationWorker.new(%{id: account.id}), MyApp.WelcomeWorker.new(%{id: account.id}, schedule_in: {1, :hour}) ]) {:ok, account} end end account.ex

Slide 59

Slide 59 text

defmodule MyApp.WelcomeWorker do use Oban.Worker, queue: :emails @impl Worker def perform(%Job{args: %{"id" => account_id}}) do {:ok, account} = MyApp.Repo.get(MyApp.Account, account_id) account |> MyApp.AccountEmail.welcome() |> MyApp.Mailer.deliver() end end welcome_worker.ex

Slide 60

Slide 60 text

1) test delivering a follow-up welcome email after sign up (MyApp.AccountTest) match (=) failed code: assert %{success: 1} = Oban.drain_queue(queue: :emails) left: %{success: 1} right: %{success: 0, failure: 0, snoozed: 0}

Slide 61

Slide 61 text

test "delivering a follow-up welcome email after sign up" do {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "[email protected]") assert %{success: 1} = Oban.drain_queue(queue: :emails, with_scheduled: true) assert_email_sent MyApp.AccountEmail.welcome(account) end account_test.exs

Slide 62

Slide 62 text

1) test delivering a follow-up welcome email after sign up (MyApp.AccountTest) match (=) failed code: assert %{success: 1} = Oban.drain_queue(queue: :emails, with_scheduled: true) left: %{success: 1} right: %{success: 0, failure: 1, snoozed: 0}

Slide 63

Slide 63 text

test "delivering a follow-up welcome email after sign up" do {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "[email protected]") assert %{success: 1} = Oban.drain_queue(queue: :emails, with_safety: false, with_scheduled: true) assert_email_sent MyApp.AccountEmail.welcome(account) end account_test.exs

Slide 64

Slide 64 text

1) test delivering a follow-up welcome email after sign up (MyApp.AccountTest) ** (MatchError) no match of right hand side value: %MyApp.Account{active: false, email: "[email protected]", id: 16, name: "ElixirConf"} code: Oban.drain_queue(queue: :emails, with_scheduled: true, with_safety: false)

Slide 65

Slide 65 text

defmodule MyApp.WelcomeWorker do use Oban.Worker, queue: :emails @impl Worker def perform(%Job{args: %{"id" => account_id}}) do {:ok, account} = MyApp.Repo.get(MyApp.Account, account_id) account |> MyApp.AccountEmail.welcome() |> MyApp.Mailer.deliver() end end welcome_worker.ex

Slide 66

Slide 66 text

defmodule MyApp.WelcomeWorker do use Oban.Worker, queue: :emails @impl Worker def perform(%Job{args: %{"id" => account_id}}) do account = MyApp.Repo.get(MyApp.Account, account_id) account |> MyApp.AccountEmail.welcome() |> MyApp.Mailer.deliver() end end welcome_worker.ex

Slide 67

Slide 67 text

. Finished in 0.2 seconds (0.2s async, 0.00s sync) 3 tests, 0 failures, 0 excluded

Slide 68

Slide 68 text

test "starting our welcome drip after sign up" do {:ok, _account} = MyApp.Account.sign_up(name: "ElixirConf", email: "[email protected]") assert %{success: 4, failure: 0, snoozed: 0} = Oban.drain_queue(queue: :emails, with_scheduled: true, with_safety: false) end account_test.exs

Slide 69

Slide 69 text

defmodule MyApp.ActivationWorker do use Oban.Worker @impl Worker def perform(%Job{args: %{"id" => account_id} = args}) do with {:activated, account} <- MyApp.Account.activate(account_id) do Oban.insert_all([ MyApp.HelloWorker.new(args, schedule_in: {1, :hour}), MyApp.IntroWorker.new(args, schedule_in: {1, :day}), MyApp.NudgeWorker.new(args, schedule_in: {1, :week}) ]) {:ok, account} end end end activation_worker.ex

Slide 70

Slide 70 text

1) test starting our welcome drip after sign up (MyApp.AccountTest) match (=) failed code: assert %{success: 4, failure: 0, snoozed: 0} = Oban.drain_queue(queue: :emails, with_scheduled: true, with_safety: false) left: %{failure: 0, snoozed: 0, success: 4} right: %{failure: 0, snoozed: 0, success: 1}

Slide 71

Slide 71 text

test "starting our welcome drip after sign up" do {:ok, _account} = MyApp.Account.sign_up(name: "ElixirConf", email: "[email protected]") assert %{success: 4, failure: 0, snoozed: 0} = Oban.drain_queue( queue: :emails, with_scheduled: true, with_safety: false, with_recursion: true ) end account_test.exs

Slide 72

Slide 72 text

. Finished in 0.2 seconds (0.2s async, 0.00s sync) 4 tests, 0 failures, 0 excluded

Slide 73

Slide 73 text

Use drain_queue/* to: • Execute All Jobs in a Particular Queue • Make Scheduled Jobs Immediately Available • Surface Job Errors and Exceptions • Recursively Execute All Jobs

Slide 74

Slide 74 text

Inline Testing

Slide 75

Slide 75 text

If you absolutely must run jobs normally during your tests. (for browser testing or live view)

Slide 76

Slide 76 text

config :my_app, Oban, name: Oban.Disabled, plugins: false, queues: false test.exs

Slide 77

Slide 77 text

setup do start_supervised_oban!(queues: [default: 5, emails: 5]) :ok end any_test.exs

Slide 78

Slide 78 text

@doc """ Set up a supvervised version of Oban with the provided options. """ def start_supervised_oban!(opts) do opts = opts |> Keyword.put_new(:name, Oban) |> Keyword.put_new(:repo, Repo) |> Keyword.put_new(:plugins, [{Repeater, interval: 25}]) |> Keyword.put_new(:poll_interval, :timer.minutes(10)) |> Keyword.put_new(:shutdown_grace_period, 25) start_supervised!({Oban, opts}, id: opts[:name]) end test_helper.exs

Slide 79

Slide 79 text

@doc """ Set up a supvervised version of Oban with the provided options. """ def start_supervised_oban!(opts) do opts = opts |> Keyword.put_new(:name, Oban) |> Keyword.put_new(:repo, Repo) |> Keyword.put_new(:plugins, [{Repeater, interval: 25}]) |> Keyword.put_new(:poll_interval, :timer.minutes(10)) |> Keyword.put_new(:shutdown_grace_period, 25) start_supervised!({Oban, opts}, id: opts[:name]) end test_helper.exs

Slide 80

Slide 80 text

@doc """ Set up a supvervised version of Oban with the provided options. """ def start_supervised_oban!(opts) do opts = opts |> Keyword.put_new(:name, Oban) |> Keyword.put_new(:repo, Repo) |> Keyword.put_new(:plugins, [{Repeater, interval: 25}]) |> Keyword.put_new(:poll_interval, :timer.minutes(10)) |> Keyword.put_new(:shutdown_grace_period, 25) start_supervised!({Oban, opts}, id: opts[:name]) end test_helper.exs

Slide 81

Slide 81 text

@doc """ Set up a supvervised version of Oban with the provided options. """ def start_supervised_oban!(opts) do opts = opts |> Keyword.put_new(:name, Oban) |> Keyword.put_new(:repo, Repo) |> Keyword.put_new(:plugins, [{Repeater, interval: 25}]) |> Keyword.put_new(:poll_interval, :timer.minutes(10)) |> Keyword.put_new(:shutdown_grace_period, 25) start_supervised!({Oban, opts}, id: opts[:name]) end test_helper.exs

Slide 82

Slide 82 text

Use start_supervised_oban!/1 to: • Run Jobs Normally in the Background • …Without Scheduling • …Without Isolation • …Without Control

Slide 83

Slide 83 text

Write tests for your Oban jobs! Whether it’s from the inside-out or the outside-in, that’s up to you.

Slide 84

Slide 84 text

💛 Thanks For Listening