Share
Beyond Mocks - Messing with Our Preconceptions of Testing
In this session, “Beyond Mocks: Messing with Our Preconceptions of Testing,” we will experiment with a different approach to testing in Elixir that steps away from traditional reliance on mocks. If you’ve ever been frustrated by broken tests that break every time you refactor, this talk might offer you a fresh perspective.
We’ll begin by discussing the challenges of using test mocks and how they can sometimes lead to unreliable test results. From there, I’ll introduce you to the Nullable pattern, an experimental technique applied to testing with external dependencies in Elixir. Instead of leaning on test mocks, this technique uses “nullable infrastructure wrappers” as an attempt to simplify dependencies and make your tests more robust and easier to maintain.
I’ll walk you through practical examples of how I tried to apply the patterns of Nullables in Elixir. While this approach is in development, it could offer a new way to think about testing that better aligns with Elixir’s strengths. And perhaps even make you use test mocks with clearer intent.
By the end of our time together, my goal is to encourage you to experiment with your own testing strategies, exploring how alternative techniques might lead to cleaner, more effective code. Let’s dive into this experiment and challenge our preconceptions about testing!
Nicholas Henry
Transcript
183 slides
-
The Impact of Infrastructure Services • Unreliable test cases •
Slow execution • Expensive to run -
Test Mocks • Different experience than our usual test flow
• From sociable, state- based to solitary, interaction-based -
Test Mocks • Different experience than our usual test flow
• From sociable, state- based to solitary, interaction-based • Design fatigue -
# terminal 1 ❯❯❯ export OPEN_WEATHER_MAP_APP_ID=$(op read op:/#some-vault/app-key) ❯❯❯ mix
server 09:43:10.195 [info] Plug now running on localhost:4000 -
# terminal 1 ❯❯❯ export OPEN_WEATHER_MAP_APP_ID=$(op read op:/#some-vault/app-key) ❯❯❯ mix
server 09:43:10.195 [info] Plug now running on localhost:4000 # terminal 2 ❯❯❯ http “http:/#localhost:4000/?lat=28.5383&lon=-81.3792” # orlando HTTP/1.1 200 OK cache-control: max-age=0, private, must-revalidate content-type: text/plain; charset=utf-8 date: Thu, 29 Aug 2024 13:43:14 GMT server: Cowboy Yes, definitey! # or No, not today! -
What to expect today? • An experiment • A mental
model of working with Infrastructure -
What to expect today? • An experiment • A mental
model of working with Infrastructure • You might learn something new about Elixir -
What to expect today? • An experiment • A mental
model of working with Infrastructure • You might learn something new about Elixir • Better understanding of test mocks -
What to expect today? • An experiment • A mental
model of working with Infrastructure • You might learn something new about Elixir • Better understanding of test mocks • Mess with your preconceptions of testing -
Test Type Test Case How Impact Duration Solitary Isolate the
SUT Change in wrapper invalidates the mock False Pass Persistent Interaction-based Verify Behaviour Change in wrapper AND SUT invalidates the mock False Failure Transient -
Test Type Test Case How Impact Duration Solitary Isolate the
SUT Change in wrapper invalidates the mock False Pass Persistent Interaction-based Verify Behaviour Change in wrapper AND SUT invalidates the mock False Failure Transient -
- + - defmodule MyUmbrella.WeatherApi.Behaviour do alias MyUmbrella.WeatherApi.Response @type coordinates
:( {float(), float()} @callback get_forecast(coordinates, duration :( :today, URI.t()) :( {:ok, Response.t()} | {:error, term} {:ok, jason :( String.t()} | {:error, term} end -
- + defmodule MyUmbrella.WeatherApi.Http do @behaviour MyUmbrella.WeatherApi.Behaviour @impl true def
get_forecast(coordinates, :today, base_url) do url = build_url(coordinates, base_url) case HTTPoison.get(url) do {:ok, %HTTPoison.Response{status_code: 200} = response} -* {:ok, Jason.decode!(response.body)} {:ok, response.body} {:ok, %HTTPoison.Response{status_code: status_code}} -* {:error, {:status, status_code}} end end end -
- + defmodule MyUmbrella.WeatherApi.HttpTest do describe "getting a forecast" do
test "handles a response with a success status code” do london = Coordinates.new(51.5098, -0.118) result = WeatherApi.get_forecast(london, :today, test_server_url) assert {:ok, data} = result assert %{"current" =, _, "hourly" =, _} = data assert %{"current" =, _, "hourly" =, _} = Jason.decode!(data) end end end -
iex(1)> MyUmbrella.for_today({28.5383, 81.3792}) *. (FunctionClauseError) no function clause matching in
Response.to_weather_report/1 The following arguments were given to MyUmbrella.WeatherApi.Response.to_weather_r # 1 "{\"lat\":28.5383,\"lon\":81.3792,\"timezone\":\"Asia/Kathmandu\"",<0 ..2 Attempted function clauses (showing 1 out of 1): def to_weather_report(%{"current" =, current, "hourly" =, hourly} = response) (my_umbrella 0.1.0) lib/my_umbrella/weather_api/response.ex:25: (my_umbrella 0.1.0) lib/my_umbrella.ex:15: MyUmbrella.for_today/2 iex:1: (file) -
❯❯❯ mix test Compiling 3 files (.ex) Running ExUnit with
seed: 169237, max_cases: 24 Excluding tags: [:exclude] .................... Finished in 0.3 seconds (0.2s async, 0.1s sync) 20 tests, 0 failures ❯❯❯ mix dialyzer Compiling 2 files (.ex) Finding suitable PLTs Checking PLT..2 Total errors: 0, Skipped: 0, Unnecessary Skips: 0 done in 0m6.13s done (passed successfully) ! ! -
defmodule MyUmbrella.WeatherApi do @behaviour MyUmbrella.WeatherApi.Behaviour @weather_api_module Application.compile_env( :my_umbrella, :weather_api_module, MyUmbrella.WeatherApi.Http
) @impl true def get_forecast(coordinates, duration, url \\ nil), do: @weather_api_module.get_forecast(coordinates, duration, url) end " Dialyzer Dynamic invocation Reference: https://elixirforum.com/t/does-dialyzer-check-behaviours-and-callbacks/2101/6 -
test "given it IS raining before end-of-day; then an umbrella
IS needed" do london = Coordinates.new(51.5098, -0.118) current_date_time_utc = DateTime.new!(~D[2000-01-01], ~T[21:30:00Z], "Etc/UTC") expect(MyUmbrella.WeatherApi.Mock, :get_forecast, fn coordinates, :today, _test_server_url -* response = %{ "lat" =, lat, "lon" =, lon, "timezone" =, "Etc/UTC", "current" =, %{ "dt" =, 946_762_200, "weather" =, [ %{ "id" =, 802, "main" =, "Clouds", "description" =, "scattered clouds" } ] Keeping in sync with the replaced dependency -
Nullables and Elixir • Object Oriented to Functional? • Dependency
Injection? • Align with Elixir? -
Nullable Patterns • Low-level Infrastructure Wrapper • Embedded Stub •
Configurable Responses -
Nullable Patterns • Low-level Infrastructure Wrapper • Embedded Stub •
Configurable Responses • Output Tracking -
Nullable Patterns • Low-level Infrastructure Wrapper • Embedded Stub •
Configurable Responses • Output Tracking -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do alias MyUmbrella.Infrastructure.JsonHttp @spec get(String.t()) :( {:ok, JsonHttp.Response.t()}
def get(url) do request = JsonHttp.Request.new(url: url) {:ok, httpoison_response} = HTTPoison.get(request.url, request.headers) response = JsonHttp.Response.new( status_code: httpoison_response.status_code, headers: httpoison_response.headers, body: Jason.decode!(httpoison_response.body) ) {:ok, response} end end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do alias MyUmbrella.Infrastructure.JsonHttp @spec get(String.t()) :( {:ok, JsonHttp.Response.t()}
def get(url) do request = JsonHttp.Request.new(url: url) {:ok, httpoison_response} = HTTPoison.get(request.url, request.headers) response = JsonHttp.Response.new( status_code: httpoison_response.status_code, headers: httpoison_response.headers, body: Jason.decode!(httpoison_response.body) ) {:ok, response} end end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do alias MyUmbrella.Infrastructure.JsonHttp @spec get(String.t()) :( {:ok, JsonHttp.Response.t()}
def get(url) do request = JsonHttp.Request.new(url: url) {:ok, httpoison_response} = HTTPoison.get(request.url, request.headers) response = JsonHttp.Response.new( status_code: httpoison_response.status_code, headers: httpoison_response.headers, body: Jason.decode!(httpoison_response.body) ) {:ok, response} end end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do alias MyUmbrella.Infrastructure.JsonHttp @spec get(String.t()) :( {:ok, JsonHttp.Response.t()}
def get(url) do request = JsonHttp.Request.new(url: url) {:ok, httpoison_response} = HTTPoison.get(request.url, request.headers) response = JsonHttp.Response.new( status_code: httpoison_response.status_code, headers: httpoison_response.headers, body: Jason.decode!(httpoison_response.body) ) {:ok, response} end end -
Nullable Patterns • Low-level Infrastructure Wrapper • Embedded Stub •
Configurable Responses • Output Tracking -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do defmodule StubbedHTTPoison do @spec get(String.t(), HTTPoison.headers()) :(
{:ok, HTTPoison.Response.t()} def get(url, _headers) do # TODO {:ok, %HTTPoison.Response{}} end end @spec get(String.t()) :( {:ok, JsonHttp.Response.t()} def get(url) do request = JsonHttp.Request.new(url: url) {:ok, httpoison_response} = HTTPoison.get(request.url, request.headers) {:ok, response} end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do defmodule StubbedHTTPoison do @spec get(String.t(), HTTPoison.headers()) :(
{:ok, HTTPoison.Response.t()} def get(url, _headers) do # TODO {:ok, %HTTPoison.Response{}} end end @spec get(String.t()) :( {:ok, JsonHttp.Response.t()} def get(url) do request = JsonHttp.Request.new(url: url) {:ok, httpoison_response} = HTTPoison.get(request.url, request.headers) {:ok, response} end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do defmodule StubbedHTTPoison do @spec get(String.t(), HTTPoison.headers()) :(
{:ok, HTTPoison.Response.t()} def get(url, _headers) do # TODO {:ok, %HTTPoison.Response{}} end end @spec get(String.t()) :( {:ok, JsonHttp.Response.t()} def get(url) do request = JsonHttp.Request.new(url: url) {:ok, httpoison_response} = HTTPoison.get(request.url, request.headers) {:ok, response} end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do defmodule StubbedHTTPoison do @spec get(String.t(), HTTPoison.headers()) :(
{:ok, HTTPoison.Response.t()} def get(url, _headers) do # TODO {:ok, %HTTPoison.Response{}} end end @spec get(String.t()) :( {:ok, JsonHttp.Response.t()} def get(url) do request = JsonHttp.Request.new(url: url) {:ok, httpoison_response} = HTTPoison.get(request.url, request.headers) {:ok, response} end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do defmodule StubbedHTTPoison do @spec get(String.t(), HTTPoison.headers()) :(
{:ok, HTTPoison.Response.t()} def get(url, _headers) do # TODO {:ok, %HTTPoison.Response{}} end end @spec get(String.t()) :( {:ok, JsonHttp.Response.t()} def get(url) do request = JsonHttp.Request.new(url: url) {:ok, httpoison_response} = HTTPoison.get(request.url, request.headers) {:ok, response} end -
# defmodule MyUmbrella.Infrastructure.JsonHttp.Client do defmodule StubbedHTTPoison do @spec get(String.t(), HTTPoison.headers())
:( {:ok, HTTPoison.Response.t()} def get(url, _headers) do # TODO {:ok, %HTTPoison.Response{}} end end @spec get(String.t()) :( {:ok, JsonHttp.Response.t()} def get(url) do request = JsonHttp.Request.new(url: url) {:ok, httpoison_response} = HTTPoison.get(request.url, request.headers) {:ok, response} end -
Dependency Replacement • Dependency Parameterization • Dependency Injection • Application
Configuration; see Mox -
Dependency Replacement • Dependency Parameterization • Dependency Injection • Application
Configuration; see Mox • State Management, e.g. Mix.Shell -
Dependency Replacement • Dependency Parameterization • Dependency Injection • Application
Configuration; see Mox • State Management, e.g. Mix.Shell • AST Manipulation, e.g. Rewire -
defmodule MyUmbrella.Infrastructure.WeatherApi.Client do alias MyUmbrella.Infrastructure.WeatherApi @type t() :( %WeatherApi.Client{ app_id:
String.t(), url: String.t() } defstruct [:app_id, :url] @spec new :( t() def new do config = Application.get_env(:my_umbrella, :weather_api) struct!(WeatherApi.Client, config) end @spec get_forecast(t(), coordinates, duration :( :today) :( {:ok, Response.t()} def get_forecast(weather_api_client, coordinates, :today) do end end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do alias MyUmbrella.Infrastructure.JsonHttp @type t() :( %JsonHttp.Client{ httpoison:
HTTPoison | JsonHttp.Client.StubbedHTTPoison } @enforce_keys [:httpoison] defstruct [:httpoison] end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do alias MyUmbrella.Infrastructure.JsonHttp alias MyUmbrella.Infrastructure.JsonHttp.Client.StubbedHTTPoison @spec create() :(
t() def create do %JsonHttp.Client{httpoison: HTTPoison} end @spec create_null() :( t() def create_null() do %JsonHttp.Client{httpoison: StubbedHTTPoison} end end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do alias MyUmbrella.Infrastructure.JsonHttp alias MyUmbrella.Infrastructure.JsonHttp.Client.StubbedHTTPoison @spec create() :(
t() def create do %JsonHttp.Client{httpoison: HTTPoison} end @spec create_null() :( t() def create_null() do %JsonHttp.Client{httpoison: StubbedHTTPoison} end end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do alias MyUmbrella.Infrastructure.JsonHttp alias MyUmbrella.Infrastructure.JsonHttp.Client.StubbedHTTPoison @spec create() :(
t() def create do %JsonHttp.Client{httpoison: HTTPoison} end @spec create_null() :( t() def create_null() do %JsonHttp.Client{httpoison: StubbedHTTPoison} end end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do alias MyUmbrella.Infrastructure.JsonHttp @spec get(t(), String.t()) :( {:ok,
JsonHttp.Response.t()} def get(http_client, url) do request = JsonHttp.Request.new(url: url) {:ok, httpoison_resp} = http_client.httpoison.get(request.url, request.headers) response = JsonHttp.Response.new( status_code: httpoison_res.status_code, headers: httpoison_resp.headers, body: Jason.decode!(httpoison_resp.body) ) {:ok, response} end HTTPoison or StubbedHTTPoison -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do defmodule StubbedHTTPoison do @spec get(String.t(), HTTPoison.headers()) :(
{:ok, HTTPoison.Response.t()} def get(url, _headers) do # TODO {:ok, %HTTPoison.Response{}} end end @spec get(String.t()) :( {:ok, JsonHttp.Response.t()} def get(url) do request = JsonHttp.Request.new(url: url) {:ok, httpoison_response} = HTTPoison.get(request.url, request.headers) {:ok, response} end -
Nullable Patterns • Low-level Infrastructure Wrapper • Embedded Stub •
Configurable Responses • Output Tracking -
test "configurable response" do url = "http:/#NOT_CONNECTED/get" responses = [
{url, Response.new(status_code: 200, body: %{"hello" =, "world"})} ] http_client = JsonHttp.Client.create_null(responses) result = JsonHttp.Client.get(http_client, url) assert {:ok, response} = result assert response.status_code =4 200 assert {"Content-Type", "application/json; charset=utf-8"} in response.headers assert %{"hello" =, "world"} =4 response.body end Configured response -
test "configurable response" do url = "http:/#NOT_CONNECTED/get" responses = [
{url, Response.new(status_code: 200, body: %{"hello" =, "world"})} ] http_client = JsonHttp.Client.create_null(responses) result = JsonHttp.Client.get(http_client, url) assert {:ok, response} = result assert response.status_code =4 200 assert {"Content-Type", "application/json; charset=utf-8"} in response.headers assert %{"hello" =, "world"} =4 response.body end Configured response -
test "configurable response" do url = "http:/#NOT_CONNECTED/get" responses = [
{url, Response.new(status_code: 200, body: %{"hello" =, "world"})} ] http_client = JsonHttp.Client.create_null(responses) result = JsonHttp.Client.get(http_client, url) assert {:ok, response} = result assert response.status_code =4 200 assert {"Content-Type", "application/json; charset=utf-8"} in response.headers assert %{"hello" =, "world"} =4 response.body end Configured response -
test "configurable response" do url = "http:/#NOT_CONNECTED/get" responses = [
{url, Response.new(status_code: 200, body: %{"hello" =, "world"})} ] http_client = JsonHttp.Client.create_null(responses) result = JsonHttp.Client.get(http_client, url) assert {:ok, response} = result assert response.status_code =4 200 assert {"Content-Type", "application/json; charset=utf-8"} in response.headers assert %{"hello" =, "world"} =4 response.body end Configured response -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do defmodule StubbedHTTPoison do @spec get(String.t(), HTTPoison.headers()) :(
{:ok, HTTPoison.Response.t()} def get(url, _headers) do # TODO: Return configurable response {:ok, %HTTPoison.Response{}} end end end end Cannot change the signature to configure responses -
defmodule Nullables.ConfigurableResponses do use Agent @type responses() :( list({term(), term()})
@spec start_link(module(), responses()) :( Agent.on_start() def start_link(module, responses \\ []) do test_pid = self() agent_name = {test_pid, module} Agent.start_link(fn -* responses end, name: agent_name) end end {#PID<0.111.0>, :httpoison} -
defmodule Nullables.ConfigurableResponses do use Agent @type responses() :( list({term(), term()})
@spec start_link(module(), responses()) :( Agent.on_start() def start_link(module, responses \\ []) do test_pid = self() agent_name = {test_pid, module} Agent.start_link(fn -* responses end, name: agent_name) end end {#PID<0.111.0>, :httpoison} -
defmodule Nullables.ConfigurableResponses do use Agent @type responses() :( list({term(), term()})
@spec start_link(module(), responses()) :( Agent.on_start() def start_link(module, responses \\ []) do test_pid = self() agent_name = {test_pid, module} Agent.start_link(fn -* responses end, name: agent_name) end end {#PID<0.111.0>, :httpoison} -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do alias MyUmbrella.Infrastructure.JsonHttp @spec create() :( t() def
create do %JsonHttp.Client{httpoison: HTTPoison} end @spec create_null(ConfigurableResponses.responses()) :( t() def create_null(responses \\ []) do {:ok, _pid} = ConfigurableResponses.start_link(:httpoison, responses) %JsonHttp.Client{httpoison: StubbedHTTPoison} end end -
defmodule StubbedHTTPoison do @spec get(String.t(), HTTPoison.headers()) :( {:ok, HTTPoison.Response.t()} def
get(url, _headers) do response = url |6 get_response |6 to_httpoison {:ok, response} end defp get_response(url) do uri = URI.parse(url) endpoint = “#8uri.scheme}:/##8uri.host}#8uri.path}" ConfigurableResponses.get_response(:httpoison, endpoint) end -
defmodule Nullables.ConfigurableResponses do use Agent @spec get_response(module(), term(), term()) :(
term() def get_response(module, key, default \\ nil) do agent_name = {test_pid(), module} Agent.get_and_update(agent_name, &pop_response(&1, key)) |9 default end defp test_pid do case Process.get(:"$callers") do [parent_id | _] -* parent_id nil -* self() end end end {#PID<0.111.0>, :httpoison} -
defmodule Nullables.ConfigurableResponses do use Agent @spec get_response(module(), term(), term()) :(
term() def get_response(module, key, default \\ nil) do agent_name = {test_pid(), module} Agent.get_and_update(agent_name, &pop_response(&1, key)) |9 default end defp test_pid do case Process.get(:"$callers") do [parent_id | _] -* parent_id nil -* self() end end end {#PID<0.111.0>, :httpoison} -
test "configurable response" do url = "http:/#NOT_CONNECTED/get" responses = [
{url, Response.new(status_code: 200, body: %{"hello" =, "world"})} ] http_client = JsonHttp.Client.create_null(responses) result = JsonHttp.Client.get(http_client, url) assert {:ok, response} = result assert response.status_code =4 200 assert {"Content-Type", "application/json; charset=utf-8"} in response.headers assert %{"hello" =, "world"} =4 response.body end -
Nullable Patterns • Low-level Infrastructure Wrapper • Embedded Stub •
Configurable Responses • Output Tracking -
defmodule Examples.MixShellTest do use ExUnit.Case, async: true setup do Mix.shell(Mix.Shell.Process)
on_exit(fn -* Mix.shell(Mix.Shell.IO) end) end test "output tracking" do Mix.shell().info("hello") assert_receive {:mix_shell, :info, [msg]} assert msg =4 "hello" end end -
defmodule Examples.MixShellTest do use ExUnit.Case, async: true setup do Mix.shell(Mix.Shell.Process)
on_exit(fn -* Mix.shell(Mix.Shell.IO) end) end test "output tracking" do Mix.shell().info("hello") assert_receive {:mix_shell, :info, [msg]} assert msg =4 "hello" end end -
defmodule Examples.MixShellTest do use ExUnit.Case, async: true setup do Mix.shell(Mix.Shell.Process)
on_exit(fn -* Mix.shell(Mix.Shell.IO) end) end test "output tracking" do Mix.shell().info("hello") assert_receive {:mix_shell, :info, [msg]} assert msg =4 "hello" end end -
defmodule Examples.MixShellTest do use ExUnit.Case, async: true setup do Mix.shell(Mix.Shell.Process)
on_exit(fn -* Mix.shell(Mix.Shell.IO) end) end test "output tracking" do Mix.shell().info("hello") assert_receive {:mix_shell, :info, [msg]} assert msg =4 "hello" end end -
defmodule Examples.MixShellTest do use ExUnit.Case, async: true setup do Mix.shell(Mix.Shell.Process)
on_exit(fn -* Mix.shell(Mix.Shell.IO) end) end test "output tracking" do Mix.shell().info("hello") assert_receive {:mix_shell, :info, [msg]} assert msg =4 "hello" end end -
defmodule Examples.Mix.Shell.Process do def info(message) do send(message_target(), {:mix_shell, :info, [message]})
:ok end defp message_target() do case Process.get(:"$callers") do [parent | _] -* parent _ -* self() end end end -
defmodule Examples.Mix.Shell.Process do def info(message) do send(message_target(), {:mix_shell, :info, [message]})
:ok end defp message_target() do case Process.get(:"$callers") do [parent | _] -* parent _ -* self() end end end -
defmodule Examples.Mix.Shell.Process do def info(message) do send(message_target(), {:mix_shell, :info, [message]})
:ok end defp message_target() do case Process.get(:"$callers") do [parent | _] -* parent _ -* self() end end end -
“Program each dependency with a tested, production-grade trackXxx() method that
tracks the otherwise-invisible writes. Have it do so regardless of whether the object is Nulled or not.” “Testing without Mocks” - James Shore -
“Telemetry is a lightweight library for dynamic dispatching of events,
with a focus on metrics and instrumentation. Any Erlang or Elixir library can use telemetry to emit events.” — Telemetry documentation -
defmodule MyUmbrella.Infrastructure.JsonHttp.ClientTest do use MyUmbrella.TestCase, async: true alias MyUmbrella.Infrastructure.JsonHttp alias
Nullables.OutputTracking describe "nullability" do test "output tracking", %{test: test} do http_client = JsonHttp.Client.create_null() ref = OutputTracking.track_output(test, [:json_http, :requested]) _result = JsonHttp.Client.get(http_client, "http:/#NOT_CONNECTED/get") assert_received {[:json_http, :requested], ^ref, %JsonHttp.Request{url: "http:/#NOT_CONNECTED/get"}} end end end -
defmodule MyUmbrella.Infrastructure.JsonHttp.ClientTest do use MyUmbrella.TestCase, async: true alias MyUmbrella.Infrastructure.JsonHttp alias
Nullables.OutputTracking describe "nullability" do test "output tracking", %{test: test} do http_client = JsonHttp.Client.create_null() ref = OutputTracking.track_output(test, [:json_http, :requested]) _result = JsonHttp.Client.get(http_client, "http:/#NOT_CONNECTED/get") assert_received {[:json_http, :requested], ^ref, %JsonHttp.Request{url: "http:/#NOT_CONNECTED/get"}} end end end -
defmodule MyUmbrella.Infrastructure.JsonHttp.ClientTest do use MyUmbrella.TestCase, async: true alias MyUmbrella.Infrastructure.JsonHttp alias
Nullables.OutputTracking describe "nullability" do test "output tracking", %{test: test} do http_client = JsonHttp.Client.create_null() ref = OutputTracking.track_output(test, [:json_http, :requested]) _result = JsonHttp.Client.get(http_client, "http:/#NOT_CONNECTED/get") assert_received {[:json_http, :requested], ^ref, %JsonHttp.Request{url: "http:/#NOT_CONNECTED/get"}} end end end -
defmodule MyUmbrella.Infrastructure.JsonHttp.ClientTest do use MyUmbrella.TestCase, async: true alias MyUmbrella.Infrastructure.JsonHttp alias
Nullables.OutputTracking describe "nullability" do test "output tracking", %{test: test} do http_client = JsonHttp.Client.create_null() ref = OutputTracking.track_output(test, [:json_http, :requested]) _result = JsonHttp.Client.get(http_client, "http:/#NOT_CONNECTED/get") assert_received {[:json_http, :requested], ^ref, %JsonHttp.Request{url: "http:/#NOT_CONNECTED/get"}} end end end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do alias MyUmbrella.Infrastructure.JsonHttp alias Nullables.OutputTracking @spec get(t(), String.t())
:( {:ok, JsonHttp.Response.t()} def get(http_client, url) do req = JsonHttp.Request.new(url: url) {:ok, httpoison_resp} = http_client.httpoison.get(req.url, req.headers) :ok = OutputTracking.emit([:http_client, :requested], req) resp = JsonHttp.Response.new( ) {:ok, resp} end end -
defmodule Nullables.OutputTracking do def track_output(handler_id, event_name) do ref = make_ref()
config = %{caller_pid: self(), ref: ref} :ok = :telemetry.attach( handler_id, event_name, &__MODULE__.handle_event/4, config) ref end def handle_event(event_name, _measurements, data, config) do [module, event_type | _] = event_name send(config.caller_pid, {[module, event_type], config.ref, data}) end end -
defmodule Nullables.OutputTracking do def track_output(handler_id, event_name) do ref = make_ref()
config = %{caller_pid: self(), ref: ref} :ok = :telemetry.attach( handler_id, event_name, &__MODULE__.handle_event/4, config) ref end def handle_event(event_name, _measurements, data, config) do [module, event_type | _] = event_name send(config.caller_pid, {[module, event_type], config.ref, data}) end end -
defmodule Nullables.OutputTracking do def track_output(handler_id, event_name) do ref = make_ref()
config = %{caller_pid: self(), ref: ref} :ok = :telemetry.attach( handler_id, event_name, &__MODULE__.handle_event/4, config) ref end def handle_event(event_name, _measurements, data, config) do [module, event_type | _] = event_name send(config.caller_pid, {[module, event_type], config.ref, data}) end end -
defmodule Nullables.OutputTracking do @spec emit(:telemetry.event_name(), term()) :( :ok def emit(event_name,
data) do :telemetry.execute(event_name, %{}, data) end end -
defmodule MyUmbrella.Infrastructure.JsonHttp.ClientTest do use MyUmbrella.TestCase, async: true alias MyUmbrella.Infrastructure.JsonHttp alias
Nullables.OutputTracking describe "nullability" do test "output tracking", %{test: test} do http_client = JsonHttp.Client.create_null() ref = OutputTracking.track_output(test, [:json_http, :requested]) _result = JsonHttp.Client.get(http_client, "http:/#NOT_CONNECTED/get") assert_received {[:json_http, :requested], ^ref, %JsonHttp.Request{url: "http:/#NOT_CONNECTED/get"}} end end end State-based testing -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do @type t() :( %JsonHttp.Client{httpoison: HTTPoison | StubbedHTTPoison}
@enforce_keys [:httpoison] defstruct [:httpoison] @spec create() :( t() def create do %JsonHttp.Client{httpoison: HTTPoison} end @spec create_null() :( t() def create_null() do %JsonHttp.Client{httpoison: StubbedHTTPoison} end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do @type t() :( %JsonHttp.Client{httpoison: HTTPoison | StubbedHTTPoison}
@enforce_keys [:httpoison] defstruct [:httpoison] @spec create() :( t() def create do %JsonHttp.Client{httpoison: HTTPoison} end @spec create_null() :( t() def create_null() do %JsonHttp.Client{httpoison: StubbedHTTPoison} end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do @type t() :( %JsonHttp.Client{httpoison: HTTPoison | StubbedHTTPoison}
@enforce_keys [:httpoison] defstruct [:httpoison] @spec create() :( t() def create do %JsonHttp.Client{httpoison: HTTPoison} end @spec create_null() :( t() def create_null() do %JsonHttp.Client{httpoison: StubbedHTTPoison} end -
defmodule MyUmbrella.Infrastructure.JsonHttp.Client do @type t() :( %JsonHttp.Client{httpoison: HTTPoison | StubbedHTTPoison}
@enforce_keys [:httpoison] defstruct [:httpoison] @spec create() :( t() def create do %JsonHttp.Client{httpoison: HTTPoison} end @spec create_null() :( t() def create_null() do %JsonHttp.Client{httpoison: StubbedHTTPoison} end -
iex> MyUmbrella.create %MyUmbrella{ weather_api_client: %MyUmbrella.Infrastructure.WeatherApi.Client{ http_client: %MyUmbrella.Infrastructure.JsonHttp.Client{ httpoison: HTTPoison },
app_id: “[REDACTED]” } } Production Dependency -
test "given it IS raining before end-of-day; then an umbrella
IS required” do london = CoordinatesFixture.example(:london) current_date_time = DateTimeFixture.Utc.example(:London) responses = %{weather_api_client: ResponseFixture.Success.example(:London)} weather_result = responses |6 MyUmbrella.create_null() |6 MyUmbrella.for_today(london, current_date_time) assert umbrella_required?(weather_result) end -
test "given it IS raining before end-of-day; then an umbrella
IS required” do london = CoordinatesFixture.example(:london) current_date_time = DateTimeFixture.Utc.example(:London) responses = %{weather_api_client: ResponseFixture.Success.example(:London)} weather_result = responses |6 MyUmbrella.create_null() |6 MyUmbrella.for_today(london, current_date_time) assert umbrella_required?(weather_result) end -
test "given it IS raining before end-of-day; then an umbrella
IS required” do london = CoordinatesFixture.example(:london) current_date_time = DateTimeFixture.Utc.example(:London) responses = %{weather_api_client: ResponseFixture.Success.example(:London)} weather_result = responses |6 MyUmbrella.create_null() |6 MyUmbrella.for_today(london, current_date_time) assert umbrella_required?(weather_result) end -
test "given it IS raining before end-of-day; then an umbrella
IS required” do london = CoordinatesFixture.example(:london) current_date_time = DateTimeFixture.Utc.example(:London) responses = %{weather_api_client: ResponseFixture.Success.example(:London)} weather_result = responses |6 MyUmbrella.create_null() |6 MyUmbrella.for_today(london, current_date_time) assert umbrella_required?(weather_result) end -
test "given it IS raining before end-of-day; then an umbrella
IS required” do london = CoordinatesFixture.example(:london) current_date_time = DateTimeFixture.Utc.example(:London) responses = %{weather_api_client: ResponseFixture.Success.example(:London)} weather_result = responses |6 MyUmbrella.create_null() |6 MyUmbrella.for_today(london, current_date_time) assert umbrella_required?(weather_result) end -
test "given it IS raining before end-of-day; then an umbrella
IS required” do london = CoordinatesFixture.example(:london) current_date_time = DateTimeFixture.Utc.example(:London) responses = %{weather_api_client: ResponseFixture.Success.example(:London)} weather_result = responses |6 MyUmbrella.create_null() |6 MyUmbrella.for_today(london, current_date_time) assert umbrella_required?(weather_result) end -
defmodule MyUmbrella do alias MyUmbrella @type t() :( %MyUmbrella{weather_api_client: WeatherApi.Client.t()}
defstruct [:weather_api_client] @spec create() :( t() def create do %MyUmbrella{weather_api_client: WeatherApi.Client.create()} end @spec create_null(%{weather_api_client: WeatherApi.Response.t()}) :( t() def create_null(responses) do null_weather_client = WeatherApi.Client.create_null(%{http_client: responses.weather_api_client}) %MyUmbrella{weather_api_client: null_weather_client} end end -
defmodule MyUmbrella.Infrastructure.WeatherApi.Client do @spec create() :( t() def create do
%WeatherApi.Client{http_client: JsonHttp.Client.create(), app_id: get_app_id()} end @spec create_null(%{http_client: WeatherApi.Response.t()}) :( t() def create_null(responses) do http_client_responses = [{@default_url <0 @request_path, responses.http_client}] %WeatherApi.Client{ http_client: JsonHttp.Client.create_null(http_client_responses), app_id: get_app_id() } end end -
Messing with preconceptions of testing • “Test code” with your
production code -
Messing with preconceptions of testing • “Test code” with your
production code • BUT Use the off-switch as a “dry-run” -
Messing with preconceptions of testing • “Test code” with your
production code • BUT Use the off-switch as a “dry-run” • Up-front investment -
Messing with preconceptions of testing • “Test code” with your
production code • BUT Use the off-switch as a “dry-run” • Up-front investment • BUT a better understanding of the client library and communication protocol -
Test Mocks • Solitary and interaction- based test cases •
Unreliable test results -
Test Mocks • Solitary and interaction- based test cases •
Unreliable test results • Fall out of sync with the dependency -
Nullables • Low-level infrastructure wrappers • Embedded Stubs • Configurable
Responses • Output Tracking -
Testing Elixir • Example: `Mix.Shell` • Embedded Stubs: `Mix.Shell.Process` •
Output Tracking: `send/2` -
Take-aways • Consider the trade-offs of test mocks • Consider
lightweight techniques -
Take-aways • Consider the trade-offs of test mocks • Consider
lightweight techniques • Consider your infrastructure wrappers -
Explore Nullables • www.jamesshore.com • Pattern language • Video series
• Complete free training course