Dispatch, a quick overview of neat Elixir features

Dispatch is a (formely internal, now open-source!) tool we built to make sure pull requests gets reviewed and looked at by the right people within a GitHub organization.

In this presentation, we’ll take a look at how we use a few neat Elixir features to make Dispatch work.

Transcript

1. Montreal Elixir Meetup — March 13, 2019 Dispatch. A quick overview of neat Elixir features.
2. MIREGO 1 Quick overview of Dispatch 2 Neat Elixir features in Dispatch 3 Questions 2 Summary. Where I’m going with this
3. MIREGO 3 Rémi Prévost Director, Web Development @ Mirego
4. MIREGO Quick overview of Dispatch Neat Elixir features Questions Dispatch. Introduction “Dispatch is a (formely internal, now open-source!) tool we built to make sure our pull requests gets reviewed and looked at by the right people within a GitHub organization.” 4
5. MIREGO Quick overview of Dispatch Neat Elixir features Questions Dispatch. Introduction 5 https://github.com/mirego/dispatch
6. MIREGO Neat Elixir features. How is this going to work 6 Quick overview of Dispatch Neat Elixir features Questions Mox How we use Mox in Dispatch Agent How we use Agent in Dispatch 1 2
7. MIREGO Neat Elixir features. Mox 7 Quick overview of Dispatch Neat Elixir features Questions Mox is a third-party library (not really an Elixir feature) that allows developers to create behaviour-based mocks to use in ExUnit tests. • It makes sure we can’t mock anything we want • It makes sure we can’t mock something only when we want it • It forces us to clearly separate what is the behaviour we need from a module and how this behaviour should be implemented Let’s see how Mox works with a simple example.
8. MIREGO Neat Elixir features. Mox 8 Quick overview of Dispatch Neat Elixir features Questions # lib/my_app/calculator.ex defmodule MyApp.Calculator do def add(first_number, second_number) do first_number + second_number end end
9. MIREGO Neat Elixir features. Mox 9 Quick overview of Dispatch Neat Elixir features Questions # lib/my_app/calculator_behaviour.ex defmodule MyApp.CalculatorBehaviour do @callback add(integer(), integer()) :: integer() end
10. MIREGO Neat Elixir features. Mox 1 0 Quick overview of Dispatch Neat Elixir features Questions # lib/my_app/calculator.ex defmodule MyApp.Calculator do @behaviour MyApp.CalculatorBehaviour def add(first_number, second_number) do first_number + second_number end end
11. MIREGO Neat Elixir features. Mox 1 1 Quick overview of Dispatch Neat Elixir features Questions # lib/my_app/my_app.ex defmodule MyApp do def add_numbers(first_number, second_number) do result = MyApp.Calculator.add(first_number, second_number) "The result is #{result}" end end
12. MIREGO Neat Elixir features. Mox 1 2 Quick overview of Dispatch Neat Elixir features Questions # lib/my_app/my_app.ex defmodule MyApp do def add_numbers(first_number, second_number) do result = calculator().add(first_number, second_number) "The result is #{result}" end def calculator do Application.get_env(:my_app, MyApp)[:calculator] end end
13. MIREGO # test/support/mocks.exs Mox.defmock(MyApp.CalculatorMock, for: MyApp.Calculator) # config/config.exs config :my_app, MyApp, calculator: MyApp.Calculator # config/config.test.exs config :my_app, MyApp, calculator: MyApp.CalculatorMock Neat Elixir features. Mox 1 3 Quick overview of Dispatch Neat Elixir features Questions
14. MIREGO Neat Elixir features. Mox 1 4 Quick overview of Dispatch Neat Elixir features Questions # test/my_app/calculator_test.exs defmodule MyApp.CalculatorTest do use ExUnit.Case async: true setup :verify_on_exit! test "add/2 adds " do expect(MyApp.CalculatorMock, :add, fn(_, _) -> 999 end) assert MyApp.add_numbers(2, 3) == "The result is 999" end end
15. MIREGO Neat Elixir features. Mock in Dispatch 1 5 Quick overview of Dispatch Neat Elixir features Questions Dispatch interacts with several APIs in order to fetch and post data. Of course, we don’t want to depend on these integrations when running our test suite. We use Mox to decouple our external service implementations from the code that actually uses them.
16. MIREGO Neat Elixir features. 1 6 Quick overview of Dispatch Neat Elixir features Questions # lib/dispatch/repositories/client_behaviour.ex defmodule Dispatch.Repositories.ClientBehaviour do @callback fetch_contributors(binary()) :: list(Map.t()) @callback request_reviewers(binary(), integer(), list(Map.t()) :: :ok end Mock in Dispatch
17. MIREGO Neat Elixir features. 1 7 Quick overview of Dispatch Neat Elixir features Questions # lib/dispatch/repositories/github_client.ex defmodule Dispatch.Repositories.GitHubClient do @behaviour Dispatch.Repositories.ClientBehaviour def fetch_contributors(repo) do url = "https://api.github.com/repos/#{repo}/stats/contributors" {:ok, %{body: %{"users" => users}} = HTTPoison.get(url) Enum.map(users, & &1.username) end def request_reviewers(repo, number, reviewers) do url = "https://api.github.com/repos/#{repo}/prs/#{number}/reviews" {:ok, _} = HTTPoison.post(url, reviewers) :ok end end Mock in Dispatch
18. MIREGO Neat Elixir features. 1 8 Quick overview of Dispatch Neat Elixir features Questions # config/config.exs config :dispatch, Dispatch, repositories_client: Dispatch.Repositories.GitHubClient # config/config.test.exs config :dispatch, Dispatch, repositories_client: Dispatch.Repositories.MockClient Mock in Dispatch
19. MIREGO Neat Elixir features. 1 9 Quick overview of Dispatch Neat Elixir features Questions # lib/dispatch/dispatch.ex defmodule Dispatch do def request_reviewers(repo, number) do client = repositories_client() contributors = client.fetch_contributors(repo) response = client.request_reviewers(repo, number, contributors) [response: response, reviewers: contributors] end defp repositories_client do Application.get_env(:dispatch, Dispatch)[:repositories_client] end end Mock in Dispatch
20. MIREGO # test/support/mocks.exs Mox.defmock(Dispatch.Repositories.MockClient, for: Dispatch.Repositories.ClientBehaviour ) # test/dispatch/dispatch_test.exs defmodule DispatchTest do setup :verify_on_exit! test "request_reviewers/2 should fetch contributors and request them" do # Tests! end end Neat Elixir features. 2 0 Quick overview of Dispatch Neat Elixir features Questions Mock in Dispatch
21. MIREGO Neat Elixir features. 2 1 Quick overview of Dispatch Neat Elixir features Questions # test/dispatch/dispatch_test.exs defmodule DispatchTest do test "request_reviewers/2 should fetch contributors and request them" do expect(MockClient, :fetch_contributors, fn "mirego/foo" -> ["bar", "baz", "omg"] end) expect(MockClient, :request_reviewers, fn "mirego/foo", 45, [] -> :ok end) [response: response, reviewers: reviewers] = Dispatch.request_reviewers("mirego/foo", 45, []) assert response == :ok assert reviewers == ["bar", "baz", "omg"] end end Mock in Dispatch
22. MIREGO Neat Elixir features. Agent 2 2 “Agents are a simple abstraction around state.” — Elixir documentation Quick overview of Dispatch Neat Elixir features Questions
23. MIREGO Neat Elixir features. Agent 2 3 “But I thought Elixir was a stateless language!?” — Anyone starting with Elixir Quick overview of Dispatch Neat Elixir features Questions
24. MIREGO Neat Elixir features. Agent 2 4 Quick overview of Dispatch Neat Elixir features Questions Processes (from Erlang) actually have a state. GenServer is an abstraction on top of processes, that provides a simple and standard way to create, supervise and interact with processes. Agent is an abstraction on top of GenServer, that provides a simple way to create processes exposing a simple API to store and retrieve state data. Let’s see how Agent works with a simple example.
25. MIREGO defmodule Counter do use Agent def start_link(initial_value) do Agent.start_link(fn -> initial_value end, name: __MODULE__) end def value do Agent.get(__MODULE__, fn state -> state end) end def increment do Agent.update(__MODULE__, fn state -> state + 1 end) end end Neat Elixir features. Agent 2 5 Quick overview of Dispatch Neat Elixir features Questions
26. MIREGO # Start the process Counter.start_link(0) # Retrieve the value from the state Counter.value => 0 # Increment the value twice Counter.increment => :ok Counter.increment => :ok # Retrieve the value from the state Counter.value => 2 Neat Elixir features. Agent 2 6 Quick overview of Dispatch Neat Elixir features Questions
27. MIREGO Neat Elixir features. Agent in Dispatch 2 7 Quick overview of Dispatch Neat Elixir features Questions Dispatch uses an external JSON file to store its configuration: • a list of experts by stack (eg. elixir, ruby, react, etc.) • a list of learners by stack • a list of blacklisted users We store this state in an Agent and make sure it’s up-to-date when we need it.
28. MIREGO Neat Elixir features. # lib/dispatch/settings/json_static_file_client.ex defmodule Dispatch.Settings.JSONStaticFileClient do use Agent def start_link do state = build_state() Agent.start_link(fn -> state end, name: __MODULE__) end end 2 8 Quick overview of Dispatch Neat Elixir features Questions Agent in Dispatch
29. MIREGO # lib/dispatch/settings/json_static_file_client.ex defmodule Dispatch.Settings.JSONStaticFileClient do defp build_state do url = Application.get_env(:dispatch, __MODULE__)[:configuration_url] {:ok, configuration} = fetch_configuration(url) %{ experts: Map.get(configuration, "experts"), learners: Map.get(configuration, "learners"), blacklist: Map.get(configuration, "blacklist"), } end defp fetch_configuration(url) do # Fetch JSON file from URL and parse it end end Neat Elixir features. 2 9 Quick overview of Dispatch Neat Elixir features Questions Agent in Dispatch
30. MIREGO # lib/dispatch/settings/json_static_file_client.ex defmodule Dispatch.Settings.JSONStaticFileClient do @behaviour Dispatch.Settings.ClientBehaviour def expert_users(stack) do Agent.get(__MODULE__, & &1.experts) |> Map.get(stack) end def learner_users(stack) do Agent.get(__MODULE__, & &1.learners) |> Map.get(stack) end def blacklisted_users do Agent.get(__MODULE__, & &1.blacklist) end end Neat Elixir features. 3 0 Quick overview of Dispatch Neat Elixir features Questions Agent in Dispatch
31. MIREGO # lib/dispatch/settings/json_static_file_client.ex defmodule Dispatch.Settings.JSONStaticFileClient do def refresh do state = build_state() Agent.update(__MODULE__, fn _ -> state end) end end Neat Elixir features. 3 1 Quick overview of Dispatch Neat Elixir features Questions Agent in Dispatch
32. MIREGO # lib/dispatch/dispatch.ex defmodule Dispatch do def fetch_reviewers(repo, number) do client = settings_client(). # Refresh settings client().refresh() # Draw the rest of the owl # client().expert_users("elixir") => […] end defp settings_client do Application.get_env(:dispatch, __MODULE__)[:settings_client] end end Neat Elixir features. 3 2 Quick overview of Dispatch Neat Elixir features Questions Agent in Dispatch
33. MIREGO Questions? 3 3 Quick overview of Dispatch Neat Elixir features Questions https://github.com/mirego/dispatch
34. Thank you.