Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Lessons Learned From an Elixir OTP Project

7c1e5b1b100ab8cfacbe14173437c998?s=47 Amanda
August 30, 2019

Lessons Learned From an Elixir OTP Project

You started an Elixir project, coming from another language, and you are curious about how everything fits, what are the most common problems encountered, and how is the best way to solve them.

Questions such as: How to organize your tests in a concurrent application? What are the pitfalls when writing process tests? How can we organize our code? How can contexts help us? Which smells can help us realize it's time to refactor our code? And how do we use the language tooling in our favor?

7c1e5b1b100ab8cfacbe14173437c998?s=128

Amanda

August 30, 2019
Tweet

Transcript

  1. Lessons Learned From an Elixir/OTP Project @amandasposito

  2. •amandasposito.com •speakerdeck.com/amandasposito •linkedin.com/in/amandasposito

  3. None
  4. None
  5. None
  6. None
  7. How was it like to start in a new language?

  8. Normally, Elixir is not the first language we learn

  9. Most people come from Object-Oriented languages

  10. Where do I start?

  11. How do I organize things?

  12. Is this code really complex or is it me that

    doesn't know FP?
  13. None
  14. A lot of new stuff

  15. A new project, a new language, a new paradigm

  16. Even though anxiety may hit harder

  17. None
  18. You started your new project

  19. How to use the language tooling in our favor?

  20. Elixir has a bunch of cool stuff to help us

    in our journey
  21. Chances are you will deal with databases in your project

  22. One of the things that may be different from other

    languages
  23. None
  24. In Object-Oriented languages we have ORMs

  25. Implicit can be dangerous sometimes

  26. Ecto is explicit

  27. This is a big change in the way we think

  28. You have to take most of the decisions dealing with

    databases
  29. Instead of accessing the data using Objects

  30. We now use Repo

  31. We write things more like SQL

  32. For example: Preloads

  33. Will I need to access this association?

  34. Repo.all from c in Course, preload: [:users]

  35. SELECT * FROM Courses; SELECT * FROM Users WHERE course_id

    = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ?
  36. SELECT * FROM Courses; SELECT * FROM Users WHERE course_id

    = ?
  37. Spend time learning all the cool features Ecto has

  38. •Repo •Changesets •Schemas •Associations •Ecto.Multi •etc.

  39. Sometimes depending on how many access to the database you

    have
  40. Or how much data you are dealing with

  41. The database may be a bottleneck

  42. In some cases what we can do is to cache

    some data to help us
  43. ETS Tables

  44. –Erlang Documentation “These provide the ability to store very large

    quantities of data in an Erlang runtime system, and to have constant access time to the data.”
  45. A common use case for ETS tables is to store

    cache
  46. However, there are some things to take into consideration

  47. When we talk about cache in memory Redis usually comes

    to mind
  48. This ETS kinda looks like Redis, and Redis I know

    how to use, so why not use this one for free?
  49. None
  50. ETS Tables are like a Hash straight into memory

  51. It does not have many optimization options

  52. They consume the memory available for the application

  53. It doesn't offer support to distribution

  54. With these constraints, is ETS what you need?

  55. Take into account the number of items you will handle

  56. There was this one time where we ended up creating

    an ETS with 76GB of memory consumption
  57. None
  58. https://moz.com/devblog/moz-analytics-db-free

  59. https://moz.com/devblog/moz-analytics-db-free

  60. None
  61. Doctest

  62. defmodule MyModuleTest do use ExUnit.Case, async: true doctest MyModule end

  63. @doc """ Sums two numbers iex> MyModule.sum(2, 2) 4 """

    def sum(a, b) do a + b end
  64. What about tests?

  65. Doctest != Test

  66. So your application is talking with the outside world, what

    now?
  67. None
  68. How do we mock or stub a request?

  69. Bypass

  70. Bypass

  71. setup do bypass = Bypass.open {:ok, bypass: bypass} end

  72. test "fetch/1 returns and formats tweets", %{bypass: bypass} do response

    = Jason.encode!([%{"text" => "Elixir Brasil 2019"}]) Bypass.expect(bypass, fn conn -> assert "/1.1/search/tweets.json" == conn.request_path assert "GET" == conn.method Plug.Conn.resp(conn, 200, response) end) tweets = TwitterClient.fetch("http://localhost:#{bypass.port}") assert tweets == [%{"text" => "Elixir Brasil 2019"}] end
  73. test "fetch/1 returns and formats tweets", %{bypass: bypass} do response

    = Jason.encode!([%{"text" => "Elixir Brasil 2019"}]) Bypass.expect(bypass, fn conn -> assert "/1.1/search/tweets.json" == conn.request_path assert "GET" == conn.method Plug.Conn.resp(conn, 200, response) end) tweets = TwitterClient.fetch("http://localhost:#{bypass.port}") assert tweets == [%{"text" => "Elixir Conf 2019"}] end
  74. def fetch(url \\ "https://api.twitter.com") do {:ok, response} = HTTPoison.get("#{url}/1.1/search/tweets.json") Jason.decode!(response.body)

    end
  75. When to use Bypass?

  76. Code that needs to make an HTTP request

  77. None
  78. Mox

  79. Timeline TwitterClient

  80. test "messages/0 lists all messages from the timeline" do TwitterMock

    |> expect(:fetch, fn -> [%{"text" => "Olá mundo"}] end) assert Timeline.messages() == {:ok, 1} end https://github.com/amandasposito/mox_example
  81. http://blog.plataformatec.com.br/2015/10/mocks-and-explicit-contracts

  82. How to test processes?

  83. GenServer

  84. It is a mindset change about the way we deal

    with the state
  85. We don't eliminate the state, we control it

  86. Functional paradigm helps you turn the state more explicit

  87. defmodule NewBank.Counter do use GenServer # Client def start_link(opts) do

    GenServer.start_link(__MODULE__, opts, name: Keyword.get(opts, :name)) end def increment() do GenServer.cast(__MODULE__, :increment) end def count() do GenServer.call(__MODULE__, :fetch) end end
  88. defmodule NewBank.CounterServer do ... # Server @impl true def handle_cast(:increment,

    _from) do default = 1 value = :ets.update_counter(:counter_table, "counter_key", 1, default) {:reply, :ok, value} end @impl true def handle_call(:fetch, _from, counter) do {:reply, :ets.lookup(:counter_table, "counter_key"), counter} end end
  89. How do I test it now?

  90. To test a GenServer Callback Is Not a Good Practice

  91. GenServer.cast

  92. We don't wait for an answer

  93. GenServer.call

  94. We expect an answer

  95. test "increments and returns the counter current number" do start_supervised!(CounterServer)

    CounterServer.increment() assert CounterServer.count() == 1 end
  96. defmodule NewBank.CounterServer do ... # Server @impl true def handle_cast(:increment,

    _from) do value = Counter.really_complex |> Counter.new_feature |> Counter.we_need_to_implement |> Counter.increment() {:reply, :ok, value} end @impl true def handle_call(:fetch, _from, counter) do value = Counter.different_way |> Counter.fetch {:reply, value, counter} end end
  97. What are the most common problems?

  98. •Everything I've learned in OOP, I'm going to throw away

    in functional programming? •How do I organize my code? •Do all the problems I had in OOP disappear in functional programming? •What about this Context? How do I use it?
  99. Chances are the code will be less complex

  100. But the problems still exist

  101. Many of the problems we see in OOP can be

    seen in FP
  102. •Long functions •Functions that are hard to test •Simple changes

    need to be done in many places •Feature Envy •Context with too many lines •Tight coupling
  103. https://youtu.be/eldYot7uxUc

  104. We still want to

  105. Minimize the number of modules affected by a change

  106. To have a reliable interface contract

  107. High-level policy to be independent of low-level details

  108. None
  109. How can we organize our code?

  110. Contexts

  111. How does it work?

  112. Where do I put my code now?

  113. What should be its responsibilities?

  114. Contexts are boundaries between your application modules

  115. None
  116. defmodule NewBank.CreditCard do @moduledoc """ The CreditCard context. """ def

    create do ... end end
  117. defmodule NewBank.CreditCard do @moduledoc """ The CreditCard context. """ def

    create do ... end def fetch do ... end def update do ... end def delete do ... end def lists do ... end def fetch_available_limit do .. end end
  118. As time goes by

  119. def fetch_transactions do .. end def count_transactions do ... end

    def total_transactions_by_user do ... end defp confirmations do ... end def list_top_credit_cards do ... end def fetch_pending_transactions do ... end def count_pending_transactions do ... end defp join_association(query, [{association, nested_preload}]) do ... end defp page_transactions do ... end defp where_transaction_has_multiple_disputes do ... end
  120. The need to interact with other schemas increase

  121. Contexts can get bigger than they should

  122. Move orthogonal functionality out of the Context

  123. def create_reward do ... end def update_reward do .. end

    def delete_reward do ... end
  124. Move queries closer to their schema

  125. def fetch_transactions do .. end def fetch_pending_transactions do ... end

    defp join_association(query, [{association, nested_preload}]) do ... end defp page_transactions do ... end defp where_transaction_has_multiple_disputes do ... end
  126. None
  127. Thank you! amandasposito.com