Lessons Learned From an Elixir OTP Project

Slide 1

Slide 1 text

Lessons Learned From an Elixir/OTP Project @amandasposito

Slide 2

Slide 2 text

•amandasposito.com •speakerdeck.com/amandasposito •linkedin.com/in/amandasposito

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

How was it like to start in a new language?

Slide 8

Slide 8 text

Normally, Elixir is not the ﬁrst language we learn

Slide 9

Slide 9 text

Most people come from Object-Oriented languages

Slide 10

Slide 10 text

Where do I start?

Slide 11

Slide 11 text

How do I organize things?

Slide 12

Slide 12 text

Is this code really complex or is it me that doesn't know FP?

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

A lot of new stuff

Slide 15

Slide 15 text

A new project, a new language, a new paradigm

Slide 16

Slide 16 text

Even though anxiety may hit harder

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

You started your new project

Slide 19

Slide 19 text

How to use the language tooling in our favor?

Slide 20

Slide 20 text

Elixir has a bunch of cool stuff to help us in our journey

Slide 21

Slide 21 text

Chances are you will deal with databases in your project

Slide 22

Slide 22 text

One of the things that may be different from other languages

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

In Object-Oriented languages we have ORMs

Slide 25

Slide 25 text

Implicit can be dangerous sometimes

Slide 26

Slide 26 text

Ecto is explicit

Slide 27

Slide 27 text

This is a big change in the way we think

Slide 28

Slide 28 text

You have to take most of the decisions dealing with databases

Slide 29

Slide 29 text

Instead of accessing the data using Objects

Slide 30

Slide 30 text

We now use Repo

Slide 31

Slide 31 text

We write things more like SQL

Slide 32

Slide 32 text

For example: Preloads

Slide 33

Slide 33 text

Will I need to access this association?

Slide 34

Slide 34 text

Repo.all from c in Course, preload: [:users]

Slide 35

Slide 35 text

SELECT * FROM Courses; SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ?

Slide 36

Slide 36 text

SELECT * FROM Courses; SELECT * FROM Users WHERE course_id = ?

Slide 37

Slide 37 text

Spend time learning all the cool features Ecto has

Slide 38

Slide 38 text

•Repo •Changesets •Schemas •Associations •Ecto.Multi •etc.

Slide 39

Slide 39 text

Sometimes depending on how many access to the database you have

Slide 40

Slide 40 text

Or how much data you are dealing with

Slide 41

Slide 41 text

The database may be a bottleneck

Slide 42

Slide 42 text

In some cases what we can do is to cache some data to help us

Slide 43

Slide 43 text

ETS Tables

Slide 44

Slide 44 text

–Erlang Documentation “These provide the ability to store very large quantities of data in an Erlang runtime system, and to have constant access time to the data.”

Slide 45

Slide 45 text

A common use case for ETS tables is to store cache

Slide 46

Slide 46 text

However, there are some things to take into consideration

Slide 47

Slide 47 text

When we talk about cache in memory Redis usually comes to mind

Slide 48

Slide 48 text

This ETS kinda looks like Redis, and Redis I know how to use, so why not use this one for free?

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

ETS Tables are like a Hash straight into memory

Slide 51

Slide 51 text

It does not have many optimization options

Slide 52

Slide 52 text

They consume the memory available for the application

Slide 53

Slide 53 text

It doesn't offer support to distribution

Slide 54

Slide 54 text

With these constraints, is ETS what you need?

Slide 55

Slide 55 text

Take into account the number of items you will handle

Slide 56

Slide 56 text

There was this one time where we ended up creating an ETS with 76GB of memory consumption

Slide 57

Slide 57 text

No content

Slide 58

Slide 58 text

https://moz.com/devblog/moz-analytics-db-free

Slide 59

Slide 59 text

https://moz.com/devblog/moz-analytics-db-free

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

Doctest

Slide 62

Slide 62 text

defmodule MyModuleTest do use ExUnit.Case, async: true doctest MyModule end

Slide 63

Slide 63 text

@doc """ Sums two numbers iex> MyModule.sum(2, 2) 4 """ def sum(a, b) do a + b end

Slide 64

Slide 64 text

What about tests?

Slide 65

Slide 65 text

Doctest != Test

Slide 66

Slide 66 text

So your application is talking with the outside world, what now?

Slide 67

Slide 67 text

No content

Slide 68

Slide 68 text

How do we mock or stub a request?

Slide 69

Slide 69 text

Bypass

Slide 70

Slide 70 text

Bypass

Slide 71

Slide 71 text

setup do bypass = Bypass.open {:ok, bypass: bypass} end

Slide 72

Slide 72 text

test "fetch/1 returns and formats tweets", %{bypass: bypass} do response = Jason.encode!([%{"text" => "Elixir Brasil 2019"}]) Bypass.expect(bypass, fn conn -> assert "/1.1/search/tweets.json" == conn.request_path assert "GET" == conn.method Plug.Conn.resp(conn, 200, response) end) tweets = TwitterClient.fetch("http://localhost:#{bypass.port}") assert tweets == [%{"text" => "Elixir Brasil 2019"}] end

Slide 73

Slide 73 text

Slide 74

Slide 74 text

def fetch(url \\ "https://api.twitter.com") do {:ok, response} = HTTPoison.get("#{url}/1.1/search/tweets.json") Jason.decode!(response.body) end

Slide 75

Slide 75 text

When to use Bypass?

Slide 76

Slide 76 text

Code that needs to make an HTTP request

Slide 77

Slide 77 text

No content

Slide 78

Slide 78 text

Mox

Slide 79

Slide 79 text

Timeline TwitterClient

Slide 80

Slide 80 text

test "messages/0 lists all messages from the timeline" do TwitterMock |> expect(:fetch, fn -> [%{"text" => "Olá mundo"}] end) assert Timeline.messages() == {:ok, 1} end https://github.com/amandasposito/mox_example

Slide 81

Slide 81 text

http://blog.plataformatec.com.br/2015/10/mocks-and-explicit-contracts

Slide 82

Slide 82 text

How to test processes?

Slide 83

Slide 83 text

GenServer

Slide 84

Slide 84 text

It is a mindset change about the way we deal with the state

Slide 85

Slide 85 text

We don't eliminate the state, we control it

Slide 86

Slide 86 text

Functional paradigm helps you turn the state more explicit

Slide 87

Slide 87 text

defmodule NewBank.Counter do use GenServer # Client def start_link(opts) do GenServer.start_link(__MODULE__, opts, name: Keyword.get(opts, :name)) end def increment() do GenServer.cast(__MODULE__, :increment) end def count() do GenServer.call(__MODULE__, :fetch) end end

Slide 88

Slide 88 text

defmodule NewBank.CounterServer do ... # Server @impl true def handle_cast(:increment, _from) do default = 1 value = :ets.update_counter(:counter_table, "counter_key", 1, default) {:reply, :ok, value} end @impl true def handle_call(:fetch, _from, counter) do {:reply, :ets.lookup(:counter_table, "counter_key"), counter} end end

Slide 89

Slide 89 text

How do I test it now?

Slide 90

Slide 90 text

To test a GenServer Callback Is Not a Good Practice

Slide 91

Slide 91 text

✉

Slide 92

Slide 92 text

GenServer.cast

Slide 93

Slide 93 text

We don't wait for an answer

Slide 94

Slide 94 text

GenServer.call

Slide 95

Slide 95 text

We expect an answer

Slide 96

Slide 96 text

test "increments and returns the counter current number" do start_supervised!(CounterServer) CounterServer.increment() assert CounterServer.count() == 1 end

Slide 97

Slide 97 text

defmodule NewBank.CounterServer do ... # Server @impl true def handle_cast(:increment, _from) do value = Counter.really_complex |> Counter.new_feature |> Counter.we_need_to_implement |> Counter.increment() {:reply, :ok, value} end @impl true def handle_call(:fetch, _from, counter) do value = Counter.different_way |> Counter.fetch {:reply, value, counter} end end

Slide 98

Slide 98 text

What are the most common problems?

Slide 99

Slide 99 text

•Everything I've learned in OOP, I'm going to throw away in functional programming? •How do I organize my code? •Do all the problems I had in OOP disappear in functional programming? •What about this Context? How do I use it?

Slide 100

Slide 100 text

Chances are the code will be less complex

Slide 101

Slide 101 text

But the problems still exist

Slide 102

Slide 102 text

Many of the problems we see in OOP can be seen in FP

Slide 103

Slide 103 text

•Long functions •Functions that are hard to test •Simple changes need to be done in many places •Feature Envy •Context with too many lines •Tight coupling

Slide 104

Slide 104 text

https://youtu.be/eldYot7uxUc

Slide 105

Slide 105 text

We still want to

Slide 106

Slide 106 text

Minimize the number of modules affected by a change

Slide 107

Slide 107 text

To have a reliable interface contract

Slide 108

Slide 108 text

High-level policy to be independent of low-level details

Slide 109

Slide 109 text

No content

Slide 110

Slide 110 text

How can we organize our code?

Slide 111

Slide 111 text

Contexts

Slide 112

Slide 112 text

How does it work?

Slide 113

Slide 113 text

Where do I put my code now?

Slide 114

Slide 114 text

What should be its responsibilities?

Slide 115

Slide 115 text

Contexts are boundaries between your application modules

Slide 116

Slide 116 text

No content

Slide 117

Slide 117 text

defmodule NewBank.CreditCard do @moduledoc """ The CreditCard context. """ def create do ... end end

Slide 118

Slide 118 text

defmodule NewBank.CreditCard do @moduledoc """ The CreditCard context. """ def create do ... end def fetch do ... end def update do ... end def delete do ... end def lists do ... end def fetch_available_limit do .. end end

Slide 119

Slide 119 text

As time goes by

Slide 120

Slide 120 text

def fetch_transactions do .. end def count_transactions do ... end def total_transactions_by_user do ... end defp confirmations do ... end def list_top_credit_cards do ... end def fetch_pending_transactions do ... end def count_pending_transactions do ... end defp join_association(query, [{association, nested_preload}]) do ... end defp page_transactions do ... end defp where_transaction_has_multiple_disputes do ... end

Slide 121

Slide 121 text

The need to interact with other schemas increase

Slide 122

Slide 122 text

Contexts can get bigger than they should

Slide 123

Slide 123 text

Move orthogonal functionality out of the Context

Slide 124

Slide 124 text

def create_reward do ... end def update_reward do .. end def delete_reward do ... end

Slide 125

Slide 125 text

Move queries closer to their schema

Slide 126

Slide 126 text

def fetch_transactions do .. end def fetch_pending_transactions do ... end defp join_association(query, [{association, nested_preload}]) do ... end defp page_transactions do ... end defp where_transaction_has_multiple_disputes do ... end