CodeEurope 2017 – Monoliths to Services with Elixir and Phoenix

Slide 1

Slide 1 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Monoliths to Services with Elixir & Phoenix PRESENTED BY Lauren Tan sugarpirate_ poteto Cinemagraph by /u/orbojunglist

Slide 2

Slide 2 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix bit.ly/elixir-microservices

Slide 3

Slide 3 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Lauren Tan sugarpirate_ poteto

Slide 4

Slide 4 text

EmberConf 2017 Confessions of an Ember Addon Author

Slide 5

Slide 5 text

EmberConf 2017 Confessions of an Ember Addon Author

Slide 6

Slide 6 text

EmberConf 2017 Confessions of an Ember Addon Author

Slide 7

Slide 7 text

From Front-End to Full Stack with Elixir & Phoenix ElixirConf 2016

Slide 8

Slide 8 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 9

Slide 9 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 10

Slide 10 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 11

Slide 11 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix https://github.com/Netflix/vizceral

Slide 12

Slide 12 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 13

Slide 13 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 14

Slide 14 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 15

Slide 15 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 16

Slide 16 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 17

Slide 17 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix http://worrydream.com/refs/Brooks-NoSilverBullet.pdf

Slide 18

Slide 18 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 19

Slide 19 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Microservices will improve our performance easily FALLACY #1

Slide 20

Slide 20 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Network latency

Slide 21

Slide 21 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix What's the bottleneck?

Slide 22

Slide 22 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix The code is cleaner FALLACY #2

Slide 23

Slide 23 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 24

Slide 24 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix You can write clean code without microservices

Slide 25

Slide 25 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix It's easier to write something that does one thing well FALLACY #3

Slide 26

Slide 26 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Application simplicity

Slide 27

Slide 27 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Infrastructure complexity

Slide 28

Slide 28 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix https://martinfowler.com/articles/microservice-trade-offs.html

Slide 29

Slide 29 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Embrace the monolith

Slide 30

Slide 30 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Microservices are an organizational optimization https://martinfowler.com/bliki/MicroservicePremium.html

Slide 31

Slide 31 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix https://www.slideshare.net/reed2001/culture-1798664/

Slide 32

Slide 32 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Conway's Law

Slide 33

Slide 33 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I. Why Elixir? II. Umbrella monoliths III. Advanced topics

Slide 34

Slide 34 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I. Why Elixir? II. Umbrella monoliths III. Advanced topics

Slide 35

Slide 35 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix The right tool for the job Cinemagraph by /u/smoothinto2nd

Slide 36

Slide 36 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Web API/apps ELIXIR IS GREAT FOR

Slide 37

Slide 37 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Soft real-time ELIXIR IS GREAT FOR

Slide 38

Slide 38 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix ELIXIR IS GREAT FOR High scalability and fault tolerance

Slide 39

Slide 39 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix ELIXIR IS GREAT FOR Concurrency

Slide 40

Slide 40 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix ELIXIR IS GREAT FOR Developer happiness

Slide 41

Slide 41 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Number crunching ELIXIR IS NOT GREAT FOR

Slide 42

Slide 42 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Anything CPU bound ELIXIR IS NOT GREAT FOR

Slide 43

Slide 43 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 44

Slide 44 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 45

Slide 45 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Highly-scalable, fault-tolerant systems ERLANG

Slide 46

Slide 46 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Pre-emptive scheduling THE ERLANG VM: BEAM BEAM SCHEDULER SCHEDULER SCHEDULER SCHEDULER Processes Processes Processes Processes OS Process OS Thread OS Thread OS Thread OS Thread https://hamidreza-s.github.io/erlang/scheduling/real-time/preemptive/migration/2016/02/09/erlang-scheduler-details.html

Slide 47

Slide 47 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix 1. Everything is a process 2. Processes are strongly isolated 3. Process creation and destruction is a lightweight operation 4. Message passing is the only way for processes to interact 5. Processes have unique names 6. If you know the name of a process you can send it a message 7. Processes share no resources 8. Error handling is non-local 9. Processes do what they are supposed to do or fail http://erlang.org/download/armstrong_thesis_2003.pdf

Slide 48

Slide 48 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix JAVA Write once, run anywhere

Slide 49

Slide 49 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix ERLANG https://www.youtube.com/watch?v=u41GEwIq2mE&t=3m59s Write once, run forever

Slide 50

Slide 50 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Garbage Collection

Slide 51

Slide 51 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix http://www.erlang-factory.com/static/upload/media/1394350183453526efsf2014whatsappscaling.pdf

Slide 52

Slide 52 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix https://www.youtube.com/watch?v=5SbWapbXhKo

Slide 53

Slide 53 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Elixir?

Slide 54

Slide 54 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix 1 + 1 #!=> 2 {:ok, "hello"} [1, 2, 3] = [head | tail] head #!=> 1 tail #!=> [2,3]

Slide 55

Slide 55 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule Math do def add(x, y), do: x + y def subtract(x, y) do x - y end end

Slide 56

Slide 56 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule Collection do def sum([x, y, z]), do: x + y + z end

Slide 57

Slide 57 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(2)> Collection.sum([1, 2, 3]) 6 iex(3)> Collection.sum([1, 2, 3, 4]) !** (FunctionClauseError) no function clause matching in Collection.sum/1 iex:2: Collection.sum([1, 2, 3, 4])

Slide 58

Slide 58 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Pipe Operator

Slide 59

Slide 59 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix foo(bar(baz(123), 456), 789)

Slide 60

Slide 60 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix 123 !|> baz() !|> bar(456) !|> foo(123)

Slide 61

Slide 61 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def serialize(user) do user !|> serialize_attributes() !|> serialize_relationships() !|> to_json() end

Slide 62

Slide 62 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def validate_presence(input) do if input !== "" do "Can't be blank" end input end

Slide 63

Slide 63 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Pattern Matching

Slide 64

Slide 64 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def validate_presence(""), do: {:err, "Can't be blank"} def validate_presence(input), do: input

Slide 65

Slide 65 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix thing = %{foo: "bar", baz: 123} %{foo: foo, baz: baz} = thing; IO.inspect(foo) # "bar" IO.inspect(baz) # 123

Slide 66

Slide 66 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix thing = %{foo: "bar", baz: 123} %{foo: foo, baz: baz, qux: qux} = thing; !** (MatchError) no match of right hand side value: %{baz: 123, foo: "bar"}

Slide 67

Slide 67 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix https://fsharpforfunandprofit.com/rop/

Slide 68

Slide 68 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def save({:ok, %Response{body: %{"data" !=> _} = body, status_code: 200}}, schema_atom), do: do_save(body, schema_atom) defp do_save(body, schema_atom) do with {:ok, parsed} !<- parse_jsonapi(body), {:ok, normalized} !<- normalize(parsed), {:ok, _} !<- insert_all(normalized, schema_atom), do: {:ok, schema_atom} end

Slide 69

Slide 69 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix with :ok !<- validate_guess(game, guess), {:ok, game} !<- record_guess(game, guess), {:ok, game, guess} !<- check_move(game, guess), {:ok, game} !<- update_letters(game, guess), {:ok, game} !<- update_guess_state(game, :good_guess) do report(game) else {:err, game, :already_guessed} !-> {:ok, game} = update_guess_state(game, :already_guessed) report(game) {:err, game, :bad_guess} !-> game !|> update_guess_state(:bad_guess) !|> subtract_turn() !|> report() end

Slide 70

Slide 70 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def transform(%{} = transformed), do: Enum.reduce(transformed, %{}, &do_transform/2) defp do_transform({k, {:date, v}}, acc), do: Map.put(acc, k, to_ecto_datetime(v)) defp do_transform({k, {:string, v}}, acc), do: Map.put(acc, k, Kernel.to_string(v)) defp do_transform({k, {:foreign_key, v}}, acc), do: Map.put(acc, k, String.downcase(Kernel.to_string(v))) defp do_transform({k, {_, v}}, acc), do: Map.put(acc, k, v) defp do_transform({_, _}, _), do: {:err, :no_transformer}

Slide 71

Slide 71 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule ID3Parser do def parse(file_name) do case File.read(file_name) do {:ok, binary} !-> mp3_byte_size = (byte_size(binary) - 128) !<< _ !:: binary-size(mp3_byte_size), id3_tag !:: binary !>> = binary !<< "TAG", title !:: binary-size(30), artist !:: binary-size(30), album !:: binary-size(30), year !:: binary-size(4), comment !:: binary-size(30), _rest !:: binary !>> = id3_tag IO.puts {title, artist, album, year, comment} _ !-> IO.puts "Couldn't open !#{file_name}" end end end ID3Parser.parse("sample.mp3") http://benjamintan.io/blog/2014/06/10/elixir-bit-syntax-and-id3/

Slide 72

Slide 72 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Macros

Slide 73

Slide 73 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix https://www.youtube.com/watch?v=zlZdOwAWcbo

Slide 74

Slide 74 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix quote do: 1!..100 !|> Enum.map(&(&1 * &1)) {:!|>, [context: Elixir, import: Kernel], [{:!.., [context: Elixir, import: Kernel], [1, 100]}, {{:., [], [{:!__aliases!__, [alias: false], [:Enum]}, :map]}, [], [{:&, [], [{:*, [context: Elixir, import: Kernel], [{:&, [], [1]}, {:&, [], [1]}]}]}]}]}

Slide 75

Slide 75 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix BEAM ELIXIR'S STRONGEST ASSET

Slide 76

Slide 76 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule SimpleQueue do use GenServer def init(state), do: {:ok, state} # GenServer API def handle_call(:dequeue, _from, [value|state]) do {:reply, value, state} end def handle_call(:dequeue, _from, []), do: {:reply, nil, []} def handle_call(:queue, _from, state), do: {:reply, state, state} def handle_cast({:enqueue, value}, state) do {:noreply, state !++ [value]} end # Client API def start_link(state, opts !\\ []) do GenServer.start_link(!__MODULE!__, state, opts) end def queue, do: GenServer.call(!__MODULE!__, :queue) def enqueue(value), do: GenServer.cast(!__MODULE!__, {:enqueue, value}) def dequeue, do: GenServer.call(!__MODULE!__, :dequeue) end

Slide 77

Slide 77 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(1)> SimpleQueue.start_link([1, 2, 3]) {:ok, #PID<0.385.0>} iex(2)> SimpleQueue.dequeue 1 iex(3)> SimpleQueue.dequeue 2 iex(4)> SimpleQueue.queue [3] iex(5)> SimpleQueue.enqueue(5) :ok iex(6)> SimpleQueue.queue [3, 5]

Slide 78

Slide 78 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(1)> {:ok, pid} = SimpleQueue.start_link([1, 2, 3]) {:ok, #PID<0.378.0>} iex(2)> GenServer.call(pid, :unknown) !** (EXIT from #PID<0.374.0>) an exception was raised: !** (FunctionClauseError) no function clause matching in SimpleQueue.handle_call/3 (data_warehouse) lib/queue.ex:7: SimpleQueue.handle_call(:unknown, {#PID<0.374.0>, #Reference<0.0.4.2238>}, [1, 2, 3]) (stdlib) gen_server.erl:615: :gen_server.try_handle_call/4 (stdlib) gen_server.erl:647: :gen_server.handle_msg/5 (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3

Slide 79

Slide 79 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(2)> GenServer.call(pid, :queue) warning: variable "pid" does not exist and is being expanded to "pid()", please use parentheses to remove the ambiguity or change the variable name iex:2 !** (CompileError) iex:2: undefined function pid/0 (stdlib) lists.erl:1354: :lists.mapfoldl/3

Slide 80

Slide 80 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule SimpleSupervisor do use Supervisor def start_link do Supervisor.start_link(!__MODULE!__, :ok) end def init(:ok) do children = [ worker(SimpleQueue, [[], [name: SimpleQueue]]) ] supervise(children, strategy: :one_for_one) end end

Slide 81

Slide 81 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(1)> {:ok, sup_pid} = SimpleSupervisor.start_link() {:ok, #PID<0.377.0>} iex(2)> SimpleQueue.enqueue(1) :ok iex(3)> SimpleQueue.queue [1] iex(4)> GenServer.call(SimpleQueue, :unknown) [error] GenServer SimpleQueue terminating !** (FunctionClauseError) no function clause matching in SimpleQueue.handle_call/3 (data_warehouse) lib/simple_queue.ex:7: SimpleQueue.handle_call(:unknown, {#PID<0.374.0>, #Reference<0.0.6.936>}, [1]) (stdlib) gen_server.erl:615: :gen_server.try_handle_call/4 (stdlib) gen_server.erl:647: :gen_server.handle_msg/5 (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3 Last message: :unknown State: [1] (elixir) lib/gen_server.ex:737: GenServer.call/3 iex(4)> SimpleQueue.queue []

Slide 82

Slide 82 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix one web-server handling 2 millions sessions ✅ 2 million web-servers handling one session each

Slide 83

Slide 83 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Distributed Elixir Cinemagraph by /u/orbojunglist

Slide 84

Slide 84 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 85

Slide 85 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(one@localhost)1> Node.list() []

Slide 86

Slide 86 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(one@localhost)2> Node.connect :two@localhost true iex(one@localhost)3> Node.list() [:two@localhost]

Slide 87

Slide 87 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix 1 2

Slide 88

Slide 88 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(three@localhost)1> Node.connect :one@localhost true iex(three@localhost)2> Node.list() [:one@localhost, :two@localhost]

Slide 89

Slide 89 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix 1 2 3

Slide 90

Slide 90 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 91

Slide 91 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(two@localhost)1> :rpc.multicall(IO, :inspect, ["Hello world!"]) "Hello world!" "Hello world!" "Hello world!" {["Hello world!", "Hello world!", "Hello world!"], []}

Slide 92

Slide 92 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(two@localhost)10> System.get_pid() "95943" iex(two@localhost)11> :rpc.multicall(System, :get_pid, []) {["95943", "95810", "96120"], []}

Slide 93

Slide 93 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(two@localhost)20> pid = :rpc.async_call(:one@localhost, Enum, :map, [[1, 2, 3], &(&1 * &1)]) #PID<0.132.0> iex(two@localhost)21> :rpc.yield(pid) [1, 4, 9]

Slide 94

Slide 94 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(two@localhost)1> Node.list [] iex(two@localhost)2> :net_adm.world_list([:localhost]) !|> Enum.map(&Node.connect/1) [true, true, true] iex(two@localhost)3> Node.list [:one@localhost, :three@localhost]

Slide 95

Slide 95 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 96

Slide 96 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 97

Slide 97 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 98

Slide 98 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Monitoring Cinemagraph by /u/orbojunglist

Slide 99

Slide 99 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(1)> :observer.start() :ok

Slide 100

Slide 100 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 101

Slide 101 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 102

Slide 102 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 103

Slide 103 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 104

Slide 104 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 105

Slide 105 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 106

Slide 106 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 107

Slide 107 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 108

Slide 108 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix The Outside World Cinemagraph by /u/orbojunglist

Slide 109

Slide 109 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 110

Slide 110 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 111

Slide 111 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 112

Slide 112 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 113

Slide 113 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I. Why Elixir? II. Umbrella monoliths III. Advanced topics

Slide 114

Slide 114 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Umbrella Applications

Slide 115

Slide 115 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix https://martinfowler.com/bliki/BoundedContext.html

Slide 116

Slide 116 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Peedy STAMP WATERMARK QUEUE WEB TONIQ REDIS HTTP POST Watermarked PDF PDFKIT.JS ERLGUTEN PDFTK 200 OK HTTP POST to Callback URL http://localhost:4000/api/v1/documents?watermark=jim.bob@example.com&callback_url=http://localhost:4000/api/v1/dev/null https://github.com/poteto/peedy

Slide 117

Slide 117 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix REST API to the umbrella PEEDY: WEB

Slide 118

Slide 118 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Router PEEDY: WEB

Slide 119

Slide 119 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule PeedyWeb.Router do use PeedyWeb.Web, :router pipeline :api do plug :accepts, ["json"] plug :fetch_session end scope "/", PeedyWeb do get "/healthcheck", HealthcheckController, :index end scope "/api", PeedyWeb do pipe_through :api scope "/v1", Api.V1 do resources "/documents", DocumentController, only: [:show, :create] post "/dev/null", DevNullController, :create end end end

Slide 120

Slide 120 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Document controller PEEDY: WEB

Slide 121

Slide 121 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule PeedyWeb.Api.V1.DocumentController do use PeedyWeb.Web, :controller alias Stamper.{Repo, Document} alias PeedyWeb.ErrorView @callback_client Application.get_env(:peedy_web, :callback_client) @whitelisted_content_types MapSet.new(~w(application/pdf)) @document_headers %{"Content-Type" !=> "multipart/form-data"} def create(conn, %{"watermark" !=> watermark_text, "callback_url" !=> callback_url} = params) do # get PDFs from post, enqueue watermark job end def create(conn, _) do conn !|> halt() !|> put_status(:bad_request) !|> render(ErrorView, "400.json", %{detail: "Watermark and/or callback_url are missing"}) end end

Slide 122

Slide 122 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defp create_callback(%Document{id: id, output: output}, filename, callback_url) do file_path = System.tmp_dir!() !<> Zarex.sanitize(filename) File.write!(file_path, output) @callback_client.do_callback(callback_url, %{file: file_path, id: id}, @document_headers) end defp whitelist_content_type(uploads) when is_list(uploads) do Enum.filter(uploads, fn %Plug.Upload{content_type: content_type} !-> MapSet.member?(@whitelisted_content_types, content_type) end) end

Slide 123

Slide 123 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def create(conn, %{"watermark" !=> watermark_text, "callback_url" !=> callback_url} = params) do params !|> Map.drop(["watermark", "callback_url"]) !|> Map.values() !|> whitelist_content_type() !|> case do [] !-> conn !|> halt() !|> put_status(:bad_request) !|> render(ErrorView, "400.json", %{detail: "No PDFs received"}) files !-> watermark = Watermarker.create(watermark_text) Enum.map(files, fn %Plug.Upload{filename: filename, path: path} !-> Peedy.F.watermark_with(watermark, &(create_callback(&1, filename, callback_url)), input_path: path, ephemeral!?: false) end) send_resp(conn, :ok, "OK") end end

Slide 124

Slide 124 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Callback client PEEDY: WEB

Slide 125

Slide 125 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule PeedyWeb.CallbackClient do use Toniq.Worker, max_concurrency: Application.get_env(:peedy_web, :max_concurrency) alias Stamper.{Repo, Document} @adapter HTTPoison def do_callback(callback_url, %{file: file, id: id}, headers, adapter !\\ @adapter) do Toniq.enqueue(PeedyWeb.CallbackClient, callback_url: callback_url, file: file, id: id, headers: headers, adapter: adapter) end def perform(callback_url: callback_url, file: file, id: id, headers: headers, adapter: adapter) do adapter.post(callback_url, multipart_encode([file: file, id: id]), headers) Repo.get!(Document, id) end defp multipart_encode(body) do body = Enum.map(body, fn {:file, path} !-> {:file, path} {key, value} !-> {to_string(key), value} end) {:multipart, body} end end

Slide 126

Slide 126 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Process watermark jobs asynchronously PEEDY: QUEUE

Slide 127

Slide 127 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 128

Slide 128 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule Peedy.F do require Logger use Toniq.Worker, max_concurrency: Application.get_env(:f, :max_concurrency) alias Stamper.Document alias Watermarker.Watermark @doc """ Queues the watermarking of a document with text and invokes the callback when complete. """ @spec watermark_with(Watermark.t, callback !:: (!!... !-> any()), opts !:: [input_path: String.t, ephemeral!?: boolean()]) !:: %{} def watermark_with(%Watermark{} = watermark, callback, input_path: input_path, ephemeral!?: ephemeral?) when is_function(callback) do # enqueue job with callback end @doc """ Queues the watermarking of a document with text. Uses the default callback. """ @spec watermark_with(Watermark.t, opts !:: [input_path: String.t, ephemeral!?: boolean()]) !:: %{} def watermark_with(%Watermark{} = watermark, input_path: input_path, ephemeral!?: ephemeral?) do # enqueue job with default callback end end

Slide 129

Slide 129 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def perform(watermark: watermark, input_path: input_path, ephemeral!?: true, callback: callback), do: do_perform(watermark, input_path, true, callback) def perform(watermark: watermark, input_path: input_path, ephemeral!?: false, callback: callback), do: do_perform(watermark, input_path, false, callback) defp do_perform(%Watermark{} = watermark, input_path, ephemeral?, callback) do %Document{} = watermark !|> Stamper.stamp_with(input_path: input_path, ephemeral!?: ephemeral?) !|> callback.() end

Slide 130

Slide 130 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Create watermark layer PEEDY: WATERMARK

Slide 131

Slide 131 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def create(text, strategy: :pdfkit) when is_binary(text), do: Pdfkit.new(text) def create(text, strategy: :erlguten) when is_binary(text), do: Erlguten.new(text)

Slide 132

Slide 132 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule Watermarker.WatermarkerBehaviour do alias Watermarker.Watermark @callback new(text !:: String.t) !:: Watermark.t end

Slide 133

Slide 133 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule Watermarker.Strategies.Pdfkit do @moduledoc """ Create a watermark by using a PDFKit. """ require Logger alias Watermarker.{Watermark,Repo} @behaviour Watermarker.WatermarkerBehaviour @nodejs Application.get_env(:watermarker, :executables)[:nodejs] @script_name Application.get_env(:watermarker, :executables)[:pdfkit] def new(text) when is_binary(text) do case Repo.get_by(Watermark, input: text) do %Watermark{} = watermark !-> watermark nil !-> sanitized_text = Zarex.sanitize(text) output_path = System.tmp_dir!() !<> "!#{sanitized_text}.pdf" output = text !|> to_pdf(output_path: output_path) !|> File.read!() %Watermark{} !|> Watermark.changeset(%{input: text, output: output}) !|> Repo.insert!() end end end

Slide 134

Slide 134 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 135

Slide 135 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def to_pdf(text, output_path: output_path) when is_binary(text) do File.touch!(output_path) %Porcelain.Result{err: nil, out: out, status: 0} = Porcelain.exec(@nodejs, [@script_name, "!--text", text, "!--output", output_path]) out end

Slide 136

Slide 136 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Stamp watermark layer onto PDF PEEDY: STAMP

Slide 137

Slide 137 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def stamp_with(%Watermark{id: id} = watermark, input_path: input_path, ephemeral!?: false) do {microseconds, document} = :timer.tc(fn !-> output_path = stamp_document(watermark, input_path) %Document{} !|> Document.changeset(%{ input: File.read!(input_path), output: File.read!(output_path), stamp_id: id}) !|> Repo.insert!() end) Logger.info("Stamped !#{document.id} in !#{microseconds / 1_000}ms") document end

Slide 138

Slide 138 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 139

Slide 139 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defp stamp_document(%Watermark{} = watermark, input_path) do output_path = System.tmp_dir!() !<> "!#{Zarex.sanitize(random_string())}.pdf" stamp_path = stamp_path(watermark) if File.exists?(input_path) do %Porcelain.Result{err: nil, out: "", status: 0} = Porcelain.exec(@pdftk, [input_path, "multistamp", stamp_path, "output", output_path]) output_path else raise "Input at !#{input_path} does not exist" end end

Slide 140

Slide 140 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix WEB ACCOUNTING PRODUCT SHOPPING PAYMENTS DELIVERY O'rly Bookstore SESSION CART CHECKOUT INVENTORY LEDGER REPORTS REVIEWS PRICES METADATA PROCESSING FRAUD REFUNDS WATERMARK PDF SHIPPING RETURNS

Slide 141

Slide 141 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix What the business does BOUNDED CONTEXTS: DOMAIN

Slide 142

Slide 142 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Smaller units of the business BOUNDED CONTEXTS: SUBDOMAIN

Slide 143

Slide 143 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I. Why Elixir? II. Umbrella monoliths III. Advanced topics

Slide 144

Slide 144 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Rate Limiting

Slide 145

Slide 145 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Simple Queue

Slide 146

Slide 146 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule Queue do @strategy Queue.SimpleQueue def enqueue(message, strategy !\\ @strategy), do: strategy.enqueue(message) def enqueue_all(messages, strategy !\\ @strategy) when is_list(messages) do messages !|> Stream.map(&strategy.enqueue/1) !|> Enum.to_list() end def queue(strategy !\\ @strategy), do: strategy.queue() def dequeue(count !\\ 1, strategy !\\ @strategy) when is_integer(count), do: strategy.dequeue(count) end

Slide 147

Slide 147 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule Queue.Application do @moduledoc false use Application def start(_type, _args) do import Supervisor.Spec, warn: false children = [ worker(Queue.SimpleQueue.Supervisor, []), worker(Queue.Pipeline.Producer, []), worker(Queue.Pipeline.Consumer, []) ] opts = [strategy: :one_for_one, name: Queue.Supervisor] Supervisor.start_link(children, opts) end end

Slide 148

Slide 148 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule Queue.Pipeline.Producer do use GenStage require Logger alias Queue.SimpleQueue def start_link(state !\\ 0) do GenStage.start_link(!__MODULE!__, state, name: !__MODULE!__) end def init(state) do {:producer, state} end def handle_cast(:check_messages, 0), do: {:noreply, [], 0} def handle_cast(:check_messages, state) when is_integer(state) do messages = SimpleQueue.dequeue(state) GenStage.cast(!__MODULE!__, :check_messages) {:noreply, messages, state - Enum.count(messages)} end def handle_demand(demand, state) when is_integer(state) do GenStage.cast(!__MODULE!__, :check_messages) {:noreply, [], demand + state} end end

Slide 149

Slide 149 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule Queue.Pipeline.Processor do require Logger @interval 5_000 def start_link(message) do {:ok, _} = Task.start_link(!__MODULE!__, :process_message, [message]) end def process_message(message) do :timer.sleep(@interval) message !|> inspect() !|> Logger.info() end end

Slide 150

Slide 150 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule Queue.Pipeline.Consumer do use ConsumerSupervisor def start_link do ConsumerSupervisor.start_link(!__MODULE!__, :ok) end def init(:ok) do children = [ worker(Queue.Pipeline.Processor, [], restart: :temporary) ] {:ok, children, strategy: :one_for_one, subscribe_to: [ {Queue.Pipeline.Producer, max_demand: 10, min_demand: 1} ]} end end

Slide 151

Slide 151 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 152

Slide 152 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 153

Slide 153 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 154

Slide 154 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Circuit Breakers

Slide 155

Slide 155 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 156

Slide 156 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 157

Slide 157 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 158

Slide 158 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 159

Slide 159 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

Slide 160

Slide 160 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule SomeApi do @fuse_name !__MODULE!__ @fuse_options { {:standard, 2, 10_000}, {:reset, 60_000} } def init do install_fuse() end def fetch_all do case :fuse.ask(@fuse_name, :sync) do :ok !-> # continue normally :blown !-> # handle failure so other applications that depend on this one won't # break end end defp do_fetch_all do case SomeApi.Adapter.get("/some/path") do {:ok, resp} !-> # do stuff with data {:error, _} !-> :fuse.melt(@fuse_name) end end defp install_fuse, do: :fuse.install(@fuse_name, @fuse_options) end

Slide 161

Slide 161 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defp do_fetch_all do case SomeApi.Adapter.get("/some/path") do {:ok, resp} !-> # do stuff with data {:error, _} !-> :fuse.melt(@fuse_name) end end

Slide 162

Slide 162 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def fetch_all do case :fuse.ask(@fuse_name, :sync) do :ok !-> # continue normally :blown !-> # handle failure so other applications that depend on this one won't # break end end

Slide 163

Slide 163 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I. Why Elixir? II. Umbrella monoliths III. Advanced topics

Slide 164

Slide 164 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I. Why Elixir? II. Umbrella monoliths III. Advanced topics

Slide 165

Slide 165 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I. Why Elixir? II. Umbrella monoliths III. Advanced topics

Slide 166

Slide 166 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I. Why Elixir? II. Umbrella monoliths III. Advanced topics

Slide 167

Slide 167 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Lauren Tan sugarpirate_ poteto

Slide 168

Slide 168 text

CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Cinemagraph by /u/fezzo Thank you sugarpirate_ poteto