Pro Yearly is on sale from $80 to $50! »

CodeEurope 2017 – Monoliths to Services with Elixir and Phoenix

CodeEurope 2017 – Monoliths to Services with Elixir and Phoenix

This talk was presented at CodeEurope 2017 by Lauren Tan. Recording available here - https://youtu.be/g2ATsmnkjbs.

Transitioning monolithic apps into a micro-service architecture isn't straightforward – in fact, it is often quite difficult. In this intermediate talk, we'll learn how Elixir umbrella apps and Phoenix utilize the Erlang VM (BEAM) to make building services less painful and more productive.

C8fccffc013096c4b465b50c284a5208?s=128

Lauren Tan

May 25, 2017
Tweet

Transcript

  1. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Monoliths

    to Services with Elixir & Phoenix PRESENTED BY Lauren Tan sugarpirate_ poteto Cinemagraph by /u/orbojunglist
  2. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix bit.ly/elixir-microservices

  3. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Lauren

    Tan sugarpirate_ poteto
  4. EmberConf 2017 Confessions of an Ember Addon Author

  5. EmberConf 2017 Confessions of an Ember Addon Author

  6. EmberConf 2017 Confessions of an Ember Addon Author

  7. From Front-End to Full Stack with Elixir & Phoenix ElixirConf

    2016
  8. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  9. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  10. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  11. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix https://github.com/Netflix/vizceral

  12. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  13. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  14. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  15. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  16. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  17. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix http://worrydream.com/refs/Brooks-NoSilverBullet.pdf

  18. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  19. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Microservices

    will improve our performance easily FALLACY #1
  20. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Network

    latency
  21. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix What's

    the bottleneck?
  22. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix The

    code is cleaner FALLACY #2
  23. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  24. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix You

    can write clean code without microservices
  25. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix It's

    easier to write something that does one thing well FALLACY #3
  26. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Application

    simplicity
  27. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Infrastructure

    complexity
  28. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix https://martinfowler.com/articles/microservice-trade-offs.html

  29. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Embrace

    the monolith
  30. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Microservices

    are an organizational optimization https://martinfowler.com/bliki/MicroservicePremium.html
  31. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix https://www.slideshare.net/reed2001/culture-1798664/

  32. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Conway's

    Law
  33. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I.

    Why Elixir? II. Umbrella monoliths III. Advanced topics
  34. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I.

    Why Elixir? II. Umbrella monoliths III. Advanced topics
  35. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix The

    right tool for the job Cinemagraph by /u/smoothinto2nd
  36. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Web

    API/apps ELIXIR IS GREAT FOR
  37. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Soft

    real-time ELIXIR IS GREAT FOR
  38. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix ELIXIR

    IS GREAT FOR High scalability and fault tolerance
  39. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix ELIXIR

    IS GREAT FOR Concurrency
  40. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix ELIXIR

    IS GREAT FOR Developer happiness
  41. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Number

    crunching ELIXIR IS NOT GREAT FOR
  42. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Anything

    CPU bound ELIXIR IS NOT GREAT FOR
  43. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  44. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  45. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Highly-scalable,

    fault-tolerant systems ERLANG
  46. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Pre-emptive

    scheduling THE ERLANG VM: BEAM BEAM SCHEDULER SCHEDULER SCHEDULER SCHEDULER Processes Processes Processes Processes OS Process OS Thread OS Thread OS Thread OS Thread https://hamidreza-s.github.io/erlang/scheduling/real-time/preemptive/migration/2016/02/09/erlang-scheduler-details.html
  47. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix 1.

    Everything is a process 2. Processes are strongly isolated 3. Process creation and destruction is a lightweight operation 4. Message passing is the only way for processes to interact 5. Processes have unique names 6. If you know the name of a process you can send it a message 7. Processes share no resources 8. Error handling is non-local 9. Processes do what they are supposed to do or fail http://erlang.org/download/armstrong_thesis_2003.pdf
  48. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix JAVA

    Write once, run anywhere
  49. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix ERLANG

    https://www.youtube.com/watch?v=u41GEwIq2mE&t=3m59s Write once, run forever
  50. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Garbage

    Collection
  51. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix http://www.erlang-factory.com/static/upload/media/1394350183453526efsf2014whatsappscaling.pdf

  52. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix https://www.youtube.com/watch?v=5SbWapbXhKo

  53. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Elixir?

  54. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix 1

    + 1 #!=> 2 {:ok, "hello"} [1, 2, 3] = [head | tail] head #!=> 1 tail #!=> [2,3]
  55. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    Math do def add(x, y), do: x + y def subtract(x, y) do x - y end end
  56. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    Collection do def sum([x, y, z]), do: x + y + z end
  57. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(2)>

    Collection.sum([1, 2, 3]) 6 iex(3)> Collection.sum([1, 2, 3, 4]) !** (FunctionClauseError) no function clause matching in Collection.sum/1 iex:2: Collection.sum([1, 2, 3, 4])
  58. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Pipe

    Operator
  59. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix foo(bar(baz(123),

    456), 789)
  60. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix 123

    !|> baz() !|> bar(456) !|> foo(123)
  61. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def

    serialize(user) do user !|> serialize_attributes() !|> serialize_relationships() !|> to_json() end
  62. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def

    validate_presence(input) do if input !== "" do "Can't be blank" end input end
  63. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Pattern

    Matching
  64. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def

    validate_presence(""), do: {:err, "Can't be blank"} def validate_presence(input), do: input
  65. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix thing

    = %{foo: "bar", baz: 123} %{foo: foo, baz: baz} = thing; IO.inspect(foo) # "bar" IO.inspect(baz) # 123
  66. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix thing

    = %{foo: "bar", baz: 123} %{foo: foo, baz: baz, qux: qux} = thing; !** (MatchError) no match of right hand side value: %{baz: 123, foo: "bar"}
  67. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix https://fsharpforfunandprofit.com/rop/

  68. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def

    save({:ok, %Response{body: %{"data" !=> _} = body, status_code: 200}}, schema_atom), do: do_save(body, schema_atom) defp do_save(body, schema_atom) do with {:ok, parsed} !<- parse_jsonapi(body), {:ok, normalized} !<- normalize(parsed), {:ok, _} !<- insert_all(normalized, schema_atom), do: {:ok, schema_atom} end
  69. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix with

    :ok !<- validate_guess(game, guess), {:ok, game} !<- record_guess(game, guess), {:ok, game, guess} !<- check_move(game, guess), {:ok, game} !<- update_letters(game, guess), {:ok, game} !<- update_guess_state(game, :good_guess) do report(game) else {:err, game, :already_guessed} !-> {:ok, game} = update_guess_state(game, :already_guessed) report(game) {:err, game, :bad_guess} !-> game !|> update_guess_state(:bad_guess) !|> subtract_turn() !|> report() end
  70. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def

    transform(%{} = transformed), do: Enum.reduce(transformed, %{}, &do_transform/2) defp do_transform({k, {:date, v}}, acc), do: Map.put(acc, k, to_ecto_datetime(v)) defp do_transform({k, {:string, v}}, acc), do: Map.put(acc, k, Kernel.to_string(v)) defp do_transform({k, {:foreign_key, v}}, acc), do: Map.put(acc, k, String.downcase(Kernel.to_string(v))) defp do_transform({k, {_, v}}, acc), do: Map.put(acc, k, v) defp do_transform({_, _}, _), do: {:err, :no_transformer}
  71. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    ID3Parser do def parse(file_name) do case File.read(file_name) do {:ok, binary} !-> mp3_byte_size = (byte_size(binary) - 128) !<< _ !:: binary-size(mp3_byte_size), id3_tag !:: binary !>> = binary !<< "TAG", title !:: binary-size(30), artist !:: binary-size(30), album !:: binary-size(30), year !:: binary-size(4), comment !:: binary-size(30), _rest !:: binary !>> = id3_tag IO.puts {title, artist, album, year, comment} _ !-> IO.puts "Couldn't open !#{file_name}" end end end ID3Parser.parse("sample.mp3") http://benjamintan.io/blog/2014/06/10/elixir-bit-syntax-and-id3/
  72. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Macros

  73. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix https://www.youtube.com/watch?v=zlZdOwAWcbo

  74. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix quote

    do: 1!..100 !|> Enum.map(&(&1 * &1)) {:!|>, [context: Elixir, import: Kernel], [{:!.., [context: Elixir, import: Kernel], [1, 100]}, {{:., [], [{:!__aliases!__, [alias: false], [:Enum]}, :map]}, [], [{:&, [], [{:*, [context: Elixir, import: Kernel], [{:&, [], [1]}, {:&, [], [1]}]}]}]}]}
  75. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix BEAM

    ELIXIR'S STRONGEST ASSET
  76. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    SimpleQueue do use GenServer def init(state), do: {:ok, state} # GenServer API def handle_call(:dequeue, _from, [value|state]) do {:reply, value, state} end def handle_call(:dequeue, _from, []), do: {:reply, nil, []} def handle_call(:queue, _from, state), do: {:reply, state, state} def handle_cast({:enqueue, value}, state) do {:noreply, state !++ [value]} end # Client API def start_link(state, opts !\\ []) do GenServer.start_link(!__MODULE!__, state, opts) end def queue, do: GenServer.call(!__MODULE!__, :queue) def enqueue(value), do: GenServer.cast(!__MODULE!__, {:enqueue, value}) def dequeue, do: GenServer.call(!__MODULE!__, :dequeue) end
  77. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(1)>

    SimpleQueue.start_link([1, 2, 3]) {:ok, #PID<0.385.0>} iex(2)> SimpleQueue.dequeue 1 iex(3)> SimpleQueue.dequeue 2 iex(4)> SimpleQueue.queue [3] iex(5)> SimpleQueue.enqueue(5) :ok iex(6)> SimpleQueue.queue [3, 5]
  78. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(1)>

    {:ok, pid} = SimpleQueue.start_link([1, 2, 3]) {:ok, #PID<0.378.0>} iex(2)> GenServer.call(pid, :unknown) !** (EXIT from #PID<0.374.0>) an exception was raised: !** (FunctionClauseError) no function clause matching in SimpleQueue.handle_call/3 (data_warehouse) lib/queue.ex:7: SimpleQueue.handle_call(:unknown, {#PID<0.374.0>, #Reference<0.0.4.2238>}, [1, 2, 3]) (stdlib) gen_server.erl:615: :gen_server.try_handle_call/4 (stdlib) gen_server.erl:647: :gen_server.handle_msg/5 (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
  79. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(2)>

    GenServer.call(pid, :queue) warning: variable "pid" does not exist and is being expanded to "pid()", please use parentheses to remove the ambiguity or change the variable name iex:2 !** (CompileError) iex:2: undefined function pid/0 (stdlib) lists.erl:1354: :lists.mapfoldl/3
  80. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    SimpleSupervisor do use Supervisor def start_link do Supervisor.start_link(!__MODULE!__, :ok) end def init(:ok) do children = [ worker(SimpleQueue, [[], [name: SimpleQueue]]) ] supervise(children, strategy: :one_for_one) end end
  81. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(1)>

    {:ok, sup_pid} = SimpleSupervisor.start_link() {:ok, #PID<0.377.0>} iex(2)> SimpleQueue.enqueue(1) :ok iex(3)> SimpleQueue.queue [1] iex(4)> GenServer.call(SimpleQueue, :unknown) [error] GenServer SimpleQueue terminating !** (FunctionClauseError) no function clause matching in SimpleQueue.handle_call/3 (data_warehouse) lib/simple_queue.ex:7: SimpleQueue.handle_call(:unknown, {#PID<0.374.0>, #Reference<0.0.6.936>}, [1]) (stdlib) gen_server.erl:615: :gen_server.try_handle_call/4 (stdlib) gen_server.erl:647: :gen_server.handle_msg/5 (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3 Last message: :unknown State: [1] (elixir) lib/gen_server.ex:737: GenServer.call/3 iex(4)> SimpleQueue.queue []
  82. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix one

    web-server handling 2 millions sessions ✅ 2 million web-servers handling one session each
  83. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Distributed

    Elixir Cinemagraph by /u/orbojunglist
  84. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  85. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(one@localhost)1>

    Node.list() []
  86. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(one@localhost)2>

    Node.connect :two@localhost true iex(one@localhost)3> Node.list() [:two@localhost]
  87. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix 1

    2
  88. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(three@localhost)1>

    Node.connect :one@localhost true iex(three@localhost)2> Node.list() [:one@localhost, :two@localhost]
  89. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix 1

    2 3
  90. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  91. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(two@localhost)1>

    :rpc.multicall(IO, :inspect, ["Hello world!"]) "Hello world!" "Hello world!" "Hello world!" {["Hello world!", "Hello world!", "Hello world!"], []}
  92. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(two@localhost)10>

    System.get_pid() "95943" iex(two@localhost)11> :rpc.multicall(System, :get_pid, []) {["95943", "95810", "96120"], []}
  93. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(two@localhost)20>

    pid = :rpc.async_call(:one@localhost, Enum, :map, [[1, 2, 3], &(&1 * &1)]) #PID<0.132.0> iex(two@localhost)21> :rpc.yield(pid) [1, 4, 9]
  94. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(two@localhost)1>

    Node.list [] iex(two@localhost)2> :net_adm.world_list([:localhost]) !|> Enum.map(&Node.connect/1) [true, true, true] iex(two@localhost)3> Node.list [:one@localhost, :three@localhost]
  95. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  96. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  97. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  98. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Monitoring

    Cinemagraph by /u/orbojunglist
  99. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix iex(1)>

    :observer.start() :ok
  100. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  101. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  102. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  103. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  104. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  105. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  106. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  107. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  108. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix The

    Outside World Cinemagraph by /u/orbojunglist
  109. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  110. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  111. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  112. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  113. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I.

    Why Elixir? II. Umbrella monoliths III. Advanced topics
  114. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Umbrella

    Applications
  115. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix https://martinfowler.com/bliki/BoundedContext.html

  116. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Peedy

    STAMP WATERMARK QUEUE WEB TONIQ REDIS HTTP POST Watermarked PDF PDFKIT.JS ERLGUTEN PDFTK 200 OK HTTP POST to Callback URL http://localhost:4000/api/v1/documents?watermark=jim.bob@example.com&callback_url=http://localhost:4000/api/v1/dev/null https://github.com/poteto/peedy
  117. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix REST

    API to the umbrella PEEDY: WEB
  118. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Router

    PEEDY: WEB
  119. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    PeedyWeb.Router do use PeedyWeb.Web, :router pipeline :api do plug :accepts, ["json"] plug :fetch_session end scope "/", PeedyWeb do get "/healthcheck", HealthcheckController, :index end scope "/api", PeedyWeb do pipe_through :api scope "/v1", Api.V1 do resources "/documents", DocumentController, only: [:show, :create] post "/dev/null", DevNullController, :create end end end
  120. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Document

    controller PEEDY: WEB
  121. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    PeedyWeb.Api.V1.DocumentController do use PeedyWeb.Web, :controller alias Stamper.{Repo, Document} alias PeedyWeb.ErrorView @callback_client Application.get_env(:peedy_web, :callback_client) @whitelisted_content_types MapSet.new(~w(application/pdf)) @document_headers %{"Content-Type" !=> "multipart/form-data"} def create(conn, %{"watermark" !=> watermark_text, "callback_url" !=> callback_url} = params) do # get PDFs from post, enqueue watermark job end def create(conn, _) do conn !|> halt() !|> put_status(:bad_request) !|> render(ErrorView, "400.json", %{detail: "Watermark and/or callback_url are missing"}) end end
  122. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defp

    create_callback(%Document{id: id, output: output}, filename, callback_url) do file_path = System.tmp_dir!() !<> Zarex.sanitize(filename) File.write!(file_path, output) @callback_client.do_callback(callback_url, %{file: file_path, id: id}, @document_headers) end defp whitelist_content_type(uploads) when is_list(uploads) do Enum.filter(uploads, fn %Plug.Upload{content_type: content_type} !-> MapSet.member?(@whitelisted_content_types, content_type) end) end
  123. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def

    create(conn, %{"watermark" !=> watermark_text, "callback_url" !=> callback_url} = params) do params !|> Map.drop(["watermark", "callback_url"]) !|> Map.values() !|> whitelist_content_type() !|> case do [] !-> conn !|> halt() !|> put_status(:bad_request) !|> render(ErrorView, "400.json", %{detail: "No PDFs received"}) files !-> watermark = Watermarker.create(watermark_text) Enum.map(files, fn %Plug.Upload{filename: filename, path: path} !-> Peedy.F.watermark_with(watermark, &(create_callback(&1, filename, callback_url)), input_path: path, ephemeral!?: false) end) send_resp(conn, :ok, "OK") end end
  124. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Callback

    client PEEDY: WEB
  125. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    PeedyWeb.CallbackClient do use Toniq.Worker, max_concurrency: Application.get_env(:peedy_web, :max_concurrency) alias Stamper.{Repo, Document} @adapter HTTPoison def do_callback(callback_url, %{file: file, id: id}, headers, adapter !\\ @adapter) do Toniq.enqueue(PeedyWeb.CallbackClient, callback_url: callback_url, file: file, id: id, headers: headers, adapter: adapter) end def perform(callback_url: callback_url, file: file, id: id, headers: headers, adapter: adapter) do adapter.post(callback_url, multipart_encode([file: file, id: id]), headers) Repo.get!(Document, id) end defp multipart_encode(body) do body = Enum.map(body, fn {:file, path} !-> {:file, path} {key, value} !-> {to_string(key), value} end) {:multipart, body} end end
  126. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Process

    watermark jobs asynchronously PEEDY: QUEUE
  127. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  128. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    Peedy.F do require Logger use Toniq.Worker, max_concurrency: Application.get_env(:f, :max_concurrency) alias Stamper.Document alias Watermarker.Watermark @doc """ Queues the watermarking of a document with text and invokes the callback when complete. """ @spec watermark_with(Watermark.t, callback !:: (!!... !-> any()), opts !:: [input_path: String.t, ephemeral!?: boolean()]) !:: %{} def watermark_with(%Watermark{} = watermark, callback, input_path: input_path, ephemeral!?: ephemeral?) when is_function(callback) do # enqueue job with callback end @doc """ Queues the watermarking of a document with text. Uses the default callback. """ @spec watermark_with(Watermark.t, opts !:: [input_path: String.t, ephemeral!?: boolean()]) !:: %{} def watermark_with(%Watermark{} = watermark, input_path: input_path, ephemeral!?: ephemeral?) do # enqueue job with default callback end end
  129. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def

    perform(watermark: watermark, input_path: input_path, ephemeral!?: true, callback: callback), do: do_perform(watermark, input_path, true, callback) def perform(watermark: watermark, input_path: input_path, ephemeral!?: false, callback: callback), do: do_perform(watermark, input_path, false, callback) defp do_perform(%Watermark{} = watermark, input_path, ephemeral?, callback) do %Document{} = watermark !|> Stamper.stamp_with(input_path: input_path, ephemeral!?: ephemeral?) !|> callback.() end
  130. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Create

    watermark layer PEEDY: WATERMARK
  131. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def

    create(text, strategy: :pdfkit) when is_binary(text), do: Pdfkit.new(text) def create(text, strategy: :erlguten) when is_binary(text), do: Erlguten.new(text)
  132. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    Watermarker.WatermarkerBehaviour do alias Watermarker.Watermark @callback new(text !:: String.t) !:: Watermark.t end
  133. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    Watermarker.Strategies.Pdfkit do @moduledoc """ Create a watermark by using a PDFKit. """ require Logger alias Watermarker.{Watermark,Repo} @behaviour Watermarker.WatermarkerBehaviour @nodejs Application.get_env(:watermarker, :executables)[:nodejs] @script_name Application.get_env(:watermarker, :executables)[:pdfkit] def new(text) when is_binary(text) do case Repo.get_by(Watermark, input: text) do %Watermark{} = watermark !-> watermark nil !-> sanitized_text = Zarex.sanitize(text) output_path = System.tmp_dir!() !<> "!#{sanitized_text}.pdf" output = text !|> to_pdf(output_path: output_path) !|> File.read!() %Watermark{} !|> Watermark.changeset(%{input: text, output: output}) !|> Repo.insert!() end end end
  134. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  135. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def

    to_pdf(text, output_path: output_path) when is_binary(text) do File.touch!(output_path) %Porcelain.Result{err: nil, out: out, status: 0} = Porcelain.exec(@nodejs, [@script_name, "!--text", text, "!--output", output_path]) out end
  136. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Stamp

    watermark layer onto PDF PEEDY: STAMP
  137. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def

    stamp_with(%Watermark{id: id} = watermark, input_path: input_path, ephemeral!?: false) do {microseconds, document} = :timer.tc(fn !-> output_path = stamp_document(watermark, input_path) %Document{} !|> Document.changeset(%{ input: File.read!(input_path), output: File.read!(output_path), stamp_id: id}) !|> Repo.insert!() end) Logger.info("Stamped !#{document.id} in !#{microseconds / 1_000}ms") document end
  138. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  139. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defp

    stamp_document(%Watermark{} = watermark, input_path) do output_path = System.tmp_dir!() !<> "!#{Zarex.sanitize(random_string())}.pdf" stamp_path = stamp_path(watermark) if File.exists?(input_path) do %Porcelain.Result{err: nil, out: "", status: 0} = Porcelain.exec(@pdftk, [input_path, "multistamp", stamp_path, "output", output_path]) output_path else raise "Input at !#{input_path} does not exist" end end
  140. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix WEB

    ACCOUNTING PRODUCT SHOPPING PAYMENTS DELIVERY O'rly Bookstore SESSION CART CHECKOUT INVENTORY LEDGER REPORTS REVIEWS PRICES METADATA PROCESSING FRAUD REFUNDS WATERMARK PDF SHIPPING RETURNS
  141. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix What

    the business does BOUNDED CONTEXTS: DOMAIN
  142. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Smaller

    units of the business BOUNDED CONTEXTS: SUBDOMAIN
  143. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I.

    Why Elixir? II. Umbrella monoliths III. Advanced topics
  144. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Rate

    Limiting
  145. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Simple

    Queue
  146. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    Queue do @strategy Queue.SimpleQueue def enqueue(message, strategy !\\ @strategy), do: strategy.enqueue(message) def enqueue_all(messages, strategy !\\ @strategy) when is_list(messages) do messages !|> Stream.map(&strategy.enqueue/1) !|> Enum.to_list() end def queue(strategy !\\ @strategy), do: strategy.queue() def dequeue(count !\\ 1, strategy !\\ @strategy) when is_integer(count), do: strategy.dequeue(count) end
  147. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    Queue.Application do @moduledoc false use Application def start(_type, _args) do import Supervisor.Spec, warn: false children = [ worker(Queue.SimpleQueue.Supervisor, []), worker(Queue.Pipeline.Producer, []), worker(Queue.Pipeline.Consumer, []) ] opts = [strategy: :one_for_one, name: Queue.Supervisor] Supervisor.start_link(children, opts) end end
  148. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    Queue.Pipeline.Producer do use GenStage require Logger alias Queue.SimpleQueue def start_link(state !\\ 0) do GenStage.start_link(!__MODULE!__, state, name: !__MODULE!__) end def init(state) do {:producer, state} end def handle_cast(:check_messages, 0), do: {:noreply, [], 0} def handle_cast(:check_messages, state) when is_integer(state) do messages = SimpleQueue.dequeue(state) GenStage.cast(!__MODULE!__, :check_messages) {:noreply, messages, state - Enum.count(messages)} end def handle_demand(demand, state) when is_integer(state) do GenStage.cast(!__MODULE!__, :check_messages) {:noreply, [], demand + state} end end
  149. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    Queue.Pipeline.Processor do require Logger @interval 5_000 def start_link(message) do {:ok, _} = Task.start_link(!__MODULE!__, :process_message, [message]) end def process_message(message) do :timer.sleep(@interval) message !|> inspect() !|> Logger.info() end end
  150. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    Queue.Pipeline.Consumer do use ConsumerSupervisor def start_link do ConsumerSupervisor.start_link(!__MODULE!__, :ok) end def init(:ok) do children = [ worker(Queue.Pipeline.Processor, [], restart: :temporary) ] {:ok, children, strategy: :one_for_one, subscribe_to: [ {Queue.Pipeline.Producer, max_demand: 10, min_demand: 1} ]} end end
  151. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  152. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  153. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  154. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Circuit

    Breakers
  155. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  156. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  157. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  158. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  159. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix

  160. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defmodule

    SomeApi do @fuse_name !__MODULE!__ @fuse_options { {:standard, 2, 10_000}, {:reset, 60_000} } def init do install_fuse() end def fetch_all do case :fuse.ask(@fuse_name, :sync) do :ok !-> # continue normally :blown !-> # handle failure so other applications that depend on this one won't # break end end defp do_fetch_all do case SomeApi.Adapter.get("/some/path") do {:ok, resp} !-> # do stuff with data {:error, _} !-> :fuse.melt(@fuse_name) end end defp install_fuse, do: :fuse.install(@fuse_name, @fuse_options) end
  161. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix defp

    do_fetch_all do case SomeApi.Adapter.get("/some/path") do {:ok, resp} !-> # do stuff with data {:error, _} !-> :fuse.melt(@fuse_name) end end
  162. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix def

    fetch_all do case :fuse.ask(@fuse_name, :sync) do :ok !-> # continue normally :blown !-> # handle failure so other applications that depend on this one won't # break end end
  163. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I.

    Why Elixir? II. Umbrella monoliths III. Advanced topics
  164. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I.

    Why Elixir? II. Umbrella monoliths III. Advanced topics
  165. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I.

    Why Elixir? II. Umbrella monoliths III. Advanced topics
  166. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix I.

    Why Elixir? II. Umbrella monoliths III. Advanced topics
  167. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Lauren

    Tan sugarpirate_ poteto
  168. CodeEurope 2017 Monoliths to Services with Elixir & Phoenix Cinemagraph

    by /u/fezzo Thank you sugarpirate_ poteto