CodeEurope 2017 – Monoliths to Services with Elixir and Phoenix

CodeEurope 2017
Monoliths to Services with Elixir & Phoenix
Monoliths
to Services
with Elixir &
Phoenix
PRESENTED BY
Lauren Tan
sugarpirate_
poteto
Cinemagraph by /u/orbojunglist

View Slide

CodeEurope 2017
bit.ly/elixir-microservices

View Slide

CodeEurope 2017
Lauren Tan
sugarpirate_
poteto

View Slide

EmberConf 2017
Confessions of an Ember Addon Author

View Slide

From Front-End to Full Stack with Elixir & Phoenix
ElixirConf 2016

View Slide

CodeEurope 2017

View Slide

CodeEurope 2017
https://github.com/Netflix/vizceral

View Slide

CodeEurope 2017

View Slide

CodeEurope 2017
http://worrydream.com/refs/Brooks-NoSilverBullet.pdf

View Slide

CodeEurope 2017

View Slide

CodeEurope 2017
Microservices will
improve our
performance easily
FALLACY #1

View Slide

CodeEurope 2017
Network latency

View Slide

CodeEurope 2017
What's the bottleneck?

View Slide

CodeEurope 2017
The code is cleaner
FALLACY #2

View Slide

CodeEurope 2017

View Slide

CodeEurope 2017
You can write
clean code without
microservices

View Slide

CodeEurope 2017
It's easier to write
something that does
one thing well
FALLACY #3

View Slide

CodeEurope 2017
Application
simplicity

View Slide

CodeEurope 2017
Infrastructure
complexity

View Slide

CodeEurope 2017
https://martinfowler.com/articles/microservice-trade-offs.html

View Slide

CodeEurope 2017
Embrace the monolith

View Slide

CodeEurope 2017
Microservices are
an organizational
optimization
https://martinfowler.com/bliki/MicroservicePremium.html

View Slide

CodeEurope 2017
https://www.slideshare.net/reed2001/culture-1798664/

View Slide

CodeEurope 2017
Conway's Law

View Slide

CodeEurope 2017
I. Why Elixir?
II. Umbrella monoliths
III. Advanced topics

View Slide

CodeEurope 2017
The right tool for the
job
Cinemagraph by /u/smoothinto2nd

View Slide

CodeEurope 2017
Web API/apps
ELIXIR IS GREAT FOR

View Slide

CodeEurope 2017
Soft real-time
ELIXIR IS GREAT FOR

View Slide

CodeEurope 2017
ELIXIR IS GREAT FOR
High scalability and fault
tolerance

View Slide

CodeEurope 2017
ELIXIR IS GREAT FOR
Concurrency

View Slide

CodeEurope 2017
ELIXIR IS GREAT FOR
Developer happiness

View Slide

CodeEurope 2017
Number crunching
ELIXIR IS NOT GREAT FOR

View Slide

CodeEurope 2017
Anything CPU bound
ELIXIR IS NOT GREAT FOR

View Slide

CodeEurope 2017

View Slide

CodeEurope 2017
Highly-scalable,
fault-tolerant
systems
ERLANG

View Slide

CodeEurope 2017
Pre-emptive
scheduling
THE ERLANG VM: BEAM
BEAM
SCHEDULER SCHEDULER SCHEDULER SCHEDULER
Processes Processes Processes Processes
OS Process
OS Thread OS Thread OS Thread OS Thread
https://hamidreza-s.github.io/erlang/scheduling/real-time/preemptive/migration/2016/02/09/erlang-scheduler-details.html

View Slide

CodeEurope 2017
1. Everything is a process
2. Processes are strongly isolated
3. Process creation and destruction is a lightweight operation
4. Message passing is the only way for processes to interact
5. Processes have unique names
6. If you know the name of a process you can send it a message
7. Processes share no resources
8. Error handling is non-local
9. Processes do what they are supposed to do or fail
http://erlang.org/download/armstrong_thesis_2003.pdf

View Slide

CodeEurope 2017
JAVA
Write once, run anywhere

View Slide

CodeEurope 2017
ERLANG
https://www.youtube.com/watch?v=u41GEwIq2mE&t=3m59s
Write once, run forever

View Slide

CodeEurope 2017
Garbage Collection

View Slide

CodeEurope 2017
http://www.erlang-factory.com/static/upload/media/1394350183453526efsf2014whatsappscaling.pdf

View Slide

CodeEurope 2017
https://www.youtube.com/watch?v=5SbWapbXhKo

View Slide

CodeEurope 2017
Elixir?

View Slide

CodeEurope 2017
1 + 1 #!=> 2
{:ok, "hello"}
[1, 2, 3] = [head | tail]
head #!=> 1
tail #!=> [2,3]

View Slide

CodeEurope 2017
defmodule Math do
def add(x, y), do: x + y
def subtract(x, y) do
x - y
end
end

View Slide

CodeEurope 2017
defmodule Collection do
def sum([x, y, z]), do: x + y + z
end

View Slide

CodeEurope 2017
iex(2)> Collection.sum([1, 2, 3])
6
iex(3)> Collection.sum([1, 2, 3, 4])
!** (FunctionClauseError) no function clause matching in Collection.sum/1
iex:2: Collection.sum([1, 2, 3, 4])

View Slide

CodeEurope 2017
Pipe Operator

View Slide

CodeEurope 2017
foo(bar(baz(123), 456), 789)

View Slide

CodeEurope 2017
123
!|> baz()
!|> bar(456)
!|> foo(123)

View Slide

CodeEurope 2017
def serialize(user) do
user
!|> serialize_attributes()
!|> serialize_relationships()
!|> to_json()
end

View Slide

CodeEurope 2017
def validate_presence(input) do
if input !== "" do
"Can't be blank"
end
input
end

View Slide

CodeEurope 2017
Pattern Matching

View Slide

CodeEurope 2017
def validate_presence(""), do: {:err, "Can't be blank"}
def validate_presence(input), do: input

View Slide

CodeEurope 2017
thing = %{foo: "bar", baz: 123}
%{foo: foo, baz: baz} = thing;
IO.inspect(foo) # "bar"
IO.inspect(baz) # 123

View Slide

CodeEurope 2017
thing = %{foo: "bar", baz: 123}
%{foo: foo, baz: baz, qux: qux} = thing;
!** (MatchError) no match of right hand side value: %{baz: 123, foo: "bar"}

View Slide

CodeEurope 2017
https://fsharpforfunandprofit.com/rop/

View Slide

CodeEurope 2017
def save({:ok, %Response{body: %{"data" !=> _} = body, status_code: 200}}, schema_atom),
do: do_save(body, schema_atom)
defp do_save(body, schema_atom) do
with {:ok, parsed} !<- parse_jsonapi(body),
{:ok, normalized} !<- normalize(parsed),
{:ok, _} !<- insert_all(normalized, schema_atom),
do: {:ok, schema_atom}
end

View Slide

CodeEurope 2017
with :ok !<- validate_guess(game, guess),
{:ok, game} !<- record_guess(game, guess),
{:ok, game, guess} !<- check_move(game, guess),
{:ok, game} !<- update_letters(game, guess),
{:ok, game} !<- update_guess_state(game, :good_guess) do
report(game)
else
{:err, game, :already_guessed} !->
{:ok, game} = update_guess_state(game, :already_guessed)
report(game)
{:err, game, :bad_guess} !->
game
!|> update_guess_state(:bad_guess)
!|> subtract_turn()
!|> report()
end

View Slide

CodeEurope 2017
def transform(%{} = transformed),
do: Enum.reduce(transformed, %{}, &do_transform/2)
defp do_transform({k, {:date, v}}, acc),
do: Map.put(acc, k, to_ecto_datetime(v))
defp do_transform({k, {:string, v}}, acc),
do: Map.put(acc, k, Kernel.to_string(v))
defp do_transform({k, {:foreign_key, v}}, acc),
do: Map.put(acc, k, String.downcase(Kernel.to_string(v)))
defp do_transform({k, {_, v}}, acc),
do: Map.put(acc, k, v)
defp do_transform({_, _}, _), do: {:err, :no_transformer}

View Slide

CodeEurope 2017
defmodule ID3Parser do
def parse(file_name) do
case File.read(file_name) do
{:ok, binary} !->
mp3_byte_size = (byte_size(binary) - 128)
!<< _ !:: binary-size(mp3_byte_size), id3_tag !:: binary !>> = binary
!<< "TAG",
title !:: binary-size(30),
artist !:: binary-size(30),
album !:: binary-size(30),
year !:: binary-size(4),
comment !:: binary-size(30),
_rest !:: binary !>> = id3_tag
IO.puts {title, artist, album, year, comment}
_ !->
IO.puts "Couldn't open !#{file_name}"
end
end
end
ID3Parser.parse("sample.mp3")
http://benjamintan.io/blog/2014/06/10/elixir-bit-syntax-and-id3/

View Slide

CodeEurope 2017
Macros

View Slide

CodeEurope 2017
https://www.youtube.com/watch?v=zlZdOwAWcbo

View Slide

CodeEurope 2017
quote do: 1!..100 !|> Enum.map(&(&1 * &1))
{:!|>, [context: Elixir, import: Kernel],
[{:!.., [context: Elixir, import: Kernel], [1, 100]},
{{:., [], [{:!__aliases!__, [alias: false], [:Enum]}, :map]}, [],
[{:&, [],
[{:*, [context: Elixir, import: Kernel],
[{:&, [], [1]}, {:&, [], [1]}]}]}]}]}

View Slide

CodeEurope 2017
BEAM
ELIXIR'S STRONGEST ASSET

View Slide

CodeEurope 2017
defmodule SimpleQueue do
use GenServer
def init(state), do: {:ok, state}
# GenServer API
def handle_call(:dequeue, _from, [value|state]) do
{:reply, value, state}
end
def handle_call(:dequeue, _from, []), do: {:reply, nil, []}
def handle_call(:queue, _from, state), do: {:reply, state, state}
def handle_cast({:enqueue, value}, state) do
{:noreply, state !++ [value]}
end
# Client API
def start_link(state, opts !\\ []) do
GenServer.start_link(!__MODULE!__, state, opts)
end
def queue, do: GenServer.call(!__MODULE!__, :queue)
def enqueue(value), do: GenServer.cast(!__MODULE!__, {:enqueue, value})
def dequeue, do: GenServer.call(!__MODULE!__, :dequeue)
end

View Slide

CodeEurope 2017
iex(1)> SimpleQueue.start_link([1, 2, 3])
{:ok, #PID<0.385.0>}
iex(2)> SimpleQueue.dequeue
1
iex(3)> SimpleQueue.dequeue
2
iex(4)> SimpleQueue.queue
[3]
iex(5)> SimpleQueue.enqueue(5)
:ok
[3, 5]

View Slide

CodeEurope 2017
iex(1)> {:ok, pid} = SimpleQueue.start_link([1, 2, 3])
{:ok, #PID<0.378.0>}
iex(2)> GenServer.call(pid, :unknown)
!** (EXIT from #PID<0.374.0>) an exception was raised:
!** (FunctionClauseError) no function clause matching in SimpleQueue.handle_call/3
(data_warehouse) lib/queue.ex:7: SimpleQueue.handle_call(:unknown, {#PID<0.374.0>, #Reference<0.0.4.2238>}, [1, 2,
3])
(stdlib) gen_server.erl:615: :gen_server.try_handle_call/4
(stdlib) gen_server.erl:647: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3

View Slide

CodeEurope 2017
iex(2)> GenServer.call(pid, :queue)
warning: variable "pid" does not exist and is being expanded to
"pid()", please use parentheses to remove the ambiguity or
change the variable name
iex:2
!** (CompileError) iex:2: undefined function pid/0
(stdlib) lists.erl:1354: :lists.mapfoldl/3

View Slide

CodeEurope 2017
defmodule SimpleSupervisor do
use Supervisor
def start_link do
Supervisor.start_link(!__MODULE!__, :ok)
end
def init(:ok) do
children = [
worker(SimpleQueue, [[], [name: SimpleQueue]])
]
supervise(children, strategy: :one_for_one)
end
end

View Slide

CodeEurope 2017
iex(1)> {:ok, sup_pid} = SimpleSupervisor.start_link()
{:ok, #PID<0.377.0>}
iex(2)> SimpleQueue.enqueue(1)
:ok
[1]
iex(4)> GenServer.call(SimpleQueue, :unknown)
[error] GenServer SimpleQueue terminating
!** (FunctionClauseError) no function clause matching in SimpleQueue.handle_call/3
(data_warehouse) lib/simple_queue.ex:7: SimpleQueue.handle_call(:unknown, {#PID<0.374.0>, #Reference<0.0.6.936>}, [1])
(stdlib) gen_server.erl:615: :gen_server.try_handle_call/4
(stdlib) gen_server.erl:647: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Last message: :unknown
State: [1]
(elixir) lib/gen_server.ex:737: GenServer.call/3
[]

View Slide

CodeEurope 2017
one web-server handling
2 millions sessions
✅ 2 million web-servers
handling one session each

View Slide

CodeEurope 2017
Distributed Elixir

View Slide

CodeEurope 2017

View Slide

CodeEurope 2017
iex(one@localhost)1> Node.list()
[]

View Slide

CodeEurope 2017
iex(one@localhost)2> Node.connect :two@localhost
true
iex(one@localhost)3> Node.list()
[:two@localhost]

View Slide

CodeEurope 2017
1 2

View Slide

CodeEurope 2017
iex(three@localhost)1> Node.connect :one@localhost
true
iex(three@localhost)2> Node.list()
[:one@localhost, :two@localhost]

View Slide

CodeEurope 2017
1 2
3

View Slide

CodeEurope 2017

View Slide

CodeEurope 2017
iex(two@localhost)1> :rpc.multicall(IO, :inspect, ["Hello world!"])
"Hello world!"
"Hello world!"
"Hello world!"
{["Hello world!", "Hello world!", "Hello world!"], []}

View Slide

CodeEurope 2017
iex(two@localhost)10> System.get_pid()
"95943"
iex(two@localhost)11> :rpc.multicall(System, :get_pid, [])
{["95943", "95810", "96120"], []}

View Slide

CodeEurope 2017
iex(two@localhost)20> pid = :rpc.async_call(:one@localhost, Enum, :map, [[1, 2, 3], &(&1 *
&1)])
#PID<0.132.0>
iex(two@localhost)21> :rpc.yield(pid)
[1, 4, 9]

View Slide

CodeEurope 2017
iex(two@localhost)1> Node.list
[]
iex(two@localhost)2> :net_adm.world_list([:localhost]) !|> Enum.map(&Node.connect/1)
[true, true, true]
iex(two@localhost)3> Node.list
[:one@localhost, :three@localhost]

View Slide

CodeEurope 2017

View Slide

CodeEurope 2017
Monitoring

View Slide

CodeEurope 2017
iex(1)> :observer.start()
:ok

View Slide

CodeEurope 2017

View Slide

CodeEurope 2017
The Outside World

View Slide

CodeEurope 2017

View Slide

CodeEurope 2017
I. Why Elixir?

View Slide

CodeEurope 2017
Umbrella Applications

View Slide

CodeEurope 2017
https://martinfowler.com/bliki/BoundedContext.html

View Slide

CodeEurope 2017
Peedy
STAMP
WATERMARK
QUEUE
WEB
TONIQ
REDIS
HTTP POST Watermarked PDF
PDFKIT.JS
ERLGUTEN
PDFTK
200 OK HTTP POST to Callback URL
http://localhost:4000/api/v1/[email protected]&callback_url=http://localhost:4000/api/v1/dev/null
https://github.com/poteto/peedy

View Slide

CodeEurope 2017
REST API to the
umbrella
PEEDY: WEB

View Slide

CodeEurope 2017
Router
PEEDY: WEB

View Slide

CodeEurope 2017
defmodule PeedyWeb.Router do
use PeedyWeb.Web, :router
pipeline :api do
plug :accepts, ["json"]
plug :fetch_session
end
scope "/", PeedyWeb do
get "/healthcheck", HealthcheckController, :index
end
scope "/api", PeedyWeb do
pipe_through :api
scope "/v1", Api.V1 do
resources "/documents", DocumentController, only: [:show, :create]
post "/dev/null", DevNullController, :create
end
end
end

View Slide

CodeEurope 2017
Document controller
PEEDY: WEB

View Slide

CodeEurope 2017
defmodule PeedyWeb.Api.V1.DocumentController do
use PeedyWeb.Web, :controller
alias Stamper.{Repo, Document}
alias PeedyWeb.ErrorView
@callback_client Application.get_env(:peedy_web, :callback_client)
@whitelisted_content_types MapSet.new(~w(application/pdf))
@document_headers %{"Content-Type" !=> "multipart/form-data"}
def create(conn, %{"watermark" !=> watermark_text, "callback_url" !=> callback_url} = params) do
# get PDFs from post, enqueue watermark job
end
def create(conn, _) do
conn
!|> halt()
!|> put_status(:bad_request)
!|> render(ErrorView, "400.json", %{detail: "Watermark and/or callback_url are missing"})
end
end

View Slide

CodeEurope 2017
defp create_callback(%Document{id: id, output: output}, filename, callback_url) do
file_path = System.tmp_dir!() !<> Zarex.sanitize(filename)
File.write!(file_path, output)
@callback_client.do_callback(callback_url, %{file: file_path, id: id}, @document_headers)
end
defp whitelist_content_type(uploads) when is_list(uploads) do
Enum.filter(uploads, fn %Plug.Upload{content_type: content_type} !->
MapSet.member?(@whitelisted_content_types, content_type)
end)
end

View Slide

CodeEurope 2017
def create(conn, %{"watermark" !=> watermark_text, "callback_url" !=> callback_url} = params) do
params
!|> Map.drop(["watermark", "callback_url"])
!|> Map.values()
!|> whitelist_content_type()
!|> case do
[] !->
conn
!|> halt()
!|> put_status(:bad_request)
!|> render(ErrorView, "400.json", %{detail: "No PDFs received"})
files !->
watermark = Watermarker.create(watermark_text)
Enum.map(files, fn %Plug.Upload{filename: filename, path: path} !->
Peedy.F.watermark_with(watermark, &(create_callback(&1, filename, callback_url)), input_path: path, ephemeral!?: false)
end)
send_resp(conn, :ok, "OK")
end
end

View Slide

CodeEurope 2017
Callback client
PEEDY: WEB

View Slide

CodeEurope 2017
defmodule PeedyWeb.CallbackClient do
use Toniq.Worker, max_concurrency: Application.get_env(:peedy_web, :max_concurrency)
alias Stamper.{Repo, Document}
@adapter HTTPoison
def do_callback(callback_url, %{file: file, id: id}, headers, adapter !\\ @adapter) do
Toniq.enqueue(PeedyWeb.CallbackClient,
callback_url: callback_url,
file: file,
id: id,
headers: headers,
adapter: adapter)
end
def perform(callback_url: callback_url, file: file, id: id, headers: headers, adapter: adapter) do
adapter.post(callback_url, multipart_encode([file: file, id: id]), headers)
Repo.get!(Document, id)
end
defp multipart_encode(body) do
body = Enum.map(body, fn {:file, path} !-> {:file, path}
{key, value} !-> {to_string(key), value} end)
{:multipart, body}
end
end

View Slide

CodeEurope 2017
Process watermark
jobs asynchronously
PEEDY: QUEUE

View Slide

CodeEurope 2017

View Slide

CodeEurope 2017
defmodule Peedy.F do
require Logger
use Toniq.Worker, max_concurrency: Application.get_env(:f, :max_concurrency)
alias Stamper.Document
alias Watermarker.Watermark
@doc """
Queues the watermarking of a document with text and invokes the callback
when complete.
"""
@spec watermark_with(Watermark.t, callback !:: (!!... !-> any()), opts !:: [input_path: String.t, ephemeral!?: boolean()]) !:: %{}
def watermark_with(%Watermark{} = watermark, callback, input_path: input_path, ephemeral!?: ephemeral?) when is_function(callback) do
# enqueue job with callback
end
@doc """
Queues the watermarking of a document with text. Uses the default callback.
"""
@spec watermark_with(Watermark.t, opts !:: [input_path: String.t, ephemeral!?: boolean()]) !:: %{}
def watermark_with(%Watermark{} = watermark, input_path: input_path, ephemeral!?: ephemeral?) do
# enqueue job with default callback
end
end

View Slide

CodeEurope 2017
def perform(watermark: watermark, input_path: input_path, ephemeral!?: true, callback: callback),
do: do_perform(watermark, input_path, true, callback)
def perform(watermark: watermark, input_path: input_path, ephemeral!?: false, callback: callback),
do: do_perform(watermark, input_path, false, callback)
defp do_perform(%Watermark{} = watermark, input_path, ephemeral?, callback) do
%Document{} =
watermark
!|> Stamper.stamp_with(input_path: input_path, ephemeral!?: ephemeral?)
!|> callback.()
end

View Slide

CodeEurope 2017
Create watermark
layer
PEEDY: WATERMARK

View Slide

CodeEurope 2017
def create(text, strategy: :pdfkit) when is_binary(text),
do: Pdfkit.new(text)
def create(text, strategy: :erlguten) when is_binary(text),
do: Erlguten.new(text)

View Slide

CodeEurope 2017
defmodule Watermarker.WatermarkerBehaviour do
alias Watermarker.Watermark
@callback new(text !:: String.t) !:: Watermark.t
end

View Slide

CodeEurope 2017
defmodule Watermarker.Strategies.Pdfkit do
@moduledoc """
Create a watermark by using a PDFKit.
"""
require Logger
alias Watermarker.{Watermark,Repo}
@behaviour Watermarker.WatermarkerBehaviour
@nodejs Application.get_env(:watermarker, :executables)[:nodejs]
@script_name Application.get_env(:watermarker, :executables)[:pdfkit]
def new(text) when is_binary(text) do
case Repo.get_by(Watermark, input: text) do
%Watermark{} = watermark !-> watermark
nil !->
sanitized_text = Zarex.sanitize(text)
output_path = System.tmp_dir!() !<> "!#{sanitized_text}.pdf"
output =
text
!|> to_pdf(output_path: output_path)
!|> File.read!()
%Watermark{}
!|> Watermark.changeset(%{input: text, output: output})
!|> Repo.insert!()
end
end
end

View Slide

CodeEurope 2017

View Slide

CodeEurope 2017
def to_pdf(text, output_path: output_path) when is_binary(text) do
File.touch!(output_path)
%Porcelain.Result{err: nil, out: out, status: 0} = Porcelain.exec(@nodejs, [@script_name, "!--text", text, "!--output", output_path])
out
end

View Slide

CodeEurope 2017
Stamp watermark
layer onto PDF
PEEDY: STAMP

View Slide

CodeEurope 2017
def stamp_with(%Watermark{id: id} = watermark, input_path: input_path, ephemeral!?: false) do
{microseconds, document} = :timer.tc(fn !->
output_path = stamp_document(watermark, input_path)
%Document{}
!|> Document.changeset(%{
input: File.read!(input_path),
output: File.read!(output_path),
stamp_id: id})
!|> Repo.insert!()
end)
Logger.info("Stamped !#{document.id} in !#{microseconds / 1_000}ms")
document
end

View Slide

CodeEurope 2017

View Slide

CodeEurope 2017
defp stamp_document(%Watermark{} = watermark, input_path) do
output_path = System.tmp_dir!() !<> "!#{Zarex.sanitize(random_string())}.pdf"
stamp_path = stamp_path(watermark)
if File.exists?(input_path) do
%Porcelain.Result{err: nil, out: "", status: 0} = Porcelain.exec(@pdftk, [input_path, "multistamp", stamp_path, "output", output_path])
output_path
else
raise "Input at !#{input_path} does not exist"
end
end

View Slide

CodeEurope 2017
WEB
ACCOUNTING
PRODUCT SHOPPING PAYMENTS DELIVERY
O'rly Bookstore
SESSION
CART
CHECKOUT
INVENTORY
LEDGER
REPORTS
REVIEWS
PRICES
METADATA
PROCESSING
FRAUD
REFUNDS WATERMARK PDF
SHIPPING
RETURNS

View Slide

CodeEurope 2017
What the business
does
BOUNDED CONTEXTS: DOMAIN

View Slide

CodeEurope 2017
Smaller units of the
business
BOUNDED CONTEXTS: SUBDOMAIN

View Slide

CodeEurope 2017
I. Why Elixir?

View Slide

CodeEurope 2017
Rate Limiting

View Slide

CodeEurope 2017
Simple Queue

View Slide

CodeEurope 2017
defmodule Queue do
@strategy Queue.SimpleQueue
def enqueue(message, strategy !\\ @strategy),
do: strategy.enqueue(message)
def enqueue_all(messages, strategy !\\ @strategy) when is_list(messages) do
messages
!|> Stream.map(&strategy.enqueue/1)
!|> Enum.to_list()
end
def queue(strategy !\\ @strategy),
do: strategy.queue()
def dequeue(count !\\ 1, strategy !\\ @strategy) when is_integer(count),
do: strategy.dequeue(count)
end

View Slide

CodeEurope 2017
defmodule Queue.Application do
@moduledoc false
use Application
def start(_type, _args) do
import Supervisor.Spec, warn: false
children = [
worker(Queue.SimpleQueue.Supervisor, []),
worker(Queue.Pipeline.Producer, []),
worker(Queue.Pipeline.Consumer, [])
]
opts = [strategy: :one_for_one, name: Queue.Supervisor]
Supervisor.start_link(children, opts)
end
end

View Slide

CodeEurope 2017
defmodule Queue.Pipeline.Producer do
use GenStage
require Logger
alias Queue.SimpleQueue
def start_link(state !\\ 0) do
GenStage.start_link(!__MODULE!__, state, name: !__MODULE!__)
end
def init(state) do
{:producer, state}
end
def handle_cast(:check_messages, 0), do: {:noreply, [], 0}
def handle_cast(:check_messages, state) when is_integer(state) do
messages = SimpleQueue.dequeue(state)
GenStage.cast(!__MODULE!__, :check_messages)
{:noreply, messages, state - Enum.count(messages)}
end
def handle_demand(demand, state) when is_integer(state) do
GenStage.cast(!__MODULE!__, :check_messages)
{:noreply, [], demand + state}
end
end

View Slide

CodeEurope 2017
defmodule Queue.Pipeline.Processor do
require Logger
@interval 5_000
def start_link(message) do
{:ok, _} = Task.start_link(!__MODULE!__, :process_message, [message])
end
def process_message(message) do
:timer.sleep(@interval)
message
!|> inspect()
!|> Logger.info()
end
end

View Slide

CodeEurope 2017
defmodule Queue.Pipeline.Consumer do
use ConsumerSupervisor
def start_link do
ConsumerSupervisor.start_link(!__MODULE!__, :ok)
end
def init(:ok) do
children = [
worker(Queue.Pipeline.Processor, [], restart: :temporary)
]
{:ok, children, strategy: :one_for_one, subscribe_to: [
{Queue.Pipeline.Producer, max_demand: 10, min_demand: 1}
]}
end
end

View Slide

CodeEurope 2017

View Slide

CodeEurope 2017
Circuit Breakers

View Slide

CodeEurope 2017

View Slide

CodeEurope 2017
defmodule SomeApi do
@fuse_name !__MODULE!__
@fuse_options {
{:standard, 2, 10_000},
{:reset, 60_000}
}
def init do
install_fuse()
end
def fetch_all do
case :fuse.ask(@fuse_name, :sync) do
:ok !->
# continue normally
:blown !->
# handle failure so other applications that depend on this one won't
# break
end
end
defp do_fetch_all do
case SomeApi.Adapter.get("/some/path") do
{:ok, resp} !->
# do stuff with data
{:error, _} !->
:fuse.melt(@fuse_name)
end
end
defp install_fuse, do: :fuse.install(@fuse_name, @fuse_options)
end

View Slide

CodeEurope 2017
defp do_fetch_all do
case SomeApi.Adapter.get("/some/path") do
{:ok, resp} !->
# do stuff with data
{:error, _} !->
:fuse.melt(@fuse_name)
end
end

View Slide

CodeEurope 2017
def fetch_all do
case :fuse.ask(@fuse_name, :sync) do
:ok !->
# continue normally
:blown !->
# handle failure so other applications that depend on this one won't
# break
end
end

View Slide

CodeEurope 2017
I. Why Elixir?

View Slide

CodeEurope 2017
Lauren Tan
sugarpirate_
poteto

View Slide

CodeEurope 2017
Cinemagraph by /u/fezzo
Thank you
sugarpirate_
poteto

View Slide

CodeEurope 2017 – Monoliths to Services with Elixir and Phoenix

CodeEurope 2017 – Monoliths to Services with Elixir and Phoenix

Lauren Tan

More Decks by Lauren Tan

Other Decks in Programming

Featured

Transcript