Redesign with Elixir/OTP

Transcript

1. Re-design with Elixir/OTP 2016 - 2017 ImgOut / On the

   fly thumbnail generator microservice using Elixir/OTP. by Mustafa Turan https://github.com/mustafaturan/imgout

2. Summary On the fly thumbnail generator microservice, a web microservice:

   - Basic Approach without Elixir/OTP - Anatomy of an Elixir Process (optional) - Fault Tolerance Systems with Supervision Tree Strategies (optional) - Approach 1 / with Supervision Tree + GenServer - Approach 2 / with Supervision Tree + GenServer + Partial Pooling - Approach 3 / with Supervision Tree + GenServer + Pooling - Metrics - Source Codes - Questions

3. Basic Approach without Elixir/OTP * * Web Based Approach without

   OTP Benefits

4. Building Blocks Thumbnail Microservice For All Languages: Inputs Outputs Image

   identifier (ID) Dimensions (width*, height*) Thumbnail (Binary) Content type (String) Tools and Requirements Sample Elixir Specific Tools Web Server Image conversion library Cache Storage or Image Source(remote/local) Metrics Cowboy Exmagick(Graphmagick NIF package) Cachex, MemcacheClient HttpPoison (Web Client) Metrex (Metrics)

5. General Internal Process Flow server cache (r) storage thumb cache

   (w)

6. Defining Modular & Functional Needs defmodule ImgOut.CacheInterface do @callback read(charlist)

   :: {:ok, binary} | charlist @callback write(charlist, binary) :: {:ok, binary} end defmodule ImgOut.StorageInterface do @callback read({:ok, binary}) @callback read({:error, integer, map}) @callback read(charlist) end defmodule ImgOut.ImageInterface do @callback thumb({:error, integer, map}, any) @callback thumb({:ok, binary}, map) end defmodule MicroserviceController do def generate_thumb(id, %{} = dimensions) do id |> Cache.read |> Storage.read |> Image.thumb(dimensions) end end defmodule AlternativeApproachController do def generate_thumb(id, %{} = dimensions) do id |> Storage.read |> Image.thumb(dimensions) end end

7. Implement CacheService using CacheInterface defmodule ImgOut.CacheInterface do @callback read(charlist) ::

   {:ok, binary} | charlist @callback write(charlist, binary) :: {:ok, binary} end defmodule ImgOut.CacheService do @behaviour ImgOut.CacheInterface def read(key) do response = Memcache.Client.get(key) case response.status do :ok -> {:ok, response.value} _ -> key end end def write(key, val) do Memcache.Client.set(key, val) {:ok, val} end end

8. Sample Directory Structure app - lib - interfaces - cache_interface.ex

   - ... - services - cache_service.ex - … - imgout.ex

9. General Internal Process Flow server cache (r) storage thumb cache

   (w)

10. How to Make Cache.write Async? server cache (r) storage thumb

    cache (w) Task.start(..) defmodule ImgOut.CacheService do ... # way 1: to make async write to cache def write(key, val) do Task.start(fn -> Memcache.Client.set(key, val) end) {:ok, val} end end

11. Anatomy of an Elixir Process What is an Erlang/Elixir Process?

12. An actor (Elixir/Erlang Process) STATE Mailbox CALCULATION FUNCTIONS (MSG LISTENERS)

    @mustafaturan

13. Erlang Virtual Machine @mustafaturan OS Process (1) Process (2) Process

    (3) …. Process (n) Erlang VM -- pid 109 -- pid 206 -- pid 3114 -- ... STATE Mailbox CALCULATION FUNCTIONS (MSG LISTENERS) (pid 109)

14. Fault Tolerance Systems Erlang/Elixir Supervision Tree Strategies

15. Supervision Tree Strategies :one_for_one S W W W S W

    W W S W W W if one worker dies respawn it down signal restart child @mustafaturan

16. Supervision Tree Strategies :one_for_all @mustafaturan S W W W S

    W W W S W W W if one worker dies supervisor respawns all down signal restart all S W W supervisor kills rest of the workers

17. Supervision Tree Strategies :rest_for_one @mustafaturan S W 2 W 3

    W 1 S W 2 W 3 W 1 S W 2 W 3 W 1 if one worker dies supervisor respawns all killed down signal restart all killed workers supervisor kills rest of the workers with start order (not W1) Note: Assumed start order of workers are like W1, W2, W3 and W4 W 4 W 4 W 4 S W 3 W 1 W 4 W 4

18. Supervision Tree Strategies :simple_one_for_one Same as :one_for_one - Needs to

    implement Supervision.Spec - You need to specify only one entry for a child - Every child spawned by this strategy is same kind of process, can not be mixed.

19. With Named GenServer & Supervision Tree * Creating Elixir Processes

    with GenServer** and Supervising ** Process discovery with ‘name’ option

20. Building Our Supervision Tree A (S) Srv (S) Im (S)

    Str (S) Ch (S) Str (w) Im (w) Srv (w) Ch (w) defmodule ImgOut do use Application def start(_type, _args) do import Supervisor.Spec, warn: false children = [ supervisor(ImgOut.WebServerSupervisor, []), supervisor(ImgOut.ImageSupervisor, []), supervisor(ImgOut.StorageSupervisor, []) ] opts = [strategy: :one_for_one, name: ImgOut.Supervisor] Supervisor.start_link(children, opts) end end

21. Sample Code For CacheSupervisor ... @doc false def init([]) do

    children = [ worker(ImgOut.CacheWorker, []) ] opts = [strategy: :one_for_one, name: __MODULE__] supervise(children, opts) end end defmodule ImgOut.CacheSupervisor do use Supervisor @doc false def start_link, do: Supervisor.start_link(__MODULE__, [], name: __MODULE__) ...

22. Sample Code For CacheWorker ... def init(_opts), do: {:ok, []}

    ## Private API @doc false def handle_call({:read, key}, _from, state), do: {:reply, ImgOut.CacheService.read(key), state} def handle_cast({:write, key, val}, state) do ImgOut.CacheService.write(key, val) {:noreply, state} end end defmodule ImgOut.CacheWorker do use GenServer ## Public API def read(key), do: GenServer.call(__MODULE__, {:read, key}) def write(key, val) do GenServer.cast(__MODULE__, {:write, key, val}) {:ok, val} end def start_link, do: GenServer.start_link(__MODULE__, [], name: __MODULE__) ...

23. Sample Directory Structure app - lib - interfaces - cache_interface.ex

    - ... - services - cache_service.ex - ... - supervisors - cache_supervisor.ex - ... - workers - cache_worker.ex - ...

24. How is the Flow server cache (r) storage thumb cache

    (w)

25. Problem: Long Running Processes server storage + cache thumb timeouts

    Thumbnail generation - A calculation - Takes too much time to process

26. Solution: Spawning More Workers On Demand … def start_link, do:

    GenServer.start_link(__MODULE__, [], name: __MODULE__) … def thumb({:ok, img}, dimensions), do: GenServer.call(__MODULE__, {:thumb, {:ok, img}, dimensions}) … def handle_call({:thumb, {:ok, img}, dimensions}, _from, state) do data = ImgOut.ImageService.thumb({:ok, img}, dimensions) {:reply, data, state} end … Cons: Spawn 1 Process Per GenServer Pros: No need to store pid to call process funcs

27. Solution: Spawning More Workers On Demand Possible Solution to spawn

    more workers on GenServer: - Process Registry - We can save pid on creation and deregister on exit - Too manual work - Reinventing wheels(special for this service) - Gproc Package - Supports pooling - Has advanced techniques to register, deregister, monitor etc... - Poolboy Package - Supports pooling - Easy

28. With Partial Pooled GenServer & Supervision Tree * Creating Elixir

    Processes with GenServer** and Supervising ** Process discovery with worker pool option (poolboy)

29. Poolboy Configuration For ImageWorker # prod.exs config :imgout, gm_pool_size: (System.get_env("GM_POOL_SIZE")

    || "25") |> String.to_integer, gm_pool_max_overflow: (System.get_env("GM_POOL_MAX_OVERFLOW") || "0") |> String.to_integer, gm_timeout: (System.get_env("GM_TIMEOUT") || "5000") |> String.to_integer # dev.exs, test.exs config :imgout, gm_pool_size: 25, gm_pool_max_overflow: 0, gm_timeout: 5000

30. Sample Code For ImageSupervisor with Poolboy def init([]) do worker_pool_options

    = [ name: {:local, :gm_worker_pool}, worker_module: ImgOut.ImageWorker, size: @pool_size, max_overflow: @pool_max_overflow ] children = [ :poolboy.child_spec(:gm_worker_pool, worker_pool_options, []) ] opts = [strategy: :one_for_one, name: __MODULE__] supervise(children, opts) end end defmodule ImgOut.ImageSupervisor do use Supervisor @pool_size Application.get_env(:imgout, :gm_pool_size) @pool_max_overflow Application.get_env(:imgout, :gm_pool_max_overflow) def start_link, do: Supervisor.start_link(__MODULE__, [], name: __MODULE__) …

31. Sample Code For ImageWorker with Poolboy def start_link(_opts), do: GenServer.start_link(__MODULE__,

    :ok, []) @doc false def init(_opts) do {:ok, []} end def handle_call({:thumb, {:ok, img}, dimensions}, _from, state) do data = ImgOut.ImageService.thumb({:ok, img}, dimensions) {:reply, data, state} end end defmodule ImgOut.ImageWorker do use GenServer @behaviour ImgOut.ImageInterface @timeout Application.get_env(:imgout, :gm_timeout) def thumb({:ok, img}, dimensions) do :poolboy.transaction(:gm_worker_pool, fn(worker) -> GenServer.call(worker, {:thumb, {:ok, img}, dimensions}, @timeout) end) end def thumb({:error, status, reason}, _), do: {:error, status, reason} …

32. Solution: Spawned (n)Thumb Process server storage + cache thumb no

    timeouts Spawned Multiple ImageWorker(s) with Poolboy - Process Registry handled by poolboy - We can change max, min spawned processes thumb thumb thumb

33. Problem: Too many request to storage and cache worker server

    storage + cache thumb no timeouts Since we solved the timeout problem for Thumbnail processor - Now storage and cache worker getting too many request - but not processing that fast with 1 instance! thumb thumb thumb timeouts

34. Solution: Spawning More Workers On Demand Possible Solution to spawn

    more workers on GenServer: - Process Registry - We can save pid on creation and deregister on exit - Too manual work - Reinventing wheels(special for this service) - Gproc Package - Supports pooling - Has advanced techniques to register, deregister, monitor etc... - Poolboy Package - Supports pooling - Easy

35. With Fully Pooled GenServer & Supervision Tree * Creating Elixir

    Processes with GenServer** and Supervising ** Process discovery with worker pool option (poolboy)

36. Why Prefer Pooling Against Free Spawning? Memcache - Connection limit

    - If you hit connection limit, you can’t get positive response Storage - Remote storage servers might have some limitation on requests - Req/per second - If you hit more, you will get error from remote

37. Metrics A very useful for microservices and system health checks,

    determining bottlenecks

38. Tracking Heartbeats of Your Elixir Processes Metrex Package - Creating

    new metric dynamically - Incrementing metric - TTL - Dumping metric data - Init and exit hooks Results: - 3260 req/min on Heroku Free Dyno - http://bit.ly/2bYRnpp config :metrex, counter: ["all"], meters: ["cache", "storage", "thumb"], ttl: 900 # Dynamically create a metric Metrex.start_meter("pageviews") # Increment a meter by 1 Metrex.Meter.increment("pageviews") # Increment a meter by x(number) Metrex.Meter.increment("pageviews", 5)

39. Tips & Source Code

40. Libraries Metrex - Has implementation for counter and meter patterns

    Exmagick - NIF package which means you can spawn graphicsmagick as Elixir Process Poolboy Cowboy Plug

41. Source Github: https://github.com/mustafaturan/imgout (Heroku Deploy Button available / Has 3

    branches for all 3 approaches) https://github.com/mustafaturan/metrex

42. QUESTIONS

43. THANK YOU