Redesign with Elixir/OTP

Transcript

1. Re-design with Elixir/OTP 2016 - 2017 ImgOut / On the fly thumbnail generator microservice using Elixir/OTP. by Mustafa Turan https://github.com/mustafaturan/imgout
2. Summary On the fly thumbnail generator microservice, a web microservice: - Basic Approach without Elixir/OTP - Anatomy of an Elixir Process (optional) - Fault Tolerance Systems with Supervision Tree Strategies (optional) - Approach 1 / with Supervision Tree + GenServer - Approach 2 / with Supervision Tree + GenServer + Partial Pooling - Approach 3 / with Supervision Tree + GenServer + Pooling - Metrics - Source Codes - Questions
3. Basic Approach without Elixir/OTP * * Web Based Approach without OTP Benefits
4. Building Blocks Thumbnail Microservice For All Languages: Inputs Outputs Image identifier (ID) Dimensions (width*, height*) Thumbnail (Binary) Content type (String) Tools and Requirements Sample Elixir Specific Tools Web Server Image conversion library Cache Storage or Image Source(remote/local) Metrics Cowboy Exmagick(Graphmagick NIF package) Cachex, MemcacheClient HttpPoison (Web Client) Metrex (Metrics)
5. General Internal Process Flow server cache (r) storage thumb cache (w)
6. Defining Modular & Functional Needs defmodule ImgOut.CacheInterface do @callback read(charlist) :: {:ok, binary} | charlist @callback write(charlist, binary) :: {:ok, binary} end defmodule ImgOut.StorageInterface do @callback read({:ok, binary}) @callback read({:error, integer, map}) @callback read(charlist) end defmodule ImgOut.ImageInterface do @callback thumb({:error, integer, map}, any) @callback thumb({:ok, binary}, map) end defmodule MicroserviceController do def generate_thumb(id, %{} = dimensions) do id |> Cache.read |> Storage.read |> Image.thumb(dimensions) end end defmodule AlternativeApproachController do def generate_thumb(id, %{} = dimensions) do id |> Storage.read |> Image.thumb(dimensions) end end
7. Implement CacheService using CacheInterface defmodule ImgOut.CacheInterface do @callback read(charlist) :: {:ok, binary} | charlist @callback write(charlist, binary) :: {:ok, binary} end defmodule ImgOut.CacheService do @behaviour ImgOut.CacheInterface def read(key) do response = Memcache.Client.get(key) case response.status do :ok -> {:ok, response.value} _ -> key end end def write(key, val) do Memcache.Client.set(key, val) {:ok, val} end end
8. Sample Directory Structure app - lib - interfaces - cache_interface.ex - ... - services - cache_service.ex - … - imgout.ex
9. General Internal Process Flow server cache (r) storage thumb cache (w)
10. How to Make Cache.write Async? server cache (r) storage thumb cache (w) Task.start(..) defmodule ImgOut.CacheService do ... # way 1: to make async write to cache def write(key, val) do Task.start(fn -> Memcache.Client.set(key, val) end) {:ok, val} end end
11. Anatomy of an Elixir Process What is an Erlang/Elixir Process?
12. An actor (Elixir/Erlang Process) STATE Mailbox CALCULATION FUNCTIONS (MSG LISTENERS) @mustafaturan
13. Erlang Virtual Machine @mustafaturan OS Process (1) Process (2) Process (3) …. Process (n) Erlang VM -- pid 109 -- pid 206 -- pid 3114 -- ... STATE Mailbox CALCULATION FUNCTIONS (MSG LISTENERS) (pid 109)
14. Fault Tolerance Systems Erlang/Elixir Supervision Tree Strategies
15. Supervision Tree Strategies :one_for_one S W W W S W W W S W W W if one worker dies respawn it down signal restart child @mustafaturan
16. Supervision Tree Strategies :one_for_all @mustafaturan S W W W S W W W S W W W if one worker dies supervisor respawns all down signal restart all S W W supervisor kills rest of the workers
17. Supervision Tree Strategies :rest_for_one @mustafaturan S W 2 W 3 W 1 S W 2 W 3 W 1 S W 2 W 3 W 1 if one worker dies supervisor respawns all killed down signal restart all killed workers supervisor kills rest of the workers with start order (not W1) Note: Assumed start order of workers are like W1, W2, W3 and W4 W 4 W 4 W 4 S W 3 W 1 W 4 W 4
18. Supervision Tree Strategies :simple_one_for_one Same as :one_for_one - Needs to implement Supervision.Spec - You need to specify only one entry for a child - Every child spawned by this strategy is same kind of process, can not be mixed.
19. With Named GenServer & Supervision Tree * Creating Elixir Processes with GenServer** and Supervising ** Process discovery with ‘name’ option
20. Building Our Supervision Tree A (S) Srv (S) Im (S) Str (S) Ch (S) Str (w) Im (w) Srv (w) Ch (w) defmodule ImgOut do use Application def start(_type, _args) do import Supervisor.Spec, warn: false children = [ supervisor(ImgOut.WebServerSupervisor, []), supervisor(ImgOut.ImageSupervisor, []), supervisor(ImgOut.StorageSupervisor, []) ] opts = [strategy: :one_for_one, name: ImgOut.Supervisor] Supervisor.start_link(children, opts) end end
21. Sample Code For CacheSupervisor ... @doc false def init([]) do children = [ worker(ImgOut.CacheWorker, []) ] opts = [strategy: :one_for_one, name: __MODULE__] supervise(children, opts) end end defmodule ImgOut.CacheSupervisor do use Supervisor @doc false def start_link, do: Supervisor.start_link(__MODULE__, [], name: __MODULE__) ...
22. Sample Code For CacheWorker ... def init(_opts), do: {:ok, []} ## Private API @doc false def handle_call({:read, key}, _from, state), do: {:reply, ImgOut.CacheService.read(key), state} def handle_cast({:write, key, val}, state) do ImgOut.CacheService.write(key, val) {:noreply, state} end end defmodule ImgOut.CacheWorker do use GenServer ## Public API def read(key), do: GenServer.call(__MODULE__, {:read, key}) def write(key, val) do GenServer.cast(__MODULE__, {:write, key, val}) {:ok, val} end def start_link, do: GenServer.start_link(__MODULE__, [], name: __MODULE__) ...
23. Sample Directory Structure app - lib - interfaces - cache_interface.ex - ... - services - cache_service.ex - ... - supervisors - cache_supervisor.ex - ... - workers - cache_worker.ex - ...
24. How is the Flow server cache (r) storage thumb cache (w)
25. Problem: Long Running Processes server storage + cache thumb timeouts Thumbnail generation - A calculation - Takes too much time to process
26. Solution: Spawning More Workers On Demand … def start_link, do: GenServer.start_link(__MODULE__, [], name: __MODULE__) … def thumb({:ok, img}, dimensions), do: GenServer.call(__MODULE__, {:thumb, {:ok, img}, dimensions}) … def handle_call({:thumb, {:ok, img}, dimensions}, _from, state) do data = ImgOut.ImageService.thumb({:ok, img}, dimensions) {:reply, data, state} end … Cons: Spawn 1 Process Per GenServer Pros: No need to store pid to call process funcs
27. Solution: Spawning More Workers On Demand Possible Solution to spawn more workers on GenServer: - Process Registry - We can save pid on creation and deregister on exit - Too manual work - Reinventing wheels(special for this service) - Gproc Package - Supports pooling - Has advanced techniques to register, deregister, monitor etc... - Poolboy Package - Supports pooling - Easy
28. With Partial Pooled GenServer & Supervision Tree * Creating Elixir Processes with GenServer** and Supervising ** Process discovery with worker pool option (poolboy)
29. Poolboy Configuration For ImageWorker # prod.exs config :imgout, gm_pool_size: (System.get_env("GM_POOL_SIZE") || "25") |> String.to_integer, gm_pool_max_overflow: (System.get_env("GM_POOL_MAX_OVERFLOW") || "0") |> String.to_integer, gm_timeout: (System.get_env("GM_TIMEOUT") || "5000") |> String.to_integer # dev.exs, test.exs config :imgout, gm_pool_size: 25, gm_pool_max_overflow: 0, gm_timeout: 5000
30. Sample Code For ImageSupervisor with Poolboy def init([]) do worker_pool_options = [ name: {:local, :gm_worker_pool}, worker_module: ImgOut.ImageWorker, size: @pool_size, max_overflow: @pool_max_overflow ] children = [ :poolboy.child_spec(:gm_worker_pool, worker_pool_options, []) ] opts = [strategy: :one_for_one, name: __MODULE__] supervise(children, opts) end end defmodule ImgOut.ImageSupervisor do use Supervisor @pool_size Application.get_env(:imgout, :gm_pool_size) @pool_max_overflow Application.get_env(:imgout, :gm_pool_max_overflow) def start_link, do: Supervisor.start_link(__MODULE__, [], name: __MODULE__) …
31. Sample Code For ImageWorker with Poolboy def start_link(_opts), do: GenServer.start_link(__MODULE__, :ok, []) @doc false def init(_opts) do {:ok, []} end def handle_call({:thumb, {:ok, img}, dimensions}, _from, state) do data = ImgOut.ImageService.thumb({:ok, img}, dimensions) {:reply, data, state} end end defmodule ImgOut.ImageWorker do use GenServer @behaviour ImgOut.ImageInterface @timeout Application.get_env(:imgout, :gm_timeout) def thumb({:ok, img}, dimensions) do :poolboy.transaction(:gm_worker_pool, fn(worker) -> GenServer.call(worker, {:thumb, {:ok, img}, dimensions}, @timeout) end) end def thumb({:error, status, reason}, _), do: {:error, status, reason} …
32. Solution: Spawned (n)Thumb Process server storage + cache thumb no timeouts Spawned Multiple ImageWorker(s) with Poolboy - Process Registry handled by poolboy - We can change max, min spawned processes thumb thumb thumb
33. Problem: Too many request to storage and cache worker server storage + cache thumb no timeouts Since we solved the timeout problem for Thumbnail processor - Now storage and cache worker getting too many request - but not processing that fast with 1 instance! thumb thumb thumb timeouts
34. Solution: Spawning More Workers On Demand Possible Solution to spawn more workers on GenServer: - Process Registry - We can save pid on creation and deregister on exit - Too manual work - Reinventing wheels(special for this service) - Gproc Package - Supports pooling - Has advanced techniques to register, deregister, monitor etc... - Poolboy Package - Supports pooling - Easy
35. With Fully Pooled GenServer & Supervision Tree * Creating Elixir Processes with GenServer** and Supervising ** Process discovery with worker pool option (poolboy)
36. Why Prefer Pooling Against Free Spawning? Memcache - Connection limit - If you hit connection limit, you can’t get positive response Storage - Remote storage servers might have some limitation on requests - Req/per second - If you hit more, you will get error from remote
37. Metrics A very useful for microservices and system health checks, determining bottlenecks
38. Tracking Heartbeats of Your Elixir Processes Metrex Package - Creating new metric dynamically - Incrementing metric - TTL - Dumping metric data - Init and exit hooks Results: - 3260 req/min on Heroku Free Dyno - http://bit.ly/2bYRnpp config :metrex, counter: ["all"], meters: ["cache", "storage", "thumb"], ttl: 900 # Dynamically create a metric Metrex.start_meter("pageviews") # Increment a meter by 1 Metrex.Meter.increment("pageviews") # Increment a meter by x(number) Metrex.Meter.increment("pageviews", 5)
39. Tips & Source Code
40. Libraries Metrex - Has implementation for counter and meter patterns Exmagick - NIF package which means you can spawn graphicsmagick as Elixir Process Poolboy Cowboy Plug
41. Source Github: https://github.com/mustafaturan/imgout (Heroku Deploy Button available / Has 3 branches for all 3 approaches) https://github.com/mustafaturan/metrex
42. QUESTIONS
43. THANK YOU