Elixir and OTP

Slides from the presentation I gave at the January DevBash in Belfast.

Markdown source is at: https://github.com/chrismcg/presentations/blob/master/elixir_devbash/elixir_devbash.md

Transcript

1. Elixir and OTP
   Chris McGrath
   @chrismcg
   chris@chrismcg.com
   

   View Slide

2. What is Elixir?
   

   View Slide

3. Elixir
   Elixir is a dynamic, functional language designed for building
   scalable and maintainable applications.
   Elixir leverages the Erlang VM, known for running low-latency,
   distributed and fault-tolerant systems, while also being
   successfully used in web development and the embedded
   software domain.
   — http://www.elixir-lang.org
   

   View Slide

4. What Elixir is NOT
   • Some sort of CoffeeScript for Erlang
   • A port of Ruby to Erlang
   • Just Erlang with nicer syntax
   

   View Slide

5. Some example code
   defmodule RedirectCounter.URL do
   @max_redirects 10
   def count_redirects(url) do
   { :ok, response } = HTTPoison.head(url)
   do_count(response.status_code, response.headers["Location"], 0)
   end
   defp do_count(_status_code, _url, @max_redirects), do: raise "To many redirects"
   defp do_count(status_code, url, redirect_count) when status_code in [301, 302, 307] do
   { :ok, response } = HTTPoison.head(url)
   do_count(response.status_code, response.headers["Location"], redirect_count + 1)
   end
   defp do_count(_status_code, _url, redirect_count), do: redirect_count
   end
   

   View Slide

6. Why learn Elixir?
   

   View Slide

7. Personal Reasons
   

   View Slide

8. "Boss" Reasons
   • Our systems are becoming more and more parallel and the
   primitives provided by most languages are quite low level
   • Runs on top of the Erlang runtime, famous for amazing
   uptimes and fault tolerance
   • Powerful macro system for creating DSLs and reducing
   boilerplate
   • OTP library and architecture makes it easier to create fault
   tolerant systems
   

   View Slide

9. What's Elixir useful for?
   • Network related tasks (from plain sockets to web servers
   and frameworks)
   • Writing reliable, distributed, and highly available software
   • MMO backends (not frontends!)
   • Using all the cores
   • (AKA Things Erlang Is Good For)
   

   View Slide

10. What Elixir adds to Erlang
    • Modules for namespacing
    • Macros
    • A focus on tooling
    • Streaming
    

    View Slide

11. What Elixir adds to Erlang
    • Much nicer string handling
    • Consistent function parameters
    • Clearer organization of standard library
    • Variable rebinding
    • Less fiddly syntax
    

    View Slide

12. Language Highlights
    • Mix project management tool
    • First class documentation and doctests
    • Toll free calling of Erlang functions
    • Macros
    • Pipeline Operator
    • Protocols
    

    View Slide

13. Mix
    • Generates and manages projects
    • Somewhat similar to leiningen
    • Like Make/Rake it can compile and runs tests
    • Like Bundler it allows dependencies to be specified
    • Like Rails or Bundler it can generate new project skeletons
    • Full integration with Erlang, Rebar, and hex.pm
    

    View Slide

14. IEx - Interactive Elixir REPL
    % iex
    iex(1)> x = 1 + 2
    3
    iex(2)> x = 4
    4
    iex(3)> IO.puts "Hello World"
    Hello World
    :ok
    

    View Slide

15. Documentation & Doctests
    defmodule ShowDoctest do
    @moduledoc """
    This module shows off an example of a doctest
    """
    @doc """
    Adds it's inputs together
    iex> ShowDoctest.add(1, 1)
    2
    """
    def add(a, b) do
    a - b
    end
    end
    

    View Slide

16. defmodule ShowDoctestTest do
    use ExUnit.Case, async: true
    doctest ShowDoctest
    end
    % mix test
    1) test doc at ShowDoctest.add/2 (1) (ShowDoctestTest)
    test/show_doctest_test.exs:3
    Doctest failed
    code: ShowDoctest.add(1, 1) === 2
    lhs: 0
    stacktrace:
    lib/show_doctest.ex:12: ShowDoctest (module)
    

    View Slide

17. IEx Doc integration
    % iex -S mix
    iex(1)> h ShowDoctest
    ShowDoctest
    This module shows off an example of a doctest
    iex(2)> h ShowDoctest.add
    def add(a, b)
    Adds it's inputs together
    

    View Slide

18. Toll free calling into erlang
    You can use any available Erlang library in your Elixir project
    % erl
    Eshell V6.3 (abort with ^G)
    1> os:timestamp().
    {1422,119363,162867}
    % iex
    iex(1)> :os.timestamp
    {1422, 119376, 391592}
    

    View Slide

19. Macros
    Lisps traditionally empowered developers because you can
    eliminate anything that's tedious through macros, and that power
    is really what people keep going back for
    — Rich Hickey
    

    View Slide

20. Macro Example
    test "some sums" do
    assert 1 + 1 == 3
    end
    1) test some math (TestProjectTest)
    ** (ExUnit.ExpectationError)
    expected: 2
    to be equal to (==): 3
    at test/test_project_test.exs:5
    

    View Slide

21. Macro Example
    iex(1)> quote do: 1 + 1 == 3
    {:==, [context: Elixir, import: Kernel],
    [{:+, [context: Elixir, import: Kernel], [1, 1]}, 3]}
    defmacro assert({ :==, _, [l, r]}) do
    # ...
    end
    defmacro assert({ :=~, _, [l, r]}) do
    # ...
    end
    

    View Slide

22. Pipelines
    people = DB.find_customers
    orders = Orders.for_customers(people)
    tax = sales_tax(orders, 2013)
    filing = prepare_filing(tax)
    

    View Slide

23. Pipelines
    filing = DB.find_customers
    |> Orders.for_customers
    |> sales_tax(2013)
    |> prepare_filing
    

    View Slide

24. Pipelines
    # rewritten to...
    filing = prepare_filing(
    sales_tax(
    Orders.for_customers(DB.find_customers),
    2013
    ))
    

    View Slide

25. Protocols
    • Let you have polymorphism in Elixir
    • Inspired heavily by Clojure
    • Can define implementation of built in protocols for your
    own types
    

    View Slide

26. Protocols: Definition1
    defprotocol Blank do
    @doc "Returns true if data is considered blank/empty"
    def blank?(data)
    end
    1 Sorry, the syntax highlighter doesn't know about protocols yet
    

    View Slide

27. Protocols: Implementation1
    # Integers are never blank
    defimpl Blank, for: Integer do
    def blank?(_), do: false
    end
    # Just empty list is blank
    defimpl Blank, for: List do
    def blank?([]), do: true
    def blank?(_), do: false
    end
    #...
    1 Sorry, the syntax highlighter doesn't know about protocols yet
    

    View Slide

28. Protocols: Calling
    iex> Blank.blank?(0)
    false
    iex> Blank.blank?([])
    true
    iex> Blank.blank?([1, 2, 3])
    false
    

    View Slide

29. Enumerable
    iex(1)> Enum.map([1, 2, 3], fn(x) -> x * x end)
    [1, 4, 9]
    iex(2)> Enum.map([1, 2, 3], &(&1 * &1))
    [1, 4, 9]
    

    View Slide

30. Enumerable
    iex(1)> stream = Stream.map([1, 2, 3], &(&1 * &1))
    #Stream<[enum: [1, 2, 3], funs: [#Function<45.29647706/1 in Stream.map/2>]]>
    iex(2)> stream = Stream.map(stream, &Integer.to_string/1)
    #Stream<[enum: [1, 2, 3],
    funs: [#Function<45.29647706/1 in Stream.map/2>,
    #Function<45.29647706/1 in Stream.map/2>]]>
    iex(3)> Enum.to_list(stream)
    ["1", "4", "9"]
    

    View Slide

31. # More example code
    defmodule RedirectCounter.Twitter do
    def configure do
    # ... boring setup ...
    end
    def links do
    configure
    ExTwitter.stream_filter(track: "link")
    |> Stream.reject(fn(t) -> t.entities["urls"] == [] end)
    |> Stream.flat_map(fn(t) ->
    Enum.map(t.entities["urls"], fn(u) -> u["expanded_url"] end)
    end)
    end
    end
    

    View Slide

32. OTP
    

    View Slide

33. OTP2
    • Large collection of libraries covering a wide range of use
    cases
    • Set of design principles encoded in behaviours
    2 Open Telephony Platform - A marketing idea gone bad
    

    View Slide

34. Behaviours
    • Specify callbacks that you implement to specialize your own
    code
    • Formalize common patterns
    • Can create your own
    • Four standard ones in Erlang
    

    View Slide

35. OTP GenServer
    

    View Slide

36. defmodule RedirectCounter.TwitterLinkStream do
    use GenServer
    def start_link do
    GenServer.start_link __MODULE__, [], name: __MODULE__
    end
    def init(_) do
    GenServer.cast __MODULE__, :stream
    { :ok, nil }
    end
    def handle_cast(:stream, state) do
    spawn_link fn ->
    RedirectCounter.Twitter.links
    |> Enum.each(&RedirectCounter.CounterSupervisor.process/1)
    end
    { :noreply, state }
    end
    end
    

    View Slide

37. defmodule RedirectCounter.Count do
    use GenServer
    def start_link do
    GenServer.start_link __MODULE__, [], name: __MODULE__
    end
    def log(redirect_count) do
    GenServer.cast __MODULE__, { :redirect_count, redirect_count }
    end
    def get do
    GenServer.call __MODULE__, :get
    end
    def init(_) do
    { :ok, %{} }
    end
    def handle_cast({:redirect_count, redirect_count}, state) do
    state = Map.update(state, redirect_count, 1, fn(n) -> n + 1 end)
    { :noreply, state }
    end
    def handle_call(:get, _from, state) do
    { :reply, state, state }
    end
    end
    

    View Slide

38. def start_link do
    GenServer.start_link __MODULE__, [], name: __MODULE__
    end
    def log(redirect_count) do
    GenServer.cast __MODULE__, { :redirect_count, redirect_count }
    end
    def get do
    GenServer.call __MODULE__, :get
    end
    

    View Slide

39. def init(_) do
    { :ok, %{} }
    end
    def handle_cast({:redirect_count, redirect_count}, state) do
    state = Map.update(state, redirect_count, 1, fn(n) -> n + 1 end)
    { :noreply, state }
    end
    def handle_call(:get, _from, state) do
    { :reply, state, state }
    end
    

    View Slide

40. iex(1)> alias RedirectCounter.Count
    nil
    iex(2)> Count.start_link
    {:ok, #PID<0.91.0>}
    iex(3)> Count.log(1)
    :ok
    iex(4)> Count.log(1)
    :ok
    iex(5)> Count.log(1)
    :ok
    iex(6)> Count.log(2)
    :ok
    iex(7)> Count.log(3)
    :ok
    iex(8)> Count.get
    %{1 => 3, 2 => 1, 3 => 1}
    

    View Slide

41. Call vs Cast
    

    View Slide

42. Cast
    • Asynchronous
    • fire & forget
    • More decoupled
    • Less control over when things happen
    

    View Slide

43. Call
    • Synchronous
    • More coupled
    • More control over order of events
    

    View Slide

44. OTP Supervisors
    

    View Slide

45. Supervisors
    • Don't do any processing
    • Start and restart workers and other supervisors
    • Prevent errors taking the entire application down
    • Shutdown system in a controlled manor
    

    View Slide

46. Supervision Trees
    

    View Slide

47. defmodule RedirectCounter.Supervisor do
    use Supervisor
    def start_link do
    Supervisor.start_link(__MODULE__, [])
    end
    def init(_) do
    children = [
    worker(RedirectCounter.Count, []),
    worker(RedirectCounter.ConsoleOutput, []),
    supervisor(RedirectCounter.CounterSupervisor, []),
    worker(RedirectCounter.TwitterLinkStream, [])
    ]
    supervise(children, strategy: :one_for_one)
    end
    end
    

    View Slide

48. defmodule RedirectCounter.CounterSupervisor do
    use Supervisor
    def start_link do
    Supervisor.start_link __MODULE__, [], name: __MODULE__
    end
    def process(url) do
    {:ok, pid} = Supervisor.start_child(__MODULE__, [url])
    GenServer.cast(pid, :count)
    end
    def init(_) do
    children = [
    worker(RedirectCounter.URLRedirectCounter, [],
    restart: :temporary, shutdown: :brutal_kill)
    ]
    supervise(children, strategy: :simple_one_for_one)
    end
    end
    

    View Slide

49. defmodule RedirectCounter.URLRedirectCounter do
    use GenServer
    def start_link(url) do
    GenServer.start_link(__MODULE__, url)
    end
    def init(url) do
    { :ok, url }
    end
    def handle_cast(:count, url) do
    redirect_count = RedirectCounter.URL.count_redirects(url)
    RedirectCounter.Count.log(redirect_count)
    { :stop, :normal, url }
    end
    end
    

    View Slide

50. Supervision Strategies
    • one_for_one
    • simple_one_for_one
    • rest_for_one
    • one_for_all
    

    View Slide

51. Restart options
    • permanent
    • temporary
    • transient
    

    View Slide

52. Error Kernel
    Good Erlang design begins with identifying the error kernel of
    the system: What part must not fail or it will bring down the whole
    system?
    — Jesper Louis Anderson
    

    View Slide

53. Error Kernel
    Whenever the kernel is about to do an operation which is
    dangerous and might crash, you "outsource" that computation to
    another process, a dumb slave worker. If he crashes and is killed,
    nothing really bad has happened - since the kernel keeps going.
    — Jesper Louis Anderson
    

    View Slide

54. Agent
    

    View Slide

55. # Plain GenServer
    defmodule RedirectCounter.Count do
    use GenServer
    def start_link do
    GenServer.start_link __MODULE__, [], name: __MODULE__
    end
    def log(redirect_count) do
    GenServer.cast __MODULE__, { :redirect_count, redirect_count }
    end
    def get do
    GenServer.call __MODULE__, :get
    end
    def init(_) do
    { :ok, %{} }
    end
    def handle_cast({:redirect_count, redirect_count}, state) do
    state = Map.update(state, redirect_count, 1, fn(n) -> n + 1 end)
    { :noreply, state }
    end
    def handle_call(:get, _from, state) do
    { :reply, state, state }
    end
    end
    

    View Slide

56. # Elixir Agent
    defmodule RedirectCounter.Count do
    def start_link do
    Agent.start_link(fn -> %{} end, name: __MODULE__)
    end
    def log(redirect_count) do
    Agent.update(__MODULE__,
    &Map.update(&1, redirect_count, 1, fn(n) -> n + 1 end))
    end
    def get do
    Agent.get(__MODULE__, fn(map) -> map end)
    end
    end
    

    View Slide

57. Task and
    Task.Supervisor
    

    View Slide

58. Simple Example
    task = Task.async(fn -> do_some_work() end)
    res = do_some_other_work()
    res + Task.await(task)
    

    View Slide

59. # Main Supervisor - Before
    defmodule RedirectCounter.Supervisor do
    use Supervisor
    def start_link do
    Supervisor.start_link(__MODULE__, [])
    end
    def init(_) do
    children = [
    worker(RedirectCounter.Count, []),
    worker(RedirectCounter.ConsoleOutput, []),
    supervisor(RedirectCounter.CounterSupervisor, []),
    worker(RedirectCounter.TwitterLinkStream, [])
    ]
    supervise(children, strategy: :one_for_one)
    end
    end
    

    View Slide

60. # Main Supervisor - After
    defmodule RedirectCounter.Supervisor do
    use Supervisor
    def start_link do
    Supervisor.start_link(__MODULE__, [])
    end
    def init(_) do
    children = [
    worker(RedirectCounter.Count, []),
    worker(RedirectCounter.ConsoleOutput, []),
    supervisor(Task.Supervisor, [[name: :counter_supervisor]]),
    worker(Task, [RedirectCounter.Twitter, :process, [&RedirectCounter.URL.process/1]])
    ]
    supervise(children, strategy: :one_for_one)
    end
    end
    

    View Slide

61. # Previous RedirectCounter.Twitter
    defmodule RedirectCounter.Twitter do
    def configure do
    # ... boring setup ...
    end
    def links do
    configure
    ExTwitter.stream_filter(track: "link")
    |> Stream.reject(fn(t) -> t.entities["urls"] == [] end)
    |> Stream.flat_map(fn(t) ->
    Enum.map(t.entities["urls"], fn(u) -> u["expanded_url"] end)
    end)
    end
    end
    

    View Slide

62. # Updated RedirectCounter.Twitter
    defmodule RedirectCounter.Twitter do
    def process(fun) do
    links |> Enum.each(fun)
    end
    # ...
    end
    

    View Slide

63. # Previous RedirectCounter.URL
    defmodule RedirectCounter.URL do
    @max_redirects 10
    def count_redirects(url) do
    { :ok, response } = HTTPoison.head(url)
    do_count(response.status_code, response.headers["Location"], 0)
    end
    defp do_count(_status_code, _url, @max_redirects), do: raise "To many redirects"
    defp do_count(status_code, url, redirect_count) when status_code in [301, 302, 307] do
    { :ok, response } = HTTPoison.head(url)
    do_count(response.status_code, response.headers["Location"], redirect_count + 1)
    end
    defp do_count(_status_code, _url, redirect_count) do
    redirect_count
    end
    end
    

    View Slide

64. # Updated RedirectCounter.URL
    defmodule RedirectCounter.URL do
    def process(url) do
    Task.Supervisor.start_child(:counter_supervisor, __MODULE__, :count_redirects, [url])
    end
    def count_redirects(url) do
    { :ok, response } = HTTPoison.head(url)
    redirect_count = do_count(response.status_code, response.headers["Location"], 0)
    RedirectCounter.Count.log(redirect_count)
    end
    # ...
    end
    

    View Slide

65. What I haven't covered
    • gen_event and gen_fsm
    • Applications (in Erlang terminology)
    • Upgrades and hot code reloading
    • Debugging, monitoring, and logging
    • The other parts of OTP (ssh, asn.1, ...)
    • ets / mnesia (built in "NoSQL" databases)
    

    View Slide

66. Interesting Elixir projects
    • Plug: Rack/WSGI like layer for Elixir
    • Phoenix: Batteries included web/websockets framework
    • Ewebmachine: Generates HTTP responses based on HTTP
    decision tree
    • Ecto: LINQ inspired database abstraction layer
    

    View Slide

67. ElixirConf.eu
    23rd - 24th April 2015
    Krakow, Poland
    http://www.elixirconf.eu
    

    View Slide

68. Thanks!
    I hope I've interested you in Elixir and Erlang/OTP
    • http://elixir-lang.org
    • Progamming Elixir - Pragmatic Programmers
    • Elixir in Action - Manning
    • Erlang and OTP in Action - Manning
    • http://www.erlang-in-anger.com/
    

    View Slide