Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What's ahead for Elixir: v1.2 and GenRouter

What's ahead for Elixir: v1.2 and GenRouter

With Elixir v1.1 just released, José Valim explains what is ahead for Elixir 1.2 and discuss what we may see in Elixir v1.3 and later.

Plataformatec

October 10, 2015
Tweet

More Decks by Plataformatec

Other Decks in Programming

Transcript

  1. @elixirlang / elixir-lang.org

  2. Elixir v1.0 • September/2014 • >180 contributors • 3 books

    out! • First Elixirconf!
  3. Extensibility • Web infrastructure • Embedded systems • Financial/Video platforms

    • Graphical User Interfaces
  4. Elixir v1.1

  5. Elixir v1.1 • September/2015 • >295 contributors • ~ 1000

    packages on hex.pm • > 6.5million downloads
  6. None
  7. Application.fetch_env!/2 Keyword.get_and_update/3 Keyword.get_lazy/3 Keyword.pop_lazy/3 Keyword.put_new_lazy/3 Map.get_and_update/3 Map.get_lazy/3 Map.pop_lazy/3 Map.put_new_lazy/3 Enum.dedup/1

    Enum.dedup_by/2 Enum.min_max/1 Enum.min_max_by/2 Enum.random/1 Enum.reduce_while/3 Enum.reverse_slice/3 Enum.take_random/2 More functions File.lstat/1 File.lstat!/1 File.rename/2 Integer.digits/2 Integer.undigits/2 GenServer.whereis/1 Process.hibernate/3 Stream.dedup/1 Stream.dedup_by/2 Stream.transform/4 String.jaro_distance/2 String.splitter/3 StringIO.flush/1 Task.yield/2 Task.shutdown/2 Tuple.append/2 URI.to_string/1
  8. Enum iex> Enum.dedup([1, 1, 2, 2, 1, 1]) [1, 2,

    1] iex> Enum.random 1..100 51 iex> Enum.take_random 1..100, 3 [14, 3, 72]
  9. iex> String.jaro_distance "helo", "help" 0.8333333333334 String $ mix helo **

    (Mix) The task "helo" could not be found. Did you mean "help"?
  10. # v1.0+ iex> task = Task.async(fn -> ... end) %Task{}

    iex> Task.await(task, 5_000) # v1.1+ iex> Task.yield(task, 5_000) {:ok, result} | nil iex> Task.shutdown(task, :brutal_kill) {:ok, result} | nil Task
  11. ExUnit v1.1

  12. Capture Log # v1.0+ import ExUnit.CaptureIO capture_io fn -> IO.puts

    "hello" end # v1.1+ import ExUnit.CaptureLog capture_log fn -> Logger.info "hello" end
  13. # Per test @tag :capture_log test "something with logging" do

    ... end # Per suite ExUnit.configure capture_log: true Capture Log
  14. Capture Log 1) test something with logging test/plug/error.exs:59 ** (RuntimeError)

    oops stacktrace: test/plug/error.exs:59 The following output was logged: 20:27:52.443 [info] log message
  15. Not implemented tests # @tag :not_implemented test "eventually will do

    X" $ mix test --exclude not_implemented $ mix test --only not_implemented
  16. • stacktrace depth is now configurable and has been increased

    to 20 • More detailed errors • Ability to skip tests • Proper line number in doctests failures More improvements
  17. Mix v1.1

  18. mix profile.fprof # CNT ACC (ms) OWN (ms) Total 200279

    1972.188 1964.579 Test.run/0 1 1972.166 0.007 Test.do_something/1 3 1972.131 0.040 Test.bottleneck/0 1 1599.490 0.007 ... $ mix profile.fprof my_script.exs
  19. [build_embedded: Mix.env == :prod, start_permanent: Mix.env == :prod] Closer to

    releases
  20. Start Permanent (true) • Shuts the node down if your

    application crashes
  21. Build Embedded (false) /_build ├── dev │ └── lib │

    ├── cowboy │ │ └── ebin -> ../../../../deps/cowboy/ebin │ ├── cowlib │ │ ├── ebin -> ../../../../deps/cowlib/ebin │ ├── phoenix │ │ ├── ebin │ │ │ ├── Elixir.Access.Phoenix.Socket.beam
  22. Build embedded (true) • Packages your whole application into _build

    (no symlinks) • Enables protocol consolidation • No longer automatically compiles
  23. config/config.exs # v1.0: config/config.exs config :my_app, :key, :value # v1.1:

    this will warn config :unknown_app, :key, :value
  24. • "mix compile" tracks only compile-time dependencies for Elixir •

    "recompile()" inside IEx • Use the safer https protocol instead of git for :github dependencies More improvements
  25. Deprecations

  26. Elixir v1.1 # v1.1 iex> "Hello \u2661" "Hello ♡" #

    v1.0 iex> "Hello \x{2661}” "Hello ♡" SOFT DEPRECATED
  27. Elixir v1.1 # v1.0 defmodule Calculator do use Behaviour defcallback

    add(number, number) :: number end SOFT DEPRECATED # v1.1 defmodule Calculator do @callback add(number, number) :: number end
  28. Elixir v1.1 defmodule Calculator do @typedoc "A complex number" @opaque

    complex :: ... @doc "Adds two numbers" @callback add(number, number) :: ... end Code.get_docs(Calculator, :type_docs)
  29. Elixir v1.1 DEPRECATED defimpl Access, for: X do ... end

  30. Access opts[key] opts[key] opts[key] opts[key] Code Server

  31. • opts[key] still works • Access defines a subset of

    the Dict behaviour • The bottleneck does not exist in prod • Common hot-paths have been inlined Access
  32. Community

  33. None
  34. None
  35. Elixir v1.2

  36. Migration • Elixir v1.0, v1.1
 Support Erlang 17 & 18+


    • Elixir v1.2
 Supports Erlang 18+
  37. Erlang 18+ • Variables in maps: %{key => value} •

    Official support for large maps
  38. Dicts in Elixir v1.1 • Keyword, Map, HashDict • Unified

    via the Dict API • Too many options. Often confusing.
  39. Dicts in Elixir v1.2 • Keyword lists - used as

    options, allow duplicate keys, user ordered • Maps - pattern matching, fast & scalable, term ordered
  40. Dict & Set • HashDict will be soft deprecated
 Use

    Map • HashSet will be soft deprecated
 Use MapSet • Dict and Set will be soft deprecated
  41. Multi-aliases alias MyApp.Foo alias MyApp.Bar alias MyApp.Baz alias MyApp.{Foo, Bar,

    Baz}
  42. December/2015

  43. Meanwhile, upgrade to Erlang 18+

  44. Elixir v1.3

  45. Collections sink (collectable) widgets |> Enum.filter(fn b -> b.color ==

    :red end) |> Enum.map(fn b -> {b.title, b.height} end) |> Enum.into(%{}) source (enumerable)
  46. Collections CSV.parse(path) |> Enum.filter(fn b -> b.color == :red end)

    |> Enum.map(fn b -> {b.title, b.height} end) |> Enum.into(IO.stream(:stdio, :inspect))
  47. Collections + Laziness CSV.parse(path) |> Stream.filter(fn b -> b.color ==

    :red end) |> Stream.map(fn b -> {b.title, b.height} end) |> Stream.into(IO.stream(:stdio, :inspect)) |> Stream.run()
  48. Pipeline parallelism CSV.parse(path) |> Stream.async() |> Stream.filter(fn b -> b.color

    == :red end) |> Stream.map(fn b -> {b.title, b.height} end) |> Stream.into(IO.stream(:stdio, :inspect)) |> Stream.run()
  49. Pipeline parallelism CSV.parse(path) |> ... |> Stream.async() |> ... |>

    Stream.async() |> ... |> Stream.async() |> ... |> Stream.run()
  50. Pipeline parallelism async CSV async async run

  51. Pipeline parallelism • Each async stage is a process •

    What happens if one of them crash? • How to provide back-pressure?
  52. Pipeline parallelism • How to make streams supervised processes? •

    Pipeline parallelism isn’t even a good strategy
  53. How to make supervised processes stream data?

  54. GenRouter

  55. GenRouter • Connects sources to sinks • Demand-driven

  56. GenRouter Router

  57. GenRouter Sink

  58. GenRouter Source

  59. GenRouter in | out

  60. GenRouter GenRouter.start_link( InPart, in_args, OutPart, out_args )

  61. GenRouter.start_link( GenRouter.SingleIn, [], GenRouter.BroadcastOut, [] ) GenRouter

  62. SingleIn - BroadcastOut router

  63. SingleIn - BroadcastOut router 1 1 1 1 1

  64. SingleIn - BroadcastOut router 1,2 1,2 1,2 1,2 1,2

  65. SingleIn - BroadcastOut router 1,2,3 1,2,3 1,2,3 1,2,3 1,2,3

  66. DynamicIn - BroadcastOut router 1,2,3 1,2,3 1,2,3 1,2,3 2 1

    3
  67. DynamicIn - MessageQueueOut router 1 2 3 4 2 1

    3
  68. Some Use Cases • A GenEvent replacement that is process-based

    • A way to load-balance jobs across a pool of processes
  69. Demand-driven

  70. Demand-driven B C A Source | Sink Source | Sink

  71. Demand-driven B C A Ask 10 Ask 10 Sends max

    10 Sends max 10
  72. Demand-driven • It is a message contract • It pushes

    back-pressure to the boundary • GenRouter is one impl of this contract • Inspired by Akka Streams
  73. Supervised TCP Acceptor # Source {:ok, acceptor} = GenRouter.TCPAcceptor.start_link(...) #

    Sink {:ok, supervisor} = GenRouter.Supervisor.start_link(...) # Source <-> Sink GenRouter.subscribe(supervisor, to: acceptor)
  74. The path forward • Define the demand-driven message contract •

    Implement GenRouter and related abstractions • Integrate with streams
  75. consulting and software engineering

  76. @elixirlang / elixir-lang.org