What's ahead for Elixir: v1.2 and GenRouter

What's ahead for Elixir: v1.2 and GenRouter

With Elixir v1.1 just released, José Valim explains what is ahead for Elixir 1.2 and discuss what we may see in Elixir v1.3 and later.

7c12adb8b5521c060ab4630360a4fa27?s=128

Plataformatec

October 10, 2015
Tweet

Transcript

  1. @elixirlang / elixir-lang.org

  2. Elixir v1.0 • September/2014 • >180 contributors • 3 books

    out! • First Elixirconf!
  3. Extensibility • Web infrastructure • Embedded systems • Financial/Video platforms

    • Graphical User Interfaces
  4. Elixir v1.1

  5. Elixir v1.1 • September/2015 • >295 contributors • ~ 1000

    packages on hex.pm • > 6.5million downloads
  6. None
  7. Application.fetch_env!/2 Keyword.get_and_update/3 Keyword.get_lazy/3 Keyword.pop_lazy/3 Keyword.put_new_lazy/3 Map.get_and_update/3 Map.get_lazy/3 Map.pop_lazy/3 Map.put_new_lazy/3 Enum.dedup/1

    Enum.dedup_by/2 Enum.min_max/1 Enum.min_max_by/2 Enum.random/1 Enum.reduce_while/3 Enum.reverse_slice/3 Enum.take_random/2 More functions File.lstat/1 File.lstat!/1 File.rename/2 Integer.digits/2 Integer.undigits/2 GenServer.whereis/1 Process.hibernate/3 Stream.dedup/1 Stream.dedup_by/2 Stream.transform/4 String.jaro_distance/2 String.splitter/3 StringIO.flush/1 Task.yield/2 Task.shutdown/2 Tuple.append/2 URI.to_string/1
  8. Enum iex> Enum.dedup([1, 1, 2, 2, 1, 1]) [1, 2,

    1] iex> Enum.random 1..100 51 iex> Enum.take_random 1..100, 3 [14, 3, 72]
  9. iex> String.jaro_distance "helo", "help" 0.8333333333334 String $ mix helo **

    (Mix) The task "helo" could not be found. Did you mean "help"?
  10. # v1.0+ iex> task = Task.async(fn -> ... end) %Task{}

    iex> Task.await(task, 5_000) # v1.1+ iex> Task.yield(task, 5_000) {:ok, result} | nil iex> Task.shutdown(task, :brutal_kill) {:ok, result} | nil Task
  11. ExUnit v1.1

  12. Capture Log # v1.0+ import ExUnit.CaptureIO capture_io fn -> IO.puts

    "hello" end # v1.1+ import ExUnit.CaptureLog capture_log fn -> Logger.info "hello" end
  13. # Per test @tag :capture_log test "something with logging" do

    ... end # Per suite ExUnit.configure capture_log: true Capture Log
  14. Capture Log 1) test something with logging test/plug/error.exs:59 ** (RuntimeError)

    oops stacktrace: test/plug/error.exs:59 The following output was logged: 20:27:52.443 [info] log message
  15. Not implemented tests # @tag :not_implemented test "eventually will do

    X" $ mix test --exclude not_implemented $ mix test --only not_implemented
  16. • stacktrace depth is now configurable and has been increased

    to 20 • More detailed errors • Ability to skip tests • Proper line number in doctests failures More improvements
  17. Mix v1.1

  18. mix profile.fprof # CNT ACC (ms) OWN (ms) Total 200279

    1972.188 1964.579 Test.run/0 1 1972.166 0.007 Test.do_something/1 3 1972.131 0.040 Test.bottleneck/0 1 1599.490 0.007 ... $ mix profile.fprof my_script.exs
  19. [build_embedded: Mix.env == :prod, start_permanent: Mix.env == :prod] Closer to

    releases
  20. Start Permanent (true) • Shuts the node down if your

    application crashes
  21. Build Embedded (false) /_build ├── dev │ └── lib │

    ├── cowboy │ │ └── ebin -> ../../../../deps/cowboy/ebin │ ├── cowlib │ │ ├── ebin -> ../../../../deps/cowlib/ebin │ ├── phoenix │ │ ├── ebin │ │ │ ├── Elixir.Access.Phoenix.Socket.beam
  22. Build embedded (true) • Packages your whole application into _build

    (no symlinks) • Enables protocol consolidation • No longer automatically compiles
  23. config/config.exs # v1.0: config/config.exs config :my_app, :key, :value # v1.1:

    this will warn config :unknown_app, :key, :value
  24. • "mix compile" tracks only compile-time dependencies for Elixir •

    "recompile()" inside IEx • Use the safer https protocol instead of git for :github dependencies More improvements
  25. Deprecations

  26. Elixir v1.1 # v1.1 iex> "Hello \u2661" "Hello ♡" #

    v1.0 iex> "Hello \x{2661}” "Hello ♡" SOFT DEPRECATED
  27. Elixir v1.1 # v1.0 defmodule Calculator do use Behaviour defcallback

    add(number, number) :: number end SOFT DEPRECATED # v1.1 defmodule Calculator do @callback add(number, number) :: number end
  28. Elixir v1.1 defmodule Calculator do @typedoc "A complex number" @opaque

    complex :: ... @doc "Adds two numbers" @callback add(number, number) :: ... end Code.get_docs(Calculator, :type_docs)
  29. Elixir v1.1 DEPRECATED defimpl Access, for: X do ... end

  30. Access opts[key] opts[key] opts[key] opts[key] Code Server

  31. • opts[key] still works • Access defines a subset of

    the Dict behaviour • The bottleneck does not exist in prod • Common hot-paths have been inlined Access
  32. Community

  33. None
  34. None
  35. Elixir v1.2

  36. Migration • Elixir v1.0, v1.1
 Support Erlang 17 & 18+


    • Elixir v1.2
 Supports Erlang 18+
  37. Erlang 18+ • Variables in maps: %{key => value} •

    Official support for large maps
  38. Dicts in Elixir v1.1 • Keyword, Map, HashDict • Unified

    via the Dict API • Too many options. Often confusing.
  39. Dicts in Elixir v1.2 • Keyword lists - used as

    options, allow duplicate keys, user ordered • Maps - pattern matching, fast & scalable, term ordered
  40. Dict & Set • HashDict will be soft deprecated
 Use

    Map • HashSet will be soft deprecated
 Use MapSet • Dict and Set will be soft deprecated
  41. Multi-aliases alias MyApp.Foo alias MyApp.Bar alias MyApp.Baz alias MyApp.{Foo, Bar,

    Baz}
  42. December/2015

  43. Meanwhile, upgrade to Erlang 18+

  44. Elixir v1.3

  45. Collections sink (collectable) widgets |> Enum.filter(fn b -> b.color ==

    :red end) |> Enum.map(fn b -> {b.title, b.height} end) |> Enum.into(%{}) source (enumerable)
  46. Collections CSV.parse(path) |> Enum.filter(fn b -> b.color == :red end)

    |> Enum.map(fn b -> {b.title, b.height} end) |> Enum.into(IO.stream(:stdio, :inspect))
  47. Collections + Laziness CSV.parse(path) |> Stream.filter(fn b -> b.color ==

    :red end) |> Stream.map(fn b -> {b.title, b.height} end) |> Stream.into(IO.stream(:stdio, :inspect)) |> Stream.run()
  48. Pipeline parallelism CSV.parse(path) |> Stream.async() |> Stream.filter(fn b -> b.color

    == :red end) |> Stream.map(fn b -> {b.title, b.height} end) |> Stream.into(IO.stream(:stdio, :inspect)) |> Stream.run()
  49. Pipeline parallelism CSV.parse(path) |> ... |> Stream.async() |> ... |>

    Stream.async() |> ... |> Stream.async() |> ... |> Stream.run()
  50. Pipeline parallelism async CSV async async run

  51. Pipeline parallelism • Each async stage is a process •

    What happens if one of them crash? • How to provide back-pressure?
  52. Pipeline parallelism • How to make streams supervised processes? •

    Pipeline parallelism isn’t even a good strategy
  53. How to make supervised processes stream data?

  54. GenRouter

  55. GenRouter • Connects sources to sinks • Demand-driven

  56. GenRouter Router

  57. GenRouter Sink

  58. GenRouter Source

  59. GenRouter in | out

  60. GenRouter GenRouter.start_link( InPart, in_args, OutPart, out_args )

  61. GenRouter.start_link( GenRouter.SingleIn, [], GenRouter.BroadcastOut, [] ) GenRouter

  62. SingleIn - BroadcastOut router

  63. SingleIn - BroadcastOut router 1 1 1 1 1

  64. SingleIn - BroadcastOut router 1,2 1,2 1,2 1,2 1,2

  65. SingleIn - BroadcastOut router 1,2,3 1,2,3 1,2,3 1,2,3 1,2,3

  66. DynamicIn - BroadcastOut router 1,2,3 1,2,3 1,2,3 1,2,3 2 1

    3
  67. DynamicIn - MessageQueueOut router 1 2 3 4 2 1

    3
  68. Some Use Cases • A GenEvent replacement that is process-based

    • A way to load-balance jobs across a pool of processes
  69. Demand-driven

  70. Demand-driven B C A Source | Sink Source | Sink

  71. Demand-driven B C A Ask 10 Ask 10 Sends max

    10 Sends max 10
  72. Demand-driven • It is a message contract • It pushes

    back-pressure to the boundary • GenRouter is one impl of this contract • Inspired by Akka Streams
  73. Supervised TCP Acceptor # Source {:ok, acceptor} = GenRouter.TCPAcceptor.start_link(...) #

    Sink {:ok, supervisor} = GenRouter.Supervisor.start_link(...) # Source <-> Sink GenRouter.subscribe(supervisor, to: acceptor)
  74. The path forward • Define the demand-driven message contract •

    Implement GenRouter and related abstractions • Integrate with streams
  75. consulting and software engineering

  76. @elixirlang / elixir-lang.org