The Upside Down Dimension of Elixir - An introduction to metaprogramming

Open the gate and enter into the Upside Down Dimension of Elixir that is metaprogramming. If you have become frustrated with boilerplate code in your modules, wanted to program in a language closer to your domain, or create new programming constructs in Elixir, then this presentation is for you. The format of this talk is a gentle introduction presenting key concepts reinforced with code demos. At the end of the presentation, you will have the tools to write basic macros and a solid foundation to explore more advanced concepts on your own.

Transcript

1. 2/61
2. Are you tired of having to write the same boilerplate code in multiple modules? The Upside Down Dimension of Elixir - @nicholasjhenry 3/61
3. Do you wish you could program in a language that more closely reflected the domain you are working in? The Upside Down Dimension of Elixir - @nicholasjhenry 4/61
4. Is there a feature you wished existed in the Elixir language? The Upside Down Dimension of Elixir - @nicholasjhenry 5/61
5. Open the gate into the Upside Down Dimension that is Metaprogramming in Elixir The Upside Down Dimension of Elixir - @nicholasjhenry 6/61
6. Metaprogramming is a tool that gives you super powers like a Mind Slayer The Upside Down Dimension of Elixir - @nicholasjhenry 7/61
7. Metaprogramming allows you to write code that writes code The Upside Down Dimension of Elixir - @nicholasjhenry 8/61
8. Remove boilerplate code iex> mix phx.new UpsideDown defmodule UpsideDownWeb do # Extends controllers, views, channels, router # ... def controller do # ... end # ... defmacro __using__(which) when is_atom(which) do apply(__MODULE__, which, []) end end defmodule UpsideDownWeb.PageController do use UpsideDownWeb, :controller def index(conn, params) do # ... end end The Upside Down Dimension of Elixir - @nicholasjhenry 9/61
9. Write Domain Specific Languages (DSL) markup do div id: "main" do h1 class: "title" do text "Welcome!" end end div class: "row" do div do p do: text "Hello!" end end end end The Upside Down Dimension of Elixir - @nicholasjhenry 10/61
10. Extend the language while x > 1 do # ... end The Upside Down Dimension of Elixir - @nicholasjhenry 11/61
11. Goal 1.Demystify metaprogramming in Elixir 2.Build a mental model and understand the core concepts 3.Read and write basic macros The Upside Down Dimension of Elixir - @nicholasjhenry 12/61
12. Prerequisite • Basic Elixir syntax • Working knowledge helpful The Upside Down Dimension of Elixir - @nicholasjhenry 13/61
13. My Experience: Typist # https://github.com/civilcode/typist defmodule Product do use Typist deftype Code :: String.t deftype Price :: Decimal.t deftype do code :: Product.Code.t price :: Product.Price.t end end The Upside Down Dimension of Elixir - @nicholasjhenry 14/61
14. How 1.Review a simple macro 2.Learn key concepts 3.Demo those concepts 4.Multiple points of review and questions 5.Revisit the macro The Upside Down Dimension of Elixir - @nicholasjhenry 15/61
15. Example • ExCalculator.add/2 • ExTracer.trace/1 iex> ExCalculator.add(1, 2) 3 iex> x = 1 ...> y = 2 ...> require ExTracer ...> ExTracer.trace(x + y) Executing: x + y Result: 3 The Upside Down Dimension of Elixir - @nicholasjhenry 16/61
16. Demo Calculator with Tracer call The Upside Down Dimension of Elixir - @nicholasjhenry 17/61
17. The 3 keys to the open the gate • defmacro/2 • quote/2 • unquote/1 The Upside Down Dimension of Elixir - @nicholasjhenry 18/61
18. How do we write code that writes code? ┌─────────────────┐ Elixir source code ──────▶│ Compiler │──────────▶ byte code └─────────────────┘ ▲ │ │ │ macros (a body of a function is expanded inline) The Upside Down Dimension of Elixir - @nicholasjhenry 19/61
19. Comparison Human World Upside Down Dimension (business logic) (code generation) run-time compile-time high level syntax internal data structure functions macros The Upside Down Dimension of Elixir - @nicholasjhenry 20/61
20. Questions The Upside Down Dimension of Elixir - @nicholasjhenry 21/61
21. 1. Run-time / Compile-time The Upside Down Dimension of Elixir - @nicholasjhenry 22/61
22. Compilation Process ┌─────────────────┐ Elixir source code ──────▶│ Compiler │──────────▶ byte code └─────────────────┘ The Upside Down Dimension of Elixir - @nicholasjhenry 23/61
23. Demo 1. Code can be executed at compile-time 2. Common confusion with run- time vs compile-time The Upside Down Dimension of Elixir - @nicholasjhenry 24/61
24. Review / Questions Run-time Compile-time execute code execute code dynamic nature static nature executes byte-code generates byte-code The Upside Down Dimension of Elixir - @nicholasjhenry 25/61
25. 2. High-level Syntax / Internal Data Structure The Upside Down Dimension of Elixir - @nicholasjhenry 26/61
26. Compilation Process ┌─────────────────┐ Elixir source code ──────▶│ compiler │──────────▶ byte code └─────────────────┘ ▲ │ │ │ macros The Upside Down Dimension of Elixir - @nicholasjhenry 27/61
27. Compilation Process ┌─────────────────┐ Elixir source code ──────▶│ parser │ └─────────────────┘ │ │ internal data structure │ ▼ ┌─────────────────┐ │ generator │──────────▶ byte code └─────────────────┘ The Upside Down Dimension of Elixir - @nicholasjhenry 28/61
28. Can we access this data structure? The Upside Down Dimension of Elixir - @nicholasjhenry 29/61
29. The first key: quote/2 The Upside Down Dimension of Elixir - @nicholasjhenry 30/61
30. Demo 1.Review quote in Tracer 2.Explore quote/2 The Upside Down Dimension of Elixir - @nicholasjhenry 31/61
31. Internal Data Structure # (1 + (2 * 3)) - 4 {:-, [context: Elixir, import: Kernel], [ {:+, [context: Elixir, import: Kernel], [1, {:*, [context: Elixir, import: Kernel], [2, 3]}]}, 4 ]} The Upside Down Dimension of Elixir - @nicholasjhenry 32/61
32. Remove metadata # (1 + (2 * 3)) - 4 {:-, [], [ {:+, [], [1, {:*, [], [2, 3]}]}, 4 ]} The Upside Down Dimension of Elixir - @nicholasjhenry 33/61
33. Remove syntax # (1 + (2 * 3)) - 4 :- :+ 1 :* 2 3 4 The Upside Down Dimension of Elixir - @nicholasjhenry 34/61
34. Tree structure # 1 + (2 * 3) - 4 ┌────┐ │ - │ (3) └────┘ │ ┌────────┴────────┐ │ │ ┌────┐ ┌────┐ │ + │ (7) │ 4 │ └────┘ └────┘ │ ┌──────┴──────┐ │ │ ┌────┐ ┌────┐ │ 1 │ │ * │ (6) └────┘ └────┘ │ ┌───────┴───────┐ │ │ ┌────┐ ┌────┐ │ 2 │ │ 3 │ └────┘ └────┘ The Upside Down Dimension of Elixir - @nicholasjhenry 35/61
35. Abstract Syntax Tree (AST) {function, metadata, [args]} => {function, metadata, [ast]} • a tree representation • syntactical structure of source code • optimized for machines • input to generate byte code The Upside Down Dimension of Elixir - @nicholasjhenry 36/61
36. Lisp • (* 2 (+ 3 4)) • parenthesized lists • form a tree • power over readability (extensibility, DSL's) The Upside Down Dimension of Elixir - @nicholasjhenry 37/61
37. Elixir's approach • human world: higher level syntax • upside down dimension: AST • power and readability (extensibility, DSL's) The Upside Down Dimension of Elixir - @nicholasjhenry 38/61
38. quote/2 • allows us to write AST in a higher level syntax # not fun :(, but possible {:-, [], [ {:+, [], [1, {:*, [], [2, 3]}]}, 4 ]} # moz def fun! quote do 1 + (2 * 3) - 4 end The Upside Down Dimension of Elixir - @nicholasjhenry 39/61
39. Elixir is written in Elixir (mostly) The Upside Down Dimension of Elixir - @nicholasjhenry 40/61
40. The Upside Down Dimension of Elixir - @nicholasjhenry 41/61
41. Demo Let's look at the humble if statement The Upside Down Dimension of Elixir - @nicholasjhenry 42/61
42. Review / Questions • quote/2 generates an AST (internal data structure) • {function, metadata, [ast]} => Macro.t • Elixir is written in Elixir (90%) The Upside Down Dimension of Elixir - @nicholasjhenry 43/61
43. 3. Functions / Macros The Upside Down Dimension of Elixir - @nicholasjhenry 44/61
44. The second key: defmacro/2 def add(x, y) do # programming for run-time # ... end defmacro trace(code) do # programming for compile-time # ... end The Upside Down Dimension of Elixir - @nicholasjhenry 45/61
45. Comparison def/2 defmacro/2 term => term Macro.t => Macro.t (ast) invoked at run-time invoked at compile-time current context macro and caller context The Upside Down Dimension of Elixir - @nicholasjhenry 46/61
46. Demo 1.Macro AST => AST 2.Macros invoked at compile time 3.Macro / Caller context The Upside Down Dimension of Elixir - @nicholasjhenry 47/61
47. Compilation Process ┌─────────────────┐ Elixir source code ──────▶│ parser │ └─────────────────┘ │ │ internal data structure │ ▼ ┌─────────────────┐ │ generator │──────────▶ byte code └─────────────────┘ The Upside Down Dimension of Elixir - @nicholasjhenry 48/61
48. Macro Expansion ┌─────────────────┐ Elixir source code ──────▶│ parser │ └─────────────────┘ │ │ raw AST │ ▼ ┌─────────────────┐ │ expansion │ (macros invoked) └─────────────────┘ │ │ expanded AST │ ▼ ┌─────────────────┐ │ generator │──────────▶ byte code └─────────────────┘ The Upside Down Dimension of Elixir - @nicholasjhenry 49/61
49. Demo • The expansion stage invokes the macro. • Macros inject into the AST • unquote/1 The Upside Down Dimension of Elixir - @nicholasjhenry 50/61
50. The third key: unquote/1 Construct the AST dynamically Demo • unquote/1 The Upside Down Dimension of Elixir - @nicholasjhenry 51/61
51. Review / Questions def/2 defmacro/2 term => term Macro.t => Macro.t (ast) invoked at run-time invoked at compile-time current context macro and caller context The Upside Down Dimension of Elixir - @nicholasjhenry 52/61
52. Final Demo • Review ExTracer.trace/1 • Annotate The Upside Down Dimension of Elixir - @nicholasjhenry 53/61
53. The Upside Down Dimension of Elixir The Upside Down Dimension of Elixir - @nicholasjhenry 54/61
54. Advanced concepts • Accumulating module attributes (ExUnit) • Compile-time hooks (__before_compile__) • Generate code from external data (files, API's, databases) The Upside Down Dimension of Elixir - @nicholasjhenry 55/61
55. Final Review Human World Upside Down Dimension (business logic) (code generation) run-time compile-time high level syntax internal data structure functions macros The Upside Down Dimension of Elixir - @nicholasjhenry 56/61
56. Features 1.Remove boilerplate code 2.Write Domain-Specific Languages (DSL) 3.Add new language constructs The Upside Down Dimension of Elixir - @nicholasjhenry 57/61
57. Books • Metaprogramming in Elixir • Programming Elixir 1.6 - Macros and Code Evaluation The Upside Down Dimension of Elixir - @nicholasjhenry 58/61
58. Online Resources • Elixir - Getting Started • Elixir Docs - Macro and Code • Elixir School - Metaprogramming • Understanding Elixir Macros - Saša Jurić The Upside Down Dimension of Elixir - @nicholasjhenry 59/61
59. Next Steps • Download this repo and play (https:// github.com/nicholasjhenry/upside-down-elixir) • Look at the source for if/2 and GenServer • Read "Understanding Elixir Macros" blog post The Upside Down Dimension of Elixir - @nicholasjhenry 60/61
60. Have fun in the Upside Down Dimension of Elixir :-) The Upside Down Dimension of Elixir - @nicholasjhenry 61/61