The Upside Down Dimension of Elixir - An introduction to metaprogramming

Slide 1

Slide 1 text

2/61

Slide 2

Slide 2 text

Are you tired of having to write the same boilerplate code in multiple modules? The Upside Down Dimension of Elixir - @nicholasjhenry 3/61

Slide 3

Slide 3 text

Do you wish you could program in a language that more closely reflected the domain you are working in? The Upside Down Dimension of Elixir - @nicholasjhenry 4/61

Slide 4

Slide 4 text

Is there a feature you wished existed in the Elixir language? The Upside Down Dimension of Elixir - @nicholasjhenry 5/61

Slide 5

Slide 5 text

Open the gate into the Upside Down Dimension that is Metaprogramming in Elixir The Upside Down Dimension of Elixir - @nicholasjhenry 6/61

Slide 6

Slide 6 text

Metaprogramming is a tool that gives you super powers like a Mind Slayer The Upside Down Dimension of Elixir - @nicholasjhenry 7/61

Slide 7

Slide 7 text

Metaprogramming allows you to write code that writes code The Upside Down Dimension of Elixir - @nicholasjhenry 8/61

Slide 8

Slide 8 text

Remove boilerplate code iex> mix phx.new UpsideDown defmodule UpsideDownWeb do # Extends controllers, views, channels, router # ... def controller do # ... end # ... defmacro __using__(which) when is_atom(which) do apply(__MODULE__, which, []) end end defmodule UpsideDownWeb.PageController do use UpsideDownWeb, :controller def index(conn, params) do # ... end end The Upside Down Dimension of Elixir - @nicholasjhenry 9/61

Slide 9

Slide 9 text

Write Domain Specific Languages (DSL) markup do div id: "main" do h1 class: "title" do text "Welcome!" end end div class: "row" do div do p do: text "Hello!" end end end end The Upside Down Dimension of Elixir - @nicholasjhenry 10/61

Slide 10

Slide 10 text

Extend the language while x > 1 do # ... end The Upside Down Dimension of Elixir - @nicholasjhenry 11/61

Slide 11

Slide 11 text

Goal 1.Demystify metaprogramming in Elixir 2.Build a mental model and understand the core concepts 3.Read and write basic macros The Upside Down Dimension of Elixir - @nicholasjhenry 12/61

Slide 12

Slide 12 text

Prerequisite • Basic Elixir syntax • Working knowledge helpful The Upside Down Dimension of Elixir - @nicholasjhenry 13/61

Slide 13

Slide 13 text

My Experience: Typist # https://github.com/civilcode/typist defmodule Product do use Typist deftype Code :: String.t deftype Price :: Decimal.t deftype do code :: Product.Code.t price :: Product.Price.t end end The Upside Down Dimension of Elixir - @nicholasjhenry 14/61

Slide 14

Slide 14 text

How 1.Review a simple macro 2.Learn key concepts 3.Demo those concepts 4.Multiple points of review and questions 5.Revisit the macro The Upside Down Dimension of Elixir - @nicholasjhenry 15/61

Slide 15

Slide 15 text

Example • ExCalculator.add/2 • ExTracer.trace/1 iex> ExCalculator.add(1, 2) 3 iex> x = 1 ...> y = 2 ...> require ExTracer ...> ExTracer.trace(x + y) Executing: x + y Result: 3 The Upside Down Dimension of Elixir - @nicholasjhenry 16/61

Slide 16

Slide 16 text

Demo Calculator with Tracer call The Upside Down Dimension of Elixir - @nicholasjhenry 17/61

Slide 17

Slide 17 text

The 3 keys to the open the gate • defmacro/2 • quote/2 • unquote/1 The Upside Down Dimension of Elixir - @nicholasjhenry 18/61

Slide 18

Slide 18 text

How do we write code that writes code? ┌─────────────────┐ Elixir source code ──────▶│ Compiler │──────────▶ byte code └─────────────────┘ ▲ │ │ │ macros (a body of a function is expanded inline) The Upside Down Dimension of Elixir - @nicholasjhenry 19/61

Slide 19

Slide 19 text

Comparison Human World Upside Down Dimension (business logic) (code generation) run-time compile-time high level syntax internal data structure functions macros The Upside Down Dimension of Elixir - @nicholasjhenry 20/61

Slide 20

Slide 20 text

Questions The Upside Down Dimension of Elixir - @nicholasjhenry 21/61

Slide 21

Slide 21 text

1. Run-time / Compile-time The Upside Down Dimension of Elixir - @nicholasjhenry 22/61

Slide 22

Slide 22 text

Compilation Process ┌─────────────────┐ Elixir source code ──────▶│ Compiler │──────────▶ byte code └─────────────────┘ The Upside Down Dimension of Elixir - @nicholasjhenry 23/61

Slide 23

Slide 23 text

Demo 1. Code can be executed at compile-time 2. Common confusion with run- time vs compile-time The Upside Down Dimension of Elixir - @nicholasjhenry 24/61

Slide 24

Slide 24 text

Review / Questions Run-time Compile-time execute code execute code dynamic nature static nature executes byte-code generates byte-code The Upside Down Dimension of Elixir - @nicholasjhenry 25/61

Slide 25

Slide 25 text

2. High-level Syntax / Internal Data Structure The Upside Down Dimension of Elixir - @nicholasjhenry 26/61

Slide 26

Slide 26 text

Compilation Process ┌─────────────────┐ Elixir source code ──────▶│ compiler │──────────▶ byte code └─────────────────┘ ▲ │ │ │ macros The Upside Down Dimension of Elixir - @nicholasjhenry 27/61

Slide 27

Slide 27 text

Compilation Process ┌─────────────────┐ Elixir source code ──────▶│ parser │ └─────────────────┘ │ │ internal data structure │ ▼ ┌─────────────────┐ │ generator │──────────▶ byte code └─────────────────┘ The Upside Down Dimension of Elixir - @nicholasjhenry 28/61

Slide 28

Slide 28 text

Can we access this data structure? The Upside Down Dimension of Elixir - @nicholasjhenry 29/61

Slide 29

Slide 29 text

The first key: quote/2 The Upside Down Dimension of Elixir - @nicholasjhenry 30/61

Slide 30

Slide 30 text

Demo 1.Review quote in Tracer 2.Explore quote/2 The Upside Down Dimension of Elixir - @nicholasjhenry 31/61

Slide 31

Slide 31 text

Internal Data Structure # (1 + (2 * 3)) - 4 {:-, [context: Elixir, import: Kernel], [ {:+, [context: Elixir, import: Kernel], [1, {:*, [context: Elixir, import: Kernel], [2, 3]}]}, 4 ]} The Upside Down Dimension of Elixir - @nicholasjhenry 32/61

Slide 32

Slide 32 text

Remove metadata # (1 + (2 * 3)) - 4 {:-, [], [ {:+, [], [1, {:*, [], [2, 3]}]}, 4 ]} The Upside Down Dimension of Elixir - @nicholasjhenry 33/61

Slide 33

Slide 33 text

Remove syntax # (1 + (2 * 3)) - 4 :- :+ 1 :* 2 3 4 The Upside Down Dimension of Elixir - @nicholasjhenry 34/61

Slide 34

Slide 34 text

Tree structure # 1 + (2 * 3) - 4 ┌────┐ │ - │ (3) └────┘ │ ┌────────┴────────┐ │ │ ┌────┐ ┌────┐ │ + │ (7) │ 4 │ └────┘ └────┘ │ ┌──────┴──────┐ │ │ ┌────┐ ┌────┐ │ 1 │ │ * │ (6) └────┘ └────┘ │ ┌───────┴───────┐ │ │ ┌────┐ ┌────┐ │ 2 │ │ 3 │ └────┘ └────┘ The Upside Down Dimension of Elixir - @nicholasjhenry 35/61

Slide 35

Slide 35 text

Abstract Syntax Tree (AST) {function, metadata, [args]} => {function, metadata, [ast]} • a tree representation • syntactical structure of source code • optimized for machines • input to generate byte code The Upside Down Dimension of Elixir - @nicholasjhenry 36/61

Slide 36

Slide 36 text

Lisp • (* 2 (+ 3 4)) • parenthesized lists • form a tree • power over readability (extensibility, DSL's) The Upside Down Dimension of Elixir - @nicholasjhenry 37/61

Slide 37

Slide 37 text

Elixir's approach • human world: higher level syntax • upside down dimension: AST • power and readability (extensibility, DSL's) The Upside Down Dimension of Elixir - @nicholasjhenry 38/61

Slide 38

Slide 38 text

quote/2 • allows us to write AST in a higher level syntax # not fun :(, but possible {:-, [], [ {:+, [], [1, {:*, [], [2, 3]}]}, 4 ]} # moz def fun! quote do 1 + (2 * 3) - 4 end The Upside Down Dimension of Elixir - @nicholasjhenry 39/61

Slide 39

Slide 39 text

Elixir is written in Elixir (mostly) The Upside Down Dimension of Elixir - @nicholasjhenry 40/61

Slide 40

Slide 40 text

The Upside Down Dimension of Elixir - @nicholasjhenry 41/61

Slide 41

Slide 41 text

Demo Let's look at the humble if statement The Upside Down Dimension of Elixir - @nicholasjhenry 42/61

Slide 42

Slide 42 text

Review / Questions • quote/2 generates an AST (internal data structure) • {function, metadata, [ast]} => Macro.t • Elixir is written in Elixir (90%) The Upside Down Dimension of Elixir - @nicholasjhenry 43/61

Slide 43

Slide 43 text

3. Functions / Macros The Upside Down Dimension of Elixir - @nicholasjhenry 44/61

Slide 44

Slide 44 text

The second key: defmacro/2 def add(x, y) do # programming for run-time # ... end defmacro trace(code) do # programming for compile-time # ... end The Upside Down Dimension of Elixir - @nicholasjhenry 45/61

Slide 45

Slide 45 text

Comparison def/2 defmacro/2 term => term Macro.t => Macro.t (ast) invoked at run-time invoked at compile-time current context macro and caller context The Upside Down Dimension of Elixir - @nicholasjhenry 46/61

Slide 46

Slide 46 text

Demo 1.Macro AST => AST 2.Macros invoked at compile time 3.Macro / Caller context The Upside Down Dimension of Elixir - @nicholasjhenry 47/61

Slide 47

Slide 47 text

Slide 48

Slide 48 text

Macro Expansion ┌─────────────────┐ Elixir source code ──────▶│ parser │ └─────────────────┘ │ │ raw AST │ ▼ ┌─────────────────┐ │ expansion │ (macros invoked) └─────────────────┘ │ │ expanded AST │ ▼ ┌─────────────────┐ │ generator │──────────▶ byte code └─────────────────┘ The Upside Down Dimension of Elixir - @nicholasjhenry 49/61

Slide 49

Slide 49 text

Demo • The expansion stage invokes the macro. • Macros inject into the AST • unquote/1 The Upside Down Dimension of Elixir - @nicholasjhenry 50/61

Slide 50

Slide 50 text

The third key: unquote/1 Construct the AST dynamically Demo • unquote/1 The Upside Down Dimension of Elixir - @nicholasjhenry 51/61

Slide 51

Slide 51 text

Review / Questions def/2 defmacro/2 term => term Macro.t => Macro.t (ast) invoked at run-time invoked at compile-time current context macro and caller context The Upside Down Dimension of Elixir - @nicholasjhenry 52/61

Slide 52

Slide 52 text

Final Demo • Review ExTracer.trace/1 • Annotate The Upside Down Dimension of Elixir - @nicholasjhenry 53/61

Slide 53

Slide 53 text

The Upside Down Dimension of Elixir The Upside Down Dimension of Elixir - @nicholasjhenry 54/61

Slide 54

Slide 54 text

Advanced concepts • Accumulating module attributes (ExUnit) • Compile-time hooks (__before_compile__) • Generate code from external data (files, API's, databases) The Upside Down Dimension of Elixir - @nicholasjhenry 55/61

Slide 55

Slide 55 text

Final Review Human World Upside Down Dimension (business logic) (code generation) run-time compile-time high level syntax internal data structure functions macros The Upside Down Dimension of Elixir - @nicholasjhenry 56/61

Slide 56

Slide 56 text

Features 1.Remove boilerplate code 2.Write Domain-Specific Languages (DSL) 3.Add new language constructs The Upside Down Dimension of Elixir - @nicholasjhenry 57/61

Slide 57

Slide 57 text

Books • Metaprogramming in Elixir • Programming Elixir 1.6 - Macros and Code Evaluation The Upside Down Dimension of Elixir - @nicholasjhenry 58/61

Slide 58

Slide 58 text

Online Resources • Elixir - Getting Started • Elixir Docs - Macro and Code • Elixir School - Metaprogramming • Understanding Elixir Macros - Saša Jurić The Upside Down Dimension of Elixir - @nicholasjhenry 59/61

Slide 59

Slide 59 text

Next Steps • Download this repo and play (https:// github.com/nicholasjhenry/upside-down-elixir) • Look at the source for if/2 and GenServer • Read "Understanding Elixir Macros" blog post The Upside Down Dimension of Elixir - @nicholasjhenry 60/61

Slide 60

Slide 60 text

Have fun in the Upside Down Dimension of Elixir :-) The Upside Down Dimension of Elixir - @nicholasjhenry 61/61