Domain Specific Languages and Metaprogramming

Transcript

1. Domain Specific Languages and Metaprogramming Andrew Summers 2019-07-24
2. #chielixirdsl Andrew Summers whoami • Software Engineer at DRW • Wrote Dialyzer pretty printer • Maintainer of Dialyxir, Erlex, elixir-lsp packages • Contributor to many others
3. #chielixirdsl Andrew Summers • What is a DSL? • Modules As Data Structures • Macros • Let’s Create a DSL
4. #chielixirdsl Andrew Summers • What is a DSL? • Modules As Data Structures • Macros • Let’s Create a DSL
5. #chielixirdsl Andrew Summers What is a DSL? • DSL = Domain Specific Language • Facilitates communication with experts within domain • Allows hiding plumbing behind macros • Examples: Ecto, Absinthe
6. #chielixirdsl Andrew Summers Why create a DSL? • Have superset of information needed to embed other DSLs • Allows you to edit entire codebase in one fell swoop • Plumbing data structures via normal functions is painful • Code base begins to solidify and data structures allow very generic pipelines
7. #chielixirdsl Andrew Summers Caveat Emptor • Warning: usually you just need functions and data, and should not create DSLs too early • Wait until data structures are somewhat stable before creating DSLs • Wait until plumbing is painful/buggy or some compile time need for macros exists before thinking about DSL • You probably do not need a DSL
8. #chielixirdsl Andrew Summers • What is a DSL? • Modules As Data Structures • Macros • Let’s Create a DSL
9. #chielixirdsl Andrew Summers defmodule ModuleAttributes do Module.register_attribute(__MODULE__, :foos, accumulate: true) Module.register_attribute(__MODULE__, :bar, accumulate: false) @foos :a @foos :b @bar :c @bar :d def foos(), do: @foos # [:b, :a] def bar(), do: @bar # :d end
10. #chielixirdsl Andrew Summers Module Attributes • Accumulate data into module attributes at compile time • Embed into function calls / data structures • Put macro data into internal data structure, call Module.put_attribute/3 after validation of opts
11. #chielixirdsl Andrew Summers Modules As Data Structures • See __schema__/1 in Ecto • Encode data into functions • Use behaviours for compile/refactor safety • Ask a module questions by passing around module name and invoking functions for use in generic pipelines
12. #chielixirdsl Andrew Summers • What is a DSL? • Modules As Data Structures • Macros • Let’s Create a DSL
13. #chielixirdsl Andrew Summers Macro Best Practices • Only do necessary work in macros and delegate out to functions ASAP • Use location: :keep to get better line numbers in errors • Validate inputs (especially unnecessary opts) • Use after_compile hooks to validate state where appropriate • Avoid dynamic name generation, optimize for greppability • Disabling lexical tracker useful for preventing deadlocks
15. #chielixirdsl Andrew Summers data >> code > macros
16. #chielixirdsl Andrew Summers DSL = Data + Code + Macros
17. #chielixirdsl Andrew Summers • What is a DSL? • Modules As Data Structures • Macro Hygiene • Let’s Create a DSL
18. #chielixirdsl Andrew Summers Let’s create a DSL • We want more data than Ecto provides in its API, e.g. description • We also want Relationships, Fields, Schema semantics • Embed Ecto Schema internally
19. #chielixirdsl Andrew Summers defmodule MyApp.Tables.ExampleTable do import Ectoo.Table, only: [table: 1] table :example do description "This is an example table" attribute :foo, :integer, required?: true, description: "foos do foo things" attribute :bar, :string, default: "none" end end
20. #chielixirdsl Andrew Summers defmodule Ectoo.Attribute do @type t :: %Ectoo.Attribute{ default: any(), description: String.t(), name: atom(), required?: boolean(), type: atom() | module() } defstruct [ :default, :description, :name, :required?, :type ] # ... end
21. #chielixirdsl Andrew Summers defmodule Ectoo.Relationship do @type t :: %Ectoo.Relationship{ cardinality: :belongs_to | :has_many | :has_one, description: String.t(), field: atom(), related: module() } defstruct [ :cardinality, :description, :field, :related ] # ... end
22. #chielixirdsl Andrew Summers defmodule Ectoo.Table do defmacro table(name, do: body) do quote location: :keep do Module.register_attribute(__MODULE__, :ectoo_description, accumulate: false) Module.register_attribute(__MODULE__, :ectoo_table, accumulate: false) Module.register_attribute(__MODULE__, :ectoo_attributes, accumulate: true) Module.register_attribute(__MODULE__, :ectoo_relationships, accumulate: true) # ... end end # ... end
23. #chielixirdsl Andrew Summers defmodule Ectoo.Table do defmacro table(name, do: body) do quote location: :keep do # ... @ectoo_table unquote(name) import Ectoo.Table try do unquote(body) after _ -> :ok end import Ectoo.Table, only: [] # ... end end # ... end
24. #chielixirdsl Andrew Summers defmodule Ectoo.Table do # ... defmacro attribute(name, type, opts \\ []) do quote location: :keep do Ectoo.Table.__attribute__(__MODULE__, unquote(name), unquote(type), unquote(opts)) end end def __attribute__(module, name, type, opts) do # ... validate opts attribute = %Ectoo.Attribute{ name: name, type: type, default: opts[:default], description: opts[:description], required?: opts[:required?] || false } Module.put_attribute(module, :ectoo_attributes, attribute) end # ... end
25. #chielixirdsl Andrew Summers defmodule Ectoo.Table do # ... defmacro description(name, description) do quote location: :keep do Ectoo.Table.__description__(__MODULE__, unquote(description)) end end def __description__(module, description) do Module.put_attribute(module, :ectoo_description, description) end # ... end
26. #chielixirdsl Andrew Summers defmodule Ectoo.Table do # ... defp expand_alias({:__aliases__, _, _} = ast, env) do Macro.expand(ast, %{env | lexical_tracker: nil}) end defp expand_alias(ast, _env) do ast end # ... end
27. #chielixirdsl Andrew Summers defmodule Ectoo.Table do # ... defmacro has_one_relationship(name, queryable, opts \\ []) do queryable = expand_alias(queryable, __CALLER__) quote do Ectoo.Table.__has_one_relationship__(__MODULE__, unquote(name), unquote(queryable), unquote(opts)) end end def __has_one_relationship__(module, name, related, opts) do # ... validate opts relationship = %Ectoo.Relationship{ field: name, cardinality: :has_one, related: related } Module.put_attribute(module, :ectoo_relationships, relationship) end # ... end
28. #chielixirdsl Andrew Summers defmodule Ectoo.Table do defmacro table(name, do: body) do quote location: :keep do # ... def attributes(), do: @ectoo_attributes def description(), do: @ectoo_description def belongs_to_relationships(), do: Enum.filter(@ectoo_relationships, & &1.cardinality == :belongs_to) def has_many_relationships(), do: Enum.filter(@ectoo_relationships, & &1.cardinality == :has_many) def has_one_relationships(), do: Enum.filter(@ectoo_relationships, & &1.cardinality == :has_one) end end # ... end
29. #chielixirdsl Andrew Summers defmodule Ectoo.Table do defmacro table(name, do: body) do quote location: :keep do # ... use Ecto.Schema schema @ectoo_table do for field <- @ectoo_attributes, do: field field.name, field.type, default: field.default for relationship = %{cardinality: :belongs_to} <- @ectoo_relationships, do: belongs_to relationship.name, relationship.related for relationship = %{cardinality: :has_many} <- @ectoo_relationships do: has_many relationship.name, relationship.related for relationship = %{cardinality: :has_one} <- @ectoo_relationships, do: has_one relationship.name, relationship.related end # ... end end # ... end
30. #chielixirdsl Andrew Summers defmodule MyApp.Tables.ExampleTable do import Ectoo.Table, only: [table: 1] table :example do description "This is an example table" attribute :foo, :integer, required?: true, description: "foos do foo things" attribute :bar, :string, default: "none" end end
31. #chielixirdsl Andrew Summers Recap • What and why DSLs? • Used module attributes as vehicle to collect compile time data • Macros • Built DSL wrapper around Ecto
32. #chielixirdsl Andrew Summers Future Steps • Expand the data structures to have all Ecto options • Build helpers to answer common questions from data • Use the data structures to build generic pipelines, e.g. Changesets • Add more macros to enable more sophisticated pipelines
33. #chielixirdsl Andrew Summers Thanks!
 
 asummers 
 
 _asummers
34. Questions?