Domain Specific Languages and Metaprogramming

Domain Specific Languages and Metaprogramming

A0845f7bdb6b3aba0ecf1560bb8c6042?s=128

Andrew Summers

July 24, 2019
Tweet

Transcript

  1. 2.

    #chielixirdsl Andrew Summers whoami • Software Engineer at DRW •

    Wrote Dialyzer pretty printer • Maintainer of Dialyxir, Erlex, elixir-lsp packages • Contributor to many others
  2. 3.

    #chielixirdsl Andrew Summers • What is a DSL? • Modules

    As Data Structures • Macros • Let’s Create a DSL
  3. 4.

    #chielixirdsl Andrew Summers • What is a DSL? • Modules

    As Data Structures • Macros • Let’s Create a DSL
  4. 5.

    #chielixirdsl Andrew Summers What is a DSL? • DSL =

    Domain Specific Language • Facilitates communication with experts within domain • Allows hiding plumbing behind macros • Examples: Ecto, Absinthe
  5. 6.

    #chielixirdsl Andrew Summers Why create a DSL? • Have superset

    of information needed to embed other DSLs • Allows you to edit entire codebase in one fell swoop • Plumbing data structures via normal functions is painful • Code base begins to solidify and data structures allow very generic pipelines
  6. 7.

    #chielixirdsl Andrew Summers Caveat Emptor • Warning: usually you just

    need functions and data, and should not create DSLs too early • Wait until data structures are somewhat stable before creating DSLs • Wait until plumbing is painful/buggy or some compile time need for macros exists before thinking about DSL • You probably do not need a DSL
  7. 8.

    #chielixirdsl Andrew Summers • What is a DSL? • Modules

    As Data Structures • Macros • Let’s Create a DSL
  8. 9.

    #chielixirdsl Andrew Summers defmodule ModuleAttributes do Module.register_attribute(__MODULE__, :foos, accumulate: true)

    Module.register_attribute(__MODULE__, :bar, accumulate: false) @foos :a @foos :b @bar :c @bar :d def foos(), do: @foos # [:b, :a] def bar(), do: @bar # :d end
  9. 10.

    #chielixirdsl Andrew Summers Module Attributes • Accumulate data into module

    attributes at compile time • Embed into function calls / data structures • Put macro data into internal data structure, call Module.put_attribute/3 after validation of opts
  10. 11.

    #chielixirdsl Andrew Summers Modules As Data Structures • See __schema__/1

    in Ecto • Encode data into functions • Use behaviours for compile/refactor safety • Ask a module questions by passing around module name and invoking functions for use in generic pipelines
  11. 12.

    #chielixirdsl Andrew Summers • What is a DSL? • Modules

    As Data Structures • Macros • Let’s Create a DSL
  12. 13.

    #chielixirdsl Andrew Summers Macro Best Practices • Only do necessary

    work in macros and delegate out to functions ASAP • Use location: :keep to get better line numbers in errors • Validate inputs (especially unnecessary opts) • Use after_compile hooks to validate state where appropriate • Avoid dynamic name generation, optimize for greppability • Disabling lexical tracker useful for preventing deadlocks
  13. 14.
  14. 17.

    #chielixirdsl Andrew Summers • What is a DSL? • Modules

    As Data Structures • Macro Hygiene • Let’s Create a DSL
  15. 18.

    #chielixirdsl Andrew Summers Let’s create a DSL • We want

    more data than Ecto provides in its API, e.g. description • We also want Relationships, Fields, Schema semantics • Embed Ecto Schema internally
  16. 19.

    #chielixirdsl Andrew Summers defmodule MyApp.Tables.ExampleTable do import Ectoo.Table, only: [table:

    1] table :example do description "This is an example table" attribute :foo, :integer, required?: true, description: "foos do foo things" attribute :bar, :string, default: "none" end end
  17. 20.

    #chielixirdsl Andrew Summers defmodule Ectoo.Attribute do @type t :: %Ectoo.Attribute{

    default: any(), description: String.t(), name: atom(), required?: boolean(), type: atom() | module() } defstruct [ :default, :description, :name, :required?, :type ] # ... end
  18. 21.

    #chielixirdsl Andrew Summers defmodule Ectoo.Relationship do @type t :: %Ectoo.Relationship{

    cardinality: :belongs_to | :has_many | :has_one, description: String.t(), field: atom(), related: module() } defstruct [ :cardinality, :description, :field, :related ] # ... end
  19. 22.

    #chielixirdsl Andrew Summers defmodule Ectoo.Table do defmacro table(name, do: body)

    do quote location: :keep do Module.register_attribute(__MODULE__, :ectoo_description, accumulate: false) Module.register_attribute(__MODULE__, :ectoo_table, accumulate: false) Module.register_attribute(__MODULE__, :ectoo_attributes, accumulate: true) Module.register_attribute(__MODULE__, :ectoo_relationships, accumulate: true) # ... end end # ... end
  20. 23.

    #chielixirdsl Andrew Summers defmodule Ectoo.Table do defmacro table(name, do: body)

    do quote location: :keep do # ... @ectoo_table unquote(name) import Ectoo.Table try do unquote(body) after _ -> :ok end import Ectoo.Table, only: [] # ... end end # ... end
  21. 24.

    #chielixirdsl Andrew Summers defmodule Ectoo.Table do # ... defmacro attribute(name,

    type, opts \\ []) do quote location: :keep do Ectoo.Table.__attribute__(__MODULE__, unquote(name), unquote(type), unquote(opts)) end end def __attribute__(module, name, type, opts) do # ... validate opts attribute = %Ectoo.Attribute{ name: name, type: type, default: opts[:default], description: opts[:description], required?: opts[:required?] || false } Module.put_attribute(module, :ectoo_attributes, attribute) end # ... end
  22. 25.

    #chielixirdsl Andrew Summers defmodule Ectoo.Table do # ... defmacro description(name,

    description) do quote location: :keep do Ectoo.Table.__description__(__MODULE__, unquote(description)) end end def __description__(module, description) do Module.put_attribute(module, :ectoo_description, description) end # ... end
  23. 26.

    #chielixirdsl Andrew Summers defmodule Ectoo.Table do # ... defp expand_alias({:__aliases__,

    _, _} = ast, env) do Macro.expand(ast, %{env | lexical_tracker: nil}) end defp expand_alias(ast, _env) do ast end # ... end
  24. 27.

    #chielixirdsl Andrew Summers defmodule Ectoo.Table do # ... defmacro has_one_relationship(name,

    queryable, opts \\ []) do queryable = expand_alias(queryable, __CALLER__) quote do Ectoo.Table.__has_one_relationship__(__MODULE__, unquote(name), unquote(queryable), unquote(opts)) end end def __has_one_relationship__(module, name, related, opts) do # ... validate opts relationship = %Ectoo.Relationship{ field: name, cardinality: :has_one, related: related } Module.put_attribute(module, :ectoo_relationships, relationship) end # ... end
  25. 28.

    #chielixirdsl Andrew Summers defmodule Ectoo.Table do defmacro table(name, do: body)

    do quote location: :keep do # ... def attributes(), do: @ectoo_attributes def description(), do: @ectoo_description def belongs_to_relationships(), do: Enum.filter(@ectoo_relationships, & &1.cardinality == :belongs_to) def has_many_relationships(), do: Enum.filter(@ectoo_relationships, & &1.cardinality == :has_many) def has_one_relationships(), do: Enum.filter(@ectoo_relationships, & &1.cardinality == :has_one) end end # ... end
  26. 29.

    #chielixirdsl Andrew Summers defmodule Ectoo.Table do defmacro table(name, do: body)

    do quote location: :keep do # ... use Ecto.Schema schema @ectoo_table do for field <- @ectoo_attributes, do: field field.name, field.type, default: field.default for relationship = %{cardinality: :belongs_to} <- @ectoo_relationships, do: belongs_to relationship.name, relationship.related for relationship = %{cardinality: :has_many} <- @ectoo_relationships do: has_many relationship.name, relationship.related for relationship = %{cardinality: :has_one} <- @ectoo_relationships, do: has_one relationship.name, relationship.related end # ... end end # ... end
  27. 30.

    #chielixirdsl Andrew Summers defmodule MyApp.Tables.ExampleTable do import Ectoo.Table, only: [table:

    1] table :example do description "This is an example table" attribute :foo, :integer, required?: true, description: "foos do foo things" attribute :bar, :string, default: "none" end end
  28. 31.

    #chielixirdsl Andrew Summers Recap • What and why DSLs? •

    Used module attributes as vehicle to collect compile time data • Macros • Built DSL wrapper around Ecto
  29. 32.

    #chielixirdsl Andrew Summers Future Steps • Expand the data structures

    to have all Ecto options • Build helpers to answer common questions from data • Use the data structures to build generic pipelines, e.g. Changesets • Add more macros to enable more sophisticated pipelines