Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Domain Specific Languages and Metaprogramming

Domain Specific Languages and Metaprogramming

Andrew Summers

July 24, 2019
Tweet

More Decks by Andrew Summers

Other Decks in Technology

Transcript

  1. Domain Specific Languages and Metaprogramming Andrew Summers 2019-07-24

  2. #chielixirdsl Andrew Summers whoami • Software Engineer at DRW •

    Wrote Dialyzer pretty printer • Maintainer of Dialyxir, Erlex, elixir-lsp packages • Contributor to many others
  3. #chielixirdsl Andrew Summers • What is a DSL? • Modules

    As Data Structures • Macros • Let’s Create a DSL
  4. #chielixirdsl Andrew Summers • What is a DSL? • Modules

    As Data Structures • Macros • Let’s Create a DSL
  5. #chielixirdsl Andrew Summers What is a DSL? • DSL =

    Domain Specific Language • Facilitates communication with experts within domain • Allows hiding plumbing behind macros • Examples: Ecto, Absinthe
  6. #chielixirdsl Andrew Summers Why create a DSL? • Have superset

    of information needed to embed other DSLs • Allows you to edit entire codebase in one fell swoop • Plumbing data structures via normal functions is painful • Code base begins to solidify and data structures allow very generic pipelines
  7. #chielixirdsl Andrew Summers Caveat Emptor • Warning: usually you just

    need functions and data, and should not create DSLs too early • Wait until data structures are somewhat stable before creating DSLs • Wait until plumbing is painful/buggy or some compile time need for macros exists before thinking about DSL • You probably do not need a DSL
  8. #chielixirdsl Andrew Summers • What is a DSL? • Modules

    As Data Structures • Macros • Let’s Create a DSL
  9. #chielixirdsl Andrew Summers defmodule ModuleAttributes do Module.register_attribute(__MODULE__, :foos, accumulate: true)

    Module.register_attribute(__MODULE__, :bar, accumulate: false) @foos :a @foos :b @bar :c @bar :d def foos(), do: @foos # [:b, :a] def bar(), do: @bar # :d end
  10. #chielixirdsl Andrew Summers Module Attributes • Accumulate data into module

    attributes at compile time • Embed into function calls / data structures • Put macro data into internal data structure, call Module.put_attribute/3 after validation of opts
  11. #chielixirdsl Andrew Summers Modules As Data Structures • See __schema__/1

    in Ecto • Encode data into functions • Use behaviours for compile/refactor safety • Ask a module questions by passing around module name and invoking functions for use in generic pipelines
  12. #chielixirdsl Andrew Summers • What is a DSL? • Modules

    As Data Structures • Macros • Let’s Create a DSL
  13. #chielixirdsl Andrew Summers Macro Best Practices • Only do necessary

    work in macros and delegate out to functions ASAP • Use location: :keep to get better line numbers in errors • Validate inputs (especially unnecessary opts) • Use after_compile hooks to validate state where appropriate • Avoid dynamic name generation, optimize for greppability • Disabling lexical tracker useful for preventing deadlocks
  14. None
  15. #chielixirdsl Andrew Summers data >> code > macros

  16. #chielixirdsl Andrew Summers DSL = Data + Code + Macros

  17. #chielixirdsl Andrew Summers • What is a DSL? • Modules

    As Data Structures • Macro Hygiene • Let’s Create a DSL
  18. #chielixirdsl Andrew Summers Let’s create a DSL • We want

    more data than Ecto provides in its API, e.g. description • We also want Relationships, Fields, Schema semantics • Embed Ecto Schema internally
  19. #chielixirdsl Andrew Summers defmodule MyApp.Tables.ExampleTable do import Ectoo.Table, only: [table:

    1] table :example do description "This is an example table" attribute :foo, :integer, required?: true, description: "foos do foo things" attribute :bar, :string, default: "none" end end
  20. #chielixirdsl Andrew Summers defmodule Ectoo.Attribute do @type t :: %Ectoo.Attribute{

    default: any(), description: String.t(), name: atom(), required?: boolean(), type: atom() | module() } defstruct [ :default, :description, :name, :required?, :type ] # ... end
  21. #chielixirdsl Andrew Summers defmodule Ectoo.Relationship do @type t :: %Ectoo.Relationship{

    cardinality: :belongs_to | :has_many | :has_one, description: String.t(), field: atom(), related: module() } defstruct [ :cardinality, :description, :field, :related ] # ... end
  22. #chielixirdsl Andrew Summers defmodule Ectoo.Table do defmacro table(name, do: body)

    do quote location: :keep do Module.register_attribute(__MODULE__, :ectoo_description, accumulate: false) Module.register_attribute(__MODULE__, :ectoo_table, accumulate: false) Module.register_attribute(__MODULE__, :ectoo_attributes, accumulate: true) Module.register_attribute(__MODULE__, :ectoo_relationships, accumulate: true) # ... end end # ... end
  23. #chielixirdsl Andrew Summers defmodule Ectoo.Table do defmacro table(name, do: body)

    do quote location: :keep do # ... @ectoo_table unquote(name) import Ectoo.Table try do unquote(body) after _ -> :ok end import Ectoo.Table, only: [] # ... end end # ... end
  24. #chielixirdsl Andrew Summers defmodule Ectoo.Table do # ... defmacro attribute(name,

    type, opts \\ []) do quote location: :keep do Ectoo.Table.__attribute__(__MODULE__, unquote(name), unquote(type), unquote(opts)) end end def __attribute__(module, name, type, opts) do # ... validate opts attribute = %Ectoo.Attribute{ name: name, type: type, default: opts[:default], description: opts[:description], required?: opts[:required?] || false } Module.put_attribute(module, :ectoo_attributes, attribute) end # ... end
  25. #chielixirdsl Andrew Summers defmodule Ectoo.Table do # ... defmacro description(name,

    description) do quote location: :keep do Ectoo.Table.__description__(__MODULE__, unquote(description)) end end def __description__(module, description) do Module.put_attribute(module, :ectoo_description, description) end # ... end
  26. #chielixirdsl Andrew Summers defmodule Ectoo.Table do # ... defp expand_alias({:__aliases__,

    _, _} = ast, env) do Macro.expand(ast, %{env | lexical_tracker: nil}) end defp expand_alias(ast, _env) do ast end # ... end
  27. #chielixirdsl Andrew Summers defmodule Ectoo.Table do # ... defmacro has_one_relationship(name,

    queryable, opts \\ []) do queryable = expand_alias(queryable, __CALLER__) quote do Ectoo.Table.__has_one_relationship__(__MODULE__, unquote(name), unquote(queryable), unquote(opts)) end end def __has_one_relationship__(module, name, related, opts) do # ... validate opts relationship = %Ectoo.Relationship{ field: name, cardinality: :has_one, related: related } Module.put_attribute(module, :ectoo_relationships, relationship) end # ... end
  28. #chielixirdsl Andrew Summers defmodule Ectoo.Table do defmacro table(name, do: body)

    do quote location: :keep do # ... def attributes(), do: @ectoo_attributes def description(), do: @ectoo_description def belongs_to_relationships(), do: Enum.filter(@ectoo_relationships, & &1.cardinality == :belongs_to) def has_many_relationships(), do: Enum.filter(@ectoo_relationships, & &1.cardinality == :has_many) def has_one_relationships(), do: Enum.filter(@ectoo_relationships, & &1.cardinality == :has_one) end end # ... end
  29. #chielixirdsl Andrew Summers defmodule Ectoo.Table do defmacro table(name, do: body)

    do quote location: :keep do # ... use Ecto.Schema schema @ectoo_table do for field <- @ectoo_attributes, do: field field.name, field.type, default: field.default for relationship = %{cardinality: :belongs_to} <- @ectoo_relationships, do: belongs_to relationship.name, relationship.related for relationship = %{cardinality: :has_many} <- @ectoo_relationships do: has_many relationship.name, relationship.related for relationship = %{cardinality: :has_one} <- @ectoo_relationships, do: has_one relationship.name, relationship.related end # ... end end # ... end
  30. #chielixirdsl Andrew Summers defmodule MyApp.Tables.ExampleTable do import Ectoo.Table, only: [table:

    1] table :example do description "This is an example table" attribute :foo, :integer, required?: true, description: "foos do foo things" attribute :bar, :string, default: "none" end end
  31. #chielixirdsl Andrew Summers Recap • What and why DSLs? •

    Used module attributes as vehicle to collect compile time data • Macros • Built DSL wrapper around Ecto
  32. #chielixirdsl Andrew Summers Future Steps • Expand the data structures

    to have all Ecto options • Build helpers to answer common questions from data • Use the data structures to build generic pipelines, e.g. Changesets • Add more macros to enable more sophisticated pipelines
  33. #chielixirdsl Andrew Summers Thanks!
 
 asummers 
 
 _asummers

  34. Questions?