Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Building Beautiful Systems with Phoenix Context...

Andrew Hao
February 10, 2018

Building Beautiful Systems with Phoenix Contexts and DDD

Phoenix contexts are a powerful code organization tool - but without a clear idea of what business domains live under the hood of your systems, naively creating contexts leads to over-engineered, fragile systems.

Today, we’ll learn about the philosophical roots of Bounded Contexts from the hard-earned wisdom of Domain-Driven Design. We’ll quickly get our hands dirty in the nuts and bolts of a Context Mapping exercise, a strategic design tool that reveals domain-specific system boundaries. With our newfound architectural vision, we’ll learn how to write context-driven Phoenix code and develop some organizational rules around communication, boundary enforcement and testing between contexts. We’ll leverage the unique powers of Elixir that make this style of architecture so natural and see how using contexts easily leads to highly-cohesive and loosely-coupled outcomes!

Given at Empex LA 2018 on Feb 10, 2018
Given at Elixir Daze 2018 on March 2, 2018

Andrew Hao

February 10, 2018
Tweet

More Decks by Andrew Hao

Other Decks in Programming

Transcript

  1. The bulk of complexity is accidental Let's ship things fast!

    Death by a thousand cuts Abstractions that might work for small systems can't scale when the system grows. 6 / 128
  2. A blast from the past Information hiding D.L. Parnas -

    "On the Criteria to Be Used in Decomposing Systems into Modules" 8 / 128
  3. "We propose instead that one begins with a list of

    dif cult design decisions or design decisions which are likely to change. "Each module is then designed to hide such a decision from the others." (Emphasis added) 10 / 128
  4. Today: We will: Introduce Phoenix contexts Build a Context Map

    and use it to introduce Domain-Driven Design concepts Apply our learnings with Elixir! 13 / 128
  5. Follow the scaffold Create a User resource in the Identity

    context $ mix phx.gen.html Identity User users \ name:string email:string 16 / 128
  6. Follow the scaffold Create a User resource in the Identity

    context $ mix phx.gen.html Identity User users \ name:string email:string 17 / 128
  7. Follow the scaffold Create a User resource in the Identity

    context $ mix phx.gen.html Identity User users \ name:string email:string 18 / 128
  8. Follow the scaffold Create a User resource in the Identity

    context $ mix phx.gen.html Identity User users \ name:string email:string 19 / 128
  9. Ecto schema # lib/my_app/identity/user.ex defmodule MyApp.Identity.User do use Ecto.Schema schema

    "users" do field :name, :string field :email, :string timestamps() end end 20 / 128
  10. Domain resource operations in the context # lib/my_app/identity/identity.ex defmodule MyApp.Identity

    do def get_user!(id), do: ... def create_user(attrs \\ %{}) do: ... def update_user(%User{} = user, attrs) do: ... def delete_user(%User{} = user) do: ... end 21 / 128
  11. Web controller for the resource # lib/my_app_web/controllers/user_controller.ex defmodule MyApp.UserController do

    def index(conn, _params) do users = Identity.list_users() render(conn, "index.html", users: users) end def show(conn, %{"id" => id}) do user = Identity.get_user!(id) render(conn, "show.html", user: user) end end 22 / 128
  12. Migration for Ecto persistence defmodule MyApp.Repo.Migrations.CreateUsers do def change do

    create table(:users) do add :name, :string add :email, :string timestamps() end end end 23 / 128
  13. Contexts, the Phoenix way: All web concerns live in MyAppWeb

    context. Contexts encapsulate persistence and domain logic 24 / 128
  14. Contexts, the Phoenix way: All web concerns live in MyAppWeb

    context. Contexts encapsulate persistence and domain logic The outer context module is the public interface to the rest of the app 24 / 128
  15. How do I know if it's too broad (coarse) or

    too speci c ( ne)? 28 / 128
  16. Usually, we reach for Horizontal layers of abstractions Web layer

    Domain layer Persistence layer Adapters 33 / 128
  17. Welcome to AutoMaxx! A used-car marketplace Sellers bring their cars

    in for an inspection and appraisal at our stores, where we buy their vehicle. Interested buyers test drive cars in our lot, purchasing them if they are interested. 36 / 128
  18. Your business is constantly changing Marketing wants us to change

    copy on the web site Finance wants us to change how we do tax calculations on a car sale by region 37 / 128
  19. Your business is constantly changing Marketing wants us to change

    copy on the web site Finance wants us to change how we do tax calculations on a car sale by region Operations wants us to build a new feature in the vehicle inventory system 37 / 128
  20. Your business is constantly changing Marketing wants us to change

    copy on the web site Finance wants us to change how we do tax calculations on a car sale by region Operations wants us to build a new feature in the vehicle inventory system Product has a new initiative to offer purchase transactions in Bitcoin 37 / 128
  21. Your business is constantly changing Marketing wants us to change

    copy on the web site Finance wants us to change how we do tax calculations on a car sale by region Operations wants us to build a new feature in the vehicle inventory system Product has a new initiative to offer purchase transactions in Bitcoin Customer support wants us to build a better support dashboard 37 / 128
  22. DDD, summarized Design your software systems according to your business

    domains by: Paying attention to the language you speak in the business 39 / 128
  23. Context mapping An exercise to help us discover all the

    concepts living in our organization, and our systems. 41 / 128
  24. Context Mapping Get everyone in a room Put up on

    a wall all the: Nouns - entities Verbs - events 42 / 128
  25. Context Mapping Group like concepts and actions together. There may

    be overlaps - that's OK if your concepts belong in multiple groups. More important to just get it down. 44 / 128
  26. De nition! Core domain The Core Domain is the primary

    area of focus of your business 47 / 128
  27. De nition! Core domain The Core Domain is the primary

    area of focus of your business AutoMaxx Core Domain: Car Sales 47 / 128
  28. De nition! Supporting domains A Supporting Domain (or Subdomain) are

    the areas of the business that play roles in making the Core Domain happen. 50 / 128
  29. De nition! Supporting domains A Supporting Domain (or Subdomain) are

    the areas of the business that play roles in making the Core Domain happen. Online Listings (put the car on the website) 50 / 128
  30. De nition! Supporting domains A Supporting Domain (or Subdomain) are

    the areas of the business that play roles in making the Core Domain happen. Online Listings (put the car on the website) Financial Transactions (charge the buyer, pay the seller) 50 / 128
  31. De nition! Supporting domains A Supporting Domain (or Subdomain) are

    the areas of the business that play roles in making the Core Domain happen. Online Listings (put the car on the website) Financial Transactions (charge the buyer, pay the seller) Optimization & Analytics (track business metrics) 50 / 128
  32. De nition! Supporting domains A Supporting Domain (or Subdomain) are

    the areas of the business that play roles in making the Core Domain happen. Online Listings (put the car on the website) Financial Transactions (charge the buyer, pay the seller) Optimization & Analytics (track business metrics) Customer Support (keep people happy) 50 / 128
  33. Take a step back You've discovered your domains! The Context

    Map points us to our business domains! These may map to real organization structures (Conway's Law) 53 / 128
  34. Ubiquitous Language The terms the business speaks within each domain!

    Put these in a Glossary Speak about them consistently in the team Use terms consistently in the code 54 / 128
  35. De nition! Bounded Context Concretely: a software system (like a

    codebase or running application) Linguistically: a delineation in your domain where concepts are "bounded", or contained 55 / 128
  36. For example: A Rating has a speci c meaning in

    the Inspection subdomain (a vehicle health score) ...but it means something else in the Customer Support subdomain (a customer support experience score) 56 / 128
  37. For example: A User in the Identity context is-- A

    Mechanic in the Inspection context, is-- A Seller or Buyer in the Financial Transaction context 57 / 128
  38. Bounded Contexts Precise language, precise models Your domains may now

    use their own speci c, nuanced terminology. Powerful modeling tool, and keeps your product in sync with the business stakeholders 58 / 128
  39. Contexts can expose domain entities defmodule AutoMaxx.Inspection do def get_vehicle!()

    do: ... def list_vehicles() do: ... def update_vehicle() do: ... def get_mechanic!() do: ... #... end 62 / 128
  40. Contexts can expose methods that perform domain actions defmodule AutoMaxx.Inspection

    do def add_vehicle_to_garage_queue(vehicle) do: ... def rate_vehicle(vehicle, mechanic, inspection_rating) do end 63 / 128
  41. defmodule AutoMaxx.VehicleInspectionRatingController do def update(conn, %{ user_id: user_id, vehicle_id: vehicle_id,

    rating: new_rating }) do with user <- Repo.get_by(User, user_id), vehicle <- Repo.get_by(Vehicle, vehicle_id) |> Repo.preload(:rating), :ok <- InspectionRatingPolicy.editable_by?(vehicle, user) do inspection_rating = vehicle.rating |> rating.changeset(%{value: new_rating}) |> Repo.insert!() render(conn, "show.html", inspection_rating: inspection_rating) else render(conn, "error.html", message: "Oops!") end end end 65 / 128
  42. defmodule AutoMaxx.VehicleInspectionRatingController do def update(conn, %{ user_id: user_id, vehicle_id: vehicle_id,

    rating: new_rating }) do with user <- Repo.get_by(User, user_id), vehicle <- Repo.get_by(Vehicle, vehicle_id) |> Repo.preload(:rating), :ok <- InspectionRatingPolicy.editable_by?(vehicle, user) do inspection_rating = vehicle.rating |> rating.changeset(%{value: new_rating}) |> Repo.insert!() render(conn, "show.html", inspection_rating: inspection_rating) else render(conn, "error.html", message: "Oops!") end end end 66 / 128
  43. defmodule AutoMaxx.VehicleInspectionRatingController do def update(conn, %{ user_id: user_id, vehicle_id: vehicle_id,

    rating: new_rating }) do with user <- Repo.get_by(User, user_id), vehicle <- Repo.get_by(Vehicle, vehicle_id) |> Repo.preload(:rating), :ok <- InspectionRatingPolicy.editable_by?(vehicle, user) do inspection_rating = vehicle.rating |> rating.changeset(%{value: new_rating}) |> Repo.insert!() render(conn, "show.html", inspection_rating: inspection_rating) else render(conn, "error.html", message: "Oops!") end end end 67 / 128
  44. Move domain entities into the Identity context # lib/automaxx/identity/user.ex defmodule

    AutoMaxx.Identity.User do schema "users" do: ... end Add domain action to the Identity context API # lib/automaxx/identity/identity.ex defmodule AutoMaxx.Identity do def get_user(user_id) do Repo.get_by(AutoMaxx.Identity.User, id: user_id) end end 68 / 128
  45. Move domain entities into the Inspection context # lib/automaxx/inspection/vehicle.ex defmodule

    AutoMaxx.Inspection.Vehicle do schema "vehicles" do belongs_to :owner, AutoMaxx.Identity.User has_one :rating, AutoMaxx.Inspection.Rating end end # lib/automaxx/inspection/rating.ex defmodule AutoMaxx.Inspection.Rating do schema "inspection_ratings" do belongs_to :vehicle, AutoMaxx.Inspection.Vehicle belongs_to :rated_by, AutoMaxx.Identity.User end end 69 / 128
  46. Add domain action to the Inspection context API defmodule AutoMaxx.Inspection

    do def rate_vehicle(vehicle, mechanic, inspection_rating) do with :ok <- Inspection.RatingPolicy.editable_by?(vehicle, mechanic) d Repo.get_by(Inspection.Vehicle, vehicle_id) |> Repo.preload(:rating) |> Inspection.Rating.changeset(%{ rated_by: mechanic.id, value: inspection_rating }) |> Repo.insert!() else {:auth_error, "Unauthorized"} end end end 70 / 128
  47. Simplify the controller with your new context APIs defmodule AutoMaxx.VehicleInspectionRatingController

    do def update(conn, %{ user_id: user_id, vehicle_id: vehicle_id, rating: new_rating }) do with user <- Identity.get_user(user_id), vehicle <- Inspection.get_vehicle(vehicle_id), {:ok, rating} <- Inspection.rate_vehicle(vehicle, user, new_rati render(conn, "show.html", inspection_rating: rating) else render(conn, "error.html", message: "Oops!") end end end 71 / 128
  48. What did you notice here? Contexts only expose methods at

    their outer layer. Contexts hide internal implementations. Use domain language when naming actions, entities and concepts. 73 / 128
  49. What happens when we need to use a User in

    a different context? Identity domain: User Inspection domain: Owner, Mechanic Marketing domain: Visitor 75 / 128
  50. A few options: 1. Direct Usage (Do Nothing): Just directly

    use schemas between contexts 76 / 128
  51. Sharing Concepts Direct Usage (Do Nothing) Mix domain models between

    contexts defmodule AutoMaxx.Inspection do def rate_vehicle(%AutoMaxx.Identity.User{} = user, vehicle, rating) do: ... end Maybe it's OK if you're refactoring, but I discourage this... 77 / 128
  52. A few options: 1. Direct Usage (Do Nothing): Just directly

    use schemas between contexts 2. Struct Conversion: convert to internal concepts at the boundaries with pure structs 78 / 128
  53. Convert a Identity.User to a Marketing.Visitor defmodule AutoMaxx.Marketing.Visitor do defstruct

    [:handle, :uuid] end defmodule AutoMaxx.Marketing do def visitor_for_user(%AutoMaxx.Identity.User{} = user) do new_mapping = user |> Map.from_struct() |> Map.delete(:email) |> Map.put(:handle, user.email) struct(AutoMaxx.Marketing.Visitor, new_mapping) end end 81 / 128
  54. Convert a Identity.User to a Marketing.Visitor defmodule AutoMaxx.Marketing.Visitor do defstruct

    [:handle, :uuid] end defmodule AutoMaxx.Marketing do def visitor_for_user(%AutoMaxx.Identity.User{} = user) do new_mapping = user |> Map.from_struct() |> Map.delete(:email) |> Map.put(:handle, user.email) struct(AutoMaxx.Marketing.Visitor, new_mapping) end end 82 / 128
  55. defmodule AutoMaxx.Marketing.EmailSubscriptionController do def create(conn, %{user_id: user_id, payload: payload}) do

    Identity.get_user(user_id) # Convert the concept at the boundaries |> Marketing.visitor_for_user() # Then proceed to perform the domain action |> Marketing.subscribe_visitor_to_mailchimp(%{payload: payload}) end end 83 / 128
  56. defmodule AutoMaxx.Marketing.EmailSubscriptionController do def create(conn, %{user_id: user_id, payload: payload}) do

    Identity.get_user(user_id) # Convert the concept at the boundaries |> Marketing.visitor_for_user() # Then proceed to perform the domain action |> Marketing.subscribe_visitor_to_mailchimp(%{payload: payload}) end end 84 / 128
  57. defmodule AutoMaxx.Marketing.EmailSubscriptionController do def create(conn, %{user_id: user_id, payload: payload}) do

    Identity.get_user(user_id) # Convert the concept at the boundaries |> Marketing.visitor_for_user() # Then proceed to perform the domain action |> Marketing.subscribe_visitor_to_mailchimp(%{payload: payload}) end end 85 / 128
  58. defmodule AutoMaxx.Marketing.EmailSubscriptionController do def create(conn, %{user_id: user_id, payload: payload}) do

    Identity.get_user(user_id) # Convert the concept at the boundaries |> Marketing.visitor_for_user() # Then proceed to perform the domain action |> Marketing.subscribe_visitor_to_mailchimp(%{payload: payload}) end end 86 / 128
  59. Pattern matching and types Strict type guarantees with pattern matching

    def subscribe_visitor_to_mailchimp(%Visitor{} = visitor) do def visitor_for_user(%User{} = user) do... 87 / 128
  60. Pattern matching and types Strict type guarantees with pattern matching

    def subscribe_visitor_to_mailchimp(%Visitor{} = visitor) do def visitor_for_user(%User{} = user) do... Even better - leverage the powers of typespecs. @spec subscribe_visitor_to_mailchimp(%Visitor{}) :: boolean @spec visitor_for_user(%User{}) :: %Visitor{} 87 / 128
  61. A few options: 1. Direct Usage (Do Nothing): Just directly

    use schemas between contexts 2. Struct Conversion: convert to internal concepts at the boundaries with pure structs 3. Collaborator Schema: Create an internal schema persisted in Ecto that uses a reference to the external schema 88 / 128
  62. Sharing Concepts Collaborator Schema Useful for Read+Write use cases defmodule

    AutoMaxx.Inspection.Mechanic do schema "mechanics" do field :is_contractor, :boolean field :certification, :string belongs_to :user, AutoMaxx.Identity.User end end 89 / 128
  63. defmodule AutoMaxx.Inspection do def create_mechanic_from_user( %AutoMaxx.Identity.User{} = user, is_contractor )

    do %AutoMaxx.Inspection.Mechanic{ is_contractor: is_contractor, user: user } |> Repo.insert!() end def mechanic_for_user(user) do Repo.get_by(AutoMaxx.Inspection.Mechanic, user_id: user.id) end end 90 / 128
  64. Sharing Concepts Avoid cross-context joins if you can defmodule AutoMaxx.Marketing.Visitor

    do: schema "marketing_visitors" do: belongs_to :user, AutoMaxx.Identity.User 92 / 128
  65. Sharing Concepts Avoid cross-context joins if you can defmodule AutoMaxx.Marketing.Visitor

    do: schema "marketing_visitors" do: belongs_to :user, AutoMaxx.Identity.User Instead, store them as external references field :user_id, :string field :user_id, :uuid 92 / 128
  66. DON'T defmodule AutoMaxx.Inspection do def get_vehicle() do: # => %Vehicle{}

    def get_vehicle_rating() do: # => %Rating{} def get_vehicle_list_price() do: # => %ListPrice{} end 97 / 128
  67. DON'T defmodule AutoMaxx.Inspection do def get_vehicle() do: # => %Vehicle{}

    def get_vehicle_rating() do: # => %Rating{} def get_vehicle_list_price() do: # => %ListPrice{} end DO defmodule AutoMaxx.Inspection do def get_vehicle() do: end # => %Vehicle{rating: %Rating{}, list_price: %ListPrice{}} 97 / 128
  68. DON'T defmodule AutoMaxx.Inspection do def update_rating(rating_id, new_rating) do... end DO

    defmodule AutoMaxx.Inspection do def rate_vehicle(vehicle_id, new_rating) do: ... end 98 / 128
  69. Leverage Aggregate Roots Expose only Aggregate Roots in your context.

    Passing Aggregates around simpli es your APIs 99 / 128
  70. Where is the coupling here? # lib/automaxx/identity/identity.ex defmodule AutoMaxx.Identity do

    def create_user(...) do do_create_user() |> AutoMaxx.Inspection.maybe_create_mechanic() |> AutoMaxx.Marketing.subscribe_user_to_email_list() |> AutoMaxx.Analytics.track_event('user_created') end end 101 / 128
  71. Where is the coupling here? # lib/automaxx/identity/identity.ex defmodule AutoMaxx.Identity do

    def create_user(...) do do_create_user() |> AutoMaxx.Inspection.maybe_create_mechanic() |> AutoMaxx.Marketing.subscribe_user_to_email_list() |> AutoMaxx.Analytics.track_event('user_created') end end 102 / 128
  72. Where is the coupling here? # lib/automaxx/identity/identity.ex defmodule AutoMaxx.Identity do

    def create_user(...) do do_create_user() |> AutoMaxx.Inspection.maybe_create_mechanic() |> AutoMaxx.Marketing.subscribe_user_to_email_list() |> AutoMaxx.Analytics.track_event('user_created') end end 103 / 128
  73. Where is the coupling here? # lib/automaxx/identity/identity.ex defmodule AutoMaxx.Identity do

    def create_user(...) do do_create_user() |> AutoMaxx.Inspection.maybe_create_mechanic() |> AutoMaxx.Marketing.subscribe_user_to_email_list() |> AutoMaxx.Analytics.track_event('user_created') end end 104 / 128
  74. Publish facts (domain events) over a bus Interested contexts can

    subscribe to domain events Refer back to your Context Map and Glossary where your business stakeholders helped you de ne events! 106 / 128
  75. Publisher: Identity context publishes identity.user.created Subscribers: Marketing context puts the

    user on the email marketing list Inspection context creates a corresponding Mechanic if the user is one Analytics context publishes metric to Google Analytics 107 / 128
  76. Using the event_bus library Github: otobus/event_bus Or any other system

    that gives you pub/sub capabilities 109 / 128
  77. # lib/automaxx/identity/identity.ex defmodule AutoMaxx.Identity do def create_user(...) do do_create_user() |>

    publish_event(:"identity.user.created") end use EventBus.EventSource defp publish_event(%User{} = user, event_name) do EventSource.notify %{id: UUID.uuid4(), topic: event_name} do %{user: user} end end end 110 / 128
  78. # lib/automaxx/identity/identity.ex defmodule AutoMaxx.Identity do def create_user(...) do do_create_user() |>

    publish_event(:"identity.user.created") end use EventBus.EventSource defp publish_event(%User{} = user, event_name) do EventSource.notify %{id: UUID.uuid4(), topic: event_name} do %{user: user} end end end 111 / 128
  79. In each context, create an EventHandler module. # lib/automaxx/marketing/event_handler.ex defmodule

    AutoMaxx.Marketing.EventHandler do def process({:"identity.user.created" = event_name, event_id}) do %{data: %{user: user}} = EventBus.fetch_event({event_name, event_id}) AutoMaxx.Marketing.subscribe_user_to_email_list(user) end def process({:"some.other.event" = event_name, event_id}) do: ... end 112 / 128
  80. In each context, create an EventHandler module. # lib/automaxx/marketing/event_handler.ex defmodule

    AutoMaxx.Marketing.EventHandler do def process({:"identity.user.created" = event_name, event_id}) do %{data: %{user: user}} = EventBus.fetch_event({event_name, event_id}) AutoMaxx.Marketing.subscribe_user_to_email_list(user) end def process({:"some.other.event" = event_name, event_id}) do: ... end 113 / 128
  81. # lib/automaxx/inspection/event_handler.ex defmodule AutoMaxx.Inspection.EventHandler do def process({:"identity.user.created" = event_name, event_id})

    do %{data: %{user: user}} = EventBus.fetch_event({event_name, event_id}) AutoMaxx.Inspection.create_mechanic_from_user(user) end def process({:"yet.another.event" = event_name, event_id}) do: ... end 114 / 128
  82. Caveats DDD concepts rst, then patterns Start with the Context

    Map. Make sure you understand the core concepts about linguistic clarity. Keep the team and the code speaking the same language! 117 / 128
  83. Caveats For systems of a certain scale If you have

    a small-scale system, this might be overkill! 118 / 128
  84. Used the full power of the BEAM Lean on message

    passing, OTP, pattern matching and typespecs 124 / 128
  85. Credits & Prior Art Evans, Eric. Domain-Driven Design: Tackling Complexity

    in the Heart of Software. Gorodinski, Lev. "Sub-domains and Bounded Contexts in Domain-Driven Design (DDD)". Hagemann, Stephan. Component-Based Rails Applications. Parnas, D.L. "On the Criteria To Be Used in Decomposing Systems into Modules". Vernon, Vaughan. Implementing Domain-Driven Design. 127 / 128
  86. Credits & Prior Art W. P. Stevens ; G. J.

    Myers ; L. L. Constantine. "Structured Design" - IBM Systems Journal, Vol 13 Issue 2, 1974. Steinegger, Giessler, Hippchen, Abeck. Overview of a Domain-Driven Design Approach to Build Microservice-Based Applications Rob Martin - Perhap: Applying DDD and Reactive Architectures: https://www.youtube.com/watch? list=PLqj39LCvnOWZMVugtyKlHMF1o2zPNntFL&time_continue=5&v=kq4qT c 128 / 128