Modular Design in Elixir

Modular Design in Elixir

A talk about modular software design given at 9th Poznań Elixir Meetup

373dd7c51433dc3c38436dcfdec79cdc?s=128

Maciej Kaszubowski

November 08, 2018
Tweet

Transcript

  1. mkaszubowski94

  2. mkaszubowski94

  3. mkaszubowski94 Maciej Kaszubowski THE BIG BALL OF NOUNS

  4. mkaszubowski94 Maciej Kaszubowski MODULAR DESIGN IN ELIXIR

  5. mkaszubowski94 Functional Programming

  6. mkaszubowski94 immutability no side effects explicit arguments

  7. mkaszubowski94 immutability no side effects explicit arguments simpler code

  8. mkaszubowski94 and yet…

  9. mkaszubowski94 time development progress complexity

  10. mkaszubowski94 why is that?

  11. mkaszubowski94 f(x) explicit input explicit output what we want

  12. mkaszubowski94 f(x) input output what we want g(x) output

  13. mkaszubowski94 input output what we have

  14. mkaszubowski94 This means a single table (or a small group

    of them), *not* the entire DB ( )
  15. mkaszubowski94

  16. mkaszubowski94

  17. mkaszubowski94

  18. mkaszubowski94

  19. mkaszubowski94 on a higher level, our systems are not very

    functional
  20. mkaszubowski94 Database mutable state shared dependency complexity

  21. mkaszubowski94

  22. mkaszubowski94 Object Object Object Object Object Object SHARED STATE SHARED

    STATE
  23. mkaszubowski94 if we don’t accept shared state for simple objects,

    why it’s ok to do this on a system level?
  24. mkaszubowski94 “In functional programming, you don’t need any design patterns”

  25. mkaszubowski94 let’s not forget the good parts of Object Oriented

    design
  26. mkaszubowski94 what can we do about it?

  27. mkaszubowski94 https://www.confluent.io/blog/event-sourcing-cqrs-stream-processing-apache-kafka-whats-connection/ CQRS

  28. mkaszubowski94 Domain Driven Design

  29. mkaszubowski94 Actor Model https://www.brianstorti.com/the-actor-model/

  30. mkaszubowski94 these are risky

  31. mkaszubowski94 relational DB is still the safest solution

  32. mkaszubowski94 Maciej Kaszubowski Modular Software Design

  33. mkaszubowski94 1972

  34. mkaszubowski94 One big problem

  35. mkaszubowski94

  36. mkaszubowski94

  37. mkaszubowski94

  38. mkaszubowski94 Modular design smaller, simpler modules strong boundaries low coupling

  39. mkaszubowski94 Each module can be understood in isolation modified in

    isolation tested in isolation refactored in isolation
  40. mkaszubowski94 ok, but how?

  41. mkaszubowski94 Maciej Kaszubowski Example Project

  42. mkaszubowski94 create a new job name image url price publish

    John Alchemist create new job manage jobs find a job logout description date
  43. mkaszubowski94 your jobs John Alchemist create new job manage jobs

    find a job logout my awesome job accept my other job cancel accept cancel yet another one publish and one more accepted this didn’t go well canceled
  44. mkaszubowski94 browse jobs John Alchemist create new job manage jobs

    find a job logout my awesome job my other job yet another one and one more filters 100$ 50$ 3500$ 99$ xxxx yyy zzz search
  45. mkaszubowski94 start this job John Alchemist create new job manage

    jobs find a job logout my awesome job Don’t ask, you’ll get paid start a job https://example.com/jobs/1234
  46. mkaszubowski94 defmodule ModularElixir.Job do use Ecto.Schema schema "jobs" do field

    :name, :string field :description, :string field :image_url, :string field :price, :integer field :datetime, :utc_datetime belongs_to :creator, ModularElixir.User belongs_to :contractor, ModularElixir.User end end
  47. mkaszubowski94 JobService - fetch_by_id() - search() - find_by_user_id() - accept()

    - cancel() - publish()
  48. mkaszubowski94 create a new job name image url price continue

    John Alchemist create new job manage jobs find a job logout description date
  49. mkaszubowski94 pay for job pay 110$ John Alchemist create new

    job manage jobs find a job logout my awesome job job price………….………..100$ fee (10%)…………………….10$ total……………………………..110$ Don’t ask, you’ll get paid Your job will be published after the payment
  50. mkaszubowski94 defmodule ModularElixir.Job do use Ecto.Schema schema "jobs" do field

    :name, :string field :description, :string field :image_url, :string field :paid, :boolean field :price, :integer field :datetime, :utc_datetime belongs_to :creator, ModularElixir.User belongs_to :contractor, ModularElixir.User end end
  51. mkaszubowski94 JobService - fetch_by_id() - search() - find_by_user_id() - accept()

    - cancel() - publish() - create()
  52. mkaszubowski94 start this job John Alchemist create new job manage

    jobs find a job logout my awesome job Don’t ask, you’ll get paid start a job https://example.com/jobs/my-awesome-job
  53. mkaszubowski94

  54. mkaszubowski94 JobService - fetch_by_id() - search() - find_by_user_id() - accept()

    - cancel() - publish() - create() - find_by_slug()
  55. mkaszubowski94 browse jobs John Alchemist create new job manage jobs

    find a job logout my awesome job my other job yet another one and one more filters 100$ 50$ 3500$ 99$ xxxx yyy zzz search Upcoming job Job starting in 54 minutes ! x
  56. mkaszubowski94 defmodule ModularElixir.Job do use Ecto.Schema schema "jobs" do field

    :name, :string field :description, :string field :image_url, :string field :paid, :boolean field :slug, :string field :started, :boolean field :price, :integer field :datetime, :utc_datetime belongs_to :creator, ModularElixir.User belongs_to :contractor, ModularElixir.User end end
  57. mkaszubowski94 JobService - fetch_by_id() - search() - find_by_user_id() - accept()

    - cancel() - publish() - create() - find_by_slug() - send_reminder()
  58. mkaszubowski94 jobs JobService UserService users join (jobs_count) Payments preload jobs

    Chat Service JobSubtask Service
  59. mkaszubowski94 jobs JobService UserService users join (jobs_count) Payments JobSubtask Service

    preload jobs Chat Service JOB USER CONVERSATION MESSAGE PAYMENT SUBTASK
  60. mkaszubowski94 In most projects: • few nouns • a lot

    of interactions between them • logic is organised around the nouns • a lot of coupling as a result
  61. mkaszubowski94 nouns are not universal

  62. mkaszubowski94 defmodule ModularElixir.Job do use Ecto.Schema schema "jobs" do field

    :name, :string field :description, :string field :image_url, :string field :paid, :boolean field :slug, :string field :started, :boolean field :price, :integer field :datetime, :utc_datetime belongs_to :creator, ModularElixir.User belongs_to :contractor, ModularElixir.User end end
  63. mkaszubowski94 Maciej Kaszubowski Extracting Behaviour

  64. mkaszubowski94 the most important question: what is the desired outcome?

  65. mkaszubowski94 1. what is the desired outcome? (what should happen?)

    2. what data do I need to do this? 3. how can I get this data? 4. what should be exposed in the interface
  66. mkaszubowski94 then, pull this out to a new, isolated module

  67. mkaszubowski94 desired outcome module

  68. mkaszubowski94 desired outcome module required data

  69. mkaszubowski94 desired outcome module required data data source

  70. mkaszubowski94 desired outcome module required data data source interface

  71. mkaszubowski94 Maciej Kaszubowski Extracting Slugs

  72. mkaszubowski94 start this job John Alchemist create new job manage

    jobs find a job logout my awesome job Don’t ask, you’ll get paid start a job https://example.com/jobs/my-awesome-job
  73. mkaszubowski94 desired outcome: jobs should be accessible via human-friendly slugs

  74. mkaszubowski94 required data: id-slug mapping for jobs

  75. mkaszubowski94 SEO Context

  76. mkaszubowski94 SEO Context slugs

  77. mkaszubowski94 SEO Context slugs - register_slug() - get_id_by_slug() - get_current_slug()

    Interface:
  78. mkaszubowski94 SEO Context - register_slug() - fetch_id_by_slug() - fetch_slug_by_id() slug

    resource_id resource_type my-awesome-job 1 job 1 1 job other-job 2 job 2 2 job slugs
  79. mkaszubowski94 def show(conn, %{"job_id"  slug}) do job = slug

     SeoContext.fetch_id_by_slug("job")  JobService.fetch_by_id() #  end
  80. mkaszubowski94 def show(conn, %{"job_id"  slug}) do job = slug

     SeoContext.fetch_id_by_slug("job")  JobService.fetch_by_id() #  end Plug?
  81. mkaszubowski94 defmodule ModularElixir.Job do use Ecto.Schema schema "jobs" do field

    :name, :string field :description, :string field :image_url, :string field :paid, :boolean field :slug, :string field :started, :boolean field :price, :integer field :datetime, :utc_datetime belongs_to :creator, ModularElixir.User belongs_to :contractor, ModularElixir.User end end
  82. mkaszubowski94 JobService - fetch_by_id() - search() - find_by_user_id() - accept()

    - cancel() - publish() - create() - find_by_slug() - send_reminder()
  83. mkaszubowski94 SEO Context • multiple slugs - easy to do

    • slugs for other resources - easy to do • slugs are resolved at the controller level • no slugs in domain logic
  84. mkaszubowski94 no slugs in domain logic /api/jobs/my-awesome-job/subtasks from( s in

    Subtask, join: j in Job, on: s.job_id  j.id where: j.id  ^job_id, # … ) /api/jobs/1/subtasks
  85. mkaszubowski94 Maciej Kaszubowski Extracting Notifications

  86. mkaszubowski94 browse jobs John Alchemist create new job manage jobs

    find a job logout my awesome job my other job yet another one and one more filters 100$ 50$ 3500$ 99$ xxxx yyy zzz search Upcoming job Job starting in 54 minutes ! x
  87. mkaszubowski94 desired outcome: notification is sent when a job is

    starting soon
  88. mkaszubowski94 required data: job information device information

  89. mkaszubowski94 Job Reminders upcoming_jobs devices

  90. mkaszubowski94 Job Reminders upcoming_jobs - put_job() - remove_job() - register_device()

    devices Interface:
  91. mkaszubowski94 Job Reminders upcoming_jobs - put_job() - remove_job() - register_device()

    devices Interface: Would probably require a transaction
  92. mkaszubowski94 - register_device() Interface: Job Service search Job Reminders sent_reminders

    devices
  93. mkaszubowski94 JobService - fetch_by_id() - search() - find_by_user_id() - accept()

    - cancel() - publish() - create() - find_by_slug() - send_reminder()
  94. mkaszubowski94 defmodule Notifications.Worker do use GenServer def handle_info(:work, state) do

    Notifications.send_notifications(now()) Process.send_after(self(), :work, @interval) {:noreply, state) end end
  95. mkaszubowski94 defmodule Notifications do defmodule Logic do def send(jobs, datetime,

    already_sent) do Enum.each(jobs, fn job -> if starting_soon?(job) && not_sent_yet?(job, already_sent) do send_notification(job) end end) end end def send_notifications(datetime) do jobs = JobBoard.search(…) already_sent = fetch_already_sent() :ok = Logic.send(jobs, datetime, already_sent) end end
  96. mkaszubowski94 Logic desired outcome Notifications Job Board required data data

    source
  97. mkaszubowski94 logic is isolated from the data source

  98. mkaszubowski94 Logic desired outcome Notifications required data events stream

  99. mkaszubowski94 Logic desired outcome Notifications required data

  100. mkaszubowski94 Logic desired outcome Notifications required data TESTS

  101. mkaszubowski94 Maciej Kaszubowski Extracting Publishing

  102. mkaszubowski94 JobService - fetch_by_id() - search() - find_by_user_id() - accept()

    - cancel() - publish() - create() - find_by_slug() - send_reminder()
  103. mkaszubowski94 problem: the same data model is used for reading

    and writing
  104. mkaszubowski94 Job Publishing jobs Job Board jobs publish

  105. mkaszubowski94 Job Publishing jobs Job Board jobs publish enforces job

    structure in params
  106. mkaszubowski94 defmodule JobBoard do defmodule Job do @keys [:name, :description,

    ] @enforce_keys @keys defstruct @keys end def publish(%Job{} = job) do #  end end
  107. mkaszubowski94 Job Publishing jobs Job Board jobs publish enforces job

    structure in params decides when the job is published
  108. mkaszubowski94 Job Publishing jobs Job Board jobs upsert semantics decides

    what are the rules for editing a job publish
  109. mkaszubowski94 defmodule ModularElixir.JobBoard.Job do use Ecto.Schema @schema_prefix “job_board” schema "jobs"

    do field :name, :string field :description, :string field :image_url, :string field :paid, :boolean field :slug, :string field :started, :boolean field :price, :integer field :datetime, :utc_datetime belongs_to :creator, ModularElixir.User belongs_to :contractor, ModularElixir.User end end
  110. mkaszubowski94 JobService - fetch_by_id() - search() - find_by_user_id() - accept()

    - cancel() - publish() - create() - find_by_slug() - send_reminder()
  111. mkaszubowski94 JobService - fetch_by_id() - search() - find_by_user_id() - accept()

    - cancel() - publish() - create() - find_by_slug() - send_reminder()
  112. mkaszubowski94 JobBoard - fetch_by_id() - search() - find_by_user_id() - accept()

    - cancel() - publish() - create() - find_by_slug() - send_reminder()
  113. mkaszubowski94 Job Publishing - create() - get_drafts() - publish()

  114. mkaszubowski94 JobBoard • can be tested in isolation (without mocking

    payments) • linear logic (without conditionals) • no risk of showing unpublished jobs
  115. mkaszubowski94 JobPublishing • rules for publishing can be changed in

    isolation • can be used alongside other modules (e.g. Admins can publish jobs without paying)
  116. mkaszubowski94 Job Board desired outcome JobBoard params are enforced by

    the interface Job Publishing
  117. mkaszubowski94 Job Board desired outcome JobBoard params are enforced by

    the interface Admin
  118. mkaszubowski94 Job Board desired outcome JobBoard params are enforced by

    the interface TESTS
  119. mkaszubowski94 JobBoard - fetch_by_id() - search() - find_by_user_id() - accept()

    - cancel() - publish() - create() - find_by_slug() - send_reminder()
  120. mkaszubowski94 JobBoard jobs Ongoing Jobs jobs

  121. mkaszubowski94 defmodule ModularElixir.JobBoard.Job do use Ecto.Schema @schema_prefix “job_board” schema "jobs"

    do field :name, :string field :description, :string field :image_url, :string field :paid, :boolean field :slug, :string field :started, :boolean field :price, :integer field :datetime, :utc_datetime belongs_to :creator, ModularElixir.User belongs_to :contractor, ModularElixir.User end end
  122. mkaszubowski94 JobBoard - fetch_by_id() - search() - find_by_user_id() - accept()

    - cancel() - publish() - create() - find_by_slug() - send_reminder()
  123. mkaszubowski94 Maciej Kaszubowski Outcomes

  124. mkaszubowski94 JobService jobs

  125. mkaszubowski94 JobService • multiple responsibilities • hard to change and

    understand • hard to test in isolation • grows with each new feature • hard to delete old features
  126. mkaszubowski94 Job Publishing jobs Job Board jobs Reminders devices SEO

    slugs Ongoing Jobs jobs Job Archive jobs
  127. mkaszubowski94 Smaller modules • clear responsibilities for each module •

    easy to change, refactor, understand • easy to test in isolation • tend to stay small • trivial to delete if no longer necessary
  128. mkaszubowski94 focused on behaviour, not nouns

  129. mkaszubowski94 nouns are not universal

  130. mkaszubowski94 defmodule ModularElixir.Job do use Ecto.Schema schema "jobs" do field

    :name, :string field :description, :string field :image_url, :string field :paid, :boolean field :slug, :string field :started, :boolean field :price, :integer field :datetime, :utc_datetime belongs_to :creator, ModularElixir.User belongs_to :contractor, ModularElixir.User end end
  131. mkaszubowski94 a “job” means something else in each module

  132. mkaszubowski94 by splitting this, we make this explicit, rather than

    relying on some values
  133. mkaszubowski94 the flow of the app is visible in the

    module structure and dependencies
  134. mkaszubowski94 Job Publishing Job Board Ongoing Jobs Job Archive Reminders

    Payments
  135. mkaszubowski94 the logic is more linear

  136. mkaszubowski94 there are no (or fewer) dependency cycles

  137. mkaszubowski94 the module hierarchy is much clearer

  138. mkaszubowski94 Maciej Kaszubowski Criteria for modularising design

  139. mkaszubowski94 start with a list of important decisions

  140. mkaszubowski94 Important decisions • what are the rules for publishing

    jobs? • when/what notifications are sent? • which jobs are visible on job board? • how is a contractor chosen for each job? • what happens after a job is started? when the job ends? • how payments work? (when creator pays for job, how the contractor receives the money? do we allow coupons? credit cards? money transfers?) • …
  141. mkaszubowski94 then, create a module for each decision

  142. mkaszubowski94 Make each module • independent • easy to test

    • easy to understand • with a strong interface
  143. mkaszubowski94 focus on behaviour, not data

  144. mkaszubowski94 Maciej Kaszubowski Summary

  145. mkaszubowski94 is this perfect?

  146. mkaszubowski94 nope

  147. mkaszubowski94 is this the best way?

  148. mkaszubowski94 I have no idea

  149. mkaszubowski94 it works for me

  150. mkaszubowski94 but the main point is to start a discussion

  151. mkaszubowski94 so argue with me over pizza

  152. mkaszubowski94 Maciej Kaszubowski THANKS!