Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Modular Design in Elixir

Modular Design in Elixir

A talk about modular software design given at 9th Poznań Elixir Meetup

Maciej Kaszubowski

November 08, 2018
Tweet

More Decks by Maciej Kaszubowski

Other Decks in Programming

Transcript

  1. mkaszubowski94

    View full-size slide

  2. mkaszubowski94

    View full-size slide

  3. mkaszubowski94
    Maciej Kaszubowski
    THE BIG BALL
    OF NOUNS

    View full-size slide

  4. mkaszubowski94
    Maciej Kaszubowski
    MODULAR
    DESIGN IN ELIXIR

    View full-size slide

  5. mkaszubowski94
    Functional Programming

    View full-size slide

  6. mkaszubowski94
    immutability
    no side effects
    explicit arguments

    View full-size slide

  7. mkaszubowski94
    immutability
    no side effects
    explicit arguments
    simpler code

    View full-size slide

  8. mkaszubowski94
    and yet…

    View full-size slide

  9. mkaszubowski94
    time
    development
    progress
    complexity

    View full-size slide

  10. mkaszubowski94
    why is that?

    View full-size slide

  11. mkaszubowski94
    f(x)
    explicit input explicit output
    what we want

    View full-size slide

  12. mkaszubowski94
    f(x)
    input output
    what we want
    g(x)
    output

    View full-size slide

  13. mkaszubowski94
    input output
    what we have

    View full-size slide

  14. mkaszubowski94
    This means a single table
    (or a small group of them),
    *not* the entire DB
    ( )

    View full-size slide

  15. mkaszubowski94

    View full-size slide

  16. mkaszubowski94

    View full-size slide

  17. mkaszubowski94

    View full-size slide

  18. mkaszubowski94

    View full-size slide

  19. mkaszubowski94
    on a higher level, our
    systems are not very
    functional

    View full-size slide

  20. mkaszubowski94
    Database
    mutable state
    shared dependency
    complexity

    View full-size slide

  21. mkaszubowski94

    View full-size slide

  22. mkaszubowski94
    Object
    Object
    Object
    Object
    Object
    Object
    SHARED
    STATE
    SHARED
    STATE

    View full-size slide

  23. mkaszubowski94
    if we don’t accept shared state
    for simple objects, why it’s ok
    to do this on a system level?

    View full-size slide

  24. mkaszubowski94
    “In functional
    programming, you don’t
    need any design patterns”

    View full-size slide

  25. mkaszubowski94
    let’s not forget the good
    parts of Object
    Oriented design

    View full-size slide

  26. mkaszubowski94
    what can we do
    about it?

    View full-size slide

  27. mkaszubowski94
    https://www.confluent.io/blog/event-sourcing-cqrs-stream-processing-apache-kafka-whats-connection/
    CQRS

    View full-size slide

  28. mkaszubowski94
    Domain Driven Design

    View full-size slide

  29. mkaszubowski94
    Actor Model
    https://www.brianstorti.com/the-actor-model/

    View full-size slide

  30. mkaszubowski94
    these are risky

    View full-size slide

  31. mkaszubowski94
    relational DB is still
    the safest solution

    View full-size slide

  32. mkaszubowski94
    Maciej Kaszubowski
    Modular Software
    Design

    View full-size slide

  33. mkaszubowski94
    1972

    View full-size slide

  34. mkaszubowski94
    One big
    problem

    View full-size slide

  35. mkaszubowski94

    View full-size slide

  36. mkaszubowski94

    View full-size slide

  37. mkaszubowski94

    View full-size slide

  38. mkaszubowski94
    Modular design
    smaller, simpler modules
    strong boundaries
    low coupling

    View full-size slide

  39. mkaszubowski94
    Each module can be
    understood in isolation
    modified in isolation
    tested in isolation
    refactored in isolation

    View full-size slide

  40. mkaszubowski94
    ok, but how?

    View full-size slide

  41. mkaszubowski94
    Maciej Kaszubowski
    Example Project

    View full-size slide

  42. mkaszubowski94
    create a new job
    name
    image url
    price
    publish
    John Alchemist
    create new job
    manage jobs
    find a job
    logout
    description
    date

    View full-size slide

  43. mkaszubowski94
    your jobs
    John Alchemist
    create new job
    manage jobs
    find a job
    logout
    my awesome job accept
    my other job
    cancel
    accept cancel
    yet another one
    publish
    and one more accepted
    this didn’t go well canceled

    View full-size slide

  44. mkaszubowski94
    browse jobs
    John Alchemist
    create new job
    manage jobs
    find a job
    logout
    my awesome job
    my other job
    yet another one
    and one more
    filters
    100$
    50$
    3500$
    99$
    xxxx
    yyy
    zzz
    search

    View full-size slide

  45. mkaszubowski94
    start this job
    John Alchemist
    create new job
    manage jobs
    find a job
    logout
    my awesome job
    Don’t ask, you’ll get paid
    start a job
    https://example.com/jobs/1234

    View full-size slide

  46. mkaszubowski94
    defmodule ModularElixir.Job do
    use Ecto.Schema
    schema "jobs" do
    field :name, :string
    field :description, :string
    field :image_url, :string
    field :price, :integer
    field :datetime, :utc_datetime
    belongs_to :creator, ModularElixir.User
    belongs_to :contractor, ModularElixir.User
    end
    end

    View full-size slide

  47. mkaszubowski94
    JobService
    - fetch_by_id()
    - search()
    - find_by_user_id()
    - accept()
    - cancel()
    - publish()

    View full-size slide

  48. mkaszubowski94
    create a new job
    name
    image url
    price
    continue
    John Alchemist
    create new job
    manage jobs
    find a job
    logout
    description
    date

    View full-size slide

  49. mkaszubowski94
    pay for job
    pay 110$
    John Alchemist
    create new job
    manage jobs
    find a job
    logout
    my awesome job
    job price………….………..100$
    fee (10%)…………………….10$
    total……………………………..110$
    Don’t ask, you’ll get paid
    Your job will be published after the payment

    View full-size slide

  50. mkaszubowski94
    defmodule ModularElixir.Job do
    use Ecto.Schema
    schema "jobs" do
    field :name, :string
    field :description, :string
    field :image_url, :string
    field :paid, :boolean
    field :price, :integer
    field :datetime, :utc_datetime
    belongs_to :creator, ModularElixir.User
    belongs_to :contractor, ModularElixir.User
    end
    end

    View full-size slide

  51. mkaszubowski94
    JobService
    - fetch_by_id()
    - search()
    - find_by_user_id()
    - accept()
    - cancel()
    - publish()
    - create()

    View full-size slide

  52. mkaszubowski94
    start this job
    John Alchemist
    create new job
    manage jobs
    find a job
    logout
    my awesome job
    Don’t ask, you’ll get paid
    start a job
    https://example.com/jobs/my-awesome-job

    View full-size slide

  53. mkaszubowski94

    View full-size slide

  54. mkaszubowski94
    JobService
    - fetch_by_id()
    - search()
    - find_by_user_id()
    - accept()
    - cancel()
    - publish()
    - create()
    - find_by_slug()

    View full-size slide

  55. mkaszubowski94
    browse jobs
    John Alchemist
    create new job
    manage jobs
    find a job
    logout
    my awesome job
    my other job
    yet another one
    and one more
    filters
    100$
    50$
    3500$
    99$
    xxxx
    yyy
    zzz
    search
    Upcoming job
    Job starting in 54 minutes
    ! x

    View full-size slide

  56. mkaszubowski94
    defmodule ModularElixir.Job do
    use Ecto.Schema
    schema "jobs" do
    field :name, :string
    field :description, :string
    field :image_url, :string
    field :paid, :boolean
    field :slug, :string
    field :started, :boolean
    field :price, :integer
    field :datetime, :utc_datetime
    belongs_to :creator, ModularElixir.User
    belongs_to :contractor, ModularElixir.User
    end
    end

    View full-size slide

  57. mkaszubowski94
    JobService
    - fetch_by_id()
    - search()
    - find_by_user_id()
    - accept()
    - cancel()
    - publish()
    - create()
    - find_by_slug()
    - send_reminder()

    View full-size slide

  58. mkaszubowski94
    jobs
    JobService
    UserService
    users
    join (jobs_count)
    Payments
    preload jobs
    Chat
    Service
    JobSubtask
    Service

    View full-size slide

  59. mkaszubowski94
    jobs
    JobService
    UserService
    users
    join (jobs_count)
    Payments
    JobSubtask
    Service
    preload jobs
    Chat
    Service
    JOB
    USER
    CONVERSATION
    MESSAGE
    PAYMENT
    SUBTASK

    View full-size slide

  60. mkaszubowski94
    In most projects:
    • few nouns
    • a lot of interactions between them
    • logic is organised around the nouns
    • a lot of coupling as a result

    View full-size slide

  61. mkaszubowski94
    nouns are not
    universal

    View full-size slide

  62. mkaszubowski94
    defmodule ModularElixir.Job do
    use Ecto.Schema
    schema "jobs" do
    field :name, :string
    field :description, :string
    field :image_url, :string
    field :paid, :boolean
    field :slug, :string
    field :started, :boolean
    field :price, :integer
    field :datetime, :utc_datetime
    belongs_to :creator, ModularElixir.User
    belongs_to :contractor, ModularElixir.User
    end
    end

    View full-size slide

  63. mkaszubowski94
    Maciej Kaszubowski
    Extracting Behaviour

    View full-size slide

  64. mkaszubowski94
    the most important question:
    what is the desired outcome?

    View full-size slide

  65. mkaszubowski94
    1. what is the desired outcome?
    (what should happen?)
    2. what data do I need to do this?
    3. how can I get this data?
    4. what should be exposed in the interface

    View full-size slide

  66. mkaszubowski94
    then, pull this out to a new,
    isolated module

    View full-size slide

  67. mkaszubowski94
    desired
    outcome
    module

    View full-size slide

  68. mkaszubowski94
    desired
    outcome
    module
    required
    data

    View full-size slide

  69. mkaszubowski94
    desired
    outcome
    module
    required
    data
    data
    source

    View full-size slide

  70. mkaszubowski94
    desired
    outcome
    module
    required
    data
    data
    source
    interface

    View full-size slide

  71. mkaszubowski94
    Maciej Kaszubowski
    Extracting Slugs

    View full-size slide

  72. mkaszubowski94
    start this job
    John Alchemist
    create new job
    manage jobs
    find a job
    logout
    my awesome job
    Don’t ask, you’ll get paid
    start a job
    https://example.com/jobs/my-awesome-job

    View full-size slide

  73. mkaszubowski94
    desired outcome:
    jobs should be accessible via
    human-friendly slugs

    View full-size slide

  74. mkaszubowski94
    required data:
    id-slug mapping for jobs

    View full-size slide

  75. mkaszubowski94
    SEO
    Context

    View full-size slide

  76. mkaszubowski94
    SEO
    Context
    slugs

    View full-size slide

  77. mkaszubowski94
    SEO
    Context
    slugs
    - register_slug()
    - get_id_by_slug()
    - get_current_slug()
    Interface:

    View full-size slide

  78. mkaszubowski94
    SEO
    Context
    - register_slug()
    - fetch_id_by_slug()
    - fetch_slug_by_id()
    slug resource_id resource_type
    my-awesome-job 1 job
    1 1 job
    other-job 2 job
    2 2 job
    slugs

    View full-size slide

  79. mkaszubowski94
    def show(conn, %{"job_id"  slug}) do
    job =
    slug
     SeoContext.fetch_id_by_slug("job")
     JobService.fetch_by_id()
    # 
    end

    View full-size slide

  80. mkaszubowski94
    def show(conn, %{"job_id"  slug}) do
    job =
    slug
     SeoContext.fetch_id_by_slug("job")
     JobService.fetch_by_id()
    # 
    end
    Plug?

    View full-size slide

  81. mkaszubowski94
    defmodule ModularElixir.Job do
    use Ecto.Schema
    schema "jobs" do
    field :name, :string
    field :description, :string
    field :image_url, :string
    field :paid, :boolean
    field :slug, :string
    field :started, :boolean
    field :price, :integer
    field :datetime, :utc_datetime
    belongs_to :creator, ModularElixir.User
    belongs_to :contractor, ModularElixir.User
    end
    end

    View full-size slide

  82. mkaszubowski94
    JobService
    - fetch_by_id()
    - search()
    - find_by_user_id()
    - accept()
    - cancel()
    - publish()
    - create()
    - find_by_slug()
    - send_reminder()

    View full-size slide

  83. mkaszubowski94
    SEO Context
    • multiple slugs - easy to do
    • slugs for other resources - easy to do
    • slugs are resolved at the controller level
    • no slugs in domain logic

    View full-size slide

  84. mkaszubowski94
    no slugs in domain logic
    /api/jobs/my-awesome-job/subtasks
    from(
    s in Subtask,
    join: j in Job, on: s.job_id  j.id
    where: j.id  ^job_id,
    # …
    )
    /api/jobs/1/subtasks

    View full-size slide

  85. mkaszubowski94
    Maciej Kaszubowski
    Extracting Notifications

    View full-size slide

  86. mkaszubowski94
    browse jobs
    John Alchemist
    create new job
    manage jobs
    find a job
    logout
    my awesome job
    my other job
    yet another one
    and one more
    filters
    100$
    50$
    3500$
    99$
    xxxx
    yyy
    zzz
    search
    Upcoming job
    Job starting in 54 minutes
    ! x

    View full-size slide

  87. mkaszubowski94
    desired outcome:
    notification is sent when a job
    is starting soon

    View full-size slide

  88. mkaszubowski94
    required data:
    job information
    device information

    View full-size slide

  89. mkaszubowski94
    Job
    Reminders
    upcoming_jobs
    devices

    View full-size slide

  90. mkaszubowski94
    Job
    Reminders
    upcoming_jobs
    - put_job()
    - remove_job()
    - register_device()
    devices
    Interface:

    View full-size slide

  91. mkaszubowski94
    Job
    Reminders
    upcoming_jobs
    - put_job()
    - remove_job()
    - register_device()
    devices
    Interface:
    Would probably
    require a transaction

    View full-size slide

  92. mkaszubowski94
    - register_device()
    Interface:
    Job
    Service
    search
    Job
    Reminders
    sent_reminders
    devices

    View full-size slide

  93. mkaszubowski94
    JobService
    - fetch_by_id()
    - search()
    - find_by_user_id()
    - accept()
    - cancel()
    - publish()
    - create()
    - find_by_slug()
    - send_reminder()

    View full-size slide

  94. mkaszubowski94
    defmodule Notifications.Worker do
    use GenServer
    def handle_info(:work, state) do
    Notifications.send_notifications(now())
    Process.send_after(self(), :work, @interval)
    {:noreply, state)
    end
    end

    View full-size slide

  95. mkaszubowski94
    defmodule Notifications do
    defmodule Logic do
    def send(jobs, datetime, already_sent) do
    Enum.each(jobs, fn job ->
    if starting_soon?(job) && not_sent_yet?(job, already_sent) do
    send_notification(job)
    end
    end)
    end
    end
    def send_notifications(datetime) do
    jobs = JobBoard.search(…)
    already_sent = fetch_already_sent()
    :ok = Logic.send(jobs, datetime, already_sent)
    end
    end

    View full-size slide

  96. mkaszubowski94
    Logic
    desired
    outcome
    Notifications
    Job
    Board
    required
    data
    data
    source

    View full-size slide

  97. mkaszubowski94
    logic is isolated from the data
    source

    View full-size slide

  98. mkaszubowski94
    Logic
    desired
    outcome
    Notifications
    required
    data
    events stream

    View full-size slide

  99. mkaszubowski94
    Logic
    desired
    outcome
    Notifications
    required
    data

    View full-size slide

  100. mkaszubowski94
    Logic
    desired
    outcome
    Notifications
    required
    data
    TESTS

    View full-size slide

  101. mkaszubowski94
    Maciej Kaszubowski
    Extracting Publishing

    View full-size slide

  102. mkaszubowski94
    JobService
    - fetch_by_id()
    - search()
    - find_by_user_id()
    - accept()
    - cancel()
    - publish()
    - create()
    - find_by_slug()
    - send_reminder()

    View full-size slide

  103. mkaszubowski94
    problem:
    the same data model is used
    for reading and writing

    View full-size slide

  104. mkaszubowski94
    Job
    Publishing
    jobs
    Job
    Board
    jobs
    publish

    View full-size slide

  105. mkaszubowski94
    Job
    Publishing
    jobs
    Job
    Board
    jobs
    publish
    enforces job
    structure in params

    View full-size slide

  106. mkaszubowski94
    defmodule JobBoard do
    defmodule Job do
    @keys [:name, :description, ]
    @enforce_keys @keys
    defstruct @keys
    end
    def publish(%Job{} = job) do
    # 
    end
    end

    View full-size slide

  107. mkaszubowski94
    Job
    Publishing
    jobs
    Job
    Board
    jobs
    publish
    enforces job
    structure in params
    decides when the
    job is published

    View full-size slide

  108. mkaszubowski94
    Job
    Publishing
    jobs
    Job
    Board
    jobs
    upsert
    semantics
    decides what are
    the rules for editing
    a job
    publish

    View full-size slide

  109. mkaszubowski94
    defmodule ModularElixir.JobBoard.Job do
    use Ecto.Schema
    @schema_prefix “job_board”
    schema "jobs" do
    field :name, :string
    field :description, :string
    field :image_url, :string
    field :paid, :boolean
    field :slug, :string
    field :started, :boolean
    field :price, :integer
    field :datetime, :utc_datetime
    belongs_to :creator, ModularElixir.User
    belongs_to :contractor, ModularElixir.User
    end
    end

    View full-size slide

  110. mkaszubowski94
    JobService
    - fetch_by_id()
    - search()
    - find_by_user_id()
    - accept()
    - cancel()
    - publish()
    - create()
    - find_by_slug()
    - send_reminder()

    View full-size slide

  111. mkaszubowski94
    JobService
    - fetch_by_id()
    - search()
    - find_by_user_id()
    - accept()
    - cancel()
    - publish()
    - create()
    - find_by_slug()
    - send_reminder()

    View full-size slide

  112. mkaszubowski94
    JobBoard
    - fetch_by_id()
    - search()
    - find_by_user_id()
    - accept()
    - cancel()
    - publish()
    - create()
    - find_by_slug()
    - send_reminder()

    View full-size slide

  113. mkaszubowski94
    Job
    Publishing
    - create()
    - get_drafts()
    - publish()

    View full-size slide

  114. mkaszubowski94
    JobBoard
    • can be tested in isolation
    (without mocking payments)
    • linear logic (without conditionals)
    • no risk of showing unpublished jobs

    View full-size slide

  115. mkaszubowski94
    JobPublishing
    • rules for publishing can be changed in
    isolation
    • can be used alongside other modules
    (e.g. Admins can publish jobs without
    paying)

    View full-size slide

  116. mkaszubowski94
    Job
    Board
    desired
    outcome
    JobBoard
    params are enforced
    by the interface
    Job
    Publishing

    View full-size slide

  117. mkaszubowski94
    Job
    Board
    desired
    outcome
    JobBoard
    params are enforced
    by the interface
    Admin

    View full-size slide

  118. mkaszubowski94
    Job
    Board
    desired
    outcome
    JobBoard
    params are enforced
    by the interface
    TESTS

    View full-size slide

  119. mkaszubowski94
    JobBoard
    - fetch_by_id()
    - search()
    - find_by_user_id()
    - accept()
    - cancel()
    - publish()
    - create()
    - find_by_slug()
    - send_reminder()

    View full-size slide

  120. mkaszubowski94
    JobBoard
    jobs
    Ongoing
    Jobs
    jobs

    View full-size slide

  121. mkaszubowski94
    defmodule ModularElixir.JobBoard.Job do
    use Ecto.Schema
    @schema_prefix “job_board”
    schema "jobs" do
    field :name, :string
    field :description, :string
    field :image_url, :string
    field :paid, :boolean
    field :slug, :string
    field :started, :boolean
    field :price, :integer
    field :datetime, :utc_datetime
    belongs_to :creator, ModularElixir.User
    belongs_to :contractor, ModularElixir.User
    end
    end

    View full-size slide

  122. mkaszubowski94
    JobBoard
    - fetch_by_id()
    - search()
    - find_by_user_id()
    - accept()
    - cancel()
    - publish()
    - create()
    - find_by_slug()
    - send_reminder()

    View full-size slide

  123. mkaszubowski94
    Maciej Kaszubowski
    Outcomes

    View full-size slide

  124. mkaszubowski94
    JobService
    jobs

    View full-size slide

  125. mkaszubowski94
    JobService
    • multiple responsibilities
    • hard to change and understand
    • hard to test in isolation
    • grows with each new feature
    • hard to delete old features

    View full-size slide

  126. mkaszubowski94
    Job
    Publishing
    jobs
    Job
    Board
    jobs
    Reminders
    devices
    SEO
    slugs
    Ongoing
    Jobs
    jobs
    Job
    Archive
    jobs

    View full-size slide

  127. mkaszubowski94
    Smaller modules
    • clear responsibilities for each module
    • easy to change, refactor, understand
    • easy to test in isolation
    • tend to stay small
    • trivial to delete if no longer necessary

    View full-size slide

  128. mkaszubowski94
    focused on
    behaviour, not nouns

    View full-size slide

  129. mkaszubowski94
    nouns are not
    universal

    View full-size slide

  130. mkaszubowski94
    defmodule ModularElixir.Job do
    use Ecto.Schema
    schema "jobs" do
    field :name, :string
    field :description, :string
    field :image_url, :string
    field :paid, :boolean
    field :slug, :string
    field :started, :boolean
    field :price, :integer
    field :datetime, :utc_datetime
    belongs_to :creator, ModularElixir.User
    belongs_to :contractor, ModularElixir.User
    end
    end

    View full-size slide

  131. mkaszubowski94
    a “job” means
    something else in each
    module

    View full-size slide

  132. mkaszubowski94
    by splitting this, we make
    this explicit, rather than
    relying on some values

    View full-size slide

  133. mkaszubowski94
    the flow of the app is
    visible in the module
    structure and dependencies

    View full-size slide

  134. mkaszubowski94
    Job
    Publishing
    Job
    Board
    Ongoing
    Jobs
    Job
    Archive
    Reminders
    Payments

    View full-size slide

  135. mkaszubowski94
    the logic is more
    linear

    View full-size slide

  136. mkaszubowski94
    there are no (or fewer)
    dependency cycles

    View full-size slide

  137. mkaszubowski94
    the module hierarchy
    is much clearer

    View full-size slide

  138. mkaszubowski94
    Maciej Kaszubowski
    Criteria for
    modularising design

    View full-size slide

  139. mkaszubowski94
    start with a list of
    important decisions

    View full-size slide

  140. mkaszubowski94
    Important decisions
    • what are the rules for publishing jobs?
    • when/what notifications are sent?
    • which jobs are visible on job board?
    • how is a contractor chosen for each job?
    • what happens after a job is started? when the job ends?
    • how payments work? (when creator pays for job, how the
    contractor receives the money? do we allow coupons?
    credit cards? money transfers?)
    • …

    View full-size slide

  141. mkaszubowski94
    then, create a module
    for each decision

    View full-size slide

  142. mkaszubowski94
    Make each module
    • independent
    • easy to test
    • easy to understand
    • with a strong interface

    View full-size slide

  143. mkaszubowski94
    focus on behaviour,
    not data

    View full-size slide

  144. mkaszubowski94
    Maciej Kaszubowski
    Summary

    View full-size slide

  145. mkaszubowski94
    is this perfect?

    View full-size slide

  146. mkaszubowski94
    nope

    View full-size slide

  147. mkaszubowski94
    is this the best
    way?

    View full-size slide

  148. mkaszubowski94
    I have no idea

    View full-size slide

  149. mkaszubowski94
    it works for me

    View full-size slide

  150. mkaszubowski94
    but the main point is
    to start a discussion

    View full-size slide

  151. mkaszubowski94
    so argue with me
    over pizza

    View full-size slide

  152. mkaszubowski94
    Maciej Kaszubowski
    THANKS!

    View full-size slide