Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building an Umbrella Project

5af347d330e59ac194cdfc355e237e44?s=47 Wojtek Mach
September 02, 2016

Building an Umbrella Project

One of the many goals that both Elixir & Phoenix share is developer’s productivity. There can be two aspects of productivity: short-term and long-term.
In this talk we’ll explore how using Elixir’s umbrella project feature can help with long-term productivity when working with potentially large codebases. As a example we’ll walk through building an imaginary system: Acme Bank. We will see how we can break up this system into a collection of smaller applications, each of which has a well defined boundary, domain model, set of responsibilities, and can be worked on independently. We’ll compare this approach to microservices-based architecture which is being adopted by many organizations and we’ll see if we can achieve some of the benefits without incurring some of the drawbacks.

5af347d330e59ac194cdfc355e237e44?s=128

Wojtek Mach

September 02, 2016
Tweet

Transcript

  1. Building an Umbrella Project Wojtek Mach @wojtekmach @wojtekmach

  2. Me • Ruby • GitHub API adapter for Ecto •

    OOP library for Elixir • Acme Bank
  3. Me • Ruby • GitHub API adapter for Ecto •

    OOP library for Elixir • Acme Bank
  4. Me • Ruby • GitHub API adapter for Ecto •

    OOP library for Elixir • Acme Bank
  5. Me • Ruby • GitHub API adapter for Ecto •

    OOP library for Elixir • Acme Bank
  6. What’s an Umbrella Project?

  7. Inspiration

  8. Inspiration Architecture: The Lost Years
 Robert C. Martin, Midwest RubyConf

    2011
 • Typical Rails app • app/{models,controllers,view} • Top-level architecture should scream it’s intent
  9. Inspiration Architecture: The Lost Years
 Robert C. Martin, Midwest RubyConf

    2011
 • Typical Rails app • app/{models,controllers,view} • Top-level architecture should scream it’s intent
  10. Inspiration Architecture: The Lost Years
 Robert C. Martin, Midwest RubyConf

    2011
 • Typical Rails app • app/{models,controllers,views,…} • Top-level architecture should scream it’s intent
  11. Inspiration Architecture: The Lost Years
 Robert C. Martin, Midwest RubyConf

    2011
 • Typical Rails app • app/{models,controllers,views,…} • Top-level architecture should scream it’s intent
  12. Inspiration Wrangling Large Rails Codebases
 Stephan Hagemann, Rocky Mountain Ruby

    2012
 • Advice about writing large apps • Components
  13. Inspiration Wrangling Large Rails Codebases
 Stephan Hagemann, Rocky Mountain Ruby

    2012
 • Advice about writing a large app • Components
  14. Inspiration Wrangling Large Rails Codebases
 Stephan Hagemann, Rocky Mountain Ruby

    2012
 • Don’t write a large app • Components
  15. Inspiration Wrangling Large Rails Codebases
 Stephan Hagemann, Rocky Mountain Ruby

    2012
 • Don’t write a large app • Components: gems & engines
  16. Inspiration The Art of Destroying Software
 Greg Young, Leetspeek 2014

    
 • organize your project in a way that makes it easy to delete code.
  17. Inspiration The Art of Destroying Software
 Greg Young, Leetspeek 2014

    
 • organize your project in a way that makes it easy to delete code.
  18. What’s an Umbrella Project?

  19. What’s an Umbrella Project? • Mixfile • apps/ directory •

    OTP applications
  20. OTP application „In Elixir (actually, in Erlang/OTP), an application is

    a component implementing some specific functionality, that can be started and stopped as a unit, and which can be re-used in other systems.” — Elixir API docs
  21. OTP application • application dependencies • configuration • start/stop behaviour

    • regular applications & library applications
  22. OTP application • application dependencies • configuration: config.exs, Application.put_env, mix.exs

    • start/stop behaviour • regular applications & library applications
  23. OTP application • application dependencies • configuration • start/stop behaviour

    • regular applications & library applications
  24. OTP application • application dependencies • configuration • start/stop behaviour

    • regular applications & library applications
  25. OTP application • application dependencies • configuration • start/stop behaviour

    • regular applications & library applications • standard
  26. OTP application „Application”

  27. OTP application Server

  28. OTP application Service

  29. OTP application Microservice

  30. Building Acme Bank

  31. $ mix new bank_platform --umbrella

  32. # mix.exs defmodule BankPlatform.Mixfile do use Mix.Project def project do

    [apps_path: "apps", deps: deps(), aliases: aliases()] end end Bank Platform
  33. First OTP App

  34. Bank

  35. $ cd apps/
 $ mix new bank --sup Bank

  36. # apps/bank/mix.exs defmodule Bank.Mixfile do use Mix.Project def project do

    [app: :bank, version: "0.1.0", build_path: "../../_build", config_path: "../../config/config.exs", deps_path: "../../deps", lockfile: "../../mix.lock", deps: deps()] end end Bank
  37. First Feature

  38. First Feature: Account Balance

  39. defmodule Bank.Account do defstruct name: "", balance: 0 end Naive

  40. defmodule Bank.Account do defstruct name: "", balance: 0 def deposit(account,

    amount) do %Account{account | balance: account.balance + amount} end end Naive
  41. Accounting 101

  42. Accounting 101 • Any movement of money happens between at

    least 2 accounts: one account is debited and the other account is credited • Example: Alice transfers $100 to Bob
  43. Accounting 101 • Any movement of money happens between at

    least 2 accounts: one account is debited and the other account is credited • Example: Alice transfers $100 to Bob
  44. Alice transfers $100 to Bob | Bob | :credit |

    100.00 | Alice | :debit | 100.00 Accounting 101
  45. Bob transfers $20 to Alice | Bob | :credit |

    100.00 | Alice | :debit | 100.00 | Alice | :credit | 20.00 | Bob | :debit | 20.00 Accounting 101
  46. Bob transfers $10 to Alice | Bob | :credit |

    100.00 | Alice | :debit | 100.00 | Alice | :credit | 20.00 | Bob | :debit | 20.00 | Bob | :debit | 10.00 | Alice | :credit | 10.00 Accounting 101
  47. Bob’s balance = $100 - $20 - $10 = $70

    | Bob | :credit | 100.00 | Alice | :debit | 100.00 | Alice | :credit | 20.00 | Bob | :debit | 20.00 | Bob | :debit | 10.00 | Alice | :credit | 10.00 Accounting 101
  48. Bob’s balance = $100 - $20 - $10 = $70

    | Bob | :credit | 100.00 | Alice | :debit | 100.00 | Alice | :credit | 20.00 | Bob | :debit | 20.00 | Bob | :debit | 10.00 | Alice | :credit | 10.00 Accounting 101
  49. Alice’s balance = -$100 + $20 + $10 = -$70

    | Bob | :credit | 100.00 | Alice | :debit | 100.00 | Alice | :credit | 20.00 | Bob | :debit | 20.00 | Bob | :debit | 10.00 | Alice | :credit | 10.00 Accounting 101
  50. Alice’s balance = -$100 + $20 + $10 = -$70

    | Bob | :credit | 100.00 | Alice | :debit | 100.00 | Alice | :credit | 20.00 | Bob | :debit | 20.00 | Bob | :debit | 10.00 | Alice | :credit | 10.00 Double-entry accounting system Accounting 101
  51. Accounting 101 Why mention this? • it’s a Bank •

    it’s functional • mathematical properties • complexity
  52. Accounting 101 Why mention this? • it’s functional • mathematical

    properties • complexity
  53. Accounting 101 Why mention this? • it’s functional • mathematical

    properties • complexity
  54. Accounting 101 Why mention this? • it’s functional • associativity

    • complexity
  55. Accounting 101 Why mention this? • it’s functional • associativity,

    commutativity • complexity
  56. Accounting 101 Why mention this? • it’s functional • associativity,

    commutativity, idempotence • complexity
  57. Accounting 101 Why mention this? • it’s functional • CRDT

    • complexity
  58. Accounting 101 Why mention this? • it’s functional • eventual

    consistency [1] • complexity [1] http://highscalability.com/blog/2013/5/1/myth-eric-brewer-on-why-banks-are-base-not-acid-availability.html
  59. Accounting 101 Why mention this? • it’s functional • eventual

    consistency [1] • complexity [1] http://highscalability.com/blog/2013/5/1/myth-eric-brewer-on-why-banks-are-base-not-acid-availability.html
  60. Accounting 101 Why mention this? • it’s functional • eventual

    consistency [1] • complexity • complexity [1] http://highscalability.com/blog/2013/5/1/myth-eric-brewer-on-why-banks-are-base-not-acid-availability.html
  61. Bank: Entities

  62. Bank: Entities • Account • Entry • Customer

  63. Bank: Entities • Account: web/models/account.ex • Entry: web/models/entry.ex • Customer:

    web/models/customer.ex
  64. Bank: Entities • Account: lib/bank/account.ex • Entry: lib/bank/entry.ex • Customer:

    lib/bank/customer.ex
  65. Bank: Entities • Account: lib/bank/account.ex • Entry: lib/bank/entry.ex • Customer:

    lib/bank/customer.ex Different rates of change
  66. Bank: Entities • Account: lib/bank/ledger/account.ex • Entry: lib/bank/ledger/entry.ex • Customer:

    lib/bank/customer.ex Different rates of change
  67. Bank: Persistence

  68. Bank: Persistence • Ecto • Where to put Repo access?

  69. Bank: Persistence • Ecto • Where to put Repo access?

  70. Bank.Ledger # lib/bank/ledger.ex defmodule Bank.Ledger do def balance(account) do Repo.one("SELECT

    SUM(...)") end def write(entries) do # ... end end
  71. Bank.Ledger # lib/bank/ledger.ex defmodule Bank.Ledger do def balance(account) do Repo.one("SELECT

    SUM(...)") end def write(entries) do # ... end end
  72. Bank.Ledger # lib/bank/ledger.ex defmodule Bank.Ledger do def balance(account) do Repo.one("SELECT

    SUM(...)") end def write(entries) do # ... end end
  73. Bank # lib/bank.ex defmodule Bank do def create_customer(params) do Bank.Customer.changeset(params)

    |> Repo.insert end end
  74. Bank # lib/bank.ex defmodule Bank do def create_customer(params) do Bank.Customer.changeset(params)

    |> Repo.insert end end
  75. Bank # lib/bank.ex defmodule Bank do def create_customer(params) do Bank.Customer.changeset(params)

    |> Repo.insert end end
  76. Bank Bank.create_customer/1
 Bank.Ledger.balance/1
 Bank.Ledger.write/1 • Use cases • Use in

    controllers, tests, iex, seeds
  77. Bank Bank.create_customer/1
 Bank.Ledger.balance/1
 Bank.Ledger.write/1 • Use cases • Use in

    controllers, tests, iex, seeds
  78. Bank Bank.create_customer/1
 Bank.Ledger.balance/1
 Bank.Ledger.write/1 • Use cases • Use in

    controllers, tests, iex, seeds
  79. Money

  80. Money • Amount + currency • Operations • Persistence •

    Isolation • OSS • Productivity
  81. Money • Amount + currency • Operations • Persistence •

    Isolation • OSS • Productivity
  82. Money • Amount + currency • Operations • Persistence •

    Isolation • OSS • Productivity
  83. Money • Amount + currency • Operations • Persistence •

    Isolation • OSS • Productivity
  84. Money • Amount + currency • Operations • Persistence •

    Isolation • OSS • Productivity
  85. Money • Amount + currency • Operations • Persistence •

    Isolation • OSS • Productivity
  86. Money # apps/bank/mix.exs defp deps do [ {:money, in_umbrella: true}

    ] end
  87. Money # apps/bank/mix.exs def application do [applications: [..., :money]] end

  88. BankWeb

  89. BankWeb $ cd apps/ $ mix phoenix.new bank_web # add

    bank as dependency
  90. BankWeb $ cd apps/ $ mix phoenix.new bank_web # add

    bank as dependency
  91. BankWeb $ cd apps/ $ mix phoenix.new bank_web # add

    bank as dependency
  92. BankWeb • boring! • connect controllers to business logic

  93. BankWeb • boring! • separation of concerns

  94. Backoffice

  95. Backoffice • ExAdmin. Works with umbrellas out of the box

    • ports setup • deployment • Phoenix is Not Your Application
  96. Backoffice • ExAdmin. Works with umbrellas out of the box

    • setup different ports in development • deployment • Phoenix is Not Your Application
  97. Backoffice • ExAdmin. Works with umbrellas out of the box

    • setup different ports in development • deployment • Phoenix is Not Your Application
  98. Backoffice • ExAdmin. Works with umbrellas out of the box

    • setup different ports in development • deployment • Phoenix is Not Your Application
  99. Backoffice • ExAdmin. Works with umbrellas out of the box

    • setup different ports in development • deployment • Phoenix is Not Your Application
  100. Deployment

  101. Deployment • Heroku • MasterProxy.

  102. Deployment • Heroku • MasterProxy.

  103. Deployment • Heroku • MasterProxy. Thanks @TheGazler! • distillery, exrm

    replacement, has 1st class support for umbrellas
  104. Deployment • Heroku • MasterProxy. Thanks @TheGazler! • distillery, exrm

    replacement, has 1st class support for umbrellas
  105. Auth

  106. Auth • Upfront design? • Features: audit logs

  107. Auth • Upfront design? • Features: strategies

  108. Auth • Upfront design? • Features: strategies, 2FA

  109. Auth • Upfront design? • Features: strategies, 2FA, tracking

  110. Auth • Upfront design? • Features: strategies, 2FA, tracking

  111. Auth • Upfront design? • Features: strategies, 2FA, tracking •

    Separate Repo, separate DB
  112. Auth • Upfront design? • Features: strategies, 2FA, tracking •

    Separate Repo, separate DB • Thanks @chris_mccord!
  113. Messenger

  114. Bank Platform

  115. BankPlatform • auth • bank • bank_web • backoffice •

    master_proxy • messenger • money
  116. BankPlatform

  117. BankPlatform

  118. BankPlatform

  119. Microservices

  120. Microservices Benefits: • Strong Module Boundaries • Independent Deployments •

    Technology Diversity http://martinfowler.com/articles/microservice-trade-offs.html
  121. Microservices Benefits: • Strong Module Boundaries • Independent Deployments •

    Technology Diversity http://martinfowler.com/articles/microservice-trade-offs.html Drawbacks: • Distribution • Eventual Consistency • Operational Complexity
  122. None
  123. None
  124. Conclusion

  125. Conclusion • keep an eye on the big picture

  126. Conclusion • keep an eye on the big picture •

    don’t write a large app
  127. Conclusion • keep an eye on the big picture •

    don’t write a large app • isolate by functionality and rate of change
  128. Conclusion • keep an eye on the big picture •

    don’t write a large app • isolate by functionality and rate of change • delete code liberally
  129. Thanks! wojtekmach/acme_bank