Save 37% off PRO during our Black Friday Sale! »

Midwest PHP 2020 - Web Scale System Design and Architecture

Midwest PHP 2020 - Web Scale System Design and Architecture

Let's walk through different system designs and architecture decisions for building a large scale PHP application.

We'll cover SOLID, Application Architectures, Platform Architectures, and walk through an example of scaling a system.

22f21d5c22b930fd35dd98f25dedf6a4?s=128

Ben Edmunds

April 03, 2020
Tweet

Transcript

  1. web scale & System Design Architecture

  2. Who is this guy? Ben Edmunds Open Source Author PHP

    Town Hall Podcast Staff Eng @ Wayfair
  3. Design Principles

  4. None
  5. None
  6. None
  7. None
  8. Design Principles Single Responsibility Principle (SRP)

  9. Design Principles

  10. Design Principles "what I mean by 'focusing one's attention upon

    some aspect': it does not mean ignoring the other aspects, it is just doing justice to the fact that from this aspect's point of view, the other is irrelevant. It is being one- and multiple-track minded simultaneously." - Edsger W. Dijkstra
  11. SRP interface User { public function getEmail(); public function savePost();

    }
  12. SRP interface User { public function getEmail(); } interface Post

    { public function setUserId(); public function save(); }
  13. SRP interface Order { public function getTotal(); public function getProducts();

    public function getPdf(); }
  14. SRP interface Order { public function getTotal(); public function getProducts();

    } interface OrderPdfReport { public function setOrder(Order $order); public function getPdf(); }
  15. None
  16. Design Principles Open-Closed Principle (OCP)

  17. Design Principles

  18. Design Principles OPEN
 "A module will be said to be

    open if it is still available for extension. For example, it should be possible to add fields to the data structures it contains, or new elements to the set of functions it performs." CLOSED
 "A module will be said to be closed if [it] is available for use by other modules. This assumes that the module has been given a well-defined, stable description (the interface in the sense of information hiding)" - Bertrand Meyer
  19. OCP class OrderPdfGeneratorJob { public function generatePdf(); } class OrderGeoLocatorJob

    { public function locate(); }
  20. OCP class ProcessWorker { public function process($job) { if ($job

    instanceof OrderPdfGeneratorJob) { $job->generatePdf(); } elseif ($job instanceof OrderGeoLocatorJob) { $job->locate(); }
  21. OCP interface Job { public function process(); } class OrderPdfGenerator

    implements Job { public function process(); } class OrderGeoLocator implements Job { public function process();
  22. OCP class ProcessWorker { public function process(Job $job) { return

    $job->process(); } }
  23. None
  24. Design Principles Liskov Substitution Principle (LSP)

  25. Design Principles

  26. Design Principles 
 "Let Φ(x) be a property provable about

    objects x of type T. Then Φ(y) should be true for objects y of type S where S is a subtype of T." - Barbara Liskov
  27. LSP interface Duck { public void walk(); public void quack();

    } class Mallard implements Duck
  28. LSP interface Duck { public void walk(); public void quack();

    } class RubberDuck implements Duck
  29. LSP interface WaterToy { public void isFun(); public void floats();

    } class RubberDuck implements WaterToy
  30. None
  31. Design Principles Interface Segregation Principle (ISP)

  32. Design Principles

  33. Design Principles 
 "Classes that have “fat” interfaces are classes

    whose interfaces are not cohesive." - Robert C. Martin
  34. ISP interface Job { public function process(); } class OrderPdfGenerator

    implements Job { public function process(); } class OrderGeoLocator implements Job { public function process();
  35. ISP interface Job { public function process(); public function getLatLong();

    } class OrderPdfGenerator implements Job { public function save(); uhOhhhh() }
  36. ISP interface Job { public function process(); } interface Geo

    { public function getLatLng(); } class OrderGeoLocator implements Job, Geo { public function process(); public function getLatLng(); }
  37. None
  38. Design Principles Dependency Inversion Principle (DIP)

  39. Design Principles

  40. Design Principles 
 "Code to the interface" - Me &

    others
  41. DIP class SuperUser {} $user = new SuperUser; class Post

    { function setUser(SuperUser $user); }
  42. DIP interface User {} class SuperUser implements User {} $user

    = new SuperUser; class Post { function setUser(User $user); }
  43. DIP interface User {} class SuperUser implements User {} class

    RegularUser implements User {} $user = new RegularUser; class Post { function setUser(User $user); }
  44. None
  45. Architecture Patterns

  46. None
  47. Application Architectures

  48. None
  49. App Architectures Layered / N-Tiered

  50. App Architectures Presentation Logic Data HTML / CSS Javascript Templating

    PHP APIs Glue Postgres Redis Config
  51. App Architectures Presentation Logic Data Authentication Product Cart

  52. App Architectures HTML / CSS PHP DB CLIENT

  53. App Architectures CLIENT Presentation Cart

  54. App Architectures Presentation Logic Data Authentication Product Cart

  55. Platform Architectures Microkernel

  56. App Architectures CORE Plugin Component Extension i n t e

    r f a c e
  57. App Architectures DB Postgres MySQL SQL Server P D O

  58. Platform Architectures

  59. Platform Architectures Microservices Monolith SOA

  60. None
  61. Platform Architectures Microservices SOA Monolith

  62. Platform Architectures Monolith

  63. Platform Architectures "Almost all the successful microservice stories have started

    with a monolith that got too big and was broken up" 
 - Martin Fowler
  64. Platform Architectures Coupling

  65. Platform Architectures Development Speed

  66. Platform Architectures Maintenance

  67. Platform Architectures Monorepo Myth

  68. Platform Architectures Trunk Based

  69. Platform Architectures Scaling

  70. Platform Architectures Microservices Monolith SOA

  71. Platform Architectures Coupling

  72. Platform Architectures Team Ownership

  73. Platform Architectures APIs as First Class Citizens

  74. Platform Architectures Scaling

  75. Platform Architectures Routing

  76. Platform Architectures Many Monoliths

  77. Platform Architectures Repo Management

  78. Platform Architectures Monolith SOA Microservices

  79. Platform Architectures Coupling

  80. Platform Architectures Deployments

  81. Platform Architectures Scaling

  82. Platform Architectures More Services = More Problems - Notorious B.I.G.

  83. Platform Architectures Microservices Monolith SOA

  84. Data Architectures

  85. Data Architectures https://github.com/UWCoffeeNCode/resources/wiki/SQL-and-NoSQL-Databases

  86. Data Architectures Relational / SQL

  87. Data Architectures NoSQL

  88. Data Architectures Time Series DBs

  89. Data Architectures Combos

  90. Data Architectures Scaling

  91. None
  92. Example Design

  93. Pastebin Example

  94. Pastebin Requirements • Save with unique URL • Retrieve code

    via URL • Don't break
  95. Pastebin Scale • Writes / second • Reads / second

    • Storage
  96. Pastebin Scale • Writes / second • Reads / second

    • Storage 1 x 86400 50 x 86400 86400x10kb = 864MB/day
  97. Pastebin Scale (864MB/day x 30) x 6 = 155 GB

    x 1.25 = 193GB
  98. Pastebin Scale (864MB/day x 30) x 6 = 155 GB

    x 1.25 = 193GB 200GB for 6 months
  99. Pastebin API • GET /$primaryKey • POST /

  100. Pastebin Database • id: serial • content: text • created_at:

    datetime
  101. Pastebin CLIENT API DB

  102. Pastebin ⚠ Scenario ⚠ • IDs becoming too long •

    Don't want users guessing other's URLs
  103. Pastebin Hashing 86400 per day 86400 x (365 x 10)

    316 million hashes
  104. Pastebin Hashing base64 encoding 64^5 >1 billion

  105. Pastebin Database • hash: varchar(5) • content: text • created_at:

    datetime
  106. Pastebin API • POST /
 -> generateHash()
 -> checkHash()
 ->

    save()

  107. Pastebin API • POST /
 -> getHash()
 -> save()


  108. Pastebin CLIENT API Key Serv DB

  109. Pastebin ⚠ Scenario ⚠ • Large pastes could fill up

    
 the DB
  110. Pastebin Database • hash: varchar(5) • created_at: datetime

  111. Pastebin Database • hash: varchar(5) • created_at: datetime Object Storage

    • {$hash}
  112. Pastebin CLIENT API Obj DB Key Serv

  113. Pastebin API • POST /
 -> getHash()
 -> saveDb((){
 saveObj()


    })

  114. Pastebin CLIENT API Obj Cache DB Key Serv

  115. Pastebin ⚠ Next Steps ⚠ • More Caching • Load

    Balancing • Separate reads / writes • Reporting System
  116. Pastebin CLIENT Obj Cache Service Service DB Key Serv

  117. Pastebin CLIENT API Obj Cache R e q u e

    s t DB Key Serv
  118. Pastebin CLIENT API Obj R e q u e s

    t API DB DB DB Key Serv
  119. Pastebin CLIENT API Obj R e q u e s

    t API DB DB DB Key Serv Iteration =
  120. Q/A

  121. Resources

  122. Resources

  123. Resources

  124. Resources

  125. Thank You @benedmunds ben.edmunds@gmail.com