Midwest PHP 2020 - Web Scale System Design and Architecture

Let's walk through different system designs and architecture decisions for building a large scale PHP application.

We'll cover SOLID, Application Architectures, Platform Architectures, and walk through an example of scaling a system.

Transcript

1. web scale
   &
   System Design
   Architecture
   

   View Slide

2. Who is this guy?
   Ben Edmunds
   Open Source
   Author
   PHP Town Hall Podcast
   Staff Eng @ Wayfair
   

   View Slide

3. Design
   Principles
   

   View Slide

4. View Slide

5. View Slide

6. View Slide

7. View Slide

8. Design Principles
   Single Responsibility
   
   Principle
   
   (SRP)
   

   View Slide

9. Design Principles
   

   View Slide

10. Design Principles
    "what I mean by 'focusing one's attention
    upon some aspect': it does not mean
    ignoring the other aspects, it is just doing
    justice to the fact that from this aspect's
    point of view, the other is irrelevant. It is
    being one- and multiple-track minded
    simultaneously."
    
    - Edsger W. Dijkstra
    

    View Slide

11. SRP
    interface User {
    
    public function getEmail();
    
    public function savePost();
    
    }
    

    View Slide

12. SRP
    interface User {
    
    public function getEmail();
    
    }
    
    interface Post {
    
    public function setUserId();
    
    public function save();
    
    }
    

    View Slide

13. SRP
    interface Order {
    
    public function getTotal();
    
    public function getProducts();
    
    public function getPdf();
    
    }
    

    View Slide

14. SRP
    interface Order {
    
    public function getTotal();
    
    public function getProducts();
    
    }
    interface OrderPdfReport {
    
    public function setOrder(Order $order);
    
    public function getPdf();
    
    }
    

    View Slide

15. View Slide

16. Design Principles
    Open-Closed Principle
    
    (OCP)
    

    View Slide

17. Design Principles
    

    View Slide

18. Design Principles
    OPEN

    "A module will be said to be open if it is still available for
    extension. For example, it should be possible to add fields
    to the data structures it contains, or new elements to the set
    of functions it performs."
    CLOSED

    "A module will be said to be closed if [it] is available for use
    by other modules. This assumes that the module has been
    given a well-defined, stable description (the interface in the
    sense of information hiding)"
    - Bertrand Meyer
    

    View Slide

19. OCP
    class OrderPdfGeneratorJob {
    
    public function generatePdf();
    
    }
    
    class OrderGeoLocatorJob {
    
    public function locate();
    
    }
    

    View Slide

20. OCP
    class ProcessWorker {
    
    public function process($job) {
    
    if ($job instanceof OrderPdfGeneratorJob)
    
    {
    
    $job->generatePdf();
    
    }
    
    elseif ($job instanceof OrderGeoLocatorJob)
    {
    
    $job->locate();
    
    }
    
    

    View Slide

21. OCP
    interface Job {
    
    public function process();
    
    }
    
    class OrderPdfGenerator implements Job {
    
    public function process();
    
    }
    
    class OrderGeoLocator implements Job {
    
    public function process();
    
    

    View Slide

22. OCP
    class ProcessWorker {
    
    public function process(Job $job) {
    
    return $job->process();
    
    }
    
    }
    

    View Slide

23. View Slide

24. Design Principles
    Liskov Substitution
    
    Principle
    
    (LSP)
    

    View Slide

25. Design Principles
    

    View Slide

26. Design Principles
    

    "Let Φ(x) be a property provable about objects x of type T.
    Then Φ(y) should be true for objects y of type S where S is
    a subtype of T."
    - Barbara Liskov
    

    View Slide

27. LSP
    interface Duck {
    
    public void walk();
    
    public void quack();
    
    }
    
    class Mallard implements Duck
    
    

    View Slide

28. LSP
    interface Duck {
    
    public void walk();
    
    public void quack();
    
    }
    
    class RubberDuck implements Duck
    
    

    View Slide

29. LSP
    interface WaterToy {
    
    public void isFun();
    
    public void floats();
    
    }
    
    class RubberDuck implements WaterToy
    
    

    View Slide

30. View Slide

31. Design Principles
    Interface Segregation
    
    Principle
    
    (ISP)
    

    View Slide

32. Design Principles
    

    View Slide

33. Design Principles
    

    "Classes that have “fat” interfaces are classes whose
    interfaces are not cohesive."
    - Robert C. Martin
    

    View Slide

34. ISP
    interface Job {
    
    public function process();
    
    }
    
    class OrderPdfGenerator implements Job {
    
    public function process();
    
    }
    
    class OrderGeoLocator implements Job {
    
    public function process();
    
    

    View Slide

35. ISP
    interface Job {
    
    public function process();
    
    public function getLatLong();
    
    }
    
    class OrderPdfGenerator implements Job {
    
    public function save();
    
    uhOhhhh()
    
    }
    
    

    View Slide

36. ISP
    interface Job {
    
    public function process();
    
    }
    
    interface Geo {
    
    public function getLatLng();
    
    }
    
    class OrderGeoLocator implements Job, Geo {
    
    public function process();
    
    public function getLatLng();
    
    }
    

    View Slide

37. View Slide

38. Design Principles
    Dependency Inversion
    
    Principle
    
    (DIP)
    

    View Slide

39. Design Principles
    

    View Slide

40. Design Principles
    

    "Code to the interface"
    - Me & others
    

    View Slide

41. DIP
    class SuperUser {}
    
    $user = new SuperUser;
    
    class Post {
    
    function setUser(SuperUser $user);
    
    }
    
    

    View Slide

42. DIP
    interface User {}
    
    class SuperUser implements User {}
    
    $user = new SuperUser;
    
    class Post {
    
    function setUser(User $user);
    
    }
    
    

    View Slide

43. DIP
    interface User {}
    
    class SuperUser implements User {}
    
    class RegularUser implements User {}
    
    $user = new RegularUser;
    
    class Post {
    
    function setUser(User $user);
    
    }
    
    

    View Slide

44. View Slide

45. Architecture
    Patterns
    

    View Slide

46. View Slide

47. Application
    Architectures
    

    View Slide

48. View Slide

49. App Architectures
    Layered / N-Tiered
    

    View Slide

50. App Architectures
    Presentation Logic Data
    HTML / CSS
    Javascript
    Templating
    PHP
    APIs
    Glue
    Postgres
    Redis
    Config
    

    View Slide

51. App Architectures
    Presentation Logic Data
    Authentication
    Product
    Cart
    

    View Slide

52. App Architectures
    HTML / CSS
    PHP
    DB
    CLIENT
    

    View Slide

53. App Architectures
    CLIENT
    Presentation
    Cart
    

    View Slide

54. App Architectures
    Presentation Logic Data
    Authentication
    Product
    Cart
    

    View Slide

55. Platform Architectures
    Microkernel
    

    View Slide

56. App Architectures
    CORE
    Plugin
    Component
    Extension
    i
    n
    t
    e
    r
    f
    a
    c
    e
    

    View Slide

57. App Architectures
    DB
    Postgres
    MySQL
    SQL Server
    P
    D
    O
    

    View Slide

58. Platform
    Architectures
    

    View Slide

59. Platform Architectures
    Microservices
    Monolith
    SOA
    

    View Slide

60. View Slide

61. Platform Architectures
    Microservices
    SOA
    Monolith
    

    View Slide

62. Platform Architectures
    Monolith
    

    View Slide

63. Platform Architectures
    "Almost all the successful
    microservice stories have started
    with a monolith that got too big
    and was broken up" 

    - Martin Fowler
    

    View Slide

64. Platform Architectures
    Coupling
    

    View Slide

65. Platform Architectures
    Development Speed
    

    View Slide

66. Platform Architectures
    Maintenance
    

    View Slide

67. Platform Architectures
    Monorepo Myth
    

    View Slide

68. Platform Architectures
    Trunk Based
    

    View Slide

69. Platform Architectures
    Scaling
    

    View Slide

70. Platform Architectures
    Microservices
    Monolith
    SOA
    

    View Slide

71. Platform Architectures
    Coupling
    

    View Slide

72. Platform Architectures
    Team Ownership
    

    View Slide

73. Platform Architectures
    APIs as
    
    First Class Citizens
    

    View Slide

74. Platform Architectures
    Scaling
    

    View Slide

75. Platform Architectures
    Routing
    

    View Slide

76. Platform Architectures
    Many Monoliths
    

    View Slide

77. Platform Architectures
    Repo Management
    

    View Slide

78. Platform Architectures
    Monolith
    SOA
    Microservices
    

    View Slide

79. Platform Architectures
    Coupling
    

    View Slide

80. Platform Architectures
    Deployments
    

    View Slide

81. Platform Architectures
    Scaling
    

    View Slide

82. Platform Architectures
    More Services
    
    =
    
    More Problems
    - Notorious B.I.G.
    

    View Slide

83. Platform Architectures
    Microservices
    Monolith
    SOA
    

    View Slide

84. Data
    Architectures
    

    View Slide

85. Data Architectures
    https://github.com/UWCoffeeNCode/resources/wiki/SQL-and-NoSQL-Databases
    

    View Slide

86. Data Architectures
    Relational / SQL
    

    View Slide

87. Data Architectures
    NoSQL
    

    View Slide

88. Data Architectures
    Time Series DBs
    

    View Slide

89. Data Architectures
    Combos
    

    View Slide

90. Data Architectures
    Scaling
    

    View Slide

91. View Slide

92. Example
    Design
    

    View Slide

93. Pastebin
    Example
    

    View Slide

94. Pastebin
    Requirements
    • Save with unique URL
    
    • Retrieve code via URL
    
    • Don't break
    

    View Slide

95. Pastebin
    Scale
    • Writes / second
    
    • Reads / second
    
    • Storage
    

    View Slide

96. Pastebin
    Scale
    • Writes / second
    
    • Reads / second
    
    • Storage
    1 x 86400
    50 x 86400
    86400x10kb = 864MB/day
    

    View Slide

97. Pastebin
    Scale
    (864MB/day x 30) x 6 =
    155 GB x 1.25 = 193GB
    

    View Slide

98. Pastebin
    Scale
    (864MB/day x 30) x 6 =
    155 GB x 1.25 = 193GB
    200GB for 6 months
    

    View Slide

99. Pastebin
    API
    • GET /$primaryKey
    
    • POST /
    

    View Slide

100. Pastebin
     Database
     • id: serial
     
     • content: text
     
     • created_at: datetime
     

     View Slide

101. Pastebin
     CLIENT API
     DB
     

     View Slide

102. Pastebin
     ⚠ Scenario ⚠
     • IDs becoming too long
     
     • Don't want users guessing
     other's URLs
     

     View Slide

103. Pastebin
     Hashing
     86400 per day
     86400 x (365 x 10)
     316 million hashes
     

     View Slide

104. Pastebin
     Hashing
     base64 encoding
     64^5
     >1 billion
     

     View Slide

105. Pastebin
     Database
     • hash: varchar(5)
     
     • content: text
     
     • created_at: datetime
     

     View Slide

106. Pastebin
     API
     • POST /

     -> generateHash()

     -> checkHash()

     -> save()

     

     View Slide

107. Pastebin
     API
     • POST /

     -> getHash()

     -> save()

     

     View Slide

108. Pastebin
     CLIENT API Key
     Serv
     DB
     

     View Slide

109. Pastebin
     ⚠ Scenario ⚠
     • Large pastes could fill up 

     the DB
     

     View Slide

110. Pastebin
     Database
     • hash: varchar(5)
     
     • created_at: datetime
     

     View Slide

111. Pastebin
     Database
     • hash: varchar(5)
     
     • created_at: datetime
     Object Storage
     • {$hash}
     

     View Slide

112. Pastebin
     CLIENT API
     Obj
     DB
     Key
     Serv
     

     View Slide

113. Pastebin
     API
     • POST /

     -> getHash()

     -> saveDb((){

     saveObj()

     })

     

     View Slide

114. Pastebin
     CLIENT API
     Obj
     Cache
     DB
     Key
     Serv
     

     View Slide

115. Pastebin
     ⚠ Next Steps ⚠
     • More Caching
     
     • Load Balancing
     
     • Separate reads / writes
     
     • Reporting System
     

     View Slide

116. Pastebin
     CLIENT
     Obj
     Cache
     Service
     Service
     DB
     Key
     Serv
     

     View Slide

117. Pastebin
     CLIENT API
     Obj
     Cache
     R
     e
     q
     u
     e
     s
     t
     DB
     Key
     Serv
     

     View Slide

118. Pastebin
     CLIENT
     API
     Obj
     R
     e
     q
     u
     e
     s
     t
     API
     DB DB
     DB
     Key
     Serv
     

     View Slide

119. Pastebin
     CLIENT
     API
     Obj
     R
     e
     q
     u
     e
     s
     t
     API
     DB DB
     DB
     Key
     Serv
     Iteration =
     

     View Slide

120. Q/A
     

     View Slide

121. Resources
     

     View Slide

122. Resources
     

     View Slide

123. Resources
     

     View Slide

124. Resources
     

     View Slide

125. Thank You
     @benedmunds
     ben.edmunds@gmail.com
     

     View Slide