$30 off During Our Annual Pro Sale. View Details »

Midwest PHP 2020 - Web Scale System Design and Architecture

Midwest PHP 2020 - Web Scale System Design and Architecture

Let's walk through different system designs and architecture decisions for building a large scale PHP application.

We'll cover SOLID, Application Architectures, Platform Architectures, and walk through an example of scaling a system.

Ben Edmunds

April 03, 2020
Tweet

More Decks by Ben Edmunds

Other Decks in Technology

Transcript

  1. web scale
    &
    System Design
    Architecture

    View Slide

  2. Who is this guy?
    Ben Edmunds
    Open Source
    Author
    PHP Town Hall Podcast
    Staff Eng @ Wayfair

    View Slide

  3. Design
    Principles

    View Slide

  4. View Slide

  5. View Slide

  6. View Slide

  7. View Slide

  8. Design Principles
    Single Responsibility

    Principle

    (SRP)

    View Slide

  9. Design Principles

    View Slide

  10. Design Principles
    "what I mean by 'focusing one's attention
    upon some aspect': it does not mean
    ignoring the other aspects, it is just doing
    justice to the fact that from this aspect's
    point of view, the other is irrelevant. It is
    being one- and multiple-track minded
    simultaneously."

    - Edsger W. Dijkstra

    View Slide

  11. SRP
    interface User {

    public function getEmail();

    public function savePost();

    }

    View Slide

  12. SRP
    interface User {

    public function getEmail();

    }

    interface Post {

    public function setUserId();

    public function save();

    }

    View Slide

  13. SRP
    interface Order {

    public function getTotal();

    public function getProducts();

    public function getPdf();

    }

    View Slide

  14. SRP
    interface Order {

    public function getTotal();

    public function getProducts();

    }
    interface OrderPdfReport {

    public function setOrder(Order $order);

    public function getPdf();

    }

    View Slide

  15. View Slide

  16. Design Principles
    Open-Closed Principle

    (OCP)

    View Slide

  17. Design Principles

    View Slide

  18. Design Principles
    OPEN

    "A module will be said to be open if it is still available for
    extension. For example, it should be possible to add fields
    to the data structures it contains, or new elements to the set
    of functions it performs."
    CLOSED

    "A module will be said to be closed if [it] is available for use
    by other modules. This assumes that the module has been
    given a well-defined, stable description (the interface in the
    sense of information hiding)"
    - Bertrand Meyer

    View Slide

  19. OCP
    class OrderPdfGeneratorJob {

    public function generatePdf();

    }

    class OrderGeoLocatorJob {

    public function locate();

    }

    View Slide

  20. OCP
    class ProcessWorker {

    public function process($job) {

    if ($job instanceof OrderPdfGeneratorJob)

    {

    $job->generatePdf();

    }

    elseif ($job instanceof OrderGeoLocatorJob)
    {

    $job->locate();

    }

    View Slide

  21. OCP
    interface Job {

    public function process();

    }

    class OrderPdfGenerator implements Job {

    public function process();

    }

    class OrderGeoLocator implements Job {

    public function process();

    View Slide

  22. OCP
    class ProcessWorker {

    public function process(Job $job) {

    return $job->process();

    }

    }

    View Slide

  23. View Slide

  24. Design Principles
    Liskov Substitution

    Principle

    (LSP)

    View Slide

  25. Design Principles

    View Slide

  26. Design Principles

    "Let Φ(x) be a property provable about objects x of type T.
    Then Φ(y) should be true for objects y of type S where S is
    a subtype of T."
    - Barbara Liskov

    View Slide

  27. LSP
    interface Duck {

    public void walk();

    public void quack();

    }

    class Mallard implements Duck

    View Slide

  28. LSP
    interface Duck {

    public void walk();

    public void quack();

    }

    class RubberDuck implements Duck

    View Slide

  29. LSP
    interface WaterToy {

    public void isFun();

    public void floats();

    }

    class RubberDuck implements WaterToy

    View Slide

  30. View Slide

  31. Design Principles
    Interface Segregation

    Principle

    (ISP)

    View Slide

  32. Design Principles

    View Slide

  33. Design Principles

    "Classes that have “fat” interfaces are classes whose
    interfaces are not cohesive."
    - Robert C. Martin

    View Slide

  34. ISP
    interface Job {

    public function process();

    }

    class OrderPdfGenerator implements Job {

    public function process();

    }

    class OrderGeoLocator implements Job {

    public function process();

    View Slide

  35. ISP
    interface Job {

    public function process();

    public function getLatLong();

    }

    class OrderPdfGenerator implements Job {

    public function save();

    uhOhhhh()

    }

    View Slide

  36. ISP
    interface Job {

    public function process();

    }

    interface Geo {

    public function getLatLng();

    }

    class OrderGeoLocator implements Job, Geo {

    public function process();

    public function getLatLng();

    }

    View Slide

  37. View Slide

  38. Design Principles
    Dependency Inversion

    Principle

    (DIP)

    View Slide

  39. Design Principles

    View Slide

  40. Design Principles

    "Code to the interface"
    - Me & others

    View Slide

  41. DIP
    class SuperUser {}

    $user = new SuperUser;

    class Post {

    function setUser(SuperUser $user);

    }

    View Slide

  42. DIP
    interface User {}

    class SuperUser implements User {}

    $user = new SuperUser;

    class Post {

    function setUser(User $user);

    }

    View Slide

  43. DIP
    interface User {}

    class SuperUser implements User {}

    class RegularUser implements User {}

    $user = new RegularUser;

    class Post {

    function setUser(User $user);

    }

    View Slide

  44. View Slide

  45. Architecture
    Patterns

    View Slide

  46. View Slide

  47. Application
    Architectures

    View Slide

  48. View Slide

  49. App Architectures
    Layered / N-Tiered

    View Slide

  50. App Architectures
    Presentation Logic Data
    HTML / CSS
    Javascript
    Templating
    PHP
    APIs
    Glue
    Postgres
    Redis
    Config

    View Slide

  51. App Architectures
    Presentation Logic Data
    Authentication
    Product
    Cart

    View Slide

  52. App Architectures
    HTML / CSS
    PHP
    DB
    CLIENT

    View Slide

  53. App Architectures
    CLIENT
    Presentation
    Cart

    View Slide

  54. App Architectures
    Presentation Logic Data
    Authentication
    Product
    Cart

    View Slide

  55. Platform Architectures
    Microkernel

    View Slide

  56. App Architectures
    CORE
    Plugin
    Component
    Extension
    i
    n
    t
    e
    r
    f
    a
    c
    e

    View Slide

  57. App Architectures
    DB
    Postgres
    MySQL
    SQL Server
    P
    D
    O

    View Slide

  58. Platform
    Architectures

    View Slide

  59. Platform Architectures
    Microservices
    Monolith
    SOA

    View Slide

  60. View Slide

  61. Platform Architectures
    Microservices
    SOA
    Monolith

    View Slide

  62. Platform Architectures
    Monolith

    View Slide

  63. Platform Architectures
    "Almost all the successful
    microservice stories have started
    with a monolith that got too big
    and was broken up" 

    - Martin Fowler

    View Slide

  64. Platform Architectures
    Coupling

    View Slide

  65. Platform Architectures
    Development Speed

    View Slide

  66. Platform Architectures
    Maintenance

    View Slide

  67. Platform Architectures
    Monorepo Myth

    View Slide

  68. Platform Architectures
    Trunk Based

    View Slide

  69. Platform Architectures
    Scaling

    View Slide

  70. Platform Architectures
    Microservices
    Monolith
    SOA

    View Slide

  71. Platform Architectures
    Coupling

    View Slide

  72. Platform Architectures
    Team Ownership

    View Slide

  73. Platform Architectures
    APIs as

    First Class Citizens

    View Slide

  74. Platform Architectures
    Scaling

    View Slide

  75. Platform Architectures
    Routing

    View Slide

  76. Platform Architectures
    Many Monoliths

    View Slide

  77. Platform Architectures
    Repo Management

    View Slide

  78. Platform Architectures
    Monolith
    SOA
    Microservices

    View Slide

  79. Platform Architectures
    Coupling

    View Slide

  80. Platform Architectures
    Deployments

    View Slide

  81. Platform Architectures
    Scaling

    View Slide

  82. Platform Architectures
    More Services

    =

    More Problems
    - Notorious B.I.G.

    View Slide

  83. Platform Architectures
    Microservices
    Monolith
    SOA

    View Slide

  84. Data
    Architectures

    View Slide

  85. Data Architectures
    https://github.com/UWCoffeeNCode/resources/wiki/SQL-and-NoSQL-Databases

    View Slide

  86. Data Architectures
    Relational / SQL

    View Slide

  87. Data Architectures
    NoSQL

    View Slide

  88. Data Architectures
    Time Series DBs

    View Slide

  89. Data Architectures
    Combos

    View Slide

  90. Data Architectures
    Scaling

    View Slide

  91. View Slide

  92. Example
    Design

    View Slide

  93. Pastebin
    Example

    View Slide

  94. Pastebin
    Requirements
    • Save with unique URL

    • Retrieve code via URL

    • Don't break

    View Slide

  95. Pastebin
    Scale
    • Writes / second

    • Reads / second

    • Storage

    View Slide

  96. Pastebin
    Scale
    • Writes / second

    • Reads / second

    • Storage
    1 x 86400
    50 x 86400
    86400x10kb = 864MB/day

    View Slide

  97. Pastebin
    Scale
    (864MB/day x 30) x 6 =
    155 GB x 1.25 = 193GB

    View Slide

  98. Pastebin
    Scale
    (864MB/day x 30) x 6 =
    155 GB x 1.25 = 193GB
    200GB for 6 months

    View Slide

  99. Pastebin
    API
    • GET /$primaryKey

    • POST /

    View Slide

  100. Pastebin
    Database
    • id: serial

    • content: text

    • created_at: datetime

    View Slide

  101. Pastebin
    CLIENT API
    DB

    View Slide

  102. Pastebin
    ⚠ Scenario ⚠
    • IDs becoming too long

    • Don't want users guessing
    other's URLs

    View Slide

  103. Pastebin
    Hashing
    86400 per day
    86400 x (365 x 10)
    316 million hashes

    View Slide

  104. Pastebin
    Hashing
    base64 encoding
    64^5
    >1 billion

    View Slide

  105. Pastebin
    Database
    • hash: varchar(5)

    • content: text

    • created_at: datetime

    View Slide

  106. Pastebin
    API
    • POST /

    -> generateHash()

    -> checkHash()

    -> save()


    View Slide

  107. Pastebin
    API
    • POST /

    -> getHash()

    -> save()


    View Slide

  108. Pastebin
    CLIENT API Key
    Serv
    DB

    View Slide

  109. Pastebin
    ⚠ Scenario ⚠
    • Large pastes could fill up 

    the DB

    View Slide

  110. Pastebin
    Database
    • hash: varchar(5)

    • created_at: datetime

    View Slide

  111. Pastebin
    Database
    • hash: varchar(5)

    • created_at: datetime
    Object Storage
    • {$hash}

    View Slide

  112. Pastebin
    CLIENT API
    Obj
    DB
    Key
    Serv

    View Slide

  113. Pastebin
    API
    • POST /

    -> getHash()

    -> saveDb((){

    saveObj()

    })


    View Slide

  114. Pastebin
    CLIENT API
    Obj
    Cache
    DB
    Key
    Serv

    View Slide

  115. Pastebin
    ⚠ Next Steps ⚠
    • More Caching

    • Load Balancing

    • Separate reads / writes

    • Reporting System

    View Slide

  116. Pastebin
    CLIENT
    Obj
    Cache
    Service
    Service
    DB
    Key
    Serv

    View Slide

  117. Pastebin
    CLIENT API
    Obj
    Cache
    R
    e
    q
    u
    e
    s
    t
    DB
    Key
    Serv

    View Slide

  118. Pastebin
    CLIENT
    API
    Obj
    R
    e
    q
    u
    e
    s
    t
    API
    DB DB
    DB
    Key
    Serv

    View Slide

  119. Pastebin
    CLIENT
    API
    Obj
    R
    e
    q
    u
    e
    s
    t
    API
    DB DB
    DB
    Key
    Serv
    Iteration =

    View Slide

  120. Q/A

    View Slide

  121. Resources

    View Slide

  122. Resources

    View Slide

  123. Resources

    View Slide

  124. Resources

    View Slide

  125. Thank You
    @benedmunds
    [email protected]

    View Slide