Midwest PHP 2020 - Web Scale System Design and Architecture

Slide 1

Slide 1 text

web scale & System Design Architecture

Slide 2

Slide 2 text

Who is this guy? Ben Edmunds Open Source Author PHP Town Hall Podcast Staff Eng @ Wayfair

Slide 3

Slide 3 text

Design Principles

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

Design Principles Single Responsibility Principle (SRP)

Slide 9

Slide 9 text

Design Principles

Slide 10

Slide 10 text

Design Principles "what I mean by 'focusing one's attention upon some aspect': it does not mean ignoring the other aspects, it is just doing justice to the fact that from this aspect's point of view, the other is irrelevant. It is being one- and multiple-track minded simultaneously." - Edsger W. Dijkstra

Slide 11

Slide 11 text

SRP interface User { public function getEmail(); public function savePost(); }

Slide 12

Slide 12 text

SRP interface User { public function getEmail(); } interface Post { public function setUserId(); public function save(); }

Slide 13

Slide 13 text

SRP interface Order { public function getTotal(); public function getProducts(); public function getPdf(); }

Slide 14

Slide 14 text

SRP interface Order { public function getTotal(); public function getProducts(); } interface OrderPdfReport { public function setOrder(Order $order); public function getPdf(); }

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

Design Principles Open-Closed Principle (OCP)

Slide 17

Slide 17 text

Design Principles

Slide 18

Slide 18 text

Design Principles OPEN  "A module will be said to be open if it is still available for extension. For example, it should be possible to add ﬁelds to the data structures it contains, or new elements to the set of functions it performs." CLOSED  "A module will be said to be closed if [it] is available for use by other modules. This assumes that the module has been given a well-deﬁned, stable description (the interface in the sense of information hiding)" - Bertrand Meyer

Slide 19

Slide 19 text

OCP class OrderPdfGeneratorJob { public function generatePdf(); } class OrderGeoLocatorJob { public function locate(); }

Slide 20

Slide 20 text

OCP class ProcessWorker { public function process($job) { if ($job instanceof OrderPdfGeneratorJob) { $job->generatePdf(); } elseif ($job instanceof OrderGeoLocatorJob) { $job->locate(); }

Slide 21

Slide 21 text

OCP interface Job { public function process(); } class OrderPdfGenerator implements Job { public function process(); } class OrderGeoLocator implements Job { public function process();

Slide 22

Slide 22 text

OCP class ProcessWorker { public function process(Job $job) { return $job->process(); } }

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

Design Principles Liskov Substitution Principle (LSP)

Slide 25

Slide 25 text

Design Principles

Slide 26

Slide 26 text

Design Principles   "Let Φ(x) be a property provable about objects x of type T. Then Φ(y) should be true for objects y of type S where S is a subtype of T." - Barbara Liskov

Slide 27

Slide 27 text

LSP interface Duck { public void walk(); public void quack(); } class Mallard implements Duck

Slide 28

Slide 28 text

LSP interface Duck { public void walk(); public void quack(); } class RubberDuck implements Duck

Slide 29

Slide 29 text

LSP interface WaterToy { public void isFun(); public void ﬂoats(); } class RubberDuck implements WaterToy

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

Design Principles Interface Segregation Principle (ISP)

Slide 32

Slide 32 text

Design Principles

Slide 33

Slide 33 text

Design Principles   "Classes that have “fat” interfaces are classes whose interfaces are not cohesive." - Robert C. Martin

Slide 34

Slide 34 text

ISP interface Job { public function process(); } class OrderPdfGenerator implements Job { public function process(); } class OrderGeoLocator implements Job { public function process();

Slide 35

Slide 35 text

ISP interface Job { public function process(); public function getLatLong(); } class OrderPdfGenerator implements Job { public function save(); uhOhhhh() }

Slide 36

Slide 36 text

ISP interface Job { public function process(); } interface Geo { public function getLatLng(); } class OrderGeoLocator implements Job, Geo { public function process(); public function getLatLng(); }

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

Design Principles Dependency Inversion Principle (DIP)

Slide 39

Slide 39 text

Design Principles

Slide 40

Slide 40 text

Design Principles   "Code to the interface" - Me & others

Slide 41

Slide 41 text

DIP class SuperUser {} $user = new SuperUser; class Post { function setUser(SuperUser $user); }

Slide 42

Slide 42 text

DIP interface User {} class SuperUser implements User {} $user = new SuperUser; class Post { function setUser(User $user); }

Slide 43

Slide 43 text

DIP interface User {} class SuperUser implements User {} class RegularUser implements User {} $user = new RegularUser; class Post { function setUser(User $user); }

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

Architecture Patterns

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

Application Architectures

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

App Architectures Layered / N-Tiered

Slide 50

Slide 50 text

App Architectures Presentation Logic Data HTML / CSS Javascript Templating PHP APIs Glue Postgres Redis Conﬁg

Slide 51

Slide 51 text

App Architectures Presentation Logic Data Authentication Product Cart

Slide 52

Slide 52 text

App Architectures HTML / CSS PHP DB CLIENT

Slide 53

Slide 53 text

App Architectures CLIENT Presentation Cart

Slide 54

Slide 54 text

App Architectures Presentation Logic Data Authentication Product Cart

Slide 55

Slide 55 text

Platform Architectures Microkernel

Slide 56

Slide 56 text

App Architectures CORE Plugin Component Extension i n t e r f a c e

Slide 57

Slide 57 text

App Architectures DB Postgres MySQL SQL Server P D O

Slide 58

Slide 58 text

Platform Architectures

Slide 59

Slide 59 text

Platform Architectures Microservices Monolith SOA

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

Platform Architectures Microservices SOA Monolith

Slide 62

Slide 62 text

Platform Architectures Monolith

Slide 63

Slide 63 text

Platform Architectures "Almost all the successful microservice stories have started with a monolith that got too big and was broken up"   - Martin Fowler

Slide 64

Slide 64 text

Platform Architectures Coupling

Slide 65

Slide 65 text

Platform Architectures Development Speed

Slide 66

Slide 66 text

Platform Architectures Maintenance

Slide 67

Slide 67 text

Platform Architectures Monorepo Myth

Slide 68

Slide 68 text

Platform Architectures Trunk Based

Slide 69

Slide 69 text

Platform Architectures Scaling

Slide 70

Slide 70 text

Platform Architectures Microservices Monolith SOA

Slide 71

Slide 71 text

Platform Architectures Coupling

Slide 72

Slide 72 text

Platform Architectures Team Ownership

Slide 73

Slide 73 text

Platform Architectures APIs as First Class Citizens

Slide 74

Slide 74 text

Platform Architectures Scaling

Slide 75

Slide 75 text

Platform Architectures Routing

Slide 76

Slide 76 text

Platform Architectures Many Monoliths

Slide 77

Slide 77 text

Platform Architectures Repo Management

Slide 78

Slide 78 text

Platform Architectures Monolith SOA Microservices

Slide 79

Slide 79 text

Platform Architectures Coupling

Slide 80

Slide 80 text

Platform Architectures Deployments

Slide 81

Slide 81 text

Platform Architectures Scaling

Slide 82

Slide 82 text

Platform Architectures More Services = More Problems - Notorious B.I.G.

Slide 83

Slide 83 text

Platform Architectures Microservices Monolith SOA

Slide 84

Slide 84 text

Data Architectures

Slide 85

Slide 85 text

Data Architectures https://github.com/UWCoffeeNCode/resources/wiki/SQL-and-NoSQL-Databases

Slide 86

Slide 86 text

Data Architectures Relational / SQL

Slide 87

Slide 87 text

Data Architectures NoSQL

Slide 88

Slide 88 text

Data Architectures Time Series DBs

Slide 89

Slide 89 text

Data Architectures Combos

Slide 90

Slide 90 text

Data Architectures Scaling

Slide 91

Slide 91 text

No content

Slide 92

Slide 92 text

Example Design

Slide 93

Slide 93 text

Pastebin Example

Slide 94

Slide 94 text

Pastebin Requirements • Save with unique URL • Retrieve code via URL • Don't break

Slide 95

Slide 95 text

Pastebin Scale • Writes / second • Reads / second • Storage

Slide 96

Slide 96 text

Pastebin Scale • Writes / second • Reads / second • Storage 1 x 86400 50 x 86400 86400x10kb = 864MB/day

Slide 97

Slide 97 text

Pastebin Scale (864MB/day x 30) x 6 = 155 GB x 1.25 = 193GB

Slide 98

Slide 98 text

Pastebin Scale (864MB/day x 30) x 6 = 155 GB x 1.25 = 193GB 200GB for 6 months

Slide 99

Slide 99 text

Pastebin API • GET /$primaryKey • POST /

Slide 100

Slide 100 text

Pastebin Database • id: serial • content: text • created_at: datetime

Slide 101

Slide 101 text

Pastebin CLIENT API DB

Slide 102

Slide 102 text

Pastebin ⚠ Scenario ⚠ • IDs becoming too long • Don't want users guessing other's URLs

Slide 103

Slide 103 text

Pastebin Hashing 86400 per day 86400 x (365 x 10) 316 million hashes

Slide 104

Slide 104 text

Pastebin Hashing base64 encoding 64^5 >1 billion

Slide 105

Slide 105 text

Pastebin Database • hash: varchar(5) • content: text • created_at: datetime

Slide 106

Slide 106 text

Pastebin API • POST /  -> generateHash()  -> checkHash()  -> save() 

Slide 107

Slide 107 text

Pastebin API • POST /  -> getHash()  -> save() 

Slide 108

Slide 108 text

Pastebin CLIENT API Key Serv DB

Slide 109

Slide 109 text

Pastebin ⚠ Scenario ⚠ • Large pastes could ﬁll up   the DB

Slide 110

Slide 110 text

Pastebin Database • hash: varchar(5) • created_at: datetime

Slide 111

Slide 111 text

Pastebin Database • hash: varchar(5) • created_at: datetime Object Storage • {$hash}

Slide 112

Slide 112 text

Pastebin CLIENT API Obj DB Key Serv

Slide 113

Slide 113 text

Pastebin API • POST /  -> getHash()  -> saveDb((){  saveObj()  }) 

Slide 114

Slide 114 text

Pastebin CLIENT API Obj Cache DB Key Serv

Slide 115

Slide 115 text

Pastebin ⚠ Next Steps ⚠ • More Caching • Load Balancing • Separate reads / writes • Reporting System