Infra as Code: The Organizational Bottleneck and the Shared Service Anti-Pattern

Slide 1

Slide 1 text

9/25/16 @evan2645 Infra as Code: The Organizational Bottleneck and the Shared Service Anti-Pattern EVAN GILMAN

Slide 2

Slide 2 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN About Me

Slide 3

Slide 3 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN

Slide 4

Slide 4 text

9/25/16 @evan2645 This Talk is NOT RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 5

Slide 5 text

9/25/16 @evan2645 This Talk IS RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 6

Slide 6 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN Infra as Code at PagerDuty Chef Evolution at PagerDuty Human Implication of Shared Services What We’re Trying and How to Avoid Agenda

Slide 7

Slide 7 text

9/25/16 @evan2645 Chef RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF Nomad + Mesos

Slide 8

Slide 8 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN Chef at PagerDuty

Slide 9

Slide 9 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN • Per-App Cookbooks Chef at PagerDuty

Slide 10

Slide 10 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN • Per-App Cookbooks • Many Common Cookbooks Chef at PagerDuty

Slide 11

Slide 11 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN • Per-App Cookbooks • Many Common Cookbooks • Mono-Repo Chef at PagerDuty

Slide 12

Slide 12 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN • Per-App Cookbooks • Many Common Cookbooks • Mono-Repo • No Versioning Chef at PagerDuty

Slide 13

Slide 13 text

9/25/16 @evan2645 Through the Ages RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 14

Slide 14 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN Baby Chef

Slide 15

Slide 15 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN Baby Chef

Slide 16

Slide 16 text

9/25/16 @evan2645 PD-Specific Tooling RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 17

Slide 17 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN • Lots of Chef Search PD-Specific Tooling

Slide 18

Slide 18 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN • Lots of Chef Search • Lots of External Calls PD-Specific Tooling

Slide 19

Slide 19 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN PD-Specific Tooling • Lots of Chef Search • Lots of External Calls • Linear Scaling == Bad

Slide 20

Slide 20 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN PD-Specific Tooling • Lots of Chef Search • Lots of External Calls • Linear Scaling == Bad • Chef as Shared Resource

Slide 21

Slide 21 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN • Lots of Chef Search • Lots of External Calls • Linear Scaling == Bad • Chef as Shared Resource • APIs as Shared Resources PD-Specific Tooling

Slide 22

Slide 22 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN • Instrument Everything • Concerted Effort • Some Specialize • Convention Emerges Optimization

Slide 23

Slide 23 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN Optimization • Instrument Everything • Concerted Effort • Some Specialize • Convention Emerges

Slide 24

Slide 24 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN Continued Growth

Slide 25

Slide 25 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN • Copy/Paste! Continued Growth

Slide 26

Slide 26 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN • Copy/Paste! • Refactors! Continued Growth

Slide 27

Slide 27 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN • Copy/Paste! • Refactors! • Services rarely identical in practice Continued Growth

Slide 28

Slide 28 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN • Copy/Paste! • Refactors! • Services rarely identical in practice • Limited Ops bandwidth Continued Growth

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Slide 31

Slide 31 text

9/25/16 @evan2645 Chef For All! RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 32

Slide 32 text

9/25/16 @evan2645 RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF Strong Convention

Slide 33

Slide 33 text

9/25/16 @evan2645 CONTRIBUTING.md RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 34

Slide 34 text

9/25/16 @evan2645 CONTRIBUTING.md RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 35

Slide 35 text

9/25/16 @evan2645 But… I just copy/pasted! RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 36

Slide 36 text

9/25/16 @evan2645 But… It’s just one query… RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 37

Slide 37 text

9/25/16 @evan2645 Hey… That’s not in CONTRIBUTING.md! RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 38

Slide 38 text

9/25/16 @evan2645 C’mon… I just wanna get stuff done RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 39

Slide 39 text

9/25/16 @evan2645 Shared Services Considered Harmful RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 40

Slide 40 text

9/25/16 @evan2645 Shared Services Considered Harmful RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 41

Slide 41 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN Solution?

Slide 42

Slide 42 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN Solution? Dev needs != Ops needs

Slide 43

Slide 43 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN Solution? Dev needs != Ops needs Chef interface complex and powerful

Slide 44

Slide 44 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN Solution? Dev needs != Ops needs Chef interface complex and powerful Like an ATV instead of a Golf Cart

Slide 45

Slide 45 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN Dev needs != Ops needs Chef interface complex and powerful Like an ATV instead of a Golf Cart Focus on Dev experience Solution?

Slide 46

Slide 46 text

9/25/16 @evan2645 Data as a Service Definition RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 47

Slide 47 text

9/25/16 @evan2645 Artifact + Service Definition RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 48

Slide 48 text

9/25/16 @evan2645 CI/CD Pipeline for Workloads RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 49

Slide 49 text

9/25/16 @evan2645 CI/CD Pipeline for Workloads RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF (with linters)

Slide 50

Slide 50 text

9/25/16 @evan2645 Work is Ongoing… RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 51

Slide 51 text

9/25/16 @evan2645 Work is Ongoing… RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF (we need help!)

Slide 52

Slide 52 text

9/25/16 @evan2645 Avoidance? RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 53

Slide 53 text

9/25/16 @evan2645 INFRA AS CODE: THE ORGANIZATIONAL BOTTLENECK AND THE SHARED SERVICE ANTI-PATTERN Always Know Your Customer Focus on the Infrastructure Interface/API Don’t Fight Conway’s Law Be Product-Focused

Slide 54

Slide 54 text

Slide 55

Slide 55 text

Slide 56

Slide 56 text

9/25/16 @evan2645 Thank You! RESILIENT INFRASTRUCTURE ORCHESTRATION WITH SERF

Slide 57

Slide 57 text

9/25/16 @evan2645 Infra as Code: The Organizational Bottleneck and the Shared Service Anti-Pattern EVAN GILMAN