Tweet
Tweet

Slide 1

Slide 1 text

Scaling Development with Kubernetes Filipe Dobreira | September 2018

Slide 2

Slide 2 text

We’re hiring! - Front-End Developer (React) - Back-end Developer (Ruby) - DevOps Engineer More positions and details at:
 jobs.impraise.com

Slide 3

Slide 3 text

Planning Analysis & Design Implementation Testing Maintenance

Slide 4

Slide 4 text

Planning Analysis & Design Implementation Testing Maintenance ???

Slide 5

Slide 5 text

Planning Analysis & Design Implementation Testing Maintenance

Slide 6

Slide 6 text

Blunder years A slow, painful and error-prone development process

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

A workflow fit for a small startup AWS Elastic Beanstalk Cumbersome interfaces, slow and unreliable update procedure Zero visibility Knowing what was deployed where was mostly done by asking through Slack or in person Manual everything Development and production deployments were almost completely manual

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

Buzz words to the rescue Docker, Kubernetes and our vision of a better workflow

Slide 14

Slide 14 text

why docker?

Slide 15

Slide 15 text

why docker?

Slide 16

Slide 16 text

Consistency and predictability
 
 Jul 20, 1969 - 10 reasons you should use Docker. Number 10 will SHOCK you! About 28,400,000 results (0.31 seconds) All Portability
 
 A brownie recipe. But first, my entire life story. Orchestration
 
 Sep 17, 1991 - The Docker secret doctors don’t want you to know! why docker?

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

Why Kubernetes? Unified management As our needs were growing, Kubernetes offered a predictable solution for managing our stack at our current and future scale Support Official and community documentation, tooling was solid Extremely flexible Kubernetes’ declarative approach and complete API means it’s easy to build on

Slide 19

Slide 19 text

Kubernetes filled our orchestration gap, and gave us a foundation to build on

Slide 20

Slide 20 text

Our assumptions about Kubernetes & developers A lot to learn We thought developers wouldn’t want to learn an entire new platform just to have their code deployed Hard to manage We thought developers wouldn’t want to deal with Kubernetes’ particulars in day-to-day operations No structure We thought Kubernetes’ bare-bones resource tooling wasn’t enough for us

Slide 21

Slide 21 text

The K stands for Kubernetes

Slide 22

Slide 22 text

The Kapow stands for whatever this is

Slide 23

Slide 23 text

Kapow Deployment es Deployment Kubernetes Ingress Kubernetes Service Kubernetes S Kubernetes Jeff Bezos Cloud Magic

Slide 24

Slide 24 text

A tool for deployment management at Impraise Tailored to fit Kapow speaks our language, and is built for our workflow Complete toolchain It includes a full toolchain for managing the development lifecycle, including monitoring and troubleshooting An API to build on Additional tools and services can build on its high- level API

Slide 25

Slide 25 text

manifestVersion: “1.1”
 name: my-app
 args:
 version: latest
 endpoints:
 exposes:
 - name: app
 host: app.impraise. containers:
 - name: app
 image: “app:{{ versio resources:
 requests:
 cpu: 1m
 env:
 - name: NODE_ENV
 1
 2
 3
 4
 5
 6
 7
 8
 9
 10
 11
 12
 13
 14
 15
 A unified configuration file

Slide 26

Slide 26 text

1
 2
 3
 4
 5
 6
 7
 8
 9
 10
 11
 12
 13
 14
 15
 manifestVersion: “1.1”
 name: my-app
 args:
 version: latest
 endpoints:
 exposes:
 - name: app
 host: app.impraise.io
 containers:
 - name: app
 image: “app:{{ version }}”
 resources:
 requests:
 cpu: 1m
 env:
 - name: NODE_ENV


Slide 27

Slide 27 text

1
 2
 3
 4
 5
 6
 7
 8
 9
 10
 11
 12
 13
 14
 15
 manifestVersion: “1.1”
 name: my-app
 args:
 version: latest
 endpoints:
 exposes:
 - name: app
 host: app.impraise.io
 containers:
 - name: app
 image: “app:{{ version }}”
 resources:
 requests:
 cpu: 1m
 env:
 - name: NODE_ENV


Slide 28

Slide 28 text

apiVersion: v1 kind: Pod metadata: name: myapp-pod labels: app: myapp spec: containers: - name: myapp-container image: node resources: requests:
 cpu: 1m
 env:
 1
 2
 3
 4
 5
 6
 7
 8
 9
 10
 11
 12
 13
 14
 15
 manifestVersion: “1.1”
 name: my-app
 args:
 version: latest
 endpoints:
 exposes:
 - name: app
 host: app.impraise.io
 containers:
 - name: app
 image: “app:{{ version }}”
 resources:
 requests:
 cpu: 1m
 env:
 - name: NODE_ENV


Slide 29

Slide 29 text

1
 2
 3
 4
 5
 6
 7
 8
 9
 10
 11
 12
 13
 14
 15
 manifestVersion: “1.1”
 name: my-app
 args:
 version: latest
 endpoints:
 exposes:
 - name: app
 host: app.impraise.io
 containers:
 - name: app
 image: “app:{{ version }}”
 resources:
 requests:
 cpu: 1m
 env:
 - name: NODE_ENV


Slide 30

Slide 30 text

Sane(r) service configuration
 Service configuration is centrally managed, and can be injected at deployment time NODE_ENV ASSET_HOST “cdn.foo App Container 1 SECRET_KEY “xR3edxE49sA4dg0” “product App Container 2

Slide 31

Slide 31 text

It’s APIs all the way down Authentication layer Before more powerful authentication solutions for Kubernetes were widely available and supported Service discovery Development releases can automatically find different versions of backend services to work with Built for our use-cases Because we know our bottlenecks, we can build Kapow to help us overcome them

Slide 32

Slide 32 text

Kapow implements our idea of what a deployment looks like at Impraise

Slide 33

Slide 33 text

Robin Kapow CLI client

Slide 34

Slide 34 text

$ robin env pr-3071-filp RAILS_ENV=production Environment variables updated for next deployment version

Slide 35

Slide 35 text

$ robin deploy impraise/project -a version:latest Created deployment with plan ID f76efe16-093b-41e5-afa0

Slide 36

Slide 36 text

$ robin ls $ robin logs pr-3071 —follow $ robin exec pr-3071 sh

Slide 37

Slide 37 text

Qarla ChatOps at Impraise

Slide 38

Slide 38 text

Visibility is central to a healthy DevOps process

Slide 39

Slide 39 text

qarla 11:36 AM APP Deployed https://github.com/impraise/project/pull/30 Auto deployed by Qarla
 Deployment of #3071 (branch-name)
 
 status commit
 pending 0fb347c
 
 commit message
 Fix everything
 
 Use ‘@qarla help’ for commands to interact with this deployment Channels
 #development
 #deploys
 #dogs 1

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

qarla 02:01 AM APP I started that deployment for you, check #deploys
 for updates Channels
 #development
 #deploys
 #dogs filipe 02:01 AM @qarla deploy web#3071

Slide 43

Slide 43 text

qarla 11:00 AM APP The deployment train is ready to depart! @filipe @arnaud your work is on the train, please review
 the changes and stick around in case of issues.
 
 The train will depart in 15 minutes. Changes for web
 5 commits (2 merge commits) Channels
 #development
 #deploys
 #dogs 1 Changes for next
 18 commits (6 merge commits) 3

Slide 44

Slide 44 text

The deployment train explained On time Developers know exactly when it’ll start, and by when they must have their work ready to catch it Strict, but forgiving Developers can ask for a 30 minute delay twice per day, but otherwise the train departs with what’s ready Highly visible The deployment train is loud and visible, and makes successful deployments a team effort and core part of the workflow

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

Lessons learned Misguided expectations, and a lesson in keeping it simple, stupid

Slide 47

Slide 47 text

Lesson 1 Resist the urge to build everything from scratch Templating and Versioning Tools like Helm grew to offer complete and capable suites for managing large Kubernetes projects Authorization RBAC and integration with solutions like Amazon Web Services’ IAM are more secure and flexible DNS and Service Discovery Tools like ExternalDNS and Kubernetes’ native service layer are a powerful toolset on their own

Slide 48

Slide 48 text

Lesson 2 You’re building a product, and the developer is the client Talk to the client Include developers in the early stages of the process, and use their feedback Start small Give yourself time and space to iterate, and resist over-building from the start Find the line Find where the line lies between Development and Ops for your team, and decide where it should be blurred

Slide 49

Slide 49 text

Lesson 3 Good visibility makes the bad days suck less Share a common language Talking about your processes is easier if everyone’s using the same words Be loud The strength of ChatOps lies in its ability to be visible and loud, use it to rally the team around what’s important, and bring clarity to confusing situations

Slide 50

Slide 50 text

Get in touch: Filipe Dobreira
 Head of Infrastructure & Security
 filipe@impraise.com Thank you!