The Journey to #GIFEE: Intro to the Open Source

Slide 1

Slide 1 text

Brandon Philips @brandonphilips | brandon.philips@coreos.com The Journey to #GIFEE Intro to the Open Source

Slide 2

Slide 2 text

Secure the Internet MISSION

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

3 Application packaging Clustering Linux at scale

Slide 10

Slide 10 text

#GIFEE Borg/Omega Linux Chubby

Slide 11

Slide 11 text

#GIFEE Borg/Omega Linux Chubby

Slide 12

Slide 12 text

#GIFEE Borg/Omega Linux Chubby

Slide 13

Slide 13 text

Avoid single points of failure Design for constant updates Consistent environment Why build #GIFEE?

Slide 14

Slide 14 text

Why build #GIFEE? Design for constant updates

Slide 15

Slide 15 text

Application Packaging 1

Slide 16

Slide 16 text

Abstract away app from the OS OS App

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

Base software managed by CoreOS kernel systemd OpenSSH

Slide 20

Slide 20 text

Protect apps from each other Isolated network namespace Isolated file system namespace Mixed versions of dependencies eg. python 3.4 & python 2.7

Slide 21

Slide 21 text

$ sudo rkt run coreos.com/etcd:v2.0.0 $ sudo rkt run coreos.com/etcd:v2.0.0 \ --cpu=750m --memory=128M $ sudo rkt run --net=host coreos.com/etcd:v2.0.0 rkt run

Slide 22

Slide 22 text

Search container metadata Identify vulnerabilities Explain update actions Clair container security auditing

Slide 23

Slide 23 text

After scanning millions of containers we found that over 80% still had Heartbleed 80% Clair container security auditing

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

In-Progress Universal Container Format Packaged Downloaded Verified

Slide 26

Slide 26 text

Linux at Scale 2

Slide 27

Slide 27 text

Patches to the OS and kernel are hard Retest after updates No automation SECURITY Dependency breakage Uptime risk APPLICATION

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

Auto-updating browsers fixed security We got HTML5 at the same time

Slide 30

Slide 30 text

Atomic operating system updates

Slide 31

Slide 31 text

Atomic operating system updates

Slide 32

Slide 32 text

Clustering 3

Slide 33

Slide 33 text

Patches to the OS and kernel are hard Retest after updates No automation SECURITY Dependency breakage Uptime risk APPLICATION

Slide 34

Slide 34 text

Patches to the OS and kernel are hard No automation SECURITY Uptime risk APPLICATION

Slide 35

Slide 35 text

Operations Paradise Easy scale out Painless app upgrades Tolerant of machine failure

Slide 36

Slide 36 text

App Req/sec: 6,000 App Healthy: True

Slide 37

Slide 37 text

App Req/sec: 6,000 App Healthy: True

Slide 38

Slide 38 text

App Req/sec: 7,000 App Healthy: True

Slide 39

Slide 39 text

App Req/sec: 8,000 App Healthy: True

Slide 40

Slide 40 text

App Req/sec: 7,000 App Healthy: True

Slide 41

Slide 41 text

App Req/sec: 6,000 App Healthy: True

Slide 42

Slide 42 text

App Req/sec: 8,000 App Healthy: True

Slide 43

Slide 43 text

App Req/sec: 7,000 App Healthy: True

Slide 44

Slide 44 text

App Req/sec: 8,000 App Healthy: True

Slide 45

Slide 45 text

App Req/sec: 8,000 App Healthy: True

Slide 46

Slide 46 text

App Req/sec: 8,000 App Healthy: True

Slide 47

Slide 47 text

When do you need cluster coordination? Leader election Cluster-wide Semaphores Service discovery Dynamic configuration

Slide 48

Slide 48 text

Hard Computer Science Problem ?

Slide 49

Slide 49 text

Hard Computer Science Problem Chubby

Slide 50

Slide 50 text

A highly-available key value store for shared configuration and service discovery

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

No content

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

No content

Slide 56

Slide 56 text

No content

Slide 57

Slide 57 text

No content

Slide 58

Slide 58 text

No existing “cloud native” solutions Simple HTTP + JSON APIs Dynamic reconfiguration Why build etcd?

Slide 59

Slide 59 text

Simple key/value “Distributed etc” Keys are versioned Changes can be watch

Slide 60

Slide 60 text

Cluster-wide reboot lock - locksmith Service discovery - vulcand, skydns Cluster orchestration - k8s, cloud foundry

Slide 61

Slide 61 text

Industry Adoption 500+ projects on Github

Slide 62

Slide 62 text

App Req/sec: 8,000 App Healthy: True

Slide 63

Slide 63 text

App Req/sec: 8,000 App Healthy: True

Slide 64

Slide 64 text

App Req/sec: 8,000 App Healthy: True

Slide 65

Slide 65 text

App Req/sec: 8,000 App Healthy: True

Slide 66

Slide 66 text

Kubernetes is our choice for orchestration platform

Slide 67

Slide 67 text

Guides & Tools coreos.com/kubernetes kube-aws Cloud-configs

Slide 68

Slide 68 text

Upstream rktnetes Auth/OIDC Node self-signed TLS

Slide 69

Slide 69 text

Scaling 15x scheduler performance 30k pods on 1k nodes SIG-scale

Slide 70

Slide 70 text

3 Application packaging Clustering Linux at scale

Slide 71

Slide 71 text

Sounds good, but... Is anyone successful with all this in prod?

Slide 72

Slide 72 text

Publically traded options exchange

Slide 73

Slide 73 text

Containers on CoreOS are powering ISE's high- throughput, low-latency financial exchange Running in production Bare metal & AWS Billions of transactions a day 150 million req/sec

Slide 74

Slide 74 text

No content

Slide 75

Slide 75 text

Secure the Internet MISSION

Slide 76

Slide 76 text

coreos.com/fest - @coreosfest May 9 & 10, 2016 - Berlin, Germany

Slide 77

Slide 77 text

No content

Slide 78

Slide 78 text

Thank you! Brandon Philips @brandonphilips | brandon@coreos.com | coreos.com We’re hiring in all departments! Email: careers@coreos.com Positions: coreos.com/ careers