Pushing Kubernetes Forward

Slide 1

Slide 1 text

Pushing Kubernetes Forward Brandon Philips @brandonphilips | brandon@coreos.com | coreos.com

Slide 2

Slide 2 text

CoreOS, Inc (2013 - today) Mission: "Secure the Internet" Started at the OS level: CoreOS Linux ● Modern, minimal operating system ● Self-updating (read-only) image ● Updates must be automatic and seamless

Slide 3

Slide 3 text

app1 app2 app3 server1 app4 app5 server2 app6 app7 server3

Slide 4

Slide 4 text

app1 app2 app3 server1 app4 app5 server2 app6 app7 server3 updating...

Slide 5

Slide 5 text

app1 app2 app3 server1 app4 app5 server2 app6 app7 server3 needs reboot

Slide 6

Slide 6 text

app1 app2 app3 server1 app4 app5 server2 app6 app7 server3 rebooting... Without orchestration

Slide 7

Slide 7 text

app1 app2 app3 server1 app4 app5 server2 app6 app7 server3 rebooting... Without orchestration

Slide 8

Slide 8 text

app1 app2 app3 server1 app4 app5 server2 app6 app7 server3 needs reboot With orchestration magical orchestrator

Slide 9

Slide 9 text

server1 app4 app5 app1 app3 server2 app6 app7 app2 server3 needs reboot With orchestration magical orchestrator

Slide 10

Slide 10 text

server1 app4 app5 app1 app3 server2 app6 app7 app2 server3 rebooting... With orchestration magical orchestrator

Slide 11

Slide 11 text

server1 app4 app5 app1 app3 server2 app6 app7 app2 server3 updated! With orchestration magical orchestrator

Slide 12

Slide 12 text

app2 app3 server1 app4 app5 app1 server2 app6 app7 server3 updated! With orchestration magical orchestrator

Slide 13

Slide 13 text

app2 app3 server1 app4 app5 app1 server2 app6 app7 server3 With orchestration

Slide 14

Slide 14 text

90+ Projects on GitHub, 1,000+ Contributors OPEN SOURCE CoreOS.com - @coreoslinux - github/coreos Secure solutions, support plans, training + more ENTERPRISE sales@coreos.com - tectonic.com - quay.io

Slide 15

Slide 15 text

Product Management via Keynote Users running Kubernetes infrastructure Community building Kubernetes Businesses building products on Kubernetes

Slide 16

Slide 16 text

Where We Are Pushing Kubernetes Simpler to deploy and configure clusters Increasing scale of clusters throughout stack Security based on good practices rkt engine powering Kubernetes nodes Standards to ensure portability

Slide 17

Slide 17 text

Simpler Deployment self-hosted k8s

Slide 18

Slide 18 text

worker kubelet worker kubelet worker kubelet scheduler & API worker kubelet w ku t worker kubelet

Slide 19

Slide 19 text

API Server scheduler controller manager

Slide 20

Slide 20 text

And a few more pieces in containers DNS addon replica set Heapster and InfluxDB Networking daemon set Identity and authz services

Slide 21

Slide 21 text

How do we install it all? Manually place configuration Cloud-config and bash Config management

Slide 22

Slide 22 text

How do we install it all? Manually place configuration Cloud-config and bash Config management

Slide 23

Slide 23 text

How do we upgrade it all?

Slide 24

Slide 24 text

$ monokube --nodes=172.17.8.101,172.17.8.102,... monokube - a prototype ssh reverse tunnel

Slide 25

Slide 25 text

$ monokube --nodes=172.17.8.101,172.17.8.102,... monokube - a prototype deploy API server

Slide 26

Slide 26 text

$ monokube --nodes=172.17.8.101,172.17.8.102,... monokube - a prototype re-configure API cfg

Slide 27

Slide 27 text

Self-hosting Kubernetes Pivot

Slide 28

Slide 28 text

kubectl Self-hosting Kubernetes Pivot

Slide 29

Slide 29 text

That seems hard, what do we get? Bootstrap requirements down to working SSH Rolling updates for Kubernetes itself! Kubelet version controlled by API Help Wanted! Goal: working in v1.3

Slide 30

Slide 30 text

Simpler Deployment join sig-high-availability

Slide 31

Slide 31 text

Increasing Scale scheduler improvements

Slide 32

Slide 32 text

https://coreos.com/blog/improving-kubernetes-scheduler-performance.html 10x Improvement in scheduler throughput Ongoing work to track upstream performance Let's make similarly large gains in v1.3 Help wanted: Kubemark dashboard!

Slide 33

Slide 33 text

Increasing Scale etcd v3 in k8s

Slide 34

Slide 34 text

etcd v3.0 - "Scaling etcd to thousands of nodes" ● Efficient transport via gRPC and HTTP/2 ● New powerful API based on k8s use-case ● Disk-backed and memory efficient storage ● Incremental snapshot for consistent performance ● Fix re-list issues with longer and memory-efficient key history

Slide 35

Slide 35 text

v3 API - Transactions ● compare and swap ○ compare: foo=bar ○ success: foo=bar2 ● multiple object transaction ○ compare: cond1=true && cond2=true ○ success: pass=true ○ failure: pass=false

Slide 36

Slide 36 text

v3 API - Watches ● support multiple keys and prefixes per stream ○ watchKey(foo) ○ watchPrefix(coreos) ● support watch from historical point ○ watchKey(foo, index_of_an_hour_ago) ○ user-driven history compaction

Slide 37

Slide 37 text

v3 API - Lease l := lease.Create(10*second) kv.Put("foo", "bar", l.ID) // key will be removed without keeping // alive the lease go KeepAlive(l.id)

Slide 38

Slide 38 text

Help Wanted: mirror maker Label queries are the new DNS Need API mirrors to give queries 100% uptime Help wanted, no work started.

Slide 39

Slide 39 text

When is the release?

Slide 40

Slide 40 text

When is it in k8s? ● etcd v3 k8s issue #22448 ○ Refactoring the storage interface ○ Proof of concept working

Slide 41

Slide 41 text

Increasing Scale join sig-scalability

Slide 42

Slide 42 text

Security Through Identity OIDC in Kubernetes

Slide 43

Slide 43 text

Dex - OIDC Provider Open source standards based identity-provider SQL, LDAP, and other identity backend connectors Applicable outside of Kubernetes but that is our use case

Slide 44

Slide 44 text

OIDC Relying Party

Slide 45

Slide 45 text

OIDC End User

Slide 46

Slide 46 text

OIDC Identity Provider

Slide 47

Slide 47 text

OIDC Identity Provider

Slide 48

Slide 48 text

OIDC Identity Provider

Slide 49

Slide 49 text

OIDC Identity Provider

Slide 50

Slide 50 text

OIDC 0. Relying party periodically syncs public key from IdP

Slide 51

Slide 51 text

1. User request protected page OIDC

Slide 52

Slide 52 text

2. User redirected to auth page OIDC

Slide 53

Slide 53 text

3. User authenticates (cookie/pw) OIDC

Slide 54

Slide 54 text

4. User given authz grant OIDC

Slide 55

Slide 55 text

5. User presents grant to client OIDC

Slide 56

Slide 56 text

6. Relying party exchanges authz code for ID token OIDC

Slide 57

Slide 57 text

7. Client gets ID token and validate claims OIDC

Slide 58

Slide 58 text

JWT JSON Web Token

Slide 59

Slide 59 text

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9. eyJzdWIiOiIyNDgyODk3NjEwMDEiLCJuYW1lI joiSmFuZSBEb2UiL... mphbmVkb2VAZXhhbXBsZS5jb20iLCJwaWN 0dXJlIjoiaHR0cDovL2V4YW1wbGUuY29tL2ph bmVkb2UvbWUuanBnIn0. TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeF ONFh7HgQ

Slide 60

Slide 60 text

{ "alg": "HS256", "typ": "JWT" }

Slide 61

Slide 61 text

{ "sub": "248289761001", "name": "Ada Richmond", "preferred_username": "ada", "email": "ada.richmond@example.com", "groups": ["read-prod", "admin-stage"] }

Slide 62

Slide 62 text

Groups and Kubernetes API server extracts user, email, groups, from OIDC token Now what?

Slide 63

Slide 63 text

Webhook Authorizer "kind": "SubjectAccessReview", "spec": { "resourceAttributes": { "namespace": "default", "verb": "GET", "group": "group3", "resource": "pods" }, "user": "ada", "group": ["read-prod", "admin-stage" ] } authorizer service OK?

Slide 64

Slide 64 text

Security Through Identity OIDC in Kubernetes

Slide 65

Slide 65 text

rkt Powered Kubernetes mid-flight engine swap

Slide 66

Slide 66 text

a modern, secure container runtime a simple, composable tool focused on kubernetes

Slide 67

Slide 67 text

no central daemon no (mandatory) API apps run directly under spawning process rkt - simple CLI tool

Slide 68

Slide 68 text

bash/systemd/kubelet rkt run ... application(s)

Slide 69

Slide 69 text

modular architecture take advantage of different technologies provide a consistent experience to users rkt internals

Slide 70

Slide 70 text

Nearly complete! 80% of end-to-end tests passing cAdvisor integration in progress rktnetes today

Slide 71

Slide 71 text

LIVE DEMO rktnetes today

Slide 72

Slide 72 text

Goal: 100% end-to-end tests working User may switch to rktnetes with zero suprises rktnetes today

Slide 73

Slide 73 text

rkt Powered Kubernetes join sig-node

Slide 74

Slide 74 text

Security TPM Log

Slide 75

Slide 75 text

● TPM, Trusted Platform Module ○ physical chip on the motherboard ○ cryptographic keys + processor ● Used to "measure" system state ● Historically just use to verify bootloader/OS (on proprietary systems) rkt TPM measurement

Slide 76

Slide 76 text

● CoreOS added support to GNU Grub ● rkt can now record information about running pods in the TPM ● attestable record of what images and pods are running on a system rkt TPM measurement

Slide 77

Slide 77 text

rkt TPM measurement

Slide 78

Slide 78 text

https://coreos.com/blog/coreos-trusted-computing.html Tectonic Trusted Computing

Slide 79

Slide 79 text

TPM Attestation in k8s 1. Generated timestamp 2. Ask TPM for sig of time + log value 3. Submit to API server in nodeStatus

Slide 80

Slide 80 text

TPM Attestation in k8s Goal: Merge nodeStatus payload upstream in k8s v1. 3

Slide 81

Slide 81 text

rkt TPM measurement For more TPM and rkt, see Matthew Garrett's talk: "Integrated trusted computing in Kubernetes" 11: 30am today

Slide 82

Slide 82 text

Security TLS Bootstrap

Slide 83

Slide 83 text

TLS Bootstrap of Nodes (#20439) 1. Generate CSR 2. Submit CSR to API server 3. Poll for approved CSR

Slide 84

Slide 84 text

TLS Bootstrap of Nodes (#20439) Goal: Merge proposal and working code into v1.3

Slide 85

Slide 85 text

Industry Movement

Slide 86

Slide 86 text

No content

Slide 87

Slide 87 text

● Coordinate promotion of Cloud Native architectures ● A home for Cloud Native OSS projects like Kubernetes ○ Technical board to evaluate additional projects ● Provides shared resources to projects like video conferencing, test servers, etc

Slide 88

Slide 88 text

● Creating technical standards for containers ● Started with runC and a runtime specification ● Large mandate to standardize an image format ○ In-progress

Slide 89

Slide 89 text

Multiple Image Formats in v1.3 API ● Today Kubernetes only supports the Docker Image Format and naming ● Use cases for executing other formats ○ OCI Image Format ○ tar archive chroots ○ jar? ○ static binary? ● Support signing and content verification

Slide 90

Slide 90 text

Help Push Kubernetes Forward Simpler to deploy and configure clusters Increasing scale of clusters throughout stack Security based on good practices rkt engine powering Kubernetes nodes Standards to ensure portability

Slide 91

Slide 91 text

No content

Slide 92

Slide 92 text

coreos.com/fest - @coreosfest May 9 & 10, 2016 - Berlin, Germany

Slide 93

Slide 93 text

Thank you! Brandon Philips @brandonphilips | brandon@coreos.com | coreos.com We’re hiring in all departments! Email: careers@coreos.com Positions: coreos.com/ careers