Pushing Kubernetes Forward

KubeCon EU 2016

Transcript

1. Pushing Kubernetes Forward Brandon Philips @brandonphilips | brandon@coreos.com | coreos.com
2. CoreOS, Inc (2013 - today) Mission: "Secure the Internet" Started at the OS level: CoreOS Linux • Modern, minimal operating system • Self-updating (read-only) image • Updates must be automatic and seamless
3. app1 app2 app3 server1 app4 app5 server2 app6 app7 server3
4. app1 app2 app3 server1 app4 app5 server2 app6 app7 server3 updating...
5. app1 app2 app3 server1 app4 app5 server2 app6 app7 server3 needs reboot
6. app1 app2 app3 server1 app4 app5 server2 app6 app7 server3 rebooting... Without orchestration
7. app1 app2 app3 server1 app4 app5 server2 app6 app7 server3 rebooting... Without orchestration
8. app1 app2 app3 server1 app4 app5 server2 app6 app7 server3 needs reboot With orchestration magical orchestrator
9. server1 app4 app5 app1 app3 server2 app6 app7 app2 server3 needs reboot With orchestration magical orchestrator
10. server1 app4 app5 app1 app3 server2 app6 app7 app2 server3 rebooting... With orchestration magical orchestrator
11. server1 app4 app5 app1 app3 server2 app6 app7 app2 server3 updated! With orchestration magical orchestrator
12. app2 app3 server1 app4 app5 app1 server2 app6 app7 server3 updated! With orchestration magical orchestrator
13. app2 app3 server1 app4 app5 app1 server2 app6 app7 server3 With orchestration
14. 90+ Projects on GitHub, 1,000+ Contributors OPEN SOURCE CoreOS.com - @coreoslinux - github/coreos Secure solutions, support plans, training + more ENTERPRISE sales@coreos.com - tectonic.com - quay.io
15. Product Management via Keynote Users running Kubernetes infrastructure Community building Kubernetes Businesses building products on Kubernetes
16. Where We Are Pushing Kubernetes Simpler to deploy and configure clusters Increasing scale of clusters throughout stack Security based on good practices rkt engine powering Kubernetes nodes Standards to ensure portability
17. Simpler Deployment self-hosted k8s
18. worker kubelet worker kubelet worker kubelet scheduler & API worker kubelet w ku t worker kubelet
19. API Server scheduler controller manager
20. And a few more pieces in containers DNS addon replica set Heapster and InfluxDB Networking daemon set Identity and authz services
21. How do we install it all? Manually place configuration Cloud-config and bash Config management
22. How do we install it all? Manually place configuration Cloud-config and bash Config management
23. How do we upgrade it all?
24. $ monokube --nodes=172.17.8.101,172.17.8.102,... monokube - a prototype ssh reverse tunnel
25. $ monokube --nodes=172.17.8.101,172.17.8.102,... monokube - a prototype deploy API server
26. $ monokube --nodes=172.17.8.101,172.17.8.102,... monokube - a prototype re-configure API cfg
27. Self-hosting Kubernetes Pivot
28. kubectl Self-hosting Kubernetes Pivot
29. That seems hard, what do we get? Bootstrap requirements down to working SSH Rolling updates for Kubernetes itself! Kubelet version controlled by API Help Wanted! Goal: working in v1.3
30. Simpler Deployment join sig-high-availability
31. Increasing Scale scheduler improvements
32. https://coreos.com/blog/improving-kubernetes-scheduler-performance.html 10x Improvement in scheduler throughput Ongoing work to track upstream performance Let's make similarly large gains in v1.3 Help wanted: Kubemark dashboard!
33. Increasing Scale etcd v3 in k8s
34. etcd v3.0 - "Scaling etcd to thousands of nodes" • Efficient transport via gRPC and HTTP/2 • New powerful API based on k8s use-case • Disk-backed and memory efficient storage • Incremental snapshot for consistent performance • Fix re-list issues with longer and memory-efficient key history
35. v3 API - Transactions • compare and swap ◦ compare: foo=bar ◦ success: foo=bar2 • multiple object transaction ◦ compare: cond1=true && cond2=true ◦ success: pass=true ◦ failure: pass=false
36. v3 API - Watches • support multiple keys and prefixes per stream ◦ watchKey(foo) ◦ watchPrefix(coreos) • support watch from historical point ◦ watchKey(foo, index_of_an_hour_ago) ◦ user-driven history compaction
37. v3 API - Lease l := lease.Create(10*second) kv.Put("foo", "bar", l.ID) // key will be removed without keeping // alive the lease go KeepAlive(l.id)
38. Help Wanted: mirror maker Label queries are the new DNS Need API mirrors to give queries 100% uptime Help wanted, no work started.
39. When is the release?
40. When is it in k8s? • etcd v3 k8s issue #22448 ◦ Refactoring the storage interface ◦ Proof of concept working
41. Increasing Scale join sig-scalability
42. Security Through Identity OIDC in Kubernetes
43. Dex - OIDC Provider Open source standards based identity-provider SQL, LDAP, and other identity backend connectors Applicable outside of Kubernetes but that is our use case
44. OIDC Relying Party
45. OIDC End User
46. OIDC Identity Provider
47. OIDC Identity Provider
48. OIDC Identity Provider
49. OIDC Identity Provider
50. OIDC 0. Relying party periodically syncs public key from IdP
51. 1. User request protected page OIDC
52. 2. User redirected to auth page OIDC
53. 3. User authenticates (cookie/pw) OIDC
54. 4. User given authz grant OIDC
55. 5. User presents grant to client OIDC
56. 6. Relying party exchanges authz code for ID token OIDC
57. 7. Client gets ID token and validate claims OIDC
58. JWT JSON Web Token
59. eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9. eyJzdWIiOiIyNDgyODk3NjEwMDEiLCJuYW1lI joiSmFuZSBEb2UiL... mphbmVkb2VAZXhhbXBsZS5jb20iLCJwaWN 0dXJlIjoiaHR0cDovL2V4YW1wbGUuY29tL2ph bmVkb2UvbWUuanBnIn0. TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeF ONFh7HgQ
60. { "alg": "HS256", "typ": "JWT" }
61. { "sub": "248289761001", "name": "Ada Richmond", "preferred_username": "ada", "email": "ada.richmond@example.com", "groups": ["read-prod", "admin-stage"] }
62. Groups and Kubernetes API server extracts user, email, groups, from OIDC token Now what?
63. Webhook Authorizer "kind": "SubjectAccessReview", "spec": { "resourceAttributes": { "namespace": "default", "verb": "GET", "group": "group3", "resource": "pods" }, "user": "ada", "group": ["read-prod", "admin-stage" ] } authorizer service OK?
64. Security Through Identity OIDC in Kubernetes
65. rkt Powered Kubernetes mid-flight engine swap
66. a modern, secure container runtime a simple, composable tool focused on kubernetes
67. no central daemon no (mandatory) API apps run directly under spawning process rkt - simple CLI tool
68. bash/systemd/kubelet rkt run ... application(s)
69. modular architecture take advantage of different technologies provide a consistent experience to users rkt internals
70. Nearly complete! 80% of end-to-end tests passing cAdvisor integration in progress rktnetes today
71. LIVE DEMO rktnetes today
72. Goal: 100% end-to-end tests working User may switch to rktnetes with zero suprises rktnetes today
73. rkt Powered Kubernetes join sig-node
74. Security TPM Log
75. • TPM, Trusted Platform Module ◦ physical chip on the motherboard ◦ cryptographic keys + processor • Used to "measure" system state • Historically just use to verify bootloader/OS (on proprietary systems) rkt TPM measurement
76. • CoreOS added support to GNU Grub • rkt can now record information about running pods in the TPM • attestable record of what images and pods are running on a system rkt TPM measurement
77. rkt TPM measurement
78. https://coreos.com/blog/coreos-trusted-computing.html Tectonic Trusted Computing
79. TPM Attestation in k8s 1. Generated timestamp 2. Ask TPM for sig of time + log value 3. Submit to API server in nodeStatus
80. TPM Attestation in k8s Goal: Merge nodeStatus payload upstream in k8s v1. 3
81. rkt TPM measurement For more TPM and rkt, see Matthew Garrett's talk: "Integrated trusted computing in Kubernetes" 11: 30am today
82. Security TLS Bootstrap
83. TLS Bootstrap of Nodes (#20439) 1. Generate CSR 2. Submit CSR to API server 3. Poll for approved CSR
84. TLS Bootstrap of Nodes (#20439) Goal: Merge proposal and working code into v1.3
85. Industry Movement
87. • Coordinate promotion of Cloud Native architectures • A home for Cloud Native OSS projects like Kubernetes ◦ Technical board to evaluate additional projects • Provides shared resources to projects like video conferencing, test servers, etc
88. • Creating technical standards for containers • Started with runC and a runtime specification • Large mandate to standardize an image format ◦ In-progress
89. Multiple Image Formats in v1.3 API • Today Kubernetes only supports the Docker Image Format and naming • Use cases for executing other formats ◦ OCI Image Format ◦ tar archive chroots ◦ jar? ◦ static binary? • Support signing and content verification
90. Help Push Kubernetes Forward Simpler to deploy and configure clusters Increasing scale of clusters throughout stack Security based on good practices rkt engine powering Kubernetes nodes Standards to ensure portability
92. coreos.com/fest - @coreosfest May 9 & 10, 2016 - Berlin, Germany
93. Thank you! Brandon Philips @brandonphilips | brandon@coreos.com | coreos.com We’re hiring in all departments! Email: careers@coreos.com Positions: coreos.com/ careers