Journey to GIFEE - Philly ETE 2016

Transcript

1. Brandon Philips @brandonphilips | brandon.philips@coreos.com The Journey to #GIFEE Intro to the Open Source
2. Secure the Internet MISSION
3. Separate Apps from OS STRATEGY
4. Make Servers Consistent STRATEGY
5. Tolerate Machine Failures STRATEGY
6. Make Servers Easy to Upgrade STRATEGY
7. Simplify Application Upgrades STRATEGY
14. 3 Application packaging Clustering Linux at scale
15. #GIFEE Borg/Omega Linux Chubby
16. #GIFEE Borg/Omega Linux Chubby
17. #GIFEE Borg/Omega Linux Chubby
18. Avoid single points of failure Design for constant updates Consistent environment Why build #GIFEE?
19. Why build #GIFEE? Design for constant updates
20. Application Packaging 1
21. Abstract away app from the OS OS App
24. Base software managed by CoreOS kernel systemd OpenSSH
25. Protect apps from each other Isolated network namespace Isolated file system namespace Mixed versions of dependencies eg. python 3.4 & python 2.7
26. $ sudo rkt run coreos.com/etcd:v2.0.0 $ sudo rkt run coreos.com/etcd:v2.0.0 \ --cpu=750m --memory=128M $ sudo rkt run --net=host coreos.com/etcd:v2.0.0 rkt run
27. Search container metadata Identify vulnerabilities Explain update actions Clair container security auditing
28. After scanning millions of containers we found that over 80% still had Heartbleed 80% Clair container security auditing
30. In-Progress Universal Container Format Packaged Downloaded Verified
31. Linux at Scale 2
32. Patches to the OS and kernel are hard Retest after updates No automation SECURITY Dependency breakage Uptime risk APPLICATION
34. Auto-updating browsers fixed security We got HTML5 at the same time
35. Atomic operating system updates
36. Atomic operating system updates
37. Clustering 3
38. Patches to the OS and kernel are hard Retest after updates No automation SECURITY Dependency breakage Uptime risk APPLICATION
39. Patches to the OS and kernel are hard No automation SECURITY Uptime risk APPLICATION
40. Operations Paradise Easy scale out Painless app upgrades Tolerant of machine failure
41. App Req/sec: 6,000 App Healthy: True
42. App Req/sec: 6,000 App Healthy: True
43. App Req/sec: 7,000 App Healthy: True
44. App Req/sec: 8,000 App Healthy: True
45. App Req/sec: 7,000 App Healthy: True
46. App Req/sec: 6,000 App Healthy: True
47. App Req/sec: 8,000 App Healthy: True
48. App Req/sec: 7,000 App Healthy: True
49. App Req/sec: 8,000 App Healthy: True
50. App Req/sec: 8,000 App Healthy: True
51. App Req/sec: 8,000 App Healthy: True
52. When do you need cluster coordination? Leader election Cluster-wide Semaphores Service discovery Dynamic configuration
53. Hard Computer Science Problem ?
54. Hard Computer Science Problem Chubby
55. A highly-available key value store for shared configuration and service discovery
63. No existing “cloud native” solutions Simple HTTP + JSON APIs Dynamic reconfiguration Why build etcd?
64. Simple key/value “Distributed etc” Keys are versioned Changes can be watch
65. Cluster-wide reboot lock - locksmith Service discovery - vulcand, skydns Cluster orchestration - k8s, cloud foundry
66. Industry Adoption 500+ projects on Github
67. When do you need cluster coordination? Leader election Cluster-wide Semaphores Service discovery Dynamic configuration
68. Hard Computer Science Problem ?
69. Hard Computer Science Problem Chubby
70. A highly-available key value store for shared configuration and service discovery
71. Kubernetes is our choice for orchestration platform
72. App Req/sec: 8,000 App Healthy: True
73. App Req/sec: 8,000 App Healthy: True
74. App Req/sec: 8,000 App Healthy: True
75. App Req/sec: 8,000 App Healthy: True
76. Guides & Tools coreos.com/kubernetes kube-aws Cloud-configs
77. Scheduler gets work to the servers
78. $ scp app host:/opt $ ssh host systemd-run /opt/app
79. $ scp app host:/opt $ ssh host systemd-run /opt/app
80. $ fab deploy:app
81. $ fab deploy:app
82. $ fab deploy:app
83. $ fab deploy:gpu-app
84. $ fab deploy:gpu-app
85. $ fab deploy:gpu-app
86. Replication controllers drive the cluster to a desired state
87. pod env=prod app=web pod env=prod app=web pod env=prod app=web rc web-prod select(env=prod,app=web) count=1
88. pod env=prod app=web pod env=prod app=web pod env=prod app=web rc web-prod select(env=prod,app=web) count=1
89. pod env=prod app=web rc web-prod select(env=prod,app=web) count=1
90. pod env=prod app=web rc web-prod select(env=prod,app=web) count=5
91. pod env=prod app=web pod env=prod app=web pod env=prod app=web pod env=prod app=web pod env=prod app=web rc web-prod select(env=prod,app=web) count=5
92. Services find scheduled apps (pods) and load balance them
93. pod env=dev app=web pod env=test app=web pod env=prod app=web
94. pod env=dev app=web pod env=test app=web pod env=prod app=web service test.example.com select(env=dev,app=web) service beta.example.com select(env=test,app=web) OR select(env=prod,app=web) service example.com select(env=prod,app=web)
95. pod env=test app=web pod env=prod app=web pod env=prod app=web pod env=dev app=web pod env=test app=web pod env=prod app=web service test.example.com select(env=dev,app=web) service beta.example.com select(env=test,app=web) OR select(env=prod,app=web) service example.com select(env=prod,app=web)
96. pod app=foo,version=1 service foo.cluster.local select(app=foo)
97. pod app=foo,version=1 pod app=foo,version=2 service foo.cluster.local select(app=foo)
98. pod app=foo,version=1 pod app=foo,version=2 service foo.cluster.local select(app=foo)
99. Upstream rktnetes Auth/OIDC Node self-signed TLS
100. Scaling 15x scheduler performance 30k pods on 1k nodes SIG-scale
101. 3 Application packaging Clustering Linux at scale
102. Sounds good, but... Is anyone successful with all this in prod?
103. Publically traded options exchange
104. Containers on CoreOS are powering ISE's high- throughput, low-latency financial exchange Running in production Bare metal & AWS Billions of transactions a day 150 million req/sec
106. Secure the Internet MISSION
107. Separate Apps from OS STRATEGY
108. Make Servers Consistent STRATEGY
109. Tolerate Machine Failures STRATEGY
110. Make Servers Easy to Upgrade STRATEGY
111. Simplify Application Upgrades STRATEGY
113. coreos.com/fest - @coreosfest May 9 & 10, 2016 - Berlin, Germany
115. Build, Store and Distribute your Containers quay.io
116. Thank you! Brandon Philips @brandonphilips | brandon@coreos.com | coreos.com We’re hiring in all departments! Email: careers@coreos.com Positions: coreos.com/ careers