Can I have Mesos and Kubernetes?

Talk at London Mesos User Group, see also http://www.meetup.com/London-Mesos-User-Group/events/224555822/

The demo is available as well, here: https://gist.github.com/mhausenblas/07c03d4230a5e6ec3b46

Transcript

1. Can I have Mesos *and* Kubernetes? Michael Hausenblas | London Mesos User Group | 2015-09-24
2. © 2015 Mesosphere, Inc. 2
3. © 2015 Mesosphere, Inc. What is Kubernetes? • open source container orchestration system • scheduling apps onto nodes + scaling • manages apps to match declared intentions • based on Google's internal cluster managers such as Borg 3
4. © 2015 Mesosphere, Inc. Kubernetes architecture 4
5. © 2015 Mesosphere, Inc. Kubernetes concepts • container—sealed application package (Docker) • pod—small group of tightly coupled containers • labels—identifying metadata attached to objects (pods, services, etc.) • selector—query against labels, producing a set result • controller—control loop driving current state towards desired state • service—set of pods that work together (incl. proxy/load balancer) 5
6. © 2015 Mesosphere, Inc. Kubernetes: pods • unit of scheduling • data sharing between its containers • shared mount point (volumes) • shared network namespace (same IP, see each other on localhost) 6
7. © 2015 Mesosphere, Inc. Kubernetes: labels & selectors • arbitrary metadata (in key-value format) attached to any API object • used for grouping, queryable by selectors • examples • pods managed by a replication controller • nodes capabilities for scheduling 7
8. © 2015 Mesosphere, Inc. Kubernetes: replication controllers • constitutes a control loop • ensures a desired state (x pods running) • scoped by selector 8
9. © 2015 Mesosphere, Inc. Kubernetes: services • compensate for dynamics of pods • provides a stable VIP (through proxy) • access policy: load-balanced or headless 9
10. © 2015 Mesosphere, Inc. Kubernetes: cluster-wide services • monitoring (e.g.: cAdvisor + InfluxDB + Heapster) • logging (e.g.: fluentd + ES + Kibana) • DNS (e.g.: skydns + kube2sky) • all of the above can be replaced by custom solutions (GCP, AWS, etc.) 10
11. © 2015 Mesosphere, Inc. Kubernetes: example pods apiVersion: v1 kind: Pod metadata: name: mysql labels: name: mysql spec: containers: - resources: limits : cpu: 0.5 image: mysql name: mysql env: - name: MYSQL_ROOT_PASSWORD value: yourpassword ports: - containerPort: 3306 name: mysql volumeMounts: - name: mysql-persistent-storage mountPath: /var/lib/mysql volumes: - name: mysql-persistent-storage gcePersistentDisk: pdName: mysql-disk fsType: ext4 11 apiVersion: v1 kind: Pod metadata: name: wordpress labels: name: wordpress spec: containers: - image: wordpress name: wordpress env: - name: WORDPRESS_DB_PASSWORD value: yourpassword ports: - containerPort: 80 name: wordpress volumeMounts: - name: wordpress-persistent-storage mountPath: /var/www/html volumes: - name: wordpress-persistent-storage gcePersistentDisk: pdName: wordpress-disk fsType: ext4 MySQL Wordpress
12. © 2015 Mesosphere, Inc. Kubernetes: example services apiVersion: v1 kind: Service metadata: labels: name: mysql name: mysql spec: ports: - port: 3306 selector: name: mysql 12 apiVersion: v1 kind: Service metadata: labels: name: wpfrontend name: wpfrontend spec: ports: - port: 80 selector: name: wordpress type: LoadBalancer MySQL Wordpress
13. © 2015 Mesosphere, Inc. 13 Kubernetes + Mesos
14. © 2015 Mesosphere, Inc. Let's talk about workloads* … 14 *) kudos to Timothy St. Clair (@timothysc) for this batch streaming PaaS MapReduce
15. © 2015 Mesosphere, Inc. Kubernetes + Mesos 15 https://github.com/kubernetes/kubernetes/blob/release-1.0/contrib/mesos/docs/architecture.md
16. © 2015 Mesosphere, Inc. Kubernetes + Mesos 16 https://github.com/kubernetes/kubernetes/blob/release-1.0/contrib/mesos/docs/architecture.md
17. © 2015 Mesosphere, Inc. Benefits • Node-level auto-scaling • Dynamic partitioning: co-location of Kubernetes with other services such as Spark, Chronos, Cassandra, etc. on the same cluster • Independence from special network infrastructure • Mesosphere DCOS specific: • High Availability • Easy installation • Easy maintenance 17
18. © 2015 Mesosphere, Inc. Use Cases • Kubernetes: 100% containerized workloads (cloud-native apps) • Mesos/Marathon: containerized & non-containerized workloads • Kubernetes+Mesos: hybrid workloads • legacy apps/microservices integration • state-less + analytical workloads 18
19. © 2015 Mesosphere, Inc. 19 $ demo
20. 20 © 2015 Mesosphere, Inc.
21. 21 © 2015 Mesosphere, Inc.
22. © 2015 Mesosphere, Inc. DCOS CE Architecture 22 https://docs.mesosphere.com/getting-started/dcosarchitecture/
23. © 2015 Mesosphere, Inc. Demo: hybrid workload 23 gen.sh distributed kernel orchestration application
24. © 2015 Mesosphere, Inc. 24 See also …
25. © 2015 Mesosphere, Inc. Scheduling at Google • Borg: http://research.google.com/pubs/pub43438.html • The Google Stack: http://j.mp/google-stack 25
26. © 2015 Mesosphere, Inc. Kubernetes • http://kubernetes.io/v1.0/docs/ • https://docs.mesosphere.com/services/kubernetes/ • http://kubernetes.io/v1.0/docs/design/architecture.html 26
27. © 2015 Mesosphere, Inc. Kubernetes + Mesos • https://youtu.be/yEwCehwPV98 — Crossing the Streams: Leveraging Kubernetes Services From Other Mesos Frameworks and Vice Versa by Timothy St. Clair, MesosCon Seattle 2015 • https://github.com/mesosphere/kubernetes-mesos 27
28. © 2015 Mesosphere, Inc. Kubernetes + Mesos • https://github.com/kubernetes/kubernetes/blob/release-1.0/contrib/mesos/ docs/architecture.md • http://kubernetes.io/v1.0/docs/getting-started-guides/mesos.html • https://github.com/jeffmendoza/kubernetes/blob/gh-pages/solutions/ mesosphere.md 28
29. © 2015 Mesosphere, Inc. We're hiring … 29 San Francisco New York Hamburg https://mesosphere.com/careers/
30. © 2015 Mesosphere, Inc. Q & A • @mhausenblas • mhausenblas.info • @mesosphere • mesosphere.io/product 30