Build applications like Google

Using containers, Kubernetes and Mesos
GoSF 2014

Transcript

1. Build applications like Google: Using containers, Kubernetes and Mesos GoSF 2014, San Francisco Patrick Reilly, Mesosphere @preillyme
3. • [TBD] • More than a Sandbox, Less than a VM • Lightweight Linux environment • Hermetically sealed, deployable system • Introspectable, runnable artifact • Recently popularized by Docker What is a container?
4. Static application environment = reliable deployments No stress deployment and update care? Repeatable, runnable artifact = portability Develop here, run there Pick your cloud solely on its merits Loosely coupled = easier to build and manage Compose applications from micro- services Mix in and extend third party services Why should developers care?
5. 1. Keep programs simple 2. Prefer smaller, stateless programs
7. Images by Connie Zhou A 2000-machine cluster will have >10 machine crashes per day DRAM errors (1% AFR) Disk failures (2-10% AFR) Machine crashes (~2/year) OS upgrades (2-6/year)
8. Agenda • Hardware => GCE • Containers => Docker • Cluster management => Mesos • Container management => Kubernetes • Platform stacks for large distributed apps • How to get started
9. Google Compute Engine • VMs as service • Europe, Asia, US • Live migration
10. Problems with raw VMs • Expensive to turn up (time) • "Hard" to make repeatable • Difficult to replicate locally for testing • High possibility of failure
11. Container • Lightweight Linux execution environment ◦ libcontainer, LXC, LMCTFY • Static application composition ◦ Reliable deployment • Unit of resource isolation ◦ Multi-tenancy without heavyweight VMs
12. Docker • Open source • Configurable layers • Reproducible • Version-controlled • Lots and lots of other peoples containers ◦ https://registry.hub.docker.com/
13. Run everything in containers!
14. Mesos facts • Supports Docker • Scales to 10,000s of nodes • Top-level Apache project • Twitter, Airbnb are major users and contributors • APIs for C++, Python, JVM-languages, Go, ... • Pluggable CPU, memory, IO isolation • Highly configurable, with easy-to-use defaults • Packages and support through Mesosphere
15. How Mesos works Application Framework Mesos Slave Executor Task Task Task Executor Scheduler Mesos Master Mesos Master Mesos Master Zookeeper Mesos Master
16. Kubernetes • Employs and promotes Docker • Simple, portable, extensible modular framework for container-based service app management • Patterned after internal systems in Google that manage internet-scale workloads • Written in Go github.com/GoogleCloudPlatform/kubernetes
17. Kubernetes Client API Server K-M Scheduler Replication Controller Mesos Master Replicated Log Mesos Slave Mesos Slave Mesos Slave K-M Executor Kubelet Pod Proxy Container A Container B DockerD
18. Kubernetes glossary (part 1) • Master: the managing machine, which oversees one or more minions. • Minion: a slave that runs tasks as delegated by the user and Kubernetes master. • Pod: an application that runs on a minion.
19. Kubernetes glossary (part 2) • Replication controller: Verifies everything that should be running is running. • Label: an arbitrary key/value pair that the Replication Controller uses for service discovery • kubecfg: the config tool • Service: an amalgamation of similar pods
20. https://raw.githubusercontent. com/GoogleCloudPlatform/kubernetes/master/docs/architecture.png Kubernetes overview
21. Connection points • Kubernetes Pod → Mesos TaskGroup • Pod Labels → Mesos TaskGroup Labels, Task Labels • LabelSelectors ↝ Mesos Label Service • ReplicationController Marathon • Kubernetes Service ↝ Marathon service discovery
22. Kubernetes-Mesos framework overview
23. Kubernetes-Mesos master and scheduler
24. Kubernetes-Mesos executors and tasks
25. What Mesos contributes • Multi-framework: weighted fair sharing, roles, preemption*, node drain* • Run pods alongside other popular frameworks (e.g. Spark, Rails, Hadoop, ...) • Run services and batch apps in the same cluster • Advanced scheduling: resources, constraints, global view • High resource availability, cluster self-healing • Proven at scale, battle-tested in production • GUI/CLI cluster management console (* in development)
26. What Kubernetes contributes • Pods: tasks to co-locate/deploy/isolate/replicate • Label Service: arbitrary metadata for tasks/pods • Tightly integrated service discovery solution • Ability to run Kubernetes API workloads
27. + + + Hardware Fault tolerance Packaging Updates Isolation High availability Service discovery Service aggregation Scalability Elasticity Multi-tenancy Batch jobs Utilization
28. Google’s Hardware + Linux Omega Mesos Kubernetes Google-like computing for everybody Batch Service Batch Service GCE
29. Some possible stack variations Google’s Hardware + Linux GCE Mesos Marathon Service Batch Service Kubernetes
30. More Possible Stack Variations Other Hardware or Cloud + Linux or CoreOS Kubernetes Mesos Service Service Portability => no vendor lock-in Kubernetes Mesos Marathon Service Batch
31. google.mesosphere.com
40. Summary • Complete stack for large distributed apps • Multi-tenancy • Resource optimization • Easy to deploy • No vendor lock-in
41. Useful Links • GCE ◦ cloud.google.com/compute • Kubernetes ◦ github.com/GoogleCloudPlatform/kubernetes • Mesosphere ◦ mesosphere.com • Kubernetes - Mesos ◦ github.com/mesosphere/kubernetes-mesos • Docker ◦ docker.com
42. Photo Credits • https://secure.flickr.com/photos/icco/7990117774/ • https://secure.flickr.com/photos/icco/9956577145/ • http://www.google.co.uk/about/datacenters/gallery/#/all/20 • http://www.google.co.uk/about/datacenters/gallery/#/all/12 • https://secure.flickr.com/photos/icco/14349351930/