Scale your Docker containers with Mesos

Scale your Docker containers with Mesos
(Docker Palo Alto Meetup Oct 2014)

Transcript

1. Scale your containers with Mesos! Docker Palo Alto Meetup Oct 2014 Timothy Chen @tnachen tim@mesosphere.io
2. About me... Distributed Systems Engineer @ Mesosphere Open source: Apache Mesos PMC / Committer Apache Drill Committer Contribute to Apache Kafka, Spark and others
3. We are Mesosphere
4.  An introduction to the world of Mesosphere  Integrating Docker into Mesos  Demo!
5. What does your production environment look like?
6. Status quo is static partitioning and use of virtual machines
7. Add some virtual machines
8. Provision Hadoop
9. Provision a web service
10. Moar data, moar Hadoop
11. Mesos let us treat a cluster of nodes...
12. As one big computer
13.  Not as individual machines  Not as VMs
14. But as computational resources like cores, memory, disks, etc.
15. Mesos is...  Open Source Apache project  Cluster Resource Manager  Scalable to 10,000s of nodes  Fault-tolerant, battle-tested  An SDK for distributed apps
16. Mesos enables dynamic resource sharing http://mesosphere.io/simulator/ – + Datacenter – + Rails – + Hadoop – + Chronos – + Cassandra – + Spark
17. Mesos is another layer of abstraction Below a PaaS Above an IaaS
18. An abstraction that let us deal with a pool of resources Instead of computers or VMs
19. Making life easier for the data center operator  Multi-tenancy  Improved resource utilization  Resource Isolation  Fault-tolerance  Easy scaling
22. The Mesos ecosystem is growing
23. High Level View of Mesos Framework = scheduler + executor Schedulers get resource offers Executors run tasks
25. We love containers
26. Most modern applications are a web of containers
27. Now what?
28.  Integrating Docker into Mesos  An introduction to the world of Mesosphere  Demo!
29. Mesos has multiple ways of achiving resource isolation Mesos Containerizer (LXC -> Cgroups) External Containerizer Docker Containerizer!
30. Mesos Task integration Docker Container as a Task Docker Container as an Executor
31. Docker Containerizer features Resize Docker Containers Report Usage Info Download and mount files into Container Log forwarding
33.  Demo!  An introduction to the world of Mesosphere  Integrating Docker into Mesos
34. Say hi to Marathon
35. Marathon does it! Start, stop, scale, update apps Nice web interface, API Highly available, no SPoF Native Docker support Pluggable event bus Rolling deploy / restart Application health checks Artifact staging
36. Run a Docker container h t t p - v P O S T h t t p : / / 1 0 . 0 . 8 5 . 9 7 : 8 0 8 0 / v 2 / a p p s @ a p p - r u b y . j s o n { " i d " : " b r i d g e d - w e b a p p " , " c m d " : " p y t h o n 3 - m h t t p . s e r v e r 8 0 8 0 " , " c p u s " : 0 . 5 , " m e m " : 6 4 . 0 , " i n s t a n c e s " : 1 , " c o n t a i n e r " : { " t y p e " : " D O C K E R " , " d o c k e r " : { " i m a g e " : " p y t h o n : 3 " , " n e t w o r k " : " B R I D G E " , " p o r t M a p p i n g s " : [ { " c o n t a i n e r P o r t " : 8 0 8 0 , " h o s t P o r t " : 0 , " p r o t o c o l " : " t c p " } ] } } }
37. Scale! { " i n s t a n c e s " : 5 }
38. Add a UNIQUE hostname constraint { " c o n s t r a i n t s " : [ [ " h o s t n a m e " , " U N I Q U E " ] ] }
41. Mesos for all the containers
42. Mesos for all the things
43. Thanks! Come and talk to us Get Mesosphere packages: Kubernetes on Mesos: Read about Marathon: Try out Mesosphere on GCE: http://mesosphere.io/downloads https://github.com/mesosphere/kubernetes-mesos https://github.com/mesosphere/marathon http://google.mesosphere.io