Scale your Docker containers with Mesos

Gitpro 2015

Transcript

1. Docker containers at scale with Apache Mesos Distributed Systems Engineer Apache Mesos PMC Timothy Chen @tnachen tim@mesosphere.io
2.  Mesoswhat?  Marathon  Chronos  Demo!
3. Status quo is static partitioning and use of virtual machines
4. Add some virtual machines
5. Provision Hadoop
6. Provision a web service
7. Moar data, moar Hadoop
8. Mesos let us treat a cluster of nodes...
9. As one big computer
10.  Not as individual machines  Not as VMs
11. But as computational resources like cores, memory, disks, etc.
12. 240 million monthly active users 500 million tweets per day Up to 150k tweets per second More than 100TB per day of compressed data
13. “Mesos is the cornerstone of our elastic compute infrastructure — it's how we build all our new services and is critical for Twitter's continued success at scale. It's one of the primary keys to our data center efficiency.” — Chris Fry, SVP of Engineering at Twitter
18. Containerization in Mesos, a brief history
19. Mesos is...  A top-level Apache project  A cluster resource negotiator  Scalable to 10,000s of nodes  Fault-tolerant, battle-tested  An SDK for distributed apps
21. Mesos for all the things
22.  Marathon  Mesoswhat?  Chronos  Demo!
23. a self-serve interface to your cluster
24. distributed "init" for long-running services
25. a private fault-tolerant PaaS
26. Marathon does it! Start, stop, scale, update apps Nice web interface, API Highly available, no SPoF Native Docker support Fully featured REST API Pluggable event bus Rolling deploy / restart Application health checks Artifact staging
27.  Chronos  Mesoswhat?  Marathon  Demo!
28. Introducing Chronos a scheduler for batch and one-off jobs
29. Distribute a graph of jobs Dependency graph for execution
30. Features Distributed job scheduler Web interface, API Highly available, no SPoF Native Docker support Easy scheduling with repeating intervals
31. Chronos  REST P U T c h r o n o s - n o d e : 8 0 8 0 / s c h e d u l e r / j o b / j o b 1 G E T c h r o n o s - n o d e : 8 0 8 0 / s c h e d u l e r / j o b s D E L E T E c h r o n o s - n o d e : 8 0 8 0 / s c h e d u l e r / t a s k / k i l l / j o b 2
32.  Demo!  Mesoswhat?  Marathon  Chronos
33. Thanks! Come and talk to me P.S., we're hiring!