Upgrade to Pro — share decks privately, control downloads, hide ads and more …

An introduction to Apache Mesos

An introduction to Apache Mesos

A short introduction to Apache Mesos, how does it help with cluster sharing and
utilisation. What are its aims and who is using it ?

Mike Frampton

March 03, 2014
Tweet

More Decks by Mike Frampton

Other Decks in Technology

Transcript

  1. Apache Mesos • What is it ? • Beyond Hadoop

    • Resource Sharing • Mesos Intentions • Architecture • Users www.semtech-solutions.co.nz [email protected]
  2. Apache Mesos – What is it ? • A cluster

    manager • Provides resource isolation / sharing • Run distributed systems on the same pool of nodes – Hadoop, Spark, Storm, Jenkins, K ... • Cluster monitoring via web UI • Tasks isolated via Linux Containers • Scalable – 10k's • Written in C++ • C++/Java/Python API's for new app dev www.semtech-solutions.co.nz [email protected]
  3. Apache Mesos – What is it ? • Its fault

    tolerant – Leader election via Zoo Keeper • Multi platform – Linux , OSX , Open Solaris • It is now a full Apache project • A Master / Slave based system www.semtech-solutions.co.nz [email protected]
  4. Apache Mesos – Beyond Hadoop What requirements are there beyond

    Hadoop ? • Cluster sharing / multi tenancy • Higher resource utilization • Faster response times • Resource sharing • A wider range of functionality • More out of the box functionality – Graphical Cluster monitoring – Graphical drag and drop cluster manipulation www.semtech-solutions.co.nz [email protected]
  5. Apache Mesos – Intentions • Avoid multiple specialised clusters •

    Avoid the need for “silo'ed” clusters • Avoid the lower utilization that this brings • Offer the ability for multi tenancy • Which offers – Lower hardware / maintenance costs – Higher utilisation – Scalability – Higher fault tolerance • Program for the Data Center www.semtech-solutions.co.nz [email protected]
  6. Apache Mesos – Users Companies using Mesos • Air bnb

    • Cloud Physics • HubSpot • Shopify • Twitter • Vimeo • Among many others www.semtech-solutions.co.nz [email protected]
  7. Contact Us • Feel free to contact us at –

    www.semtech-solutions.co.nz – [email protected] • We offer IT project consultancy • We are happy to hear about your problems • You can just pay for those hours that you need • To solve your problems