Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Apache Mesos NYC Meetup

benh
August 20, 2013

Apache Mesos NYC Meetup

benh

August 20, 2013
Tweet

More Decks by benh

Other Decks in Technology

Transcript

  1. Benjamin  Hindman    –  @benh  
    Apache  Mesos  
    mesos.apache.org  
    @ApacheMesos  

    View full-size slide

  2. download  and  install  
    http://www.apache.org/dyn/closer.cgi/mesos/0.12.1  
    $  tar  zxf  mesos-­‐0.12.1.tar.gz  
    $  cd  mesos-­‐0.12.1  
    $  ./configure  -­‐-­‐prefix=/path/to/install/directory  
    $  make  install  

    View full-size slide

  3. releases  
    maintained   stable   development  
    0.12.1   0.13.0  
    (0.13.0-­‐rc7)  
    0.14.0  
    (0.14.0-­‐rc1)  

    View full-size slide

  4. development  release  
    $  git  clone  https://git.apache.org/mesos.git  
    $  cd  mesos  
    $  ./bootstrap  
    $  ./configure  -­‐-­‐prefix=/path/to/install/directory  
    $  make  install  

    View full-size slide

  5. packages  
    https://s3.amazonaws.com/mesos-­‐pkg/ubuntu/12.10/mesos_0.14.0_amd64.deb  
    https://s3.amazonaws.com/mesos-­‐pkg/ubuntu/12.04/mesos_0.14.0_amd64.deb  
    https://s3.amazonaws.com/mesos-­‐pkg/debian/7.0/mesos_0.14.0_amd64.deb  
    Contact  [email protected]  for  more  information  or  other  packages  

    View full-size slide

  6. packages  
    https://github.com/deric/mesos-­‐deb-­‐packaging  
    packaging  support  in  0.15.0  
    nightly/weekly  snapshots  of  development  

    View full-size slide

  7. starting  a  master  
    $  mesos-­‐master  -­‐-­‐help  
    $  mesos-­‐master  -­‐-­‐ip=a.b.c.d  
    $  MESOS_ip=a.b.c.d  mesos-­‐master  

    View full-size slide

  8. starting  a  (fault-­‐tolerant)  master  
    $  mesos-­‐master  -­‐-­‐zk=zk://ip1:port1,ip2:port2,…/mesos  

    View full-size slide

  9. mesos  
    Apache  
    ZooKeeper  

    View full-size slide

  10. mesos  
    Apache  
    ZooKeeper  

    View full-size slide

  11. starting  a  slave  
    $  mesos-­‐slave  –help  
    $  mesos-­‐slave  -­‐-­‐master=ip:port  
    $  mesos-­‐slave  -­‐-­‐master=zk://ip1:port1,ip2:port2,…/mesos  

    View full-size slide

  12. mesos  
    Apache  
    ZooKeeper  

    View full-size slide

  13. mesos  
    Apache  
    ZooKeeper  

    View full-size slide

  14. mesos  
    Apache  
    ZooKeeper  

    View full-size slide

  15. now  what?  

    View full-size slide

  16. launch  frameworks  

    View full-size slide

  17. what’s  a  framework?  

    View full-size slide

  18. framework  
    ≈  
    distributed  system  

    View full-size slide

  19. frameworks  
    •  Hadoop  (github.com/mesos/hadoop)  
    •  Spark  (github.com/mesos/spark)  
    •  DPark  (github.com/douban/dpark)  
    •  Storm  (github.com/nathanmarz/storm)  
    •  Chronos  (github.com/airbnb/chronos)  
    •  MPICH2  (not  well  maintained,  email  mailing  list)  

    View full-size slide

  20. framework  commonality  
    run  processes  simultaneously  (distributed)  
    handle  process  failures  (fault-­‐tolerance)  
    optimize  execution  (elasticity,  scheduling)  

    View full-size slide

  21. mesos  
    Apache  
    ZooKeeper  
    Apache  
    Hadoop  
    Chronos  

    View full-size slide

  22. mesos  
    Apache  
    ZooKeeper  
    Apache  
    Hadoop  
    Chronos  

    View full-size slide

  23. mesos  
    Apache  
    ZooKeeper  
    Apache  
    Hadoop  
    Chronos  

    View full-size slide

  24. mesos  
    Apache  
    ZooKeeper  
    Apache  
    Hadoop  
    Chronos  

    View full-size slide

  25. mesos  
    Apache  
    ZooKeeper  
    Apache  
    Hadoop  
    Chronos  

    View full-size slide

  26. but  why?  

    View full-size slide

  27. origins  
    Berkeley  research  project  including  Benjamin  
    Hindman,  Andy  Konwinski,  Matei  Zaharia,  Ali  
    Ghodsi,  Anthony  D.  Joseph,  Randy  Katz,  Scott  
    Shenker,  Ion  Stoica  
    mesos.apache.org/documentation  

    View full-size slide

  28. static  partitioning  
    Apache  
    Hadoop  
    Chronos  

    View full-size slide

  29. static  partitioning  
    considered  harmful    
    Apache  
    Hadoop  
    Chronos  

    View full-size slide

  30. static  partitioning  
    considered  harmful    
    Apache  
    Hadoop  
    Chronos  
    hard to utilize machines
    (e.g., 72 GB RAM and 24 CPUs)
    (1)  

    View full-size slide

  31. static  partitioning  
    considered  harmful    
    Apache  
    Hadoop  
    Chronos  
    hard to utilize machines
    (e.g., 72 GB RAM and 24 CPUs)
    (1)  

    View full-size slide

  32. static  partitioning  
    considered  harmful    
    Apache  
    Hadoop  
    Chronos  
    hard to scale elastically
    (to take advantage of statistical multiplexing)
    (2)  

    View full-size slide

  33. static  partitioning  
    considered  harmful    
    Apache  
    Hadoop  
    Chronos  
    hard to scale elastically
    (to take advantage of statistical multiplexing)
    (2)  

    View full-size slide

  34. static  partitioning  
    considered  harmful    
    Apache  
    Hadoop  
    Chronos  
    hard to scale elastically
    (to take advantage of statistical multiplexing)
    (2)  

    View full-size slide

  35. static  partitioning  
    considered  harmful    
    Apache  
    Hadoop  
    Chronos  
    hard to scale elastically
    (to take advantage of statistical multiplexing)
    (2)  

    View full-size slide

  36. static  partitioning  
    considered  harmful    
    Apache  
    Hadoop  
    Chronos  
    hard to deal with failures
    (3)  

    View full-size slide

  37. static  partitioning  
    considered  harmful    
    Apache  
    Hadoop  
    Chronos  
    hard to deal with failures
    (3)  

    View full-size slide

  38. static  partitioning  
    considered  harmful    
    Apache  
    Hadoop  
    Chronos  
    hard to deal with failures
    (3)  

    View full-size slide

  39. mesos  –  level  of  indirection  
    Apache  
    Hadoop  
    Chronos  

    View full-size slide

  40. mesos  –  level  of  indirection  
    Apache  
    Hadoop  
    Chronos  

    View full-size slide

  41. mesos  –  level  of  indirection  
    Apache  
    Hadoop  
    Chronos  

    View full-size slide

  42. a  “kernel”  for  the  datacenter  
    Apache  
    Hadoop  
    Chronos  

    View full-size slide

  43. primitives  
    scheduler  –  distributed  system  “master”  
    (executor  –  lower-­‐level  control  of  task  
    execution,  optional)  
    requests/offers  –  resource  allocations  
    tasks  –  “threads”  of  the  distributed  system  
    state  –  working  set  of  the  distributed  system  
    …  

    View full-size slide

  44. scheduler  
    Apache  
    Hadoop  
    Chronos  

    View full-size slide

  45. scheduler  
    (1)  brokers  for  resources  (with  master)  
    (2)  launches  tasks  
    (3)  handles  task  termination  

    View full-size slide

  46. brokering  for  resources  
    (1)  make  resource  requests  
                   2  CPUs  
                   1  GB  RAM  
                   slave  *  
    (2)  respond  to  resource  offers  
                   4  CPUs  
                   4  GB  RAM  
                   slave  foo.bar.com  

    View full-size slide

  47. offers:  non-­‐blocking  resource  allocation  
    exist  to  answer  the  question:  
    “what  should  mesos  do  if  it  can’t  satisfy  a  request?”  
    (1)  wait  until  it  can  
    (2)  offer  the  best  allocation  it  can  immediately  

    View full-size slide

  48. offers:  non-­‐blocking  resource  allocation  
    exist  to  answer  the  question:  
    “what  should  mesos  do  if  it  can’t  satisfy  a  request?”  
    (1)  wait  until  it  can  
    (2)  offer  the  best  allocation  it  can  immediately  

    View full-size slide

  49. “two-­‐level  scheduling”  
    mesos:  controls  resource  allocations  to  
    schedulers  
    schedulers:  make  decisions  about  what  to  run  
    given  allocated  resources  

    View full-size slide

  50. end-­‐to-­‐end  principle  
    “application-­‐specific  functions  ought  to  
    reside  in  the  end  hosts  of  a  network  
    rather  than  intermediary  nodes”  

    View full-size slide

  51. tasks  
    either  a  concrete  command  line  or  an  opaque  
    description  (which  requires  a  framework  
    executor  to  execute)  
    a  consumer  of  resources  

    View full-size slide

  52. task  operations  
    launching/killing  
    health  monitoring/reporting  (failure  detection)  
    resource  usage  monitoring  (statistics)  

    View full-size slide

  53. state  (and  replicated  log)  
    …  when  your  distributed  system  needs  state  (the  
    “working  set”,  often  10’s  to  100’s  of  MB),  what  do  you  
    do?  
    » a  database  is  overkill  (yet  another  system  to  manage)  
    » ZooKeeper  can  work  (but  you  probably  want  to  use  a  
    higher  level  abstraction,  and  if  you  have  more  than  
    1MB  you  could  be  out  of  luck,  and  …)  
    » can  build  your  own  distributed  state  machine  …  

    View full-size slide

  54. state  (and  replicated  log)  
    you  probably  don’t  want  Paxos  ,  you  want  Multi-­‐Paxos  
    and  Multi-­‐Paxos  is  just  a  replicated  log  (i.e.,  a  replicated  
    log  is  an  implementation  of  Multi-­‐Paxos  but  with  a  nicer  
    interface)  
    in  Mesos  since  0.9.0  (including  Java/JNI  bindings),  used  in  
    production  for  ~2  years  

    View full-size slide

  55. state  (and  replicated  log)  
    even  a  replicated  log  is  fairly  low-­‐level  (one  of  the  
    reasons  ZooKeeper  is  so  popular)  …  enter  “state”  
    State*  state  =  new  State(new  ReplicatedLogStorage());  
    Future>  fetch  =  state-­‐>fetch(“registry”);  
    Variable  variable  =  fetch.get();  
    Registry  registry  =  variable.get();  
    registry.makeSomeUpdates();  
    Variable  variable_  =  variable.mutate(registry);  
    Future>>  store  =  state-­‐>store(variable_);  

    View full-size slide

  56. Mesos  
    Mesos  
    Node   Node   Node   Node  
    Hadoop  
    Node   Node   Node   Node  
    Spark  
    Node   Node  
    MPI  
    Node  

    View full-size slide

  57. Mesos  
    Mesos  
    Node   Node   Node   Node  
    Hadoop  
    Node   Node   Node   Node  
    Spark  
    Node   Node  
    MPI   Storm  
    Node  

    View full-size slide

  58. Mesos  
    Mesos  
    Node   Node   Node   Node  
    Hadoop  
    Node   Node   Node   Node  
    Spark  
    Node   Node  
    MPI   Storm  
    Node  
    Chronos   …

    View full-size slide

  59. Mesos  
    Mesos  
    Node   Node   Node   Node  
    Hadoop  
    Node   Node   Node   Node  
    Spark  
    Node   Node  
    MPI   Storm  
    Node  
    Chronos  

    View full-size slide

  60. Questions  …  

    View full-size slide