Tweet
Tweet

Slide 1

Slide 1 text

© 2015 Mesosphere, Inc. All Rights Reserved. FROM BIG DATA TO MICRO SERVICE:
 A JOURNEY WITH ADAM, AKKA, CASSANDRA, DCOS, SPARK AND SPARK NOTEBOOK 1 Devoxx, Antwerp | Michael Hausenblas | 2015-11-10

Slide 2

Slide 2 text

© 2015 Mesosphere, Inc. All Rights Reserved. WORKLOADS* … 2 *) kudos to Timothy St. Clair, @timothysc batch streaming PaaS MapReduce

Slide 3

Slide 3 text

© 2015 Mesosphere, Inc. All Rights Reserved. APACHE MESOS 3

Slide 4

Slide 4 text

© 2015 Mesosphere, Inc. All Rights Reserved. • A top-level ASF project • A cluster resource negotiator • Scalable to 10,000s of nodes but also useful for a handful of nodes • Fault-tolerant, battle-tested • An SDK for distributed apps • Native Docker support 4 Apache Mesos

Slide 5

Slide 5 text

© 2015 Mesosphere, Inc. All Rights Reserved. ARCHITECTURE 5 Mesos http://mesos.berkeley.edu/mesos_tech_report.pdf *) now: agent *

Slide 6

Slide 6 text

© 2015 Mesosphere, Inc. All Rights Reserved. RESOURCES 6 Mesos • resource == anything a task/executor consumes in order to do their work • standard resources: cpu, mem, disk, ports • DRF

Slide 7

Slide 7 text

© 2015 Mesosphere, Inc. 7

Slide 8

Slide 8 text

© 2015 Mesosphere, Inc. 8

Slide 9

Slide 9 text

© 2015 Mesosphere, Inc. 9

Slide 10

Slide 10 text

© 2015 Mesosphere, Inc. 10

Slide 11

Slide 11 text

© 2015 Mesosphere, Inc. 11

Slide 12

Slide 12 text

© 2015 Mesosphere, Inc. 12

Slide 13

Slide 13 text

© 2015 Mesosphere, Inc. 13

Slide 14

Slide 14 text

© 2015 Mesosphere, Inc. 14

Slide 15

Slide 15 text

© 2015 Mesosphere, Inc. 15

Slide 16

Slide 16 text

© 2015 Mesosphere, Inc. 16

Slide 17

Slide 17 text

© 2015 Mesosphere, Inc. 17

Slide 18

Slide 18 text

© 2015 Mesosphere, Inc. 18

Slide 19

Slide 19 text

© 2015 Mesosphere, Inc. 19

Slide 20

Slide 20 text

© 2015 Mesosphere, Inc. 20

Slide 21

Slide 21 text

© 2015 Mesosphere, Inc. 21

Slide 22

Slide 22 text

© 2015 Mesosphere, Inc. 22

Slide 23

Slide 23 text

© 2015 Mesosphere, Inc. All Rights Reserved. MEET THE DATACENTER OPERATING SYSTEM (DCOS) 23

Slide 24

Slide 24 text

© 2015 Mesosphere, Inc. All Rights Reserved. LOCAL OS VS. DISTRIBUTED OS 24 http://bitly.com/os-vs-dcos

Slide 25

Slide 25 text

© 2015 Mesosphere, Inc. All Rights Reserved. DCOS IS A DISTRIBUTED OPERATING SYSTEM 25 • kernel (Apache Mesos, written in C++) scales to 10,000 of nodes • fault-tolerant in all components, rolling upgrades throughout • containers first class citizens (LXC, Docker) • local OS per node (+container enabled) • scheduling (long-lived, batch) • service discovery, monitoring, logging, debugging

Slide 26

Slide 26 text

© 2015 Mesosphere, Inc. All Rights Reserved. DCOS HIGH LEVEL VIEW 26 Any Service or Container Any Infrastructure Mesosphere DCOS Your favorite services, container formats, and those yet to come Build apps once on DCOS, and run it anywhere Runs distributed apps anywhere 
 as simply as running apps on your laptop

Slide 27

Slide 27 text

© 2015 Mesosphere, Inc. All Rights Reserved. DCOS BENEFITS 27 • Run stateless services such as Web servers, app servers (via Marathon) and stateful services like Spark, Kafka, HDFS, Cassandra, ArangoDB etc. together on one cluster • Dynamic partitioning of your cluster, depending on your needs (business requirements) • Increased utilization (10% → 80% an more)

Slide 28

Slide 28 text

© 2015 Mesosphere, Inc. All Rights Reserved. DCOS ARCHITECTURE 28 https://docs.mesosphere.com/getting-started/dcosarchitecture/

Slide 29

Slide 29 text

© 2015 Mesosphere, Inc. All Rights Reserved. 29

Slide 30

Slide 30 text

© 2015 Mesosphere, Inc. All Rights Reserved. 30

Slide 31

Slide 31 text

© 2015 Mesosphere, Inc. All Rights Reserved. DEMO TIME! 31

Slide 32

Slide 32 text

© 2015 Mesosphere, Inc. All Rights Reserved. REFERENCES 32 Look up stuff: FAQ: help.mesosphere.com Docs: docs.mesosphere.com Chat with us and others from the community: Slack: chat.mesosphere.com Intercom (embedded in DCOS) IRC: #mesos on irc.freenode.net

Slide 33

Slide 33 text

© 2015 Mesosphere, Inc. All Rights Reserved. SEE ALSO … 33 • @mhausenblas • mhausenblas.info • @mesosphere • mesosphere.io/product http://shop.oreilly.com/product/9781939902184.do http://p24e.io http://shop.oreilly.com/product/0636920039952.do https://manning.com/books/mesos-in-action

Slide 34

Slide 34 text

© 2015 Mesosphere, Inc. All Rights Reserved. WE ARE HIRING, WORLDWIDE … San Francisco New York Hamburg https://mesosphere.com/careers/