Slide 1

Slide 1 text

The Comeback of Grids aka. Compute Clusters

Slide 2

Slide 2 text

Evgeny Zislis Co-Founder / CTO www.devops.co.il www.devopspro.co.uk

Slide 3

Slide 3 text

Agenda ● Buzzing trends in Soft. Architecture ● Containers ● Schedulers ● Containers inside schedulers

Slide 4

Slide 4 text

architecture of SaaS applications

Slide 5

Slide 5 text

aws.amazon.com/architecture Software HA

Slide 6

Slide 6 text

_...still Software HA_

Slide 7

Slide 7 text

monolithic vs. micro yobriefca.se/blog/2013/04/29/micro-service-architecture/ Circa 2000 Circa 2014

Slide 8

Slide 8 text

What is MicroServices Architecture? Loosely coupled service oriented architecture with bounded contexts not updated at the same time minimal knowledge of own environment

Slide 9

Slide 9 text

monitoring configuration router layout data access queue cache scheduler micro services infrastructure services authentication authorization form handling file uploads web scraping indexing searching application services e-mail pdf generation payments workflow engine pagination load balancer

Slide 10

Slide 10 text

Death-star graph of microservices at Netflix

Slide 11

Slide 11 text

“If you can’t feed a team with two pizzas, it’s too large. That limits a task force to five to seven people, depending on their appetites” - Jeff Bezos medium.com/@benorama/ the-evolution-of-software-architecture-bd6ea674c477

Slide 12

Slide 12 text

Gartner Hype Cycle

Slide 13

Slide 13 text

last three years... Docker Containers

Slide 14

Slide 14 text

Docker API host containers

Slide 15

Slide 15 text

time to provision in seconds 10000 100 1 Bare Metal VM Container

Slide 16

Slide 16 text

kubernetes Orchestration

Slide 17

Slide 17 text

kubernetes mesos / marathon or Orchestration

Slide 18

Slide 18 text

kubernetes coreos / fleet geard mesos or or or Orchestration

Slide 19

Slide 19 text

Discovery kubernetes coreos / fleet geard mesos or or or consul helios etcd centurion

Slide 20

Slide 20 text

Infrastructure kubernetes coreos / fleet geard mesos or or or consul helios etcd centurion EC2 Rack space GCE DO Tutum Heroku

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

Mesos is a Distributed OS kernel ● ● ● ● ● mesosphere.com/blog/2015/04/23/apple-details-j-a-r-v-i-s-the-mesos-framework-that-runs-siri/ www.wired.com/2013/03/google-borg-twitter-mesos

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

Mesos - abstract resource manager virtual machines docker App A App A MESOS host | host | host | host | host | host

Slide 25

Slide 25 text

blog.twitter.com/2012/incubating-apache-mesos nerds.airbnb.com/introducing-chronos/ dev.hubspot.com/blog/mesos-hubspot-and-the-singularity ebaytechblog.com/2014/05/12/delivering-ebays-ci-solution-with-apache-mesos-part-ii/

Slide 26

Slide 26 text

mesosphere.com/blog/2015/11/10/mesos-nvidia-gpus/

Slide 27

Slide 27 text

mesosphere.com/blog/2015/08/20/mesos-everywhere-apache-mesos-for-windows-server/ Mesos on Windows Server

Slide 28

Slide 28 text

but, but ... What about Containers?

Slide 29

Slide 29 text

github.com/mesosphere/marathon ● Highly available schedulers ● Constraints for application instances ● Service discovery - DNS & Load balancing ● Health checks ● Event subscription endpoint ● Graphical UI ● JSON / REST APIs for scriptability ● Metrics Mesosphere Marathon

Slide 30

Slide 30 text

● Docker compatible ● Grid/cluster management ● AWS APIs ● Custom & default scheduler ● Docker Compose support for local dev ● Monitoring / Logging ● Custom docker repositories + AWS Repo Elastic Container Service

Slide 31

Slide 31 text

● Self healing ● Service discovery & load balancing ● Secrets & config management ● Automated rollouts & rollbacks ● Storage orchestration ● Automatic binpacking

Slide 32

Slide 32 text

Docker Swarm ● Docker Engine API ● Integrated Networking & Volumes ● High Scalability - 1k hosts 50k containers ● High Availability with failovers ● Pluggable scheduler ● Pluggable host discovery

Slide 33

Slide 33 text

● Task Groups == K8s Pods == Docker Compose ● Service discovery - Consul ● Health checks - HTTP ● Containers - or - ○ Docker / Rocket / runc … ○ Single binary apps: Java / Go / … ● or! Virtual Machines (like OpenStack Nova) ● “Run Everywhere”, “Batch”, “Service”, “Cron” ● Constraints ● Multi Datacenter + Continental ● Easy installation using GO & single binary ● Lack of Zookeeper - uses Consul instead ● Monitoring ● Pluggable service discovery: Consul / ZK / etcd / … ● Self healing ● Automatic binpacking ● Orhestration of tasks on Windows

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

Thank you! We invite you to join Operations UK Facebook group on dvps.me/OpsUK www.devopspro.co.uk