The Comeback of Grids aka. Compute Clusters

Evgeny Zislis Co-Founder / CTO www.devops.co.il www.devopspro.co.uk

Agenda ● Buzzing trends in Soft. Architecture ● Containers ● Schedulers ● Containers inside schedulers

architecture of SaaS applications

aws.amazon.com/architecture Software HA

_...still Software HA_

monolithic vs. micro yobriefca.se/blog/2013/04/29/micro-service-architecture/ Circa 2000 Circa 2014

What is MicroServices Architecture? Loosely coupled service oriented architecture with bounded contexts not updated at the same time minimal knowledge of own environment

monitoring configuration router layout data access queue cache scheduler micro services infrastructure services authentication authorization form handling file uploads web scraping indexing searching application services e-mail pdf generation payments workflow engine pagination load balancer

Death-star graph of microservices at Netflix

“If you can’t feed a team with two pizzas, it’s too large. That limits a task force to five to seven people, depending on their appetites” - Jeff Bezos medium.com/@benorama/ the-evolution-of-software-architecture-bd6ea674c477

Gartner Hype Cycle

last three years... Docker Containers

Docker API host containers

time to provision in seconds 10000 100 1 Bare Metal VM Container

kubernetes Orchestration

kubernetes mesos / marathon or Orchestration

kubernetes coreos / fleet geard mesos or or or Orchestration

Discovery kubernetes coreos / fleet geard mesos or or or consul helios etcd centurion

Infrastructure kubernetes coreos / fleet geard mesos or or or consul helios etcd centurion EC2 Rack space GCE DO Tutum Heroku

No content

Mesos is a Distributed OS kernel ● ● ● ● ● mesosphere.com/blog/2015/04/23/apple-details-j-a-r-v-i-s-the-mesos-framework-that-runs-siri/ www.wired.com/2013/03/google-borg-twitter-mesos

No content

Mesos - abstract resource manager virtual machines docker App A App A MESOS host | host | host | host | host | host

blog.twitter.com/2012/incubating-apache-mesos nerds.airbnb.com/introducing-chronos/ dev.hubspot.com/blog/mesos-hubspot-and-the-singularity ebaytechblog.com/2014/05/12/delivering-ebays-ci-solution-with-apache-mesos-part-ii/

mesosphere.com/blog/2015/11/10/mesos-nvidia-gpus/

mesosphere.com/blog/2015/08/20/mesos-everywhere-apache-mesos-for-windows-server/ Mesos on Windows Server

but, but ... What about Containers?

github.com/mesosphere/marathon ● Highly available schedulers ● Constraints for application instances ● Service discovery - DNS & Load balancing ● Health checks ● Event subscription endpoint ● Graphical UI ● JSON / REST APIs for scriptability ● Metrics Mesosphere Marathon

● Docker compatible ● Grid/cluster management ● AWS APIs ● Custom & default scheduler ● Docker Compose support for local dev ● Monitoring / Logging ● Custom docker repositories + AWS Repo Elastic Container Service

● Self healing ● Service discovery & load balancing ● Secrets & config management ● Automated rollouts & rollbacks ● Storage orchestration ● Automatic binpacking

Docker Swarm ● Docker Engine API ● Integrated Networking & Volumes ● High Scalability - 1k hosts 50k containers ● High Availability with failovers ● Pluggable scheduler ● Pluggable host discovery

● Task Groups == K8s Pods == Docker Compose ● Service discovery - Consul ● Health checks - HTTP ● Containers - or - ○ Docker / Rocket / runc … ○ Single binary apps: Java / Go / … ● or! Virtual Machines (like OpenStack Nova) ● “Run Everywhere”, “Batch”, “Service”, “Cron” ● Constraints ● Multi Datacenter + Continental ● Easy installation using GO & single binary ● Lack of Zookeeper - uses Consul instead ● Monitoring ● Pluggable service discovery: Consul / ZK / etcd / … ● Self healing ● Automatic binpacking ● Orhestration of tasks on Windows

No content

No content

No content

Thank you! We invite you to join Operations UK Facebook group on dvps.me/OpsUK www.devopspro.co.uk