Tweet
Tweet

Slide 1

Slide 1 text

Cloud-native application stacks From orchestrators to monitoring A lightly-opinionated survey

Slide 2

Slide 2 text

These are just, like, my opinions, man.

Slide 3

Slide 3 text

Amazon Web Services • Google Cloud Platform • Microsoft Azure
 Mesosphere • Apache Mesos • DC/OS • Marathon • Aurora Chronos • Kubernetes • Docker • Docker Swarm • Deis Rancher • Kismatic • Apprenda • Cloud Foundry • OpenShift OpenStack • CoreOS • Project Atomic • Snappy Ubuntu Joyent Triton • SmartOS • RancherOS • Vagrant • Terraform Consul • Serf • Packer • Nomad • Flocker • Ceph • Flannel
 Weave • Canal • Prometheus • InfluxDB • Nagios • Zabbix
 Elasticsearch • OpenZipkin • LightStep • Appdash

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

The Datacenter as a Computer An Introduction to the Design of Warehouse-Scale Machines http://www.cs.berkeley.edu/~rxin/db-papers/WarehouseScaleComputing.pdf

Slide 6

Slide 6 text

https://twitter.com/alicemazzy/status/655306196128280576

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

http://architecturebestinfo.blogspot.de/2015/09/computer-architecture.html

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

CPU Mem Disk Net

Slide 13

Slide 13 text

CPU Mem Disk Net Cooling Circuits Power App OS User Silicon

Slide 14

Slide 14 text

CPU Mem Disk Net Cooling Circuits Power App OS User Silicon

Slide 15

Slide 15 text

CPU Mem Disk Net App OS

Slide 16

Slide 16 text

App OS CPU Mem Disk Net

Slide 17

Slide 17 text

CPU Mem Disk Net Data Batch Biz logic Resource mgmt Abstraction Index

Slide 18

Slide 18 text

CPU Mem Disk Net Data Batch Biz logic Resource mgmt Abstraction Index

Slide 19

Slide 19 text

CPU Mem Disk Net Data Batch Biz logic Resource mgmt Compute resources Distributed storage Local disk Physical network Software defined network Abstraction Index

Slide 20

Slide 20 text

CPU Mem Disk Net Data Batch Biz logic Resource mgmt Compute resources Distributed storage Local disk Physical network Software defined network Scheduler Service discovery Config mgmt Abstraction Virtualization Index

Slide 21

Slide 21 text

CPU Mem Disk Net Data Batch Biz logic Resource mgmt Compute resources Distributed storage Local disk Physical network Software defined network Scheduler Service discovery Config mgmt Abstraction Virtualization Database Data warehouse Biz logic Index

Slide 22

Slide 22 text

Compute resources Distributed storage Local disk Physical network Software defined network Scheduler Service discovery Config mgmt Virtualization Database Data warehouse Biz logic

Slide 23

Slide 23 text

Compute resources Distributed storage Local disk Physical network Software defined network Virtualization Database Data warehouse Biz logic Scheduler Service discovery Config mgmt

Slide 24

Slide 24 text

Compute resources Distributed storage Local disk Physical network Software defined network Virtualization Database Data warehouse Biz logic Scheduler Config mgmt Service discovery

Slide 25

Slide 25 text

Database RDBMS AP CP Document Transactional Oracle MySQL Postgres Dynamo Voldemort Cassandra Riak Chubby ZooKeeper etcd Consul Mongo Couch Redis F1 Cockroach Rethink

Slide 26

Slide 26 text

Data warehouse Hadoop RedShift BigQuery Spark Mahout Hbase Pig ···

Slide 27

Slide 27 text

Lambda Architecture Realtime + Batch Storm Kafka Druid

Slide 28

Slide 28 text

Compute resources Distributed storage Local disk Physical network Software defined network Virtualization Database Data warehouse Biz logic Scheduler Config mgmt Service discovery

Slide 29

Slide 29 text

Compute resources Distributed storage Local disk Physical network Software defined network Virtualization Data Biz logic Scheduler Service discovery Config mgmt

Slide 30

Slide 30 text

Compute resources Distributed storage Local disk Physical network Software defined network Virtualization Data Biz logic Scheduler Service discovery Config mgmt

Slide 31

Slide 31 text

Biz logic Monolith

Slide 32

Slide 32 text

Biz logic Foo service Bar service Baz service Users service Auth service Search service Reports service Audit service

Slide 33

Slide 33 text

Biz logic Foo service Bar service Baz service Users service Auth service Search service Reports service Audit service

Slide 34

Slide 34 text

Biz logic Foo service Bar service Baz service Users service Auth service Search service Reports service Audit service Continuous delivery Continuous integration Monitoring Tracing Safety Application logging Event sourcing

Slide 35

Slide 35 text

Sean Treadway, SoundCloud

Slide 36

Slide 36 text

Sean Treadway, SoundCloud

Slide 37

Slide 37 text

Sean Treadway, SoundCloud

Slide 38

Slide 38 text

Sean Treadway, SoundCloud

Slide 39

Slide 39 text

Compute resources Distributed storage Local disk Physical network Software defined network Scheduler Service discovery Config mgmt Virtualization Data Biz logic

Slide 40

Slide 40 text

Monitoring Compute resources Distributed storage Local disk Physical network Software defined network Scheduler Service discovery Config mgmt Virtualization Data Biz logic

Slide 41

Slide 41 text

Virtualization Distributed storage Local disk Physical network Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Compute resources

Slide 42

Slide 42 text

Virtualization Distributed storage Local disk Physical network Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Compute resources

Slide 43

Slide 43 text

Physical network Virtualization Distributed storage Local (scratch Software defined
 network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Compute resources

Slide 44

Slide 44 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization

Slide 45

Slide 45 text

Application Platform Infrastructure Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization

Slide 46

Slide 46 text

http://trapptechnology.com/saas-iaas-paas-decoding-cloud-acronyms/

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

Software defined network Monitoring Distr. storage Scheduler Config mgmt Data Service discovery Compute resources Virtualization

Slide 49

Slide 49 text

¯\_(ϑ)_/¯ Software defined network Monitoring Distr. storage Scheduler Config mgmt Data Service discovery Compute resources Virtualization

Slide 50

Slide 50 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization Our new warehouse computer

Slide 51

Slide 51 text

Cloud platforms

Slide 52

Slide 52 text

Virtualization OS Middleware Runtime Data Applications Servers Storage Networking { }

Slide 53

Slide 53 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization AWS

Slide 54

Slide 54 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization EC2 S3 RedShift VPC Route53 CloudWatch CloudFormation RDS Machine Learning Kinesis Dynamo ELB ECS Lambda

Slide 55

Slide 55 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization GCP

Slide 56

Slide 56 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization GCE GKE GAE BigTable SQL Datastore BigQuery Cloud Monitoring Data* Deployment manager Cloud Storage Cloud Networking

Slide 57

Slide 57 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization Azure?

Slide 58

Slide 58 text

Newer

Slide 59

Slide 59 text

Open-source platforms

Slide 60

Slide 60 text

Virtualization OS Middleware Runtime Data Applications Servers Storage Networking { }

Slide 61

Slide 61 text

https://mesos.apache.org/ https://mesosphere.com/

Slide 62

Slide 62 text

https://thecustomizewindows.com/2014/12/mesosphere-data-center-operating-system/

Slide 63

Slide 63 text

https://aphyr.com/posts/326-call-me-maybe-chronos

Slide 64

Slide 64 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization Master Slave Slave Slave ZooKeeper Marathon Aurora Chronos DNS Master Master

Slide 65

Slide 65 text

http://kubernetes.io/

Slide 66

Slide 66 text

http://www.slideshare.net/brendandburns/defrag-2014-41815642

Slide 67

Slide 67 text

http://kubernetes.io/

Slide 68

Slide 68 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring API server Kubelet Kubelet Kubelet etcd SkyDNS Various Volume mounts Controllers

Slide 69

Slide 69 text

http://devopscube.com/how-to-setup-and-configure-docker-swarm-cluster/

Slide 70

Slide 70 text

http://blog.daocloud.io/swarm_analysis_part1/

Slide 71

Slide 71 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization Docker Docker Docker Swarm Scheduler Node discovery libnetwork libkv

Slide 72

Slide 72 text

No content

Slide 73

Slide 73 text

http://deis.io/ https://www.engineyard.com/

Slide 74

Slide 74 text

https://rancher.com

Slide 75

Slide 75 text

http://kismatic.com http://apprenda.com

Slide 76

Slide 76 text

https://pivotal.io/platform

Slide 77

Slide 77 text

https://www.openstack.org/

Slide 78

Slide 78 text

No content

Slide 79

Slide 79 text

https://www.openshift.com

Slide 80

Slide 80 text

More enterprisey?

Slide 81

Slide 81 text

Less opinionated

Slide 82

Slide 82 text

Cloud-native operating systems

Slide 83

Slide 83 text

} Virtualization OS Middleware Runtime Data Applications Servers Storage Networking

Slide 84

Slide 84 text

https://coreos.com/

Slide 85

Slide 85 text

https://en.wikipedia.org/wiki/CoreOS

Slide 86

Slide 86 text

http://projectatomic.io/

Slide 87

Slide 87 text

https://access.redhat.com/products/red-hat-atomic-enterprise-platform

Slide 88

Slide 88 text

https://developer.ubuntu.com/en/snappy/

Slide 89

Slide 89 text

https://rancher.com OS

Slide 90

Slide 90 text

https://smartos.org/ https://joyent.com

Slide 91

Slide 91 text

http://www.hypervizor.com/diagrams/

Slide 92

Slide 92 text

???

Slide 93

Slide 93 text

Open-source components

Slide 94

Slide 94 text

→ → ← → ← Virtualization OS Middleware Runtime Data Applications Servers Storage Networking ←

Slide 95

Slide 95 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization

Slide 96

Slide 96 text

Virtualization Vagrant Docker rkt

Slide 97

Slide 97 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization

Slide 98

Slide 98 text

Schedulers Kubernetes Swarm Nomad Marathon Aurora Chronos

Slide 99

Slide 99 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization

Slide 100

Slide 100 text

Config management Chubby Consul etcd ZooKeeper

Slide 101

Slide 101 text

Config management Chubby Consul etcd ZooKeeper CFEngine Chef Puppet Salt Ansible Terraform CloudFormation Cloud
 Deployment Manager

Slide 102

Slide 102 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization

Slide 103

Slide 103 text

Distributed storage EBS Ceph GlusterFS Flocker

Slide 104

Slide 104 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization

Slide 105

Slide 105 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization

Slide 106

Slide 106 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization

Slide 107

Slide 107 text

Software defined network Weave Canal Open vSwitch

Slide 108

Slide 108 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization

Slide 109

Slide 109 text

Service discovery DNS/SRV Eureka Consul etcd ZooKeeper SmartStack

Slide 110

Slide 110 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization Proxying &
 Load balancing

Slide 111

Slide 111 text

Proxying & Load balancing HAProxy nginx+ Traefik SmartStack Finagle Linkerd

Slide 112

Slide 112 text

Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization

Slide 113

Slide 113 text

Monitoring Prometheus InfluxData Riemann Nagios Zabbix Elasticsearch
 ELK stack OpenZipkin Lightstep AppDash Weave Scope Sysdig

Slide 114

Slide 114 text

No content

Slide 115

Slide 115 text

https://cncf.io/

Slide 116

Slide 116 text

No content

Slide 117

Slide 117 text

Thank you! Questions? http://weave.works • @peterbourgon