Cloud-native Application Stacks: a lightly-opinionated survey (May 2016)

A survey of the platforms and projects that make up the cloud-native computing landscape, circa May 2016.

Transcript

1. Cloud-native application stacks From orchestrators to monitoring A lightly-opinionated survey
2. These are just, like, my opinions, man.
3. Amazon Web Services • Google Cloud Platform • Microsoft Azure
 Mesosphere • Apache Mesos • DC/OS • Marathon • Aurora Chronos • Kubernetes • Docker • Docker Swarm • Deis Rancher • Kismatic • Apprenda • Cloud Foundry • OpenShift OpenStack • CoreOS • Project Atomic • Snappy Ubuntu Joyent Triton • SmartOS • RancherOS • Vagrant • Terraform Consul • Serf • Packer • Nomad • Flocker • Ceph • Flannel
 Weave • Canal • Prometheus • InfluxDB • Nagios • Zabbix
 Elasticsearch • OpenZipkin • LightStep • Appdash
5. The Datacenter as a Computer An Introduction to the Design of Warehouse-Scale Machines http://www.cs.berkeley.edu/~rxin/db-papers/WarehouseScaleComputing.pdf
6. https://twitter.com/alicemazzy/status/655306196128280576
10. http://architecturebestinfo.blogspot.de/2015/09/computer-architecture.html
12. CPU Mem Disk Net
13. CPU Mem Disk Net Cooling Circuits Power App OS User Silicon
14. CPU Mem Disk Net Cooling Circuits Power App OS User Silicon
15. CPU Mem Disk Net App OS
16. App OS CPU Mem Disk Net
17. CPU Mem Disk Net Data Batch Biz logic Resource mgmt Abstraction Index
18. CPU Mem Disk Net Data Batch Biz logic Resource mgmt Abstraction Index
19. CPU Mem Disk Net Data Batch Biz logic Resource mgmt Compute resources Distributed storage Local disk Physical network Software defined network Abstraction Index
20. CPU Mem Disk Net Data Batch Biz logic Resource mgmt Compute resources Distributed storage Local disk Physical network Software defined network Scheduler Service discovery Config mgmt Abstraction Virtualization Index
21. CPU Mem Disk Net Data Batch Biz logic Resource mgmt Compute resources Distributed storage Local disk Physical network Software defined network Scheduler Service discovery Config mgmt Abstraction Virtualization Database Data warehouse Biz logic Index
22. Compute resources Distributed storage Local disk Physical network Software defined network Scheduler Service discovery Config mgmt Virtualization Database Data warehouse Biz logic
23. Compute resources Distributed storage Local disk Physical network Software defined network Virtualization Database Data warehouse Biz logic Scheduler Service discovery Config mgmt
24. Compute resources Distributed storage Local disk Physical network Software defined network Virtualization Database Data warehouse Biz logic Scheduler Config mgmt Service discovery
25. Database RDBMS AP CP Document Transactional Oracle MySQL Postgres Dynamo Voldemort Cassandra Riak Chubby ZooKeeper etcd Consul Mongo Couch Redis F1 Cockroach Rethink
26. Data warehouse Hadoop RedShift BigQuery Spark Mahout Hbase Pig ···
27. Lambda Architecture Realtime + Batch Storm Kafka Druid
28. Compute resources Distributed storage Local disk Physical network Software defined network Virtualization Database Data warehouse Biz logic Scheduler Config mgmt Service discovery
29. Compute resources Distributed storage Local disk Physical network Software defined network Virtualization Data Biz logic Scheduler Service discovery Config mgmt
30. Compute resources Distributed storage Local disk Physical network Software defined network Virtualization Data Biz logic Scheduler Service discovery Config mgmt
31. Biz logic Monolith
32. Biz logic Foo service Bar service Baz service Users service Auth service Search service Reports service Audit service
33. Biz logic Foo service Bar service Baz service Users service Auth service Search service Reports service Audit service
34. Biz logic Foo service Bar service Baz service Users service Auth service Search service Reports service Audit service Continuous delivery Continuous integration Monitoring Tracing Safety Application logging Event sourcing
35. Sean Treadway, SoundCloud
36. Sean Treadway, SoundCloud
37. Sean Treadway, SoundCloud
38. Sean Treadway, SoundCloud
39. Compute resources Distributed storage Local disk Physical network Software defined network Scheduler Service discovery Config mgmt Virtualization Data Biz logic
40. Monitoring Compute resources Distributed storage Local disk Physical network Software defined network Scheduler Service discovery Config mgmt Virtualization Data Biz logic
41. Virtualization Distributed storage Local disk Physical network Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Compute resources
42. Virtualization Distributed storage Local disk Physical network Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Compute resources
43. Physical network Virtualization Distributed storage Local (scratch Software defined
 network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Compute resources
44. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization
45. Application Platform Infrastructure Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization
46. http://trapptechnology.com/saas-iaas-paas-decoding-cloud-acronyms/
48. Software defined network Monitoring Distr. storage Scheduler Config mgmt Data Service discovery Compute resources Virtualization
49. ¯\_(ϑ)_/¯ Software defined network Monitoring Distr. storage Scheduler Config mgmt Data Service discovery Compute resources Virtualization
50. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization Our new warehouse computer
51. Cloud platforms
52. Virtualization OS Middleware Runtime Data Applications Servers Storage Networking { }
53. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization AWS
54. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization EC2 S3 RedShift VPC Route53 CloudWatch CloudFormation RDS Machine Learning Kinesis Dynamo ELB ECS Lambda
55. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization GCP
56. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization GCE GKE GAE BigTable SQL Datastore BigQuery Cloud Monitoring Data* Deployment manager Cloud Storage Cloud Networking
57. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization Azure?
58. Newer
59. Open-source platforms
60. Virtualization OS Middleware Runtime Data Applications Servers Storage Networking { }
61. https://mesos.apache.org/ https://mesosphere.com/
62. https://thecustomizewindows.com/2014/12/mesosphere-data-center-operating-system/
63. https://aphyr.com/posts/326-call-me-maybe-chronos
64. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization Master Slave Slave Slave ZooKeeper Marathon Aurora Chronos DNS Master Master
65. http://kubernetes.io/
66. http://www.slideshare.net/brendandburns/defrag-2014-41815642
67. http://kubernetes.io/
68. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring API server Kubelet Kubelet Kubelet etcd SkyDNS Various Volume mounts Controllers
69. http://devopscube.com/how-to-setup-and-configure-docker-swarm-cluster/
70. http://blog.daocloud.io/swarm_analysis_part1/
71. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization Docker Docker Docker Swarm Scheduler Node discovery libnetwork libkv
73. http://deis.io/ https://www.engineyard.com/
74. https://rancher.com
75. http://kismatic.com http://apprenda.com
76. https://pivotal.io/platform
77. https://www.openstack.org/
79. https://www.openshift.com
80. More enterprisey?
81. Less opinionated
82. Cloud-native operating systems
83. } Virtualization OS Middleware Runtime Data Applications Servers Storage Networking
84. https://coreos.com/
85. https://en.wikipedia.org/wiki/CoreOS
86. http://projectatomic.io/
87. https://access.redhat.com/products/red-hat-atomic-enterprise-platform
88. https://developer.ubuntu.com/en/snappy/
89. https://rancher.com OS
90. https://smartos.org/ https://joyent.com
91. http://www.hypervizor.com/diagrams/
92. ???
93. Open-source components
94. → → ← → ← Virtualization OS Middleware Runtime Data Applications Servers Storage Networking ←
95. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization
96. Virtualization Vagrant Docker rkt
97. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization
98. Schedulers Kubernetes Swarm Nomad Marathon Aurora Chronos
99. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization
100. Config management Chubby Consul etcd ZooKeeper
101. Config management Chubby Consul etcd ZooKeeper CFEngine Chef Puppet Salt Ansible Terraform CloudFormation Cloud
 Deployment Manager
102. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization
103. Distributed storage EBS Ceph GlusterFS Flocker
104. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization
105. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization
106. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization
107. Software defined network Weave Canal Open vSwitch
108. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization
109. Service discovery DNS/SRV Eureka Consul etcd ZooKeeper SmartStack
110. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization Proxying &
 Load balancing
111. Proxying & Load balancing HAProxy nginx+ Traefik SmartStack Finagle Linkerd
112. Compute resources Distributed storage Software defined network Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization
113. Monitoring Prometheus InfluxData Riemann Nagios Zabbix Elasticsearch
 ELK stack OpenZipkin Lightstep AppDash Weave Scope Sysdig
115. https://cncf.io/
117. Thank you! Questions? http://weave.works • @peterbourgon