Cloud-native Application Stacks: a lightly-opinionated survey (May 2016)

Cloud-native Application Stacks: a lightly-opinionated survey (May 2016)

A survey of the platforms and projects that make up the cloud-native computing landscape, circa May 2016.

Eb68eb9603a0e8822eda69e7ca721866?s=128

Peter Bourgon

May 25, 2016
Tweet

Transcript

  1. Cloud-native application stacks From orchestrators to monitoring A lightly-opinionated survey

  2. These are just, like, my opinions, man.

  3. Amazon Web Services • Google Cloud Platform • Microsoft Azure


    Mesosphere • Apache Mesos • DC/OS • Marathon • Aurora Chronos • Kubernetes • Docker • Docker Swarm • Deis Rancher • Kismatic • Apprenda • Cloud Foundry • OpenShift OpenStack • CoreOS • Project Atomic • Snappy Ubuntu Joyent Triton • SmartOS • RancherOS • Vagrant • Terraform Consul • Serf • Packer • Nomad • Flocker • Ceph • Flannel
 Weave • Canal • Prometheus • InfluxDB • Nagios • Zabbix
 Elasticsearch • OpenZipkin • LightStep • Appdash
  4. None
  5. The Datacenter as a Computer An Introduction to the Design

    of Warehouse-Scale Machines http://www.cs.berkeley.edu/~rxin/db-papers/WarehouseScaleComputing.pdf
  6. https://twitter.com/alicemazzy/status/655306196128280576

  7. None
  8. None
  9. None
  10. http://architecturebestinfo.blogspot.de/2015/09/computer-architecture.html

  11. None
  12. CPU Mem Disk Net

  13. CPU Mem Disk Net Cooling Circuits Power App OS User

    Silicon
  14. CPU Mem Disk Net Cooling Circuits Power App OS User

    Silicon
  15. CPU Mem Disk Net App OS

  16. App OS CPU Mem Disk Net

  17. CPU Mem Disk Net Data Batch Biz logic Resource mgmt

    Abstraction Index
  18. CPU Mem Disk Net Data Batch Biz logic Resource mgmt

    Abstraction Index
  19. CPU Mem Disk Net Data Batch Biz logic Resource mgmt

    Compute resources Distributed storage Local disk Physical network Software defined network Abstraction Index
  20. CPU Mem Disk Net Data Batch Biz logic Resource mgmt

    Compute resources Distributed storage Local disk Physical network Software defined network Scheduler Service discovery Config mgmt Abstraction Virtualization Index
  21. CPU Mem Disk Net Data Batch Biz logic Resource mgmt

    Compute resources Distributed storage Local disk Physical network Software defined network Scheduler Service discovery Config mgmt Abstraction Virtualization Database Data warehouse Biz logic Index
  22. Compute resources Distributed storage Local disk Physical network Software defined

    network Scheduler Service discovery Config mgmt Virtualization Database Data warehouse Biz logic
  23. Compute resources Distributed storage Local disk Physical network Software defined

    network Virtualization Database Data warehouse Biz logic Scheduler Service discovery Config mgmt
  24. Compute resources Distributed storage Local disk Physical network Software defined

    network Virtualization Database Data warehouse Biz logic Scheduler Config mgmt Service discovery
  25. Database RDBMS AP CP Document Transactional Oracle MySQL Postgres Dynamo

    Voldemort Cassandra Riak Chubby ZooKeeper etcd Consul Mongo Couch Redis F1 Cockroach Rethink
  26. Data warehouse Hadoop RedShift BigQuery Spark Mahout Hbase Pig ···

  27. Lambda Architecture Realtime + Batch Storm Kafka Druid

  28. Compute resources Distributed storage Local disk Physical network Software defined

    network Virtualization Database Data warehouse Biz logic Scheduler Config mgmt Service discovery
  29. Compute resources Distributed storage Local disk Physical network Software defined

    network Virtualization Data Biz logic Scheduler Service discovery Config mgmt
  30. Compute resources Distributed storage Local disk Physical network Software defined

    network Virtualization Data Biz logic Scheduler Service discovery Config mgmt
  31. Biz logic Monolith

  32. Biz logic Foo service Bar service Baz service Users service

    Auth service Search service Reports service Audit service
  33. Biz logic Foo service Bar service Baz service Users service

    Auth service Search service Reports service Audit service
  34. Biz logic Foo service Bar service Baz service Users service

    Auth service Search service Reports service Audit service Continuous delivery Continuous integration Monitoring Tracing Safety Application logging Event sourcing
  35. Sean Treadway, SoundCloud

  36. Sean Treadway, SoundCloud

  37. Sean Treadway, SoundCloud

  38. Sean Treadway, SoundCloud

  39. Compute resources Distributed storage Local disk Physical network Software defined

    network Scheduler Service discovery Config mgmt Virtualization Data Biz logic
  40. Monitoring Compute resources Distributed storage Local disk Physical network Software

    defined network Scheduler Service discovery Config mgmt Virtualization Data Biz logic
  41. Virtualization Distributed storage Local disk Physical network Software defined network

    Scheduler Biz logic Service discovery Config mgmt Data Monitoring Compute resources
  42. Virtualization Distributed storage Local disk Physical network Software defined network

    Scheduler Biz logic Service discovery Config mgmt Data Monitoring Compute resources
  43. Physical network Virtualization Distributed storage Local (scratch Software defined
 network

    Scheduler Biz logic Service discovery Config mgmt Data Monitoring Compute resources
  44. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization
  45. Application Platform Infrastructure Compute resources Distributed storage Software defined network

    Scheduler Biz logic Service discovery Config mgmt Data Monitoring Virtualization
  46. http://trapptechnology.com/saas-iaas-paas-decoding-cloud-acronyms/

  47. None
  48. Software defined network Monitoring Distr. storage Scheduler Config mgmt Data

    Service discovery Compute resources Virtualization
  49. ¯\_(ϑ)_/¯ Software defined network Monitoring Distr. storage Scheduler Config mgmt

    Data Service discovery Compute resources Virtualization
  50. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization Our new warehouse computer
  51. Cloud platforms

  52. Virtualization OS Middleware Runtime Data Applications Servers Storage Networking {

    }
  53. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization AWS
  54. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization EC2 S3 RedShift VPC Route53 CloudWatch CloudFormation RDS Machine Learning Kinesis Dynamo ELB ECS Lambda
  55. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization GCP
  56. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization GCE GKE GAE BigTable SQL Datastore BigQuery Cloud Monitoring Data* Deployment manager Cloud Storage Cloud Networking
  57. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization Azure?
  58. Newer

  59. Open-source platforms

  60. Virtualization OS Middleware Runtime Data Applications Servers Storage Networking {

    }
  61. https://mesos.apache.org/ https://mesosphere.com/

  62. https://thecustomizewindows.com/2014/12/mesosphere-data-center-operating-system/

  63. https://aphyr.com/posts/326-call-me-maybe-chronos

  64. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization Master Slave Slave Slave ZooKeeper Marathon Aurora Chronos DNS Master Master
  65. http://kubernetes.io/

  66. http://www.slideshare.net/brendandburns/defrag-2014-41815642

  67. http://kubernetes.io/

  68. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring API server Kubelet Kubelet Kubelet etcd SkyDNS Various Volume mounts Controllers
  69. http://devopscube.com/how-to-setup-and-configure-docker-swarm-cluster/

  70. http://blog.daocloud.io/swarm_analysis_part1/

  71. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization Docker Docker Docker Swarm Scheduler Node discovery libnetwork libkv
  72. None
  73. http://deis.io/ https://www.engineyard.com/

  74. https://rancher.com

  75. http://kismatic.com http://apprenda.com

  76. https://pivotal.io/platform

  77. https://www.openstack.org/

  78. None
  79. https://www.openshift.com

  80. More enterprisey?

  81. Less opinionated

  82. Cloud-native operating systems

  83. } Virtualization OS Middleware Runtime Data Applications Servers Storage Networking

  84. https://coreos.com/

  85. https://en.wikipedia.org/wiki/CoreOS

  86. http://projectatomic.io/

  87. https://access.redhat.com/products/red-hat-atomic-enterprise-platform

  88. https://developer.ubuntu.com/en/snappy/

  89. https://rancher.com OS

  90. https://smartos.org/ https://joyent.com

  91. http://www.hypervizor.com/diagrams/

  92. ???

  93. Open-source components

  94. → → ← → ← Virtualization OS Middleware Runtime Data

    Applications Servers Storage Networking ←
  95. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization
  96. Virtualization Vagrant Docker rkt

  97. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization
  98. Schedulers Kubernetes Swarm Nomad Marathon Aurora Chronos

  99. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization
  100. Config management Chubby Consul etcd ZooKeeper

  101. Config management Chubby Consul etcd ZooKeeper CFEngine Chef Puppet Salt

    Ansible Terraform CloudFormation Cloud
 Deployment Manager
  102. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization
  103. Distributed storage EBS Ceph GlusterFS Flocker

  104. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization
  105. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization
  106. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization
  107. Software defined network Weave Canal Open vSwitch

  108. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization
  109. Service discovery DNS/SRV Eureka Consul etcd ZooKeeper SmartStack

  110. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization Proxying &
 Load balancing
  111. Proxying & Load balancing HAProxy nginx+ Traefik SmartStack Finagle Linkerd

  112. Compute resources Distributed storage Software defined network Scheduler Biz logic

    Service discovery Config mgmt Data Monitoring Virtualization
  113. Monitoring Prometheus InfluxData Riemann Nagios Zabbix Elasticsearch
 ELK stack OpenZipkin

    Lightstep AppDash Weave Scope Sysdig
  114. None
  115. https://cncf.io/

  116. None
  117. Thank you! Questions? http://weave.works • @peterbourgon