Minimum Viable Infrastructure

I'm building Convox, an open-source platform that expertly configures AWS to run and manage any application. In doing this I have discovered a minimal set of 11 cloud infrastructure services that form an architecture that supports any application.

Here I'll introduce the 11 service types that support our secure computing, application workload, and operational needs. I'll show how these services are used together to manage a secure computing cluster that can build Docker images for an app, roll out new images to production with zero downtime, and scale to handle any load.

Finally we'll look at the cloud computing landscape to see how all the cloud providers and software projects compare with offering these 11 service types.

Transcript

1. Minimum Viable Infrastructure Noah Zoschke noah@convox.com @nzoschke LA DevOps 09-26-2016
2. Goal • Take a web app • Run it in the cloud forever • Sleep well every night 10000 Year Clock from http://www.mawer.com/insights/articles/the-10000-year-clock/
3. Strategy • Developer provides app code and manifest • Community develops strategy and architecture to run it • Providers operate cloud services to support it • Software automates setup, monitoring and maintenance Leroy Buffington skyscraper (1888) from http://patentroom.com/architecture
4. Single Node Architecture Well Understood Secure Compute • CPU • Memory • Network • Crypto ┌────────────────────────┐ ┌──┤ HAProxy ├─┐ │┌─┤ Load Balancer ├┐│ ││ └────────────────────────┘││ ││ ┌──────┐┌──────┐┌──────┐ ││ ││ │apache││ php ││python│ ││ ││ └──────┘└──────┘└──────┘ ││ ││ iptables ││ ││ Network Security ││ │└───────────────────────────┘│ │┌───────┐┌────────┐┌────────┐│ ││syslogd││collectd││Postgres││ ││ Log ││ Metric ││Database││ │└───────┘└────────┘└────────┘│ │┌───────────────────────────┐│ ││ ┌──────────┐ ┌──────────┐ ││ ││ │ key.pem │ │ /var/ │ ││ ││ │ Crypto │ │ Data │ ││ ││ └──────────┘ └──────────┘ ││ ││ ext4 ││ ││ File System ││ │└───────────────────────────┘│ │ Ubuntu OS │ └─────────────────────────────┘ ┌───┐┌───┐┌───┐┌───┐ │CPU││RAM││ETH││SSD│ └───┘└───┘└───┘└───┘ App Workload • Package • Config • Data • Proxy Visibility • Logs • Metrics • Events
5. Single Node Challenges Well Understood • OS and Software Versions • Single Point of Failure • Coarse Scaling • Constant Pricing ┌────────────────────────┐ ┌──┤ HAProxy ├─┐ │┌─┤ Load Balancer ├┐│ ││ └────────────────────────┘││ ││ ┌──────┐┌──────┐┌──────┐ ││ ││ │apache││ php ││python│ ││ ││ └──────┘└──────┘└──────┘ ││ ││ iptables ││ ││ Network Security ││ │└───────────────────────────┘│ │┌───────┐┌────────┐┌────────┐│ ││syslogd││collectd││Postgres││ ││ Log ││ Metric ││Database││ │└───────┘└────────┘└────────┘│ │┌───────────────────────────┐│ ││ ┌──────────┐ ┌──────────┐ ││ ││ │ key.pem │ │ /var/ │ ││ ││ │ Crypto │ │ Data │ ││ ││ └──────────┘ └──────────┘ ││ ││ ext4 ││ ││ File System ││ │└───────────────────────────┘│ │ Ubuntu OS │ └─────────────────────────────┘ ┌───┐┌───┐┌───┐┌───┐ │CPU││RAM││ETH││SSD│ └───┘└───┘└───┘└───┘
6. Service Oriented Architecture Evolving Best Practices Secure Compute • CPU/Memory → VM • Network → VPC • Crypto → Crypto App Workload • Package → Image, Container • Config → Crypto, Blob • Data → Database • Proxy → Load Balancer Visibility • Logs → Logs • Metrics → Metrics • Events → KV, Blob
7. Minimum Viable Infrastructure Simple Monolith → Simple SOA ┌────────────────────────┐ ┌──┤ HAProxy ├─┐ │┌─┤ Load Balancer ├┐│ ││ └────────────────────────┘││ ││ ┌──────┐┌──────┐┌──────┐ ││ ││ │apache││apache││python│ ││ ││ └──────┘└──────┘└──────┘ ││ ││ iptables ││ ││ Network Security ││ │└───────────────────────────┘│ │┌───────┐┌────────┐┌────────┐│ ││syslogd││collectd││Postgres││ ││ Log ││ Metric ││Database││ │└───────┘└────────┘└────────┘│ │┌───────────────────────────┐│ ││ ┌──────────┐ ┌──────────┐ ││ ││ │ key.pem │ │ /var/ │ ││ ││ │ Crypto │ │ Data │ ││ ││ └──────────┘ └──────────┘ ││ ││ ext4 ││ ││ File System ││ │└───────────────────────────┘│ │ Linux OS │ └─────────────────────────────┘ ┌───┐┌───┐┌───┐┌───┐ │CPU││RAM││ETH││SSD│ └───┘└───┘└───┘└───┘ ┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┌────────┐ │ │ │Database│ │ │ └────────┘ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐┌───┐┌──────┐┌──┐┌────┐ │Crypto││Image││Log││Metric││KV││Blob│ └──────┘└─────┘└───┘└──────┘└──┘└────┘
8. SOA Advantages Somewhat Understood • Service Level Agreements • Versioned APIs • Independent Scaling • Utility Pricing ┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┌────────┐ │ │ │Database│ │ │ └────────┘ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐┌───┐┌──────┐┌──┐┌────┐ │Crypto││Image││Log││Metric││KV││Blob│ └──────┘└─────┘└───┘└──────┘└──┘└────┘
9. SOA Challenges Somewhat Understood • Competing providers, services and software solutions • Configuration • Customization • Impedance Mismatch ┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┌────────┐ │ │ │Database│ │ │ └────────┘ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐┌───┐┌──────┐┌──┐┌────┐ │Crypto││Image││Log││Metric││KV││Blob│ └──────┘└─────┘└───┘└──────┘└──┘└────┘
10. Virtual Machine (VM) Secure Compute A Virtual Machine (VM) service provides CPU, Memory, Networking and a server Operating System. ┏━━━━━━━━━━━━━━━━━┓ ┃ ┃ ┃ ┃ ┃ ┃ ┃ VM ┃ ┗━━━━━━━━━━━━━━━━━┛
11. Virtual Private Cloud (VPC) Secure Compute A Virtual Private Cloud (VPC) service provides private networking. ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ ┃ ┃ ┃ ┃ ┌─────────────────┐ ┃ ┃ │ │ ┃ ┃ │ │ ┃ ┃ │ │ ┃ ┃ │ VM │ ┃ ┃ └─────────────────┘ ┃ ┃ ┃ ┃ ┃ ┃ VPC ┃ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
12. Cryptography Secure Compute A Cryptography (Crypto) service provides a way to create, import and rotate an unguessable key for encrypting data, and provides a way to get and audit access to the key for decrypting data. ┌──────────────────────────────────────┐ │ │ │ │ │ ┌─────────────────┐ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ VM │ │ │ └─────────────────┘ │ │ │ │ │ │ VPC │ └──────────────────────────────────────┘ ┏━━━━━━┓ ┃Crypto┃ ┗━━━━━━┛
13. Image App Workload An Image service provides a private place to push, store and pull binary application and dependency data. ┌──────────────────────────────────────┐ │ │ │ │ │ ┌─────────────────┐ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ VM │ │ │ └─────────────────┘ │ │ │ │ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┏━━━━━┓ │Crypto│┃Image┃ └──────┘┗━━━━━┛
14. Container App Workload A Container service provides a way to run many specialized process types, defined by Images and commands, on fewer homogeneous VMs. ┌──────────────────────────────────────┐ │ │ │ │ │┌─────────────────┐┌─────────────────┐│ ││┏━━━━━┓┏━━━━━━━━┓││ ┏━━━━━┓ ││ ││┃web 1┃┃worker 1┃││ ┃web 2┃ ││ ││┗━━━━━┛┗━━━━━━━━┛││ ┗━━━━━┛ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ │ │ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐ │Crypto││Image│ └──────┘└─────┘
15. Load Balancer App Workload A Load Balancer service provides a single, stable hostname that accepts network requests and proxies them to one or more healthy Containers. ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┌┫ Load Balancer ┣┐ │┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛│ │ │ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ │ │ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐ │Crypto││Image│ └──────┘└─────┘
16. Database App Workload A Database service provides a single network hostname that is used to save, update and delete application data records concurrently by one or more Containers. ┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┏━━━━━━━━┓ │ │ ┃Database┃ │ │ ┗━━━━━━━━┛ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐ │Crypto││Image│ └──────┘└─────┘
17. Log Visibility A Log service provides a place to send ordered text from app Containers’ stdout and stderr streams, so all the application events can be tailed in real-time and searched later. ┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┌────────┐ │ │ │Database│ │ │ └────────┘ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐┏━━━┓ │Crypto││Image│┃Log┃ └──────┘└─────┘┗━━━┛
18. Metric Visibility A Metric service provides a place to save numerical data from apps and underlying cloud services so operational properties of the entire system can be aggregated, analyzed and graphed in real-time and reviewed later. ┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┌────────┐ │ │ │Database│ │ │ └────────┘ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐┌───┐┏━━━━━━┓ │Crypto││Image││Log│┃Metric┃ └──────┘└─────┘└───┘┗━━━━━━┛
19. Key-Value (KV) Visibility A Key-Value (KV) service provides a way to save small amounts of structured data in a highly-available (HA) fashion while still being easy to query for reporting purposes. ┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┌────────┐ │ │ │Database│ │ │ └────────┘ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐┌───┐┌──────┐┏━━┓ │Crypto││Image││Log││Metric│┃KV┃ └──────┘└─────┘└───┘└──────┘┗━━┛
20. Blob Visibility A Blob service provides a way to save and retrieve large amounts unstructured data in a HA fashion. ┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┌────────┐ │ │ │Database│ │ │ └────────┘ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐┌───┐┌──────┐┌──┐┏━━━━┓ │Crypto││Image││Log││Metric││KV│┃Blob┃ └──────┘└─────┘└───┘└──────┘└──┘┗━━━━┛
21. MVI Apps
22. Rails App Production Grade ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ web: │ build: . │ labels: │ - convox.port.443.protocol=tls │ - convox.port.443.proxy=true │ links: │ - db ┌───────────────────┐ │ - redis │ │ TLS Load Balancer │ ports: ┌┤https + websockets ├┐ ┌─────────┐ ┌─────────┐ │ - 80:4000 │ │└────────┬─┬────────┘│ │┌───────┐│ │┌───────┐│ - 443:4001 │ ┌─────┐ │ │ ┌─────┐ │ ││ rake ││ ││ rake ││ │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ │ │nginx│ │ │ │nginx│ │ ││resque ││ ││resque ││ worker: Rails Image │ │ └─────┘ │ │ └─────┘ │ │└───────┘│ │└───────┘│ │ build: . │ │ │ ┌─────┐ │ │ ┌─────┐ │ │ worker │ │ worker │ command: rake resque work ──────▶ Ubuntu 16.04 OS │──────▶│ │ruby │ │ │ │ruby │ │ │Container│ │Container│ │ │ │ pg, redis gems │ │puma │ │ │ │puma │ │ └─────────┘ └─────────┘ db: + code │ │ └─────┘ │ │ └─────┘ │ ┌─────────┐ ┌─────────┐ │ image: convox/postgres │ └ ─ ─ ─ ─ ─ ─ ─ ─ │ web │ │ web │ │┌───────┐│ │┌───────┐│ labels: │Container│ │Container│ ││ rake ││ ││ rake ││ │ - convox.health.timeout=60 │ └─────────┘ └─────────┘ ││resque ││ ││resque ││ ports: ┌─────────┐ ┌─────────┐ │└───────┘│ │└───────┘│ │ - 5432 │ │Postgres │ │ Redis │ │ worker │ │ worker │ volumes: │Database │ │Database │ │Container│ │Container│ │ - /var/lib/postgresql/data │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │redis: │ image: convox/redis │ ports: │ - 6433 └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘
23. Challenges • Containers, Images, Load balancers and data services only • Just like Heroku / Twelve-Factor, but more ports and protocols • Constraints are good • Orchestration • Rolling deploys • Health Checks • Resilience is good
24. MVI Systems
25. Build System API docker build Load Balancer Blob KV Container daemon Container Image ┌┐ ┌┐ ┌┐ ┌┐ ┌┐ ┌┐ ┌┐ POST code ││ ││ ││ ││ ││ ││ ││ ──────────────▶││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ PUT obj ││ ││ ││ ││ ││ ││ ││──────────────▶││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ PUT item ││ ││ ││ ││ ││ ││ ││───────────────┼┼──────────────▶││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ RUN container ││ ││ ││ RUN container ││ ││ ││ ││───────────────┼┼───────────────┼┼──────────────▶││─ ─ ─ ─ ─ ─ ─ ▶││ ││ ││ ││ ││ ││ ││ ││ ││ ││ GET logs ││ ││ ││ ││ ││ ││ ││ ──────────────▶││─ ─ ─ ─ ─ ─ ─ ─│┼ ─ ─ ─ ─ ─ ─ ─ ┼│─ ─ ─ ─ ─ ─ ─ ─│┼ ─ ─ ─ ─ ─ ─ ─▶││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ GET obj ││ ││ ││ ││◀──────────────┼┼───────────────┼┼───────────────┼┼───────────────││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ BUILD ││ ││ ││ ││ ││ ││ ││◀──────────────││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ TAG ││ ││ ││ ││ ││ ││ ││◀──────────────││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ PUSH ││ ││ ││ ││ ││ ││ ││◀──────────────││ ││ ││ ││ ││ ││ ││─ ─ ─ ─ ─ ─ ─ ─│┼ ─ ─ ─ ─ ─ ─ ─▶││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ UPDATE item ││ ││ ││ ││ ││◀──────────────┼┼───────────────┼┼───────────────││ ││ ││ ││ ││ ││ ││ ││ ││ GET result ││ ││ ││ ││ ││ ││ ││ ──────────────▶││─ ─ ─ ─ ─ ─ ─ ─│┼ ─ ─ ─ ─ ─ ─ ─▶││ ││ ││ ││ ││ └┘ └┘ └┘ └┘ └┘ └┘ └┘
26. MVI Landscape
27. IaaS Providers Amazon Google Microsoft VM EC2 Google Compute Engine Azure Virtual Machines VPC VPC GCE Firewall Rules Azure Virtual Network Crypto KMS Azure Key Vault Image ECR GCE Registry Container ECS Google Container Engine (Kubernetes) Azure Container Service (DC/OS or Swarm) Load Balancer ELB Google Cloud Load Balancing Azure Load Balancer Database RDS Google Cloud SQL Azure SQL Log CloudWatch Logs Stackdriver Logging Azure Log Analytics Metric CloudWatch Metrics Stackdriver Monitoring Azure Monitor KV DynamoDB Google Cloud Bigtable Azure Tables Storage Blob S3 Google Cloud Storage Azure Blob Storage https://cloud.google.com/docs/compare/aws/ https://azure.microsoft.com/en-us/services/
28. Service Providers VM VPC Crypto Image Container LB DB Log Metric KV Blob Digital Ocean Heroku CloudFlare Heroku Postgres Sumo logic MongoLab Rackspace Cloud Files Linode CircleCI Compose Loggly DataDog Compose Rackspace Docker Hub Papertrail Newrelic Quay Librato
29. Software Projects VM VPC Crypto Image Container LB DB Log Metric KV Blob KVM OpenStack Vault Docker Distribution Docker Swarm HAProxy Postgres Elastic Search RRD Redis minio Xen Knox GitLab Kubernetes nginx MySQL Splunk Graphite MongoDB VMWare Jenkins Mesos Prometheus Cassandra VirtualBox Nomad CouchDB
30. MVI Open Questions • Future additions or simplifications? • Three database types? • Shared File System • Lambda • Anti-patterns? • Service discovery • Overlay networking • Naming Things • What knobs and gauges?
31. MVI Key Takeaways • We can support any app or system with 11 component SOA • Building with MVI services has advantages: • Off-the-shelf components • Privacy • Reliability • Scale • Time to first deploy • Total cost of ownership
32. Thanks! • Feedback and questions to @nzoschke • See MVI in action • https://github.com/convox/rack • https://github.com/nzoschke/mvi