Anatomy of a Cloud-Native Application Ahmet Alp Balkan (@ahmetb) SEPTEMBER 28, 2017

There has been never been a better time to build, deploy, run and monitor applications using open source software. “ ”

A bit about myself Software Engineer at Google Cloud ● Kubernetes open source project ● Google Container Engine (GKE) developer experience Software Engineer at Microsoft Azure ● Docker on Windows ● Azure Container Registry ● Docker images for ASP.NET ● Developed official C#/.NET library for Docker Say hello at @ahmetb.

GIFEE (Google Infrastructure for Everyone Else) Recently, tools similar to those used internally at Google became available as open source, through various channels: ● People reading research papers from Google are implementing the software as open source. ● Companies like Google started to open source tools they are using internally ● Many people who left Google needed similar tools at their new jobs/projects.

In 2015, a few companies decided to form a group. The mission was to make sure open source projects that make it possible to run software, distributed on tens of thousands of shared nodes successful. It’s part of The Linux Foundation. Founding members: Google, CoreOS, Docker, eBay, Twitter, IBM, Intel, Mesosphere, VMWare and many others. Currently 100+ members, including AWS, Microsoft, Oracle and Red Hat. Proprietary + Confidential Platinum Members

Some CNCF projects Proven to be successful open source projects. Critical for creating a future with scalable infrastructure and distributed software. They no longer belong to the companies who built them in the first place, CNCF owns them.

Anatomy of a Cloud Native Application Networking Cluster management RPC Continuous deployment Observability Application identity Service broker Runtime/packaging

Application runtime & packaging

Simplified application development Microservices Easier mental model for services that have separate roles. Developed independently. Scaled independently. Crashes independently. Dynamic scheduling Apps crash and nodes die all the time. Make peace with microservices axiom and you can focus on less failure modes. Packaging Containers revolutionized how we package and deliver software using containers. Portable, atomic, isolated. Docker has showed it is possible.

Cluster management

Cluster management with Kubernetes Dynamically managed infrastructure Capability to add/remove compute nodes to your infrastructure. Multi-tenancy (can multiple users/teams use your clusters safely) Meet Kubernetes Kubernetes solves the scale needs elegantly It drives the clusters to the goal state defined by declarative configuration (i.e. manifest files) Enterprise-grade, production-ready. Declarative application management Infrastructure as code. To what extent can you declaratively describe your apps, configuration, secrets, networks? Can an intern deploy a copy of your whole stack?

guestbook-app.yaml Kubernetes resources are created declaratively, through manifest files. You can check these files into Git, and as you deploy new versions of the app, you can just apply the new manifest to the cluster. apiVersion: apps/v1beta1 kind: Deployment metadata: name: guestbook spec: selector: matchLabels: run: guestbook tier: frontend template: metadata: labels: run: guestbook tier: frontend spec: replicas: 3 containers: - image: gcr.io/ahmetb/guestbook:v2 name: frontend ports: - containerPort: 80

guestbook-app.yaml apiVersion: apps/v1beta1 kind: Deployment metadata: name: guestbook spec: selector: matchLabels: run: guestbook tier: frontend template: metadata: labels: run: guestbook tier: frontend spec: replicas: 3 containers: - image: gcr.io/ahmetb/guestbook:v2 name: frontend ports: - containerPort: 80 Kubernetes resources are created declaratively, through manifest files. You can check these files into Git, and as you deploy new versions of the app, you can just apply the new manifest to the cluster.

guestbook-app.yaml Kubernetes resources are created declaratively, through manifest files. You can check these files into Git, and as you deploy new versions of the app, you can just apply the new manifest to the cluster. apiVersion: apps/v1beta1 kind: Deployment metadata: name: guestbook spec: selector: matchLabels: run: guestbook tier: frontend template: metadata: labels: run: guestbook tier: frontend spec: replicas: 3 containers: - image: gcr.io/ahmetb/guestbook:v2 name: frontend ports: - containerPort: 80

apiVersion: apps/v1beta1 kind: Deployment metadata: name: guestbook spec: selector: matchLabels: run: guestbook tier: frontend template: metadata: labels: run: guestbook tier: frontend spec: replicas: 3 containers: - image: gcr.io/ahmetb/guestbook:v2 name: frontend ports: - containerPort: 80 guestbook-app.yaml Kubernetes resources are created declaratively, through manifest files. You can check these files into Git, and as you deploy new versions of the app, you can just apply the new manifest to the cluster.

apiVersion: apps/v1beta1 kind: Deployment metadata: name: guestbook spec: selector: matchLabels: run: guestbook tier: frontend template: metadata: labels: run: guestbook tier: frontend spec: replicas: 3 containers: - image: gcr.io/ahmetb/guestbook:v2 name: frontend ports: - containerPort: 80 guestbook-app.yaml Kubernetes resources are created declaratively, through manifest files. You can check these files into Git, and as you deploy new versions of the app, you can just apply the new manifest to the cluster.

Production readiness Kubernetes is mature, and has a strong open source community. Google Container Engine (GKE) gives you a production-ready cluster in a few minutes. Google engineers do on-calls for your cluster. Easy migration to/from cloud Run your Docker containers locally in Minikube, then move to the cloud using the same manifest files: $ kubectl apply -f manifests/*.yaml Service discovery kube-dns add-on helps you communicate any service in the cluster by its name Example: $ curl http://guestbook:80 Some things you get with Kubernetes

Continuous integration & deployment

Repeatable deployments Hermetic builds help you pinpoint source+dependencies of a particular binary running in production. Rollback strategy Even when you use manifest files, can you rollback a service and its dependencies, configuration, etc safely? Tooling does not matter Most tools are fine. But, Spinnaker (by Netflix, now open source) offers advanced primitives and ways to build pipelines. (If your deployments require complex machinery). Automate your deployments Engineering time spent on each deployments is a waste. Automate your way out! CI/CD

Microservices networking

Meet the “Service mesh” Service traffic is routed over a proxy. No change to your application code. Proxy sync with a rule/routing list. Only configure the service mesh dynamically. Modern cluster networking You can enforce cluster-wide: ● retry policies ● timeouts ● access control policies ● encryption ● authorization ● custom routing

Service mesh model Pod app1 Pod app2

Service mesh model Pod app1 proxy Pod app2 proxy

Service mesh model Pod app1 proxy Pod app2 proxy machine-1 machine-2

Istio service mesh model Pod app1 envoy Pod app2 envoy

Istio service mesh model Pod app1 envoy Pod app2 envoy Istio pilot (control plane, rule list, ...) Program this component and the cluster will follow.

Application interfaces (RPC)

Binary API protocols Google has been using Protocol Buffers (protobuf) since 2001. Open sourced in 2008. Everything inside Google is protobuf. Use a RPC framework You probably stop using JSON REST APIs between services in your company. It’s prone to human errors, cannot easily revision the API, difficult to do streaming and compression. RPC frameworks gRPC Google also has released gRPC, a framework that makes it easy to use protobuf. Easy client/server code generation. Used by many open source projects, Square, Netflix etc.

example.proto service UserService { rpc GetUser (UserRequest) returns (UserReply) {} } message UserRequest { string id = 1; bool includeDetails = 2; } message UserReply { string id = 1; string username = 2; string email = 3; } The client and server code for this UserService is automatically generated, including the message types. Compile-time safety.

Observability

Tracing Trace a request (from user or internal) as it passes through microservices: ● Which microservices does the request pass through ● How long time is spent on each microservice ● How long a function is taking in a service Often sampled: 1/100 of requests It’s not working correctly, if you’re not observing. Black-box monitoring Looking at application or node memory/CPU/network metrics from outside. Also, health of an application as the user sees it. White-box monitoring Application exposes its internal metrics through an HTTP endpoint, collected by a tool. Counter: http.requests=5256 Gauge: http.connections=12 Metrics can be aggregated: → all instances of a service → all services in a region → apps labeled as staging=true Prometheus is the de-facto solution.

$ curl http://my-service.cluster.local:9000/metrics Http.requests 1412 http.errors.500 12 http.errors.404 765 cpu.threads 22 uptime_minutes 1128 accounts.created 12 accounts.logins.success 560 accounts.logins.failure 4 rpc.sent.count 880 rpc.sent.avg_sec 1.3525677 Whitebox: a /metrics endpoint Write queries to find out averages/percentiles/sums of these metrics, for all instances of this app. Take queries, make them alerts: - if no successful logins in past 5m, alert! - if failed logins are >75%, alert! - if mean uptime goes below 5 mins, alert! - if cpu avg stays above 80% for 15m, alert Assume this is for a frontend service that handles the /login, /signup requests.

Google Cloud Stackdriver trace

Uber Jaeger trace

Application identity

Authentication Who is sending me this request? Is this a “trusted” requestor? Your applications should have an identity verifiable by a public-key infrastructure (PKI). Use JWT. Istio also provides strong identity automatically, no changes to application code. In-cluster traffic security Can you identify and allow/block traffic to microservices? Authorization A trusted app does not mean it should be able to call all other services in the cluster. ACLs (access control lists) Kubernetes offers Network Policies (like firewalls)

Service Brokers

Open Service Broker API Delivering “services” to “applications” running within cloud-native platforms. Originated from Cloud Foundry. Examples: ● “Give me a MySQL database and deliver credentials to the app.” ● “Provision a storage bucket for my app.” Service brokerage Terminology Service Broker: a server that provisions resources. Service Instance: a request for provisioning new instance of a service (e.g.creating a database) Service Instance Binding: associating a service instance with an application. i.e. delivering credentials and access details. Service Catalog: orchestrates the concepts above in a platform, like Kubernetes.

apiVersion: servicecatalog.k8s.io/v1alpha1 kind: Broker metadata: name: mysql-broker spec: url: http://mysql-broker.svc.cluster.local Terminology Service Broker: a server that provisions resources. Kubernetes Service catalog – example concept (alpha, subject to change)

apiVersion: servicecatalog.k8s.io/v1alpha1 kind: Instance metadata: name: guestbook-db spec: serviceClassName: mysql-database planName: smalldb Terminology Service Broker: a server that provisions resources. Service Instance: a request for provisioning new instance of a service (e.g.creating a database) Kubernetes Service catalog – example concept (alpha, subject to change)

apiVersion: servicecatalog.k8s.io/v1alpha1 kind: Binding metadata: name: guestbook-binding spec: instanceRef: name: guestbook-db secretName: guestbook-db-password Terminology Service Broker: a server that provisions resources. Service Instance: a request for provisioning new instance of a service (e.g.creating a database) Kubernetes Service catalog – example concept (alpha, subject to change)

Anatomy of a Cloud Native Application Networking Cluster management RPC Continuous deployment Observability Application identity Service broker Runtime/packaging

Thanks! Ahmet Alp Balkan (@ahmetb)