Scalable Microservices with gRPC, Kubernetes, and Containers (DevFest Ukraine)

Slide 1

Slide 1 text

Scalable Microservices with gRPC, Kubernetes, and Containers

Slide 2

Slide 2 text

#dfua, @tekgrrl, @googlecloud Mandy Waite Developer Advocate +MandyWaite @tekgrrl

Slide 3

Slide 3 text

Image by Connie Zhou

Slide 4

Slide 4 text

#dfua, @tekgrrl, @googlecloud job hello_world = { runtime = { cell = 'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 } replicas = 5 // Number of tasks } 10000 Developer View

Slide 5

Slide 5 text

#dfua, @tekgrrl, @googlecloud Developer View

Slide 6

Slide 6 text

#dfua, @tekgrrl, @googlecloud web browsers BorgMaster link shard UI shard BorgMaster link shard UI shard BorgMaster link shard UI shard BorgMaster link shard UI shard Scheduler borgcfg web browsers scheduler Borglet Borglet Borglet Borglet Config file BorgMaster link shard UI shard persistent store (Paxos) Binary What just happened? Cell Storage Developer View

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Image by Connie Zhou Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world!

Slide 13

Slide 13 text

#dfua, @tekgrrl, @googlecloud Images by Connie Zhou Observations: ● Containers make everyone more productive ● Everything at Google runs in a Container! ● The Datacenter is one big System ○ On which we schedule Containers See this video for more on Scheduling at Google: https://youtu.be/elHbkoJOBNE?t=4426 http://goo.gl/1C4nuo (Borg paper)

Slide 14

Slide 14 text

Kubernetes

Slide 15

Slide 15 text

#dfua, @tekgrrl, @googlecloud Greek for “Helmsman”; also the root of the word “Governor” • Orchestrator for Docker containers • Supports multi-cloud environments • Inspired and informed by Google’s experiences and internal systems • Open source, written in Go Manage applications, not machines Kubernetes

Slide 16

Slide 16 text

#dfua, @tekgrrl, @googlecloud web browsers y Kubelet Kubelet Kubelet Kubelet Kubernetes Master Replication Controller Scheduler API Server Kube-UI Container Registry kubectl Proxy web browsers Developer View (Kubernetes)

Slide 17

Slide 17 text

#dfua, @tekgrrl, @googlecloud The atom of scheduling for containers Represents an application specific logical host Hosts containers and volumes Each has its own routable (no NAT) IP address Ephemeral • Pods are functionally identical and therefore ephemeral and replaceable Pod Web Server Volume Consumers Pods

Slide 18

Slide 18 text

#dfua, @tekgrrl, @googlecloud Can be used to group multiple containers & shared volumes Containers within a pod are tightly coupled Shared namespaces • Containers in a pod share a network IP and port namespace • Pods Pod Git Synchronizer Node.js App Container Volume Consumers Github

Slide 19

Slide 19 text

#dfua, @tekgrrl, @googlecloud Pod Networking (across nodes) Pods have IPs which are routable Pods can reach each other without NAT ● Even across nodes No Brokering of Port Numbers These are fundamental requirements Many solutions ● Flannel, Weave, Cloud Provider 10.1.2.0/24 10.1.1.0/24 10.1.1.211 10.1.1.2 10.1.2.106 10.1.3.0/24 10.1.3.45 10.1.3.17 10.1.3.0/24

Slide 20

Slide 20 text

#dfua, @tekgrrl, @googlecloud Dashboard show: type = FE Pod Pod frontend Pod frontend Pod type = FE version = v2 type = FE ● Metadata with semantic meaning ● Membership identifier ● The only Grouping Mechanism Behavior Benefits ➔ Allow for intent of many users (e.g. dashboards) ➔ Build higher level systems … ➔ Queryable by Selectors Labels ← These are important

Slide 21

Slide 21 text

#dfua, @tekgrrl, @googlecloud Dashboard show: type = FE Pod Pod frontend Pod frontend Pod Pod Dashboard show: version = v2 type = FE version = v2 type = FE version = v2 ● Metadata with semantic meaning ● Membership identifier ● The only Grouping Mechanism Behavior Benefits ➔ Allow for intent of many users (e.g. dashboards) ➔ Build higher level systems … ➔ Queryable by Selectors Labels ← These are important

Slide 22

Slide 22 text

#dfua, @tekgrrl, @googlecloud Replication Controller Pod Pod frontend Pod version= v1 version = v1 Replication Controller #pods = 2 version = v1 show: version = v2 Behavior Benefits ● Keeps Pods running ● Gives direct control of Pod #s ● Grouped by Label Selector ➔ Recreates Pods, maintains desired state ➔ Fine-grained control for scaling ➔ Standard grouping semantics Replication Controllers

Slide 23

Slide 23 text

#dfua, @tekgrrl, @googlecloud Replication Controller Pod Pod frontend Pod frontend Pod Pod Replication Controller #pods = 1 version = v2 show: version = v2 version= v1 version = v1 version = v2 Replication Controller #pods = 2 version = v1 show: version = v2 Behavior Benefits ● Keeps Pods running ● Gives direct control of Pod #s ● Grouped by Label Selector ➔ Recreates Pods, maintains desired state ➔ Fine-grained control for scaling ➔ Standard grouping semantics Replication Controllers

Slide 24

Slide 24 text

#dfua, @tekgrrl, @googlecloud Portal (VIP) Client Pod Container Pod Container Pod Container Container A logical grouping of pods that perform the same function • grouped by label selector Load balances incoming requests across constituent pods Choice of pod is random but supports session affinity (ClientIP) Gets a stable virtual IP and port • also a DNS name Services

Slide 25

Slide 25 text

#dfua, @tekgrrl, @googlecloud Developer View (Replication Controller) spec: containers: - name: php-guestbook image: php-guestbook:containercamp resources: limits: memory: "128Mi" cpu: "500m" ports: - containerPort: 80 protocol: TCP replicas: 5

Slide 26

Slide 26 text

Slide 27

Slide 27 text

#dfua, @tekgrrl, @googlecloud Kubernetes Cluster Applications Ops Dev App A App B App C App A

Slide 28

Slide 28 text

HTTP/2, ProtoBufs and gRPC

Slide 29

Slide 29 text

● Improve end-user perceived latency ● Address the "head of line blocking" ● Not require multiple connections ● Retain the semantics of HTTP/1.1 "HTTP/2 is a protocol designed for low- latency transport of content over the World Wide Web"

Slide 30

Slide 30 text

#dfua, @tekgrrl, @googlecloud HTTP/2 HTTP/1.1 http://www.http2demo.io/

Slide 31

Slide 31 text

#dfua, @tekgrrl, @googlecloud ● One TCP connection ● Request → Stream ○ Streams are multiplexed ○ Streams are prioritized ● Binary framing layer ○ Prioritization ○ Flow control ○ Server push ● Header compression HTTP/2 in one slide…

Slide 32

Slide 32 text

#dfua, @tekgrrl, @googlecloud What are Protocol Buffers? Structured representation of data Google's lingua franca for data ● 48k+ Message Types ● 12k+ Proto files Evolutionary Development Incrementally solved problems, Now used for: ● RPC Systems ● Persistent Data Storage

Slide 33

Slide 33 text

#dfua, @tekgrrl, @googlecloud Protocol buffers: ● Efficient and compact binary data representation ● Clear compatibility rules; can easily be extended over time ● Generates idiomatic, easy to use classes for many languages ● Strongly typed; less error prone Why Protocol Buffers? Why?

Slide 34

Slide 34 text

#dfua, @tekgrrl, @googlecloud Protocol Buffers language version 3 Specified by syntax = “proto3”; proto2 continues to be supported All fields are optional in proto3 No user specified default values No groups (FYI for those that use them) Message Format (proto3) syntax = “proto3”; message Person { string name = 1; int32 id = 2; string email = 3; enum PhoneType { MOBILE = 0; HOME = 1; WORK = 2; } message PhoneNumber { string number = 1; PhoneType type = 2; } repeated PhoneNumber phone = 4; }

Slide 35

Slide 35 text

#dfua, @tekgrrl, @googlecloud Add new fields without breaking backwards-compatibility old implementations ignore the new fields when parsing In proto3 any field can be removed, but don’t renumber existing fields Extensible syntax = “proto3”; message Person { string name = 1; int32 id = 2; string email = 3; address addr = 4; message address { string firstLine = 1; string secondLine = 2; string postalCode = 3; string country = 4; } ... }

Slide 36

Slide 36 text

#dfua, @tekgrrl, @googlecloud https://github.com/google/protobuf Protocol Buffer Compiler

Slide 37

Slide 37 text

#dfua, @tekgrrl, @googlecloud gRPC goals Enable developers to build microservice-based applications Build an open source, standards-based, best-of-breed, feature-rich RPC system Create easy-to-use, efficient and idiomatic libraries micro-services performant and scalable efficient and idiomatic Provide a performant and scalable RPC framework

Slide 38

Slide 38 text

“The biggest issue in changing a monolith into microservices lies in changing the communication pattern.” - Martin Fowler

Slide 39

Slide 39 text

#dfua, @tekgrrl, @googlecloud IDL to describe an API Automatically generated servers and clients in 10+ languages Takes advantage of feature set of HTTP/2 Lightweight open connections Point to point Streaming! Bidirectional streaming! gRPC in a nutshell

Slide 40

Slide 40 text

#dfua, @tekgrrl, @googlecloud Getting Started Define a service in a .proto file using Protocol Buffers IDL Generate server and client code using the protocol buffer compiler Use the gRPC API to write a simple client and server for your service in the languages of your choice

Slide 41

Slide 41 text

#dfua, @tekgrrl, @googlecloud gRPC Language Support Implementations ● C core ○ Native bindings in C++, Node.js, Python, Ruby, ObjC, PHP, C# ● Java using Netty or OkHttp (+ inProcess for testing) ● Go

Slide 42

Slide 42 text

Microservices

Slide 43

Slide 43 text

JavaScript var http = require('http'); var server = http.createServer(function (request, response) { response.writeHead(200, {"Content-Type": "text/plain"}); response.end("Hello World\n"); }); server.listen(3000); A Hello World Microservice

Slide 44

Slide 44 text

#dfua, @tekgrrl, @googlecloud Build and tag a Container Image Push Image to Container Repository Create Service and ReplicationController config files Create ReplicationController Create Service From Code to deployed Microservice Microservice A Microservice C Microservice D Microservice B

Slide 45

Slide 45 text

#dfua, @tekgrrl, @googlecloud YAML spec: replicas: 2 selector: type: FE version: 1.0 template: metadata: labels: type: FE version: 1.0 spec: containers: - image: gcr.io/project-id/frontend:1.0 name: frontend ports: - containerPort: 3000 name: http-server Frontend Replication Controller Spec or ‘DesiredState’ for Pods Specifies: ● The number of Pods to maintain ● A template for Containers in the pod ○ Container Image ○ Labels ○ Ports ○ Env Vars ○ Volume Mount Points ● Volume Configurations ● Label Selector for Pod constituency

Slide 46

Slide 46 text

#dfua, @tekgrrl, @googlecloud YAML apiVersion: v1 kind: Service metadata: name: frontend labels: name: frontend spec: type: LoadBalancer ports: - port: 80 targetPort: 3000 protocol: TCP selector: name: frontend Frontend Service Service Definition Defines: ● Port Mappings ● Label Selector for Service constituency ● Whether the Service has an external Load Balancer

Slide 47

Slide 47 text

#dfua, @tekgrrl, @googlecloud Build and tag a Container Images Push Images to Container Repository Create backend Service and ReplicationController config files Create backend ReplicationController Create backend Service Update frontend to use new image Scale as needed Add a Backend and Update Frontend Microservice A Microservice C Microservice D Microservice B

Slide 48

Slide 48 text

#dfua, @tekgrrl, @googlecloud ProtoBuf package geo; service GeoService { rpc DistanceBetween (Points) returns (Distance) {} } message Point { float lat = 1; float lng = 2; } message Points { Point origin = 1; Point destination = 2; } message Distance { float distance = 1; } IDL File (ProtoBuf)

Slide 49

Slide 49 text

JavaScript var grpc = require('grpc'), proto = grpc.load('interface.proto'), GeoService = grpc.buildServer([proto.geo.GeoService.service]); var server = new GeoService({ 'geo.GeoService': { distanceBetween: function(call, callback) { callback(null, getDistance(call.request)); } } }); … server.bind('0.0.0.0:50051'); server.listen(); Backend Code

Slide 50

Slide 50 text

JavaScript var grpc = require('grpc'), proto = grpc.load('interface.proto'), client = new proto.geo.GeoService('backend:50051'); var request = { origin: { lat: 0, lng: 0 }, destination: { lat: 15, lng: 15 } } client.distanceBetween(request, function(error, distance) { // error handling response.end("Distance = " + JSON.stringify(distance) + "\n"); }); Frontend Code

Slide 51

Slide 51 text

YAML Backend Replication Controller YAML spec: replicas: 4 selector: name: backend template: metadata: labels: name: backend spec: containers: - image: gcr.io/project-id/backend:1.0 name: backend ports: - containerPort: 50051 name: grpc-server

Slide 52

Slide 52 text

YAML Backend Service YAML apiVersion: v1 kind: Service metadata: name: backend labels: name: backend spec: ports: - port: 50051 targetPort: 50051 selector: name: backend

Slide 53

Slide 53 text

Demo - Visualization

Slide 54

Slide 54 text

#dfua, @tekgrrl, @googlecloud Node3 Kubelet Proxy Pod Container Container Container Container Pod Container Container Container Container Node3 Kubelet Proxy Pod Container Container Container Container Pod Container Container Container Container Node1 Kubelet Proxy Pod Container Container Pod $ kubectl proxy --www=k8s-visualizer/ Visualizing Kubernetes Master APIs Scheduling REST (pods, services, controllers) AuthN Scheduler Replication Controller Container

Slide 55

Slide 55 text

#dfua, @tekgrrl, @googlecloud State of our Microservices Service Label selectors: version = 1.0 type = Frontend Service name = Frontend Label selector: type = FE Replication Controller version= 1.0 Replication Controller version = v1 #pods = 1 show: version = v2 type = FE Replication Controller version = v1 #pods = show: version = v2 Pod Replication Controller version = 1.0 type = FE #pods = 2 Pod frontend Pod version = 1.0 type = FE Service Label selectors: version = 1.0 type = Frontend Service name = backend Label selector: type = BE Replication Controller Pod Pod frontend Pod version= 1.0 version = 1.0 Replication Controller version = 1.0 type = BE #pods = 2 type = BE type = BE

Slide 56

Slide 56 text

#dfua, @tekgrrl, @googlecloud Service Label selectors: version = 1.0 type = Frontend Service name = frontend Label selector: type = BE Replication Controller version= v1 Replication Controller version = v1 #pods = 1 show: version = v2 type = FE Scaling Example Replication Controller version = v1 #pods = 2 show: version = v2 Pod Replication Controller version = v1 type = FE #pods = 1 show: version = v2

Slide 57

Slide 57 text

#dfua, @tekgrrl, @googlecloud Service Label selectors: version = 1.0 type = Frontend Service name = frontend Label selector: type = BE Replication Controller version= v1 Replication Controller version = v1 #pods = 1 show: version = v2 type = FE Pod Replication Controller version = v1 #pods = 2 show: version = v2 Pod Replication Controller version = v1 type = FE #pods = 2 show: version = v2 version = v1 type = FE Scaling Example

Slide 58

Slide 58 text

#dfua, @tekgrrl, @googlecloud Service Label selectors: version = 1.0 type = Frontend Service name = frontend Label selector: type = BE Replication Controller Pod frontend Pod version= v1 version = v1 Replication Controller version = v1 #pods = 1 show: version = v2 type = FE type = FE Scaling Example Pod frontend Pod version = v1 type = FE Replication Controller version = v1 #pods = 2 show: version = v2 Pod Pod Replication Controller version = v1 type = FE #pods = 4 show: version = v2 version = v1 type = FE

Slide 59

Slide 59 text

#dfua, @tekgrrl, @googlecloud Replication Controller Replication Controller - Name = “backend” - Selector = {“name”: “backend”} - Template = { ... } - NumReplicas = 4 API Server 3 Start 1 more OK 4 How many? How many? Canonical example of control loops Have one job: ensure N copies of a pod ● if too few, start new ones ● if too many, kill some ● group == selector Replicated pods are fungible ● No implied order or identity Replication Controllers

Slide 60

Slide 60 text

#dfua, @tekgrrl, @googlecloud Pod Rolling Update Example Service Label selectors: version = 1.0 type = Frontend Service name = backend Label selector: type = BE Replication Controller Pod version= v1 version = v1 Replication Controller version = v1 type = BE #pods = 2 show: version = v2 type = BE type = BE

Slide 61

Slide 61 text

#dfua, @tekgrrl, @googlecloud Rolling Update Example Service Label selectors: version = 1.0 type = Frontend Service name = backend Label selector: type = BE Replication Controller Pod Pod frontend version= v1 Replication Controller version = v1 type = BE #pods = 2 show: version = v2 type = BE Replication Controller version = v2 type = BE #pods = 2 show: version = v2 Pod version = v2 type = BE Pod version = v1 type = BE

Slide 62

Slide 62 text

Slide 63

Slide 63 text

#dfua, @tekgrrl, @googlecloud Rolling Update Example Service Label selectors: version = 1.0 type = Frontend Service name = backend Label selector: type = BE Replication Controller Pod Pod frontend Pod version= v1 version = v1 Replication Controller version = v1 type = BE #pods = 2 show: version = v2 type = BE type = BE Replication Controller version = v2 type = BE #pods = 2 show: version = v2 Pod version = v2 type = BE version = v2

Slide 64

Slide 64 text

#dfua, @tekgrrl, @googlecloud Rolling Update Example Service Label selectors: version = 1.0 type = Frontend Service name = backend Label selector: type = BE Replication Controller Pod frontend Pod version = v1 Replication Controller version = v1 type = BE #pods = 2 show: version = v2 type = BE Replication Controller version = v2 type = BE #pods = 2 show: version = v2 Pod version = v2 type = BE version = v2

Slide 65

Slide 65 text

#dfua, @tekgrrl, @googlecloud Rolling Update Example Service Label selectors: version = 1.0 type = Frontend Service name = backend Label selector: type = BE Replication Controller Pod frontend Pod version = v1 type = BE Replication Controller version = v2 type = BE #pods = 2 show: version = v2 Pod version = v2 type = BE version = v2

Slide 66

Slide 66 text

#dfua, @tekgrrl, @googlecloud Freedom to pick the language independently for each micro-service, based on performance, library availability, team expertise etc MicroServices using gRPC Loosely coupled development Blocks of functionality can be broken off into separate MicroService. Allows organic growth Multi-language High Performance Make use of the strengths of HTTP/2 and Protocol Buffers

Slide 67

Slide 67 text

#dfua, @tekgrrl, @googlecloud Kubernetes 1.0 as of mid July • Formerly announced at OSCON in July Open sourced in June, 2014 • won the BlackDuck “rookie of the year” award Google launched Google Container Engine (GKE) • hosted Kubernetes • https://cloud.google.com/container-engine/ Roadmap: • https://github.com/GoogleCloudPlatform/kubernetes/milestones Kubernetes Status

Slide 68

Slide 68 text

#dfua, @tekgrrl, @googlecloud Google Container Engine (GA) Managed Kubernetes (Kubernetes v1) Manages Kubernetes master uptime Manages Updates Cluster Resize via Managed Instance Groups Centralised Logging Google Cloud VPN support

Slide 69

Slide 69 text

#dfua, @tekgrrl, @googlecloud Kubernetes is Open Source We want your help! http://kubernetes.io https://github.com/GoogleCloudPlatform/kubernetes irc.freenode.net #google-containers @kubernetesio

Slide 70

Slide 70 text

#dfua, @tekgrrl, @googlecloud Tweet questions to: @tekgrrl Slides: http://bit.ly/1i2PsgE Questions

Slide 71

Slide 71 text

#dfua, @tekgrrl, @googlecloud Service Label selectors: version = 1.0 type = Frontend Service name = backend Label selector: type = BE Replication Controller Pod Pod frontend Pod version= v1 version = v1 Replication Controller version = v1 type = BE #pods = 2 show: version = v2 type = BE type = BE Canary Example Replication Controller Replication Controller version = v2 type = BE #pods = 1 show: version = v2 Pod frontend Pod version = v2 type = BE