Scalable Microservices with gRPC, Kubernetes, and Containers (DevFest Ukraine)

Scalable Microservices with gRPC,
Kubernetes, and Containers

View Slide

#dfua, @tekgrrl, @googlecloud
Mandy Waite
Developer Advocate
+MandyWaite
@tekgrrl

View Slide

Image by Connie
Zhou

View Slide

job hello_world = {
runtime = { cell = 'ic' } // Cell (cluster) to run in
binary = '.../hello_world_webserver' // Program to run
args = { port = '%port%' } // Command line parameters
requirements = { // Resource requirements
ram = 100M
disk = 100M
cpu = 0.1
}
replicas = 5 // Number of tasks
}
10000
Developer View

View Slide

Developer View

View Slide

web browsers
BorgMaster
link shard
UI
shard
BorgMaster
link shard
UI
shard
BorgMaster
link shard
UI
shard
BorgMaster
link shard
UI
shard
Scheduler
borgcfg web browsers
scheduler
Borglet Borglet Borglet Borglet
Config
file
BorgMaster
link shard
UI
shard
persistent store
(Paxos)
Binary
What just
happened?
Cell
Storage
Developer View

View Slide

Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world! Hello
world! Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world! Hello
world!
Hello
world!
Hello
world!
Hello
world!
Image by Connie
Zhou
Hello
world!
Hello
world!
Hello
world! Hello
world!
Hello
world! Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world! Hello
world!
Hello
world! Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world!
Hello
world! Hello
world!
Hello
world! Hello
world!
Hello
world!
Hello
world!

View Slide

Images by Connie
Zhou
Observations:
● Containers make everyone more
productive
● Everything at Google runs in a
Container!
● The Datacenter is one big System
○ On which we schedule Containers
See this video for more on Scheduling at
Google:
https://youtu.be/elHbkoJOBNE?t=4426
http://goo.gl/1C4nuo (Borg paper)

View Slide

Kubernetes

View Slide

Greek for “Helmsman”; also the root of the
word “Governor”
• Orchestrator for Docker containers
• Supports multi-cloud environments
• Inspired and informed by Google’s
experiences and internal systems
• Open source, written in Go
Manage applications, not machines
Kubernetes

View Slide

web browsers
y
Kubelet Kubelet Kubelet Kubelet
Kubernetes Master
Replication
Controller
Scheduler
API Server
Kube-UI
Container
Registry
kubectl
Proxy

web browsers
Developer View (Kubernetes)

View Slide

The atom of scheduling for containers
Represents an application specific
logical host
Hosts containers and volumes
Each has its own routable (no NAT) IP
address
Ephemeral
• Pods are functionally identical and therefore
ephemeral and replaceable
Pod
Web Server
Volume
Consumers
Pods

View Slide

Can be used to group multiple
containers & shared volumes
Containers within a pod are tightly
coupled
Shared namespaces
• Containers in a pod share a network IP
and port namespace
•
Pods
Pod
Git
Synchronizer
Node.js App
Container
Volume
Consumers
Github

View Slide

Pod Networking (across nodes)
Pods have IPs which are routable
Pods can reach each other without NAT
● Even across nodes
No Brokering of Port Numbers
These are fundamental requirements
Many solutions
● Flannel, Weave, Cloud Provider
10.1.2.0/24
10.1.1.0/24
10.1.1.211 10.1.1.2
10.1.2.106
10.1.3.0/24
10.1.3.45
10.1.3.17
10.1.3.0/24

View Slide

Dashboard
show: type = FE
Pod
Pod
frontend
Pod
frontend
Pod
type = FE
version = v2
type = FE
● Metadata with semantic meaning
● Membership identifier
● The only Grouping Mechanism
Behavior Benefits
➔ Allow for intent of many users (e.g. dashboards)
➔ Build higher level systems …
➔ Queryable by Selectors
Labels ← These are important

View Slide

Dashboard
show: type = FE
Pod
Pod
frontend
Pod
frontend
Pod Pod
Dashboard
show: version = v2
type = FE
version = v2
type = FE version = v2
● Metadata with semantic meaning
● Membership identifier
● The only Grouping Mechanism
Behavior Benefits
➔ Allow for intent of many users (e.g. dashboards)
➔ Build higher level systems …
➔ Queryable by Selectors
Labels ← These are important

View Slide

Replication
Controller
Pod
Pod
frontend
Pod
version= v1 version = v1
Replication
Controller
#pods = 2
version = v1
show: version = v2 Behavior Benefits
● Keeps Pods running
● Gives direct control of Pod #s
● Grouped by Label Selector
➔ Recreates Pods, maintains desired state
➔ Fine-grained control for scaling
➔ Standard grouping semantics
Replication Controllers

View Slide

Replication
Controller
Pod
Pod
frontend
Pod
frontend
Pod Pod
Replication
Controller
#pods = 1
version = v2
show: version = v2
version= v1 version = v1 version = v2
Replication
Controller
#pods = 2
version = v1
show: version = v2 Behavior Benefits
● Keeps Pods running
● Gives direct control of Pod #s
● Grouped by Label Selector
➔ Recreates Pods, maintains desired state
➔ Fine-grained control for scaling
➔ Standard grouping semantics

View Slide

Portal (VIP)
Client
Pod
Container
Pod
Container
Pod
Container
Container
A logical grouping of pods that perform the
same function
• grouped by label selector
Load balances incoming requests across
constituent pods
Choice of pod is random but supports
session affinity (ClientIP)
Gets a stable virtual IP and port
• also a DNS name
Services

View Slide

Developer View (Replication Controller)
spec:
containers:
- name: php-guestbook
image: php-guestbook:containercamp
resources:
limits:
memory: "128Mi"
cpu: "500m"
ports:
- containerPort: 80
protocol: TCP
replicas: 5

View Slide

Developer View (Replication Controller)
spec:
containers:
- name: php-guestbook
image: php-guestbook:containercamp
resources:
limits:
memory: "128Mi"
cpu: "500m"
ports:
- containerPort: 80
protocol: TCP
replicas: 1
10000

View Slide

Kubernetes
Cluster
Applications
Ops Dev
App A
App B
App C
App A

View Slide

HTTP/2, ProtoBufs and gRPC

View Slide

● Improve end-user perceived latency
● Address the "head of line blocking"
● Not require multiple connections
● Retain the semantics of HTTP/1.1
"HTTP/2 is a protocol designed for low-
latency transport of content
over the World Wide Web"

View Slide

HTTP/2
HTTP/1.1
http://www.http2demo.io/

View Slide

● One TCP connection
● Request → Stream
○ Streams are multiplexed
○ Streams are prioritized
● Binary framing layer
○ Prioritization
○ Flow control
○ Server push
● Header compression
HTTP/2 in one slide…

View Slide

What are Protocol Buffers?
Structured representation of data
Google's lingua franca for data
● 48k+ Message Types
● 12k+ Proto files
Evolutionary Development
Incrementally solved problems, Now
used for:
● RPC Systems
● Persistent Data Storage

View Slide

Protocol buffers:
● Efficient and compact binary data
representation
● Clear compatibility rules; can
easily be extended over time
● Generates idiomatic, easy to use
classes for many languages
● Strongly typed; less error prone
Why Protocol Buffers?
Why?

View Slide

Protocol Buffers language version 3
Specified by syntax = “proto3”;
proto2 continues to be supported
All fields are optional in proto3
No user specified default values
No groups (FYI for those that use them)
Message Format (proto3) syntax = “proto3”;
message Person {
string name = 1;
int32 id = 2;
string email = 3;
enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber {
string number = 1;
PhoneType type = 2;
}
repeated PhoneNumber phone = 4;
}

View Slide

Add new fields without breaking
backwards-compatibility
old implementations ignore the new
fields when parsing
In proto3 any field can be removed, but
don’t renumber existing fields
Extensible
syntax = “proto3”;
message Person {
string name = 1;
int32 id = 2;
string email = 3;
address addr = 4;
message address {
string firstLine = 1;
string secondLine = 2;
string postalCode = 3;
string country = 4;
}
...
}

View Slide

https://github.com/google/protobuf
Protocol Buffer Compiler

View Slide

gRPC goals
Enable developers to
build micro-
service-based
applications
Build an open source, standards-based, best-of-breed, feature-rich RPC system
Create easy-to-use,
efficient and idiomatic
libraries
micro-services
performant and scalable
efficient and idiomatic
Provide a performant
and scalable RPC
framework

View Slide

“The biggest issue in changing a
monolith into microservices lies in
changing the communication
pattern.”
- Martin Fowler

View Slide

IDL to describe an API
Automatically generated servers and
clients in 10+ languages
Takes advantage of feature set of
HTTP/2
Lightweight open connections
Point to point
Streaming! Bidirectional streaming!
gRPC in a nutshell

View Slide

Getting Started
Define a service in a .proto file using
Protocol Buffers IDL
Generate server and client code using
the protocol buffer compiler
Use the gRPC API to write a simple
client and server for your service in the
languages of your choice

View Slide

gRPC Language Support
Implementations
● C core
○ Native bindings in C++, Node.js, Python, Ruby, ObjC,
PHP, C#
● Java using Netty or OkHttp (+ inProcess for testing)
● Go

View Slide

Microservices

View Slide

JavaScript
var http = require('http');
var server = http.createServer(function (request, response) {
response.writeHead(200, {"Content-Type": "text/plain"});
response.end("Hello World\n");
});
server.listen(3000);
A Hello World Microservice

View Slide

Build and tag a Container Image
Push Image to Container Repository
Create Service and
ReplicationController config files
Create ReplicationController
Create Service
From Code to deployed Microservice
Microservice A
Microservice C
Microservice D
Microservice B

View Slide

YAML
spec:
replicas: 2
selector:
type: FE
version: 1.0
template:
metadata:
labels:
type: FE
version: 1.0
spec:
containers:
- image: gcr.io/project-id/frontend:1.0
name: frontend
ports:
- containerPort: 3000
name: http-server
Frontend Replication Controller
Spec or ‘DesiredState’ for Pods
Specifies:
● The number of Pods to maintain
● A template for Containers in the pod
○ Container Image
○ Labels
○ Ports
○ Env Vars
○ Volume Mount Points
● Volume Configurations
● Label Selector for Pod constituency

View Slide

YAML
apiVersion: v1
kind: Service
metadata:
name: frontend
labels:
name: frontend
spec:
type: LoadBalancer
ports:
- port: 80
targetPort: 3000
protocol: TCP
selector:
name: frontend
Frontend Service
Service Definition
Defines:
● Port Mappings
● Label Selector for Service constituency
● Whether the Service has an external
Load Balancer

View Slide

Build and tag a Container Images
Push Images to Container Repository
Create backend Service and
ReplicationController config files
Create backend ReplicationController
Create backend Service
Update frontend to use new image
Scale as needed
Add a Backend and Update Frontend
Microservice A
Microservice C
Microservice D
Microservice B

View Slide

ProtoBuf
package geo;
service GeoService {
rpc DistanceBetween (Points) returns (Distance) {}
}
message Point {
float lat = 1;
float lng = 2;
}
message Points {
Point origin = 1;
Point destination = 2;
}
message Distance {
float distance = 1;
}
IDL File (ProtoBuf)

View Slide

JavaScript
var grpc = require('grpc'),
proto = grpc.load('interface.proto'),
GeoService = grpc.buildServer([proto.geo.GeoService.service]);
var server = new GeoService({
'geo.GeoService': {
distanceBetween: function(call, callback) {
callback(null, getDistance(call.request));
}
}
});
…
server.bind('0.0.0.0:50051');
server.listen();
Backend Code

View Slide

JavaScript
var grpc = require('grpc'),
proto = grpc.load('interface.proto'),
client = new proto.geo.GeoService('backend:50051');
var request = {
origin: { lat: 0, lng: 0 },
destination: { lat: 15, lng: 15 }
}
client.distanceBetween(request, function(error, distance) {
// error handling
response.end("Distance = " + JSON.stringify(distance) + "\n");
});
Frontend Code

View Slide

YAML
Backend Replication Controller
YAML
spec:
replicas: 4
selector:
name: backend
template:
metadata:
labels:
name: backend
spec:
containers:
- image: gcr.io/project-id/backend:1.0
name: backend
ports:
- containerPort: 50051
name: grpc-server

View Slide

YAML
Backend Service
YAML
apiVersion: v1
kind: Service
metadata:
name: backend
labels:
name: backend
spec:
ports:
- port: 50051
targetPort: 50051
selector:
name: backend

View Slide

Demo - Visualization

View Slide

Node3
Kubelet Proxy
Pod
Container
Container
Container
Container
Pod
Container
Container
Container
Container
Node3
Kubelet Proxy
Pod
Container
Container
Container
Container
Pod
Container
Container
Container
Container
Node1
Kubelet Proxy
Pod
Container
Container
Pod
$ kubectl proxy --www=k8s-visualizer/
Visualizing Kubernetes
Master
APIs
Scheduling
REST
(pods, services,
controllers)
AuthN
Scheduler
Replication
Controller
Container

View Slide

State of our Microservices
Service
Label
selectors:
version = 1.0
type =
Frontend
Service
name = Frontend
Label selector:
type = FE
Replication
Controller
version= 1.0
Replication
Controller
version = v1
#pods = 1
show: version =
v2
type = FE
Replication
Controller
version = v1
#pods =
show: version =
v2
Pod
Replication
Controller
version = 1.0
type = FE
#pods = 2
Pod
frontend
Pod
version = 1.0
type = FE
Service
Label
selectors:
version = 1.0
type =
Frontend
Service
name = backend
Label selector:
type = BE
Replication
Controller Pod Pod
frontend
Pod
version= 1.0 version = 1.0
Replication
Controller
version = 1.0
type = BE
#pods = 2
type = BE type = BE

View Slide

Service
Label selectors:
version = 1.0
type = Frontend
Service
name = frontend
Label selector:
type = BE
Replication
Controller
version= v1
Replication
Controller
version = v1
#pods = 1
show: version = v2
type = FE
Scaling Example
Replication
Controller
version = v1
#pods = 2
show: version = v2
Pod
Replication
Controller
version = v1
type = FE
#pods = 1
show: version = v2

View Slide

Service
Label selectors:
version = 1.0
type = Frontend
Service
name = frontend
Label selector:
type = BE
Replication
Controller
version= v1
Replication
Controller
version = v1
#pods = 1
show: version = v2
type = FE
Pod
Replication
Controller
version = v1
#pods = 2
show: version = v2
Pod
Replication
Controller
version = v1
type = FE
#pods = 2
show: version = v2
version = v1
type = FE
Scaling Example

View Slide

Service
Label selectors:
version = 1.0
type = Frontend
Service
name = frontend
Label selector:
type = BE
Replication
Controller Pod
frontend
Pod
Replication
Controller
version = v1
#pods = 1
show: version = v2
type = FE type = FE
Scaling Example
Pod
frontend
Pod
version = v1
type = FE
Replication
Controller
version = v1
#pods = 2
show: version = v2
Pod Pod
Replication
Controller
version = v1
type = FE
#pods = 4
show: version = v2
version = v1
type = FE

View Slide

Replication Controller
Replication Controller
- Name = “backend”
- Selector = {“name”: “backend”}
- Template = { ... }
- NumReplicas = 4
API Server
3
Start 1
more
OK 4
How
many?
How
many?
Canonical example of control loops
Have one job: ensure N copies of a pod
● if too few, start new ones
● if too many, kill some
● group == selector
Replicated pods are fungible
● No implied order or identity

View Slide

Pod
Rolling Update Example
Service
Label selectors:
version = 1.0
type = Frontend
Service
name = backend
Label selector:
type = BE
Replication
Controller
Pod
Replication
Controller
version = v1
type = BE
#pods = 2
show: version = v2
type = BE type = BE

View Slide

Service
Label selectors:
version = 1.0
type = Frontend
Service
name = backend
Label selector:
type = BE
Replication
Controller
Pod
Pod
frontend
version= v1
Replication
Controller
version = v1
type = BE
#pods = 2
show: version = v2
type = BE
Replication
Controller
version = v2
type = BE
#pods = 2
show: version = v2
Pod
version = v2
type = BE
Pod
version = v1
type = BE

View Slide

Service
Label selectors:
version = 1.0
type = Frontend
Service
name = backend
Label selector:
type = BE
Replication
Controller
Pod
Pod
frontend
version= v1
Replication
Controller
version = v1
type = BE
#pods = 2
show: version = v2
type = BE
Replication
Controller
version = v2
type = BE
#pods = 2
show: version = v2
Pod
version = v2
type = BE

View Slide

Service
Label selectors:
version = 1.0
type = Frontend
Service
name = backend
Label selector:
type = BE
Replication
Controller
Pod
Pod
frontend
Pod
Replication
Controller
version = v1
type = BE
#pods = 2
show: version = v2
type = BE type = BE
Replication
Controller
version = v2
type = BE
#pods = 2
show: version = v2
Pod
version = v2
type = BE
version = v2

View Slide

Service
Label selectors:
version = 1.0
type = Frontend
Service
name = backend
Label selector:
type = BE
Replication
Controller Pod
frontend
Pod
version = v1
Replication
Controller
version = v1
type = BE
#pods = 2
show: version = v2
type = BE
Replication
Controller
version = v2
type = BE
#pods = 2
show: version = v2
Pod
version = v2
type = BE
version = v2

View Slide

Service
Label selectors:
version = 1.0
type = Frontend
Service
name = backend
Label selector:
type = BE
Replication
Controller Pod
frontend
Pod
version = v1
type = BE
Replication
Controller
version = v2
type = BE
#pods = 2
show: version = v2
Pod
version = v2
type = BE
version = v2

View Slide

Freedom to pick the language independently for each micro-service, based on
performance, library availability, team expertise etc
MicroServices using gRPC
Loosely coupled development
Blocks of functionality can be broken off into separate MicroService. Allows
organic growth
Multi-language
High Performance
Make use of the strengths of HTTP/2 and Protocol Buffers

View Slide

Kubernetes 1.0 as of mid July
• Formerly announced at OSCON in July
Open sourced in June, 2014
• won the BlackDuck “rookie of the year” award
Google launched Google Container Engine (GKE)
• hosted Kubernetes
• https://cloud.google.com/container-engine/
Roadmap:
• https://github.com/GoogleCloudPlatform/kubernetes/milestones
Kubernetes Status

View Slide

Google Container Engine (GA)
Managed Kubernetes (Kubernetes v1)
Manages Kubernetes master uptime
Manages Updates
Cluster Resize via Managed Instance
Groups
Centralised Logging
Google Cloud VPN support

View Slide

Kubernetes is Open Source
We want your help!
http://kubernetes.io
https://github.com/GoogleCloudPlatform/kubernetes
irc.freenode.net #google-containers
@kubernetesio

View Slide

Tweet questions to:
@tekgrrl
Slides: http://bit.ly/1i2PsgE
Questions

View Slide

Service
Label selectors:
version = 1.0
type = Frontend
Service
name = backend
Label selector:
type = BE
Replication
Controller
Pod
Pod
frontend
Pod
Replication
Controller
version = v1
type = BE
#pods = 2
show: version = v2
type = BE type = BE
Canary Example
Replication
Controller
Replication
Controller
version = v2
type = BE
#pods = 1
show: version = v2
Pod
frontend
Pod
version = v2
type = BE

View Slide

Scalable Microservices with gRPC, Kubernetes, and Containers (DevFest Ukraine)

Scalable Microservices with gRPC, Kubernetes, and Containers (DevFest Ukraine)

GoogleCloudPlatform

More Decks by GoogleCloudPlatform

Other Decks in Technology

Featured

Transcript