Tweet
Tweet

Slide 1

Slide 1 text

gRPC xDS Load Balancing Dev Mountain Tech Festival 2022-03-19

Slide 2

Slide 2 text

Contents #connect people to good stuff ● What is gRPC ● Envoy & Universal data plane API ● xDS ● What’s coming next

Slide 3

Slide 3 text

Who is this talk for #connect people to good stuff ● What is gRPC « Dev ● Envoy & Universal data plane API « Ops ● xDS « Ops ● What’s coming next « Ops

Slide 4

Slide 4 text

About Me #connect people to good stuff ● Manatsawin Hanmongkolchai ● Senior Architect @ LINE MAN Wongnai ● We’re hiring! ○ https://careers.lmwn.com

Slide 5

Slide 5 text

gRPC

Slide 6

Slide 6 text

What is gRPC #connect people to good stuff ● It’s API definition language - think Swagger ● But the ecosystem is heavily defined by Google ○ So you get usable client & server codegen in many languages - PHP, Python, Ruby, JavaScript, C#, Java, Go

Slide 7

Slide 7 text

What is gRPC #connect people to good stuff 1 syntax = "proto3"; 2 3 package sms.v2; 4 5 service SMSService { 6 rpc SendSMS(SendSMSRequest) returns (SendSMSResponse); 7 } 8 9 message SendSMSRequest { 10 string phone_no = 1; 11 string message = 2; 12 }

Slide 8

Slide 8 text

What is gRPC #connect people to good stuff func main() { conn, _ := grpc.Dial("sms:3000", …) client := smsv2.NewSMSServiceClient(conn) resp, err := client.SendSMS(context.TODO(), &smsv2.SendSMSRequest{ PhoneNo: "+6620000000", Message: "hello world", }) }

Slide 9

Slide 9 text

Why gRPC #connect people to good stuff 1. You get models generated in many languages 2. Errors are handled as exceptions (no need for raise_for_error()) export interface SendSMSRequest { /** * Phone number in E164 format */ phoneNo: string; message: string; }

Slide 10

Slide 10 text

Why gRPC #connect people to good stuff class SendSMSRequest(Message): phone_no: typing.Text = ... "Phone number in E164 format" message: typing.Text = ... def __init__( self, *, phone_no: typing.Text = ..., message: typing.Text = ..., ) -> None: ... 1. You get models generated in many languages 2. Errors are handled as exceptions (no need for raise_for_error())

Slide 11

Slide 11 text

Why gRPC #connect people to good stuff 3. It simplify team communication

Slide 12

Slide 12 text

Why gRPC #connect people to good stuff 4. It allow schema first development

Slide 13

Slide 13 text

Envoy and the Universal Data Plane API

Slide 14

Slide 14 text

What is Envoy #connect people to good stuff ● It’s reverse proxy originally created by Lyft (now hosted by CNCF) ● Now used by Istio, Consul Connect, Ambassador, Gloo, Contour

Slide 15

Slide 15 text

Why Envoy #connect people to good stuff

Slide 16

Slide 16 text

How do you config Envoy? #connect people to good stuff ● Load config from YAML file ● Load config from gRPC server ○ Config changes can be sent via gRPC stream

Slide 17

Slide 17 text

How do you config Envoy? #connect people to good stuff ● Load config from YAML file ● Load config from gRPC server ○ Config changes can be sent via gRPC stream

Slide 18

Slide 18 text

How do you config Envoy? #connect people to good stuff ● So yes, you can codegen Envoy schema routeConfig := &routev3.RouteConfiguration{ Name: targetHostPortNumber, VirtualHosts: []*routev3.VirtualHost{ { Name: targetHostPort, Domains: []string{fullName, targetHostPort, targetHostPortNumber, svc.Name}, Routes: []*routev3.Route{{ Name: "default", Match: &routev3.RouteMatch{ PathSpecifier: &routev3.RouteMatch_Prefix{}, },

Slide 19

Slide 19 text

xDS #connect people to good stuff ● Envoy config protocol is called xDS ● xDS will be the base of Universal Dataplane Platform - one config format for L4/L7 data plane

Slide 20

Slide 20 text

xDS #connect people to good stuff ● ?DS stands for ○ Cluster Discovery Service (CDS) - nginx upstream object ○ Endpoint Discovery Service (EDS) - nginx upstream items ○ Listener Discovery Service (LDS) - nginx server blocks ○ Route Discovery Service (RDS) - nginx location blocks ○ Aggregated Discovery Service (ADS) ○ etc.

Slide 21

Slide 21 text

Recap #connect people to good stuff ● Envoy is reverse proxy like nginx ● Envoy have configuration API (xDS) ○ Load balancing pool, listeners, routes ● xDS is going to be universal dataplane API

Slide 22

Slide 22 text

gRPC xDS where I no longer care about service mesh

Slide 23

Slide 23 text

Problem with gRPC https://speakerdeck.com/whs/a-date-with-grpc Cool event, but not public

Slide 24

Slide 24 text

Problem with gRPC https://speakerdeck.com/whs/a-date-with-grpc

Slide 25

Slide 25 text

Problem with gRPC https://speakerdeck.com/whs/a-date-with-grpc

Slide 26

Slide 26 text

Problem with gRPC https://speakerdeck.com/whs/a-date-with-grpc

Slide 27

Slide 27 text

Problem with gRPC https://speakerdeck.com/whs/a-date-with-grpc

Slide 28

Slide 28 text

Problem with gRPC https://speakerdeck.com/whs/a-date-with-grpc

Slide 29

Slide 29 text

Problem with gRPC https://speakerdeck.com/whs/a-date-with-grpc

Slide 30

Slide 30 text

2022 Update #connect people to good stuff ● You still could do it… ○ My team did try it ● But it’s deprecated ● And you have to do it per-language

Slide 31

Slide 31 text

Problem with gRPC https://speakerdeck.com/whs/a-date-with-grpc

Slide 32

Slide 32 text

Problem with gRPC https://speakerdeck.com/whs/a-date-with-grpc

Slide 33

Slide 33 text

Problem with gRPC https://speakerdeck.com/whs/a-date-with-grpc

Slide 34

Slide 34 text

2022 Update #connect people to good stuff ● Istio didn’t work out well for us ○ Both Wongnai & LINE MAN team tried, too much tuning works for unclear losses & gains ● Linkerd didn’t work out well for us ○ More operational overhead ○ Insufficient load testing

Slide 35

Slide 35 text

2022 Update #connect people to good stuff ● But, option 3 is coming in 2022

Slide 36

Slide 36 text

xDS is coming to gRPC !! #connect people to good stuff ● gRPC is going to support xDS without sidecar ● Client side load balancing for all languages, fault injection, routing, circuit breaking, mTLS

Slide 37

Slide 37 text

How to add gRPC xDS to your application #connect people to good stuff ● Python 1. Upgrade to grpcio v1.36.0 2. There is no 2

Slide 38

Slide 38 text

How to add gRPC xDS to your application #connect people to good stuff ● Go 1. import _ "google.golang.org/grpc/xds" 2. There is no 2

Slide 39

Slide 39 text

How to add gRPC xDS to your application #connect people to good stuff ● Node (@grpc/js only) 1. import * as grpcJsXds from '@grpc/grpc-js-xds' 2. grpcJsXds.register()

Slide 40

Slide 40 text

Config #connect people to good stuff ● Where does the config comes from?

Slide 41

Slide 41 text

xDS server #connect people to good stuff LMWN’s xDS server for Kubernetes ● https://github.com/wongnai/xds ● We’re running it in production since Feb 2022 ○ www.wongnai.com/cooking is mostly powered by xDS

Slide 42

Slide 42 text

xDS server #connect people to good stuff ● Use 5% CPU & 100MB to support 100 pods ● ~2% application CPU overhead to handle config changes ● Handle only config, not data - not a point of failure

Slide 43

Slide 43 text

xDS server #connect people to good stuff ● Cons: gRPC xDS support is buggy currently ○ C-based (Python), Go is stable ○ Node.js has a race condition ○ Java has another race condition

Slide 44

Slide 44 text

xDS server #connect people to good stuff ● What you still need service mesh: observability ○ But you can already integrate OpenTelemetry for that

Slide 45

Slide 45 text

Future where the mastermind show themself

Slide 46

Slide 46 text

The mastermind #connect people to good stuff ● Why is Google building xDS v2 & gRPC xDS?

Slide 47

Slide 47 text

The mastermind #connect people to good stuff

Slide 48

Slide 48 text

Traffic director #connect people to good stuff

Slide 49

Slide 49 text

Traffic director #connect people to good stuff

Slide 50

Slide 50 text

Google Cloud Load Balancer (Preview) #connect people to good stuff

Slide 51

Slide 51 text

Google Cloud Load Balancer (Preview) #connect people to good stuff

Slide 52

Slide 52 text

My questions to the future #connect people to good stuff ● Once Istio control plane become SaaS, what is the next step for Istio ● What’s the grand scheme for Envoy & xDS on Google Cloud?

Slide 53

Slide 53 text

Q & A Get this slide deck - speakerdeck.com/whs LMWN is hiring! - careers.lmwn.com Slides