Envoy API/xDS Deep Dive

Transcript

1. Rei Shimizu (@_iy4) 2020-06-20 Envoy API/xDS Deep Dive
2. 自己紹介 • 早稲田大学大学院 修士1年 • Tetrate.io でソフトウェアエンジニアのアルバイト中 • Envoy / Istio の開発を主にやっています。 ◦ 必要に応じて様々な OSSにパッチを投げたりも • Twitter: @_iy4
3. 目標 • 話すこと ◦ Data Plane API の概略 ◦ Data Plane API がどのように Envoy コミュニティで運用されているか ◦ xDS の概略 ◦ xDS のコードを読むときに把握しておくと役に立つこといくつか ◦ 直近の xDS 周りの動向 • 話さないこと ◦ Control Plane の実装方法・Tips ◦ サービスメッシュの運用 ◦ Envoy の設定の詳細事項 Envoy の xDS や API 周りのコードを読むときに頭の片隅においておくとよさそうな事をまとめました。
4. Envoy Data Plane API
6. • The interface to manage Envoy • We can pass the configurations to Envoy as JSON / YAML. • These configurations illustrate as Protocol Buffers in Envoy. • Optimized for machine generation. Introduction to Envoy Data Plane API
8. • We can manage all of API definitions from a single interface. • Flexibility to change. • Rich and flexible primitive types. ◦ e.g.: String, google.protobuf.Struct, google.protobuf.Any, etc... • Extensibility with protoc plugins. ◦ e.g.: We can generate documentations as reStructuredText from protobuf definitions on Envoy. • Availability to add metadatas by annotations. Why Protocol Buffers?
9. Annotation on Protocol Buffers • Protocol Buffers have an ability to attach metadata for messages, fields, etc.... • Protoc plugin can understand these metadatas. • We use this ability to automate the changes of Envoy API.
11. • It is implemented as user-friendly interface by using typed config. • This is illustrated as google.protobuf.Any in Protocol Buffers • Convert this fuzzy type by referring type_url. API flexibility with Typed Config
12. • Protocol Buffers have a type level validator. • But, it can’t validate user passed contents. ◦ e.g.: Protocol buffers can’t reject an address when the field is considered to input a phone number. • protoc-gen-validate can validate contents. ◦ e.g. It is enabled to validate contents with regexp. • protoc-gen-validate is implemented as protoc plugin. • Currently It supports Go / C++ / Java / Python. https://github.com/envoyproxy/protoc-gen-validate protoc-gen-validate
14. Data Plane API lifecycle • Envoy API has versioning system ◦ Currently, envoy community supports V2 and V3 API as stable version. • Breaking changes are not allowed to apply stable version of API. • These breaking changes will be applied to alpha version of API. ◦ Currently, V4alpha is used as candidate version.
15. Data Plane API lifecycle 2020/1 2021/? V2 V3 V4Alpha V2 V3 V4 V5Alpha 2020/12 V2 V3 V4Alpha V2 will be supported only a year after changed to previous major version even if the specifications of V4 are not fixed. https://docs.google.com/document/d/1afQ9wthJxofwOMCAle8qF-ckEfNWUuWtxbmdhxji0sI/edit?ts=5ec10fb3#heading=h.n2ql27ddmi4x
16. The strategy for seamless migration of API • Supports multiple versions of API ◦ V2 and V3 are being supported as major version on 1.14.2. • Annotation based API migration ◦ When we apply breaking changes on current version of API, we don’t delete fields but add an annotation which specifies that is deprecated. • To enable updating API version automatically → protoxform
17. • Protoxform is implemented as protoc plugin. • Major version of API as input, next version of API as output. • If we have breaking changes on major version, protoxform outputs the next version of API, with maintaining backward compatibility by using annotation. ◦ e.g. If we have deprecated fields on current version of API, protoxform understand the annotation and destroy the fields. Automation to change API with protoxform Protoxform V3 V4Alpha
18. Automation to change API with protoxform This annotations are attached by protoxform This field will be deleted in alpha version by protoxform Protoxform V2 V3Alpha V3 Protoxform V4Alpha
19. • API dependency tree is illustrated as DAG. • We can also pass the version of envoy explicitly. • If not, Envoy decide valid API version automatically, by using abilities of Protobuf. How to enable co-existing multiple versions of API? V3 Bootstrap V3 Admin V3 Static Resources ... ... V3 Router Filter V2 Hoge Filter
20. Universal Data Plane API • The objective of UDPA (Universal Data Plane API) is to standardize the existing data plane API. • This API covers all of capabilities for data plane such as service discovery, listener configurations, etc… • UDPA-WG (Working Group) is proceeding these movements. https://blog.envoyproxy.io/the-universal-data-plane-api-d15cec7a
21. xDS
22. What is xDS? • The ability to update configurations of data plane without stopping. • A characteristic functionality on Envoy. • x Discovery Service ◦ LDS (Listener Discovery Service) ◦ CDS (Cluster Discovery Service) ◦ EDS (Endpoint Discovery Service) ◦ SDS (Secret Discovery Service) ◦ RDS (Route Discovery Service) ◦ RTDS (Runtime Discovery Service) ◦ etc...
23. xDS Protocol • gRPC / REST / File based protocol is used for xDS. ◦ Especially, bidirectional gRPC streaming is applied for gRPC approach. ◦ Polling is used for REST. Envoy xDS Server xDS Server xDS Stream xDS Stream Subscription Subscription Envoy xDS Server
24. Basic xDS Protocol xDS Server xDS stream starts with a DiscoveryRequest. ◦ it has a type_url (which distinguishes what type of xDS) ◦ It has resource_names (which specifies the name of configuration.) Envoy DiscoveryRequest
25. Basic xDS Protocol Envoy xDS Server DiscoveryResponse DiscoveryRequest xDS server returns a DiscoveryResponse. ◦ It has a version of configuration. ◦ It has resources (actual configurations) Envoy
26. Basic xDS Protocol Envoy xDS Server DiscoveryResponse DiscoveryRequest After that, Envoy sends second DiscoveryRequest as a response of DiscoveryResponse, to inform whether the updates were succeeded. DiscoveryRequest Envoy
27. Basic xDS Protocol Envoy xDS Server DiscoveryResponse DiscoveryRequest DiscoveryRequest Envoy DiscoveryResponse DiscoveryRequest Update Config
28. SoTW vs Incremental • SoTW (State of The World) xDS ◦ Envoy needs to send all resource names when request. ◦ xDS server needs to send configs for all subscriptions even if that is not changed… • Incremental xDS (Delta xDS) ◦ Another approach for xDS ◦ What is to be less effectively which I described is not needed. ◦ gRPC is only supported. ◦ More performant approach when we manage a lot of resources!
29. Architecture Overview https://github.com/envoyproxy/envoy/blob/master/source/common/config/xDS_code_diagram.png
30. Update listener config via gRPC SoTW LDS Listener Manager Grpc Subscription Grpc Mux Grpc Subscription LDS API Subscription Watch Grpc Stream API Listener API State API State Watch
31. Update listener config via gRPC SoTW LDS Listener Manager Grpc Subscription Grpc Mux Grpc Subscription LDS API Subscription Watch Grpc Stream API State Grpc Subscription Watch API State API State Watch API Listener API Listener
32. Update listener config via gRPC SoTW LDS Listener Manager Grpc Subscription Grpc Mux Grpc Subscription LDS API Subscription Watch Grpc Stream API Listener API State Grpc Subscription Watch API State API State Watch API State API Listener API Listener
33. Update listener config via gRPC SoTW LDS Listener Manager Grpc Subscription Grpc Mux Grpc Subscription LDS API Subscription Watch Grpc Stream API Listener API State Grpc Subscription Watch API State API State Watch API State Watch API Listener API Listener
34. Update listener config via gRPC SoTW LDS Listener Manager Grpc Subscription Grpc Mux Grpc Subscription LDS API Subscription Watch Grpc Stream API Listener API State Grpc Subscription Watch API State API State Watch API State Watch DiscoveryResponse API Listener API Listener
35. Update listener config via gRPC SoTW LDS Listener Manager Grpc Subscription Grpc Mux Grpc Subscription LDS API Subscription Watch Grpc Stream API Listener API State Grpc Subscription Watch API State API State Watch API State Watch DiscoveryResponse Callback Flow API Listener API Listener
36. ADS (Aggregated Discovery Service) • In Basic xDS, xDS connection must be created per single type_url. ◦ The number of connections can be increased explosively… • ADS brings us to manage multiple type_urls by single xDS connection.
37. Control Plane for large-scaled Service Mesh • Send DiscoveryResponses to managed envoys. ◦ Must maintain all of xDS connections. • Marshal / Unmarshal envoy configurations. • Cache DiscoveryResponses High computational cost may be occured when we must manage many side-car proxies... Service A Service A Service 1 Service A Service A Service N Control Plane Thousands of Envoys
38. Introduce aggregation layers • To manage thousand of xDS connection, we can introduce an aggregation layer. ◦ It will reduce computational costs for Control Plane ◦ DiscoveryResponse which will be shared by some of side-car proxies can be aggregated. • Called xds-relay ◦ It was started developing from 2020 and WIP. Service A Service A Service 1 Service A Service A Service N Control Plane Thousands of Envoys xds-relay N xDS Connections A group to share the same DiscoveryResponse https://github.com/envoyproxy/xds-relay
39. xds-relay components Orchestrator • Mapper ◦ Generate aggregation key ◦ Key creation rule can be configured by YAML. ◦ In general, key is generated from node name. • Cache ◦ Like Key-Value data structure. ◦ Manage DIscoveryResponses per aggregated connection. • Orchestrator ◦ DiscoveryResponse handler Mapper Cache DiscoveryRequest Aggregated Key Aggregated Key
40. Thank you for listening!
41. Appendix
42. Manage multiple resources via xDS Resource 1 Resource 2 Resource N Resource 1 Subscription 2 Subscription N gRPC Subscription Watch API State gRPC Mux Watch Watch