Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Envoy API/xDS Deep Dive

Envoy API/xDS Deep Dive

9cbac3a8b2ea3f8f9c79d01aeb050ad3?s=128

Rei Shimizu

June 20, 2020
Tweet

Transcript

  1. Rei Shimizu (@_iy4) 2020-06-20 Envoy API/xDS Deep Dive

  2. 自己紹介 • 早稲田大学大学院 修士1年 • Tetrate.io でソフトウェアエンジニアのアルバイト中 • Envoy /

    Istio の開発を主にやっています。 ◦ 必要に応じて様々な OSSにパッチを投げたりも • Twitter: @_iy4
  3. 目標 • 話すこと ◦ Data Plane API の概略 ◦ Data

    Plane API がどのように Envoy コミュニティで運用されているか ◦ xDS の概略 ◦ xDS のコードを読むときに把握しておくと役に立つこといくつか ◦ 直近の xDS 周りの動向 • 話さないこと ◦ Control Plane の実装方法・Tips ◦ サービスメッシュの運用 ◦ Envoy の設定の詳細事項 Envoy の xDS や API 周りのコードを読むときに頭の片隅においておくとよさそうな事をまとめました。
  4. Envoy Data Plane API

  5. None
  6. • The interface to manage Envoy • We can pass

    the configurations to Envoy as JSON / YAML. • These configurations illustrate as Protocol Buffers in Envoy. • Optimized for machine generation. Introduction to Envoy Data Plane API
  7. None
  8. • We can manage all of API definitions from a

    single interface. • Flexibility to change. • Rich and flexible primitive types. ◦ e.g.: String, google.protobuf.Struct, google.protobuf.Any, etc... • Extensibility with protoc plugins. ◦ e.g.: We can generate documentations as reStructuredText from protobuf definitions on Envoy. • Availability to add metadatas by annotations. Why Protocol Buffers?
  9. Annotation on Protocol Buffers • Protocol Buffers have an ability

    to attach metadata for messages, fields, etc.... • Protoc plugin can understand these metadatas. • We use this ability to automate the changes of Envoy API.
  10. None
  11. • It is implemented as user-friendly interface by using typed

    config. • This is illustrated as google.protobuf.Any in Protocol Buffers • Convert this fuzzy type by referring type_url. API flexibility with Typed Config
  12. • Protocol Buffers have a type level validator. • But,

    it can’t validate user passed contents. ◦ e.g.: Protocol buffers can’t reject an address when the field is considered to input a phone number. • protoc-gen-validate can validate contents. ◦ e.g. It is enabled to validate contents with regexp. • protoc-gen-validate is implemented as protoc plugin. • Currently It supports Go / C++ / Java / Python. https://github.com/envoyproxy/protoc-gen-validate protoc-gen-validate
  13. None
  14. Data Plane API lifecycle • Envoy API has versioning system

    ◦ Currently, envoy community supports V2 and V3 API as stable version. • Breaking changes are not allowed to apply stable version of API. • These breaking changes will be applied to alpha version of API. ◦ Currently, V4alpha is used as candidate version.
  15. Data Plane API lifecycle 2020/1 2021/? V2 V3 V4Alpha V2

    V3 V4 V5Alpha 2020/12 V2 V3 V4Alpha V2 will be supported only a year after changed to previous major version even if the specifications of V4 are not fixed. https://docs.google.com/document/d/1afQ9wthJxofwOMCAle8qF-ckEfNWUuWtxbmdhxji0sI/edit?ts=5ec10fb3#heading=h.n2ql27ddmi4x
  16. The strategy for seamless migration of API • Supports multiple

    versions of API ◦ V2 and V3 are being supported as major version on 1.14.2. • Annotation based API migration ◦ When we apply breaking changes on current version of API, we don’t delete fields but add an annotation which specifies that is deprecated. • To enable updating API version automatically → protoxform
  17. • Protoxform is implemented as protoc plugin. • Major version

    of API as input, next version of API as output. • If we have breaking changes on major version, protoxform outputs the next version of API, with maintaining backward compatibility by using annotation. ◦ e.g. If we have deprecated fields on current version of API, protoxform understand the annotation and destroy the fields. Automation to change API with protoxform Protoxform V3 V4Alpha
  18. Automation to change API with protoxform This annotations are attached

    by protoxform This field will be deleted in alpha version by protoxform Protoxform V2 V3Alpha V3 Protoxform V4Alpha
  19. • API dependency tree is illustrated as DAG. • We

    can also pass the version of envoy explicitly. • If not, Envoy decide valid API version automatically, by using abilities of Protobuf. How to enable co-existing multiple versions of API? V3 Bootstrap V3 Admin V3 Static Resources ... ... V3 Router Filter V2 Hoge Filter
  20. Universal Data Plane API • The objective of UDPA (Universal

    Data Plane API) is to standardize the existing data plane API. • This API covers all of capabilities for data plane such as service discovery, listener configurations, etc… • UDPA-WG (Working Group) is proceeding these movements. https://blog.envoyproxy.io/the-universal-data-plane-api-d15cec7a
  21. xDS

  22. What is xDS? • The ability to update configurations of

    data plane without stopping. • A characteristic functionality on Envoy. • x Discovery Service ◦ LDS (Listener Discovery Service) ◦ CDS (Cluster Discovery Service) ◦ EDS (Endpoint Discovery Service) ◦ SDS (Secret Discovery Service) ◦ RDS (Route Discovery Service) ◦ RTDS (Runtime Discovery Service) ◦ etc...
  23. xDS Protocol • gRPC / REST / File based protocol

    is used for xDS. ◦ Especially, bidirectional gRPC streaming is applied for gRPC approach. ◦ Polling is used for REST. Envoy xDS Server xDS Server xDS Stream xDS Stream Subscription Subscription Envoy xDS Server
  24. Basic xDS Protocol xDS Server xDS stream starts with a

    DiscoveryRequest. ◦ it has a type_url (which distinguishes what type of xDS) ◦ It has resource_names (which specifies the name of configuration.) Envoy DiscoveryRequest
  25. Basic xDS Protocol Envoy xDS Server DiscoveryResponse DiscoveryRequest xDS server

    returns a DiscoveryResponse. ◦ It has a version of configuration. ◦ It has resources (actual configurations) Envoy
  26. Basic xDS Protocol Envoy xDS Server DiscoveryResponse DiscoveryRequest After that,

    Envoy sends second DiscoveryRequest as a response of DiscoveryResponse, to inform whether the updates were succeeded. DiscoveryRequest Envoy
  27. Basic xDS Protocol Envoy xDS Server DiscoveryResponse DiscoveryRequest DiscoveryRequest Envoy

    DiscoveryResponse DiscoveryRequest Update Config
  28. SoTW vs Incremental • SoTW (State of The World) xDS

    ◦ Envoy needs to send all resource names when request. ◦ xDS server needs to send configs for all subscriptions even if that is not changed… • Incremental xDS (Delta xDS) ◦ Another approach for xDS ◦ What is to be less effectively which I described is not needed. ◦ gRPC is only supported. ◦ More performant approach when we manage a lot of resources!
  29. Architecture Overview https://github.com/envoyproxy/envoy/blob/master/source/common/config/xDS_code_diagram.png

  30. Update listener config via gRPC SoTW LDS Listener Manager Grpc

    Subscription Grpc Mux Grpc Subscription LDS API Subscription Watch Grpc Stream API Listener API State API State Watch
  31. Update listener config via gRPC SoTW LDS Listener Manager Grpc

    Subscription Grpc Mux Grpc Subscription LDS API Subscription Watch Grpc Stream API State Grpc Subscription Watch API State API State Watch API Listener API Listener
  32. Update listener config via gRPC SoTW LDS Listener Manager Grpc

    Subscription Grpc Mux Grpc Subscription LDS API Subscription Watch Grpc Stream API Listener API State Grpc Subscription Watch API State API State Watch API State API Listener API Listener
  33. Update listener config via gRPC SoTW LDS Listener Manager Grpc

    Subscription Grpc Mux Grpc Subscription LDS API Subscription Watch Grpc Stream API Listener API State Grpc Subscription Watch API State API State Watch API State Watch API Listener API Listener
  34. Update listener config via gRPC SoTW LDS Listener Manager Grpc

    Subscription Grpc Mux Grpc Subscription LDS API Subscription Watch Grpc Stream API Listener API State Grpc Subscription Watch API State API State Watch API State Watch DiscoveryResponse API Listener API Listener
  35. Update listener config via gRPC SoTW LDS Listener Manager Grpc

    Subscription Grpc Mux Grpc Subscription LDS API Subscription Watch Grpc Stream API Listener API State Grpc Subscription Watch API State API State Watch API State Watch DiscoveryResponse Callback Flow API Listener API Listener
  36. ADS (Aggregated Discovery Service) • In Basic xDS, xDS connection

    must be created per single type_url. ◦ The number of connections can be increased explosively… • ADS brings us to manage multiple type_urls by single xDS connection.
  37. Control Plane for large-scaled Service Mesh • Send DiscoveryResponses to

    managed envoys. ◦ Must maintain all of xDS connections. • Marshal / Unmarshal envoy configurations. • Cache DiscoveryResponses High computational cost may be occured when we must manage many side-car proxies... Service A Service A Service 1 Service A Service A Service N Control Plane Thousands of Envoys
  38. Introduce aggregation layers • To manage thousand of xDS connection,

    we can introduce an aggregation layer. ◦ It will reduce computational costs for Control Plane ◦ DiscoveryResponse which will be shared by some of side-car proxies can be aggregated. • Called xds-relay ◦ It was started developing from 2020 and WIP. Service A Service A Service 1 Service A Service A Service N Control Plane Thousands of Envoys xds-relay N xDS Connections A group to share the same DiscoveryResponse https://github.com/envoyproxy/xds-relay
  39. xds-relay components Orchestrator • Mapper ◦ Generate aggregation key ◦

    Key creation rule can be configured by YAML. ◦ In general, key is generated from node name. • Cache ◦ Like Key-Value data structure. ◦ Manage DIscoveryResponses per aggregated connection. • Orchestrator ◦ DiscoveryResponse handler Mapper Cache DiscoveryRequest Aggregated Key Aggregated Key
  40. Thank you for listening!

  41. Appendix

  42. Manage multiple resources via xDS Resource 1 Resource 2 Resource

    N Resource 1 Subscription 2 Subscription N gRPC Subscription Watch API State gRPC Mux Watch Watch