Envoy API/xDS Deep Dive

Transcript

1. Rei Shimizu (@_iy4)
   2020-06-20
   Envoy API/xDS Deep Dive
   

   View Slide

2. 自己紹介
   ● 早稲田大学大学院 修士1年
   ● Tetrate.io でソフトウェアエンジニアのアルバイト中
   ● Envoy / Istio の開発を主にやっています。
   ○ 必要に応じて様々な OSSにパッチを投げたりも
   ● Twitter: @_iy4
   

   View Slide

3. 目標
   ● 話すこと
   ○ Data Plane API の概略
   ○ Data Plane API がどのように Envoy コミュニティで運用されているか
   ○ xDS の概略
   ○ xDS のコードを読むときに把握しておくと役に立つこといくつか
   ○ 直近の xDS 周りの動向
   ● 話さないこと
   ○ Control Plane の実装方法・Tips
   ○ サービスメッシュの運用
   ○ Envoy の設定の詳細事項
   Envoy の xDS や API 周りのコードを読むときに頭の片隅においておくとよさそうな事をまとめました。
   

   View Slide

4. Envoy Data Plane API
   

   View Slide

5. View Slide

6. ● The interface to manage Envoy
   ● We can pass the configurations to Envoy as JSON / YAML.
   ● These configurations illustrate as Protocol Buffers in Envoy.
   ● Optimized for machine generation.
   Introduction to Envoy Data Plane API
   

   View Slide

7. View Slide

8. ● We can manage all of API definitions from a single interface.
   ● Flexibility to change.
   ● Rich and flexible primitive types.
   ○ e.g.: String, google.protobuf.Struct, google.protobuf.Any, etc...
   ● Extensibility with protoc plugins.
   ○ e.g.: We can generate documentations as reStructuredText from protobuf definitions on Envoy.
   ● Availability to add metadatas by annotations.
   Why Protocol Buffers?
   

   View Slide

9. Annotation on Protocol Buffers
   ● Protocol Buffers have an ability to attach metadata for messages, fields, etc....
   ● Protoc plugin can understand these metadatas.
   ● We use this ability to automate the changes of Envoy API.
   

   View Slide

10. View Slide

11. ● It is implemented as user-friendly interface by using typed config.
    ● This is illustrated as google.protobuf.Any in Protocol Buffers
    ● Convert this fuzzy type by referring type_url.
    API flexibility with Typed Config
    

    View Slide

12. ● Protocol Buffers have a type level validator.
    ● But, it can’t validate user passed contents.
    ○ e.g.: Protocol buffers can’t reject an address when the field is considered to input a phone
    number.
    ● protoc-gen-validate can validate contents.
    ○ e.g. It is enabled to validate contents with regexp.
    ● protoc-gen-validate is implemented as protoc plugin.
    ● Currently It supports Go / C++ / Java / Python.
    https://github.com/envoyproxy/protoc-gen-validate
    protoc-gen-validate
    

    View Slide

13. View Slide

14. Data Plane API lifecycle
    ● Envoy API has versioning system
    ○ Currently, envoy community supports V2 and V3 API as stable version.
    ● Breaking changes are not allowed to apply stable version of API.
    ● These breaking changes will be applied to alpha version of API.
    ○ Currently, V4alpha is used as candidate version.
    

    View Slide

15. Data Plane API lifecycle
    2020/1
    2021/?
    V2
    V3
    V4Alpha
    V2
    V3
    V4
    V5Alpha
    2020/12
    V2
    V3
    V4Alpha
    V2 will be supported only a year after changed to previous
    major version even if the specifications of V4 are not fixed.
    https://docs.google.com/document/d/1afQ9wthJxofwOMCAle8qF-ckEfNWUuWtxbmdhxji0sI/edit?ts=5ec10fb3#heading=h.n2ql27ddmi4x
    

    View Slide

16. The strategy for seamless migration of API
    ● Supports multiple versions of API
    ○ V2 and V3 are being supported as major version on 1.14.2.
    ● Annotation based API migration
    ○ When we apply breaking changes on current version of API, we don’t delete fields but add an
    annotation which specifies that is deprecated.
    ● To enable updating API version automatically → protoxform
    

    View Slide

17. ● Protoxform is implemented as protoc plugin.
    ● Major version of API as input, next version of API as
    output.
    ● If we have breaking changes on major version, protoxform
    outputs the next version of API, with maintaining backward
    compatibility by using annotation.
    ○ e.g. If we have deprecated fields on current version of API,
    protoxform understand the annotation and destroy the fields.
    Automation to change API with protoxform
    Protoxform
    V3
    V4Alpha
    

    View Slide

18. Automation to change API with protoxform
    This annotations are attached by protoxform
    This field will be deleted in alpha version by protoxform
    Protoxform
    V2 V3Alpha
    V3 Protoxform V4Alpha
    

    View Slide

19. ● API dependency tree is illustrated as DAG.
    ● We can also pass the version of envoy
    explicitly.
    ● If not, Envoy decide valid API version
    automatically, by using abilities of Protobuf.
    How to enable co-existing multiple versions of API?
    V3 Bootstrap
    V3 Admin
    V3 Static
    Resources
    ...
    ...
    V3 Router
    Filter
    V2 Hoge Filter
    

    View Slide

20. Universal Data Plane API
    ● The objective of UDPA (Universal Data Plane API) is to standardize the
    existing data plane API.
    ● This API covers all of capabilities for data plane such as service discovery,
    listener configurations, etc…
    ● UDPA-WG (Working Group) is proceeding these movements.
    https://blog.envoyproxy.io/the-universal-data-plane-api-d15cec7a
    

    View Slide

21. xDS
    

    View Slide

22. What is xDS?
    ● The ability to update configurations of data plane without stopping.
    ● A characteristic functionality on Envoy.
    ● x Discovery Service
    ○ LDS (Listener Discovery Service)
    ○ CDS (Cluster Discovery Service)
    ○ EDS (Endpoint Discovery Service)
    ○ SDS (Secret Discovery Service)
    ○ RDS (Route Discovery Service)
    ○ RTDS (Runtime Discovery Service)
    ○ etc...
    

    View Slide

23. xDS Protocol
    ● gRPC / REST / File based protocol is used for xDS.
    ○ Especially, bidirectional gRPC streaming is applied for gRPC approach.
    ○ Polling is used for REST.
    Envoy
    xDS
    Server
    xDS
    Server
    xDS Stream
    xDS Stream
    Subscription
    Subscription
    Envoy
    xDS
    Server
    

    View Slide

24. Basic xDS Protocol
    xDS
    Server
    xDS stream starts with a DiscoveryRequest.
    ○ it has a type_url (which distinguishes what type of xDS)
    ○ It has resource_names (which specifies the name of
    configuration.)
    Envoy
    DiscoveryRequest
    

    View Slide

25. Basic xDS Protocol
    Envoy xDS
    Server
    DiscoveryResponse
    DiscoveryRequest
    xDS server returns a DiscoveryResponse.
    ○ It has a version of configuration.
    ○ It has resources (actual configurations)
    Envoy
    

    View Slide

26. Basic xDS Protocol
    Envoy xDS
    Server
    DiscoveryResponse
    DiscoveryRequest
    After that, Envoy sends second DiscoveryRequest as a
    response of DiscoveryResponse, to inform whether the updates
    were succeeded.
    DiscoveryRequest
    Envoy
    

    View Slide

27. Basic xDS Protocol
    Envoy xDS
    Server
    DiscoveryResponse
    DiscoveryRequest
    DiscoveryRequest
    Envoy
    DiscoveryResponse
    DiscoveryRequest
    Update Config
    

    View Slide

28. SoTW vs Incremental
    ● SoTW (State of The World) xDS
    ○ Envoy needs to send all resource names when request.
    ○ xDS server needs to send configs for all subscriptions even if that is not changed…
    ● Incremental xDS (Delta xDS)
    ○ Another approach for xDS
    ○ What is to be less effectively which I described is not needed.
    ○ gRPC is only supported.
    ○ More performant approach when we manage a lot of resources!
    

    View Slide

29. Architecture Overview
    https://github.com/envoyproxy/envoy/blob/master/source/common/config/xDS_code_diagram.png
    

    View Slide

30. Update listener config via gRPC SoTW LDS
    Listener
    Manager
    Grpc
    Subscription
    Grpc Mux
    Grpc
    Subscription
    LDS API
    Subscription
    Watch
    Grpc Stream
    API Listener
    API State
    API State
    Watch
    

    View Slide

31. Update listener config via gRPC SoTW LDS
    Listener
    Manager
    Grpc
    Subscription
    Grpc Mux
    Grpc
    Subscription
    LDS API
    Subscription
    Watch
    Grpc Stream
    API State
    Grpc
    Subscription
    Watch
    API State
    API State
    Watch
    API Listener
    API Listener
    

    View Slide

32. Update listener config via gRPC SoTW LDS
    Listener
    Manager
    Grpc
    Subscription
    Grpc Mux
    Grpc
    Subscription
    LDS API
    Subscription
    Watch
    Grpc Stream
    API Listener
    API State
    Grpc
    Subscription
    Watch
    API State
    API State
    Watch
    API State
    API Listener
    API Listener
    

    View Slide

33. Update listener config via gRPC SoTW LDS
    Listener
    Manager
    Grpc
    Subscription
    Grpc Mux
    Grpc
    Subscription
    LDS API
    Subscription
    Watch
    Grpc Stream
    API Listener
    API State
    Grpc
    Subscription
    Watch
    API State
    API State
    Watch
    API State
    Watch
    API Listener
    API Listener
    

    View Slide

34. Update listener config via gRPC SoTW LDS
    Listener
    Manager
    Grpc
    Subscription
    Grpc Mux
    Grpc
    Subscription
    LDS API
    Subscription
    Watch
    Grpc Stream
    API Listener
    API State
    Grpc
    Subscription
    Watch
    API State
    API State
    Watch
    API State
    Watch
    DiscoveryResponse
    API Listener
    API Listener
    

    View Slide

35. Update listener config via gRPC SoTW LDS
    Listener
    Manager
    Grpc
    Subscription
    Grpc Mux
    Grpc
    Subscription
    LDS API
    Subscription
    Watch
    Grpc Stream
    API Listener
    API State
    Grpc
    Subscription
    Watch
    API State
    API State
    Watch
    API State
    Watch
    DiscoveryResponse
    Callback Flow
    API Listener
    API Listener
    

    View Slide

36. ADS (Aggregated Discovery Service)
    ● In Basic xDS, xDS connection must be created per single type_url.
    ○ The number of connections can be increased explosively…
    ● ADS brings us to manage multiple type_urls by single xDS connection.
    

    View Slide

37. Control Plane for large-scaled Service Mesh
    ● Send DiscoveryResponses to managed envoys.
    ○ Must maintain all of xDS connections.
    ● Marshal / Unmarshal envoy configurations.
    ● Cache DiscoveryResponses
    High computational cost may be occured when we must
    manage many side-car proxies...
    Service A
    Service A
    Service 1
    Service A
    Service A
    Service N
    Control Plane
    Thousands of Envoys
    

    View Slide

38. Introduce aggregation layers
    ● To manage thousand of xDS connection, we can introduce
    an aggregation layer.
    ○ It will reduce computational costs for Control Plane
    ○ DiscoveryResponse which will be shared by some of side-car proxies
    can be aggregated.
    ● Called xds-relay
    ○ It was started developing from 2020 and WIP.
    Service A
    Service A
    Service 1
    Service A
    Service A
    Service N
    Control Plane
    Thousands of Envoys
    xds-relay
    N xDS Connections
    A group to share the same
    DiscoveryResponse
    https://github.com/envoyproxy/xds-relay
    

    View Slide

39. xds-relay components
    Orchestrator
    ● Mapper
    ○ Generate aggregation key
    ○ Key creation rule can be configured by YAML.
    ○ In general, key is generated from node name.
    ● Cache
    ○ Like Key-Value data structure.
    ○ Manage DIscoveryResponses per aggregated
    connection.
    ● Orchestrator
    ○ DiscoveryResponse handler
    Mapper Cache
    DiscoveryRequest
    Aggregated Key
    Aggregated Key
    

    View Slide

40. Thank you for listening!
    

    View Slide

41. Appendix
    

    View Slide

42. Manage multiple resources via xDS
    Resource 1
    Resource 2
    Resource N
    Resource 1
    Subscription 2
    Subscription N
    gRPC
    Subscription
    Watch
    API State
    gRPC Mux
    Watch
    Watch
    

    View Slide