single interface. • Flexibility to change. • Rich and flexible primitive types. ◦ e.g.: String, google.protobuf.Struct, google.protobuf.Any, etc... • Extensibility with protoc plugins. ◦ e.g.: We can generate documentations as reStructuredText from protobuf definitions on Envoy. • Availability to add metadatas by annotations. Why Protocol Buffers?
it can’t validate user passed contents. ◦ e.g.: Protocol buffers can’t reject an address when the field is considered to input a phone number. • protoc-gen-validate can validate contents. ◦ e.g. It is enabled to validate contents with regexp. • protoc-gen-validate is implemented as protoc plugin. • Currently It supports Go / C++ / Java / Python. https://github.com/envoyproxy/protoc-gen-validate protoc-gen-validate
◦ Currently, envoy community supports V2 and V3 API as stable version. • Breaking changes are not allowed to apply stable version of API. • These breaking changes will be applied to alpha version of API. ◦ Currently, V4alpha is used as candidate version.
V3 V4 V5Alpha 2020/12 V2 V3 V4Alpha V2 will be supported only a year after changed to previous major version even if the specifications of V4 are not fixed. https://docs.google.com/document/d/1afQ9wthJxofwOMCAle8qF-ckEfNWUuWtxbmdhxji0sI/edit?ts=5ec10fb3#heading=h.n2ql27ddmi4x
versions of API ◦ V2 and V3 are being supported as major version on 1.14.2. • Annotation based API migration ◦ When we apply breaking changes on current version of API, we don’t delete fields but add an annotation which specifies that is deprecated. • To enable updating API version automatically → protoxform
of API as input, next version of API as output. • If we have breaking changes on major version, protoxform outputs the next version of API, with maintaining backward compatibility by using annotation. ◦ e.g. If we have deprecated fields on current version of API, protoxform understand the annotation and destroy the fields. Automation to change API with protoxform Protoxform V3 V4Alpha
can also pass the version of envoy explicitly. • If not, Envoy decide valid API version automatically, by using abilities of Protobuf. How to enable co-existing multiple versions of API? V3 Bootstrap V3 Admin V3 Static Resources ... ... V3 Router Filter V2 Hoge Filter
Data Plane API) is to standardize the existing data plane API. • This API covers all of capabilities for data plane such as service discovery, listener configurations, etc… • UDPA-WG (Working Group) is proceeding these movements. https://blog.envoyproxy.io/the-universal-data-plane-api-d15cec7a
is used for xDS. ◦ Especially, bidirectional gRPC streaming is applied for gRPC approach. ◦ Polling is used for REST. Envoy xDS Server xDS Server xDS Stream xDS Stream Subscription Subscription Envoy xDS Server
◦ Envoy needs to send all resource names when request. ◦ xDS server needs to send configs for all subscriptions even if that is not changed… • Incremental xDS (Delta xDS) ◦ Another approach for xDS ◦ What is to be less effectively which I described is not needed. ◦ gRPC is only supported. ◦ More performant approach when we manage a lot of resources!
Subscription Grpc Mux Grpc Subscription LDS API Subscription Watch Grpc Stream API Listener API State Grpc Subscription Watch API State API State Watch API State Watch DiscoveryResponse API Listener API Listener
Subscription Grpc Mux Grpc Subscription LDS API Subscription Watch Grpc Stream API Listener API State Grpc Subscription Watch API State API State Watch API State Watch DiscoveryResponse Callback Flow API Listener API Listener
managed envoys. ◦ Must maintain all of xDS connections. • Marshal / Unmarshal envoy configurations. • Cache DiscoveryResponses High computational cost may be occured when we must manage many side-car proxies... Service A Service A Service 1 Service A Service A Service N Control Plane Thousands of Envoys
we can introduce an aggregation layer. ◦ It will reduce computational costs for Control Plane ◦ DiscoveryResponse which will be shared by some of side-car proxies can be aggregated. • Called xds-relay ◦ It was started developing from 2020 and WIP. Service A Service A Service 1 Service A Service A Service N Control Plane Thousands of Envoys xds-relay N xDS Connections A group to share the same DiscoveryResponse https://github.com/envoyproxy/xds-relay
Key creation rule can be configured by YAML. ◦ In general, key is generated from node name. • Cache ◦ Like Key-Value data structure. ◦ Manage DIscoveryResponses per aggregated connection. • Orchestrator ◦ DiscoveryResponse handler Mapper Cache DiscoveryRequest Aggregated Key Aggregated Key