$30 off During Our Annual Pro Sale. View Details »

PiCoP en

Hiroya Onoe
September 25, 2023

PiCoP en

Protocol-Independent Context Propagation for Sharing Microservices in Multiple Environments

Hiroya Onoe

September 25, 2023
Tweet

More Decks by Hiroya Onoe

Other Decks in Research

Transcript

  1. Protocol-Independent Context
    Propagation for Sharing Microservices
    in Multiple Environments
    IC2E 2023
    Hiroya ONOE, Daisuke KOTANI, Yasuo OKABE
    1
    Slide URL: https://onoe.dev/picop-en

    View Slide

  2. Background: Production-Like Environment
    Require Production-Like Environments in CI/CD
    For testing, staging, debugging, and previewing
    →Resources are wasted due to low access
    Microservice Architecture
    A single application consists of
    several microservices
    →A large number of microservices
    can easily lead to a high waste
    2

    View Slide

  3. Background: Production-Like Environment
    A large number of microservices lead to a high waste
    3
    https://github.com/microservices-demo/microservices-demo/blob/master/internal-docs/design.md
    Microservice Demo: Sock Shop

    View Slide

  4. Background: Sharing Microservices
    Reduce resource usage by sharing stateless and non-updated
    services in multiple environments
    (proposed by Wantedly, Mercari, Lyft, and Ambassador Labs)
    1. Propagate contexts that
    identify environments by
    adding them to requests
    2. Route requests
    based on the contexts
    4

    View Slide

  5. Add contexts to HTTP header or gRPC metadata
    1. Propagate contexts with OpenTelemetry
    OpenTelemetry
    Provide specifications and libraries
    for distributed tracing,
    including context propagation
    Background: Sharing Microservices
    5
    func handler(w http.ResponseWriter, r *http.Request) {
    propagator := otelprop.TextMapPropagator
    // Extract
    envID := h.propagator.Extract(r.Context(), “EnvID”)
    // Propagate
    log.Print(envID)
    // Inject
    ctx := propagator.Inject(“EnvID”, envID)
    http.Post(ctx, “https://service2.local/api”)
    ...
    }

    View Slide

  6. Background: Sharing Microservices
    Add contexts to HTTP header or gRPC metadata
    2. Route requests with Envoy and Istio
    6
    Istio
    Build service mesh
    with Envoy in Kubernetes
    Envoy
    A cloud-native proxy
    deployed as a sidecar
    Provide observability, telemetry, and traffic management
    with cloud-native applications

    View Slide

  7. Background: Sharing Microservices
    Problem
    Propagating and Routing depend on specific protocols
    • Not available for other protocols,
    especially those that cannot have
    optional data
    (MySQL, Redis, MQTT, etc.)
    • Require instrumentation
    for each protocol
    7
    ?

    View Slide

  8. Related Works: Context Propagation
    Distributed Tracing
    Trace distributed systems by adding contexts to requests and
    propagating them (Dapper, Pinpoint, X-Trace, Pip, and Canopy)
    Context Propagation Frameworks
    Reduce the effort in instrumenting by standardizing the context
    propagation mechanisms
    (Tracing Plane, Canopy, Pivot Tracing, and Pythia)
    →All methods depend on specific protocols and platforms
    8

    View Slide

  9. Proposal: PiCoP
    A framework to propagate contexts and route requests
    independently of application layer protocols
    Protocol
    Propagate contexts
    without interpreting
    application layer protocols
    Proxy
    Route requests to the
    appropriate environments
    by using the protocol
    9

    View Slide

  10. Proposal: Why Protocol Independence?
    • Enable to propagate contexts in any application
    layer protocol, including those that cannot have
    optional data
    • Enable to use the same proxy to route requests
    of any application layer protocol
    10

    View Slide

  11. Proposal: PiCoP Protocol
    Add contexts to the front of the TCP byte stream
    (as with PROXY Protocol)
    →Available regardless of the type of application layer protocols
    PROXY Protocol
    Protocol-independently propagate
    source client information through
    a reverse proxy server
    the concept is similar,
    but the content and format of
    propagated information is different
    11
    PiCoP Protocol

    View Slide

  12. Proposal: PiCoP Protocol
    Extend the signature of PROXY Protocol version 2
    →No conflicts with application layer protocols
    12
    Byte Description
    1-12 The same signature as PROXY Protocol version 2
    13
    The upper 4 bits: the signature
    The lower 4 bits: the version

    View Slide

  13. Proposal: PiCoP Protocol
    Satisfy Propagators API defined by OpenTelemetry
    →Easy to instrument into applications
    →Easy to use for context propagation for other purposes
    Propagate environment ID by using PiCoP Protocol
    13
    Byte Description
    14,15 The byte length of the context
    16- The context in the form of key-value pairs
    (Satisfied Propagators API)

    View Slide

  14. Proposal: PiCoP Proxy
    Route requests of any application layer protocol to the
    appropriate environment based on environment ID
    Receive the correspondence between an environment ID and a
    destination service in advance(route information)
    14
    Environment ID Destination Service
    EnvA Service[EnvA]
    EnvB Service[Base](default route)
    Other Service[Base](default route)

    View Slide

  15. Proposal: PiCoP Proxy
    Some services do not accept PiCoP Protocol (e.g., MySQL server)
    Services at the end of request processing in an application
    do not need to propagate environment ID
    →Enable to choose whether or not to propagate PiCoP Protocol
    15

    View Slide

  16. Implementation
    Proxy Controller
    Manage proxy and
    route information in DB
    Receive information
    from administrator
    Send information
    to each proxy
    Support scaling out of proxies
    →For large cloud-scale clusters
    16

    View Slide

  17. Evaluation: Proxy Communication Delay

    Send HTTP requests with 1kB payload to NGINX server

    Measure response time of 10000 requests in 10 seconds

    Set the number of simultaneous connections from 1 to 64

    Add ‘Env-Id:Main’ context to HTTP header or PiCoP Protocol
    Four Conditions
    With vs. Without PiCoP Proxy PiCoP Proxy vs. Istio Proxy
    17

    View Slide

  18. Evaluation: Proxy Communication Delay
    Comparison with no PiCoP Proxy
    (base vs. base+picop)
    18
    3.2ms-12.3ms
    slower
    6.7ms-13.3ms
    slower

    View Slide

  19. Comparison with Istio Proxy
    (base+gw+istio vs. base+gw+picop)
    Almost equivalent to Istio, widely used in existing methods
    →The delay of PiCoP Proxy is within practical range
    Evaluation: Proxy Communication Delay
    19
    almost
    equal
    0.3ms-2.7ms
    slower

    View Slide

  20. Evaluation: Resource Reduction
    • Load PiCoP Proxy and NGINX server by continuously sending HTTP requests
    • Scale out to 80% or less CPU utilization for all containers
    • Compare the number of containers of proxy and NGINX server
    • Each container’s resource is limited
    • Change the number of environments
    from 1 to 100
    • Send 100 or 1000 requests per second
    in total for all environments from clients
    equal to the number of environments
    20

    View Slide

  21. Evaluation: Resource Reduction
    PiCoP reduce resource usage
    when the number of environments exceeds a specific value
    The more environments there are, the more significant the reduction is
    21

    View Slide

  22. Discussion: Protocol Independence
    Inter-Service Propagation
    Intra-Service Propagation
    22

    View Slide

  23. Discussion: Protocol Independence
    Inter-Service Propagation
    PiCoP can propagate contexts between services in the same
    mechanism and proxy for all protocols, including those that
    cannot have optional data
    23
    Cannot have optional data Can have optional data
    MySQL, PostgreSQL, Memcached, Redis,
    MongoDB Wire Protocol, MQTT, Kafka Wire
    Protocol, TDS(Microsoft SQL Server)
    HTTP, gRPC, AMQP, Cassandra Native Protocol

    View Slide

  24. Discussion: Protocol Independence
    Inter-Service Propagation
    Restriction on connection reuse
    (e.g., persistent HTTP connections)
    Overhead of establishing a TCP connection each time
    24

    View Slide

  25. Discussion: Protocol Independence
    Inter-Service Propagation
    Need to implement a proxy for each transport layer protocol
    Available for UDP, QUIC, etc. in addition to TCP
    Add PiCoP Protocol to the front of the datagram or stream
    Unidirectional Communication
    Deploy PiCoP Proxy for services receiving requests
    Bidirectional Communication
    Deploy PiCoP Proxy for both services
    25

    View Slide

  26. Discussion: Protocol Independence
    Intra-Service Propagation
    In services in the middle of request processing in an
    application (e.g., a web server),
    instrumentation required (as with existing methods)
    26
    Need instrumentation libraries
    per each language and protocol
    Enable to use instrumentation libraries
    provided by OpenTelemetry

    View Slide

  27. Discussion: Protocol Independence
    Intra-Service Propagation
    In services at the end of request processing in an application
    (e.g., a database), no instrumentation required
    The services do not need environment ID
    27

    View Slide

  28. Discussion: Protocol Independence
    Intra-Service Propagation
    In asynchronous processing
    (e.g., a message queue),
    need to extend PiCoP Proxy
    Unable to propagate contexts
    in one direction through
    asynchronous processing
    28
    Extended
    Proxy

    View Slide

  29. Conclusion
    • Propose PiCoP for protocol-independent microservice sharing
    • PiCoP protocol propagates contexts without interpreting application layer protocols
    • PiCoP proxy routes requests to the appropriate environments by using the protocol
    • Show practicality in terms of communication delay and resource reduction
    • Show constraints on connection reuse and asynchronous processing
    • Available as OSS ( https://github.com/picop-rd )
    Future Work
    • Resolve the above constraints
    • Evaluate at large cloud-scale clusters
    • Analysis the applicability of PiCoP protocol to context propagation for other purposes
    29
    Slide URL: https://onoe.dev/picop-en

    View Slide