Spring Cloud Gateway: Resilience, Security, and Observability (Golden Path to SpringOne 2023)

Do you want to use a microservices architecture? Are you looking for a solution to manage access to single services from clients? How can you ensure resilience and security for your entire system? Spring Cloud Gateway is a project based on Reactor, Spring WebFlux, and Spring Boot which provides an effective way to route traffic to your APIs and address cross-cutting concerns.

In this session, I’ll show you how to configure an API gateway to route traffic to your microservices and improve the resilience of your system with patterns like circuit breakers, retries, fallbacks, and rate limiters using Spring Cloud Circuit Breaker and Resilience4J. Since the gateway is the entry point of your system, it’s also an excellent candidate to address security concerns like user authentication. I’ll show you how to do that with Spring Security, OAuth2, and OpenID Connect. Finally, I’ll show you how to improve the observability of your system using Spring Boot Actuator, OpenTelemetry, and Grafana.

Transcript

1. Thomas Vitale
   The Golden Path to SpringOne
   Feb 23rd, 2023
   Spring Cloud Gateway
   Resilience, Security, and Observability
   @vitalethomas
   

   View Slide

2. Systematic
   • Software Engineer and Cloud
   Architect.
   
   • Author of “Cloud Native Spring
   in Action” (Manning).
   
   • OSS contributor (Java, Spring,
   Cloud Native Technologies)
   Thomas Vitale
   thomasvitale.com @vitalethomas
   

   View Slide

3. API Gateway
   @vitalethomas
   

   View Slide

4. Scenarios
   Di
   ff
   erent clients need
   di
   ff
   erent APIs
   Cross-cutting concerns in
   distributed systems
   Uni
   fi
   ed interface for
   microservices
   Strangling the monolith
   @vitalethomas
   

   View Slide

5. Account Service
   [Container: Spring Boot]
   Provides functionality for
   managing members
   accounts.
   Loan Service
   [Container: Spring Boot]
   Provides functionality for
   managing book loans.
   Library
   [Software System]
   Uses
   [REST/HTTP]
   Uses
   [REST/HTTP]
   Edge Service
   [Container: Spring Boot]
   Provides API gateway and
   cross-cutting concerns.
   User
   [Person]
   A member of the Library.
   Uses
   Book Service
   [Container: Spring Boot]
   Provides functionality for
   managing the library books.
   Uses
   [REST/HTTP]
   

   View Slide

6. Routing
   @vitalethomas
   

   View Slide

7. The Architecture
   @vitalethomas
   

   View Slide

8. Observability
   @vitalethomas
   

   View Slide

9. Monitoring and management
   Operating applications in production
   Spring Boot Actuator
   ‣Health (liveness and readiness)
   
   
   ‣Metrics (Prometheus, OpenMetrics)
   
   
   ‣Flyway, Thread Dumps, Heap Dumps
   Micrometer Tracing
   
   
   (Spring Cloud Sleuth)
   ‣Distributed tracing
   
   
   ‣Instrumentation
   
   
   ‣OpenZipkin and OpenTelemetry
   @vitalethomas
   

   View Slide

10. grafana.com
    

    View Slide

11. View Slide

12. View Slide

13. Resilience
    @vitalethomas
    

    View Slide

14. Retry
    @vitalethomas
    

    View Slide

15. Retry
    Book Route Retry Book Controller
    Edge Service Book Service
    t t t
    1. Send HTTP request
    2. Receive HTTP 503 error
    3. Retry HTTP request
    4. Receive HTTP 503 error
    5. Retry HTTP request
    6. Receive successfull HTTP response after second retry attempt
    @vitalethomas
    

    View Slide

16. Request Rate Limiter
    @vitalethomas
    

    View Slide

17. Rate Limiter https://stripe.com/blog/rate-limiters
    @vitalethomas
    

    View Slide

18. Circuit Breaker
    @vitalethomas
    

    View Slide

19. Circuit Breaker
    CLOSED
    HALF_OPEN
    OPEN
    Trip breaker when
    failure rate above
    threshold
    Attempt reset after
    wait duration
    Trip breaker after
    failure rate above
    threshold
    Reset breaker when
    failure rate below
    threshold
    @vitalethomas
    

    View Slide

20. Time Limiter
    @vitalethomas
    

    View Slide

21. Time Limiter and Fallback
    Book Route
    Time Limiter
    Fallback
    Time Limiter Book Controller
    Edge Service Book Service
    t t t t
    1. Send HTTP request
    2a. Receive successfull HTTP response within the time limit
    2b. Throw exception when timeout expires and no fallback defined
    2c. Return fallback when defined and timeout expires
    @vitalethomas
    

    View Slide

22. User Authentication
    @vitalethomas
    

    View Slide

23. Inventory Service
    [Container: Spring Boot]
    Provides functionality for
    managing the bookshop
    inventory.
    Order Service
    [Container: Spring Boot]
    Provides functionality for
    managing book orders.
    Polar Bookshop
    [Software System]
    Uses
    [REST/HTTP]
    Uses
    [REST/HTTP]
    Edge Service
    [Container: Spring Boot]
    Provides API gateway and
    cross-cutting concerns.
    User
    [Person]
    An employee of the
    bookshop.
    Uses
    Book Service
    [Container: Spring Boot]
    Provides functionality for
    managing the library books.
    Uses
    [REST/HTTP]
    Auth Service
    Delegates authentication to
    Strategy ?
    Protocol?
    Data Format?
    

    View Slide

24. OpenID Connect
    A protocol built on top of OAuth2 that enables
    
    an application (Client) to verify the identity of
    
    a user based on the authentication performed
    
    by a trusted party (Authorization Server).
    @vitalethomas
    

    View Slide

25. Keycloak
    [Container: Wildfly]
    Provides identity and
    access management.
    Inventory Service
    [Container: Spring Boot]
    Provides functionality for
    managing the bookshop
    inventory.
    Order Service
    [Container: Spring Boot]
    Provides functionality for
    managing book orders.
    Polar Bookshop
    [Software System]
    Uses
    [REST/HTTP]
    Uses
    [REST/HTTP]
    Edge Service
    [Container: Spring Boot]
    Provides API gateway and
    cross-cutting concerns.
    User
    [Person]
    An employee of the
    bookshop.
    Uses
    Book Service
    [Container: Spring Boot]
    Provides functionality for
    managing the library books.
    Uses
    [REST/HTTP]
    Delegates authentication to
    OAuth2 Client
    OAuth2 Authorization Server
    Uses
    {
    
    "iss": “keycloak",
    
    "sub": "isabelle",
    
    "exp": 1626439022
    
    }
    ID Token
    ID Token
    OIDC
    

    View Slide

26. Keycloak
    [Container: Wildfly]
    Provides identity and
    access management.
    Inventory Service
    [Container: Spring Boot]
    Provides functionality for
    managing the bookshop
    inventory.
    Order Service
    [Container: Spring Boot]
    Provides functionality for
    managing book orders.
    Polar Bookshop
    [Software System]
    Uses
    [REST/HTTP]
    Uses
    [REST/HTTP]
    Edge Service
    [Container: Spring Boot]
    Provides API gateway and
    cross-cutting concerns.
    User
    [Person]
    An employee of the
    bookshop.
    Uses
    Book Service
    [Container: Spring Boot]
    Provides functionality for
    managing the library books.
    Uses
    [REST/HTTP]
    Delegates authentication to
    OAuth2 Client
    OAuth2 Authorization Server
    Uses
    Security context
    propagation ?
    Authorized access?
    

    View Slide

27. OAuth2
    An authorization framework that enables an
    application (Client) to obtain limited access to a
    protected resource provided by another
    application (called Resource Server)
    
    on behalf of a user.
    @vitalethomas
    

    View Slide

28. Keycloak
    [Container: Wildfly]
    Provides identity and
    access management.
    Inventory Service
    [Container: Spring Boot]
    Provides functionality for
    managing the bookshop
    inventory.
    Order Service
    [Container: Spring Boot]
    Provides functionality for
    managing book orders.
    Polar Bookshop
    [Software System]
    Uses
    [REST/HTTP]
    Uses
    [REST/HTTP]
    Edge Service
    [Container: Spring Boot]
    Provides API gateway and
    cross-cutting concerns.
    User
    [Person]
    An employee of the
    bookshop.
    Uses
    Book Service
    [Container: Spring Boot]
    Provides functionality for
    managing the library books.
    Uses
    [REST/HTTP]
    Delegates authentication to
    OAuth2 Client
    OAuth2 Authorization Server
    Uses
    OAuth2 Resource Server
    OAuth2 Resource Server
    OAuth2 Resource Server
    {
    
    "iss": “keycloak",
    
    "sub": "isabelle",
    
    "exp": 1626439022
    
    }
    Access Token
    Access Token
    OAuth2
    

    View Slide

29. Token Relay
    Browser Edge Service Book
    Service
    Access Token
    Session Cookie
    Resource
    Server
    Access Token
    Resource
    Server
    Access Token
    Keeps mapping
    Session <---> Access Token
    OAuth2
    @vitalethomas
    

    View Slide

30. Resources
    @vitalethomas
    

    View Slide

31. https://github.com/ThomasVitale/awesome-spring
    

    View Slide

32. Resources
    Source code
    • Sample project:
    
    • https://github.com/ThomasVitale/spring-cloud-gateway-resilience-
    security-observability
    
    • Spring Cloud Gateway:
    
    • https://spring.io/projects/spring-cloud-gateway
    
    • Spring Security, OAuth2, OpenID Connect:
    
    • https://www.youtube.com/watch?v=g7Dwv1BKnkg
    @vitalethomas
    

    View Slide

33. Thomas Vitale
    The Golden Path to SpringOne
    Feb 23rd, 2023
    Spring Cloud Gateway
    Resilience, Security, and Observability
    @vitalethomas
    

    View Slide