Spring Cloud Gateway: Resilience, Security, and Observability

Do you want to use a microservices architecture? Are you looking for a solution to manage access to single services from clients? How can you ensure resilience and security for your entire system? Spring Cloud Gateway is a project based on Reactor, Spring WebFlux, and Spring Boot which provides an effective way to route traffic to your APIs and address cross-cutting concerns.

In this session, I'll show you how to configure an API gateway to route traffic to your microservices architecture and implement solutions to improve the resilience of your system with patterns like circuit breakers, retries, fallbacks, and rate limiters using Spring Cloud Circuit Breaker and Resilience4J. Since the gateway is the entry point of your system, it’s also an excellent candidate to implement security concerns like user authentication. I'll show you how to do that with Spring Security, OAuth2, and OpenID Connect, relying on Spring Redis Reactive to manage sessions. Finally, I'll show you how to improve the observability of your system using Spring Boot Actuator and Spring Cloud Sleuth and relying on the Grafana stack.

Transcript

1. Thomas Vitale
   Devoxx UK
   May 12th, 2021
   Spring Cloud Gateway
   Resilience, Security, and Observability
   @vitalethomas
   

   View Slide

2. Systematic
   • Software Architect at
   Systematic, Denmark.
   
   • Author of “Cloud Native Spring
   in Action” (Manning).
   
   • Spring Security and Spring
   Cloud contributor.
   Thomas Vitale
   thomasvitale.com @vitalethomas
   

   View Slide

3. API Gateway
   thomasvitale.com @vitalethomas
   

   View Slide

4. Scenarios
   Di
   ff
   erent clients need
   di
   ff
   erent APIs
   Cross-cutting concerns in
   distributed systems
   Uni
   fi
   ed interface for
   microservices
   Strangling the monolith
   thomasvitale.com @vitalethomas
   

   View Slide

5. $FFRXQW6HUYLFH
   >&RQWDLQHU6SULQJ%RRW@
   3URYLGHVIXQFWLRQDOLW\IRU
   PDQDJLQJPHPEHUV
   DFFRXQWV
   /RDQ6HUYLFH
   >&RQWDLQHU6SULQJ%RRW@
   3URYLGHVIXQFWLRQDOLW\IRU
   PDQDJLQJERRNORDQV
   /LEUDU\
   >6RIWZDUH6\VWHP@
   8VHV
   >5(67+773@
   8VHV
   >5(67+773@
   (GJH6HUYLFH
   >&RQWDLQHU6SULQJ%RRW@
   3URYLGHV$3,JDWHZD\DQG
   FURVVFXWWLQJFRQFHUQV
   8VHU
   >3HUVRQ@
   
   $PHPEHURIWKH/LEUDU\
   8VHV
   %RRN6HUYLFH
   >&RQWDLQHU6SULQJ%RRW@
   3URYLGHVIXQFWLRQDOLW\IRU
   PDQDJLQJWKHOLEUDU\ERRNV
   8VHV
   >5(67+773@
   

   View Slide

6. Reactive Spring
   thomasvitale.com @vitalethomas
   

   View Slide

7. Thread-per-request
   thomasvitale.com @vitalethomas
   7KUHDG3RRO
   ,QWHQVLYH
   2SHUDWLRQ
   7KUHDG
   7KUHDG
   7KUHDG
   5HTXHVW
   5HTXHVW
   5HTXHVW
   %ORFNLQJ
   ZDLWIRUUHVXOW
   2QHWKUHDG
   SHUUHTXHVW
   

   View Slide

8. Event Loop
   thomasvitale.com @vitalethomas
   ,QWHQVLYH
   2SHUDWLRQ
   1RQ%ORFNLQJ
   QRQZDLWLQJIRUUHVXOW
   -XVWDIHZWKUHDGV
   SURFHVVLQJPXOWLSOH
   UHTXHVWV
   (YHQW/RRS
   (YHQW4XHXH
   5HTXHVW5HVSRQVH
   VFKHGXOH
   HYHQW
   UHJLVWHU
   FDOOEDFN
   RSHUDWLRQ
   FRPSOHWH
   WULJJHU
   FDOOEDFN
   

   View Slide

9. thomasvitale.com @vitalethomas
   

   View Slide

10. Routing
    thomasvitale.com @vitalethomas
    

    View Slide

11. The Architecture
    thomasvitale.com @vitalethomas
    

    View Slide

12. Observability
    thomasvitale.com @vitalethomas
    

    View Slide

13. grafana.com
    

    View Slide

14. View Slide

15. Monitoring and management
    thomasvitale.com @vitalethomas
    Operating applications in production
    Spring Boot Actuator
    ‣Health (liveness and readiness)
    
    
    ‣Metrics (Prometheus, OpenMetrics)
    
    
    ‣Flyway, Thread Dumps, Heap Dumps
    Spring Cloud Sleuth
    
    
    (Micrometer Tracing)
    ‣Distributed tracing
    
    
    ‣Instrumentation
    
    
    ‣OpenZipkin and OpenTelemetry
    

    View Slide

16. Resilience
    thomasvitale.com @vitalethomas
    

    View Slide

17. Retry
    thomasvitale.com @vitalethomas
    

    View Slide

18. Retry
    thomasvitale.com @vitalethomas
    %RRN5RXWH 5HWU\ %RRN&RQWUROOHU
    (GJH6HUYLFH %RRN6HUYLFH
    W W W
    6HQG+773UHTXHVW
    5HFHLYH+773HUURU
    5HWU\+773UHTXHVW
    5HFHLYH+773HUURU
    5HWU\+773UHTXHVW
    5HFHLYHVXFFHVVIXOO+773UHVSRQVHDIWHUVHFRQGUHWU\DWWHPSW
    

    View Slide

19. Request Rate Limiter
    thomasvitale.com @vitalethomas
    

    View Slide

20. Rate Limiter
    thomasvitale.com @vitalethomas
    https://stripe.com/blog/rate-limiters
    

    View Slide

21. Circuit Breaker
    thomasvitale.com @vitalethomas
    

    View Slide

22. Circuit Breaker
    thomasvitale.com @vitalethomas
    &/26('
    +$/)B23(1
    23(1
    7ULSEUHDNHUZKHQ
    IDLOXUHUDWHDERYH
    WKUHVKROG
    $WWHPSWUHVHWDIWHU
    ZDLWGXUDWLRQ
    7ULSEUHDNHUDIWHU
    IDLOXUHUDWHDERYH
    WKUHVKROG
    5HVHWEUHDNHUZKHQ
    IDLOXUHUDWHEHORZ
    WKUHVKROG
    

    View Slide

23. Time Limiter
    thomasvitale.com @vitalethomas
    

    View Slide

24. Time Limiter and Fallback
    thomasvitale.com @vitalethomas
    %RRN5RXWH
    7LPH/LPLWHU
    )DOOEDFN
    7LPH/LPLWHU %RRN&RQWUROOHU
    (GJH6HUYLFH %RRN6HUYLFH
    W W W W
    6HQG+773UHTXHVW
    D5HFHLYHVXFFHVVIXOO+773UHVSRQVHZLWKLQWKHWLPHOLPLW
    E7KURZH[FHSWLRQZKHQWLPHRXWH[SLUHVDQGQRIDOOEDFNGHILQHG
    F5HWXUQIDOOEDFNZKHQGHILQHGDQGWLPHRXWH[SLUHV
    

    View Slide

25. User Authentication
    thomasvitale.com @vitalethomas
    

    View Slide

26. ,QYHQWRU\6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHVIXQFWLRQDOLW\IRU
    PDQDJLQJWKHERRNVKRS
    LQYHQWRU\
    2UGHU6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHVIXQFWLRQDOLW\IRU
    PDQDJLQJERRNRUGHUV
    3RODU%RRNVKRS
    >6RIWZDUH6\VWHP@
    8VHV
    >5(67+773@
    8VHV
    >5(67+773@
    (GJH6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHV$3,JDWHZD\DQG
    FURVVFXWWLQJFRQFHUQV
    8VHU
    >3HUVRQ@
    $QHPSOR\HHRIWKH
    ERRNVKRS
    8VHV
    %RRN6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHVIXQFWLRQDOLW\IRU
    PDQDJLQJWKHOLEUDU\ERRNV
    8VHV
    >5(67+773@
    $XWK6HUYLFH
    'HOHJDWHVDXWKHQWLFDWLRQWR
    Strategy ?
    Protocol?
    Data Format?
    

    View Slide

27. Login
    thomasvitale.com @vitalethomas
    /LEUDU\
    >6RIWZDUH6\VWHP@
    (GJH6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHV$3,JDWHZD\DQG
    FURVVFXWWLQJFRQFHUQV
    8VHU
    >3HUVRQ@
    
    $PHPEHURIWKHOLEUDU\
    8VHV
    2$XWK&OLHQW
    2$XWK8VHU
    .H\FORDN
    >&RQWDLQHU:LOG)O\@
    3URYLGHVLGHQWLW\DQGDFFHVV
    PDQDJHPHQW
    2$XWK$XWKRUL]DWLRQ6HUYHU
    8VHV 'HOHJDWHVDXWKHQWLFDWLRQDQG
    WRNHQPDQDJHPHQWWR
    OAuth2 + OIDC
    

    View Slide

28. OpenID Connect
    A protocol built on top of OAuth2 that enables
    
    an application (Client) to verify the identity of
    
    a user based on the authentication performed
    
    by a trusted party (Authorization Server).
    thomasvitale.com @vitalethomas
    

    View Slide

29. .H\FORDN
    >&RQWDLQHU:LOGIO\@
    3URYLGHVLGHQWLW\DQG
    DFFHVVPDQDJHPHQW
    ,QYHQWRU\6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHVIXQFWLRQDOLW\IRU
    PDQDJLQJWKHERRNVKRS
    LQYHQWRU\
    2UGHU6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHVIXQFWLRQDOLW\IRU
    PDQDJLQJERRNRUGHUV
    3RODU%RRNVKRS
    >6RIWZDUH6\VWHP@
    8VHV
    >5(67+773@
    8VHV
    >5(67+773@
    (GJH6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHV$3,JDWHZD\DQG
    FURVVFXWWLQJFRQFHUQV
    8VHU
    >3HUVRQ@
    $QHPSOR\HHRIWKH
    ERRNVKRS
    8VHV
    %RRN6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHVIXQFWLRQDOLW\IRU
    PDQDJLQJWKHOLEUDU\ERRNV
    8VHV
    >5(67+773@
    'HOHJDWHVDXWKHQWLFDWLRQWR
    2$XWK&OLHQW
    2$XWK$XWKRUL]DWLRQ6HUYHU
    8VHV
    {
    
    "iss": “keycloak",
    
    "sub": "isabelle",
    
    "exp": 1626439022
    
    }
    ID Token
    ID Token
    

    View Slide

30. .H\FORDN
    >&RQWDLQHU:LOGIO\@
    3URYLGHVLGHQWLW\DQG
    DFFHVVPDQDJHPHQW
    ,QYHQWRU\6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHVIXQFWLRQDOLW\IRU
    PDQDJLQJWKHERRNVKRS
    LQYHQWRU\
    2UGHU6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHVIXQFWLRQDOLW\IRU
    PDQDJLQJERRNRUGHUV
    3RODU%RRNVKRS
    >6RIWZDUH6\VWHP@
    8VHV
    >5(67+773@
    8VHV
    >5(67+773@
    (GJH6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHV$3,JDWHZD\DQG
    FURVVFXWWLQJFRQFHUQV
    8VHU
    >3HUVRQ@
    $QHPSOR\HHRIWKH
    ERRNVKRS
    8VHV
    %RRN6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHVIXQFWLRQDOLW\IRU
    PDQDJLQJWKHOLEUDU\ERRNV
    8VHV
    >5(67+773@
    'HOHJDWHVDXWKHQWLFDWLRQWR
    2$XWK&OLHQW
    2$XWK$XWKRUL]DWLRQ6HUYHU
    8VHV
    Security context
    propagation ?
    Authorized access?
    

    View Slide

31. OAuth2
    An authorization framework that enables an
    application (Client) to obtain limited access to a
    protected resource provided by another
    application (called Resource Server)
    
    on behalf of a user.
    thomasvitale.com @vitalethomas
    

    View Slide

32. .H\FORDN
    >&RQWDLQHU:LOGIO\@
    3URYLGHVLGHQWLW\DQG
    DFFHVVPDQDJHPHQW
    ,QYHQWRU\6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHVIXQFWLRQDOLW\IRU
    PDQDJLQJWKHERRNVKRS
    LQYHQWRU\
    2UGHU6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHVIXQFWLRQDOLW\IRU
    PDQDJLQJERRNRUGHUV
    3RODU%RRNVKRS
    >6RIWZDUH6\VWHP@
    8VHV
    >5(67+773@
    8VHV
    >5(67+773@
    (GJH6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHV$3,JDWHZD\DQG
    FURVVFXWWLQJFRQFHUQV
    8VHU
    >3HUVRQ@
    $QHPSOR\HHRIWKH
    ERRNVKRS
    8VHV
    %RRN6HUYLFH
    >&RQWDLQHU6SULQJ%RRW@
    3URYLGHVIXQFWLRQDOLW\IRU
    PDQDJLQJWKHOLEUDU\ERRNV
    8VHV
    >5(67+773@
    'HOHJDWHVDXWKHQWLFDWLRQWR
    2$XWK&OLHQW
    2$XWK$XWKRUL]DWLRQ6HUYHU
    8VHV
    2$XWK5HVRXUFH6HUYHU
    2$XWK5HVRXUFH6HUYHU
    2$XWK5HVRXUFH6HUYHU
    {
    
    "iss": “keycloak",
    
    "sub": "isabelle",
    
    "exp": 1626439022
    
    }
    Access Token
    Access Token
    

    View Slide

33. Token Relay
    thomasvitale.com @vitalethomas
    %URZVHU (GJH6HUYLFH %RRN
    6HUYLFH
    $FFHVV7RNHQ
    6HVVLRQ&RRNLH
    5HVRXUFH
    6HUYHU
    $FFHVV7RNHQ
    5HVRXUFH
    6HUYHU
    $FFHVV7RNHQ
    .HHSVPDSSLQJ
    6HVVLRQ!$FFHVV7RNHQ
    OAuth2
    

    View Slide

34. Resources
    Source code
    • Sample project:
    
    • https://github.com/ThomasVitale/devoxx-uk-2022-spring-cloud-
    gateway
    
    • Spring Cloud Gateway:
    
    • https://spring.io/projects/spring-cloud-gateway
    
    • Spring Security, OAuth2, OpenID Connect:
    
    • https://www.youtube.com/watch?v=g7Dwv1BKnkg
    

    View Slide

35. Thomas Vitale
    Devoxx UK
    May 12th, 2021
    Spring Cloud Gateway
    Resilience, Security, and Observability
    @vitalethomas
    

    View Slide