Spring Cloud Gateway: Resilience, Security, and Observability

Do you want to use a microservices architecture? Are you looking for a solution to manage access to single services from clients? How can you ensure resilience and security for your entire system? Spring Cloud Gateway is a project based on Reactor, Spring WebFlux, and Spring Boot which provides an effective way to route traffic to your APIs and address cross-cutting concerns.
In this session, I’ll show you how to configure an API gateway to route traffic to your microservices architecture and implement solutions to improve the resilience of your system with patterns like circuit breakers, retries, fallbacks, and rate limiters using Spring Cloud Circuit Breaker and Resilience4J. Since the gateway is the entry point of your system, it’s also an excellent candidate to implement security concerns like user authentication. I’ll show you how to do that with Spring Security, OAuth2, and OpenID Connect, relying on Spring Redis Reactive to manage sessions. Finally, I’ll show you how to improve the observability of your system using Spring Boot Actuator and Spring Cloud Sleuth and relying on the Grafana stack.

Transcript

1. Thomas Vitale Spring I/O May 26th, 2022 Spring Cloud Gateway Resilience, Security, and Observability @vitalethomas
2. Systematic • Software Architect at Systematic, Denmark. • Author of “Cloud Native Spring in Action” (Manning). • Spring Security and Spring Cloud contributor. Thomas Vitale thomasvitale.com @vitalethomas
3. API Gateway thomasvitale.com @vitalethomas
4. Scenarios Di ff erent clients need di ff erent APIs Cross-cutting concerns in distributed systems Uni fi ed interface for microservices Strangling the monolith thomasvitale.com @vitalethomas
5. $FFRXQW6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHVIXQFWLRQDOLW\IRU PDQDJLQJPHPEHUV DFFRXQWV /RDQ6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHVIXQFWLRQDOLW\IRU PDQDJLQJERRNORDQV /LEUDU\ >6RIWZDUH6\VWHP@ 8VHV >5(67+773@ 8VHV >5(67+773@ (GJH6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHV$3,JDWHZD\DQG FURVVFXWWLQJFRQFHUQV 8VHU >3HUVRQ@  $PHPEHURIWKH/LEUDU\ 8VHV %RRN6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHVIXQFWLRQDOLW\IRU PDQDJLQJWKHOLEUDU\ERRNV 8VHV >5(67+773@
6. Reactive Spring thomasvitale.com @vitalethomas
7. Thread-per-request thomasvitale.com @vitalethomas 7KUHDG3RRO ,QWHQVLYH 2SHUDWLRQ 7KUHDG 7KUHDG 7KUHDG 5HTXHVW 5HTXHVW 5HTXHVW %ORFNLQJ ZDLWIRUUHVXOW 2QHWKUHDG SHUUHTXHVW
8. Event Loop thomasvitale.com @vitalethomas ,QWHQVLYH 2SHUDWLRQ 1RQ%ORFNLQJ QRQZDLWLQJIRUUHVXOW -XVWDIHZWKUHDGV SURFHVVLQJPXOWLSOH UHTXHVWV (YHQW/RRS (YHQW4XHXH 5HTXHVW5HVSRQVH VFKHGXOH HYHQW UHJLVWHU FDOOEDFN RSHUDWLRQ FRPSOHWH WULJJHU FDOOEDFN
9. thomasvitale.com @vitalethomas
10. Routing thomasvitale.com @vitalethomas
11. The Architecture thomasvitale.com @vitalethomas
12. Observability thomasvitale.com @vitalethomas
13. grafana.com
15. Monitoring and management thomasvitale.com @vitalethomas Operating applications in production Spring Boot Actuator ‣Health (liveness and readiness) ‣Metrics (Prometheus, OpenMetrics) ‣Flyway, Thread Dumps, Heap Dumps Spring Cloud Sleuth (Micrometer Tracing) ‣Distributed tracing ‣Instrumentation ‣OpenZipkin and OpenTelemetry
16. Resilience thomasvitale.com @vitalethomas
17. Retry thomasvitale.com @vitalethomas
18. Retry thomasvitale.com @vitalethomas %RRN5RXWH 5HWU\ %RRN&RQWUROOHU (GJH6HUYLFH %RRN6HUYLFH W W W 6HQG+773UHTXHVW 5HFHLYH+773HUURU 5HWU\+773UHTXHVW 5HFHLYH+773HUURU 5HWU\+773UHTXHVW 5HFHLYHVXFFHVVIXOO+773UHVSRQVHDIWHUVHFRQGUHWU\DWWHPSW
19. Request Rate Limiter thomasvitale.com @vitalethomas
20. Rate Limiter thomasvitale.com @vitalethomas https://stripe.com/blog/rate-limiters
21. Circuit Breaker thomasvitale.com @vitalethomas
22. Circuit Breaker thomasvitale.com @vitalethomas &/26(' +$/)B23(1 23(1 7ULSEUHDNHUZKHQ IDLOXUHUDWHDERYH WKUHVKROG $WWHPSWUHVHWDIWHU ZDLWGXUDWLRQ 7ULSEUHDNHUDIWHU IDLOXUHUDWHDERYH WKUHVKROG 5HVHWEUHDNHUZKHQ IDLOXUHUDWHEHORZ WKUHVKROG
23. Time Limiter thomasvitale.com @vitalethomas
24. Time Limiter and Fallback thomasvitale.com @vitalethomas %RRN5RXWH 7LPH/LPLWHU )DOOEDFN 7LPH/LPLWHU %RRN&RQWUROOHU (GJH6HUYLFH %RRN6HUYLFH W W W W 6HQG+773UHTXHVW D5HFHLYHVXFFHVVIXOO+773UHVSRQVHZLWKLQWKHWLPHOLPLW E7KURZH[FHSWLRQZKHQWLPHRXWH[SLUHVDQGQRIDOOEDFNGHILQHG F5HWXUQIDOOEDFNZKHQGHILQHGDQGWLPHRXWH[SLUHV
25. User Authentication thomasvitale.com @vitalethomas
26. ,QYHQWRU\6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHVIXQFWLRQDOLW\IRU PDQDJLQJWKHERRNVKRS LQYHQWRU\ 2UGHU6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHVIXQFWLRQDOLW\IRU PDQDJLQJERRNRUGHUV 3RODU%RRNVKRS >6RIWZDUH6\VWHP@ 8VHV >5(67+773@ 8VHV >5(67+773@ (GJH6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHV$3,JDWHZD\DQG FURVVFXWWLQJFRQFHUQV 8VHU >3HUVRQ@ $QHPSOR\HHRIWKH ERRNVKRS 8VHV %RRN6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHVIXQFWLRQDOLW\IRU PDQDJLQJWKHOLEUDU\ERRNV 8VHV >5(67+773@ $XWK6HUYLFH 'HOHJDWHVDXWKHQWLFDWLRQWR Strategy ? Protocol? Data Format?
27. OpenID Connect A protocol built on top of OAuth2 that enables an application (Client) to verify the identity of a user based on the authentication performed by a trusted party (Authorization Server). thomasvitale.com @vitalethomas
28. Login thomasvitale.com @vitalethomas /LEUDU\ >6RIWZDUH6\VWHP@ (GJH6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHV$3,JDWHZD\DQG FURVVFXWWLQJFRQFHUQV 8VHU >3HUVRQ@  $PHPEHURIWKHOLEUDU\ 8VHV 2$XWK&OLHQW 2$XWK8VHU .H\FORDN >&RQWDLQHU:LOG)O\@ 3URYLGHVLGHQWLW\DQGDFFHVV PDQDJHPHQW 2$XWK$XWKRUL]DWLRQ6HUYHU 8VHV 'HOHJDWHVDXWKHQWLFDWLRQDQG WRNHQPDQDJHPHQWWR OAuth2 + OIDC
29. .H\FORDN >&RQWDLQHU:LOGIO\@ 3URYLGHVLGHQWLW\DQG DFFHVVPDQDJHPHQW ,QYHQWRU\6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHVIXQFWLRQDOLW\IRU PDQDJLQJWKHERRNVKRS LQYHQWRU\ 2UGHU6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHVIXQFWLRQDOLW\IRU PDQDJLQJERRNRUGHUV 3RODU%RRNVKRS >6RIWZDUH6\VWHP@ 8VHV >5(67+773@ 8VHV >5(67+773@ (GJH6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHV$3,JDWHZD\DQG FURVVFXWWLQJFRQFHUQV 8VHU >3HUVRQ@ $QHPSOR\HHRIWKH ERRNVKRS 8VHV %RRN6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHVIXQFWLRQDOLW\IRU PDQDJLQJWKHOLEUDU\ERRNV 8VHV >5(67+773@ 'HOHJDWHVDXWKHQWLFDWLRQWR 2$XWK&OLHQW 2$XWK$XWKRUL]DWLRQ6HUYHU 8VHV { "iss": “keycloak", "sub": "isabelle", "exp": 1626439022 } ID Token ID Token
30. .H\FORDN >&RQWDLQHU:LOGIO\@ 3URYLGHVLGHQWLW\DQG DFFHVVPDQDJHPHQW ,QYHQWRU\6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHVIXQFWLRQDOLW\IRU PDQDJLQJWKHERRNVKRS LQYHQWRU\ 2UGHU6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHVIXQFWLRQDOLW\IRU PDQDJLQJERRNRUGHUV 3RODU%RRNVKRS >6RIWZDUH6\VWHP@ 8VHV >5(67+773@ 8VHV >5(67+773@ (GJH6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHV$3,JDWHZD\DQG FURVVFXWWLQJFRQFHUQV 8VHU >3HUVRQ@ $QHPSOR\HHRIWKH ERRNVKRS 8VHV %RRN6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHVIXQFWLRQDOLW\IRU PDQDJLQJWKHOLEUDU\ERRNV 8VHV >5(67+773@ 'HOHJDWHVDXWKHQWLFDWLRQWR 2$XWK&OLHQW 2$XWK$XWKRUL]DWLRQ6HUYHU 8VHV Security context propagation ? Authorized access?
31. OAuth2 An authorization framework that enables an application (Client) to obtain limited access to a protected resource provided by another application (called Resource Server) on behalf of a user. thomasvitale.com @vitalethomas
32. Token Relay thomasvitale.com @vitalethomas %URZVHU (GJH6HUYLFH %RRN 6HUYLFH $FFHVV7RNHQ 6HVVLRQ&RRNLH 5HVRXUFH 6HUYHU $FFHVV7RNHQ 5HVRXUFH 6HUYHU $FFHVV7RNHQ .HHSVPDSSLQJ 6HVVLRQ!$FFHVV7RNHQ OAuth2
33. .H\FORDN >&RQWDLQHU:LOGIO\@ 3URYLGHVLGHQWLW\DQG DFFHVVPDQDJHPHQW ,QYHQWRU\6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHVIXQFWLRQDOLW\IRU PDQDJLQJWKHERRNVKRS LQYHQWRU\ 2UGHU6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHVIXQFWLRQDOLW\IRU PDQDJLQJERRNRUGHUV 3RODU%RRNVKRS >6RIWZDUH6\VWHP@ 8VHV >5(67+773@ 8VHV >5(67+773@ (GJH6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHV$3,JDWHZD\DQG FURVVFXWWLQJFRQFHUQV 8VHU >3HUVRQ@ $QHPSOR\HHRIWKH ERRNVKRS 8VHV %RRN6HUYLFH >&RQWDLQHU6SULQJ%RRW@ 3URYLGHVIXQFWLRQDOLW\IRU PDQDJLQJWKHOLEUDU\ERRNV 8VHV >5(67+773@ 'HOHJDWHVDXWKHQWLFDWLRQWR 2$XWK&OLHQW 2$XWK$XWKRUL]DWLRQ6HUYHU 8VHV 2$XWK5HVRXUFH6HUYHU 2$XWK5HVRXUFH6HUYHU 2$XWK5HVRXUFH6HUYHU { "iss": “keycloak", "sub": "isabelle", "exp": 1626439022 } Access Token Access Token
34. Discount codes Manning • 45% discount code, valid for all products in all format • ctwspringio22 • manning.com
35. https://github.com/ThomasVitale/awesome-spring
36. Resources Source code • Sample project: • https://github.com/ThomasVitale/spring-io-2022-spring-cloud-gateway • Spring Cloud Gateway: • https://spring.io/projects/spring-cloud-gateway • Spring Security, OAuth2, OpenID Connect: • https://www.youtube.com/watch?v=g7Dwv1BKnkg
37. Thomas Vitale Spring I/O May 26th, 2022 Spring Cloud Gateway Resilience, Security, and Observability @vitalethomas