.NET Day 2023: Fault Handling with retry and circuit breaker design patterns

Polly can improve the customers' user experience in mobile and web applications. But do you know you can also use Polly in microservices communication? In this talk, I will show you a step-by-step process of building two simple .NET 6 Services and working with the Polly library to ensure we can handle any transient faults that may occur when one service calls the other. I will start with different types of exposing your microservices like direct, facade, and API gateway. Then what is Retry Policy? What is Circuit Breaker Policy? What is Infrastructure-based resiliency? And how to implement them in DotNET apps.

Transcript

4. Fault handling with retry and circuit breaker design patterns
5. • Exposing Microservices • Retry Policy • Circuit Breaker Pattern • Demo of Retry Policy and Circuit Breaker Pattern in .NET 7 web apps Agenda
9. Why web service requests fail?
11. • infrastructure upgrade • faulty network connection • null reference on someone else’s code • server unavailable • bad deployment • and more
12. Retry Policy
17. ✅
18. ✔️ ❌
19. downstream microservice failure (aka a ‘dependency failure’)
21. ??? 3 ways to expose microservices 1. Directly 2. Façade(s) 3. API Gateway
22. Option 1. Direct Access
23. Option 1. Direct Access Pros: • No additional components – lazy approach • Backward compatibility is addressed in each microservice – a little less work
24. Option 1. Direct Access Pros: • No additional components – lazy approach • Backward compatibility is addressed in each microservice – a little less work Cons: • Multiple entry points – extra complexity • The internal architecture is directly exposed to the world – evolution is difficult to impossible • There is no clear perimeter; every microservice must be secured, adding extra latency and complexity.
25. Option 1. Direct Access When to use: • Don’t • TU es nicht
27. FAÇADE Option 2-A. Single Universal Façade
28. FAÇADE Option 2-A. Single Universal Façade Pros: • The internal architecture is hidden – easy evolution • Single entry point – simpler and better security • Place to tailor and optimize external requests • Consistent development and delivery practices • Platform agnostic – will work anywhere • Feature-rich – multi-protocol, async notifications and more
29. FAÇADE Option 2-A. Single Universal Façade Pros: • The internal architecture is hidden – easy evolution • Single entry point – simpler and better security • Place to tailor and optimize external requests • Consistent development and delivery practices • Platform agnostic – will work anywhere • Feature-rich – multi-protocol, async notifications and more Cons: • Extra work to implement • Constant changes
30. BFF 2 BFF 1 App 1 App 2 Option 2-B. Backend For Frontend (BFF)
31. BFF 2 BFF 1 App 1 App 2 Option 2-B. Backend For Frontend (BFF) Pros: • The internal architecture is hidden – easy evolution • Single entry point – simpler and better security • Consistent development and delivery practices • Platform agnostic – will work anywhere • Feature-rich – multi-protocol, async notifications and more • Extra optimization of application requests
32. BFF 2 BFF 1 App 1 App 2 Option 2-B. Backend For Frontend (BFF) Pros: • The internal architecture is hidden – easy evolution • Single entry point – simpler and better security • Consistent development and delivery practices • Platform agnostic – will work anywhere • Feature-rich – multi-protocol, async notifications and more • Extra optimization of application requests Cons: • Extra work to implement
33. FAÇADE 1 FAÇADE 2 Option 2-C. Multiple Façades
34. FAÇADE 1 FAÇADE 2 Pros: • The internal architecture is hidden – easy evolution • Consistent development and delivery practices • Platform agnostic – will work anywhere • Feature-rich – multi-protocol, async notifications, and more • Extra optimization of application requests • Smaller APIs – easier changes Option 2-C. Multiple Façades
35. FAÇADE 1 FAÇADE 2 Pros: • The internal architecture is hidden – easy evolution • Consistent development and delivery practices • Platform agnostic – will work anywhere • Feature-rich – multi-protocol, async notifications, and more • Extra optimization of application requests • Smaller APIs – easier changes Cons: • Multiple entry points - higher complexity • Extra work to implement Option 2-C. Multiple Façades
36. API Gateway Option 3. API Gateway
37. API Gateway Option 3. API Gateway Cloud-native: • AWS API Gateway • Microsoft Azure API Management Simple free cloud agnostic: • KongHQ • Nginx Plus API Management: • Apigee (Google) • 3scale (Red Hat)
42. API Gateway Option 3. API Gateway Pros: • The internal architecture is hidden – easy evolution • Single entry point – simpler and better security • Prebuilt cross-cutting services – less work to add authorization, throttling, tracing, audit logging • Pre-integrated with cloud services (cloud only) • Extra security – feature reach well tested 3rd party implementation • API productization - website, documentation, SDK generation, and more
43. API Gateway Option 3. API Gateway Pros: • The internal architecture is hidden – easy evolution • Single entry point – simpler and better security • Prebuilt cross-cutting services – less work to add authorization, throttling, tracing, audit logging • Pre-integrated with cloud services (cloud only) • Extra security – feature reach well tested 3rd party implementation • API productization - website, documentation, SDK generation, and more Cons: • Extra technology to learn and maintain • Limited functionality – hard to support complex processing, multi-protocol or async notifications • Inconsistent development and delivery practices
44. Terminologies
45. Retry Policy A Retry policy is exactly what the name implies. The request is retried after a short wait if an error response is received. The wait time is configurable as shown in the following table: Wait time value Example sequence Constant 2s, 2s, 2s, 2s… Exponentially increasing 2s 4s, 8s, 16s… Specified list 100ms, 100ms, 100ms, 2s, 4s. 8s
46. Circuit Breaker Policy A Circuit Breaker policy gives the target service a break after a repeated number of failures. Such a state might indicate that the service is experiencing a serious problem and is temporarily unable to response. After a defined number of consecutive failures, the connection attempts are paused temporarily, “opening” the circuit.
47. Closed: In electrical circuits, when a circuit breaker is closed indicates that electric current is passing. In the resilience pattern it means that you are passing “requests”
48. Open: in this state, the circuit does not pass the requests to the destination, only returns to the requester that the service is out, avoiding to overload the destination with new requests, besides providing feedback faster, without having to wait for the end of a request that probably will fail. The circuit goes to the open state after reaching a predefined threshold of faults and will also remain open for a predefined time.
49. Half-Open: This is a check state. After reaching the time in the Open state, the system goes to this state and the next request will be forwarded to the destination. If the destination returns a fault, the circuit returns to the Open state, if it returns successfully it goes to the Closed state.
50. Infrastructure-based resiliency To implement infrastructure-based resiliency, you can use a service mesh. • A management interface, which could be a UI or an API • For example, Istio, with Istio you can define retry policies with a traffic rule, offloading this logic to your services’ sidecar proxies
53. Code-based resiliency To implement code-based resiliency, you can write a logic or use a 3rd party library for resilience and transient failure handling. github.com/App-vNext/Polly
54. API Gateway FAÇADE 1 FAÇADE 2 Retry policy Retry policy Retry policy
55. DEMO
56. Other Usage Scenarios of a Retry Policy • Reading/writing to file system • Using databases • Reading/writing to message queue
57. thepollyproject.org/2023/03/03/we-want-your-feedback-introducing-polly-v8 Upcoming version
58. Summary Retry Policy - the request is retried after a short wait if an error response is received. Circuit Breaker Policy - a Circuit Breaker policy gives the target service a break after a repeated number of failures.
59. Resiliency approaches There are two fundamental approaches to resiliency: • Infrastructure • Code Review resiliency concepts Resiliency is the ability to recover from transient failures Summary
60. Library Polly Demo github.com/webmasterdevlin/PollyDemo
61. Danke @DevlinDuldulao