Designing Resilient Platforms with Workflow Orchestration

Transcript

1. Mehmet Sezer Designing Resilient Platforms with Workflow Orchestration
2. About me
3. Reconciliation Process Developer I need a database Desired State
4. Platform Desired State Current State Reconciliation Process Observe Adjust
5. Resilient Requirements Replication Database Creation
6. Requirements Database Creation
7. Database Creation if there is a problem it can TAKE WEEKS to create Can run parallel cluster creation for different regions
8. Requirements Replication
9. Replication Istanbul Ankara replication
10. Replication Can create replication between multi-region clusters Need to WAIT for two clusters to be created successfully
11. Resilient Requirements
12. Resilient Need to overcome challenges to ensure EVENTUALLY success
13. How? Resilient Requirements Replication Database Creation
14. Let’s Code
15. Traditional Programming
16. Traditional Programming what if it takes days?
17. Traditional Programming what if it returns error?
18. Kafka Event streaming platform Publish, subscribe, and process messages efficiently. Ideal for building data pipelines and event-driven systems, ensuring high throughput and reliability.
19. Kafka
20. Kafka Need to store database created events to understand all region of the cluster is created. Race condition (what happen if two events come in same time?)
21. Kafka Still need a custom error management
22. Kafka Need to manage Kafka components
23. Kafka Cognitive Load
24. Business Process Modeling Notation(BPMN) Flow chart method that models the steps of a planned business process from end to end
25. Create Replication Activity Istanbul Database Creation Activity Ankara Database Creation Activity Done Developer Request Multi- Region Database Business Process Modeling Notation(BPMN)
26. Temporal Temporal is an open-source workflow orchestration platform. State Persistence Built-in support for retries, timeouts, scheduled and long-running tasks. Support: Go, Java, PHP, Python, TypeScript .NET, Clojure, Scala, Ruby
27. Temporal
28. Temporal
29. Temporal
30. Temporal States Scheduled Scheduled Execution Started Started Started Execution Completed Completed Completed Execution Failed failed failed Ready to process, worker can poll and start execution
31. Temporal
32. Temporal
33. Temporal Execution of code is stopped Will continue from BEGINNING when an activity is completed.
34. Temporal Both activities worked successfully
35. Event Status Payload Istanbul Region Database Creation Completed ... Ankara Region Database Creation Completed ... Temporal Temporal Event History Example
36. Event Status Payload Istanbul Region Database Creation Completed ... Ankara Region Database Creation Completed ... Temporal Temporal Event History Example
37. Event Status Payload Istanbul Region Database Creation Completed ... Ankara Region Database Creation Completed ... Temporal Event History Example Temporal
38. Event Status Payload Istanbul Region Database Creation Completed ... Ankara Region Database Creation Completed ... Temporal Event History Example Temporal
39. Event Status Payload Istanbul Region Database Creation Completed ... Ankara Region Database Creation Completed ... Temporal Event History Example Temporal
40. Event Status Payload Istanbul Region Database Creation Completed ... Ankara Region Database Creation Completed ... Temporal Event History Example Temporal
41. Event Status Payload Istanbul Region Database Creation Completed ... Ankara Region Database Creation Completed ... Replication Completed ... Temporal Temporal Event History Example
42. Temporal vs Traditional
43. Temporal Still can take long-time
44. Temporal Need to create single-region database
45. Temporal
46. Temporal
47. Temporal
48. Temporal
49. Temporal
50. Temporal
51. Temporal
52. mehmetsezerr Thank You mhmtszr Interested in joining us? mehmet.sezer