Bulletproof communication in distributed systems

Transcript

1. @jkubrynski / kubrynski.com @jkubrynski / kubrynski.com BULLETPROOF BULLETPROOF COMMUNICATION COMMUNICATION JAKUB KUBRYNSKI JAKUB KUBRYNSKI jk@devskiller.com / @jkubrynski / kubrynski.blog
2. $ WHOAMI $ WHOAMI DEVSKILLER CO-FOUNDER DEVSKILLER CO-FOUNDER BOTTEGA TRAINER BOTTEGA TRAINER DEVOXX.PL PROGRAM COMMITTEE MEMBER DEVOXX.PL PROGRAM COMMITTEE MEMBER SPRING CLOUD CONTRACT CO-AUTHOR SPRING CLOUD CONTRACT CO-AUTHOR
3. DEVSKILLER IS LOOKING FOR DEVSKILLER IS LOOKING FOR HELLO-WORLD DEVELOPERS HELLO-WORLD DEVELOPERS
4. WHY DISTRIBUTED SYSTEMS? WHY DISTRIBUTED SYSTEMS? SCALABILITY SCALABILITY POLYGLOT POLYGLOT RESILIENCE RESILIENCE
5. DISTRIBUTED SYSTEMS FALLACIES DISTRIBUTED SYSTEMS FALLACIES THE NETWORK IS RELIABLE THE NETWORK IS RELIABLE LATENCY IS ZERO LATENCY IS ZERO THE NETWORK IS SECURE THE NETWORK IS SECURE TOPOLOGY DOESN'T CHANGE TOPOLOGY DOESN'T CHANGE
6. IF YOU WON'T RESPECT FALLACIES IF YOU WON'T RESPECT FALLACIES YOU'LL BUILD DISTRIBUTED MONOLITH YOU'LL BUILD DISTRIBUTED MONOLITH
7. BRACE YOURSELF BRACE YOURSELF FAILURE IS COMING FAILURE IS COMING
8. IN MONOLITH ALL COLLABORATORS ARE IN MONOLITH ALL COLLABORATORS ARE ALWAYS AVAILABLE ALWAYS AVAILABLE
9. IN DISTRIBUTED SYSTEMS THEY ARE NOT IN DISTRIBUTED SYSTEMS THEY ARE NOT NETWORK ISSUES NETWORK ISSUES REDEPLOYS REDEPLOYS OUTAGES OUTAGES
10. AVAILABILITY VS DOWNTIME AVAILABILITY VS DOWNTIME 9 9 . 9 9 9 % 35d 4d 8h 50m 5m per year 2.5h 14m 1.5m 8s 0.8s per day
11. 99.9 (1.5 MIN/DAY) 99.9 (1.5 MIN/DAY) 5 * 99.9 => 99.5 (7 MIN/DAY) 5 * 99.9 => 99.5 (7 MIN/DAY)
12. TEMPORAL COUPLING TEMPORAL COUPLING
13. DESIGN FOR FAILURE DESIGN FOR FAILURE
14. PREFER ASYNCHRONOUS PREFER ASYNCHRONOUS OVER SYNCHRONOUS OVER SYNCHRONOUS
15. ASYNCHRONOUS ACTORS ASYNCHRONOUS ACTORS COMMANDS (QUEUES) COMMANDS (QUEUES) EVENTS (TOPICS) EVENTS (TOPICS)
16. ASYNCHRONOUS PRINCIPLES ASYNCHRONOUS PRINCIPLES PERSISTENT PERSISTENT AT-LEAST-ONCE DELIVERY AT-LEAST-ONCE DELIVERY EXPONENTIAL RETRIES EXPONENTIAL RETRIES
17. SYNCHRONOUS SYNCHRONOUS IS USED ONLY WHEN REQUIRED IS USED ONLY WHEN REQUIRED
18. WHAT COULD POSSIBLY GO WRONG? WHAT COULD POSSIBLY GO WRONG? ¯\_(ツ)_/¯ ¯\_(ツ)_/¯
19. TIMEOUT?!?! TIMEOUT?!?! RETRY RETRY
20. CIRCUIT BREAKER CIRCUIT BREAKER KEEP CALM AND FALLBACK KEEP CALM AND FALLBACK
21. IN MONOLITH TOPOLOGY IS FIXED IN MONOLITH TOPOLOGY IS FIXED
22. IN DISTRIBUTED SYSTEMS IT'S NOT IN DISTRIBUTED SYSTEMS IT'S NOT CLOUD NATIVE / CONTAINERS CLOUD NATIVE / CONTAINERS DYNAMIC SCALABILITY DYNAMIC SCALABILITY
23. ALWAYS USE SERVICE DISCOVERY ALWAYS USE SERVICE DISCOVERY
24. LOAD BALANCERS SUCKS LOAD BALANCERS SUCKS IT'S SINGLE POINT OF FAILURE IT'S SINGLE POINT OF FAILURE
25. ALWAYS USE CLIENT SIDE LOAD BALACING ALWAYS USE CLIENT SIDE LOAD BALACING
26. IT'S ALL ABOUT COMMUNICATION IT'S ALL ABOUT COMMUNICATION SO NICE TO KNOW IT WORKS SO NICE TO KNOW IT WORKS
27. DISTRIBUTED SYSTEMS DISTRIBUTED SYSTEMS VS VS MICROSERVICES MICROSERVICES
28. MICROSERVICES BRING MICROSERVICES BRING AUTONOMY AUTONOMY
29. AUTONOMY IS HARD AUTONOMY IS HARD BECAUSE WE NEED TO COORDINATE DEPENDENCIES BECAUSE WE NEED TO COORDINATE DEPENDENCIES AND BE BACKWARD COMPATIBLE AND BE BACKWARD COMPATIBLE
30. LOOSE COUPLING LOOSE COUPLING OVER OVER CANONICAL MODEL CANONICAL MODEL
31. END-TO-END TESTS ARE HARD END-TO-END TESTS ARE HARD FALSE-NEGATIVE FALSE-NEGATIVE FALSE-POSITIVE FALSE-POSITIVE
32. VERIFY CONTRACTS VERIFY CONTRACTS A == C && B == C A == C && B == C A == B A == B
33. CONTRACT CONVERTS TO CONTRACT CONVERTS TO SERVER TEST SERVER TEST CLIENT STUB CLIENT STUB
34. PERFORMANCE? PERFORMANCE?
35. MONITORING IS THE NEW TESTING MONITORING IS THE NEW TESTING
36. MONITORING MONITORING EXCEPTIONS EXCEPTIONS BUSINESS METRICS BUSINESS METRICS APPLICATION PERFORMANCE MANAGEMENT APPLICATION PERFORMANCE MANAGEMENT
37. ANSCOMBE'S QUARTET ANSCOMBE'S QUARTET Mean of x => 9 Sample variance of x => 11 Mean of y => 7.50 Sample variance of y => 4.125 Correlation => 0.816 Linear regression => y = 3.00 + 0.500x
38. PERCENTILES PERCENTILES P50, P90, P95, P99 P50, P90, P95, P99
39. MONITOR MONITOR P99.9 OR P99.95 P99.9 OR P99.95
40. THANKS THANKS
41. DROGA.DEV DROGA.DEV
42. QUESTIONS? QUESTIONS?
43. "Have no fear of perfection - you'll never reach it." ― Salvador Dalí