Re-architect for the Cloud - Cloud Expo Europe

Your move to Cloud doesn't have to be lift&shift! You get the most benefits of the Cloud, re-architecting to become "just enough" Cloud-Native is the way to go. Learn more about this approach, illustrated with two little experience reports, and borrowing from Domain-Driven Design with bounded contexts.

Transcript

1. RÉ-ARCHITECTURER VOTRE EXISTANT POUR MAXIMISER LE BÉNÉFICE DU CLOUD CHALLENGES & OPPORTUNITIES @AROLLAFR Cyrille Martraire Marouane Benamara
2. @arollafr We want to move to cloud!
3. @arollafr "Datacenter -Zero" by Q4!
4. @arollafr WHY?
5. @arollafr We want to scale the business 5x! #elastic #BlackFridayProof
6. @arollafr We want to reduce cost 50%! #PayAsYouGo #costeffective #opex
7. @arollafr Managing boxes is NOT our CORE expertise! #secure #reliable #current #global (It’s someone else’s expertise)
8. @arollafr We want to deliver new features 3x faster! #time2market #self-service-provisioning #pre-approved-budget
9. @arollafr We want to innovate quick & fast! #cheap-experiment #reversible
10. Cloud-benefits: how to catch them all?
11. how it started how it’s going
12. full cloud lift&shift
13. SINCE 1999 CYRILLE MARTRAIRE @CYRIUX @AROLLAFR CO-FOUNDER & CTO AROLLA DEVELOPER, CONSULTANT, TRAINER FOUNDER PARIS SOFTWARE CRAFTERS SPEAKER & AUTHOR
14. @cyriux BUY MY BOOKS!
15. MAROUANE BENAMARA @CYRIUX @AROLLAFR SENIOR AROLLA DEVELOPER AZURE & AZURE SERVERLESS ARCHITECT 10+ YEARS COMMERCIAL EXPERIENCE
16. arolla.fr @arollafr TDD | BDD | DDD
17. arolla.fr @arollafr crafting architecture
18. arolla.fr @arollafr Trainings Consulting Contracting
19. @arollafr ideal cloud = ?
20. CLOUD-NATIVE
21. Energy Flexibility Trading QUICK CLOUD-NATIVE EXPERIENCE REPORT
22. Vehicle to Grid Technology (V2G)
23. 23 Inject energy from vehicles into the grid
24. 24 Value at Stake: innovating! Never done before? Feasibility? How to make it work?
25. 25 Value at Stake: innovating! Never done before? Feasibility? How to make it work? Proof of Concept!
26. 26 Serverless + some PaaS
27. FROM ZERO TO DEMO IN 1 MONTH Setup CI/CD (Azure Devops): a few days Provisioning Function + DB: one half-day
28. @arollafr Deliver new features fast! #Innovate #time2market #SafeToFail #Experimentations
29. CLOUD-NATIVE ARCHITECTURE
30. CLOUD-NATIVE ARCHITECTURE Fine-grained modularity Requests Scheduling #NotMonolithic
31. CLOUD-NATIVE ARCHITECTURE Fine-grained update-ability Requests Scheduling Deploy #NotMonolithic
32. CLOUD-NATIVE ARCHITECTURE Fine-grained update-ability Requests Scheduling Deploy #NotMonolithic
33. CLOUD-NATIVE ARCHITECTURE Fine-grained modularity Async communication Serverless bus #LooselyCoupled
34. CLOUD-NATIVE ARCHITECTURE Fine-grained modularity Async communication Read-models WRITE READ #LooselyCoupled
35. @arollafr Speed! #CloudNative #PaaS #Serverless
36. @arollafr Why these two boundaries?
37. Evolve for different reasons DIFFERENT EVOLUTION, DIFFERENT AREAS Requests Scheduling like "Purchasing" like "Shipping"
38. Homogenous by technical pressure HOMOGENOUS AREAS Requests <<interactive>> <<High-Availability>> <<transactional>> <<low-volume>> <<always-on>> Scheduling <<batch>> <<low-downtime>> <<stop&go>> <<optimisation>> <<medium-volume>>
39. Homogenous by technical pressure HOMOGENOUS AREAS Requests <<interactive>> <<High-Availability>> <<transactional>> <<low-volume>> <<always-on>> Scheduling <<batch>> <<low-downtime>> <<stop&go>> <<optimisation>> <<medium-volume>>
40. Homogenous by technical pressure HOMOGENOUS AREAS Requests <<interactive>> <<High-Availability>> <<transactional>> <<low-volume>> <<always-on>> Scheduling <<batch>> <<low-downtime>> <<stop&go>> <<optimisation>> <<medium-volume>>
41. Homogenous by technical pressure HOMOGENOUS AREAS Requests <<interactive>> <<High-Availability>> <<always-on>> <<transactional>> <<low-volume>> Scheduling <<batch>> <<low-downtime>> <<stop&go>> <<optimisation>> <<medium-volume>>
42. @arollafr How to find them? #ServicesBoundaries
43. @arollafr @CYRIUX
44. BOUNDED CONTEXTS @CYRIUX
45. BOUNDED CONTEXTS @CYRIUX since 2003
46. BOUNDED CONTEXTS @CYRIUX before it was cool
47. @arollafr FlexRequest Booking Response Partial Response Future Capacity Estimated Flexibility Calendar Slot (Asset) Charge Point Demand Response Command Power (state of charge) Telemetry Set Point (target) Estimated Flexibility Calendar Slot Priority Requests <<interactive>> <<High-Availability>> <<fast>> <<transactional>> <<low-volume>> Scheduling <<batch>> <<low-downtime>> <<stop&go>> <<optimisation>> <<medium-volume>> … DOMAIN LANGUAGE!
48. @arollafr FlexRequest Booking Response Partial Response Future Capacity Estimated Flexibility Calendar Slot (Asset) Charge Point Demand Response Command Power (state of charge) Telemetry Set Point (target) Estimated Flexibility Calendar Slot Priority Requests <<interactive>> <<High-Availability>> <<fast>> <<transactional>> <<low-volume>> Scheduling <<batch>> <<low-downtime>> <<stop&go>> <<optimisation>> <<medium-volume>> … CLUSTERS OF WORDS
49. @arollafr FlexRequest Booking Response Partial Response Future Capacity Estimated Flexibility Calendar Slot (Asset) Charge Point Demand Response Command Power (state of charge) Telemetry Set Point (target) Estimated Flexibility Calendar Slot Priority Requests <<interactive>> <<High-Availability>> <<fast>> <<transactional>> <<low-volume>> Scheduling <<batch>> <<low-downtime>> <<stop&go>> <<optimisation>> <<medium-volume>> … SUGGEST SUB-DOMAIN Booking Scheduling
50. By sub-domain DERIVE THE TECHNICAL PRESSURES Requests <<interactive>> <<High-Availability>> <<always-on>> <<transactional>> <<low-volume>> Scheduling <<batch>> <<low-downtime>> <<stop&go>> <<optimisation>> <<medium-volume>>
51. BOUNDED CONTEXTS @CYRIUX since 2003 - plenty of experience & heuristics now!
52. @arollafr grrrrrr okay
53. @arollafr But we have a legacy monolith!
54. 54 Second-hand car trading (B2B) QUICK LEGACY-TO-CLOUD EXPERIENCE REPORT
55. Sales Fixed price (to first buyer) Auctions (à la eBay) Hidden price (…)
56. 375k vehicles 1000’s of buyers/sellers (before the project)
57. 57 Value at Stake: growth! Data Volume++ Availability++ (to 99,99%) Scalability++ Performance++
58. 58 Value at Stake: growth! Data Volume++ Availability++ (to 99,99%) Scalability++ Performance++ Monolith says NO!
59. 59 PaaS as target
60. #SUCCESS! x2 Volume New features++ (ML recommendation…)
61. Typical On-Premise Legacy Web Server - IIS Windows Services SQL Server DB •Deployment requests 48h before •Deployment document to write •Heavyweight process: • 3 approvals, 5 validations • 8 firewalls, 14 hops, 62 proxies to access production machines A legacy monolith A legacy process
62. Typical On-Premise Legacy Web Server - IIS Windows Services SQL Server DB •Recurring performance problems •Almost no monitoring •Regular unplanned downtime
63. @arollafr fedakv.medium.com/ MOVE 2 CLOUD Scalability High-availability
64. RE- ARCHITECTING for the PaaS!
65. A MONOLITH… "Car Trading System"
66. IDENTIFY THE SUB-DOMAINS… Front-Of fi ce Back-Of fi ce
67. SPLIT IN 2 FOR THE CLOUD Front-Of fi ce Back-Of fi ce APP Services <<PaaS>>
68. FROM DB-POLLING TO ASYNC QUEUE Front-Of fi ce Back-Of fi ce Use data Provide data Serverless bus
69. Front-Of fi ce Back-Of fi ce APP Services <<PaaS>> Too much work/not enough bene fi ts to go full PasS HALF PAAS / HALF IAAS KEEP AS-IS EXTRACT & ADAPT <<IaaS>>
70. OF COURSE, IT WENT WRONG
71. 1- MULTIPLE INSTANCES + SESSIONS = ❤ Front-Of fi ce Session state Session state Session state
72. STATEFUL → STATELESS Front-Of fi ce Session state Azure Redis #win
73. 2 - BIG QUERIES WORKED ON-PREM Front-Of fi ce SELECT p.ID , LAG (p.ID, 1) OVER (ORDER BY p.OwnerUserId DESC) AS PreviousID , LEAD (p.ID, 1) OVER (ORDER BY p.OwnerUserId DESC) AS NextI D FROM Posts p JOIN Users u ON u.Id = p.OwnerUserI d WHERE DisplayName = 'CecilPhillip ' AND p.PostTypeId = 1 AND p.OwnerUserId = 333082 Web Server - IIS Windows Services SQL Server DB Co-colocated
74. BIG SQL QUERIES TOO SLOW ON PAAS DB Front-Of fi ce SELECT p.ID , LAG (p.ID, 1) OVER (ORDER BY p.OwnerUserId DESC) AS PreviousID , LEAD (p.ID, 1) OVER (ORDER BY p.OwnerUserId DESC) AS NextI D FROM Posts p JOIN Users u ON u.Id = p.OwnerUserI d WHERE DisplayName = 'CecilPhillip ' AND p.PostTypeId = 1 AND p.OwnerUserId = 333082 SQL Always-On Cluster
75. QUICK WINS FOR PERFORMANCE! Azure CDN for big media Replace old cache with Azure Redis Cache Cron tasks moved to background jobs Pre-fetch data in read models (caches) via message bus #Cloud-powered
76. QUICK WINS FOR AUTOMATED DELIVERY! Proving scalability with automated performance tests Zero-Downtime deployment with Blue-Green deployment (using built-in Cloud "slots") #Cloud-powered
77. 77 Split into 2-3 parts. Add queues/event bus. Switch stores. Automate CI/CD. Enjoy!
78. Pareto
79. Minimum Viable Cloud-Native re-Architecting
80. @arollafr New features faster!
81. New type of auction New vehicle-aware search-engine New vehicle-aware search-engine (using another, better-suited Cloud service) Business logs streaming Log Storage & Exploitation NEW FEATURES FASTER (FROM NOW ON)
82. MORE OPTIONS TO EVOLVE Event bus
83. MORE OPTIONS TO EVOLVE Event bus Ne w Service
84. MORE OPTIONS TO EVOLVE Event bus Saa S Service Ne w Service
85. New Architectural Opportunities
86. CLOUD = OPTIONS TO GO FASTER Self-Service Provisioning: Try stuff yourself No upfront cost: Try stuff cheap API or EVENTS everywhere to plug new stuff easily A lot of ready-made stuff The autonomy to JUST DELIVER with minimal coordination
87. @arollafr Re-architect faster Ongoing, while delivering new value
88. Back-Office progressively moved to Functions SQL DB progressively replaced by PaaS or Serverless storages More async messaging everywhere ONGOING RE-ARCHITECTING, (FROM NOW ON)
89. Just enough re-Architecting
90. @arollafr BIG-SPLIT RE-ARCHITECTING SYNDROME $3M 1 YEAR 7 PEOPLE 100% IT4IT SPLIT INTO 5 DOCKER CONTAINERS #wasteful-effort #capex-heavy
91. @arollafr BIG-SPLIT RE-ARCHITECTING SYNDROME $3M 1 YEAR 7 PEOPLE 100% IT4IT SPLIT INTO 5 DOCKER CONTAINERS #wasteful-effort #capex-heavy "Re-architecting sucks"
92. Ok.
93. @arollafr We are not fast with our cloud!
94. BEWARE CLOUD POISONS! Don’t degrade the time-to-market with process Pre-approved budget & own (sub)-account Pre-approved list of cloud services & patterns you can use Self-service provisioning in (no ticketing…) No mandatory technical layer on top of each service
95. @arollafr I don’t want to be locked to a cloud provider!
96. PORTABILITY BY DESIGN AWS SNS S3 Azure Blob Storage Service Bus e.g. hexagonal architecture, GUID, decoupling identity management What’s good for testing is also great for portability Dev+Test Test Robot Mocks
97. @arollafr Ok, but now… how to up-skill the teams?
98. LOTS OF SKILLS REQUIRED! Distributed Architecture Tolerant contracts Blue-Green Deployment Stateless EDA Correlation ID / GUID Modularity Tech Pressure Areas Bounded Contexts Evolution Pressure Areas Legacy Split Concerns Lazy Loading Large Transactions Temporal Coupling Shared "Uber" DB Migration Concerns Shared auth/ directory Slow queries/join Strangler App+Sync mechanism Cloud- native Patterns Read- models Async everywhere Fast & Reversible Direct Access download /upload Lean but not Naive Frugality Last Responsible Moment Keep Options Open Trying over Thinking Idempotence Extract Repair Rewrite Replace Hexagonal Architecture Change Data Capture Data Polling DB Views
99. THE RIGHT SKILLS FOR THE RIGHT GROUP Tolerant contracts Blue-Green Deployment Stateless EDA Correlation ID / GUID Tech Pressure Areas Bounded Contexts Large Transactions Shared "Uber" DB Strangler App+Sync mechanis m Read- models Direct Access download /upload Last Responsible Moment Keep Options Open Idempotence Extract Repair Rewrite Replace Hexagonal Architecture Change Data Capture Data Polling Group 1 Group 2 Group 3 Tolerant contracts Stateless EDA Correlation ID / GUID
100. @cyriux LEARNING THROUGH SPECIFICALLY DESIGNED EXERCISES
101. @arollafr IN CLOSING
102. Cloud is no more about hosting
103. New Architectural Opportunities
104. LIFT & SHIFT IS NOT ENOUGH FULL RE-ARCHITECTING IS TOO MUCH DDD BOUNDED CONTEXT CAN HELP EMBRACE PAAS/SERVERLESS ASAP TO GO FASTER EVERYTHING!
105. Minimum Viable Cloud-Native re-Architecting
106. THE END THANK YOU @AROLLA @CYRIUX