Re-architect for the Cloud - Cloud Expo Europe

Slide 1

Slide 1 text

RÉ-ARCHITECTURER VOTRE EXISTANT POUR MAXIMISER LE BÉNÉFICE DU CLOUD CHALLENGES & OPPORTUNITIES @AROLLAFR Cyrille Martraire Marouane Benamara

Slide 2

Slide 2 text

@arollafr We want to move to cloud!

Slide 3

Slide 3 text

@arollafr "Datacenter -Zero" by Q4!

Slide 4

Slide 4 text

@arollafr WHY?

Slide 5

Slide 5 text

@arollafr We want to scale the business 5x! #elastic #BlackFridayProof

Slide 6

Slide 6 text

@arollafr We want to reduce cost 50%! #PayAsYouGo #costeffective #opex

Slide 7

Slide 7 text

@arollafr Managing boxes is NOT our CORE expertise! #secure #reliable #current #global (It’s someone else’s expertise)

Slide 8

Slide 8 text

@arollafr We want to deliver new features 3x faster! #time2market #self-service-provisioning #pre-approved-budget

Slide 9

Slide 9 text

@arollafr We want to innovate quick & fast! #cheap-experiment #reversible

Slide 10

Slide 10 text

Cloud-benefits: how to catch them all?

Slide 11

Slide 11 text

how it started how it’s going

Slide 12

Slide 12 text

full cloud lift&shift

Slide 13

Slide 13 text

SINCE 1999 CYRILLE MARTRAIRE @CYRIUX @AROLLAFR CO-FOUNDER & CTO AROLLA DEVELOPER, CONSULTANT, TRAINER FOUNDER PARIS SOFTWARE CRAFTERS SPEAKER & AUTHOR

Slide 14

Slide 14 text

@cyriux BUY MY BOOKS!

Slide 15

Slide 15 text

MAROUANE BENAMARA @CYRIUX @AROLLAFR SENIOR AROLLA DEVELOPER AZURE & AZURE SERVERLESS ARCHITECT 10+ YEARS COMMERCIAL EXPERIENCE

Slide 16

Slide 16 text

arolla.fr @arollafr TDD | BDD | DDD

Slide 17

Slide 17 text

arolla.fr @arollafr crafting architecture

Slide 18

Slide 18 text

arolla.fr @arollafr Trainings Consulting Contracting

Slide 19

Slide 19 text

@arollafr ideal cloud = ?

Slide 20

Slide 20 text

CLOUD-NATIVE

Slide 21

Slide 21 text

Energy Flexibility Trading QUICK CLOUD-NATIVE EXPERIENCE REPORT

Slide 22

Slide 22 text

Vehicle to Grid Technology (V2G)

Slide 23

Slide 23 text

23 Inject energy from vehicles into the grid

Slide 24

Slide 24 text

24 Value at Stake: innovating! Never done before? Feasibility? How to make it work?

Slide 25

Slide 25 text

25 Value at Stake: innovating! Never done before? Feasibility? How to make it work? Proof of Concept!

Slide 26

Slide 26 text

26 Serverless + some PaaS

Slide 27

Slide 27 text

FROM ZERO TO DEMO IN 1 MONTH Setup CI/CD (Azure Devops): a few days Provisioning Function + DB: one half-day

Slide 28

Slide 28 text

@arollafr Deliver new features fast! #Innovate #time2market #SafeToFail #Experimentations

Slide 29

Slide 29 text

CLOUD-NATIVE ARCHITECTURE

Slide 30

Slide 30 text

CLOUD-NATIVE ARCHITECTURE Fine-grained modularity Requests Scheduling #NotMonolithic

Slide 31

Slide 31 text

CLOUD-NATIVE ARCHITECTURE Fine-grained update-ability Requests Scheduling Deploy #NotMonolithic

Slide 32

Slide 32 text

CLOUD-NATIVE ARCHITECTURE Fine-grained update-ability Requests Scheduling Deploy #NotMonolithic

Slide 33

Slide 33 text

CLOUD-NATIVE ARCHITECTURE Fine-grained modularity Async communication Serverless bus #LooselyCoupled

Slide 34

Slide 34 text

CLOUD-NATIVE ARCHITECTURE Fine-grained modularity Async communication Read-models WRITE READ #LooselyCoupled

Slide 35

Slide 35 text

@arollafr Speed! #CloudNative #PaaS #Serverless

Slide 36

Slide 36 text

@arollafr Why these two boundaries?

Slide 37

Slide 37 text

Evolve for different reasons DIFFERENT EVOLUTION, DIFFERENT AREAS Requests Scheduling like "Purchasing" like "Shipping"

Slide 38

Slide 38 text

Homogenous by technical pressure HOMOGENOUS AREAS Requests <> <> <> <> <> Scheduling <> <> <> <> <>

Slide 39

Slide 39 text

Homogenous by technical pressure HOMOGENOUS AREAS Requests <> <> <> <> <> Scheduling <> <> <> <> <>

Slide 40

Slide 40 text

Homogenous by technical pressure HOMOGENOUS AREAS Requests <> <> <> <> <> Scheduling <> <> <> <> <>

Slide 41

Slide 41 text

Homogenous by technical pressure HOMOGENOUS AREAS Requests <> <> <> <> <> Scheduling <> <> <> <> <>

Slide 42

Slide 42 text

@arollafr How to find them? #ServicesBoundaries

Slide 43

Slide 43 text

@arollafr @CYRIUX

Slide 44

Slide 44 text

BOUNDED CONTEXTS @CYRIUX

Slide 45

Slide 45 text

BOUNDED CONTEXTS @CYRIUX since 2003

Slide 46

Slide 46 text

BOUNDED CONTEXTS @CYRIUX before it was cool

Slide 47

Slide 47 text

@arollafr FlexRequest Booking Response Partial Response Future Capacity Estimated Flexibility Calendar Slot (Asset) Charge Point Demand Response Command Power (state of charge) Telemetry Set Point (target) Estimated Flexibility Calendar Slot Priority Requests <> <> <> <> <> Scheduling <> <> <> <> <> … DOMAIN LANGUAGE!

Slide 48

Slide 48 text

Slide 49

Slide 49 text

Slide 50

Slide 50 text

By sub-domain DERIVE THE TECHNICAL PRESSURES Requests <> <> <> <> <> Scheduling <> <> <> <> <>

Slide 51

Slide 51 text

BOUNDED CONTEXTS @CYRIUX since 2003 - plenty of experience & heuristics now!

Slide 52

Slide 52 text

@arollafr grrrrrr okay

Slide 53

Slide 53 text

@arollafr But we have a legacy monolith!

Slide 54

Slide 54 text

54 Second-hand car trading (B2B) QUICK LEGACY-TO-CLOUD EXPERIENCE REPORT

Slide 55

Slide 55 text

Sales Fixed price (to first buyer) Auctions (à la eBay) Hidden price (…)

Slide 56

Slide 56 text

375k vehicles 1000’s of buyers/sellers (before the project)

Slide 57

Slide 57 text

57 Value at Stake: growth! Data Volume++ Availability++ (to 99,99%) Scalability++ Performance++

Slide 58

Slide 58 text

58 Value at Stake: growth! Data Volume++ Availability++ (to 99,99%) Scalability++ Performance++ Monolith says NO!

Slide 59

Slide 59 text

59 PaaS as target

Slide 60

Slide 60 text

#SUCCESS! x2 Volume New features++ (ML recommendation…)

Slide 61

Slide 61 text

Typical On-Premise Legacy Web Server - IIS Windows Services SQL Server DB •Deployment requests 48h before •Deployment document to write •Heavyweight process: • 3 approvals, 5 validations • 8 firewalls, 14 hops, 62 proxies to access production machines A legacy monolith A legacy process

Slide 62

Slide 62 text

Typical On-Premise Legacy Web Server - IIS Windows Services SQL Server DB •Recurring performance problems •Almost no monitoring •Regular unplanned downtime

Slide 63

Slide 63 text

@arollafr fedakv.medium.com/ MOVE 2 CLOUD Scalability High-availability

Slide 64

Slide 64 text

RE- ARCHITECTING for the PaaS!

Slide 65

Slide 65 text

A MONOLITH… "Car Trading System"

Slide 66

Slide 66 text

IDENTIFY THE SUB-DOMAINS… Front-Of fi ce Back-Of fi ce

Slide 67

Slide 67 text

SPLIT IN 2 FOR THE CLOUD Front-Of fi ce Back-Of fi ce APP Services <>

Slide 68

Slide 68 text

FROM DB-POLLING TO ASYNC QUEUE Front-Of fi ce Back-Of fi ce Use data Provide data Serverless bus

Slide 69

Slide 69 text

Front-Of fi ce Back-Of fi ce APP Services <> Too much work/not enough bene fi ts to go full PasS HALF PAAS / HALF IAAS KEEP AS-IS EXTRACT & ADAPT <>

Slide 70

Slide 70 text

OF COURSE, IT WENT WRONG

Slide 71

Slide 71 text

1- MULTIPLE INSTANCES + SESSIONS = ❤ Front-Of fi ce Session state Session state Session state

Slide 72

Slide 72 text

STATEFUL → STATELESS Front-Of fi ce Session state Azure Redis #win

Slide 73

Slide 73 text

2 - BIG QUERIES WORKED ON-PREM Front-Of fi ce SELECT p.ID , LAG (p.ID, 1) OVER (ORDER BY p.OwnerUserId DESC) AS PreviousID , LEAD (p.ID, 1) OVER (ORDER BY p.OwnerUserId DESC) AS NextI D FROM Posts p JOIN Users u ON u.Id = p.OwnerUserI d WHERE DisplayName = 'CecilPhillip ' AND p.PostTypeId = 1 AND p.OwnerUserId = 333082 Web Server - IIS Windows Services SQL Server DB Co-colocated

Slide 74

Slide 74 text

BIG SQL QUERIES TOO SLOW ON PAAS DB Front-Of fi ce SELECT p.ID , LAG (p.ID, 1) OVER (ORDER BY p.OwnerUserId DESC) AS PreviousID , LEAD (p.ID, 1) OVER (ORDER BY p.OwnerUserId DESC) AS NextI D FROM Posts p JOIN Users u ON u.Id = p.OwnerUserI d WHERE DisplayName = 'CecilPhillip ' AND p.PostTypeId = 1 AND p.OwnerUserId = 333082 SQL Always-On Cluster

Slide 75

Slide 75 text

QUICK WINS FOR PERFORMANCE! Azure CDN for big media Replace old cache with Azure Redis Cache Cron tasks moved to background jobs Pre-fetch data in read models (caches) via message bus #Cloud-powered

Slide 76

Slide 76 text

QUICK WINS FOR AUTOMATED DELIVERY! Proving scalability with automated performance tests Zero-Downtime deployment with Blue-Green deployment (using built-in Cloud "slots") #Cloud-powered

Slide 77

Slide 77 text

77 Split into 2-3 parts. Add queues/event bus. Switch stores. Automate CI/CD. Enjoy!

Slide 78

Slide 78 text

Pareto

Slide 79

Slide 79 text

Minimum Viable Cloud-Native re-Architecting

Slide 80

Slide 80 text

@arollafr New features faster!

Slide 81

Slide 81 text

New type of auction New vehicle-aware search-engine New vehicle-aware search-engine (using another, better-suited Cloud service) Business logs streaming Log Storage & Exploitation NEW FEATURES FASTER (FROM NOW ON)

Slide 82

Slide 82 text

MORE OPTIONS TO EVOLVE Event bus

Slide 83

Slide 83 text

MORE OPTIONS TO EVOLVE Event bus Ne w Service

Slide 84

Slide 84 text

MORE OPTIONS TO EVOLVE Event bus Saa S Service Ne w Service

Slide 85

Slide 85 text

New Architectural Opportunities

Slide 86

Slide 86 text

CLOUD = OPTIONS TO GO FASTER Self-Service Provisioning: Try stuff yourself No upfront cost: Try stuff cheap API or EVENTS everywhere to plug new stuff easily A lot of ready-made stuff The autonomy to JUST DELIVER with minimal coordination

Slide 87

Slide 87 text

@arollafr Re-architect faster Ongoing, while delivering new value

Slide 88

Slide 88 text

Back-Office progressively moved to Functions SQL DB progressively replaced by PaaS or Serverless storages More async messaging everywhere ONGOING RE-ARCHITECTING, (FROM NOW ON)

Slide 89

Slide 89 text

Just enough re-Architecting

Slide 90

Slide 90 text

@arollafr BIG-SPLIT RE-ARCHITECTING SYNDROME $3M 1 YEAR 7 PEOPLE 100% IT4IT SPLIT INTO 5 DOCKER CONTAINERS #wasteful-effort #capex-heavy

Slide 91

Slide 91 text

@arollafr BIG-SPLIT RE-ARCHITECTING SYNDROME $3M 1 YEAR 7 PEOPLE 100% IT4IT SPLIT INTO 5 DOCKER CONTAINERS #wasteful-effort #capex-heavy "Re-architecting sucks"

Slide 92

Slide 92 text

Ok.

Slide 93

Slide 93 text

@arollafr We are not fast with our cloud!

Slide 94

Slide 94 text

BEWARE CLOUD POISONS! Don’t degrade the time-to-market with process Pre-approved budget & own (sub)-account Pre-approved list of cloud services & patterns you can use Self-service provisioning in (no ticketing…) No mandatory technical layer on top of each service

Slide 95

Slide 95 text

@arollafr I don’t want to be locked to a cloud provider!

Slide 96

Slide 96 text

PORTABILITY BY DESIGN AWS SNS S3 Azure Blob Storage Service Bus e.g. hexagonal architecture, GUID, decoupling identity management What’s good for testing is also great for portability Dev+Test Test Robot Mocks

Slide 97

Slide 97 text

@arollafr Ok, but now… how to up-skill the teams?

Slide 98

Slide 98 text

LOTS OF SKILLS REQUIRED! Distributed Architecture Tolerant contracts Blue-Green Deployment Stateless EDA Correlation ID / GUID Modularity Tech Pressure Areas Bounded Contexts Evolution Pressure Areas Legacy Split Concerns Lazy Loading Large Transactions Temporal Coupling Shared "Uber" DB Migration Concerns Shared auth/ directory Slow queries/join Strangler App+Sync mechanism Cloud- native Patterns Read- models Async everywhere Fast & Reversible Direct Access download /upload Lean but not Naive Frugality Last Responsible Moment Keep Options Open Trying over Thinking Idempotence Extract Repair Rewrite Replace Hexagonal Architecture Change Data Capture Data Polling DB Views

Slide 99

Slide 99 text

THE RIGHT SKILLS FOR THE RIGHT GROUP Tolerant contracts Blue-Green Deployment Stateless EDA Correlation ID / GUID Tech Pressure Areas Bounded Contexts Large Transactions Shared "Uber" DB Strangler App+Sync mechanis m Read- models Direct Access download /upload Last Responsible Moment Keep Options Open Idempotence Extract Repair Rewrite Replace Hexagonal Architecture Change Data Capture Data Polling Group 1 Group 2 Group 3 Tolerant contracts Stateless EDA Correlation ID / GUID

Slide 100

Slide 100 text

@cyriux LEARNING THROUGH SPECIFICALLY DESIGNED EXERCISES

Slide 101

Slide 101 text

@arollafr IN CLOSING

Slide 102

Slide 102 text

Cloud is no more about hosting

Slide 103

Slide 103 text

New Architectural Opportunities

Slide 104

Slide 104 text

LIFT & SHIFT IS NOT ENOUGH FULL RE-ARCHITECTING IS TOO MUCH DDD BOUNDED CONTEXT CAN HELP EMBRACE PAAS/SERVERLESS ASAP TO GO FASTER EVERYTHING!