Slide 1

Slide 1 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Platinum Sponsor: Database migrations: the missing link to continuous delivery Filippo Bosi @filippobosi [email protected] Luca Acquaviva @lacquaviva [email protected] Stefano Monti @ste_monti [email protected] http://www.imolainformatica.it

Slide 2

Slide 2 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL

Slide 3

Slide 3 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL …that’s nothing new

Slide 4

Slide 4 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL …that’s nothing new Continuous Delivery

Slide 5

Slide 5 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Continuous Delivery with DATABASES & NO DOWNTIME

Slide 6

Slide 6 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL …that’s nothing new Continuous Delivery with DATABASES & NO DOWNTIME

Slide 7

Slide 7 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Principles & Application lifecycle Patterns Tools Continuous Delivery with Zero Downtime

Slide 8

Slide 8 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Simple Scenario “Single application with dedicated database” VS Complex Scenarios “Multiple applications with shared database” E.g. Monolith-to-microservices migrations Continuous Delivery with Zero Downtime

Slide 9

Slide 9 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Principles & Application lifecycle - Avoid breaking database changes à Maintain backward compatibility - Treat your database as code (Versioning & lifecycle) - Automate DB Testing Patterns Tools Single application with dedicated database

Slide 10

Slide 10 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Treat your database as code • Database changeset format (plain SQL vs XML/Liquibase) • Database objects: DDL vs data vs stored procedures • Version-control code AND database • Relate application code and database via version numbers and explicit dependencies Single application with dedicated database https://www.slideshare.net/axelfontaine/continuous-delivery-and-zero-downtime

Slide 11

Slide 11 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Tips • Single project for Database and Code • Database Code (DDL+Logic) and application share the same lifecycle • Release and deploy Application and Database Dedicated database - lifecycle one team one project one repository one pipeline one deploy

Slide 12

Slide 12 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Continuous Integration Deployment Env Artifact Repo Deploy Continuous Integration Deployment Env Artifact Repo Deploy

Slide 13

Slide 13 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Principles & Application lifecycle Patterns - Expand & Contract - Blue/Green Deploy (or Feature Toggle) - Adopt Delivery Pipelines Tools Single application with dedicated database

Slide 14

Slide 14 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Expand and contract https://dotnetvibes.com/2018/04/14/handling-rollback-of-database-deployments-with-a-single-click/ Make sure that database changes are backward compatibile E.g. Renaming Column SurName to LastName Name SurName LastName Expand: - add a new column LastName - add Trigger or Logic for dat a alignment Transition: - adapt application logic to use the new column - test extensively Contract: - remove old column and trigger Name LastName Name SurName LastName

Slide 15

Slide 15 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Blue/green deploy https://martinfowler.com/bliki/BlueGreenDeployment.html

Slide 16

Slide 16 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL if (flag){ //access v1.1 ... } else{ //access v1.0 ... } Feature toggle https://martinfowler.com/articles/feature-toggles.html Name Surname LastName Name Surname LastName v1.0 v1.1

Slide 17

Slide 17 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Name Surname V 1.0 V 1.0 Blue/Green+Expand&Contract (1/4)

Slide 18

Slide 18 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL V 1.1 Name Surname LastName V 1.1 V 2.0 #1 Deploy - Expand Blue/Green+Expand&Contract (2/4)

Slide 19

Slide 19 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Blue/Green+Expand&Contract (3/4) Name Surname LastName V 1.1 V 2.0 V 1.1 #2 Transition

Slide 20

Slide 20 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Blue/Green+Expand&Contract (4/4) Name LastName V 2.0 V 2.0 V 1.1 #3 Deploy -Contract

Slide 21

Slide 21 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Principles & Application lifecycle Patterns Tools - Database automation, e.g., Liquibase vs Flyway - Continuous Integration/Delivery , e.g., Jenkins Continuous Delivery with Zero downtime

Slide 22

Slide 22 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Active Migrations Tools

Slide 23

Slide 23 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Tools are required but not enough

Slide 24

Slide 24 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Principles, Patterns and tools mitigate problems, however… …migrations can fail! YOU NEED TO PLAN FOR (AUTOMATED) DATABASE ROLLBACKS! Open points – #1 migration failures

Slide 25

Slide 25 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL WHAT DDL (provided backward compatibility is guaranteed) Stored Procedures/Logic (provided backward compatibility is guaranteed) DATA: WARNING STATIC/DOMAIN DATA (no env-dependent data!) LIVE/BUSINESS DATA (only fake data for testing purposes) Open points – #2 migration logic

Slide 26

Slide 26 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Continuous Delivery with Zero downtime Simple Scenario “Single application with dedicated database” VS Complex Scenarios “Multiple applications with shared database” E.g. Monolith-to-microservices migrations

Slide 27

Slide 27 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL A typical monolith-to-microservices roadmap… Monolith-to-microservices migration 2. Modular Application with Single Database 1. Monolith 3. Microservices

Slide 28

Slide 28 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Monolith-to-microservices migration: an example router t0 router router router t1 t2 t3 t4 router

Slide 29

Slide 29 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL • Many applications/modules depend on the same database àTreat database as a dependency: database artifact with version àMore complex and extensive automated database testing • Many teams need to modify the database àMore complex application lifecycles and delivery pipelines Monolith-to-microservices migration: issues Worst case

Slide 30

Slide 30 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Testing strategies – App modules UNIT TEST END 2 END TEST ACCEPTANCE ENV

Slide 31

Slide 31 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Testing strategies – App modules https://martinfowler.com/articles/microservice-testing/ UNIT TEST CONTRACT TEST END 2 END TEST LOCAL/SANDBOXED ENV ACCEPTANCE ENV

Slide 32

Slide 32 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Testing strategies – DB module UNIT TEST CONTRACT TEST M1 LOCAL/SANDBOXED ENV CONTRACT TEST M2 ACCEPTANCE ENV

Slide 33

Slide 33 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL multiple distinct repo Tips • Dedicated project for Database and Code • Database code (DDL+Logic) and applications code have different lifecycle • Consider Database like another dependency Shared database many teams multiple projects multiple repositories multiple pipelines multiple deployments multiple distinct project multiple distinct pipeline depends

Slide 34

Slide 34 text

ACCEPTANCE PRO D END TO END TESTING CONTINUOUS DB INTEGRATION PIPELINE CONTINUOUS DB DELIVERY PIPELINE depends

Slide 35

Slide 35 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL Demo

Slide 36

Slide 36 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL • Development complexity increases • Delivery Governance complexity increases Open points

Slide 37

Slide 37 text

@lacquaviva @filippobosi @ste_monti #DevoxxPL THANK YOU !

Slide 38

Slide 38 text

No content