Slide 1

Slide 1 text

DEPLOYMENT PIPELINE FOR DATABASES Eduardo Piairo @EdPiairo #devopsdayscph

Slide 2

Slide 2 text

DEPLOYMENT PIPELINE FOR DATABASES Eduardo Piairo @EdPiairo #devopsdayscph

Slide 3

Slide 3 text

ABOUT ME Deployment pipeline for databases @EdPiairo, #devopsdayscph @EdPiairo https://pt.linkedin.com/in/jesuspiairo [email protected] http://www.eduardopiairo.com/ Eduardo Piairo DevOps Coach @ Natixis DevOps Porto Founder

Slide 4

Slide 4 text

AGENDA • Deployment pipeline • Database challenges • Deployment pipeline stages • Deployment pipeline scenarios • Final thoughts Deployment pipeline for databases @EdPiairo, #devopsdayscph

Slide 5

Slide 5 text

DEPLOYMENT PIPELINE DEFINITION A deployment pipeline should enable collaboration between the various groups involved in delivering software and provide everyone visibility about the flow of changes in the system, together with a thorough audit trail. by Martin Fowler @ https://martinfowler.com/bliki/DeploymentPipeline.html Deployment pipeline for databases @EdPiairo, #devopsdayscph

Slide 6

Slide 6 text

DEPLOYMENT PIPELINE STAGES Deployment pipeline for databases Stage 1 – Change description (Source Control) Stage 2 – Change validation (Continuous Integration) Stage 3 – Change implementation (Continuous Delivery) Source Control Continuous Integration Continuous Delivery @EdPiairo, #devopsdayscph

Slide 7

Slide 7 text

DATABASE & DEPLOYMENT PIPELINE Deployment pipeline for databases Source Control Continuous Integration Continuous Delivery Database + Application @EdPiairo, #devopsdayscph

Slide 8

Slide 8 text

WHAT’S SO SPECIAL ABOUT DATABASES? Deployment pipeline for databases @EdPiairo, #devopsdayscph

Slide 9

Slide 9 text

DATABASE CHALLENGES Deployment pipeline for databases • Database and application changes (most of the time) are handled differently • Different deployment pipeline • Synchronization needed • Database is not included (manual work) @EdPiairo, #devopsdayscph

Slide 10

Slide 10 text

DATABASE CHALLENGES Deployment pipeline for databases • Manual work … • Lack of traceability of database changes (changes history) • It’s the most expensive and risky work type • Prevent CI and CD utilization in their full extent • Promote the fear of changes @EdPiairo, #devopsdayscph

Slide 11

Slide 11 text

DATABASE CHALLENGES Deployment pipeline for databases Databases become a bottleneck in an agile delivery process @EdPiairo, #devopsdayscph

Slide 12

Slide 12 text

AUTOMATION Deployment pipeline for databases @EdPiairo, #devopsdayscph

Slide 13

Slide 13 text

THE VALUE OF AUTOMATION Deployment pipeline for databases • Enable control over database development • Increase speed of response to change • Greater reliability of the release process • Remove/reduce human intervention in the release process From (Re)Learn and forget it to Improve and forget it @EdPiairo, #devopsdayscph

Slide 14

Slide 14 text

THE VALUE OF AUTOMATION Deployment pipeline for databases Fearless database changes @EdPiairo, #devopsdayscph

Slide 15

Slide 15 text

SOURCE CONTROL Deployment pipeline for databases • First step in your database deployment pipeline • Traceability through change history • SQL as documentation • Shared code-base and shared process • Enforceable standards to reduce conflicts SQL Script @EdPiairo, #devopsdayscph

Slide 16

Slide 16 text

• State based solutions • How the database should be • Migrations based solutions • How the database should change MIGRATIONS VS STATE Deployment pipeline for databases Migration Migration State Delta @EdPiairo, #devopsdayscph

Slide 17

Slide 17 text

SCRIPTING GUIDELINES Deployment pipeline for databases • One script, one operation type, one object (small batches) • Merge conflicts management • Patterns identification • File system scripts history search @EdPiairo, #devopsdayscph

Slide 18

Slide 18 text

CONTINOUS INTEGRATION Deployment pipeline for databases • Integrate and validate changes • Tests: Unit, Integration • Small batches, less risk • Considerations: • What should be tested? And When? • Can the application help on this? @EdPiairo, #devopsdayscph

Slide 19

Slide 19 text

CONTINUOUS DELIVERY Deployment pipeline for databases • Delivering change(s) in the target environment • Considerations • Downtime • Time to recover • Affected applications @EdPiairo, #devopsdayscph

Slide 20

Slide 20 text

FAILED DEPLOYMENT Deployment pipeline for databases • Rollback/roll forward scripts • Very fast • Please, do not make mistakes • It’s like working backwards • Backups (before deploy) • Safer but slower • Keep databases small • Avoid share databases between applications @EdPiairo, #devopsdayscph

Slide 21

Slide 21 text

SCENARIO #1 Deployment pipeline for databases Source Control Continuous Integration Continuous Delivery APP DB • Independent deployment pipelines • High need of synchronization between DBA and development team @EdPiairo, #devopsdayscph

Slide 22

Slide 22 text

SCENARIO #2 Deployment pipeline for databases Source Control Continuous Integration Continuous Delivery APP DB • Different code repositories • Independent CI process • The contact point will be the deployment moment @EdPiairo, #devopsdayscph

Slide 23

Slide 23 text

SCENARIO #3 Deployment pipeline for databases Source Control Continuous Integration Continuous Delivery APP DB • Different code repositories • Connected/dependent CI process @EdPiairo, #devopsdayscph

Slide 24

Slide 24 text

SCENARIO #4 Deployment pipeline for databases Source Control Continuous Integration Continuous Delivery APP DB • Share the some code repository, CI process and CD process • No need of synchronization • Promotes learning within teams (database code reviews) @EdPiairo, #devopsdayscph

Slide 25

Slide 25 text

MY DEPLOYMENT PIPELINE DEFINITION Pipeline: Cultural and technical tool for managing changes in the software development process (should contemplate databases, applications and infrastructure). by me ;) Show me your pipeline and I will tell you how you work! Deployment pipeline for databases @EdPiairo, #devopsdayscph

Slide 26

Slide 26 text

DEVOPS AND DATABASES Deployment pipeline for databases • DevOps way • Increase flow visibility • Increase feedback • Increase knowledge base @EdPiairo, #devopsdayscph

Slide 27

Slide 27 text

READING MATERIAL Deployment pipeline for databases @EdPiairo, #devopsdayscph

Slide 28

Slide 28 text

Q&A Deployment pipeline for databases @EdPiairo, #devopsdayscph @EdPiairo https://pt.linkedin.com/in/jesuspiairo [email protected] http://www.eduardopiairo.com/