Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ADDO_2021: Why and how to include database chan...

ADDO_2021: Why and how to include database changes in the deployment pipeline

Avatar for Eduardo Piairo

Eduardo Piairo

October 28, 2021
Tweet

More Decks by Eduardo Piairo

Other Decks in Technology

Transcript

  1. TRACK: CI/CD CONTINUOUS EVERYTHING NOVEMBER 12, 2020 Eduardo Piairo Why

    and how to include database changes in the deployment pipeline
  2. TRACK: CI/CD CONTINUOUS EVERYTHING • Engineering Operations Director @ Deeper

    Insights • Co-Founder and Leader @ DevOps Porto About Me @EdPiairo https://pt.linkedin.com/in/eduardopiairo eduardopiairo@gmail.com https://www.eduardopiairo.com
  3. TRACK: CI/CD CONTINUOUS EVERYTHING • Deployment pipeline • Database challenges

    • Deployment pipeline stages • Deployment pipeline scenarios • Final thoughts Agenda
  4. TRACK: CI/CD CONTINUOUS EVERYTHING A deployment pipeline should enable collaboration

    between the various groups involved in delivering software and provide everyone visibility about the flow of changes in the system, together with a thorough audit trail. by Martin Fowler @ https://martinfowler.com/bliki/DeploymentPipeline.html Definition
  5. TRACK: CI/CD CONTINUOUS EVERYTHING • Stage 1 – Change description

    (Source Control) • Stage 2 – Change validation (Continuous Integration) • Stage 3 – Change implementation (Continuous Delivery) Stages Source Control Continuous Integration Continuous Delivery
  6. TRACK: CI/CD CONTINUOUS EVERYTHING Database & Deployment Pipeline Source Control

    Continuous Integration Continuous Delivery Database + Application
  7. TRACK: CI/CD CONTINUOUS EVERYTHING Database Challenges • Database and application

    changes (most of the time) are handled differently • Different deployment pipelines - synchronization needed • Database is not included (manual work)
  8. TRACK: CI/CD CONTINUOUS EVERYTHING Database Challenges • Manual work …

    • Lack of traceability of database changes (changes history) • It’s the most expensive and risky work type • Prevent CI and CD utilization in their full extent • Promote the fear of changes
  9. TRACK: CI/CD CONTINUOUS EVERYTHING The Value Of Automation • Enable

    control over database development • Increase speed of response to change • Greater reliability, repeatability and consistency • Enables failing in a controlled and known way • Remove/reduce human intervention in the release process
  10. TRACK: CI/CD CONTINUOUS EVERYTHING Source Control • Every database change

    can be described with an SQL script • Traceability through change history • SQL as documentation • Shared code-base and shared process • Enforce standards to reduce conflicts
  11. TRACK: CI/CD CONTINUOUS EVERYTHING Migrations vs State • State based

    solutions • How the database should be • Migrations based solutions • How the database should change Migration Migration State Delta
  12. TRACK: CI/CD CONTINUOUS EVERYTHING Scripting Guideline • One script, one

    operation type, one object (small batches) • Merge conflicts management • Patterns identification • File system scripts history search
  13. TRACK: CI/CD CONTINUOUS EVERYTHING Continuous Integration • Integrate and validate

    changes • Unit and integration tests • Small batches, less risk • Considerations • What should be tested? And when? • How can the application contribute to this stage?
  14. TRACK: CI/CD CONTINUOUS EVERYTHING Continuous Delivery • Delivering change(s) in

    the target environment • Considerations • Downtime • Time to recover • Affected applications
  15. TRACK: CI/CD CONTINUOUS EVERYTHING Failed Deployment • Scripts (roll-back, roll-front)

    • Very fast • (Please) Do not make mistakes • Working backwards
  16. TRACK: CI/CD CONTINUOUS EVERYTHING Failed Deployment • Backups (before deploy)

    • Safer but slower • Keep databases small • Avoid shared databases
  17. TRACK: CI/CD CONTINUOUS EVERYTHING Scenario #1 Source Control Continuous Integration

    Continuous Delivery APP DB • Independent deployment pipelines • High need of synchronization between DBA and development team
  18. TRACK: CI/CD CONTINUOUS EVERYTHING Scenario #2 Source Control Continuous Integration

    Continuous Delivery APP DB • Different code repositories • Independent CI process • The contact point will be the deployment moment
  19. TRACK: CI/CD CONTINUOUS EVERYTHING Scenario #3 Source Control Continuous Integration

    Continuous Delivery APP DB • Different code repositories • Connected/dependent CI process • Short learning step towards scenario #4
  20. TRACK: CI/CD CONTINUOUS EVERYTHING Scenario #4 Source Control Continuous Integration

    Continuous Delivery APP DB • Share the some code repository, CI process and CD process • No need of synchronization • Promotes learning within teams (database code reviews)
  21. TRACK: CI/CD CONTINUOUS EVERYTHING My Definition Pipeline: Cultural and technical

    tool for managing changes in the software development process (should contemplate databases, applications and infrastructure).
  22. TRACK: CI/CD CONTINUOUS EVERYTHING DevOps Way • Increase flow visibility

    • Increase feedback • Increase knowledge base and exploration