Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ADDO_2021: Why and how to include database changes in the deployment pipeline

ADDO_2021: Why and how to include database changes in the deployment pipeline

Eduardo Piairo

October 28, 2021
Tweet

More Decks by Eduardo Piairo

Other Decks in Technology

Transcript

  1. TRACK: CI/CD CONTINUOUS EVERYTHING
    NOVEMBER 12, 2020
    Eduardo Piairo
    Why and how to
    include database
    changes in the
    deployment
    pipeline

    View Slide

  2. TRACK: CI/CD CONTINUOUS EVERYTHING
    ● Engineering Operations Director @ Deeper Insights
    ● Co-Founder and Leader @ DevOps Porto
    About Me
    @EdPiairo
    https://pt.linkedin.com/in/eduardopiairo
    [email protected]
    https://www.eduardopiairo.com

    View Slide

  3. TRACK: CI/CD CONTINUOUS EVERYTHING
    ● Deployment pipeline
    ● Database challenges
    ● Deployment pipeline stages
    ● Deployment pipeline scenarios
    ● Final thoughts
    Agenda

    View Slide

  4. TRACK: CI/CD CONTINUOUS EVERYTHING
    A deployment pipeline should enable collaboration between the various
    groups involved in delivering software and provide everyone visibility
    about the flow of changes in the system, together with a thorough audit
    trail.
    by Martin Fowler
    @ https://martinfowler.com/bliki/DeploymentPipeline.html
    Definition

    View Slide

  5. TRACK: CI/CD CONTINUOUS EVERYTHING
    ● Stage 1 – Change description (Source Control)
    ● Stage 2 – Change validation (Continuous Integration)
    ● Stage 3 – Change implementation (Continuous Delivery)
    Stages
    Source
    Control
    Continuous
    Integration
    Continuous
    Delivery

    View Slide

  6. TRACK: CI/CD CONTINUOUS EVERYTHING
    Database & Deployment Pipeline
    Source
    Control
    Continuous
    Integration
    Continuous
    Delivery
    Database
    +
    Application

    View Slide

  7. TRACK: CI/CD CONTINUOUS EVERYTHING
    What’s so special about
    databases?

    View Slide

  8. TRACK: CI/CD CONTINUOUS EVERYTHING
    Database Challenges
    • Database and application changes (most of the time) are
    handled differently
    • Different deployment pipelines - synchronization needed
    • Database is not included (manual work)

    View Slide

  9. TRACK: CI/CD CONTINUOUS EVERYTHING
    Database Challenges
    • Manual work …
    • Lack of traceability of database changes (changes history)
    • It’s the most expensive and risky work type
    • Prevent CI and CD utilization in their full extent
    • Promote the fear of changes

    View Slide

  10. TRACK: CI/CD CONTINUOUS EVERYTHING
    Databases become a bottleneck
    in the delivery process

    View Slide

  11. TRACK: CI/CD CONTINUOUS EVERYTHING

    View Slide

  12. TRACK: CI/CD CONTINUOUS EVERYTHING
    The Value Of Automation
    • Enable control over database development
    • Increase speed of response to change
    • Greater reliability, repeatability and consistency
    • Enables failing in a controlled and known way
    • Remove/reduce human intervention in the release process

    View Slide

  13. TRACK: CI/CD CONTINUOUS EVERYTHING
    Fearless database changes

    View Slide

  14. TRACK: CI/CD CONTINUOUS EVERYTHING
    Source Control
    • Every database change can be described with an SQL script
    • Traceability through change history
    • SQL as documentation
    • Shared code-base and shared process
    • Enforce standards to reduce conflicts

    View Slide

  15. TRACK: CI/CD CONTINUOUS EVERYTHING
    Migrations vs State
    • State based solutions
    • How the database should be
    • Migrations based solutions
    • How the database should change
    Migration Migration
    State
    Delta

    View Slide

  16. TRACK: CI/CD CONTINUOUS EVERYTHING
    Scripting Guideline
    • One script, one operation type, one object (small batches)
    • Merge conflicts management
    • Patterns identification
    • File system scripts history search

    View Slide

  17. TRACK: CI/CD CONTINUOUS EVERYTHING
    Continuous Integration
    • Integrate and validate changes
    • Unit and integration tests
    • Small batches, less risk
    • Considerations
    • What should be tested? And when?
    • How can the application contribute to this stage?

    View Slide

  18. TRACK: CI/CD CONTINUOUS EVERYTHING
    Continuous Delivery
    • Delivering change(s) in the target environment
    • Considerations
    • Downtime
    • Time to recover
    • Affected applications

    View Slide

  19. TRACK: CI/CD CONTINUOUS EVERYTHING
    Failed Deployment
    • Scripts (roll-back, roll-front)
    • Very fast
    • (Please) Do not make mistakes
    • Working backwards

    View Slide

  20. TRACK: CI/CD CONTINUOUS EVERYTHING
    Failed Deployment
    • Backups (before deploy)
    • Safer but slower
    • Keep databases small
    • Avoid shared databases

    View Slide

  21. TRACK: CI/CD CONTINUOUS EVERYTHING
    Scenario #1
    Source Control Continuous Integration Continuous Delivery
    APP
    DB
    • Independent deployment pipelines
    • High need of synchronization between DBA and development team

    View Slide

  22. TRACK: CI/CD CONTINUOUS EVERYTHING
    Scenario #2
    Source Control Continuous Integration Continuous Delivery
    APP
    DB
    • Different code repositories
    • Independent CI process
    • The contact point will be the deployment moment

    View Slide

  23. TRACK: CI/CD CONTINUOUS EVERYTHING
    Scenario #3
    Source Control Continuous Integration Continuous Delivery
    APP
    DB
    • Different code repositories
    • Connected/dependent CI process
    • Short learning step towards scenario #4

    View Slide

  24. TRACK: CI/CD CONTINUOUS EVERYTHING
    Scenario #4
    Source Control Continuous Integration Continuous Delivery
    APP
    DB
    • Share the some code repository, CI process and CD process
    • No need of synchronization
    • Promotes learning within teams (database code reviews)

    View Slide

  25. TRACK: CI/CD CONTINUOUS EVERYTHING
    My Definition
    Pipeline: Cultural and technical tool for managing changes in the
    software development process (should contemplate databases, applications
    and infrastructure).

    View Slide

  26. TRACK: CI/CD CONTINUOUS EVERYTHING
    DevOps Way
    • Increase flow visibility
    • Increase feedback
    • Increase knowledge base and exploration

    View Slide

  27. TRACK: CI/CD CONTINUOUS EVERYTHING
    Q&A
    @EdPiairo
    https://pt.linkedin.com/in/eduardopiairo
    [email protected]
    http://www.eduardopiairo.com/
    #2021-ci-cd

    View Slide