Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Delivering changes for applications and databases @ Azure

Delivering changes for applications and databases @ Azure

Talk presented @ PortoData (28-03-2017)

7b47fb5bb7f32e0ffdbd7bba5598ba98?s=128

Eduardo Piairo

March 28, 2017
Tweet

Transcript

  1. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS @AZURE Eduardo Piairo

  2. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Eduardo Piairo, DevOps Engineer

    About me @EdPiairo https://pt.linkedin.com/in/jesuspiairo eduardopiairo@outlook.com http://www.eduardopiairo.com/
  3. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS 1. The value stream

    2. The problem to fix Chapter 1: Introduction
  4. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS 1. Create web app

    Chapter 2: Application
  5. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS 1. Physical machines 2.

    Virtual machines 3. Platform as a Service (PaaS) 4. Serverless 5. Infrastructure as Code (IaC) Chapter 3: Infrastructure
  6. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS 1. What’s so special

    about databases 2. Database challenges 3. Why database automation 4. The 1st step: Source Control 5. Migrations vs State 6. Flyway 7. Version Control System Chapter 4: Database
  7. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS 1. Deployment pipeline 2.

    The DevOps way Chapter 5: DevOps as engineering practice
  8. INTRODUCTION Chapter 1

  9. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS The value stream Source

    Control Continuous Integration Continuous Delivery Database + Application
  10. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Database and application most

    of the times are not both included in the value stream Have different deployment pipelines Database is not included in the deployment pipeline As result, database is out of pace with application development The problem to fix
  11. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Databases becoming a bottleneck

    in an agile delivery process The problem to fix
  12. APPLICATION Chapter 2

  13. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Let’s create a web

    app… Application
  14. INFRASTRUCTURE Chapter 3

  15. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS How can you have

    the required consistency without including infrastructure? If you practice (development) and play (production) in different fields, why do you expect the same behavior/performance? Infrastructure
  16. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS High hardware cost High

    human cost Very low flexibility Physical machines
  17. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS In the cloud Upfront

    cost First step to IaC Requires configuration management Virtual machines
  18. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Very flexible approach Costs

    grow with the application Fast and simple development Bring infrastructure/operations closer to the development Platform-as-a-Service (PaaS)
  19. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Azure Functions AWS Lambda

    Pay what you use Serverless (FaaS)
  20. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Thank you Virtualization!! Managing

    and provisioning infrastructure Is not simply writing scripts Allows to apply proven software development practices Version control, testing, small deployments Infrastructure as Code (Iac)
  21. DATABASE Chapter 4

  22. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Databases are out of

    pace with application development Lack of traceability of database changes (changes history) Manual databases processes prevent the CI and CD utilization Lack of testability Database setup time for new environments Bugs in production Database challenges
  23. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Databases becoming a bottleneck

    in an agile delivery process Database challenges
  24. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Database challenge

  25. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Enable control over database

    development Increase speed of response to change Keep a versioned “history” of database states and changes Reliability of the release process Increase release frequency though repeatability of processes Reduce time spent fixing bugs - automated tests Remove/reduce human intervention in the release process Why database automation
  26. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Why database automation Fearless

    databases changes
  27. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS First step in your

    database deployment pipeline Traceability through change history SQL as documentation Shared code-base and shared process Enforceable standards to reduce conflicts The 1st step: Source Control
  28. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Fundamental resource The 1st

    step: Source Control SQL Script
  29. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS State based solutions Script

    represents the current database state Your source of truth is how the database should be Migrations based solutions Script represents a migration Migration represents how to transition to the next database version Your source of truth is how the database should change Migrations vs State
  30. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Flyway – “Database Migrations

    made Easy” http://flywaydb.org/ Open source database migration tool Flyway command-line enabled Simplicity: easy to setup, no need to install Zero dependencies (java + jdbc) Scripts are written in SQL Flyway
  31. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Flyway commands: migrate, clean,

    info, validate, baseline, repair Metadata table: Used to track the state of the database If the database is empty, flyway won’t find it and will create it Flyway scans the migrations directory and check migrations against the metadata table Migrations are sorted based on their version number and applied in order Flyway: How it works
  32. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Demo

  33. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Rule 1: Script version

    (timestamp) Rule 2: Operation type Rule 3: Object type Rule 4: Object name Version Control System Example: V20170311.1200__Create_TB_MyTable.sql
  34. DEVOPS AS ENGINEERING PRACTICE Chapter 5

  35. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Deployment pipeline

  36. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Devops: contract for collaboration

    and communication (change management “framework”) Change description (Source Control) Change validation (Continuous Integration) Change implementation (Continuous Delivery) Applications, databases, infrastructure The DevOps way
  37. READING MATERIAL Appendix

  38. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Reading material

  39. DELIVERING CHANGES FOR DATABASES AND APPLICATIONS Q&A @EdPiairo https://pt.linkedin.com/in/jesuspiairo eduardopiairo@outlook.com

    http://www.eduardopiairo.com/