Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Workshop: Delivering chnages for applications and databases

Workshop: Delivering chnages for applications and databases

Workshop presented @ Scrum Gathering Portugal (02-12-2016) by Eduardo Piairo and Miguel Alho

Eduardo Piairo

December 02, 2016

More Decks by Eduardo Piairo

Other Decks in Technology


  1. Chapter 1: Introduction 1. The “DevOps” context 2. The problem

    that we want to fix 3. The database challenge 4. Infrastructure 5. Collaboration & Culture 6. Continuous Integration 7. Continuous Delivery
  2. Chapter 2: Application 1. Microservice repository creation 2. Creating the

    first functionality with TDD 3. Implement a build job / CI pipeline
  3. Chapter 3: Infrastructure 1. Physical machines 2. Virtual machines 3.

    Platform as a Service 4. Serverless 5. Infrastructure as Code 6. Provision and deployment for our microservice
  4. Chapter 4: Database 1. The special case of the database

    2. Database automation 3. Migrations vs State 4. Flyway 5. Database integration into our microservice
  5. Chapter 5: Managing Changes 1. Business requirements changes 2. How

    interaction is important to determine how to move forward 3. Versioning strategies 4. Deployment strategies 5. Implementing changes in our Microservice
  6. Stakeholders Sponsors / Customers Product Owner / Project Manager Coaches

    and Scrum Masters Business Stakeholders Architects, Leads and Developers Quality Assurance Engineers Release Engineers DBAs IT Operations Development Stakeholders User
  7. Business stakeholders want... Working software Software that fix the customer's

    problems Software that add value New features and fast Incremental features Frequent changes (Continuous Change) Business development and continuity Sponsors / Costumers Product Owner / Project Manager Coaches and Scrum Masters Business Stakeholders Sponsors / Customers Product Owner / Project Manager Coaches and Scrum Masters Business Stakeholders
  8. Development stakeholders want... Development Stakeholders Develop quality solutions Secure and

    stable solutions Avoid bugs and midnight calls Few obstacles Architects, Leads and Developers Quality Assurance Engineers Release Engineers DBAs IT Operations
  9. Dichotomy of interests Traditionally there is collision of interests. Devops

    intends to break this collision with communication, empathy and collaboration improvement, along with useful technical practices, principally automation New features Corrections and improvements Optimizations Stability Security Troubleshooting Operations Devs
  10. Delivery of new features quickly Devs must be able to

    make changes to software quickly to integrate new features Software should be developed in order to integrate changes, instead of simply passing the tests Requires: • Use of adequate design patterns • Automation of the build process • Testing in different environments/levels • Fast functionality validation • Regression prevention • Quality measurement
  11. Software delivery consistently Putting software in production should not be

    a headache Requires: • Continuous Integration • Continuous Delivery • Process automation – consistency guarantee • Visibility – quality and metrics
  12. Scalability Rapidly answer to new needs for delivery as well

    as users load requirements Requires: • Provisioning automation • Deployment automation • Instance generation simplification • Monitoring
  13. Database as bottleneck • Databases traditionally are out of pace

    with application development • Easy target to blame • Lack of traceability of databases changes (changes history) • Manual database processes prevent CI and CD
  14. Database as bottleneck Requires: • Automation • Source Control database

    changes (1st step for db deployment pipeline) • Synchronization between application and database
  15. Infrastructure How can you have the required consistency without including

    infrastructure? If you practice (development) and play (production) in different fields, why do you expect the same behavior/performance?
  16. Knight(Mare) • Knight Capital Group • 3.3 billion trades daily

    • $21.5 billion traded daily • $365M cash & equivalents ----- • NYSE launches Retail Liquidity Program on August 1, 2012, an automated routing system for equity orders • Group updates software to comply and repurpose 8-year old feature flag
  17. Knight(mare) “During the deployment of the new code, however, one

    of Knight’s technicians did not copy the new code to one of the eight SMARs computer servers.” SEC Filing | Release No. 70694 | October 16, 2013 ( http://tinyurl.com/Devops-Knightmare )
  18. Knight(mare) • 9:30 AM - Market Opens • 212 small

    retail “parent-orders” • 7 servers processing “child-orders” correctly. • 8th server using the old feature flag code failed to recognize parent-orders were fulfilled. • 8th server sent cumulative child.-orders in rapid succession.
  19. Knight(mare) • No automated fail-safe • No procedures for how

    to react • Knight uninstalled the correct SMARS code from the seven servers where it had been deployed correctly • 9:30 AM to 10:15 • 4 million executions in 154 stocks • 357 million shares
  20. DevOps changes in Technical Practices • Continuous Integration • Continuous

    Delivery • Provisioning (Infrastructure as Code) • Containerization • Microservices • Database migrations
  21. Continuous Integration Increases your opportunities feedback (fast) feedback Build software

    at every change Features: A connection to a version control repository A build script A feedback mechanism A process for integrating the source code changes
  22. Continuous Integration To implement it, we need: • Source Control

    - Git, Hg, Svn, TFS, ... • Build System ◦ Scripting - Cake, Psake, FAKE, Gulp … ◦ Compiler and tools - MsBuild, Node/npm, GitVersion, … ◦ Build Server - VSTS, TeamCity, AppVeyor, CircleCi • Testing Framework - nUnit, xUnit, Fixie, tSQLt, ... • Packaging mechanism - Nuget, Docker container, zip file…
  23. Here’s what we are going to do: • Create a

    project in VSTS • Create a repository for a Microservice, with in memory data • Implement the microservice using TDD • Create a basic build job for our microservice in VSTS • Se this whole setup run
  24. Virtual Machines In the cloud Upfront cost First step to

    IaC Requires configuration management
  25. Platform as a Service (PaaS) Very flexible approach Costs grow

    with the application Fast and simple development Bring infrastructure/operations closer to the development
  26. Infrastructure as Code (IaC) Thank you Virtualization!! Managing and provisioning

    infrastructure Is not simply writing scripts Allows to apply proven software development practices Version control, testing, small deployments
  27. Database challenges Databases are out of pace with application development

    Lack of traceability of database changes (changes history) Manual databases processes prevent the CI and CD utilization Lack of testability Database setup time for new environments Bugs in production
  28. Why database automation? Enable control over database development Increase speed

    of response to change Keep a versioned “history” of database states and changes Reliability of the release process Increase release frequency though repeatability of processes Reduce time spent fixing bugs - automated tests Remove/reduce human intervention in the release process
  29. 1st Step: Source Control First step in your database deployment

    pipeline Traceability through change history SQL as documentation Shared code-base and shared process Enforceable standards to reduce conflicts
  30. Migrations vs State State based solutions Script represents the current

    database state Your source of truth is how the database should be Migrations based solutions Script represents a migration Migration represents how to transition to the next database version Your source of truth is how the database should change
  31. Flyway Flyway – “Database Migrations made Easy” http://flywaydb.org/ Open source

    database migration tool Flyway command-line enabled Simplicity: easy to setup, no need to install Zero dependencies (java + jdbc) Scripts are written in SQL
  32. Flyway: How it works Flyway commands: migrate, clean, info, validate,

    baseline, repair Metadata table: Used to track the state of the database If the database is empty, flyway won’t find it and will create it Flyway scans the migrations directory and check migrations against the metadata table Migrations are sorted based on their version number and applied in order
  33. Version Control System Two components Database scripts repo (git) Flyway

    - database migrations Rule 1: Script version (timestamp) Rule 2: Operation type Rule 3: Object type Rule 4: Object name Example: V20160728.1100__Create_TB_MyTable.sql
  34. Time to add a database to our service • Provision

    development Database • Add test project in Code • Implement database features in our microservice • Test • Add steps in our build system to support this • Deploy changes
  35. Change is unavoidable Businesses change naturally • competitive advantage requires

    enhancements • products need to move towards customer’s real needs • social and economic context affects how business is done Software NEEDS to change to enable this!
  36. Change is not easy in software Ideally, changes contained in

    one component should NOT break all the others! Unfortunately, this is not always possible • more than one app consumes a service • more than one app consumes a database • binary data contracts (dlls) need to be updated everywhere • environment variables and changes affect program’s expected behaviour • lack of tests brings uncertainty to change outcome Many of these are anti-patterns!
  37. Collaboration is key • Need for change needs to be

    communicated • Risk of change needs to be analysed • Effects need to be listed to prepare for possible problems and avoid fire- fighting • Teams need to announce how changes will impact existing processes and work. • Teams need to determine if the change is done in a single step or in multiple iterations.
  38. Versioning strategies • Versioning allows team to tag artifacts to

    a specific state • Collaborating components can assert their validity based on versions ◦ component A requires B version X or higher • Semantic versioning gives meaning to number ◦ prefer SemVer • Automate versioning as much as possible ◦ tooling and commit messages can be enough to indicate the type of change • Important part of a CI pipeline - end result of a build is a specific version of an
  39. SemVer 1.3.5-beta0003 major minor patch prerelease increment on a breaking

    change increment on a feature addition increment on a bug fix increment while unreleased
  40. Versioning is not easy • But it is extremely important

    • Semantics matter ◦ minors and patches mean clients shouldn’t have to change anything, only update ◦ Majors are breaking so clients need to take some maintenance and realignment action • It’s also automatic documentation
  41. Deployment strategies You choose based on what your context is

    (traffic volume, working hours, SLAs) • Single Server or service ◦ Stop service, upgrade, restart • Multiple Servers ◦ Stop service, upgrade, restart ◦ Blue-Green deployment ◦ Canary Deployment Whatever the scenario, automation is key
  42. Deployments with DBs make things harder • Database’s data is

    mutable. It’s super hard to rollback! ◦ How do you rollback changes that delete a column of data? • Application is typically immutable. It’s way easier to rollback ◦ Just need the database schema to be compatible This means that we typically need to make changes in multiple steps to avoid breaking the service. Schema should valid for backward compatible to at least 1 version of the application. Marking things as deprecated can / should make a difference.
  43. Why DevOps Developing software is not enough, you have to

    deliver it Communication framework for change management You can not stop change, but you can control it Perspectives Need for speed (time-to- market) (management people) Need for control (error control) (operations people)
  44. DevOps: contract for collaboration and communication Contract - change communication

    management tool Change description (Source Control) Change validation (Continuous Integration) Change implementation (Continuous Delivery) Applications, databases, infrastructure
  45. DevOps advices Operations requirements in Backlog Operations engineers in Product

    Team Shared goals Shared responsibilities Devs monitor apps Devs deploy apps
  46. Demo application “Purchases-DbMigration-sample” https://github.com/MiguelAlho/Purchases-DbMigration-sample Flyway https://flywaydb.org/ DbUp http://dbup.github.io/ Octopus Deploy

    https://octopus.com/ http://docs.octopusdeploy.com/display/OD/Deploying+ASP.NET+Core+Web+Applications Visual Studio https://www.visualstudio.com/