.NET Day 22 - Fast feedback with pull request deployments by Marc Müller

A modern DevOps process would no longer be possible without feature branches and pull request workflows. How are the changes of a pull request verified? Often, only a deployment into an isolated PR environment provides clarity. This allows testing in the overall context with surrounding systems and real data persistence, the deployment and upgrade process can be tested as well and last but not least a product owner can look at the change on a running system. This talk will show everything that is needed for a pull request deployment and also show approaches for challenges like database schema deployment with data initialization as well as regression testing. The examples will be presented using Azure DevOps and Azure Kubernetes Services.

Transcript

7. ▪ ▪ ▪ ▪ ▪ ▪ ▪
10. Code Test & Stabilize Code Test & Stabilize Code Complete Planning
12. ▪ ▪ ▪ ▪ ▪
13. ▪ ▪ ▪ ▪
15. ▪ ▪ ▪
17. ▪ ▪ ▪ ▪ ▪
18. ▪ ▪ ▪
20. ▪ ▪ ▪ ▪ ▪
21. ▪ ▪ ▪ ▪
23. Base Infrastructure CD Service CD Base Infrastructure Template Resource Template Resource Template Dev/Test Prod Service CI Build Stage PR Stage Testing PreProd Prod Compile Service DB Schema Compile System Tests Infrastructure Artifacts Pipeline Artifacts Task / Job Templates Task / Job Templates Resource Groups, vNets, VMs, Azure SQL, CosmosDB, … App Deployment, DB instance, DB Schema, Managed Identities, Storage, … Deplyoment Verification
24. ▪ ▪ ▪ ▪ ▪
25. CI build for every changed service
26. Require a successful deployment for every changed service
27. Additional security scans
28. Frontend deployment Backend deployment
29. ▪ ▪ ▪ ▪ ▪ ▪ ▪
37. ▪ ▪ ▪ ▪ ▪
38. PR Version Release Version
40. Service and DB Schema Testdata Automated system tests
41. Identify PR Context Add PR status check which is used as policy
42. ▪ ▪ ▪ ▪
43. PR Version Release Version Build Version + Deployment Attempt
44. CI pipeline as resource (trigger and artifacts)
45. Calculate release number CI version + release revision
46. Calculate PR specific variables Setup PR environment (k8s namespace, database, identities) Deploy service Initialize test data
47. Run system tests in PR environment
48. Update PR status (collect deployment and test results)
49. Calculate the overall status Update the PR status
50. Extract deployment URL (PR specific) Only run for PR Add a PR comment with deployment info
51. ▪ ▪ ▪ ▪ ▪
52. ▪ ▪
54. ▪ ▪ ▪ ▪
55. ▪ ▪ ▪
56. ▪ ▪ ▪
57. ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪
61. https://dev.azure.com/4tecture-demo/_apis/public/distributedtask/webhooks/prupdated?api-version=6.0-preview
63. Method: POST URI: https://dev.azure.com/4tecture-demo/_apis/public/distributedtask/webhooks/prupdated?api-version=6.0-preview HTTP Version: 1.1 Headers: { Content-Type: application/json; charset=utf-8 } Content: { "subscriptionId": "1de80ac8-b9a7-42d0-a2fe-5441c2b7ffc2", "notificationId": 14, "id": "af07be1b-f3ad-44c8-a7f1-c4835f2df06b", "eventType": "git.pullrequest.updated", "publisherId": "tfs", "message": { "text": "Jamal Hartnett marked the pull request as completed", "html": "Jamal Hartnett marked the pull request as completed", "markdown": "Jamal Hartnett marked the pull request as completed" }, "detailedMessage": { "text": "Jamal Hartnett marked the pull request as completed\r\n\r\n- Merge status: Succeeded\r\n- Merge commit: eef717(https://fabrikam.visualstudio.com/DefaultCollection/_apis/git/repositories/4bc14d40-c903-45e2-872e-0462c7748079/commit "html": "Jamal Hartnett marked the pull request as completed\r\n<ul>\r\n<li>Merge status: Succeeded</li>\r\n<li>Merge commit: <a href=\"https://fabrikam.visualstudio.com/DefaultCollection/_apis/git/repositories/4bc14d40-c903-45e2-872e- 0462c7748079/commits/eef717f69257a6333f221566c1c987dc94cc0d72\">eef717</a></li>\r\n</ul>", "markdown": "Jamal Hartnett marked the pull request as completed\r\n\r\n+ Merge status: Succeeded\r\n+ Merge commit: [eef717](https://fabrikam.visualstudio.com/DefaultCollection/_apis/git/repositories/4bc14d40-c903-45e2-872e-0462c7748079/ }, "resource": { "repository": { "id": "4bc14d40-c903-45e2-872e-0462c7748079", "name": "Fabrikam", "url": "https://fabrikam.visualstudio.com/DefaultCollection/_apis/git/repositories/4bc14d40-c903-45e2-872e-0462c7748079", "project": { "id": "6ce954b1-ce1f-45d1-b94d-e6bf2464ba2c", "name": "Fabrikam", "url": "https://fabrikam.visualstudio.com/DefaultCollection/_apis/projects/6ce954b1-ce1f-45d1-b94d-e6bf2464ba2c", "state": "wellFormed", "visibility": "unchanged", "lastUpdateTime": "0001-01-01T00:00:00" }, "defaultBranch": "refs/heads/master", "remoteUrl": "https://fabrikam.visualstudio.com/DefaultCollection/_git/Fabrikam" }, "pullRequestId": 1, "status": "completed", "createdBy": { "displayName": "Jamal Hartnett", "url": "https://fabrikam.vssps.visualstudio.com/_apis/Identities/54d125f7-69f7-4191-904f-c5b96b6261c8", "id": "54d125f7-69f7-4191-904f-c5b96b6261c8", "uniqueName": "fabrikamfiber4@hotmail.com", "imageUrl": "https://fabrikam.visualstudio.com/DefaultCollection/_api/_common/identityImage?id=54d125f7-69f7-4191-904f-c5b96b6261c8" }, "creationDate": "2014-06-17T16:55:46.589889Z", "closedDate": "2014-06-30T18:59:12.3660573Z", "title": "my first pull request", "description": " - test2\r\n", "sourceRefName": "refs/heads/mytopic",
64. Incoming WebHook as resource with trigger Additional conditions for dynamic payload evaluation
65. Calculate PR specific variables Delete deployment Delete database Delete Identities
67. • • • • IDE MSBuild IntelliSense Validation Code Base Consistency Design Compare CI CD
68. • • • DB Scheme Migrations (Static & Dynamic SQL) Single Pre- and Post Script Logic Microsoft.Data.Tools.Msbuild (NuGet)
69. • • • • •
70. • • • • •
71. Local DEV DB Visual Studio DB Project Git Repo → QA Dump Prod Dump
72. ▪ ▪ ▪ ▪ ▪
73. Start Deployment End Deployment GENERATE Deployment Script Run Script Pre-Deployment Script Run Script Post-Deployment Script Scheme Migration Deployment Script (DacPac) Database Deployment Build Process Run Script Reference Data Deployment Script
74. • • • • • • •
75. • • • •
76. Release v2 Deploy DB Schema Deploy Binaries Prod Environment v2 Binaries v1 Prod Environment v2 Binaries v2 Release v2 Deploy Binaries Deploy DB Schema Prod Environment v1 Binaries v1 Binaries v2 Factory Prod Environment v2 Binaries v1 Binaries v2 Factory
77. CD PR CI Checkout Build App Run Unit Test Build Dacpac Publish Dacpac Publish App Create / restore DB Deploy DB Schema Deploy App QA Deploy DB Schema Deploy App Pre-Prod Clone Prod DB Deploy DB Schema Deploy App Prod Deploy DB Schema Deploy App CI Type
78. steps: - pwsh: | /opt/sqlpackage/sqlpackage /a:Publish /p:BlockOnPossibleDataLoss= ${{ parameters.blockOnDataLoss }} /p:GenerateSmartDefaults=True /tcs:"${{ parameters.connection }} /sf:"${{ parameters.dacpacFile }}" displayName: 'Deploy DACPACs' https://docs.microsoft.com/de-de/sql/tools/sqlpackage/sqlpackage-download?view=sql-server-ver16
79. ▪ ▪ ▪ ▪
80. ▪ ▪ ▪ ▪ ▪ https://hub.docker.com/_/microsoft-mssql-server
81. ▪ ▪ ▪
82. ▪ ▪ ▪ ▪
83. ▪ ▪ ▪ ▪ ▪ ▪
85. ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪
86. ✓ ✓ ✓ ✓
87. Infrastructure QA Namespace Fronend Backend PR-xy Namespace Fronend Backend Database Server Prod PR-xy
88. Infrastructure QA Namespace Fronend Backend PR-xy Namespace Fronend Backend Database Server Prod PR-xy
89. Infrastructure QA Namespace Fronend Backend PR-xy Namespace Fronend Database Server Prod Backend PR-xy
90. ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪
91. ▪ ▪ ▪ ▪
92. Helm Release Deployment App Pod App Container Init Container Job DB Migration Pod DB Migration Container Service Ingress DB
94. ▪ ▪ ▪
95. ▪ ▪ ▪ Pipeline Agent Docker Container Pipeline Job
96. trigger: - master pool: #vmImage: 'windows-latest' default container: mcr.microsoft.com/dotnet/framework/sdk:4.7.2-windowsservercore-ltsc2019 variables: solution: '**/*.sln' buildPlatform: 'Any CPU' buildConfiguration: 'Release' steps: - task: NuGetToolInstaller@1 - task: NuGetCommand@2 inputs: restoreSolution: '$(solution)' - task: VSBuild@1 inputs: solution: '$(solution)' platform: '$(buildPlatform)' configuration: '$(buildConfiguration)' - task: VSTest@2 inputs: testSelector: 'testAssemblies’ searchFolder: '$(System.DefaultWorkingDirectory)' vstestLocationMethod: 'location' vstestLocation: 'C:\Program Files (x86)\Microsoft Visual Studio\2019\TestAgent\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe' platform: '$(buildPlatform)' configuration: '$(buildConfiguration)' - publish: $(System.DefaultWorkingDirectory)\ConsoleApp\bin\Release artifact: ConsoleApp
97. ▪ ▪ ▪
98. Pipeline Agent Docker Network Container Pipeline Job Build Container Publish Container Container Job Container API App Container SQL Server Deploy DB Run Test Docker Registry
99. Container Resources
100. Matrix → run test in different environments in parallel Dynamic service container selection based on matrix variable
101. DB schema deployment Run integration / system tests
102. Docker network container alias Docker network container alias
104. ▪ ▪ ▪ ▪