M365Con: Combining the power of Azure Verified Modules and private modules in a hybrid setup

In the rapidly evolving landscape of cloud infrastructure, the Azure Bicep stands out as Azure native declarative Infrastructure as Code (IaC) language and a tool designed to simplify the deployment of Azure resources. When your organization matures with its IaC adoption, new challenges emerge like applying the DRY (Don’t Repeat Yourself) principle, improving collaboration across teams, writing a secure code that follows Microsoft recommended practices, etc.

In this session, we will explore the world of Bicep modules that are published to Microsoft’s Public Registry through the Azure Verified Modules (AVM) initiative and explain how they can be combined with private modules that organizations can develop to fit their specific needs and publish to their private registries.

We will focus on a practical example on how to build a ‘Private Modules Library’, a publishing engine on GitHub that can follow recommended practice for testing and validating your modules before they are published to your private registry.

The goal is to demonstrate practically how you can combine the use of public modules, curated by Microsoft, with your own code in a consistent way and how you can empower your cloud engineers to use this Library together with a custom Web Catalogue that can help the understand and use the modules better.

The demo environment (and repository) will contain all code artefacts for building the Private Modules Library, provisioning all necessary prerequisites, and deploying a ‘solution’ using a private ‘pattern’ module.

The main building blocks of this solution are Bicep, AVM, GitHub Actions workflows, PowerShell, PSRule for Azure, PSDocs, MKDocs, and more.

The session is based on a real-life customer project where such Modules Library was used for deploying Azure Virtual Desktop environments automatically through code.

Transcript

1. Combining the power of Azure Verified Modules and private modules in a hybrid setup David Pazdera azurescholar.cloud
2. About me o solution architect @ o Microsoft MVP Azure o meetups, conferences, ACP, communities (ALZ, Azure Arc, Bicep, AVM, Terraform in Azure) o GitHub | LinkedIn | Sessionize | SpeakerDeck | X : pazdedav handle o Blog: azurescholar.cloud azurescholar.cloud
3. Today’s menu azurescholar.cloud Concepts Infrastructure modules refresher (Bicep and Terraform) AVM Private Modules Library design Demo Featuring Bicep and GitHub combo Building Private Modules Library Role-play
4. What are infrastructure modules o composable, reusable files - set of related resources o used in deployment templates / root modules o embed your requirements (defined naming conventions and security requirements and policies) o contract = defined input variables / parameters and outputs o software packages for IaC world (dependency) o authoring styles: configuration set vs. maximum customization azurescholar.cloud
5. Terminology azurescholar.cloud User input Parameters Variables Internal variables Variables Locals User output Outputs Outputs Input values files Parameter files TFVars files Provider definition Extension or Import block Providers block Configuration bicepconfig.json Terraform block
6. Module structure azurescholar.cloud
7. Good practices azurescholar.cloud o az bicep format o az bicep lint o az bicep generate-params o az bicep restore o terraform fmt o terraform validate o terraform init | terraform get
8. Module sources azurescholar.cloud o Local paths o Bicep registries (pub, priv) o Template Specs o Local paths o Terraform registry (pub, priv) o GitHub, Bitbucket, generic Git, Mercurial repo o HTTP URLs o S3 bucket, GCS bucket o (package sub-directory)
9. Consuming modules azurescholar.cloud module hostPool 'br/public:avm/res/desktop-virtualization/host-pool:0.3.0' = { scope: resourceGroup('${workloadSubsId}', '${serviceObjectsRgName}’) name: 'HostPool-${time}’ params: { } } module "vpc" { source = "terraform-aws-modules/vpc/aws" version = "3.18.1" name = var.vpc_name }
10. Publishing modules - Bicep azurescholar.cloud Bicep public registry o N/A – Microsoft only allows ‘internal’ publishing Bicep private registry o ACR instance, permissions, az cli or posh az bicep publish --file storage.bicep --target br:exampleregistry.azurecr.io/bicep/modules/storage:v1 --documentation-uri https://www.contoso.com/examplereg.html --with-source
11. Publishing modules - Terraform azurescholar.cloud Terraform public registry – registry.terraform.io o Compliant GitHub repo (public, naming convention, 1 module per repo, standard module structure, description, x.y.z tags o sign-in to the registry with GitHub (authorize app) o tag-based workflow o Community tier Terraform private registry - app.terraform.io/example_corp o Requires Terraform Cloud account o Connection to VCS provider o Tag-based vs. branch-based publishing workflow
12. Azure Verified Modules in a nutshell o MSFT official initiative to set the standards for IaC modules o Flexible, generalized, multi-purpose with integrated child and extension resources o Resource and Pattern Modules o Bicep and Terraform azurescholar.cloud
13. Definition of Verified o supported by MSFT CSS o aligned to AVM specs with enforced consistency (interfaces) o up-to-date with product roadmaps o aligned to WAF High-priority recommendations, Reliability Hub, and APRL o documented (with examples) o tested azurescholar.cloud aka.ms/avm
14. External contributions o Modules must be owned by MSFT FTEs o Create issue for missing module or feature o Fork the repo and contribute via PR o All tests must pass azurescholar.cloud
15. What if you… o need a specific resource composition / module o don't want to publish modules externally, but o don’t want to create and maintain general-purpose resource modules, or o need to temporarily deviate from AVM to fix a bug / enable feature azurescholar.cloud Build your own pattern modules but use AVM resource modules
16. Private Modules Library azurescholar.cloud
17. Building blocks [1/4] Azure Container Registry o SKUs: o Basic and Standard SKUs uses private as default o Repositories o AuthN: Microsoft Entra ID or keys o AuthZ: RBAC Roles o Least privilege: AcrPull, AcrPush o Reader has ‘pull image’ permission o Owner and Contributor have ‘push image’ permission azurescholar.cloud
18. Building blocks [2/4] Code repository o Structure o Bicep – can use multiple-module single-repo model o Terraform – single-module single-repo model o Branching o Main for production version of infra modules o Feature branches for updates and new modules azurescholar.cloud
19. Building blocks [3/4] CI/CD pipelines o Tested on both GitHub Action workflows and Azure Pipelines o Generic scripts / CLI commands – easy to port on other pipelines o Workflows: o CI – linting, validation, testing o CD – publishing to ACR azurescholar.cloud
20. Building blocks [4/4] Module Web Catalog o Auto-generated documentation (markdown): PSDocs o Rendering from markdown to HTML: MKDocs o Publishing to a web service: Azure Static Apps o Separate workflow o Can be integrated with Entra ID azurescholar.cloud
21. azurescholar.cloud Demo time… https://github.com/pazdedav/private-modules-library
22. Challenges 1/2 o access management to registry o adding MIs to ACR in ‘vending machine’ o group memberships for engineers o lifecycle management – upstream modules o change feed o all or some o test before publish o publishing cascade azurescholar.cloud
23. Challenges 2/2 o flexibility can lead to complexity and verbosity o e.g., storage-account module (json) has 5281 lines of code o authoring and debugging o template size limits o external dependency - software supply chain azurescholar.cloud
24. Want to learn more? o aka.ms/avm o aka.ms/learnbicep o https://github.com/pazdedav/private-modules-library azurescholar.cloud