@shahiddev Q&A go to slido.com event #Y860 Agenda Popular common options for provisioning cloud infrastructure Limitations of the common approaches Pulumi Title slide photo by Oscar Nord on Unsplash
@shahiddev Q&A go to slido.com event #Y860 Caveats Focus on Azure & .NET but principles are same on other platforms and languages Talk is inspired by my experience dealing with Cloud infrastructure whilst working with smaller teams I don’t work for Pulumi – i.e. not a sales pitch ☺
@shahiddev Q&A go to slido.com event #Y860 Who am I Freelance Azure consultant specialising in Azure, Kubernetes & Cloud native technologies. Over a decade of experience as a developer (mostly .NET) Microsoft MVP Co-organise meetup in the UK (Milton Keynes .NET) https://linkedin.shahid.dev [email protected] https://blog.headforcloud.com
@shahiddev Q&A go to slido.com event #Y860 Portal cons • Not easy to repeat consistently • Options can be buried deep in submenus • Provisioning lots of resources takes time (not easy to run in parallel) • You have to resolve the resource dependencies
@shahiddev Q&A go to slido.com event #Y860 Infrastructure as code (IaC) is the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools. https://en.wikipedia.org/wiki/Infrastructure_as_code
@shahiddev Q&A go to slido.com event #Y860 Pros of Cloud templating options • Can be automated • Idempotent • Declarative – desired state • Easier to provision multiple resources • Resource dependency (mostly) automatically resolved
@shahiddev Q&A go to slido.com event #Y860 Cons of cloud templating • Verbose • Need to learn domain specific language • Specific to each cloud provider • Closed source
@shahiddev Q&A go to slido.com event #Y860 Terraform • Tool from Hashicorp • Uses their own markup language (HCL) • Multi-cloud/platform • Open source • Written in Go
@shahiddev Q&A go to slido.com event #Y860 Cons with Terraform • Special language used by Terraform only • Support in Terraform lags behind cloud provider • State management
@shahiddev Q&A go to slido.com event #Y860 Infrastructure as code (IaC) is the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools. https://en.wikipedia.org/wiki/Infrastructure_as_code
@shahiddev Q&A go to slido.com event #Y860 Infrastructure as code (IaC) is the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools.
@shahiddev Q&A go to slido.com event #Y860 Pulumi • Open source* • Uses modern programming languages • Supports many platforms • Declarative *Except for the Pulumi console
@shahiddev Q&A go to slido.com event #Y860 class Program { static Task Main() { return Deployment.RunAsync(() => { // Create an Azure Resource Group var resourceGroup = new ResourceGroup("resourceGroup"); // Create an Azure Storage Account var storageAccount = new Account("storage", new AccountArgs { ResourceGroupName = resourceGroup.Name, AccountReplicationType = "LRS", AccountTier = "Standard", }); // Export the connection string for the storage account return new Dictionary { { "connectionString", storageAccount.PrimaryConnectionString }, }; }); } }
@shahiddev Q&A go to slido.com event #Y860 class Program { static Task Main() { return Deployment.RunAsync(() => { // Create an Azure Resource Group var resourceGroup = new ResourceGroup("resourceGroup"); // Create an Azure Storage Account var storageAccount = new Account("storage", new AccountArgs { ResourceGroupName = resourceGroup.Name, AccountReplicationType = "LRS", AccountTier = "Standard", }); // Export the connection string for the storage account return new Dictionary { { "connectionString", storageAccount.PrimaryConnectionString }, }; }); }
@shahiddev Q&A go to slido.com event #Y860 class Program { static Task Main() { return Deployment.RunAsync(() => { // Create an Azure Resource Group var resourceGroup = new ResourceGroup("resourceGroup"); // Create an Azure Storage Account var storageAccount = new Account("storage", new AccountArgs { ResourceGroupName = resourceGroup.Name, AccountReplicationType = "LRS", AccountTier = "Standard", }); // Export the connection string for the storage account return new Dictionary { { "connectionString", storageAccount.PrimaryConnectionString }, }; }); }
@shahiddev Q&A go to slido.com event #Y860 Pulumi components https://www.pulumi.com/docs/intro/concepts/how-pulumi-works/ Local disk Cloud storage Pulumi console - $
@shahiddev Q&A go to slido.com event #Y860 Invoking 3rd party APIs Templating and Terraform options usually require dropping into a script inside the template. • Not easy to test • Relies on 3rd party tools to be available on the machine running the template • Yet another language (bash/PowerShell)
@shahiddev Q&A go to slido.com event #Y860 Azure storage static website hosting Run static websites/SPAs from storage account Feature can’t be enabled from Azure Resource Manager API • ARM templates/Terraform/Pulumi cannot enable this easily • Need to invoke script/Azure CLI to enable feature Can we use Azure storage .NET SDK? (spoiler alert – Yes!) https://github.com/pulumi/examples/tree/master/azure-cs-static-website
@shahiddev Q&A go to slido.com event #Y860 // The code in the Apply method must be idempotent. if (!Deployment.Instance.IsDryRun) storageAccount.PrimaryBlobConnectionString.Apply(async v => await EnableStaticSites(v) ); // Upload the files var files = new[]{"index.html", "404.html"}; foreach (var file in files) { var uploadedFile = new Blob(file, new BlobArgs { Name = file, StorageAccountName = storageAccount.Name, StorageContainerName = "$web", Type = "block", Source = $"./wwwroot/{file}", ContentType = "text/html", }); }
@shahiddev Q&A go to slido.com event #Y860 // The code in the Apply method must be idempotent. if (!Deployment.Instance.IsDryRun) storageAccount.PrimaryBlobConnectionString.Apply(async v => await EnableStaticSites(v) ); // Upload the files var files = new[]{"index.html", "404.html"}; foreach (var file in files) { var uploadedFile = new Blob(file, new BlobArgs { Name = file, StorageAccountName = storageAccount.Name, StorageContainerName = "$web", Type = "block", Source = $"./wwwroot/{file}", ContentType = "text/html", }); }
@shahiddev Q&A go to slido.com event #Y860 // The code in the Apply method must be idempotent. if (!Deployment.Instance.IsDryRun) storageAccount.PrimaryBlobConnectionString.Apply(async v => await EnableStaticSites(v) ); // Upload the files var files = new[]{"index.html", "404.html"}; foreach (var file in files) { var uploadedFile = new Blob(file, new BlobArgs { Name = file, StorageAccountName = storageAccount.Name, StorageContainerName = "$web", Type = "block", Source = $"./wwwroot/{file}", ContentType = "text/html", }); }
@shahiddev Q&A go to slido.com event #Y860 Changes: Type Name Operation + azure:core:ResourceGroup mystaticsite created + azure:storage:Account mysite created + pulumi:pulumi:Stack azure-cs-static-website-dev created + azure:storage:Blob index.html created + azure:storage:Blob 404.html created Resources: + created 5 Duration: 28s
@shahiddev Q&A go to slido.com event #Y860 Primary region Application gateway Web apps Secondary region Application gateway Web apps Traffic manager SQL Server Failover group Geo-replication High availability configuration
@shahiddev Q&A go to slido.com event #Y860 Steps for single region • Provision App service plan and n-web apps, capturing app urls • Provision Application gateway and configure routes to backend web apps (using the app urls). • Adding SSL certificates • Configure security headers • Configure healthchecks • Configure SQL server and create database, capturing server and db details
@shahiddev Q&A go to slido.com event #Y860 Steps for multi-region deployment • Provision into primary and secondary regions, capturing app gateway addresses • Add SQL geo-replication • Configure SQL failover group and capture the failover group connection string • Add Traffic manager and wire up backends to app gateways
@shahiddev Q&A go to slido.com event #Y860 Challenges for smaller teams • No dedicated person/team to manage cloud resources • Complex templates or duplicated code • Often several steps that need to be coordinated • Need for team to understand not only their own app code but also cloud platform + templating language
@shahiddev Q&A go to slido.com event #Y860 Where Pulumi can help • Likelihood that .NET (or other languages) are more understood by team • Pulumi console means everyone sees the same picture ($)
@shahiddev Q&A go to slido.com event #Y860 Re-usable components • Create a stack for a group of resources that you can deploy together • Can be packaged into a Nuget package to use in my org • Compose more complex deployments by re-using stacks whilst ensuring everyone is using consistent configuration
@shahiddev Q&A go to slido.com event #Y860 Policy as Code Allows you to define policies which “intercept” deployments and will prevent certain things from being deployed. Better option than trying to abstract the Pulumi api and hide certain options from teams.
@shahiddev Q&A go to slido.com event #Y860 Working with existing resources • Flexible approach to working with existing resources • Co-exist along side previously deployed resources – no interference • Adopt existing resources into Pulumi (doesn’t generate the .NET code!) • Re-write/generate Pulumi code from existing resources • Tool to generate Pulumi code from Terraform – Tf2pulumi https://www.pulumi.com/docs/guides/adopting/
@shahiddev Q&A go to slido.com event #Y860 Pulumi pros • Using languages your teams are familiar with already • With the power of modern langaguages and 3rd SDKs can achieve most things directly in the code • Easy to get started • Free to use and OSS (unless you want the optional console)
@shahiddev Q&A go to slido.com event #Y860 Pulumi cons • Fewer platforms supported vs Terraform • A lag between feature release and support in Pulumi • Azure provider is using Terraform provider – dependency on competitor product • Still need to learn/discover the cloud provider resource API (not specific to Pulumi) • Some errors not apparent until Pulumi up • Can get into bad state – especially if you cancel mid-way* *it does warn you to not cancel mid-way!
@shahiddev Q&A go to slido.com event #Y860 What about SDKs for the Cloud? • Abstract REST calls behind code • Imperative • Not necessarily idempotent • Difficult to reason about current state vs new state
@shahiddev Q&A go to slido.com event #Y860 Transpilers • Take general purpose language and convert to cloud templating • AWS CDK – generates Cloud Formation templates • Farmer – F# code generates ARM templates.
shahiddev
line_developers
orinbou
junki
kobayang
miyashino
coconala_engineer
salaboy
kmwebnet
konifar
hamadakoji
caitiem20
mclloyd
swwweet
csswizardry
lauravandoore
ddemaree
reverentgeek
gr2m
garrettdimon
ammeep
chrisshort
panda_program