Utilising AWS for .NET

This presentation showcases some of the best AWS services available for .NET, through the journey of Tigerspike's .NET development.

Transcript

1. Hello .Net+ Syd Meetup November 2019 Utilising AWS for .Net A CONCENTRIX COMPANY
2. Intro Tierspike is a digital products company. We help our partners and customers deliver better services through technology. Products we’re known for in Australia include: Opal Travel app, 7Eleven FuelApp, FaCS Housing apps and platform, VicRoads and Public Transport Victoria apps and platforms, Youth off the Streets digital transformation, SHI Be a Looper and many more. What we do? 01 About Tigerspike.
3. Intro We engage with our partners and customers at different stages throughout their journey. What we do? Strategy Experience design Development Hosting & Support
4. New York Singapore Melbourne Dubai Tokyo San Francisco Sydney Wrocław London Tigerspike around the world. 4 Mumbai Brisbane
5. Intro Working with different products and different requirements means our teams have to be multi-skilled with SMEs in many fields.
6. Intro We use a wide range of technologies.
7. React .Net Mobile Web Database iOS watch OS Swift Objective C Android Java SQL Server Oracle MySQL Postgre SQL Mongo DB Dynamo DB Angular C# jQuery Java Script Node.js Vue.js php Rails ERP & CRP Authentication & Authorization okta ADFS Rest:api SAP Sitecore Sales Force Word Express Umbraco Share Point Auth0 Service Desk & Analytics Google Analytics Jira Tag Manager Confluence Continuous Delivery & Deployment Ops Works CHEF Jenkins doker Octopus Deploy Code Build Code Pipeline Azure DevOps Infrastructure Lambda Load Balancer App Service DNS CDN AWS Servers Azure Aurora Intro
8. Intro What AWS services do we utilise for our .NET applications?
9. Journey Going through Tigerspike’s journey with AWS is a logical way of showcasing our utilisation of AWS for .NET. Like any digital company in late 2000s, we were hosting our products and client services in a datacentre. We owned our routers, switches, storage systems and servers. They were expensive, powerful and solid; but they were limited and required regular visits to maintain. When these devices approached their EoL, we needed to get out of there quick. It was also time to grow and scale but gradually. That’s when we decided to move to AWS, and the rest was history. The Journey 02 The Journey.
10. The datacentre A couple of powerful physical servers running VMware and a massive storage system connected through a gigabit network running through switches, a router and a firewall. Everything is designed with High Availability and DR in mind. The usual expensive design. This was serving mainly APAC clients, with a few in EMEA and AMER. Journey
11. The migration We chose to “replatform”. That is to lift, tinker-to-a-limit, and shift. The key is not to touch the core architecture of the applications. Why? Since AWS offers better networking experience, different instance types, newer OSs and a range of storage solutions, why not make use of this and update parts of our environments where it is easy to update. Journey Lookup “aws migration strategies”
12. Moving from VMs to EC2 instances is the simplest and most logical option. While you could use AWS Connector for vCenter or AWS Server Migration Service to migrate your VMs now, we migrated (redeployed) our applications to new EC2 instances. The migration Journey Lookup “aws products”
13. A simple architecture.. Keeping IIS, MSSQL and Windows services as our core platforms, we migrated our applications to freshly built and up to date instances. We made use of AWS Route53, Availability Zones, ELBs, S3, and EC2 license-included images to build a highly available and reliable hosting platform. Using CloudFormation templates, we replicated this architecture in multiple Regions closest to our APAC, EMEA and AMER’s clients. Over time, we decoupled some of the services and started using CloudFront, ElastiCache, SQS, and other services. Journey
14. ..with a simple CI/CD Using EC2 instances, we rebuilt our CI/CD pipeline with a mix of modern open-source and licensed tools. This is accompanied with highly durable Backups and cost effective Reserved instances. The .NET build agents are launched on-demand on EC2 Spot instances from our custom frequently-updated AMIs. Journey
15. Journey As simple as that, we achieved a more-reliable yet flexible and cost-effective infrastructure.
16. Isolated environments.. When bigger clients with specific security requirements jumped onboard, we decided to host them in isolated environments. And to simplify management of these environments, we chose to use AWS OpsWorks. OpsWorks for Chef takes care of instances configuration, deployments, auto-healing, auto-scaling, and load balancer and database attachments. This was also a good time to move to RDS and make use of other available AWS services, e.g. SQS. Journey
17. ..also with a simple CI/CD The CI/CD for these OpsWorks environments is not far from the simple CI/CD we started with. Instead of deploying direct to the instances, Octopus Deploy uploads packages to an S3 location and notifies OpsWorks to deploy the App to the specified Stack and Layer. The custom Octopus steps were simply written with Powershell, making use of AWS modules for Powershell. Journey
18. Journey With OpsWorks, we were able to focus more on improving our managed systems and less on maintenance and fire fighting.
19. A complex environment.. A digital bank jumped onboard… The bank’s application requires building decoupled APIs running on different groups of instances, i.e. microservices. These groups should be able to accommodate loads between 10 and 12,000 RPM in a single day. The “Golden Image” approach was the best candidate for this. AMIs or Golden Images are built using Packer. And once approved, the Launch Configuration of every Autoscaling Group is updated with its relevant AMI. The Autoscaling Groups then will take care of replacing and scaling instances as configured. Caching and data storage had to be moved away from these disposable instances. Also, we moved away Windows Services to Lambda Functions. Journey
20. ..yet not as complex CI/CD Packer launches a Spot Instance from a defined base image, configures it and install packages using scripts, deploy the application then create an AMI. Once created, the AMI ID is updated in the relevant Launch Configurations. We run Packer on Jenkins as well as on CodeBuild for AWS native CI/CD pipelines. We also use Packer to build Linux and Windows AMIs. Journey
21. A new serverless-world.. Some of our modern mobile apps require simple yet efficient APIs, i.e. a thin middleware layer to interact with complex backend systems. Serverless Lambda Functions is the best for this. We divided our middleware into 2 layers: functions that respond to mobile calls, i.e. API Functions, and functions that do tasks in the background, i.e. Backend Functions. There’s not a single EC2 instance in these environments. The only services that are running 24/7 are RDS (which you can run serverless now) and ElastiCache. The rest is billed per usage. Journey
22. Also, some of our modern managed APIs require supporting consistent baseloads with sudden and frequent spikes in usage. This is where AWS ECS helped us. Most of our ECS environments utilise AWS Fargate, which is serverless, while some require running on managed and custom ECS Clusters. A new serverless-world.. Journey
23. We utilize AWS native tools for our CI/CD pipelines. We build and test on CodeBuild, run automation jobs on CodeBuild or Lambda functions, and deploy using CloudFormation or CodeDeploy. All are managed through AWS CodePipeline and everything is written in code and version controlled including the pipelines themselves. ..with efficient CI/CD Journey
24. Recap “We use a wide range of technologies.”
25. Let’s recap - our server based .NET utilisation: Recap EC2 - traditional • EC2 instances • Launched through EC2 • Configured through a 3rd party Configuration Manager (e.g. Chef), Scripts (e.g. Bash or Powershell), or Active Directory (e.g. GPOs) • Applications are deployed via CodeDeploy or 3rd party tools (e.g. Octopus Deploy) EC2 - OpsWorks • EC2 instances • Launched through OpsWorks • Configured through OpsWorks for Chef, or a 3rd party Configuration Manager (e.g. Chef) • Applications are deployed via OpsWorks or 3rd party tools (e.g. Octopus Deploy) EC2 - Golden Images • EC2 instances • Launched through EC2 Autoscaling Groups from a selection of custom AMIs built using Packer • Configured through Packer during builds • Applications are deployed via Packer during builds ECS Clusters and Tasks • Containers running on EC2 instances • Clusters: launched through ECS from a choice of ECS Optimised AMIs • Containers: launched through ECS Services from ECS Task Definitions • Applications are deployed and configured via a container builder (e.g. Docker) during builds + +
26. Let’s recap - our serverless .NET utilisation: Recap ECS Fargate • Containers only • Serverless - clusters are managed by AWS • Containers are launched through ECS Services from ECS Task Definitions • Applications are deployed and configured via a container builder (e.g. Docker) during builds Lambda Functions • Code execution on demand • Serverless - runtime is managed by AWS • Launched through AWS Lambda • Deployed to via CLI, CodeDeploy or 3rd party tools
27. Q&A Any Questions? 03 Ask me.
28. Thank you Sydney — Melbourne — Brisbane — San Francisco — New York — London — Wrocław — Dubai — Mumbai — Singapore — Tokyo LinkedIn Facebook Twitter Instagram Tigerspike We deliver business value, fast. tigerspike.com Moha Alsouli Snr DevOps Engineer moha.alsouli@tigerspike.com