AWS Bucharest Loft 2019 - Control Your infrastructure and configuration on AWS

It's easy to say - "Hey I will use the cloud and be scalable and elastic!" - But it is not easy managing all that at scale, and keeping it flexible! Let's talk about Infrastructure as Code and Configuration as Code! This session will help you grasp the available toolset and best practices when it comes to managing your infrastructure and configuration on AWS. It will show you how can you make any changes to your workload with a single 'git push origin master'

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Manage Your Infrastructure and Configuration on AWS Darko Meszaros Specialist Solutions Architect @darkosubotica
3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. $(whoami) Darko Mesaroš / Darko Meszaros / Дарко Месарош ! → " → # → $ → % Berlin % @darkosubotica ln/darko-mesaros
4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. How we used to do it?
5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Jerry
6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. *click click click* Jerry
7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Web Application BI Tooling Super Secure FTP Servers yum install Jerry yum install yum install
8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Web Application BI Tooling Super Secure FTP Servers Jerry
9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Web Application BI Tooling Super Secure FTP Servers scale out scale out Jerry
10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. What issues do we see here? • Not scalable • Not elastic • Need for limitless documentation procedures • Very difficult to repeat/replicate • Slow reaction to changes
11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Before we get started, let’s take a look at some fundamentals.
12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. What is a Modern Application?
13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Approaches to modern application development • Simplify environment management • Reduce the impact of code changes • Automate operations • Accelerate the delivery of new, high-quality services • Gain insight across resources and applications • Protect customers and the business
14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Approaches to modern application development • Simplify environment management with serverless technologies • Reduce the impact of code changes with microservice architectures • Automate operations by modeling applications & infrastructure as code • Accelerate the delivery of new, high-quality services with CI/CD • Gain insight across resources and applications by enabling observability • Protect customers and the business with end-to-end security & compliance
15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Approaches to modern application development • Simplify environment management with serverless technologies • Reduce the impact of code changes with microservice architectures • Automate operations by modeling applications & infrastructure as code • Accelerate the delivery of new, high-quality services with CI/CD • Gain insight across resources and applications by enabling observability • Protect customers and the business with end-to-end security & compliance
16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Pillars of releasing modern applications
17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Pillars of releasing modern applications Infrastructure as code
18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Infrastructure as code goals 1. Make infrastructure changes repeatable and predictable 2. Release infrastructure changes using the same tools as code changes 3. Replicate production environment in a staging environment to enable continuous testing
19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Release infrastructure-as-code “Master” branch Prepare template Create & execute change set Create & execute change set
20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Release infrastructure-as-code process stages Source Build Test Production
21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Infrastructure as Code on AWS
23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CloudFormation
24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CloudFormation at a glance Code in YAML or JSON directly or use sample templates Upload local files or from an S3 bucket Create stack using console, API or CLI Stacks and resources are provisioned
25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. CloudFormation
26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Manage multiple stacks CloudFormation
27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. CloudFormation View your stacks as diagrams
28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Build Test Promote CloudFormation – Infrastructure CI/CD AWS Cloud Region Developers Git Push Templates Taskcat Source Staging Production Testing
29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. CloudFormation Now that we have infrastructure, how do we configure it?
30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Configuration as Code
31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Configuration as Code • Version Control • Declarative Code • Resource Providers • Testing • Communities • Portability
32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Configuration as Code AWS Systems Manager with State Manager / +
33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Configuration as Code • Install packages • Configure users • Configure server settings • Setup Services • …
34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Configuration as Code Run Ansible
35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Configuration as Code Run on Schedule
36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lets Automate This!
37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Jerry
38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. There are still a bunch of clicks … *click click click* SET UPLOAD UPDATE Jerry
39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Can we work this magic? git push origin master magic Jerry
40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Web Application BI Tooling Super Secure FTP Servers Jerry Ansible playbooks configure
41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Web Application BI Tooling Super Secure FTP Servers Jerry configure
42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Release infrastructure-as-code “Master” branch Prepare template Create & execute change set Create & execute change set
43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Release infrastructure-as-code process stages Source Build Test Production
44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Tale of Two Pipelines
45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Demo time
46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Notable mentions
47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Model function environments with AWS Serverless Application Model (SAM) • Open source framework for building serverless applications on AWS • Shorthand syntax to express functions, APIs, databases, and event source mappings • Transforms and expands SAM syntax into AWS CloudFormation syntax on deployment • Supports all AWS CloudFormation resource types https://aws.amazon.com/serverless/sam/
48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SAM template AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: GetFunction: Type: AWS::Serverless::Function Properties: Handler: index.get Runtime: nodejs8.10 CodeUri: src/ Policies: - DynamoDBReadPolicy: TableName: !Ref MyTable Events: GetResource: Type: Api Properties: Path: /resource/{resourceId} Method: get MyTable: Type: AWS::Serverless::SimpleTable Just 20 lines to create: • Lambda function • IAM role • API Gateway • DynamoDB table
49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Use SAM CLI to package and deploy SAM templates pip install --user aws-sam-cli sam logs sam validate sam local sam init sam build sam package sam deploy sam publish New
50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Model container environments with AWS Cloud Development Kit (CDK) • Open source framework to define cloud infrastructure • JavaScript, TypeScript, and Python, (Java, and C# in developer preview) • Provides library of higher-level resource types (“construct” classes) that have AWS best practices built in by default • Provisions resources with CloudFormation • Supports all CloudFormation resource types AWS CDK https://awslabs.github.io/aws-cdk
51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDK template import ec2 = require('@aws-cdk/aws-ec2'); import ecs = require('@aws-cdk/aws-ecs'); import cdk = require('@aws-cdk/cdk'); class BonjourFargate extends cdk.Stack { constructor(parent: cdk.App, name: string, props?: cdk.StackProps) { super(parent, name, props); const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 }); const cluster = new ecs.Cluster(this, 'Cluster', { vpc }); new ecs.LoadBalancedFargateService( this, "FargateService", { cluster, image: ecs.DockerHub.image("amazon/amazon-ecs-sample"), }); } } const app = new cdk.App(); new BonjourFargate(app, 'Bonjour'); app.run();
52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. import ec2 = require('@aws-cdk/aws-ec2'); import ecs = require('@aws-cdk/aws-ecs'); import cdk = require('@aws-cdk/cdk'); class BonjourFargate extends cdk.Stack { constructor(parent: cdk.App, name: string, props?: cdk.StackProps) { super(parent, name, props); const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 }); const cluster = new ecs.Cluster(this, 'Cluster', { vpc }); new ecs.LoadBalancedFargateService( this, "FargateService", { cluster, image: ecs.DockerHub.image("amazon/amazon-ecs-sample"), }); } } const app = new cdk.App(); new BonjourFargate(app, 'Bonjour'); app.run(); CDK template
53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDK template import ec2 = require('@aws-cdk/aws-ec2'); import ecs = require('@aws-cdk/aws-ecs'); import cdk = require('@aws-cdk/cdk'); class BonjourFargate extends cdk.Stack { constructor(parent: cdk.App, name: string, props?: cdk.StackProps) { super(parent, name, props); const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 }); const cluster = new ecs.Cluster(this, 'Cluster', { vpc }); new ecs.LoadBalancedFargateService( this, "FargateService", { cluster, image: ecs.DockerHub.image("amazon/amazon-ecs-sample"), }); } } const app = new cdk.App(); new BonjourFargate(app, 'Bonjour'); app.run();
54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Wrap up!
55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Recap • How do we manage infrastructure on AWS – CloudFormation • How do we manage Configuration on AWS - Systems Manager + Ansible • Automating the deployment of changes with Pipelines
56. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. So, should I use Terraform or CloudFormation??? Or, should I use Ansible, or Chef, or Puppet???
57. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Yes.
58. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank You! @darkosubotica ln/darko-mesaros
59. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.