Infrastructure IS Code on AWS

In the age of The Cloud, Serverless, Containers and Microservices, our infrastructure is part of our application more than ever. So why do we still treat it as something else? Our Application is Code, our Business Logic is code, well Infrastructure also IS code. One of the parts of doing things properly at scale is being able to describe your infrastructure as code and deploy it as such. If we already treat our infrastructure as code, why not apply all the best practices of software delivery to infrastructure delivery. In this session we look into Infrastructure as Code solutions, best practices and patterns on AWS.

Transcript

1. © 2020, Amazon Web Services, Inc. or its Affiliates.
   Infrastructure IS Code on
   AWS
   Darko Meszaros
   Developer Advocate – Amazon Web Services
   @darkosubotica
   ln/darko-mesaros
   twitch.tv/ruptwelve
   youtube.com/ruptwelve
   

   View Slide

2. © 2020, Amazon Web Services, Inc. or its Affiliates.
   Agenda for today
   • What is Infrastructure as Code
   • AWS CloudFormation
   • AWS Serverless Application Model (SAM)
   • AWS Cloud Development Kit (CDK)
   • Other Tools out there
   • Some Best Practices
   • Wrap up
   

   View Slide

3. © 2020, Amazon Web Services, Inc. or its Affiliates.
   What is Infrastructure as Code?
   

   View Slide

4. © 2020, Amazon Web Services, Inc. or its Affiliates.
   

   View Slide

5. © 2020, Amazon Web Services, Inc. or its Affiliates.
   

   View Slide

6. © 2020, Amazon Web Services, Inc. or its Affiliates.
   Infrastructure as code
   ✓
   Make infrastructure
   changes repeatable and
   predictable
   ✓
   Release infrastructure
   changes using the same
   tools as code changes
   ✓
   Replicate production in
   a staging environment
   to enable continuous
   testing
   

   View Slide

7. © 2020, Amazon Web Services, Inc. or its Affiliates.
   Infrastructure as code
   Declarative
   I tell you
   what I need
   I tell you
   what to do
   Imperative
   

   View Slide

8. © 2020, Amazon Web Services, Inc. or its Affiliates.
   $(whoami) Darko Mesaroš / Darko Meszaros /
   Дарко Месарош
   ! → " → # → $ → %
   Berlin !
   @darkosubotica
   ln/darko-mesaros
   twitch.tv/ruptwelve
   youtube.com/ruptwelve
   

   View Slide

9. © 2020, Amazon Web Services, Inc. or its Affiliates.
   AWS CloudFormation
   

   View Slide

10. © 2020, Amazon Web Services, Inc. or its Affiliates.
    AWS CloudFormation
    • Infrastructure as code (IaC)
    • Provides a common language for you to
    describe and provision all the infrastructure
    resources in your cloud environment
    • Build and rebuild your infrastructure and
    applications, without having to perform
    manual actions or write custom scripts
    https://aws.amazon.com/cloudformation/
    

    View Slide

11. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Sample AWS CloudFormation code
    • Code is written in files
    called templates
    • A stack is generated from
    a template
    • Templates primarily define
    resources for an application
    • AWS CloudFormation can create
    over 490 types of resources
    • Each resource is configured
    based on its available properties
    • Dependencies can be explicitly
    declared or implicitly discovered
    AWSTemplateFormatVersion: "2010-09-09"
    Description: A CodeCommit Repo and Cloud9 Environment
    Resources:
    MyRepo:
    Type: "AWS::CodeCommit::Repository"
    Properties:
    RepositoryName: MyRepo
    RepositoryDescription: Sample Repository for Demo
    MyC9Environment:
    Type: "AWS::Cloud9::EnvironmentEC2"
    Properties:
    Repositories:
    - PathComponent: /cfn
    RepositoryUrl: !GetAtt MyRepo.CloneUrlHttp
    InstanceType: t2.micro
    

    View Slide

12. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Anatomy of an AWS CloudFormation template
    • Resources
    • Parameters and Mappings
    • Conditions
    • Outputs
    

    View Slide

13. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Testing via pipelines
    • As you would with other application code, templates should be version
    controlled and tested via CI/CD pipelines
    • The linter can be run in an AWS CodeBuild step to ensure that teams comply
    with rules and standards
    • Additional tools, like taskcat (available on GitHub), allow tests across regions
    Git push
    Templates
    AWS
    CodeCommit
    AWS
    CodePipeline
    AWS
    CodeBuild
    AWS
    CloudFormation
    Region
    AWS
    CodeDeploy
    

    View Slide

14. © 2020, Amazon Web Services, Inc. or its Affiliates.
    But I use resources outside of AWS!
    

    View Slide

15. © 2020, Amazon Web Services, Inc. or its Affiliates.
    CloudFormation
    registry
    Open
    CLI
    Open
    providers
    Introducing the AWS CloudFormation registry
    An open approach to managing external resources
    

    View Slide

16. © 2020, Amazon Web Services, Inc. or its Affiliates.
    AWS CloudFormation registry and CLI
    • Allows AWS CloudFormation to
    support native and
    non-AWS resources while inheriting
    many core benefits like rollbacks
    • Use the AWS CloudFormation CLI
    tool to create resource providers
    using JSON schema-driven
    development, generating many of
    the code assets for you
    • Use third-party resource providers
    as you would use native AWS
    resource types
    

    View Slide

17. © 2020, Amazon Web Services, Inc. or its Affiliates.
    AWS Serverless Application
    Model (SAM)
    

    View Slide

18. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Model function environments with AWS Serverless
    Application Model (SAM)
    • Open source framework for building serverless
    applications on AWS
    • Shorthand syntax to express functions, APIs,
    databases, and event source mappings
    • Transforms and expands SAM syntax into AWS
    CloudFormation syntax on deployment
    • Supports all AWS CloudFormation resource types
    https://aws.amazon.com/serverless/sam/
    

    View Slide

19. © 2020, Amazon Web Services, Inc. or its Affiliates.
    SAM template
    AWSTemplateFormatVersion: '2010-09-09’
    Transform: AWS::Serverless-2016-10-31
    Resources:
    MySimpleTableFunction:
    Type: AWS::Serverless::Function
    Properties:
    Handler: mySimpleTableFunction.handler
    Runtime: nodejs12.x
    CodeUri: ./functions
    Policies:
    - DynamoDBReadPolicy:
    TableName: !Ref MySimpleTable
    Events:
    MySimpleFunctionApi:
    Type: Api
    Properties:
    Path: /simpleTable
    Method: GET
    MySimpleTable:
    Type: AWS::Serverless::SimpleTable
    Just 20 lines to create:
    • Lambda function
    • IAM role
    • API Gateway
    

    View Slide

20. © 2020, Amazon Web Services, Inc. or its Affiliates.
    AWS SAM CLI
    • Create, build, test, and deploy
    AWS SAM applications
    • Step-through debugging and
    IDE support
    • Open source!
    • https://github.com/awslabs/aw
    s-sam-cli
    

    View Slide

21. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Use SAM CLI to package and deploy SAM templates
    pip install --user aws-sam-cli # Or even better use native installers
    sam init --name my-app --runtime python
    cd my-app/
    sam local ... # generate-event/invoke/start-api/start-lambda
    sam validate # The SAM template
    sam build # Depending on the runtime
    sam package --s3-bucket my-packages-bucket \
    --output-template-file packaged.yaml
    sam deploy --template-file packaged.yaml \
    --stack-name my-stack-prod
    sam logs -n MyFunction --stack-name my-stack-prod -t # Tail
    sam publish # To the Serverless Application Repository
    CodePipeline
    Use
    CloudFormation
    deployment
    actions with any
    SAM application
    Jenkins
    Use SAM CLI
    plugin
    O
    pen
    Source
    

    View Slide

22. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Did you just say local tests?
    

    View Slide

23. © 2020, Amazon Web Services, Inc. or its Affiliates.
    

    View Slide

24. © 2020, Amazon Web Services, Inc. or its Affiliates.
    AWS Cloud Development Kit
    (CDK)
    

    View Slide

25. View Slide

26. © 2020, Amazon Web Services, Inc. or its Affiliates.
    

    View Slide

27. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Shorten the learning curve with AWS Cloud Development
    Kit (CDK)
    Late 2020
    Brings cloud infrastructure to developers in ways the can understand
    • Build cloud infrastructure with the languages they already know
    • Use their existing tools and workflows
    • Helpful abstractions that remove the need to learn the details
    • Vibrant and fast-growing community of developers
    

    View Slide

28. © 2020, Amazon Web Services, Inc. or its Affiliates.
    AWS Cloud Development Kit (AWS CDK)
    A multi-language development framework for modeling infrastructure as reusable components
    

    View Slide

29. © 2020, Amazon Web Services, Inc. or its Affiliates.
    From constructs to the cloud
    

    View Slide

30. © 2020, Amazon Web Services, Inc. or its Affiliates.
    AWS CDK Constructs
    

    View Slide

31. © 2020, Amazon Web Services, Inc. or its Affiliates.
    All the tests!
    

    View Slide

32. © 2020, Amazon Web Services, Inc. or its Affiliates.
    How do we do testing with CDK?
    • Snapshot tests
    • Fine-grained assertions
    • Validation tests
    npm install --save-dev jest @types/jest @aws-cdk/assert
    

    View Slide

33. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Additional Tools and
    Frameworks
    

    View Slide

34. © 2020, Amazon Web Services, Inc. or its Affiliates.
    

    View Slide

35. © 2020, Amazon Web Services, Inc. or its Affiliates.
    

    View Slide

36. © 2020, Amazon Web Services, Inc. or its Affiliates.
    npm install -g cdktf-cli
    

    View Slide

37. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Some Terraform to my CDK
    

    View Slide

38. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Best Practices
    

    View Slide

39. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Best practices (1/5)
    • Layer your application to
    reduce blast radius when
    updating resources
    • Use multiple, isolated
    environments for testing,
    production, development,
    staging, etc.
    • Smaller files are easier to
    write, test, and troubleshoot
    Instances, Auto Scaling groups
    API endpoints, functions
    Alarms, dashboards
    VPCs, NAT gateways, VPNs, subnets
    IAM users, groups, roles, policies
    Front-end
    resources
    Backend
    services
    Stateful
    resources
    Base
    network
    Identity &
    security
    Monitoring
    resources
    Databases and clusters, queues
    

    View Slide

40. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Quotes
    “Please for the love of everything you hold dear, separate critical persistent
    storage from the rest of your IAC so you don’t accidentally remove it.
    Please.”
    - An Infrastructure as Code Developer with scars to prove it
    

    View Slide

41. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Best practices (2/5)
    • Start small and don’t try to
    boil the ocean
    • Work out simple resources
    first to get the hang of it.
    • Do not specify every little
    detail right from the start.
    

    View Slide

42. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Best practices (3/5)
    • It’s okay to repeat yourself
    • Do not engineer a whole new
    construct/library just so you
    prevent yourself from typing
    twice.
    • Do not overengineer things –
    this will help you out in the
    long run
    

    View Slide

43. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Quotes
    “It’s okay to repeat yourself in CDK. It’s not normal code, Don’t engineer a
    whole new construct just to prevent yourself from typing something twice”
    - An experience CDK developer
    “Keep in mind the operational aspects coming after you build the infra and
    make it as simple as possible to support. And more importantly,
    straightforward to troubleshoot. Operations will pay dearly for crazy
    abstractions and dependencies in your code”
    - Someone who had a run in with Operations
    

    View Slide

44. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Best practices (4/5)
    • Parameters and Mappings
    • Secrets Manager and SSM
    Parameter store
    • Do not hardcode sensitive
    information
    Resources:
    MyRDSDB:
    Type: "AWS::RDS::DBInstance"
    Properties:
    DBInstanceClass: db.t2.medium
    AllocatedStorage: ’20’
    Engine: mariadb
    EngineVersion: ’10.2’
    MasterUsername: appadmin
    MasterUserPassword: ‘{{resolve:ssm-secure:ssbRDSmEcntl:1}}'
    

    View Slide

45. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Best practices (5/5)
    • Test, Test, Test
    • Put proper guidelines in
    place before.
    • Introduce peer reviews of
    your infrastructure code.
    

    View Slide

46. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Quotes
    “Automation makes it easy to destroy your entire org with a few lines of
    yaml. Be paranoid about peer review, promoting changes through test
    environments, and privilege segregation"
    - An IaC Developer who, apparently, destroyed an entire org.
    “Automation let’s you mess up, at scale!”
    - Developer whor an the ‘* destroy’ command
    

    View Slide

47. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Takeaways!
    • Get into Infrastructure as Code Early, it will help you manage scale in
    thelong run. ☁
    • Treat infrastructure as code as any other code!
    • Use the tools that best fit your needs! ⚒
    

    View Slide

48. © 2020, Amazon Web Services, Inc. or its Affiliates.
    

    View Slide

49. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Yes
    

    View Slide

50. © 2020, Amazon Web Services, Inc. or its Affiliates.
    @darkosubotica
    ln/darko-mesaros
    twitch.tv/ruptwelve
    youtube.com/ruptwelve
    

    View Slide