Supercharge product development with cloud best practices - Web Summit 2019

Supercharge product development with cloud best practices - Web Summit 2019

AWS Developer Theater @ Web Summit 2019

36ca8ae7e19067021d0e39c0b72acc2a?s=128

Alex Casalboni

November 05, 2019
Tweet

Transcript

  1. 1.

    © 2019, Amazon Web Services, Inc. or its Affiliates. Supercharge

    product development with cloud best practices Alex Casalboni Technical Evangelist, AWS @alex_casalboni 5 November 2019
  2. 2.

    © 2019, Amazon Web Services, Inc. or its Affiliates. About

    me Software Engineer & Web Developer Data science background Worked in a startup for 4.5 years ServerlessDays global committee (Happy) AWS customer since 2013
  3. 3.

    © 2019, Amazon Web Services, Inc. or its Affiliates. Development

    transformation @ Amazon Modern applications approach Infrastructure as Code Continuous Integration Continuous Deployment Demo Agenda
  4. 4.

    Development transformation at Amazon: 2001–2002 monolithic application + teams 2001

    Lesson learned: decompose for agility 2002 microservices + 2 pizza teams
  5. 8.

    Microservice development lifecycle developers services monitor release test build delivery

    pipelines monitor release test build monitor release test build monitor release test build monitor release test build monitor release test build
  6. 10.

    What changes do you need to make to adopt these

    best practices? Serverless No provisioning/management Automatic scaling Pay for value billing Availability and resiliency Microservices Componentization Business capabilities Products not projects Infrastructure automation DevOps Cultural philosophies Cross-disciplinary teams CI/CD Automation tools DEV OPS Architectural patterns Operational Model Software Delivery
  7. 12.

    Operational responsibility AWS Lambda Serverless functions AWS Fargate Serverless containers

    ECS/EKS Container-management as a service EC2 Infrastructure-as-a-Service More opinionated Less opinionated AWS manages Customer manages • Data source integrations • Physical hardware, software, networking, and facilities • Provisioning • Application code • Container orchestration, provisioning • Cluster scaling • Physical hardware, host OS/kernel, networking, and facilities • Application code • Data source integrations • Security config and updates, network config, management tasks • Container orchestration control plane • Physical hardware software, networking, and facilities • Application code • Data source integrations • Work clusters • Security config and updates, network config, firewall, management tasks • Physical hardware software, networking, and facilities • Application code • Data source integrations • Scaling • Security config and updates, network config, management tasks • Provisioning, managing scaling and patching of servers
  8. 13.

    Approaches to modern application development • Simplify environment management •

    Reduce the impact of code changes • Automate operations • Accelerate the delivery of new, high-quality services • Gain insight across resources and applications • Protect customers and the business Simplify environment management with serverless technologies Reduce the impact of code changes with microservice architectures Automate operations by modeling applications & infrastructure as code Accelerate the delivery of new, high-quality services with CI/CD Gain insight across resources and applications by enabling observability Protect customers and the business with end-to-end security & compliance
  9. 14.

    Approaches to modern application development • Simplify environment management with

    serverless technologies • Reduce the impact of code changes with microservice architectures • Automate operations by modeling applications & infrastructure as code • Accelerate the delivery of new, high-quality services with CI/CD • Gain insight across resources and applications by enabling observability • Protect customers and the business with end-to-end security & compliance
  10. 15.

    Approaches to modern application development Serverless containers Long-running Abstracts the

    OS Fully managed orchestration Fully managed cluster scaling Serverless functions Event-driven Many language runtimes Data source integrations No server management
  11. 16.

    Approaches to modern application development • Simplify environment management with

    serverless technologies • Reduce the impact of code changes with microservice architectures • Automate operations by modeling applications & infrastructure as code • Accelerate the delivery of new, high-quality services with CI/CD • Gain insight across resources and applications by enabling observability • Protect customers and the business with end-to-end security & compliance
  12. 20.

    Release process stages Source Build Test Production • Integration tests

    • Load testing • UI/UX tests • Security testing • Check-in source code (git) • Peer reviews • Compile code • Unit tests • Style checkers • Create artifacts (images & deployment packages) • Deployment to production • Monitor prod to detect errors
  13. 22.

    © 2019, Amazon Web Services, Inc. or its Affiliates. Pillars

    of releasing modern applications Infrastructure as code
  14. 23.

    Infrastructure as code (IaC) Declarative I tell you what I

    need I tell you what to do Imperative
  15. 24.

    IaC goals 1. Make infrastructure changes repeatable and predictable 2.

    Release infrastructure changes using the same tools as code changes 3. Replicate production environment in a staging environment to enable continuous testing
  16. 25.

    AWS Serverless Application Model (SAM) • Open source framework for

    building serverless applications on AWS • Shorthand syntax to express functions, APIs, databases, and event source mappings • Transforms and expands SAM syntax into AWS CloudFormation syntax on deployment • Supports all AWS CloudFormation resource types github.com/awslabs/serverless-application-model
  17. 26.

    SAM template AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: GetFunction: Type: AWS::Serverless::Function

    Properties: Handler: index.get Runtime: nodejs8.10 CodeUri: src/ Policies: - DynamoDBReadPolicy: TableName: !Ref MyTable Events: GetResource: Type: Api Properties: Path: /resource/{resourceId} Method: get MyTable: Type: AWS::Serverless::SimpleTable Just 20 lines to create: • Lambda function • IAM role • API Gateway • DynamoDB table
  18. 27.

    SAM CLI: test, package and deploy pip install --user aws-sam-cli

    sam logs sam validate sam local sam init sam build sam package sam deploy sam publish github.com/awslabs/aws-sam-cli
  19. 28.

    AWS Cloud Development Kit (CDK) • Open source framework to

    define cloud infrastructure in TypeScript, Python, Java, C#, … • Provides library of higher-level resource types (“construct” classes) with built-in AWS best practices, packaged as npm/pip/maven modules • Provisions resources with CloudFormation • Supports all CloudFormation resource types AWS CDK github.com/aws/aws-cdk
  20. 29.

    CDK template import ec2 = require('@aws-cdk/aws-ec2'); import ecs = require('@aws-cdk/aws-ecs');

    import cdk = require('@aws-cdk/cdk'); class BonjourFargate extends cdk.Stack { constructor(parent: cdk.App, name: string, props?: cdk.StackProps) { super(parent, name, props); const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 }); const cluster = new ecs.Cluster(this, 'Cluster', { vpc }); new ecs.LoadBalancedFargateService( this, "FargateService", { cluster, image: ecs.DockerHub.image("amazon/amazon-ecs-sample"), }); } } const app = new cdk.App(); new BonjourFargate(app, 'Bonjour'); app.run();
  21. 30.

    import ec2 = require('@aws-cdk/aws-ec2'); import ecs = require('@aws-cdk/aws-ecs'); import cdk

    = require('@aws-cdk/cdk'); class BonjourFargate extends cdk.Stack { constructor(parent: cdk.App, name: string, props?: cdk.StackProps) { super(parent, name, props); const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 }); const cluster = new ecs.Cluster(this, 'Cluster', { vpc }); new ecs.LoadBalancedFargateService( this, "FargateService", { cluster, image: ecs.DockerHub.image("amazon/amazon-ecs-sample"), }); } } const app = new cdk.App(); new BonjourFargate(app, 'Bonjour'); app.run(); CDK template
  22. 31.

    22 Lines 400 lines CDK template import ec2 = require('@aws-cdk/aws-ec2');

    import ecs = require('@aws-cdk/aws-ecs'); import cdk = require('@aws-cdk/cdk'); class BonjourFargate extends cdk.Stack { constructor(parent: cdk.App, name: string, props?: cdk.StackProps) { super(parent, name, props); const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 }); const cluster = new ecs.Cluster(this, 'Cluster', { vpc }); new ecs.LoadBalancedFargateService( this, "FargateService", { cluster, image: ecs.DockerHub.image("amazon/amazon-ecs-sample"), }); } } const app = new cdk.App(); new BonjourFargate(app, 'Bonjour'); app.run();
  23. 32.

    CDK CLI: synthesize and deploy npm install -g aws-cdk cdk

    init app --language typescript cdk synth cdk deploy
  24. 33.

    © 2019, Amazon Web Services, Inc. or its Affiliates. Pillars

    of releasing modern applications Infrastructure as code
  25. 34.

    © 2019, Amazon Web Services, Inc. or its Affiliates. Pillars

    of releasing modern applications Continuous integration
  26. 36.

    Continuous integration goals 1. Automatically kick off a new release

    when new code is checked in 2. Build and test code in a consistent, repeatable environment 3. Continually have an artifact ready for deployment 4. Continually close feedback loop when build fails
  27. 37.

    AWS CodePipeline • Continuous delivery service for fast and reliable

    application updates • Model and visualize software release process • Builds, tests, and deploys your code every time there is a code change • Integrates with third-party tools
  28. 38.

    AWS CodePipeline: Supported sources Pick branch AWS CodeCommit GitHub Pick

    object or folder Amazon S3 Pick Docker tag Amazon ECR
  29. 39.

    AWS CodePipeline: Supported triggers Amazon CloudWatch Events • Scheduled (nightly

    release) • AWS Health events (Fargate platform retirement) Webhooks • DockerHub • Quay • Artifactory
  30. 40.
  31. 41.
  32. 42.

    AWS CodeBuild • Fully managed build service that compiles source

    code, runs tests, and produces software packages • Scales continuously and processes multiple builds concurrently • No build servers to manage • Pay by the minute, only for the compute resources you use • Monitor builds through CloudWatch Events
  33. 43.

    AWS CodeBuild: Lambda buildspec version: 0.2 phases: build: commands: -

    npm ci - npm test - > aws cloudformation package --template-file template.yaml --output-template packaged.yaml --s3-bucket $BUCKET artifacts: type: zip files: - packaged.yaml
  34. 44.

    AWS CodeBuild: Lambda buildspec with SAM CLI version: 0.2 phases:

    install: commands: - pip install --upgrade awscli aws-sam-cli build: commands: - sam build - sam package --s3-bucket $BUCKET --output-template-file packaged.yaml artifacts: type: zip files: - packaged.yaml
  35. 45.

    AWS CodeBuild: Docker buildspec version: 0.2 phases: build: commands: -

    $(aws ecr get-login --no-include-email) - docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG . - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $ECR_REPO:$IMAGE_TAG - docker push $ECR_REPO:$IMAGE_TAG
  36. 46.
  37. 47.
  38. 48.
  39. 49.

    © 2019, Amazon Web Services, Inc. or its Affiliates. Pillars

    of releasing modern applications Continuous integration
  40. 50.

    © 2019, Amazon Web Services, Inc. or its Affiliates. Pillars

    of releasing modern applications Continuous deployment
  41. 52.

    Continuous deployment goals 1. Automatically deploy changes to staging environments

    for testing 2. Deploy to production safely without impacting customers 3. Deliver to customers faster: increase frequency, reduce failure rate
  42. 53.

    AWS CodeDeploy • Automates code deployments to any instance or

    function • Handles the complexity of updating your applications • Avoid downtime during deployment • Roll back automatically if failure is detected • Deploy to Amazon EC2, Lambda, or on-premises
  43. 54.

    CodeDeploy – Lambda deployments Enable in your serverless application template

    Resources: GetFunction: Type: AWS::Serverless::Function Properties: DeploymentPreference: Type: Canary10Percent10Minutes Alarms: - !Ref ErrorsAlarm Hooks: PreTraffic: !Ref PreTrafficHook Canary10Percent30Minutes Canary10Percent5Minutes Canary10Percent10Minutes Canary10Percent15Minutes Linear10PercentEvery10Minutes Linear10PercentEvery1Minute Linear10PercentEvery2Minutes Linear10PercentEvery3Minutes AllAtOnce
  44. 55.

    CodeDeploy – Lambda canary deployment API Gateway Lambda function weighted

    alias “live” v1 code 100% Run PreTraffic hook against v2 code before it receives traffic v2 code 0%
  45. 56.

    CodeDeploy – Lambda canary deployment API Gateway Lambda function weighted

    alias “live” v1 code 90% Wait for 10 minutes, roll back in case of alarm v2 code 10%
  46. 57.

    CodeDeploy – Lambda canary deployment API Gateway Lambda function weighted

    alias “live” v1 code 0% Run PostTraffic hook and complete deployment v2 code 100%
  47. 60.

    © 2019, Amazon Web Services, Inc. or its Affiliates. Pillars

    of releasing modern applications Continuous deployment
  48. 62.

    © 2019, Amazon Web Services, Inc. or its Affiliates. Takeaways

    1. Manage your infrastructure as code 2. Frequently build and integrate your code to get a first feedback 3. Continuously release in production using canary releases with monitoring and automated rollbacks 4. Use canary releases to get both technical and business feedback
  49. 63.

    14:00 Choosing the right Database for your Applications [Steven Bryen]

    15:00 Making Sense of Machine Learning for Your Organization [Antje Barth] 16:00 Getting Started with Serverless Chatbots [Marcia Villalba] Today
  50. 64.

    10:00 How to build on AWS without knowing much about

    AWS [Sebastien Stormacq] 11:00 Chaos Engineering: Why Breaking Things Should be Practiced [Boaz Ziniman] 12:00 Data lakes and analytics in the Cloud for developers and founders [Javier Ramirez] 13:00 An Introduction to Deep Learning [Antje Barth] 14:00 Tools for Building your MVP on AWS [Rob de Feo] 15:00 Improving your security posture with the AWS Cloud [Steven Bryen] 16:00 Understanding Graph Databases [Robert Zhu] Tomorrow
  51. 65.

    10:00 How to build on AWS without knowing much about

    AWS [Alex Casalboni] 11:00 Microservices and containers [Frank Munz] 12:00 Building modern APIs with GraphQL [Robert Zhu] 13:00 Why adding a Service Mesh to your containers? [Frank Munz] 14:00 Immutable & distributed transactions: your ledger databases & blockchain in the cloud [Javier Ramirez] 15:00 Improving UX through observability [Enrique Duvos] 16:00 Build a mobile app with machine learning [Nicki Stone] (no ML expertise required) Thursday
  52. 66.

    © 2019, Amazon Web Services, Inc. or its Affiliates. ©

    2019, Amazon Web Services, Inc. or its Affiliates. Thank you! Alex Casalboni Technical Evangelist, AWS @alex_casalboni