Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DVC15: AWS Fargate instead of Amazon EC2 instances for one-off tasks

Sathya
December 05, 2019
990

DVC15: AWS Fargate instead of Amazon EC2 instances for one-off tasks

Presented at AWS re:Invent 2019 as part of DevChat, DV15.

This talk at looks at a use case where AWS Fargate is better for running one-off tasks instead of utility EC2 instances

Sathya

December 05, 2019
Tweet

Transcript

  1. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. AWS Fargate instead of Amazon EC2 instances for one-off tasks Sathyajith Bhat D V C 1 5 Senior DevOps Engineer Adobe
  2. About Me • Sathyajith Bhat • Senior DevOps Engineer, Adobe

    I/O • Author, Practical Docker With Python • AWS Community Hero & Organizer, Bangalore AWS Users Group
  3. What’s the Problem? • Need to get Database export from

    an RDS cluster • Goals • One click access to requested data • Should not be perpetual or require complicated logins • Ideally, it should be cleaned up automatically
  4. What’s the Problem? • Compute workload runs in Adobe Managed

    Infrastructure • No compute instances running in our AWS Accounts • Backing stores(Aurora MySQL, ElastiCache) in our AWS Accounts • VPC Peered with Managed Infra account • SpringBoot based Java microservice • Migrations on Container startup
  5. What’s the Problem? • Developers need to test database migrations

    • Local testing only covers so much • RDS snapshots are a pain • Single-AZ DB will result in suspended I/O • Snapshots can’t be exported to anything except RDS • MultiAZ DB export with mysqldump was ok • No compute instances, how do we get the data out? • Repeat multiple times, lot of wasted time
  6. What’s the Problem? • Data export process • Spin up

    an instance • Wait for instance to be ready • Fumble with SSH keys • Fumble with mysqldump command • Wait for data to be exported • Fumble with scp • Where to store the data export and how long?
  7. The Lambda Approach • Write a Lambda function which does

    equivalent of mysqldump • Wasted time in re-engineering the equivalent of mysqldump • Add mysqldump binary to package • system(“mysqldump…”) • Eh… • Problems? • Lambda limits to only 512 MB on /tmp • Lambda runtime limits(15 minutes currently, was 5 minutes earlier)
  8. Enter AWS Fargate • Managed Container service • (“compute engine

    for Amazon ECS that allows you to run containers without having to manage servers or clusters”) • ECS mode – “Fargate launch type” • No cluster* to manage, no nodes to run • Provide a Docker image, CPU & memory specs • Define networking, IAM • Launch a task *you still must create a “cluster” but don’t have to manage infra
  9. Enter AWS Fargate • Advantages • BYOD(aka Bring Your Own

    Docker image) • Private registries supported • Run in VPC • Invoke when required • Per usage billing • Starts from image pull, ends until task terminates • Run as task instead of service • Let the container exit when done • Save on costs • Export logs to CloudWatch, Splunk, FireLens, fluentd • Higher ephemeral storage space
  10. Enter AWS Fargate • Disadvantages • No free tier (#AWSWishlist,

    anyone?) • Minimum billing is 1 minute • Savings Plans brings reservations but no spot prices NEW! Fargate Spot now GA • Environment variable values listed on AWS Console • Use SSM Parameter store for secrets • Play CPU/Memory combo bingo • No support for persistent storage(or EBS/EFS) yet, but on the roadmap • https://github.com/aws/containers-roadmap/issues/53 • https://github.com/aws/containers-roadmap/issues/64
  11. How do I use this? # Create a cluster aws

    ecs create-cluster --cluster-name mysql- export --region=us-east-1 # Create a repository (optional) aws ecr create-repository --repository-name sathyabhat/mysql-export --region=us-east-1 # Create a log group aws logs create-log-group --log-group-name /ecs/mysql-export
  12. How do I use this? # Register a task definition

    aws ecs register-task-definition --cli-input- json file://mysql-export.json --region=us-east-1
  13. How do I use this? # Run the task task_arn=$(aws

    ecs run-task \ --cluster fargate-cluster \ --task-definition mysql-export:1 \ --overrides '{"containerOverrides":[{"name":"mysql-export","environment":[ {"name":"dbhost", "value":'\"$dbhost\“’}, {"name":"dbname","value":'\"$database\"'}]}]}' \ --launch-type FARGATE \ --network-configuration 'awsvpcConfiguration={subnets=['subnet- 1a2b3c4d'],securityGroups=['sg-0fa6089fc94b1438'],assignPublicIp='DISABLED'}' \ --region=us-east-1 | jq -r .tasks[].taskArn)
  14. How do I use this? # Wait for the task

    to be completed echo "Waiting for export to complete.. " aws ecs wait tasks-stopped \ --cluster fargate-cluster \ --tasks $task_arn \ --region=us-east-1
  15. What did we get by this move? • Old approach:

    • Spin up instance and wait for instance to be ready: 2 minutes • Fumble with the right set of SSH keys: 2 minutes • SSH to bastion: 10 seconds • Alerts fired, investigate: 5 minutes • Lunch time!
  16. What did we get by this move? • Old approach:

    • Recall why did I connect to instance? • Find out the right set of Database host and credentials: 10 minutes • Run the command: 2 seconds • Get the command wrong and search for the right command: 5 minutes • Phew, command worked. • Try to SCP the file out: 5 minutes • Realize you can’t share the file and copy to S3: 2 minutes
  17. What did we get by this move? • Fargate approach:

    • Run the Jenkins / other task runner tool job: 10 seconds
  18. We have the data, what next? • Goals: • One

    click access to requested data • Should not be perpetual or require complicated logins but still be “secure enough” • Data exports should be cleaned up automatically.
  19. Enter S3 • Simple Storage Service - simple to use

    but offers quite a bit of features • Upload data exports to a private bucket • Generate a pre-signed URL valid for an hour • Setup lifecycle policies to expire out the objects after a day. • Goals • One click access to requested data • Should not be perpetual or require complicated logins • Ideally, it should be cleaned up automatically
  20. Tying it all together • Setup Jenkins/your favorite task runner

    to Invoke the Fargate task on demand • Profit! • Code • https://u.sbhat.me/dvc15
  21. Code & Related Breakouts • https://u.sbhat.me/dvc15 • DVC02 Cutting-edge architectures

    based on AWS AppSync, Lambda, and Fargate • CON208 Build your microservices application on AWS Fargate • CON423 AWS Fargate under the hood
  22. Thank you! © 2019, Amazon Web Services, Inc. or its

    affiliates. All rights reserved. Sathyajith Bhat Twitter/GitHub/LinkedIn: SathyaBhat