DVC15: AWS Fargate instead of Amazon EC2 instances for one-off tasks

Presented at AWS re:Invent 2019 as part of DevChat, DV15.

This talk at looks at a use case where AWS Fargate is better for running one-off tasks instead of utility EC2 instances

Transcript

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Fargate instead of Amazon EC2 instances for one-off tasks Sathyajith Bhat D V C 1 5 Senior DevOps Engineer Adobe
3. Agenda About Me Background and Problem at Hand AWS Fargate Why Not Lambda Show the codez!
4. About Me • Sathyajith Bhat • Senior DevOps Engineer, Adobe I/O • Author, Practical Docker With Python • AWS Community Hero & Organizer, Bangalore AWS Users Group
5. What’s the Problem? • Need to get Database export from an RDS cluster • Goals • One click access to requested data • Should not be perpetual or require complicated logins • Ideally, it should be cleaned up automatically
6. What’s the Problem? • Compute workload runs in Adobe Managed Infrastructure • No compute instances running in our AWS Accounts • Backing stores(Aurora MySQL, ElastiCache) in our AWS Accounts • VPC Peered with Managed Infra account • SpringBoot based Java microservice • Migrations on Container startup
7. What’s the Problem? • Developers need to test database migrations • Local testing only covers so much • RDS snapshots are a pain • Single-AZ DB will result in suspended I/O • Snapshots can’t be exported to anything except RDS • MultiAZ DB export with mysqldump was ok • No compute instances, how do we get the data out? • Repeat multiple times, lot of wasted time
8. What’s the Problem? • Data export process • Spin up an instance • Wait for instance to be ready • Fumble with SSH keys • Fumble with mysqldump command • Wait for data to be exported • Fumble with scp • Where to store the data export and how long?
9. The Lambda Approach • Write a Lambda function which does equivalent of mysqldump • Wasted time in re-engineering the equivalent of mysqldump • Add mysqldump binary to package • system(“mysqldump…”) • Eh… • Problems? • Lambda limits to only 512 MB on /tmp • Lambda runtime limits(15 minutes currently, was 5 minutes earlier)
10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
11. Enter AWS Fargate • Managed Container service • (“compute engine for Amazon ECS that allows you to run containers without having to manage servers or clusters”) • ECS mode – “Fargate launch type” • No cluster* to manage, no nodes to run • Provide a Docker image, CPU & memory specs • Define networking, IAM • Launch a task *you still must create a “cluster” but don’t have to manage infra
12. Enter AWS Fargate • Advantages • BYOD(aka Bring Your Own Docker image) • Private registries supported • Run in VPC • Invoke when required • Per usage billing • Starts from image pull, ends until task terminates • Run as task instead of service • Let the container exit when done • Save on costs • Export logs to CloudWatch, Splunk, FireLens, fluentd • Higher ephemeral storage space
13. Enter AWS Fargate • Disadvantages • No free tier (#AWSWishlist, anyone?) • Minimum billing is 1 minute • Savings Plans brings reservations but no spot prices NEW! Fargate Spot now GA • Environment variable values listed on AWS Console • Use SSM Parameter store for secrets • Play CPU/Memory combo bingo • No support for persistent storage(or EBS/EFS) yet, but on the roadmap • https://github.com/aws/containers-roadmap/issues/53 • https://github.com/aws/containers-roadmap/issues/64
14. How do I use this? # Create a cluster aws ecs create-cluster --cluster-name mysql- export --region=us-east-1 # Create a repository (optional) aws ecr create-repository --repository-name sathyabhat/mysql-export --region=us-east-1 # Create a log group aws logs create-log-group --log-group-name /ecs/mysql-export
15. How do I use this? # Register a task definition aws ecs register-task-definition --cli-input- json file://mysql-export.json --region=us-east-1
16. How do I use this? # Run the task task_arn=$(aws ecs run-task \ --cluster fargate-cluster \ --task-definition mysql-export:1 \ --overrides '{"containerOverrides":[{"name":"mysql-export","environment":[ {"name":"dbhost", "value":'\"$dbhost\“’}, {"name":"dbname","value":'\"$database\"'}]}]}' \ --launch-type FARGATE \ --network-configuration 'awsvpcConfiguration={subnets=['subnet- 1a2b3c4d'],securityGroups=['sg-0fa6089fc94b1438'],assignPublicIp='DISABLED'}' \ --region=us-east-1 | jq -r .tasks[].taskArn)
17. How do I use this? # Wait for the task to be completed echo "Waiting for export to complete.. " aws ecs wait tasks-stopped \ --cluster fargate-cluster \ --tasks $task_arn \ --region=us-east-1
18. What did we get by this move? • Old approach: • Spin up instance and wait for instance to be ready: 2 minutes • Fumble with the right set of SSH keys: 2 minutes • SSH to bastion: 10 seconds • Alerts fired, investigate: 5 minutes • Lunch time!
19. What did we get by this move? • Old approach: • Recall why did I connect to instance? • Find out the right set of Database host and credentials: 10 minutes • Run the command: 2 seconds • Get the command wrong and search for the right command: 5 minutes • Phew, command worked. • Try to SCP the file out: 5 minutes • Realize you can’t share the file and copy to S3: 2 minutes
20. What did we get by this move? • Fargate approach: • Run the Jenkins / other task runner tool job: 10 seconds
21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
22. We have the data, what next? • Goals: • One click access to requested data • Should not be perpetual or require complicated logins but still be “secure enough” • Data exports should be cleaned up automatically.
23. Enter S3 • Simple Storage Service - simple to use but offers quite a bit of features • Upload data exports to a private bucket • Generate a pre-signed URL valid for an hour • Setup lifecycle policies to expire out the objects after a day. • Goals • One click access to requested data • Should not be perpetual or require complicated logins • Ideally, it should be cleaned up automatically
24. Tying it all together • Setup Jenkins/your favorite task runner to Invoke the Fargate task on demand • Profit! • Code • https://u.sbhat.me/dvc15
25. Code & Related Breakouts • https://u.sbhat.me/dvc15 • DVC02 Cutting-edge architectures based on AWS AppSync, Lambda, and Fargate • CON208 Build your microservices application on AWS Fargate • CON423 AWS Fargate under the hood
26. Thank you! © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Sathyajith Bhat Twitter/GitHub/LinkedIn: SathyaBhat
27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.