Introduction to Docker on AWS

Docker containers are a fantastic technology for packaging, delivering, and running your production workloads. AWS offers a variety of services to help you run containers in the cloud. This deck is an overview of those services, what they do, and how you can benefit from using them.

Transcript

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Nathan Peck, Developer Advocate for Container Services March 21, 2018 Getting Started with Docker on AWS
2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Why Docker containers? FizzBuzz!
3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Application environment components Runtime Engine Code Dependencies
4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Local Laptop Staging / QA Production On-Premise Different environments
5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Local Laptop Staging / QA Production On-Prem It worked on my machine, why not in production? v6.0.0 v7.0.0 v4.0.0 v7.0.0
6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Docker Lightweight container virtualization platform. Licensed under the Apache 2.0 license. First released March 2013 Built by Docker, Inc.
7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Dependencies Binaries Code docker build
8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Docker Image Read only image that is used as a template to launch a container. Start from base images that have your dependencies, add your custom code. Docker file for easy, reproducible builds. bootfs kernel Base image Image Image W ritable Container add nginx add nodejs U buntu References parent image
9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Local Laptop Staging / QA Production On-Prem Four environments, same container
10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. VM vs Container Server (Host) Host OS Hypervisor Guest OS Guest OS Guest OS Bins/Libs Bins/Libs Bins/Libs App 1 App 2 App 3 VM Server (Host) Host OS Docker Bins/Libs Bins/Libs Bins/Libs App 1 App 2 App 3 Container
11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Benefits Portable runtime application environment Package application and dependencies in a single artifact Run different application versions (different dependencies) simultaneously Faster development & deployment cycles Better resource utilization
12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use Cases Consistent environment between Development & Production Service-Oriented Architectures & Microservices Short lived workflows, batch jobs, cron jobs Isolated environments for testing
13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Just four commands to start using Docker docker build (Create an image) docker tag (Set a version for the image) docker push (Store image in a registry to run later) docker run (Run the image on a machine)
14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Server Guest OS Bins/Libs Bins/Libs App2 App1 Using Docker is easy!
15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. A few hosts? Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS
16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lots of hosts! Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS
17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. How do I deliver container images to all those hosts?
18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Elastic Container Registry (Amazon ECR) • Cloud-based Docker image registry • Fully managed • Secure – images encrypted at rest, integrated with IAM • Scalable and Highly Available • Integrated with Amazon ECS and the Docker CLI
19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. How can I get containers running on my hosts?
20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Elastic Container Service (Amazon ECS) • Container management service • Fully managed • Scalable and Highly Available • Microservices, batch workers, machine learning applications • Integrated with • Amazon ECR • AWS networking, storage, management tools • AWS Fargate
22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon ECS EC2 INSTANCES ECS AGENT TASK Container TASK Container ECS AGENT TASK Container TASK Container AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT TASK Container TASK Container Internet LOAD BALANCER LOAD BALANCER
23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Cluster of hosts EC2 INSTANCES ECS AGENT TASK Container TASK Container ECS AGENT TASK Container TASK Container AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT TASK Container TASK Container Internet LOAD BALANCER LOAD BALANCER EC2 INSTANCES ECS AGENT TASK Container TASK Container ECS AGENT TASK Container TASK Container AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT TASK Container TASK Container Internet
24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lightweight agent on each host EC2 INSTANCES ECS AGENT TASK Container TASK Container ECS AGENT TASK Container TASK Container AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT TASK Container TASK Container Internet LOAD BALANCER LOAD BALANCER EC2 INSTANCES ECS AGENT TASK Container TASK Container ECS AGENT TASK Container TASK Container AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT TASK Container TASK Container Internet
25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. API for launching containers on the cluster EC2 INSTANCES ECS AGENT TASK Container TASK Container ECS AGENT TASK Container TASK Container AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT TASK Container TASK Container Internet LOAD BALANCER LOAD BALANCER EC2 INSTANCES ECS AGENT TASK Container TASK Container ECS AGENT TASK Container TASK Container AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT TASK Container TASK Container Internet
26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Container task is placed on a host EC2 INSTANCES ECS AGENT TASK Container TASK Container ECS AGENT TASK Container TASK Container AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT TASK Container TASK Container Internet LOAD BALANCER LOAD BALANCER
27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Traffic is sent to your host EC2 INSTANCES ECS AGENT TASK Container TASK Container ECS AGENT TASK Container TASK Container AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT TASK Container TASK Container Internet LOAD BALANCER LOAD BALANCER
28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. PRODUCTION WORKLOADS ON AWS AWS VPC networking mode Advanced task placement Deep integration with AWS platform ECS CLI … { } Global footprint Powerful scheduling engines Auto scaling CloudWatch metrics Load balancers
29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Customers Using Containers at Scale
30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CUSTOMERS ARE OUR KEY! 50+ releases since 2015 CUSTOMERS RELEASE FEEDBACK / NEW USE CASES LEARN EXPERIMENT, INNOVATE, & BUILD FEATURES
31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What we did with ECS in 2017 Container access to environmental metadata Network Load Balancer support Console support for SpotFleet Override parameters for RunTask and StartTask APIs Task Elastic Network Interface Application Load Balancer Support HIPAA eligibility Console UX improvements CLI V1.0 Container instance draining Windows containers Cron and Cloudwatch Event Task scheduling Support for Docker Privileged Mode Lifecycle Policies for container images Beijing Region Support for Device and Init flags Add attributes during boot Seoul Region Linux capabilities
32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. I don’t want to deal with hosts at all!
33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Cluster Management is a relic of physical infrastructure
34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ENABLE FOCUS ON APPLICATIONS
35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. INTRODUCING FARGATE!
36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CHANGING COMPUTE CONSUMPTION MODEL No instances to manage Task native API Resource based pricing Simple, easy to use, powerful – and new consumption model
37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. PRODUCTION WORKLOADS ON AWS AWS VPC networking mode Advanced task placement Deep integration with AWS platform ECS CLI … { } Global footprint Powerful scheduling engines Auto scaling CloudWatch metrics Load balancers
38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. I want to use more open source in my environment
39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Run Kubernetes for me.”
42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Native AWS Integrations.”
43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ”An Open Source Kubernetes Experience.”
44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. = + E L A S T I C C O N T A I N E R S E R V I C E F O R K U B E R N E T E S (EKS)
45. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. mycluster.eks.amazonaws.com Availability Zone 1 Availability Zone 2 Availability Zone 3 Kubectl
46. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GENERALLY AVAILABLE 2018
47. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. In Summary • Docker allows you to easily run different code across different machines in a standardized, easily defined environment • Amazon Elastic Container Service allows you to schedule and run Docker containers on AWS • Amazon Elastic Container Service for Kubernetes is a managed service for running Kubernetes on AWS • Amazon Elastic Container Registry is a secure, private registry for Docker container images
48. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. AMAZON CONTAINER SERVICES Choose your orchestration tool 1 Choose your launch type 2 ECS EKS EC2 Fargate EC2 Fargate We give you the power to choose:
49. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The awesome-ecs project: https://github.com/nathanpeck/awesome-ecs
50. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. We want to hear from all of you! More focus on supporting Tasks as compute primitive, more focus on removing undifferentiated heavy lifting. Our roadmap is driven by feedback:
51. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you!