And you thought you knew EC2, GOT edition

Presented at AWS re:Invent 2017

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. And you thought you knew EC2… Ben Whaley D i r e c t o r , S e c u r i t y a n d O p e r a t i o n s K o u n t a b l e
2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Pertinent announcements since June 2017 Docker Device and Init Flags in Container Task Definitions Amazon ECS Allows Containers to Directly Access Environmental Metadata Announcing New AWS Deep Learning AMI for Amazon EC2 P3 Instances
 Amazon EC2 Systems Manager Parameter Store Adds Versioning Support Amazon EC2 Systems Manager Now Integrates With GitHub Application Load Balancers Now Support Multiple TLS Certificates With Smart Selection Using SNI Introducing Amazon EC2 P3 Instances Introducing Lifecycle Policies for Amazon EC2 Container Registry Application Load Balancers now support multiple SSL certificates EC2 Per second billing ECS Adds Support for Adding or Dropping Linux Capabilities to Containers Network Load Balancer now supports load balancing to IP addresses as targets Amazon EC2 Spot Can Now Stop and Start Your Spot Instances Amazon EC2 Systems Manager Run Command Adds Tag-Based Permissions and Multi- Tag Support Auto Scaling Lifecycle Hooks Enhancements A new addition to the Amazon EC2 memory-optimized X1 Instance family – x1e.32xlarge Amazon EC2 Container Service Now Integrated with Network Load Balancer Application Load Balancer Adds Support for New RequestCountPerTarget CloudWatch Metric EC2 Systems Manager Now Supports Linux Patching Sync Amazon EC2 Systems Manager Inventory Data to Amazon S3 Buckets ECS RunTask and StartTask APIs now support additional override parameters EC2 Systems Manager Adds Hierarchy, Tagging, and Notification Support for Parameter Store Announcing Network Load Balancer for Elastic Load Balancing Announcing improved networking performance for Amazon EC2 instances Application Load Balancer now supports load balancing to IP addresses as targets Amazon EC2 Systems Manager Adds Configuration Compliance Reporting and Auto- Remediation Amazon ECS is now HIPAA Eligible Amazon EC2 Systems Manager now HIPAA eligible Tag Your Spot Fleet EC2 Instances Introducing Amazon EC2 G3 Instances, the next-generation of GPU-powered instances for graphics-intensive applications Support for LCU metrics on Classic Load Balancer Amazon EC2 Systems Manager Adds Cross-Platform and Multi-Step Document Support Amazon EC2 Systems Manager Adds Raspbian OS and Raspberry Pi Support Introducing Target Tracking Scaling Policies for Auto Scaling
3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. C5 Compute-optimized with 3.0GHz Intel Skylake P3 Next-gen GPU instances suitable for ML, HPC R4 Memory optimized (up to 488GiB) I3 High I/O with NVMe SSD 10k-300k IOPS X1e In-memory databases - up to 3.8TiB, 128vCPU D2 Up to 43TiB HDD EC2 Instance Types
4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EBS Volume Types Type Description I/O Throughput Cost io1 Provisioned IOPS SSD Highest High Highest gp2 General purpose SSD High Low(ish) Medium st1 Throughput-optimized HDD Low Highest Low sc1 Non-optimized HDD Low Medium Lowest
5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Miscellaneous changes, features ▪ IPv6 ▪ KVM hypervisor for new instance types ▪ Elastic Network Adapter - 25 Gbps ▪ Per second billing ▪ Elastic GPUs ▪ Target Tracking Scaling Policies for Auto Scaling ▪ New regions 2016 - Ohio, Canada, London, Mumbia, Seoul Soon - China (Ningxia), Paris, Stockholm, Hong Kong, Bahrain
 #awswishlist - Kenya
6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CloudWatch Integration Scheduled tasks Event handling Metrics Logs
7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CloudWatch Events ▪ Scheduled Events ▪ Instance state changes ▪ Systems manager ▪ Maintenance window ▪ ECS event stream ▪ EBS snapshots, encryption/decryption ▪ Cross-account event delivery
8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CloudWatch Telemetry ▪ High resolution (1s/3H) custom metrics ▪ High resolution alarms (10s) ▪ collectd CloudWatch plugin ▪ Dashboards (GUI, CLI) ▪ Percentile statistics (p50, p90, p99…) ▪ Logs ▪ Metric filters for log parsing and alarms
9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EC2 Container Service ▪ Elastic Container Registry ▪ Docker Device and Init Flags in Container Task Definitions ▪ CloudWatch metrics for CPU and memory utilization across the cluster ▪ IAM roles for ECS tasks ▪ github.com/blox/blox marching towards v1.0 ▪ 3rd party tooling (Convox, Empire) ▪ Integration with Application Load Balancer ▪ Run tasks on a schedule ▪ Execute tasks in response to CloudWatch events
10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EC2 Systems Manager Superpowers for EC2 instances and on-premises systems. ▪ Remote command execution with Run Commands ▪ Controlled secrets and configuration data with the Parameter Store ▪ Periodic tasks with the State Manager and Maintenance Windows ▪ Stepwise Automation workflows for initializing nodes ▪ Collect and query Inventory and Patch status
11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Key Systems Manager Benefits ✓ All actions recorded in CloudTrail (e.g. immutable audit trail) ✓ Trigger SNS, Lambda from Systems Manager events ✓ Store command history and output to S3 ✓ Fine-grained access control to Run Commands ✓ Integration with Config to track changes over time
12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The SSM Agent ▪ Open source (Golang) executable for Linux and Windows ▪ Available for cloud and on-premises systems ▪ Assign IAM role with permissions to interface with SSM API ▪ Install at boot or on existing systems ▪ Polls for commands to execute
13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. { "schemaVersion":"2.0", "description":"Run a script", "parameters":{ "commands":{ "type":"String", “description”:"Commands to run" } }, "mainSteps":[ { "action":"aws:runShellScript", "name":"runShellScript", "inputs":{ "runCommand":"{{ commands }}" } } ] } Systems Manager Documents
14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. github.com/kountable/pssh A s h e l l f o r t h e E C 2 P a r a m e t e r S t o r e
15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. λ pssh CloudWatch Event EC2 Parameter Store Game of Thrones Transfer of Power CloudWatch Metrics
16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Scoring System -40 Illegitimate -30 Handicapped -20 Exile 0 Deceased 30 Distinction 50 Noble 50 Royalty 100 Magical
17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Community Curated Pro Tips
18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Automated infrastructure is table stakes.
19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Ansible for provisioning and fleet control. Lambdas for glue. Packer for building images.
20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Single node ASGs for self-healing and resilience.
21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use Config Rules to enforce compliance with tagging schemes, EBS snapshots, security group rules, or other site preferences.
22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Many EC2 IAM actions do not support resource-level permissions. Exercise caution. { "Statement": [ { "Action": [ "ec2:AllocateAddress", "ec2:AssociateAddress", "ec2:DescribeAddresses", "ec2:DisassociateAddress", "ec2:CreateKeyPair", "ec2:DeleteKeyPair" ], "Effect": "Allow", "Resource": [ "*" ] } ] }
23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use the BurstBalance CloudWatch metric to monitor I/O credit balance for gp2, st1, sc1 EBS volumes.
24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use the CPUCreditUsage and CPUCreditBalance metrics in CloudWatch to track CPU burst usage on T2 instance types.
25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. github.com/awslabs/aws-shell T h e r e a l h e r o
26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Network throughput increases substantially with instance type. (Don't forget to enable enhanced networking)
27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. In CloudFormation, explicitly request SSD ephemeral disks or you may not get them.
28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Require MFA for SSH access and enable Fail2ban to block IPs with failed login attempts.
29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Improve your SSH experience with ControlPersist.
30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use ssh -D and the SwitchyOmega Chrome extension for convenient access to services in a private network.
31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Running multiple apps per instance? Use AssumeRole to assign granular permissions to each app.
32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Protect the EC2 metadata and userdata.
33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use the instance identity document to validate the authenticity of EC2 instances.
34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use Linux >= 4.4 for best results.
35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. checkip.amazonaws.com is HTTP only hence it cannot be trusted. Use icanhazip.com instead.
36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. #!/bin/bash -x # Save userdata cmd output to a log exec > /var/log/userdata.log 2>&1 # Initialize instance …
37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Pace yourself tonight. It’s going to be a busy week.
38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Recently released!
39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ` Ben Whaley D i r e c t o r , S e c u r i t y a n d O p e r a t i o n s K o u n t a b l e