Upgrade to Pro — share decks privately, control downloads, hide ads and more …

And you thought you knew EC2

And you thought you knew EC2

EC2 features, best practices, and pro tips. Presented at AWS Community Day 2017 in San Francisco.

Ben Whaley

June 15, 2017

More Decks by Ben Whaley

Other Decks in Technology


  1. Instance Types Family Intended use C5 Compute-optimized with Intel Skylake*

    R4 Memory-optimized, up to 488GiB** I3 High I/O - Watch for sharp corners F1 FPGA P2 GPU - 2x perf of G2 at 1.5x the cost D2 Dense Storage, up to 48TiB HDD & 10GiB network * Coming soon? ** Only 244GiB currently
  2. 4 EBS Volume Types Type Description I/O Throughput Cost io1

    Provisioned IOPS SSD Highest High Highest gp2 General purpose SSD High Low(ish) Medium st1 Throughput-optimized HDD Low Highest Low sc1 Non-optimized HDD Low Medium Lowest
  3. Miscellaneous improvements & features • IPv6! • VPC endpoints for

    DynamoDB, S3 • Lightsail - EC2 quick start • Elastic GPUs • New regions • 2016 - Ohio, Canada, London, Mumbia, Seoul • Ningxia, Paris, Stockholm
  4. EC2 Systems Manager Superpowers for EC2 instances and on-premises systems.

    • Remote command execution with Run Commands • Controlled secrets and configuration data with the Parameter Store • Periodic tasks with the State Manager and Maintenance Windows • Stepwise Automation workflows for initializing nodes • Collect and query Inventory and Patch status
  5. Systems Manager Essentials • The SSM agent • Open source

    (Golang) executable for Linux and Windows • Available for cloud and on-premises systems • Assign IAM role with permissions to interface with SSM API • Install at boot or on existing systems • Polls for commands to execute • All actions recorded in CloudTrail (e.g. immutable audit trail) • Trigger SNS, Lambda from Systems Manager events • Store command history and output to S3 • Fine-grained access control to Run Commands • Integration with Config to track changes over time
  6. Systems Manager Documents JSON Schema describing actions for the systems

    manager { "schemaVersion":"2.0", "description":"Run a script", "parameters":{ "commands":{ "type":"String", “description”:"Commands to run" } }, "mainSteps":[ { "action":"aws:runShellScript", "name":"runShellScript", "inputs":{ "runCommand":"{{ commands }}" } } ] }
  7. Quick & dirty SSM demo: 1. Scale up ASG of

    managed instances 2. Run a command on remote agents 3. Store and retrieve a parameter Virtual Private Cloud Private subnet EC2 Instances Systems Manager ssm-agent Workstations
  8. Five Ways to Provision Instances Configuration management runs at boot,

    registers with server, converges a configuration 3
  9. Five Ways to Provision Instances Autoscaling lifecycle hook —> CloudWatch

    event —> Run Command —> execute provisioning documents 4 (Alternative: CloudWatch event —> Lambda)
  10. Application Load Balancer • Almost as good as HAProxy or

    NGINX. Almost. • Host- and path-based routing • Additional metrics (# active connections, total traffic) • Improved health checks • Websockets • HTTP/2 • Integration with X-Ray (adds X-Amzn-Trace-Id header) • Integration with ECS • Integration with WAF
  11. ELB IP 1 ELB IP 2 Clients Clients Clients Clients

    Moar clients? Feeling… weak… must… scale up. ELB IP 1 ELB IP 2 Clients Clients Couple of clients? Come at me, bro.
  12. ELB IP 3 ELB IP 4 Clients Clients I sense

    a disturbance in the force… as if hundreds of clients suddenly cried out in pain Clients Clients ELB IP 1 ELB IP 2 Bueller? Bueller? … moments later
  13. Clients Lookup A record for myservice.example.com Route53 IP 1 IP

    2 Clients Client Connect to IP 1 Node with IP 1 Client Connect to IP 2 Node with IP 2 health checks ASG
  14. Clients Connect to IP 1 Clients Connect to IP 2

    ASG Node with IP 1 Node with IP 2 Clients Clients Clients Clients Clients Clients 1. Autoscale via custom CloudWatch metric 2. New node boots with IP 3 3. Autoscaling lifecycle hook adds IP 3 to myservice.example.com A record Moar clients!
  15. Clients Connect to IP 1 Clients Connect to IP 2

    ASG Node with IP 1 Node with IP 2 Clients Clients Clients Clients Clients Clients Clients Connect to IP 3 Node with IP 3 Clients Clients Clients
  16. EC2 Container Service Updates • ECR - best registry to

    use for AWS container workloads • Support for volumes • CloudWatch metrics for CPU and memory utilization across the cluster (set alarms for autoscaling) • IAM roles for ECS tasks • Blox allows custom schedulers (github.com/blox/blox) • 3rd party tooling (Convox, Empire) • Integration with ALB • Run tasks on a schedule • Execute tasks in response to CloudWatch events
  17. Use ssh -D and the SwitchyOmega Chrome extension for convenient

    access to services in a private network.