Slide 1

Slide 1 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. And you thought you knew EC2… Ben Whaley D i r e c t o r , S e c u r i t y a n d O p e r a t i o n s K o u n t a b l e

Slide 2

Slide 2 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Pertinent announcements since June 2017 Docker Device and Init Flags in Container Task Definitions Amazon ECS Allows Containers to Directly Access Environmental Metadata Announcing New AWS Deep Learning AMI for Amazon EC2 P3 Instances
 Amazon EC2 Systems Manager Parameter Store Adds Versioning Support Amazon EC2 Systems Manager Now Integrates With GitHub Application Load Balancers Now Support Multiple TLS Certificates With Smart Selection Using SNI Introducing Amazon EC2 P3 Instances Introducing Lifecycle Policies for Amazon EC2 Container Registry Application Load Balancers now support multiple SSL certificates EC2 Per second billing ECS Adds Support for Adding or Dropping Linux Capabilities to Containers Network Load Balancer now supports load balancing to IP addresses as targets Amazon EC2 Spot Can Now Stop and Start Your Spot Instances Amazon EC2 Systems Manager Run Command Adds Tag-Based Permissions and Multi- Tag Support Auto Scaling Lifecycle Hooks Enhancements A new addition to the Amazon EC2 memory-optimized X1 Instance family – x1e.32xlarge Amazon EC2 Container Service Now Integrated with Network Load Balancer Application Load Balancer Adds Support for New RequestCountPerTarget CloudWatch Metric EC2 Systems Manager Now Supports Linux Patching Sync Amazon EC2 Systems Manager Inventory Data to Amazon S3 Buckets ECS RunTask and StartTask APIs now support additional override parameters EC2 Systems Manager Adds Hierarchy, Tagging, and Notification Support for Parameter Store Announcing Network Load Balancer for Elastic Load Balancing Announcing improved networking performance for Amazon EC2 instances Application Load Balancer now supports load balancing to IP addresses as targets Amazon EC2 Systems Manager Adds Configuration Compliance Reporting and Auto- Remediation Amazon ECS is now HIPAA Eligible Amazon EC2 Systems Manager now HIPAA eligible Tag Your Spot Fleet EC2 Instances Introducing Amazon EC2 G3 Instances, the next-generation of GPU-powered instances for graphics-intensive applications Support for LCU metrics on Classic Load Balancer Amazon EC2 Systems Manager Adds Cross-Platform and Multi-Step Document Support Amazon EC2 Systems Manager Adds Raspbian OS and Raspberry Pi Support Introducing Target Tracking Scaling Policies for Auto Scaling

Slide 3

Slide 3 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. C5 Compute-optimized with 3.0GHz Intel Skylake P3 Next-gen GPU instances suitable for ML, HPC R4 Memory optimized (up to 488GiB) I3 High I/O with NVMe SSD 10k-300k IOPS X1e In-memory databases - up to 3.8TiB, 128vCPU D2 Up to 43TiB HDD EC2 Instance Types

Slide 4

Slide 4 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EBS Volume Types Type Description I/O Throughput Cost io1 Provisioned IOPS SSD Highest High Highest gp2 General purpose SSD High Low(ish) Medium st1 Throughput-optimized HDD Low Highest Low sc1 Non-optimized HDD Low Medium Lowest

Slide 5

Slide 5 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Miscellaneous changes, features ■ IPv6 ■ KVM hypervisor for new instance types ■ Elastic Network Adapter - 25 Gbps ■ Per second billing ■ Elastic GPUs ■ Target Tracking Scaling Policies for Auto Scaling ■ New regions 2016 - Ohio, Canada, London, Mumbia, Seoul Soon - China (Ningxia), Paris, Stockholm, Hong Kong, Bahrain
 #awswishlist - Kenya

Slide 6

Slide 6 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CloudWatch Integration Scheduled tasks Event handling Metrics Logs

Slide 7

Slide 7 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CloudWatch Events ■ Scheduled Events ■ Instance state changes ■ Systems manager ■ Maintenance window ■ ECS event stream ■ EBS snapshots, encryption/decryption ■ Cross-account event delivery

Slide 8

Slide 8 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CloudWatch Telemetry ■ High resolution (1s/3H) custom metrics ■ High resolution alarms (10s) ■ collectd CloudWatch plugin ■ Dashboards (GUI, CLI) ■ Percentile statistics (p50, p90, p99…) ■ Logs ■ Metric filters for log parsing and alarms

Slide 9

Slide 9 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EC2 Container Service ■ Elastic Container Registry ■ Docker Device and Init Flags in Container Task Definitions ■ CloudWatch metrics for CPU and memory utilization across the cluster ■ IAM roles for ECS tasks ■ github.com/blox/blox marching towards v1.0 ■ 3rd party tooling (Convox, Empire) ■ Integration with Application Load Balancer ■ Run tasks on a schedule ■ Execute tasks in response to CloudWatch events

Slide 10

Slide 10 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EC2 Systems Manager Superpowers for EC2 instances and on-premises systems. ■ Remote command execution with Run Commands ■ Controlled secrets and configuration data with the Parameter Store ■ Periodic tasks with the State Manager and Maintenance Windows ■ Stepwise Automation workflows for initializing nodes ■ Collect and query Inventory and Patch status

Slide 11

Slide 11 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Key Systems Manager Benefits ✓ All actions recorded in CloudTrail (e.g. immutable audit trail) ✓ Trigger SNS, Lambda from Systems Manager events ✓ Store command history and output to S3 ✓ Fine-grained access control to Run Commands ✓ Integration with Config to track changes over time

Slide 12

Slide 12 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The SSM Agent ■ Open source (Golang) executable for Linux and Windows ■ Available for cloud and on-premises systems ■ Assign IAM role with permissions to interface with SSM API ■ Install at boot or on existing systems ■ Polls for commands to execute

Slide 13

Slide 13 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. { "schemaVersion":"2.0", "description":"Run a script", "parameters":{ "commands":{ "type":"String", “description”:"Commands to run" } }, "mainSteps":[ { "action":"aws:runShellScript", "name":"runShellScript", "inputs":{ "runCommand":"{{ commands }}" } } ] } Systems Manager Documents

Slide 14

Slide 14 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. github.com/kountable/pssh A s h e l l f o r t h e E C 2 P a r a m e t e r S t o r e

Slide 15

Slide 15 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. λ pssh CloudWatch Event EC2 Parameter Store Game of Thrones Transfer of Power CloudWatch Metrics

Slide 16

Slide 16 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Scoring System -40 Illegitimate -30 Handicapped -20 Exile 0 Deceased 30 Distinction 50 Noble 50 Royalty 100 Magical

Slide 17

Slide 17 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Community Curated Pro Tips

Slide 18

Slide 18 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Automated infrastructure is table stakes.

Slide 19

Slide 19 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Ansible for provisioning and fleet control. Lambdas for glue. Packer for building images.

Slide 20

Slide 20 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Single node ASGs for self-healing and resilience.

Slide 21

Slide 21 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use Config Rules to enforce compliance with tagging schemes, EBS snapshots, security group rules, or other site preferences.

Slide 22

Slide 22 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Many EC2 IAM actions do not support resource-level permissions. Exercise caution. { "Statement": [ { "Action": [ "ec2:AllocateAddress", "ec2:AssociateAddress", "ec2:DescribeAddresses", "ec2:DisassociateAddress", "ec2:CreateKeyPair", "ec2:DeleteKeyPair" ], "Effect": "Allow", "Resource": [ "*" ] } ] }

Slide 23

Slide 23 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use the BurstBalance CloudWatch metric to monitor I/O credit balance for gp2, st1, sc1 EBS volumes.

Slide 24

Slide 24 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use the CPUCreditUsage and CPUCreditBalance metrics in CloudWatch to track CPU burst usage on T2 instance types.

Slide 25

Slide 25 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. github.com/awslabs/aws-shell T h e r e a l h e r o

Slide 26

Slide 26 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Network throughput increases substantially with instance type. (Don't forget to enable enhanced networking)

Slide 27

Slide 27 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. In CloudFormation, explicitly request SSD ephemeral disks or you may not get them.

Slide 28

Slide 28 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Require MFA for SSH access and enable Fail2ban to block IPs with failed login attempts.

Slide 29

Slide 29 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Improve your SSH experience with ControlPersist.

Slide 30

Slide 30 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use ssh -D and the SwitchyOmega Chrome extension for convenient access to services in a private network.

Slide 31

Slide 31 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Running multiple apps per instance? Use AssumeRole to assign granular permissions to each app.

Slide 32

Slide 32 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Protect the EC2 metadata and userdata.

Slide 33

Slide 33 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use the instance identity document to validate the authenticity of EC2 instances.

Slide 34

Slide 34 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use Linux >= 4.4 for best results.

Slide 35

Slide 35 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. checkip.amazonaws.com is HTTP only hence it cannot be trusted. Use icanhazip.com instead.

Slide 36

Slide 36 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. #!/bin/bash -x # Save userdata cmd output to a log exec > /var/log/userdata.log 2>&1 # Initialize instance …

Slide 37

Slide 37 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Pace yourself tonight. It’s going to be a busy week.

Slide 38

Slide 38 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Recently released!

Slide 39

Slide 39 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ` Ben Whaley D i r e c t o r , S e c u r i t y a n d O p e r a t i o n s K o u n t a b l e