Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AWS re:Invent 2017 Recap for London Startups

AWS re:Invent 2017 Recap for London Startups

London, December 13th, 2017

Danilo Poccia

December 13, 2017
Tweet

More Decks by Danilo Poccia

Other Decks in Programming

Transcript

  1. © 2017, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. re:Cap Danilo Poccia Technical Evangelist [email protected] @danilop danilop
  2. 4 3 , 0 0 0 + attendees 1 ,

    3 0 0 + technical sessions 6 0 , 0 0 0 + live stream registrations
  3. 516 24 48 61 82 159 280 722 1,017 L

    A U N C H E S 2008 2009 2010 2011 2012 2013 2014 2015 2016 1,300+ 2017 P A C E O F I N N O V A T I O N New capabilities daily
  4. Most robust, fully featured technology infrastructure platform CORE SERVICES Integrated

    Networking Rules Engine Device Shadows Device SDKs Device Gateway Registry Local Compute Custom Model Training & Hosting Conversational Chatbots Virtual Desktops App Streaming Schema Conversion Image & Scene Recognition Sharing & Collaboration Exabyte-Scale Data Migration Text to Speech Corporate Email Application Migration Database Migration Regions Availability Zones Points of Presence Data Warehousing Business Intelligence Elasticsearch Hadoop/Spark Data Pipelines Streaming Data Collection ETL Streaming Data Analysis Interactive SQL Queries Queuing & Notifications Workflow Email Transcoding Deep Learning (Apache MXNet, TensorFlow, & others) Server Migration Communications MARKETPLACE Business Apps Business Intelligence DevOps Tools Security Networking Storage Databases API Gateway Single Integrated Console Identity Sync Mobile Analytics Mobile App Testing Targeted Push Notifications One-click App Deployment DevOps Resource Management Application Lifecycle Management Containers Triggers Resource Templates Build & Test Analyze & Debug Identity Management Key Management & Storage Monitoring & Logs Configuration Compliance Web Application Firewall Assessment & Reporting Resource & Usage Auditing Access Control Account Grouping DDOS Protection TECHNICAL & BUSINESS SUPPORT Support Professional Services Optimization Guidance Partner Ecosystem Training & Certification Solutions Management Account Management Security & Billing Reports Personalized Dashboard Monitoring Manage Resources Data Integration Integrated Identity & Access Integrated Resource & Deployment Management Integrated Devices & Edge Systems Resource Templates Configuration Tracking Server Management Service Catalogue Search MIGRATION HYBRID ARCHITECTURE ENTERPRISE APPS MACHINE LEARNING IoT MOBILE SERVICES DEV OPS ANALYTICS APP SERVICES INFRASTRUCTURE SECURITY & COMPLIANCE MANAGEMENT TOOLS Compute VMs, Auto-scaling, Load Balancing, Containers, Virtual Private Servers, Batch Computing, Cloud Functions, Elastic GPUs, Edge Computing Storage Object, Blocks, File, Archivals, Import/Export, Exabyte-scale data transfer CDN Databases Relational, NoSQL, Caching, Migration, PostgreSQL compatible Networking VPC, DX, DNS Facial Recognition & Analysis Facial Search Patching Contact Center
  5. A W S V E N D O R 2

    V E N D O R 4 V E N D O R 3 6 RELATIONAL DATABASE ENGINES GENERALLY AVAILABLE 41 OBJECT AND BLOCK KEY STORAGE CAPABILITIES 71 INTEGRATIONS WITH PARTNER STORAGE SOLUTIONS 147 KEY SECURITY, GOVERNANCE AND COMPLIANCE CAPABILITIES 18 SERVERLESS EVENT SOURCES 66 KEY MANAGEMENT TOOL CAPABILITIES
  6. First 5 years: 4 regions 2016–2018: 11 regions Next 5

    years: 7 regions A W S REGIONAL EXPANSION
  7. Amazon EC2 M5 Instances (GA) Next generation of EC2 general

    purpose instances • Powered by 2.5 GHz Intel Xeon Platinum 8000- series ”Skylake” Processor • New larger instance size – m5.24xlarge with 96 vCPUs and 384 GiB of memory • Improved network and EBS performance on smaller instance sizes • Support for Intel AVX-512 • Powered by new lightweight Nitro Hypervisor 14% Price / Performance Improvement With M5 M4 M5
  8. Amazon EC2 H1 Instances (GA) New dense storage instance family

    for big data workloads New Storage-optimized instance Up to 16TB of locally attached HDD storage Up to 25 Gbps network bandwidth with ENA Big Data Clusters Kafka Streaming MapReduce
  9. I3.metal (Bare Metal) • Two Intel Xeon E5-2686v4 2.3GHz (72

    logical processors) • 512GiB Memory • 15.2 TB local SSD NVMe storage • 25 Gbps ENA-based enhanced networking
  10. More Computing • T2 Unlimited • Spread Placement • Launch

    Templates • Spot Enhancements • AWS PrivateLink • Inter-Region VPC Peering • Amazon Time Sync Service • AWS Batch – Array Jobs • Migrate Hyper-V VMs to AWS
  11. BROADEST SPECTRUM OF COMPUTE INSTANCES Burstable T 2 Big Data

    Optimized H 1 Memory Optimized R 4 In-memory X 1 High I/O I 3 Compute Intensive C 5 Graphics Intensive G 3 General Purpose GPU P 3 Memory Intensive X 1 e General Purpose M 5 Virtual Private Servers Bare Metal High I/O I 3 m Dense Storage D 2 F 1 FPGA A m a z o n L i g h t s a i l EC2 Elastic GPUs Graphics acceleration for EC2 instances EC2 Spot Instances • Hibernation • No Bid Pricing N E W ! NEW! NEW! NEW! NEW! NEW!
  12. Service integrations are at the container level Scales to support

    clusters and applications of any size Integration with entire AWS platform 3 ALB, Auto Scaling, Batch, Elastic Beanstalk, CloudFormation, CloudTrail, CloudWatch Events, CloudWatch Logs, CloudWatch Metrics, ECR, EC2 Spot, IAM, NLB, Parameter Store, and VPC The easiest way to deploy and manage containers Amazon Elastic Container Service (ECS) 2 1
  13. WHAT ABOUT KUBENETES? “According to a recent survey, 63% of

    users were already hosting Kubernetes on Amazon EC2. This was up from 44 percent a year ago.” https://www.cncf.io/blog/2017/06/28/survey-shows-kubernetes-leading-orchestration-platform/
  14. Manag e d K ub e r ne te s

    o n A W S Available in preview today Amazon Elastic Container Service for Kubenetes (EKS) Hybrid cloud compatible Highly available Automated upgrades and patches Integrated with AWS Services CloudTrail, CloudWatch, ELB, IAM, VPC, PrivateLink NEW!
  15. … B U T W H A T E L

    S E ? M A N A G E D C L U S T E R S A R E G R E A T …
  16. R u n c o n t a i n

    e r s w i t h o u t m a n a g i n g s e r v e r s o r c l u s t e r s Available for ECS today Available for EKS in 2018 AWS Fargate No clusters to manage Manages underlying infrastructure Easy to run, easy to scale NEW!
  17. No server is easier to manage than no server! (2015)

    ALL THE CODE YOU EVER WRITE IS BUSINESS LOGIC No server management Flexible scaling High availability No idle capacity
  18. AWS LAMBDA IS EVERYWHERE AWS Lambda Amazon S3 Amazon CloudFormation

    AWS IoT Amazon API Gateway Amazon DynamoDB Amazon CloudWatch Logs AWS IoT Button AWS Step Functions Amazon Kinesis Streams Amazon CloudWatch Events AWS Greengrass AWS X-Ray Amazon Kinesis Firehose AWS CodeCommit AWS Snowball Edge Amazon SNS AWS Config AWS Lambda@Edge Amazon SES Amazon Lex Amazon Cognito Amazon CloudFront AWS IoT AWS Lambda Event-driven services Event sources Lambda inside
  19. AWS Serverless Application Repository 4 new powerful features for AWS

    Lambda 3GB memory support Concurrency controls API Gateway VPC integration .NET Core 2.0 language support (pre-announce) …and Go Discover a collection of serverless applications Easily deploy to your AWS account in a few clicks Publish your own applications to share with the community D i s c o v e r , d e p l o y , a n d p u b l i s h s e r v e r l e s s a p p s Available in preview today NEW!
  20. HOW HAS DEVELOPMENT CHANGED? More mobile More collaborative More services

    More security aware QA & Ops are integrated More languages
  21. AWS Cloud9 A c l o u d I D

    E f o r w r i t i n g , r u n n i n g , a n d d e b u g g i n g c o d e Generally available today Zero to deploy FAST Deep integration Real-time pair programming Broad debugging support AWS CodePipeline AWS CodeDeploy AWS CodeCommit AWS CodeBuild PUBLISH DIRECTLY INTO CODESTAR TOOLS Integration with Serverless App NEW!
  22. RELATIONAL DATABASE Old Guards expensive, proprietary AMAZON AURORA MySQL and

    PostgreSQL compatible Several times faster than EC2/RDS Highly available and durable 1/10th the cost of commercial grade databases re:Invent 2015: Thousands of customers re:Invent 2016: 3.5X more customers Today: Tens of thousands of customers
  23. A U R O R A I S T H

    E F A S T E S T G R O W I N G S E R V I C E I N T H E H I S T O R Y O F A W S
  24. Aurora Multi-Masters Seamless recovery from read replica failures Auto-scale new

    read replicas Up to 15 read replicas across 3 availability zones Application Read Replica 1 Master Node Read Replica 2 Shared Distributed Storage Volume Availability Zone 1 Availability Zone 2 Availability Zone 3 Application Read/Write Master 2 Read/Write Master 1 Shared Distributed Storage Volume Availability Zone 1 Availability Zone 2 Availability Zone 3 Read/Write Master 3 Zero application downtime from ANY node failure Zero application downtime from ANY AZ failure Multi-region coming in 2018 Faster write performance NEW! First relational database service with scale-out both read and write across multiple datacenters
  25. Sign up for the preview today Aurora Severless On-demand, auto-scaling

    database for applications with unpredictable or cyclical workloads Automatically scales capacity up and down Pay per second and only for the database capacity you use Starts up on demand and shuts down when not in use No need to provision instances NEW!
  26. EVOLUTION OF DATABASES Amazon DynamoDB Amazon ElastiCache KEY VALUE DOCUMENT

    IN-MEMORY STORE AURORA Amazon RDS COMMERCIAL COMMUNITY N o n - r e l a t i o n a l d a t a b a s e s R e l a t i o n a l d a t a b a s e s
  27. Build high performance, globally distributed applications Low latency reads and

    writes to locally available tables Disaster proof with multi-region redundancy Easy to setup and no application re-writes required DYNAMODB GLOBAL TABLES NEW! First fully managed, multi-master, multi-region database Generally available today DYNAMODB BACKUP & RESTORE Only cloud database to provide on-demand and continuous backups On-Demand Backups for long term data archival and compliance Point In Time Restore for short term retention and protection against application errors Backup hundreds of TB instantaneously with NO performance impact On-Demand Backup generally available today Point In Time Restore coming early 2018
  28. EVOLUTION OF DATABASES Amazon DynamoDB Amazon ElastiCache KEY VALUE DOCUMENT

    IN-MEMORY STORE AURORA Amazon RDS COMMERCIAL COMMUNITY HIGHLY CONNECTED DATA N o n - r e l a t i o n a l d a t a b a s e s R e l a t i o n a l d a t a b a s e s Social news feed Restaurant recommendations Retail fraud detection
  29. CHALLENGES BUILDING APPS WITH HIGHLY CONNECTED DATA Difficult to maintain

    high availability Difficult to scale Relational databases Existing graph databases Limited support for open standards Too expensive Unnatural for querying graph Inefficient graph processing Rigid schema inflexible for changing graphs
  30. Available in preview today F A S T A N

    D S C A L A B L E E A S Y Build powerful queries easily with Gremlin and SPARQL 6 replicas of your data across 3 AZs with full backup and restore R E L I A B L E Supports Apache TinkerPopTM and W3C RDF graph models OPEN F u l l y m a n a g e d g r a p h d a t a b a s e Store billions of relationships and query with milliseconds latency Amazon Neptune NEW!
  31. EVOLUTION OF DATABASES Amazon DynamoDB Amazon ElastiCache KEY VALUE DOCUMENT

    IN-MEMORY STORE AURORA Amazon RDS COMMERCIAL COMMUNITY N o n - r e l a t i o n a l d a t a b a s e s R e l a t i o n a l d a t a b a s e s Amazon Neptune GRAPH MULTI- MASTERS SERVERLESS GLOBAL TABLES BACLUP& RESTORE
  32. Data Lake on AWS Most ways to bring data in

    Best security, compliance, and audit capabilities Object-level controls Unmatched durability, availability, and scalability Twice as many partner integrations Business insights into your data AMAZON S3 IS THE MOST POPULAR CHOICE FOR DATA LAKES
  33. Data Lake on AWS Amazon Redshift + R e d

    s h i f t S p e c t r u m Amazon QuickSight Amazon EMR H a d o o p , S p a r k , P r e s t o , P i g , H i v e … 1 9 t o t a l Amazon Athena Amazon Kinesis Amazon Elasticsearch Service AWS Glue S3 DATA LAKE Objects in your S3 data lake v v v v v v v v v v v v v v v v BUT, MOST ANALYTICS JOBS INVOLVE PROCESSING ONLY A SUBSET OF OBJECT DATA MAKING PETABYTE-SCALE ANALYTICS ACCESSIBLE TO COMPANIES OF ALL SIZES
  34. New API to select and retrieve data within objects Accelerate

    any application that processes a subset of object data in S3 Improve data access performance by up to 400% NEW! v Available in preview today Powerful new S3 capability to pull out only the object data you need using standard SQL expressions S3 SELECT 8 seconds Without S3 Select 1.8 seconds With S3 Select 4.5x faster 3 aggregations 1 table 4 filters COMPLEX PRESTO QUERY Against a standard TPC-DS dataset 6 sub-queries with each containing: Glacier SELECT Generally available today Run queries directly on data stored in Glacier Run queries on data stored at rest in Amazon Glacier Any application can query Glacier data Retrieve only what you need Makes Glacier part of your data lake
  35. A LONG HERITAGE OF MACHINE LEARNING AT AMAZON Personalized recommendations

    Inventing entirely new customer experiences Fulfillment automation and inventory management Drones Voice driven interactions
  36. F R A M E W O R K S

    A N D I N T E R F A C E S A W S D E E P L E A R N I N G A P I Apache MXNet TensorFlow Caffe2 Torch Keras CNTK PyTorch Theano Gluon Machine Learning on AWS NVIDIA Tesla V100 GPUs P3 1 Petaflop of compute NVLink 2.0 5,120 Tensor cores 128GB of memory ~14X faster than P2 P3 Instance Deep Learning AMI Frameworks
  37. Collect and prepare training data Choose and optimize your ML

    algorithm Set up and manage environments for training Train and tune model (trial and error) Deploy model in production Scale and manage the production environment ML IS STILL TOO COMPLICATED FOR EVERYDAY DEVELOPERS
  38. Easily build, train, and deploy machine learning models Generally available

    today Amazon SageMaker Fully managed hosting with auto-scaling DEPLOY One-click deployment BUILD Pre-built notebooks for common problems Built-in, high performance algorithms One-click training TRAIN Hyperparameter optimization NEW!
  39. T he wo r l d’ s f i r

    st w i r e l e ss, de e p l e ar ni ng e nab l e d v i de o c am e r a f o r de v e l o p e r s AWS DeepLens HD video camera Custom-designed deep learning inference engine Micro-SD Mini-HDMI USB USB Reset Audio out Power HD video camera with on- board compute optimized for deep learning Tutorials, examples, demos, and pre-built models From unboxing to first inference in <10 minutes Integrates with Amazon SageMaker and AWS Lambda 10 MIN NEW!
  40. V I S I O N L A N G

    U A G E A P P L I C A T I O N S E R V I C E S Amazon Rekognition Amazon Polly Amazon Lex Machine Learning on AWS
  41. Search, analyze, and organize millions of images A M A

    Z O N R E K O G N I T I O N Objects and scenes Facial analysis and recognition Inappropriate content detection Celebrity recognition Image in text recognition Amazon Rekognition
  42. R e a l - t i m e a

    n d b a t c h v i d e o a n a l y t i c s Generally available today NEW! Amazon Rekognition Video Video in. People, activities, and details out. Objects, scenes, and activities Person detection and recognition Person tracking Celebrity recognition Inappropriate content detection Amazon Kinesis Video Streams Generally available today S e c u r e l y i n g e s t a n d s t o r e v i d e o , a u d i o , a n d o t h e r t i m e - e n c o d e d d a t a
  43. A u t o m a t i c s

    p e e c h r e c o g n i t i o n Available in preview today NEW! Amazon Transcribe Support for telephony audio Timestamp generation Intelligent punctuation and formatting Recognize multiple speakers Custom vocabulary Multiple languages Amazon Translate A u t o m a t i c a l l y t r a n s l a t e s t e x t b e t w e e n l a n g u a g e s Available in preview today Real-time translation Batch analysis Automatic language recognition Low cost
  44. F u l l y m a n a g

    e d n a t u r a l l a n g u a g e p r o c e s s i n g Generally available today NEW! Amazon Comprehend Discover valuable insights from text Entities Key Phrases Language Sentiment Amazon Comprehend STORM WORLD SERIES STOCK MARKET WASHINGTON LIBRARY OF NEWS ARTICLES Amazon Comprehend
  45. Alexa Ecosystem Create great content: ASK is how you connect

    to your consumer Alexa Voice Service Unparalleled distribution: AVS allows your content to be everywhere Lives in the cloud Automated Speech Recognition (ASR) Natural Language Understanding (NLU) Always learning Alexa Skills Kit
  46. Alexa For Business NEW! “Alexa, start the meeting.” “Alexa, dial

    555-8000.” “Alexa, lower the blinds.” “Alexa, ask Salesforce which big deals closed today.” Your intelligent assistant for work Provision and manage shared devices Configure conference rooms Enroll users and assign skills Build custom skills
  47. G R O W I N G C O M

    M U N I T Y O F P A R T N E R S Voice-enabled office Quick access to business metrics Conference automation with Amazon Chime Inventory status check on ingredients IT procurement status update Voice-enabled Polycom conference rooms Hands-free conference room experience Conference automation with Amazon Chime A L E X A F O R B U S I N E S S C U S T O M E R S
  48. Amazon Sumerian NEW! The fastest and easiest way to create

    VR, AR, and 3D experiences Available in preview today
  49. V I S I O N L A N G

    U A G E A P P L I C A T I O N S E R V I C E S Amazon Rekognition Amazon Polly Amazon Lex Machine Learning on AWS Amazon Rekognition Video Amazon Transcribe Amazon Translate Amazon Comprehend Alexa for Business V R / I R Amazon Sumerian Amazon Kinesis Video Streams
  50. AWS ML Customers APPLICATION SERVICES Amazon Lex Amazon Polly Amazon

    Comprehend Amazon Translate Amazon Transcribe Amazon Rekognition Image Amazon Rekognition Video PLATFORM SERVICES Amazon SageMaker AWS DeepLens FRAMEWORKS AND INTERFACES AWS Deep Learning AMI Apache MXNet Caffe2 CNTK PyTorch TensorFlow Theano Torch Gluon Keras AWS ML Platform DATA LAKE STORAGE Amazon S3 SECURITY Access Control Encryption COMPUTE Powerful GPU and CPU Instances ANALYTICS Amazon Athena Amazon Redshift and Redshift Spectrum Amazon EMR (Spark, Hive, Presto, Pig) AWS Glue Amazon Kinesis Amazon QuickSight Amazon Macie AWS Organizations Complementary Services BROADEST ML PLATFORM THAT’S THE EASIEST TO USE WITH THE MOST CUSTOMERS
  51. GETTING STARTED Available in preview today N E W !

    AWS IoT 1-Click One-click creation of an AWS Lambda trigger for any device
  52. DEVICE MANAGEMENT N E W ! Generally available today Provision

    entire fleet with one click Organize device inventory Monitor and query fleet of devices Remotely manage devices AWS IoT Device Management Securely onboard, organize, monitor and remotely manage IoT devices at scale
  53. SECURITY N E W ! AWS IoT Device Defender Define

    and enforce security policies for fleets of devices Coming in 2018 Audit device policies Monitor device behavior Identify anomalies and out of compliance behaviors Generate alerts
  54. ANALYTICS N E W ! AWS IoT Analytics Fully managed

    analytics service that cleans, processes, stores, and analyzes IoT device data Integrated with AWS IoT Core Performs ad hoc queries, sophisticated analysis Stores device fleet data in time series data store Cleans and processes data from device fleet Prepares data for Machine Learning Available in preview today
  55. SMALLER DEVICES N E W ! Amazon FreeRTOS IoT connected

    operating system for microcontroller-based edge devices Generally available today Send data to AWS for analysis and action Based on FreeRTOS Easily connect to nearby AWS Greengrass devices Libraries to connect, update, and secure IoT devices Security credentials, key management
  56. Use AWS Greengrass console to transfer models to your devices

    Inference on the device Devices take action quickly – even when disconnected AWS Greengrass ML Inference Available in preview today Build and train models in the cloud N E W ! Run Machine Learning at the edge MACHINE LEARNING
  57. Build video- on-demand workflows in minutes not months Deliver broadcast-

    grade quality video Integrate with other AWS services AWS Elemental MediaConvert Compress video files for delivery to televisions or connected devices Generally available today NEW!
  58. Broadcast live with the push of a button Scale to

    deliver any number of live channels Encode broadcast- quality live video AWS Elemental MediaLive Encode live video for delivery to televisions or connected devices Generally available today NEW!
  59. Deliver video from one source to many devices Protect valuable

    content with advanced DRM capabilities Enable a DVR experience AWS Elemental MediaPackage Customize and protect content requested by devices using just-in-time format conversion Generally available today NEW!
  60. Dependable storage for video workloads Low, predictable latencies Consistent URL

    for media assets AWS Elemental MediaStore High-performance storage for live and on-demand video Generally available today NEW!
  61. Deliver unique ads to viewers Provide broadcast- grade ad experience

    Improve tracking of ad views AWS Elemental MediaTailor Personalize and monetize internet video content Generally available today NEW!
  62. © 2017, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Offload broker set up and administration High availability and durability Easily migrate from standards-based brokers Launch a message broker in minutes Pay as you go – get started for free Amazon MQ (available in GA) M a n a g e d m e s s a g e b r o k e r s e r v i c e f o r A p a c h e A c t i v e M Q N E W !
  63. • Gain Operational Insights and Take Action on AWS Resources

    • New Resource Groups API for grouping tagged resources programmatically • Operate on groups of resources intelligently • All of the EC2 Systems Manager features AWS Systems Manager N E W !
  64. AWS AppSync (preview) B u i l d d a

    t a d r i v e n m o b i l e a n d w e b a p p s w i t h r e a l - t i m e u p d a t e s a n d a n o f f l i n e p r o g r a m m i n g m o d e l N E W !
  65. Generally available today N E W ! Amazon GuardDuty I

    n t e l l i g e n t p r o t e c t i o n o f y o u r A W S a c c o u n t s a n d w o r k l o a d s
  66. © 2017, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. re:Cap Danilo Poccia Technical Evangelist [email protected] @danilop danilop