Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AWS re:Cap Webinar

AWS re:Cap Webinar

Welcome to re:Invent Recap Edition 2018. This online talk is designed for executives, IT professionals, and anyone looking to leverage the AWS Cloud to build and transform their businesses.

Whether you are new to the cloud or an experienced user, you will learn something new at AWS Innovate. This free online conference is designed to inspire and educate you about AWS services and help you develop the skills to design, deploy, and operate infrastructure and applications.

Each November, AWS hosts re:Invent, our global cloud computing conference featuring Keynotes, Product and Service Announcements, a Partner Expo Hall, Breakout Sessions, Chalk Talks, Workshops, Builders Sessions, Training and Certification, and Hackathons that cover AWS core topics and embrace the emerging technologies we are developing.

This special Innovate edition brings you a recap of the event, breakout sessions, launches, AWS insights, live Q&A access to AWS experts, and more.

Julio Faerman

January 22, 2019
Tweet

More Decks by Julio Faerman

Other Decks in Technology

Transcript

  1. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. AWS re:Invent Recap Julio Faerman @faermanj
  2. 5 0 , 0 0 0 + attendees 2 ,

    1 0 0 + technical sessions 1 0 0 , 0 0 0 + live stream registrations 2018
  3. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Business Update 4 6 % Y/Y G r ow t h Q3 2018 vs Q3 2017 $2 7 B Revenue run rate ANNUALIZED FROM Q3 2018
  4. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. The broadest and deepest cloud platform for today’s builders
  5. Amazon Global Network • Redundant 100GbE network • Redundant private

    capacity between all Regions except China Over 150 Global CloudFront PoPs 89 Direct Connect Locations a e o q i h Paris Sweden AWS GovCloud East First 5 years: 4 regions 2016–2020: 13 regions Next 5 years: 7 regions A W S REGIONAL EXPANSION 1 9 R e g i o n s 5 7 A Z s d m c g b n s k v i i i i i i i i Milan i Cape Town
  6. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Computing - Instances
  7. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. EC2 Instance Types Burstable T 3 Big Data Optimized H 1 Memory Optimized R 5 High I/O I 3 Compute Intensive C 5 Graphics Intensive G 3 General Purpose GPU P 3 Memory Intensive X 1 X 1 e General Purpose M 5 V i r t u a l P r i v a t e S e r v e r s Bare Metal High I/O I 3 m Dense Storage D 2 F 1 FPGA A m a z o n L i g h t s a i l High-Memory Intensive Z 1 Powered by M 5 a R 5 a • Choose between processors on AWS general purpose and memory optimized instances • 10% lower prices on AMD-based instances • Most applications can run on AMD-based variants with little to no modification M 5 d R 5 d C 5 d Z 1 d • NVMe-based SSD block level instance storage physically connected to the host server • High-speed, low latency local block storage • EBS PIOPS to 1GB/s (64,000)
  8. Management Tools Introducing Predictive Scaling Ramp capacity before you need

    it with Predictive Scaling Time On-premise capacity provisioning Load/Capacity Time Capacity provisioning with target tracking Load/Capacity Time Capacity provisioning with predictive scaling and target tracking Load/Capacity © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  9. N E W ! Up to 45% lower cost for

    scale-out workloads
  10. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Other EC2 Updates • Hibernation now supported on EC2 instances • EBS PIOPS doubled • Heterogenous fleets and purchase options in Autoscaling groups
  11. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Computing - Containers
  12. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Micro Services to Service Mesh Service Mesh Plane (Routing, Monitoring, Security) Svc A Svc B Side Car Side Car
  13. N E W ! App Mesh with Amazon ECS and

    Amazon EKS to better run containerized microservices at scale Microservice A Microservice B Microservice C Microservice A Microservice B Microservice C Microservice C Microservice C Before After
  14. N E W ! Easily create and maintain custom maps

    of your applications Before After
  15. Introducing AWS Marketplace for Containers AWS Marketplace Software Container products

    available in AWSMarketplace Choose from more than 160 curated and trusted container products in AWS Marketplace and run them on AWS Container product images are verified and scanned Deploy container products on Amazon ECS, AWS Fargate, or EKS Products are available with free, bring your own license, and usage-based pricing models AWS Marketplace for Containers
  16. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Computing - Serverless
  17. N E W ! GE N E R AL L

    Y AVAI L AB L E A W S C l o u d 9 AWS Toolkit for PyCharm AWS Toolkit for IntelliJ AWS Toolkit for VS Code GENERALLY AVAILABLE IN DEVELOPER PREVIEW IN DEVELOPER PREVIEW Open source toolkits meeting you where and how you like to work AWS Toolkits for popular IDEs + IDEs
  18. N E W ! Languages Lambda support for Ruby +

    Bring any Linux compatible language runtime; Powered by new Runtime API - Codifies the runtime calling conventions and integration points Same technology powering Ruby support in AWS Lambda Bring any Linux compatible language runtime Custom Runtimes + AWS OPEN SOURCE o f f e r e d b y o f f e r e d b y o f f e r e d b y o f f e r e d b y PARTNER SUPPORTED
  19. N E W ! Extend the Lambda execution environment with

    any binaries, dependencies, or runtimes Lambda Layers BUSINESS LOGIC LIB A LIB B BUSINESS LOGIC LIB A LIB B BUSINESS LOGIC LIB A LIB B BUSINESS LOGIC LIB A LIB B Programming Model Before BUSINESS LOGIC BUSINESS LOGIC BUSINESS LOGIC BUSINESS LOGIC LIB A LIB B After
  20. N E W ! Programming Model Store, share, and deploy

    serverless applications Serverless Application Repository Compose application architectures from reusable building blocks Nested Applications using Serverless Application Repository Deploy new architectures as a set of serverless apps (nesting) Foster best organizational practices and reduce duplication of effort Share components, modules and full applications privately with teams or publicly with others to improve agility + + Integrate Lambda functions into existing web architectures ALB Support for Lambda AWS Lambda Applicatio n Load Balancer AWS Fargate Amazon EC2
  21. N E W ! MOBILE APPS CHAT DASHBOARDS IoT DEVICES

    Amazon API Gateway WebSockets API LAMBDA FUNCTIONS PUBLIC ENDPOINTS ON AMAZON EC2 AMAZON KINESIS ANY OTHER AWS SERVICE Stateful connection A L L P U B L I C L Y A C C E S S I B L E E N D P O I N T S Stateful connection Programming Model This new type of API will enable customers to build real-time two way communication applications backed by Lambda functions or other API Gateway integrations. Web Socket support for API Gateway
  22. N E W ! WorkFlow Step Functions Process photo Resize

    image Extract metadata Facial recognition Load in Database P A R A L L E L S T E P S Start E n d Glue AWS services together without writing code Step Functions API Connectors Amazon ECS AWS Fargate Amazon DynamoD B Amazon SNS AWS Batch Amazon SQS AWS Glue Amazon SageMak er +
  23. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. IDEs Languages AWS Cloud9 AWS Toolkit for PyCharm AWS Toolkit for IntelliJ AWS Toolkit for VS Code AWS Lambda layers AWS Nested apps Websocket support for API Gateway ALB support for Lambda Workflows Programming Model Step Functions API Connectors Amazon Managed Streaming for Kafka Build increasingly powerful applications, faster
  24. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Firecracker Architecture and Benefits • Firecracker microVMs have the same security as KVM VMs • Designed for low overhead, high density, and fast start times • Built-in fair sharing
  25. N E W ! Amazon EFS IA Amazon S3 Intelligent-Tiering

    AWS DataSync AWS Transfer for SFTP
  26. N E W ! Amazon Glacier Deep Archive No tape

    to manage $0.00099/GB/month Less than 1/4th the cost of Glacier Designed for 99.999999999 durability Recover data in hours vs. days/weeks L o w e s t c o s t s t o r a g e a v a i l a b l e i n t h e c l o u d … e v e n l o w e r t h a n o n - p r e m i s e s t a p e
  27. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Set up Storage 1 Move data 2 Cleanse and prep data 3 Configure and enforce security and compliance policies 4 Make data accessible for analytics 5 Steps for building and managing a data lake
  28. N E W ! Enforce security policies across multiple services

    Gain and manage new insights Move, store, catalog, and clean your data faster with machine learning A service that allows you to build a secure data lake in days Amazon Lake Formation
  29. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Worldwide Windows Public Cloud IaaS Instances by Cloud Provider
  30. N E W ! Amazon FSx for Windows File Server

    Windows native for fully compatible Windows File System experience No hardware or software to manage Secure and compliant including PCI-DSS, ISO, and HIPAA Up to 10s of GB/s throughput with sub- millisecond latencies (Compatibility with AD, Windows access control, and native Windows Explorer experience) Fully managed Windows file system built on native Windows file servers
  31. N E W ! High throughput, low latency – 100s

    of GBs/s and millions of IOPS Seamless integration with Amazon S3 Secure and compliant including PCI-DSS, ISO and HIPAA Fully managed file system for high compute intensive workloads Amazon FSx for Lustre
  32. N E W ! Fully managed, highly- available Kafka clusters

    Highly available with rolling upgrades Fully compatible – Run your Kafka applications with zero code changes Fully managed and highly available Apache Kafka service Amazon Managed Streaming for Kafka
  33. Management Tools Introducing CloudWatch Logs Insights Pay-as-you-go log analytics service

    for CloudWatch Purpose-built for log diving Resolve operational problems faster Connect the dots between logs and metrics Fully managed
  34. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Other Updates • QuickSight ML Insights • QuickSight Embeddable dashboards and graphs
  35. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. This is the moment for #databasefreedom
  36. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Aurora is the foundation for cloud native Databases Amazon Aurora NEW Global Database
  37. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Provisioning capacity for DynamoDB ( S o m e t i m e s i t ’ s h a r d t o k n o w w h a t ’ s b e s t ) HIGH-SCALE APPLICATIONS Estimating how much throughput capacity to provision can be guesswork Not enough experience with app can cause unexpected extreme app usage Spikey traffic can be costly to maintain availability and performance Auto-scaling can cause lag time apps can’t afford
  38. N E W ! Amazon DynamoDB Read/Write Capacity On Demand

    N o m o r e c a p a c i t y p l a n n i n g – p a y o n l y f o r w h a t y o u u s e No capacity planning No need to specify how much read/write throughput you expect to use Pay only for what you use Pay-per-request pricing Ideal for unpredictable workloads Ramp from zero to tens of thousands of requests per second on demand
  39. N E W ! Amazon DynamoDB Transactions N a t

    i v e , s e r v e r - s i d e s u p p o r t f o r t r a n s a c t i o n s
  40. Existing time-series databases Relational databases Difficult to scale Manual effort

    needed for enterprise-grade availability and reliability Limited data lifecycle management capabilities Unnatural for time-series data Rigid schema inflexible for fast-moving time- series data Building with time-series data is challenging Lack time-series analytic functions like smoothing, approximation, and interpolation 1 2 3 Clickstream data IoT Sensor Readings DevOps data
  41. N E W ! Timestream is 1,000X faster and 1/10th

    the cost of relational databases Trillions of daily events Serverless Time-series analytics (interpolation, smoothing, approximation) built in Multiple orders of magnitude improvement in query performance Fast, scalable, fully managed time series database Amazon Timestream
  42. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. How do we think about Blockchain?
  43. N E W ! Cryptographically Verifiable All changes are cryptographically

    chained and verifiable Transparent Full visibility into entire data lineage Immutable Append-only, immutable journal tracks history of all changes Highly scalable Automatically scale up or down Easy to use Query with familiar SQL operators Fast Execute 2-3X more transactions Fully managed ledger database that provides a transparent, immutable, cryptographically verifiable transaction log owned by a central trusted authority Amazon Quantum Ledger Database
  44. N E W ! Create and manage scalable blockchain networks

    Amazon Managed Blockchain Choose Hyperledger Fabric or Ethereum Create blockchain networks with a few clicks; manage them with simple API calls Scales to support thousands of applications running millions of transactions Easy to move data into QLDB for further analysis
  45. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. This is the moment for #databasefreedom
  46. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Machine Learning on AWS to All Developers
  47. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Fastest training times for TensorFlow 30m training time (Mountain View) ResNet50 only Specialized hardware only available in beta 14m training time (Seattle) ResNet50, convolutional neural networks (for images), recurrent neural networks (for language recommendations Optimized for P3 with global availability F A S T E S T T R A I N I N G T I M E Scaling TensorFlow 65% STOCK TENSORFLOW scaling efficiency with 256 GPUs 90% AWS-OPTIMIZED TENSORFLOW scaling efficiency with 256 GPUs NEW Thinking TensorFlow on Cloud
  48. N E W ! EC2 Instance EC2 Instance EC2 Instance

    GPU Add GPU acceleration to any Amazon EC2 instance for faster inference at much lower cost (up to 75% savings) Amazon Elastic Inference P 3 . 8 X L P 3 P 3 P 3 P 3 Amazon Elastic Inference 36 TOPS GPU M5.large Amazon Elastic Inference
  49. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Starting at 1 TFLOPS Any instance family Simple speech and language models Up to 32 TFLOPS Recommendation engines or fraud detection models Provision Elastic Inference capacity inside VPC 360,000 ResNet-50 Computer vision deep learning model images per hour, inference $0.22 per hour on medium EI accelerator 75% lower cost L O W E S T C O S T A V A I L A B L E Elastic Inference - Scale & Cost
  50. N E W ! High throughput Low latency Hundreds of

    TOPS Multiple data types Multiple ML Frameworks INT8, FP16, mixed precision TensorFlow, MXNet, PyTorch, Caffe2, ONNX EC2 instances Amazon SageMaker Amazon Elastic Inference High performance machine learning inference chip, custom designed by AWS AWS Inferentia
  51. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon SageMaker Bringing Machine Learning to all developers One-click training Set up and manage environments for training Pre-built notebooks for common problems Collect and prepare training data Built-in, high performance algorithms Choose and optimize your ML algorithm Optimization Train and tune model(trial and error) One-click deployment Deploy model in production Fully managed with auto- scaling, health checks, automatic handling of node failures, and security checks Scale and manage the production environment
  52. N E W ! Human annotations Data in S3 Automatic

    annotations Training data Simple, pre-built workflows Mechanical Turk Your own employees Third party vendors Active Learning model >80% confidence <80% confidence Build highly accurate training datasets and reduce data labeling costs by up to 70% using Machine Learning Amazon SageMaker - Ground Truth
  53. N E W ! Natural Language Processing Computer Vision Speech

    Recognition Text Clustering Text Generation Text Classification Grammar and Parsing Named Entity Recognition Text to Speech Handwriting Recognition Object Detection in Images 3D Images Text OCR Video Classification Speaker Identification Ranking Regression Anomaly Detection Register with AWS Marketplace Automatic validation on SageMaker Package algorithm, models and configuration Self-service listing on AWS Marketplace Browse or search AWS Marketplace Subscribe in a single click Available through Amazon SageMaker Over a hundred algorithms and models that can be deployed directly to Amazon SageMaker AWS Marketplace for Machine Learning S e l l i n g a l g o r i t h m s & m o d e l s o n A W S M a r k e t p l a c e
  54. N E W ! Increase or decrease the number of

    nodes available during training Dynamic Training with MXNet
  55. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. SOPHISTICATION OF ML MODELS A M O U N T O F T R A I N I N G D A T A R E Q U I R E D Reinforcement Learning (RL)
  56. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. How does Reinforcement Learning work? Algorithm controls Pac-Man Learn to play to get the highest score possible Maximize rewards and minimize penalties Learn advanced strategies
  57. N E W ! Fully managed reinforcement learning algorithms TensorFlow,

    MXNet, Intel Coach, and Ray RL 2D and 3D simulation environments via OpenGym Simulate environments with Amazon Sumerian and AWS RoboMaker Example notebooks and tutorials New machine learning capabilities in Amazon SageMaker to build, train, and deploy with Reinforcement Learning Amazon SageMaker RL
  58. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. AWS DeepRacer: How does it work? 3D simulator with virtual car and track Rewards RL algorithm
  59. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Build reinforcement learning model DeepRacer League Races at AWS Summits Winners of each DRL Race and top points getters compete in Championship Cup at re:Invent 2019 Virtual tournaments through the year AWS DeepRacer League World’s first global autonomous racing league, open to anyone
  60. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. No master algorithm for personalization and recommendation Music recommendation Tracks, artists, albums Film recommendation Actors, directors, genres Product recommendation Pricing, category, offers Article recommendation Themes, geography, breaking news
  61. N E W ! Activity stream from app Views, signups,

    conversion, etc. Inventory Articles, products, videos, etc. Demographics (optional) LOAD DATA (EMR Cluster) INSPECT DATA IDENTIFY FEATURES SELECT ALGORITHMS SELECT HYPERPARAMETERS TRAIN MODELS OPTIMIZE MODELS HOST MODELS BUILD FEATURE STORE CREATE REAL-TIME CACHES Customized personalization & recommendation API F u l l y m a n a g e d b y A m a z o n P e r s o n a l i z e Amazon Personalize Age, location, etc. Real-time personalization and recommendation service, based on the same technology used at Amazon.com Amazon Personalize
  62. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. The perils of poor predictions in forecasting TIME SALES FORECAST SALES
  63. N E W ! Amazon Forecast Any historical time-series Integrates

    with SAP and Oracle Supply Chain Custom forecasts with 3 clicks 50% more accurate 1/10th the cost Integrates with Amazon Timestream Retail demand Travel demand AWS usage Revenue forecasts Web traffic Advertising demand Generate forecasts for: Accurate time-series forecasting service, based on the same technology used at Amazon.com
  64. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Historical data Supply chain, inventory, etc. Customized forecasting API Inspect data Identify features Select from 8 algorithms Select Hyperparameters Host models Load data Train models Optimize models Related “causal” data Weather, special offers, product details F u l l y m a n a g e d b y A m a z o n F o r e c a s t Amazon Forecast Amazon Forecast: Machine Learning time-series forecasting
  65. N E W ! OCR++ service to easily extract text

    and data from virtually any document Amazon Textract
  66. N E W ! Augment machine translations with custom terms

    or branding Amazon Translate Custom Terminology "Have you ever shopped with Amazon Family?" Avez-vous déjà fait des achats avec Amazon Family? Avez-vous déjà fait des achats avec Amazon Famille?
  67. N E W ! Extract information from unstructured medical text

    accurately and quickly No machine learning experience required Amazon Comprehend Medical
  68. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Integrate ML to AWS Services
  69. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. #1 Security Priority on AWS Cloud
  70. Introducing AWS SecurityHub Security Public Beta Centrally view and manage

    security alerts & automate compliance checks • Enabled in minutes to aggregate security findings from AWS and Partner services across your accounts • Quickly assess security and compliance in one location and take action on findings • Built-in and customizable insights help you track security issues that are unique to your environment • Improve compliance with automated, continuous account-level configuration and compliance checks
  71. Introducing AWS Resource AccessManager Security Simple, secure service to share

    AWSresources eliminating the need to provision duplicate resources in every account AWS IAM policies to govern the consumption of shared resources, and AWS CloudWatch and AWS CloudTrail to provide visibility efficiently using your resources across different departments Share Route 53 Resolver Rules, License Manager Config, Transit Gateways and Subnets
  72. Introducing AWS Control Tower Management Tools Limited Preview Set-up a

    multi-account environment in a single location to govern AWS workloads Automate the creation of a landing zone with best practice blueprints AWS Control Tower automates the set-up of a well-architected multi-account environment with best practices, and guides you through a step-by-step process to customize it to your organization Guardrails for policy enforcement Control Tower offers curated guardrails. Guardrails are high-level rules that provide on-going governance for your overall AWS environment Dashboard for continuous visibility The Control Tower dashboard gives you continuous visibility into your AWS environment. You can view the number of organizational units and accounts provisioned, guardrails enabled, and the compliance status of your enabled guardrails
  73. Introducing PrivateMarketplace AWS Marketplace Software Build a private marketplace that

    includes approved vendors and software from AWSMarketplace Ensure your AWS users are purchasing and launching softwarethat meets the company’s procurement, legal, and security controls Customize your private marketplace with company branding, such as logo, color, and messaging Always know when products are added or removed through notifications fromAWS Define and control permissions for your users in AWSPrivate Marketplace
  74. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Manage with your existing VMware tools Seamlessly migrate workloads Run the same VMware software on AWS that you run in your data centers VMware Cloud on AWS
  75. N E W ! AWS coming to a data center

    near you in two ways AWS Outposts VMware Cloud on AWS AWS Native AWS DESIGNED HARDWARE The same that we run in our own data centers OPTION 1 OPTION 2
  76. N E W ! AWS Well- Architected Tool Measure and

    improve your architecture using AWS Well-Architected best practices Implement workplans to improve your architecture Stay up to date as your architecture evolves Review workloads against best practices
  77. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. AWS IoT – Broad and Deep Integrations with Cloud End-point Gateway Edge Network AWS Cloud Secure Device Connection & Messaging Fleet on-boarding, Softeare Update Fleet Audit& Security Data Analytics & Intelligence Easily collect and organize data from industrial equipment NEW Easily detect and respond to events from IoT sensors and applications NEW Visually develop IoT applications NEW
  78. N E W ! Easily develop, test, and deploy intelligent

    robotics applications AWS RoboMaker Development Environment Simulation Cloud Extensions for ROS Fleet Management
  79. N E W ! Easily control satellites and ingest data

    with fully managed Ground Station as a Service AWS Ground Station Low Earth Orbit (LEO) Medium Earth Orbit (MEO) Simultaneous narrowband S-band, X-band and UHF downlink Receive satellite data into Amazon VPC and Process data in AWS Cloud
  80. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Here are all of the new launches! https://aws.amazon.com/new/reinvent
  81. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Follow it all on twitch.tv/aws !
  82. Thank you! © 2018, Amazon Web Services, Inc. or its

    affiliates. All rights reserved. Julio Faerman @faermanj