re:Invent 2021 re:Cap – Part II

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA Data Exchange for APIs RESTful or GraphQL API calls directly to AWS Data Exchange Use the AWS SDK in the programming language of your choice. Manage API access centrally with AWS Identity and Access Management (IAM)

Slide 5

Slide 5 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA Granular access control Define row- and cell-level access policies on both governed and traditional Amazon S3 tables. Manage and consistently enforce policies in a single place. Governed tables A new type of Amazon S3 table that allows you to ingest and manage data at scale, easily and reliably. Storage optimization Automatically optimize how data is stored to improve query performance as data volume increases. A new storage API to simplify how data is ingested, stored, and managed, together with row-level security to protect your data AWS Lake Formation Governed Tables

Slide 6

Slide 6 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA New storage API that solves for those key challenges 7 Fine-grained access controls Simple to ingest and manage data Integration APIs Define column-, row-, and cell- level permissions in a single place and enforce them consistently across AWS native and third-party tools Ingest streaming and batch data with ACID transactions Automatic small file compaction and data management Unified data access APIs make it easy to securely integrate your applications and tools with data in the data lake

Slide 7

Slide 7 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview Amazon Athena ACID transactions Write, delete, update, and time travel operations Enables concurrent users to make row-level modifications to Amazon S3 data Use Athena's console, API, and ODBC and JDBC drivers Use ACID transactions either on Lake Formation Governed tables, or using the Apache Iceberg table format

Slide 8

Slide 8 text

Slide 9

Slide 9 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview Faster time to market Don’t worry about data warehouse management Pay for what you use Compute provisioning Automated patching Routine maintenance Backup and recovery Automatic scaling Automatic failover Security and industry compliance Advanced monitoring YOU focus on insights takes care of the rest Automatically provisions and scales data warehouse capacity

Slide 10

Slide 10 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview Amazon Redshift Serverless – Easy Start Experience Activate Amazon Redshift Serverless for your AWS account Connect from your favorite BI tool or Amazon Redshift Query Editor BI tool Developer and business analyst Load data seamlessly and Amazon Redshift executes queries by automatically provisioning capacity Amazon Redshift Pay for compute and storage used during analysis Pay Easy to get started SQL features Performance at scale Security Scalability Analyze all your data No compromises

Slide 11

Slide 11 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview Redshift Materialized views auto refresh & query rewrite S P E E D U P Q U E R Y P E R F O R M A N C E B Y O R D E R S O F M A G N I T U D E W I T H P R E C O M P U T E D R E S U L T S item store cust price i1 s1 c1 12.00 i2 s2 c1 3.00 i3 s2 c2 7.00 store owner loc s1 Joe SF s2 Ann NY s3 Lisa SF loc total_sales SF 12.00 NY 10.00 sales store_info loc_sales Simplify and accelerate iterative and predictable workloads, such as ETL, BI/dashboarding queries MVs can be based on one or more Amazon Redshift tables or external tables (Spectrum, Federated) Efficient incremental maintenance Scheduled, automatic, or manually timed refresh Amazon Redshift auto query rewrite optimizes queries by replacing native tables with materialized views “The Amazon Redshift materialized view auto query rewrite feature reduced dashboard load times from 8 minutes to just 500 ms. The best part is that this is completely transparent for Tableau and the business user.” —Arman Nasrollahi, Home24

Slide 12

Slide 12 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview Redshift Query Editor v2 – Notebook Support Data users engaging in advanced analytics work on multiple queries at a time to perform various tasks for their data analysis. organize related queries by saving them together in a folder, or combining them into a single saved query with multiple statements. The Notebooks support provides an alternative way to embed all queries required for a complete data analysis in a single document using SQL cells. You can share your Notebooks with team members, similar to how you share your saved queries in Query Editor V2. Documenting your work precisely enables further collaboration with other users.

Slide 13

Slide 13 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview Amazon EMR Serverless Easily run petabyte-scale data analytics in the cloud without managing, tuning, optimizing, securing, and operating clusters Quickly run large-scale applications that use open-source frameworks of your choice, including Spark, Hive, and Presto Automatically scale resources up and down as needed based on the changing requirements of your application Pay only for the resources you use for data analytics at scale

Slide 14

Slide 14 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview Easily run Apache Kafka clusters without rightsizing cluster capacity Instantly scale I/O without worrying about scaling capacity up and down or reassigning partitions Pay for the data volume you stream and retain Amazon MSK Serverless

Slide 15

Slide 15 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA Simple to use Simplify streaming data processing by eliminating capacity management Flexible scaling Automatically scale capacity in response to changing data volumes Automated high availability Provide built-in availability and fault tolerance by default Lower your costs Pay per gigabyte of data written, read, and stored Amazon Kinesis Data Streams on demand

Slide 16

Slide 16 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Glue Autoscaling Cost = f(compute) t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 Without Autoscaling With Autoscaling Job execution timeline List operation Wide transform Uneven distribution of data partitions AWS Glue job Tear-down Setup Potential savings Preview

Slide 17

Slide 17 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Streaming Autoscaling (preview) Cost = f(compute) t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 Without Autoscaling With Autoscaling Job execution timeline Low stream activity Peak streaming record arrival rate Scale down as stream activity decreases Glue job Tear-down Setup Potential Savings Initial scale up Scale down to match stream activity

Slide 18

Slide 18 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Glue Studio Notebook (Preview) Interactive AWS Glue jobs development Submit AWS Glue jobs from the AWS Glue Studio notebook Use notebook magic to define transforms in SQL and control cost Built-in monitoring support Preview

Slide 19

Slide 19 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Glue Interactive Sessions Development tool of your choice Rapid development Built-in cost control Preview A dedicated cluster created in seconds for developers to interactively build data pipelines with choice of IDE…

Slide 20

Slide 20 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. … and advanced transforms like PII detection for advanced cleansing Scan Data Detect Entities Remeditate 1 2 3 Full Scan Sample Scan Built-in Entities (e.g. SSN, passport) Custom Entities Store results Redact/mask results

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA Amazon RDS Custom Amazon RDS Custom is a managed database service for legacy, custom, and packaged applications that require access to the underlying operating system and database environment Amazon RDS Custom is available for RDS for Oracle and for SQL Server

Slide 24

Slide 24 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA ML-powered database advisor for Amazon RDS First released for Amazon Aurora Delivers insights and recommendations to help resolve issues in minutes Uses ML models informed by years of operating thousands of databases for Amazon.com Automatically detects and diagnoses hard-to-find performance bottlenecks and operational issues Continuously monitors database workloads at scale No ML experience required Amazon DevOps Guru for RDS

Slide 25

Slide 25 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA Amazon DynamoDB Standard- Infrequent Access (Standard-IA) table class The Standard-IA table class offers 60% lower storage costs than DynamoDB Standard tables. Standard-IA tables offer the same performance, durability, data availability, and massive scalability as existing DynamoDB Standard tables. Switch between table classes with a single click in the DynamoDB console, or using the AWS CLI or AWS SDK. Also, use the same DynamoDB APIs and service endpoints. Lower storage costs No performance trade-offs No developer overhead

Slide 26

Slide 26 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA What’s new in Amazon Timestream R E A L - T I M E A N A L Y T I C S A R E O R D E R S O F M A G N I T U D E F A S T E R A N D C H E A P E R Multi-measure records Scheduled queries Magnetic store writes • A new data modeling capability for cost-effective data storage, performant data access, and ease of use • Store multiple measurements and events in a single table row, instead of storing one measure per table row • Improve query performance and reduce costs by an order of magnitude • Fully managed, serverless, and scalable solution for calculating and storing aggregates, rollups, and other real-time analytics typically used to power dashboards, business reports, and other applications • Simply define the computation/query to precompute and its schedule. Amazon Timestream will automatically and periodically run the queries and reliably write the results into a separate table. • Flexible and cost-effective solution for late-arriving data • Ability to ingest data directly into the magnetic store • As you ingest data, Amazon Timestream will automatically determine whether the data gets written to the memory store or the magnetic store, based on the timestamp of the data and the configured data retention window for the memory and magnetic stores

Slide 27

Slide 27 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview AWS DMS Fleet Advisor A C C E L E R A T E D A T A B A S E A N D A N A L Y T I C S M I G R A T I O N S W I T H A U T O M A T E D I N V E N T O R Y A N D M I G R A T I O N R E C O M M E N D A T I O N S Generates a list of possible migration targets Automatically gathers database fleet information Results in hours instead of weeks and months

Slide 28

Slide 28 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA Database Migration Service – Time Travel Store and encrypt AWS DMS logs using Amazon S3, and view, download and obfuscate the logs within a certain time frame Log and debug replication tasks Time Travel can be used in all AWS Regions with DMS-supported PostgreSQL source endpoints and DMS-supported PostgreSQL and MySQL target endpoints

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview Amazon EC2 Trn1 Instances for ML Training • Training larger models requires an increase in compute power, which also increases costs • Increasing costs become a barrier to innovation and growth INCREASING ML COMPLEXITY • Businesses need higher precision in their model predictions • Larger and more complex models result in more frequent retraining INCREASING TRAINING COSTS

Slide 33

Slide 33 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview Amazon EC2 Trn1 Instances for ML Training 16 AWS Trainium chips, purpose-built for deep learning training Most cost efficient for large- scale distributed training AWS Neuron SDK integrated with TensorFlow and PyTorch Deployed in Amazon EC2 UltraClusters with tens of thousands of Trainium chips and petabit-scale networking 800 Gbps networking and ultra- highspeed interconnect between accelerators Fastest ML training in the cloud

Slide 34

Slide 34 text

Slide 35

Slide 35 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA Amazon SageMaker Notebooks P E R F O R M D A T A E N G I N E E R I N G , A N A L Y T I C S , A N D M L W O R K F L O W S I N O N E N O T E B O O K Discover, manage, create, terminate, and connect to Amazon EMR clusters Interactively access, transform, and analyze a wide range of data Build, train, and deploy models using your preferred framework Amazon SageMaker

Slide 36

Slide 36 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA Amazon SageMaker Training Compiler A C C E L E R A T E T H E T R A I N I N G O F D E E P L E A R N I N G M O D E L S Training compiler Accelerate training times by up to 50% through more efficient use of GPUs Reduce training costs Save time and money associated with training Integrated with existing frameworks Use familiar framework APIs with minimal code changes Amazon SageMaker

Slide 37

Slide 37 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA Amazon SageMaker Inference Recommender A C C E L E R A T E T H E D E P L O Y M E N T O F M O D E L E N D P O I N T S Automatic deployment recommendations Optimal instance type/count and container parameters Reduce inference costs Receive endpoint configuration recommends for best price performance Fully managed load testing Run and review fully managed load tests on a set of instance types to evaluate performance and trade-offs Amazon SageMaker

Slide 38

Slide 38 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview Amazon SageMaker Serverless Inference D E P L O Y M L M O D E L S F O R I N F E R E N C E W I T H O U T C O N F I G U R I N G O R M A N A G I N G U N D E R L Y I N G I N F R A S T R U C T U R E Optimize for intermittent or unpredictable workloads Automatic and fast scaling when usage is required Pay per use Pay only for the resources consumed, not idle time Fully managed Underlying infrastructure, security, patching, monitoring and logging is all managed Amazon SageMaker

Slide 39

Slide 39 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA Amazon SageMaker MLOps T R A C K , A S S E S S , A N D M O N I T O R Y O U R M L M O D E L S Improved lineage management Retrieve all ML lineage across accounts and in a single query, from data to endpoint Model quality and bias detection in workflows SageMaker Model Monitor and SageMaker Clarify integrated into SageMaker Pipeline workflows Monitor endpoints from SageMaker Model Registry Track endpoints, custom metadata, and model metrics Amazon SageMaker

Slide 40

Slide 40 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA Amazon SageMaker Ground Truth Plus D E L I V E R H I G H - Q U A L I T Y T R A I N I N G D A T A S E T S F A S T A N D R E D U C E D A T A L A B E L L I N G C O S T S Increase data quality through ML-powered data labeling Access expert data labelers Reduce data labeling costs with assistive labeling features Improve operational efficiency by reviewing project metrics Make data labeling accessible to data operations and program managers

Slide 41

Slide 41 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA Amazon SageMaker Canvas G E N E R A T E M L P R E D I C T I O N S – N O C O D E R E Q U I R E D Quickly access and prepare data sources for ML AutoML built in to generate accurate predictions Share ML models with data science teams Amazon SageMaker

Slide 42

Slide 42 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview Amazon SageMaker Studio Lab L E A R N A N D E X P E R I M E N T W I T H M A C H I N E L E A R N I N G Amazon SageMaker Studio Lab Free, no-configuration development environment All you need is an email address to get started As many CPU and GPU sessions as needed Automatically saves session data Models ready for production

Slide 43

Slide 43 text

Slide 44

Slide 44 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA Amazon Textract S P E C I A L I Z E D S U P P O R T F O R I D E N T I T Y D O C U M E N T S Amazon Textract OUTPUT First Name: JORGE Last Name: SOUZA Middle Name: Address Line1: 100 MAIN STREET Address Line 2: City: ANYTOWN State: MA Document Number: 820BAC729CBAC Expiration Date: 01/20/2020 Date of Birth: 03/18/1978 ID Type: Driver License Date of Issue: 03/18/1978 Issued By: MASSACHUSETTS Class: D Restrictions: NONE Endorsements: NONE DOCUMENT

Slide 45

Slide 45 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA Amazon Personalize I N T E L L I G E N T U S E R S E G M E N T A T I O N Amazon Personalize • Identify users interested in a genre, category, or any other item attribute • Identify users interested in a given item such as a movie, product, and so on • More effective campaigns through marketing channels • Acquire users for new product categories, genres, channels, and more • Improve return on investment for your marketing spend Action movie fans

Slide 46

Slide 46 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA Amazon Personalize U S E - C A S E O P T I M I Z E D R E C O M M E N D E R S F O R R E T A I L , A N D M E D I A A N D E N T E R T A I N M E N T Retail • Recommended for you • Customers who viewed this also viewed • Frequently bought together • Most viewed • Best sellers Media and entertainment • Top picks for you • Because you watched X • More like Y • Most popular Amazon Personalize

Slide 47

Slide 47 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview Amazon Lex A U T O M A T E D C H A T B O T D E S I G N E R Amazon Lex Analyze thousands of lines of transcripts in minutes Discover the most common intents and the information needed to fulfill them Accelerate chatbot deployment Allows you to design chatbots in hours rather than weeks Improve customer experience Helps ensure that intents are well defined and separated

Slide 48

Slide 48 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AI & ML Scholarship Program Learn ML faster with AWS DeepRacer Student An all-new free service for students over the age of 16 Fast-track your career with scholarships 2,000 students per year to be awarded scholarships as part of the AWS AI & ML Scholarship program Receive mentorship from AWS and Intel ML experts Students in the AWS AI & ML Scholarship program receive mentoring and career guidance from industry experts

Slide 49

Slide 49 text

Slide 50

Slide 50 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. New services: • AWS IoT ExpressLink • AWS IoT TwinMaker • AWS IoT FleetWise • AWS IoT RoboRunner • FreeRTOS out-of-the-box AWS IoT Announcements Other: • AWS IoT Greengrass & Systems Manager • AWS IoT Greengrass Software Catalog • AWS IoT SiteWise Hot/Cold Storage Tiers • AWS IoT Fleet Indexing enhancements • FreeRTOS extended maintenance

Slide 51

Slide 51 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview AWS IoT ExpressLink AWS IoT ExpressLink packages all the complex and security-critical code into a single hardware component. A 3-wire serial interface is all you need to connect any embedded device to the cloud. TX RX GND WAKE INT RST

Slide 52

Slide 52 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview AWS IoT ExpressLink IoT library Hardware security Radio library Your application Host processor Hardware security Radio library Your application Wireless module IoT library AWS IoT ExpressLink Host processor Hardware security library Without AWS IoT ExpressLink With AWS IoT ExpressLink

Slide 53

Slide 53 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview AWS IoT ExpressLink int main() { print("AT+CONNECT\n"); while(1){ print("AT+SEND data {\"A\"=%d}", getSensorA()); delays(1); } } In less than 10 lines of code

Slide 54

Slide 54 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview AWS IoT ExpressLink – Four evaluation kits Vendor Evaluation Kit Technology Buy Espressif ESP32-C3-DevKitM1 Wi-Fi Infineon CCM Eval Kit Wi-Fi u-blox Nora-W2 Mini Evaluation kit Wi-Fi u-blox Sara – R5 AWS Evaluation kit Cellular

Slide 55

Slide 55 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview AWS IoT TwinMaker E A S I L Y C R E A T E D I G I T A L T W I N S O F R E A L - W O R L D S Y S T E M S T O O P T I M I Z E O P E R A T I O N S Easily access data no matter where it lives Accurately model your built environment Create immersive 3D views

Slide 56

Slide 56 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview AWS IoT TwinMaker – How it works AWS IoT TwinMaker Digital twin Operations Planning Operational and maintenance history Real-time process data Real-time IoT sensor data CAD/BIM models Point cloud scans Video feeds Geo-spatial metadata Statistical and physics-based models Integrated asset model 3D visualization Insights AWS Cloud End-user web application

Slide 57

Slide 57 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview AWS IoT TwinMaker – Features Data connectors Access data from diverse sources using built-in connectors, or easily create your own connectors to AWS and third-party data sources. Model builder Create entities to virtually represent physical systems, specify relationships between them, and connect them to different data sources to form a digital twin graph Scene composer Combine existing 3D visual models with real- world data from the digital twin graph to compose an interactive 3D view of your physical environment App toolkit Integrate digital twins into 3D-enabled applications using Amazon Managed Grafana or AWS Partners for end users to monitor and improve operations

Slide 58

Slide 58 text

Slide 59

Slide 59 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview Build virtual representations of vehicles and apply a common data format to structure and label vehicle attributes, sensors, and signals Easily access standardized, fleet-wide vehicle data AWS IoT FleetWise Easily collect, transform, and transfer vehicle data to the cloud at scale Reduce costs with intelligent data filtering Detect and mitigate problems more quickly by surfacing vehicle data in near real time Select which data to transfer, define rules and events for when to transfer it, and automatically reduce redundant data through dynamic data selection Take quick, corrective action by notifying operations or manufacturing groups in near real time when problems occur

Slide 60

Slide 60 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Preview AWS IoT RoboRunner B U I L D A P P L I C A T I O N S T H A T H E L P R O B O T S W O R K T O G E T H E R S E A M L E S S L Y Connect robots and work management systems to a common infrastructure Enable robots from different vendors to work together Simplify building applications for optimizing robot fleets

Slide 61

Slide 61 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA IoT devices Hub/gateway AWS IoT Greengrass AWS Systems Manager integrates AWS IoT Greengrass Systems Manager OS upgrades, schedule maintenance tasks, and remotely access their edge device fleet

Slide 62

Slide 62 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Greengrass Software Catalog A W S I O T G R E E N G R A S S S O F T W A R E C O M P O N E N T S D E V E L O P E D B Y T H E G R E E N G R A S S C O M M U N I T Y

Slide 63

Slide 63 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. GA AWS IoT SiteWise cold tier storage support 1 0 1 0 1 1 0 0 1 0 1 0 1 0 0 1 0 1 0 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 101011010 01010 10101 0101001010 01010 10101 1010101010 1010101010 1010101010 1 0 1 0 1 1 0 0 1 0 1 0 1 0 0 1 0 1 0 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 01010101 ASSET MODEL Property: Attribute tool_id : Property: Time-series pressforce-ch1 : Property: Formula Avg Pressforce : f (pressforce-ch1) Ingest equipment data into AWS in minutes Structure data and specify performance metrics for your equipment and processes Store data in a time-series- optimized data store Create and share dashboards to visualize live and historical equipment data