Open Source re:Invent re:Cap

This deck walks you through some of the important open source related news and announcements both during pre:Invent and re:Invent.

Transcript

1. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved.
2. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS re:Invent recap Ricardo Sueiras OPEN SOURCE EDITION Developer Advocate, Open Source Amazon Web Services
3. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. pre:Invent
4. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Summary Amazon Linux 2022 Database and analytics Grafana AWS Proton Syne Tune AWS Amplify
5. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Linux 2022 Predictable two-year major release cycle and long-term support Frequent and flexible updates Improved security posture Repository locking Kernel hardening Kernel live patching Amazon Linux 2022 (AL2022) is our next generation of Amazon Linux. It provides a security focused, stable, and high-performance execution environment to develop and run cloud applications. AL2022 uses Fedora upstream, and is available for you to use in your AWS environments or you can download and use it in your own environments. https://github.com/amazonlinux/amazon-linux-2022 Public Preview
6. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Database and Analytics • Amazon Athena ACID transactions powered by Apache Iceberg [preview] • Redis 6.2 now available in Amazon ElastiCache for Redis • MSK Connect makes it easy to configure and deploy a connector using Kafka Connect with a just few clicks • Amazon Neptune JDBC driver and .NET SigV4 library • MySQL 8.0 support in Amazon Aurora • Multi AZ deployment options for Amazon RDS PostgreSQL and MySQL
7. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Grafana • Grafana 8.2 version release • New data source support including CloudFlare, Zabbix, and Splunk • Amazon Managed Grafana announces new data source plugins for Amazon Athena and Amazon Redshift • Geomap visualisations
8. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Proton • Terraform Open Source for infrastructure provisioning • Git management of infrastructure as code templates • Sync templates that are sourced in your git repository Public Preview Help customers on their journey to Gitops
9. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. syne-tune • Python library for large scale distributed hyperparameter and neural architecture optimization (HPO) • Implementations of several state-of-the-art global optimizers, such as Bayesian optimization, Hyperband, and population-based training • Bring your own global optimization algorithm. https://github.com/awslabs/syne-tune
10. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Amplify • New GraphQL Transformer (v2) • Amplify UI Authenticator for React/Angular/Vue • Export Amplify backends to CDK stacks • Add custom resources using AWS CDK and Cloudformation • New in-app messaging powered by Pinpoint (Developer Preview)
11. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Other open source launches • ECS optimised AMIs – HashiCorp packer recipes used to build the base images for Amazon ECS
12. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. re:Invent
13. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Modernising Mainframes with open source Amazon FSx for OpenZFS Amazon FSx for Lustre Containers Anywhere Amazon ECR Karpenter AWS Bug Bust AWS IoT Greengrass with AWS System Manager AWS Control Tower Account Factory for Terraform Serverless open source analytics Amazon DevOps Guru for RDS AWS CDKv2 and Construct Hub AWS SDKs
14. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon FSx for OpenZFS • FSx for OpenZFS is powered by AWS Graviton • Up to 1 million IOPS, 12GB/s compressed throughput • High performance, close to local storage • Deploy and then access volumes in AWS, on-premises • Windows/Mac/Linux clients via NFS (v3,v4,v4,1 and v4.2)
15. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon FSx for Lustre • Bi-directional syncronisation of your filesystems with Amazon S3 (including deleted files) • Synchronise your filesystems with multiple Amazon S3 buckets or prefixes New capabilities • New generation (powered by AWS Graviton processors) file system up to 5x higher throughput, 400 Gbs network bandwidth and up to 60% lower cost
16. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers Anywhere • AWS Marketplace for Containers Anywhere – find, subscribe and deploy Kubernetes applications to your Kubernetes clusters • Deploy to your Amazon EKS clusters, or your on premises clusters using EKS Anywhere • Find and deploy your favourite open source software • Configuration instructions provided during deployment process
17. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon ECR • Pull through cache for your containers - cache public images into your ECR registry and leverage all the benefits of Amazon ECR without the operational burden of syncing public images. • You can make Docker official images available in your Amazon ECR Public repositories • The images are available to browse in the ECR Public gallery at https://gallery.ecr.aws/docker • Additional container image scanning and security tools • Sign images with Notary v2
18. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon ECR
19. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Karpenter • Provides flexible configuration options for auto scaling resources to support your Kubernetes workloads • Karpenter provisioner, a custom resource where you define your scaling logic • Architecture separates provider and carpenter logic and provides extension points to extend to other providers • Apache 2.0 https://github.com/aws/karpenter
20. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS BugBust • Identify and resolve bugs in open source projects (Java and Python) • Competition with prizes and bragging rights • Amazon CodeGuru will be used to identify potential bugs, which builders can select to fix • Amazon CodeGuru will validate fix and submit PR back to source projects
21. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS IoT Greengrass & AWS Systems Manager • Simplify the deployment, configuration and management of AWS IoT Greengrass on your devices using AWS Systems Manager • Deploy AWS Systems Manager to your devices, and then manage/patch them via consistent operational policies • Single, consolidated view of your cloud, on premises and edge infrastructure
22. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Mainframe Modernisation AWS Mainframe Modernization provides tools and resources to help you plan and implement migration and modernization from mainframes to AWS managed runtime environments. It provides tools for analyzing existing mainframe applications, developing or updating mainframe applications using COBOL or PL/I, and implementing an automated pipeline for continuous integration and continuous delivery (CI/CD) of the applications. Modernise your mainframe applications using projects like gnuCOBOL
23. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Control Tower Account Factory for Terraform • You can now provision AWS Control Tower via Terraform Infrastructure as Code • AWS Control Tower Account Factory for Terraform (AFT), is a new Terraform module maintained by the AWS Control Tower team https://github.com/aws-ia/terraform-aws-control_tower_account_factory
24. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Serverless Analytics • Amazon EMR – specify your open source frameworks, and Amazon EMR provisions, manages and scales resources • Data Pipelines, Shared Clusters and Interactive workload use cases • Amazon MSK – provision Apache Kafka secure and highly available clusters that automatically scale as your application I/O scales Public Preview A M A Z O N E M R S E R V E R L E S S A M A Z O N M S K S E R V E R L E S S
25. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon EMR from SageMaker Studio • Discover, manage, create, terminate, and connect to Amazon EMR clusters from within SageMaker Studio • Utilize “templates” – a new way to configure and provision clusters for your workload needs with support from seasoned DevOps practitioners • Connect to, debug, and monitor Spark jobs running on an Amazon EMR cluster from within a SageMaker Studio Notebook
26. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon DevOps Guru for RDS • Detect, diagnose, and resolve performance and operational issues in Amazon Aurora PostgreSQL and MySQL • It uses ML to automatically identify and analyse issues such as performance- related database issues, over-utilization of host resources, database bottlenecks, or misbehaviour of SQL queries • Issues found trigger events that you can use to trigger automatic/real-time resolutions
27. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CDK • CDK v2 package changes that consolidate all constructs into a single aws-cdk-lib package • Stable APIs via semantic versioning (semver) to make it easier to update to new minor versions and better way to incorporate experimental constructs • Available across all languages CDK supports and Go preview • New capabilities: CDK Watch ,CDK Pipelines, CDK assertions
28. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CDK Construct Hub • A centralised repository for all the open source constructs for AWS CDK • CDK, cdk8s, cfktf and all languages • Best practices for publishing to CDK Construct Hub - https://go.aws/3plaRGp
29. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. New AWS SDKs Developer Preview Kotlin Swift Rust
30. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Other announcements • ROSA now integrates with AWS Licence Manager making it easy to share subscriptions within an AWS Organisation • Amazon Corretto launched the preview of the generational model to the Shenandoah GC - higher throughput and better response times • New simplified FreeRTOS out of the box integration with AWS IoT • New FreeRTOS Extended maintenance plan for ten years • New O3DE stable release, 21.11
31. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. ROS: AWS Robotics Startup Accelerator • four-week program with AWS and MassRobotics industry experts and AWS robotics experts • Up to $10,000 in promotional credits for use of AWS IoT, Robotics, and ML services • Key dates - registration opens on November 29, and closes on January 16, 2022
32. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. re:Invent Open Source sessions A selection of must watch sessions covering open source technologies
33. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. OPN207: Top 10 observability challenges.. When building and operating cloud- native applications, such as containerized microservices in Kubernetes or serverless applications, you don’t want to proceed by guesswork. In other words, you want to capture and collect all relevant signals (logs, metrics, traces) from your applications and infrastructure so that you can act upon it in a timely manner. This is called observability. Speakers Michael Hausenblas Abstract Link - > TBA
34. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. OPN204: Introduction to GraphQL GraphQL is an open-source API query language. This session provides an introduction to GraphQL theory and practice. Learn the three superpowers of GraphQL, how GraphQL compares to REST API, and how to take GraphQL into production. Speakers Robert Zhu Abstract Link - > TBA
35. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. OPN301: Using Rust to minimize environmental impact Rust is one of the most energy-efficient and safe programming languages. With Rust, it may be possible to reduce the environmental impact of the IT industry by 50% and prevent 70% of all high- severity CVEs. In this session, dive into the “super powers” of Rust, hear about the work ahead to give those powers to every engineer, and hear the ways you can contribute. Speakers Carl Lerche and Shane Miller Abstract Link - > TBA
36. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. CON2026: Kubernetes at AWS Learn about the AWS vision for Amazon EKS and how recent enhancements to ease of use, security, scale, and reliability make it a smart choice for your applications. Hear how the platform abstracts away the infrastructure responsibilities of service owners, meets internal requirements around availability and security, and simplifies the developer experience. Speakers Allan Naim, Anubhav Dhoot Abstract Link - > TBA
37. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. CON304: Deep Dive on Amazon EKS Amazon EKS is a fully managed Kubernetes service. This session covers recent enhancements to EKS and dives deep into the latest features. Learn how EKS gives you the flexibility to start, run, and scale Kubernetes applications in the AWS Cloud or on premises and how customers trust EKS to run their most sensitive and mission-critical applications. Speakers Mike Stefaniak Abstract Link - > TBA
38. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. DAT304: Deep dive on Amazon ElastiCache for Redis Learn about the key capabilities of ElastiCache for Redis, including caching to accelerate performance and reduce load for your operational databases and data stores, auto scaling Redis clusters to easily provision resources to match workload demands, and achieving secure, cross-Region replication with global datastores. Plus, dive into the latest ElastiCache for Redis features. Speakers Joseph Travaglini, Lindsey Berg Abstract Link - > TBA
39. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. OPN308: ML with Metaflow At AWS re:Invent 2019, Netflix addressed the data science challenge by open- sourcing Metaflow, their human-centric framework for data science. This session explores data science pipelines on Amazon EKS and other open platforms and details how Metaflow is bringing its human-centric vision to the Kubernetes ecosystem. Speakers Rob Hilton, Romain Cledat Abstract Link - > TBA
40. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. OPN307: OpenSearch: Building the future of search together In this session, dive deep into how OpenSearch was created. Gain insight into new functionalities being built for OpenSearch, including improved performance and reliability, lower cost at scale, observability analytics, machine learning capabilities, and extensibility enhancements designed to enable anyone to build and share their own OpenSearch products and features. Speakers Jules Graybill, Saurabh Mehta Abstract Link - > TBA
41. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Apache Airflow Learn how to use Apache Airflow to orchestrate the export of time series data from Amazon Timestream into your data lake using the open source Data Wrangler python library. Speakers Danilo Poccia, Javier Ramirez and Ricardo Sueiras Abstract Link - > TBA
42. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you! © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved.
43. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Please complete the session survey © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved.