JumpStart to Build Generative AI with Amazon SageMaker

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

© 2023, Amazon Web Services, Inc. or its affiliates. Agenda Build Generative AI with SageMaker JumpStart How To Customize Pre-trained ML Models Optimize ML Inference with SageMaker Inference Recommender Benefits of ML in Cloud

Slide 4

Slide 4 text

© 2023, Amazon Web Services, Inc. or its affiliates. Generative AI – Text Generation Jack went to the university in the early 1970s as a graduate student and started the company that would become McDonald’s. Though he was a long time away from his family, he was an avid cook. “My mom is a huge foodie,” he says. “She likes to know things about what I’m eating.” Her recipes helped him develop an idea of how to go about the food business and eventually made him a fast-food millionaire in 1993. “I mad sure that my family was always watching,” he says. “And I always liked to tell them about everything I was doing on a regular basis.” That early obsession with recipes led to his own idea of what a McDonald’s menu would look like: a menu that would make people say hello to their food, even though they’ve been away.

Slide 5

Slide 5 text

Slide 6

Slide 6 text

© 2023, Amazon Web Services, Inc. or its affiliates. Key trends in AI/ML • Models are becoming more complex, with end users moving from classical ML to deep learning • State-of-the-art deep learning models are getting larger and larger as we find that larger models generalize better Source: NVIDIA Transformers 65M BERT 340M GPT-2 1.5B GPT-2 8B 8.3B T5 11B Turing-NLG 17B GPT-3 175B Megatron-Turing 530B GPT-3 1T 1 trillion MID 2017 2018 2019 MID 2019 LATE 2019 2020 MID 2020 LATE 2021 2022 Model size Time 15,000x increase in 5 years

Slide 7

Slide 7 text

Slide 8

Slide 8 text

© 2023, Amazon Web Services, Inc. or its affiliates. A fully managed service that enables data scientists and developers to quickly and easily build machine-learning based models into production smart applications. Amazon SageMaker

Slide 9

Slide 9 text

Slide 10

Slide 10 text

© 2023, Amazon Web Services, Inc. or its affiliates. End-to-End Machine Learning Platform Zero setup Flexible Model Training Pay by the second $ Amazon SageMaker 손 쉬 운 기 계 학 습 모 델 생 성 , 훈 련 및 서 비 스 배포 완전 관리 서비 스

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

© 2023, Amazon Web Services, Inc. or its affiliates. Tasks Algori thms/models Vision Text Tabular Audio SageMaker JumpStart: ML hub for SageMaker Customers 400+ algorithms and pre-trained, state-of-the-art, open-source models from PyTorch Hub, TensorFlow Hub, and Hugging Face, etc.

Slide 16

Slide 16 text

© 2023, Amazon Web Services, Inc. or its affiliates. Browse and search SageMaker JumpStart content Search for topics or problem types, and get relevant results across all content Browse by content type to explore solutions, models, example notebooks, blogs, and video tutorials

Slide 17

Slide 17 text

© 2023, Amazon Web Services, Inc. or its affiliates. Easily launch pre-built solutions Launch solutions through cloud formation with a single click Easily manage assets from Amazon SageMaker JumpStart Open pre-populated notebooks for solutions to solve the business problems end to end

Slide 18

Slide 18 text

© 2023, Amazon Web Services, Inc. or its affiliates. Easily deploy or fine-tune models Deploy or fine-tune pre- trained models with a single click Open pre-populated notebooks to perform inference on deployed models 400+ pre-trained open source models from Hugging Face, PyTorch, TensorFlow Hubs

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

© 2023, Amazon Web Services, Inc. or its affiliates. How to Fine-tune and Train pre-trained ML Models? ML Problem Framing Real-World Problem Define ML Problem Data Preparation Build Training Deploy Amazon SageMaker 한국어 Fine-tuning 가나다라 마바사아 … 1 2

Slide 22

Slide 22 text

© 2023, Amazon Web Services, Inc. or its affiliates. How to Fine-tune and Train pre-trained Models • Train Fine-tunable pre-trained models • Fine-tune and train from scratch with Hugging Face, or other open source models.

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

© 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates. Hugging Face on Amazon SageMaker Scale and accelerate your NLP projects with Hugging Face Integrations

Slide 26

Slide 26 text

© 2023, Amazon Web Services, Inc. or its affiliates. What are the Hugging Face libraries? Open-source Datasets, Tokenizers and Transformers Popular 52k+ GitHub stars (March 2021), 1M+ downloads per month Intuitive NLP-specific Python frontends based on PyTorch or TensorFlow State of the art Transformer-based models are state-of-the-art, enable transfer-learning and scale Comprehensive Model zoo with 7000+ model architectures, 160+ languages

Slide 27

Slide 27 text

© 2023, Amazon Web Services, Inc. or its affiliates. A strong partnership to make NLP easy & accessible for all Hugging Face is the most popular Open Source company providing state of the art NLP technology Hugging Face SageMaker offers high performance resources to train and use NLP Models AWS https://huggingface.co/ https://aws.amazon.com/sagemaker/

Slide 28

Slide 28 text

Slide 29

Slide 29 text

© 2023, Amazon Web Services, Inc. or its affiliates. How to Fine-tune and Train pre-trained ML Models? ML Problem Framing Real-World Problem Define ML Problem Data Preparation Build Training Deploy 한국어 Fine-tuning 가나다라 마바사아 … 1 2 + Amazon SageMaker Hugging Face

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Slide 35

Slide 35 text

© 2023, Amazon Web Services, Inc. or its affiliates. Amazon SageMaker Training How does training happen HuggingFace validation(optional) test(optional) ECR(1) S3 ML Instance ml.p3.2xlarge HuggingFace linear-learner PCA DeepAR BlazingText Image classification … Object Detection DLC Images(2) S3 SageMaker Notebook SageMaker Training Job train Model (1) ECR: Elastic Container Registry (2) DLC: Deep Learning Container

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Slide 39

Slide 39 text

Slide 40

Slide 40 text

© 2023, Amazon Web Services, Inc. or its affiliates. Amazon SageMaker Deployment Hosting Services Inference Image Training Image Training Data Model artifacts Amazon SageMaker Amazon S3 Amazon ECR Model artifacts

Slide 41

Slide 41 text

Slide 42

Slide 42 text

© 2023, Amazon Web Services, Inc. or its affiliates. Amazon SageMaker Deployment Hosting Services Inference Image Training Image Training Data Model artifacts Endpoint Amazon SageMaker Amazon S3 Amazon ECR Model artifacts Inference Image

Slide 43

Slide 43 text

© 2023, Amazon Web Services, Inc. or its affiliates. Amazon SageMaker Deployment SageMaker Endpoints (Private API) Auto Scaling group Availability Zone 1 Availability Zone 2 Availability Zone 3 Elastic Load Balancing Model Endpoint Client Deployment / Hosting Amazon SageMaker ML Compute Instances Input Data (Request) Prediction (Response)

Slide 44

Slide 44 text

© 2023, Amazon Web Services, Inc. or its affiliates. Amazon SageMaker Deployment SageMaker Endpoints (Public API) Auto Scaling group Availability Zone 1 Availability Zone 2 Availability Zone 3 Elastic Load Balancing Model Endpoint Amazon API Gateway Client Deployment / Hosting Amazon SageMaker ML Compute Instances Input Data (Request) Prediction (Response)

Slide 45

Slide 45 text

© 2023, Amazon Web Services, Inc. or its affiliates. SageMaker ML instance options B A L A N C I N G B E T W E E N C O S T A N D P E R F O R M A N C E High throughput, and low-latency access to CUDA GPU INSTANCES P3 G4 Low throughput, low cost, most flexible CPU INSTANCES C5 Inf1: High throughput, high performance, and lowest cost in the cloud CUSTOM CHIP Inf1

Slide 46

Slide 46 text

© 2023, Amazon Web Services, Inc. or its affiliates. ML instance ML instance Endpoint Load testing K N O W Y O U R E N D P O I N T S Artificial requests Amazon SageMaker endpoint Endpoint Auto-scaling group Availability Zone 1 Availability Zone 2 ML instance ML instance ML instance ML instance Amazon CloudWatch Elastic Load Balancing

Slide 47

Slide 47 text

© 2023, Amazon Web Services, Inc. or its affiliates. SageMaker Inference Recommender F E A T U R E S Designed for MLOps engineers and data scientists to reduce time to get models into production Run extensive load tests that include production requirements – throughput, latency Load tests Get endpoint configuration settings that meet your production requirements Endpoint recommendations Instance recommendations Instance type recommendation for initial deployments

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

Slide 50

Slide 50 text

Slide 51

Slide 51 text

Slide 52

Slide 52 text

Slide 53

Slide 53 text

Slide 54

Slide 54 text

© 2023, Amazon Web Services, Inc. or its affiliates. Fully integrated with Amazon SageMaker Monitor Monitor for bias and feature attribution drifts Automate pipeline Deploy Flexible deployment with best-price performance Fine-tune Customization for specific domains Human-in- the-Loop Provide feedback, label data, active learning through human-in-loop

Slide 55

Slide 55 text

© 2023, Amazon Web Services, Inc. or its affiliates. End-to-End Machine Learning with Amazon SageMaker ML Problem Framing Real-World Problem Define ML Problem Data Preparation Build Training Deploy SageMaker JumpStart + Hugging Face • SageMaker Endpoint • SageMaker Inference Recommender SageMaker Training Job SageMaker Studio Raw Data

Slide 56

Slide 56 text

Slide 57

Slide 57 text

© 2023, Amazon Web Services, Inc. or its affiliates. Call-to-Action • AWS Machine Learning Low-Code Immersion Day • https://catalog.us-east-1.prod.workshops.aws/workshops/f560a788-af64- 4e5a-a02c-a6c88516ab02/en-US • SageMaker Immersion Day • https://sagemaker-immersionday.workshop.aws/ • Using HuggingFace Models on Amazon SageMaker • https://catalog.us-east-1.prod.workshops.aws/workshops/74da6491-2f74- 466a-86a5-3bcce6541904/en-US • Amazon SageMaker Canvas Immersion Day • https://catalog.workshops.aws/canvas-immersion-day/en-US

Slide 58

Slide 58 text