Slide 1

Slide 1 text

© 2021, Amazon Web Services, Inc. or its Affiliates. Sungmin Kim Solutions Architect, AWS Build Visual Search Application powered by Amazon SageMaker

Slide 2

Slide 2 text

Agenda • Amazon Rekognition • Amazon SageMaker • SageMaker Studio Notebooks • SageMaker Training Job • SageMaker Endpoints

Slide 3

Slide 3 text

AnyCompany Dream is

Slide 4

Slide 4 text

Which Advertisement Is Better?

Slide 5

Slide 5 text

(B) may be Better, isn’t It?

Slide 6

Slide 6 text

Why (B) is Better? Climbing sunglasses Climbing shoes How about climbing gloves?

Slide 7

Slide 7 text

How to Choose the Most Relevant Advertisement? • may be easy to find, if a image has correct tags. • If so, how to tag millions of images in fast, cost-efficient way? • Machine Learning may save your life

Slide 8

Slide 8 text

Automatic Image Tagging Automatic Image Recognition

Slide 9

Slide 9 text

Amazon SageMaker Label data Aggregate & prepare data Store & share features Auto ML Spark/R Detect bias Visualize in notebooks Pick algorithm Train models Tune parameters Debug & profile Deploy in production Manage & monitor CI/CD Human review Ground Truth Data Wrangler Feature store Autopilot Processing Clarify Studio Notebooks Built-in or Bring-your-own Experiments Spot Training Distributed Training Automatic Model Tuning Debugger Model Hosting Multi-model Endpoints Model Monitor Pipelines Augmented AI AMAZON SAGEMAKER EDGE MANAGER SAGEMAKER STUDIO IDE AMAZON SAGEMAKER JUMPSTART VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD CONTACT CENTERS Deep Learning AMIs & Containers GPUs & CPUs Elastic Inference Trainium Inferentia FPGA AI SERVICES ML SERVICES FRAMEWORKS & INFRASTRUCTURE DeepGraphLibrary Amazon Rekognition Amazon Polly Amazon Transcribe +Medical Amazon Lex Amazon Personalize Amazon Forecast Amazon Comprehend +Medical Amazon Textract Amazon Kendra Amazon CodeGuru Amazon Fraud Detector Amazon Translate INDUSTRIAL AI CODE AND DEVOPS Amazon DevOps Guru Voice ID For Amazon Connect Contact Lens Amazon Monitron AWS Panorama + Appliance Amazon Lookout for Vision Amazon Lookout for Equipment Amazon HealthLake HEALTHCARE AI Amazon Lookout for Metrics ANOMOLY DETECTION Amazon Transcribe for Medical Amazon Comprehend for Medical 모든 개발자를 위한 다양한 인공 지능 도구 제공

Slide 10

Slide 10 text

Amazon SageMaker Label data Aggregate & prepare data Store & share features Auto ML Spark/R Detect bias Visualize in notebooks Pick algorithm Train models Tune parameters Debug & profile Deploy in production Manage & monitor CI/CD Human review Ground Truth Data Wrangler Feature store Autopilot Processing Clarify Studio Notebooks Built-in or Bring-your-own Experiments Spot Training Distributed Training Automatic Model Tuning Debugger Model Hosting Multi-model Endpoints Model Monitor Pipelines Augmented AI AMAZON SAGEMAKER EDGE MANAGER SAGEMAKER STUDIO IDE AMAZON SAGEMAKER JUMPSTART VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD CONTACT CENTERS Deep Learning AMIs & Containers GPUs & CPUs Elastic Inference Trainium Inferentia FPGA AI SERVICES ML SERVICES FRAMEWORKS & INFRASTRUCTURE DeepGraphLibrary Amazon Rekognition Amazon Polly Amazon Transcribe +Medical Amazon Lex Amazon Personalize Amazon Forecast Amazon Comprehend +Medical Amazon Textract Amazon Kendra Amazon CodeGuru Amazon Fraud Detector Amazon Translate INDUSTRIAL AI CODE AND DEVOPS Amazon DevOps Guru Voice ID For Amazon Connect Contact Lens Amazon Monitron AWS Panorama + Appliance Amazon Lookout for Vision Amazon Lookout for Equipment Amazon HealthLake HEALTHCARE AI Amazon Lookout for Metrics ANOMOLY DETECTION Amazon Transcribe for Medical Amazon Comprehend for Medical 모든 개발자를 위한 다양한 인공 지능 도구 제공

Slide 11

Slide 11 text

Amazon Rekognition 기계 학습을 통한 이미지 및 비디오 분석 자동화

Slide 12

Slide 12 text

Amazon Rekognition Image and Video CELEBRITY RECOGNITION FACE COMPARE & SEARCH PATHING FACE DETECTION AND ANALYSIS LIVE STREAM VIDEO TEXT CONTENT MODERATION CUSTOM LABELS OBJECT, SCENE, AND ACTIVITY VIDEO SEGMENTS PPE DETECTION

Slide 13

Slide 13 text

Architecture: Automatic Image Tagging Amazon OpenSearch Service Amazon Kinesis Data Streams Amazon Rekognition Images AWS Lambda Amazon S3 bucket with images Amazon API Gateway https://tinyurl.com/kxydd2wn Automatic Image Recognition

Slide 14

Slide 14 text

Demo: Automatic Image Tagging https://github.com/aws-samples/aws-realtime-image-analysis

Slide 15

Slide 15 text

1 Year Later

Slide 16

Slide 16 text

Visual Search Similar results bonzai Provence lavender Cozy places Top 10 cozy apartments in Brooklyn

Slide 17

Slide 17 text

Building a Visual Search Application https://aws.amazon.com/blogs/machine-learning/building-a-visual-search-application-with- amazon-sagemaker-and-amazon-es/

Slide 18

Slide 18 text

As-Is: Automatic Image Tagging Amazon Kinesis Data Streams Client AWS Lambda Amazon S3 bucket with images Amazon API Gateway Amazon OpenSearch Service Amazon Rekognition

Slide 19

Slide 19 text

To-Be: Visual Search Amazon Kinesis Data Streams Client AWS Lambda Amazon S3 bucket with images Amazon API Gateway Amazon OpenSearch Service

Slide 20

Slide 20 text

Machine Learning is like this Model Training Prediction Output New Data Labeled Data Cat

Slide 21

Slide 21 text

Typical Machine Learning Process Collect, prepare and label training data Choose and optimize ML algorithm Train and tune ML models Set up and manage environments for training Deploy models in production Scale and manage the production environment 1 2 3

Slide 22

Slide 22 text

Set up and track experiment Choose model Debug, compare, and evaluate experiments Monitor quality, detect drift, and retrain Share, review, and collaborate Machine Learning is iterative

Slide 23

Slide 23 text

Amazon SageMaker Label data Aggregate & prepare data Store & share features Auto ML Spark/R Detect bias Visualize in notebooks Pick algorithm Train models Tune parameters Debug & profile Deploy in production Manage & monitor CI/CD Human review Ground Truth Data Wrangler Feature store Autopilot Processing Clarify Studio Notebooks Built-in or Bring-your-own Experiments Spot Training Distributed Training Automatic Model Tuning Debugger Model Hosting Multi-model Endpoints Model Monitor Pipelines Augmented AI AMAZON SAGEMAKER EDGE MANAGER SAGEMAKER STUDIO IDE AMAZON SAGEMAKER JUMPSTART VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD CONTACT CENTERS Deep Learning AMIs & Containers GPUs & CPUs Elastic Inference Trainium Inferentia FPGA AI SERVICES ML SERVICES FRAMEWORKS & INFRASTRUCTURE DeepGraphLibrary Amazon Rekognition Amazon Polly Amazon Transcribe +Medical Amazon Lex Amazon Personalize Amazon Forecast Amazon Comprehend +Medical Amazon Textract Amazon Kendra Amazon CodeGuru Amazon Fraud Detector Amazon Translate INDUSTRIAL AI CODE AND DEVOPS Amazon DevOps Guru Voice ID For Amazon Connect Contact Lens Amazon Monitron AWS Panorama + Appliance Amazon Lookout for Vision Amazon Lookout for Equipment Amazon HealthLake HEALTHCARE AI Amazon Lookout for Metrics ANOMOLY DETECTION Amazon Transcribe for Medical Amazon Comprehend for Medical 모든 개발자를 위한 다양한 인공 지능 도구 제공

Slide 24

Slide 24 text

Amazon SageMaker Label data Aggregate & prepare data Store & share features Auto ML Spark/R Detect bias Visualize in notebooks Pick algorithm Train models Tune parameters Debug & profile Deploy in production Manage & monitor CI/CD Human review Ground Truth Data Wrangler Feature store Autopilot Processing Clarify Studio Notebooks Built-in or Bring-your-own Experiments Spot Training Distributed Training Automatic Model Tuning Debugger Model Hosting Multi-model Endpoints Model Monitor Pipelines Augmented AI AMAZON SAGEMAKER EDGE MANAGER SAGEMAKER STUDIO IDE AMAZON SAGEMAKER JUMPSTART VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD CONTACT CENTERS Deep Learning AMIs & Containers GPUs & CPUs Elastic Inference Trainium Inferentia FPGA AI SERVICES ML SERVICES FRAMEWORKS & INFRASTRUCTURE DeepGraphLibrary Amazon Rekognition Amazon Polly Amazon Transcribe +Medical Amazon Lex Amazon Personalize Amazon Forecast Amazon Comprehend +Medical Amazon Textract Amazon Kendra Amazon CodeGuru Amazon Fraud Detector Amazon Translate INDUSTRIAL AI CODE AND DEVOPS Amazon DevOps Guru Voice ID For Amazon Connect Contact Lens Amazon Monitron AWS Panorama + Appliance Amazon Lookout for Vision Amazon Lookout for Equipment Amazon HealthLake HEALTHCARE AI Amazon Lookout for Metrics ANOMOLY DETECTION Amazon Transcribe for Medical Amazon Comprehend for Medical 모든 개발자를 위한 다양한 인공 지능 도구 제공

Slide 25

Slide 25 text

End-to-End Machine Learning Platform Zero setup Flexible Model Training Pay by the second $ Amazon SageMaker 손쉬운 기계 학습 모델 생성, 훈련 및 서비스 배포 완전 관리 서비스

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

Amazon SageMaker Studio 기계 학습 모델 개발 및 배포를 위한 최초의 완전 통합 개발 환경 (IDE) Collaboration at scale 코드 의존성 추적 없이 확장 가능한 노트북 공유 Easy experiment management 수천 개의 모델 실험을 구성, 추적 및 비교 Automatic model generation 코드 작성 없이 데이터를 가지고 자동 모델 생성 Higher quality ML models 오류 자동 디버깅 및 실시간 오류 경보 모델 모니터링 및 고품질 유지 Increased productivity 완전 자동화된 머신 러닝 워크플로 구축

Slide 28

Slide 28 text

Amazon SageMaker Studio 기계 학습 모델 개발 및 배포를 위한 최초의 완전 통합 개발 환경 (IDE)

Slide 29

Slide 29 text

Amazon SageMaker Studio 시작 화면

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

Notebook Model Models bucket SageMaker Endpoint Train Amazon SageMaker End to End Training and Deployment

Slide 32

Slide 32 text

From Amazon SageMaker Notebooks training

Slide 33

Slide 33 text

Amazon SageMaker Training Specify Training Infrastructure

Slide 34

Slide 34 text

Amazon SageMaker Training Use Algorithm and Start training Execution Role SageMaker Estimator

Slide 35

Slide 35 text

Amazon SageMaker Training How does training happen XGBoost validation(optional) test(optional ECR S3 ML Instance ml.m4.xlarge xgboost linear-learner PCA DeepAR BlazingText Image classification … Object Detection Images S3 SageMaker Notebook SageMaker Training Job train Model

Slide 36

Slide 36 text

Model Deployment training

Slide 37

Slide 37 text

Model artifacts Amazon SageMaker Deployment Hosting Services Inference Image Training Image Training Data Model artifacts Endpoint Amazon SageMaker Amazon S3 Amazon ECR Inference Image

Slide 38

Slide 38 text

Amazon SageMaker Deployment SageMaker Endpoints (Private API) Auto Scaling group Availability Zone 1 Availability Zone 2 Availability Zone 3 Elastic Load Balancing Model Endpoint Client Deployment / Hosting Amazon SageMaker ML Compute Instances Input Data (Request) Prediction (Response)

Slide 39

Slide 39 text

Amazon SageMaker Deployment SageMaker Endpoints (Public API) Auto Scaling group Availability Zone 1 Availability Zone 2 Availability Zone 3 Elastic Load Balancing Model Endpoint Amazon API Gateway Client Deployment / Hosting Amazon SageMaker ML Compute Instances Input Data (Request) Prediction (Response)

Slide 40

Slide 40 text

Define Estimator Object created deploy() predict() Object created fit() Amazon SageMaker End to End Training and Deployment

Slide 41

Slide 41 text

Classification • Linear Learner • XGBoost • KNN Working with Text • BlazingText • Supervised • Unsupervised* Recommendation • Factorization Machines Forecasting • DeepAR Topic Modeling • LDA • NTM Amazon SageMaker에서 제공하는 Built-in Algorithms Sequence Translation • Seq2Seq* Clustering • KMeans Feature Reduction • PCA • Object2Vec Anomaly Detection • Random Cut Forests • IP Insights Computer Vision • Image Classification • Object Detection • Semantic Segmentation Regression • Linear Learner • XGBoost • KNN https://docs.aws.amazon.com/sagemaker/latest/dg/algos.html

Slide 42

Slide 42 text

Model Deployment: Built-in Algorithms xgboost linear-learner PCA DeepAR BlazingText Image classification … Object Detection Built-in Algorithm Images Elastic Container Registry

Slide 43

Slide 43 text

Model Deployment: Bring Your Own Model

Slide 44

Slide 44 text

BYOM Deployment: Model Artifacts (model.tar.gz) model.tar.gz/ |- model.pth |- code/ |- inference.py |- requirements.txt model.tar.gz/ |- model | |- [model_version_number] | |- variables | |- saved_model.pb |- code |- inference.py |- requirements.txt TensorFlow (>= 1.11) PyTorch (>=1.2.0)

Slide 45

Slide 45 text

def model_fn(model_dir): # Loads a model for inference model = Your_Model() return model def input_fn(input_data, content_type): # Deserializes the input data return decoder.decode(input_data, content_type) def predict_fn(input_data, model): # Calls a model on data deseralized by input_fn return model(input_data) def output_fn(prediction, content_type): # Serializes predictions from predict_fn return encoder.encode(prediction, content_type) https://github.com/aws/sagemaker-inference-toolkit BYOM Deployment: Inference Handler Script https://github.com/tensorflow/serving def input_handler(data, context): # Pre-process request input before it is sent to TensorFlow Serving REST API if context.request_content_type == 'application/json': pass if context.request_content_type == 'text/csv': pass def output_handler(response, context): # Post-process TensorFlow Serving output before it is returned to the client response_content_type = context.accept_header prediction = response.content return prediction, response_content_type

Slide 46

Slide 46 text

A fully managed service that enables data scientists and developers to quickly and easily build machine-learning based models into production smart applications. Amazon SageMaker

Slide 47

Slide 47 text

Amazon SageMaker Studio 기계 학습 모델 개발 및 배포를 위한 최초의 완전 통합 개발 환경 (IDE) 학습 모델 구축 및 협업 SageMaker Notebooks SageMaker Pipelines 완전 자동화된 머신 러닝 워크플로 구축 학습 모델 훈련 및 검증 SageMaker Training Job One-click 배포 , 모델 모니터링 및 고품질 유지 SageMaker Endpoints 학습 모델 최적화 및 다중 알고리즘 튜닝 SageMaker HPO

Slide 48

Slide 48 text

Let’s build a Machine Learning powered Application

Slide 49

Slide 49 text

Building a Visual Search Application https://aws.amazon.com/blogs/machine-learning/building-a-visual-search-application-with- amazon-sagemaker-and-amazon-es/

Slide 50

Slide 50 text

Visual Search Amazon Kinesis Data Streams Client AWS Lambda Amazon S3 bucket with images Amazon API Gateway Amazon SageMaker Amazon OpenSearch Service Notebook Model Models bucket SageMaker Endpoint Train

Slide 51

Slide 51 text

Demo: Visual Search Application https://tinyurl.com/53yubfzw

Slide 52

Slide 52 text

Demo: Visual Search Application https://tinyurl.com/53yubfzw

Slide 53

Slide 53 text

Demo: Visual Search Application https://tinyurl.com/53yubfzw

Slide 54

Slide 54 text

Demo: Visual Search Application https://tinyurl.com/53yubfzw

Slide 55

Slide 55 text

Your Dream Comes True

Slide 56

Slide 56 text

문제 해결에 가장 적합한 AI 서비스 선택 문제 해결을 위한 ML 모델 개발 ML 모델 배포 및 모니터링 서비스 개발 문제 정의 AI 서비스 활용 Business Problem – ML problem framing SageMaker 활용 서비스 출시 How to choose the most relevant advertisement? How to tag millions of images in fast, cost-efficient way? How to build our own ML model? How to customize labels? Wrap-up Our Business Problem

Slide 57

Slide 57 text

AI/ML Stack을 활용한 비즈니스 문제 해결 과정 문제 해결에 가장 적합한 AI 서비스 선택 문제 해결을 위한 ML 모델 개발 ML 모델 배포 및 모니터링 서비스 개발 문제 정의 AI 서비스 활용 Business Problem – ML problem framing SageMaker 활용 서비스 출시

Slide 58

Slide 58 text

Put machine learning in the hands of every developer Our mission at

Slide 59

Slide 59 text

Amazon SageMaker Label data Aggregate & prepare data Store & share features Auto ML Spark/R Detect bias Visualize in notebooks Pick algorithm Train models Tune parameters Debug & profile Deploy in production Manage & monitor CI/CD Human review Ground Truth Data Wrangler Feature store Autopilot Processing Clarify Studio Notebooks Built-in or Bring-your-own Experiments Spot Training Distributed Training Automatic Model Tuning Debugger Model Hosting Multi-model Endpoints Model Monitor Pipelines Augmented AI AMAZON SAGEMAKER EDGE MANAGER SAGEMAKER STUDIO IDE AMAZON SAGEMAKER JUMPSTART VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD CONTACT CENTERS Deep Learning AMIs & Containers GPUs & CPUs Elastic Inference Trainium Inferentia FPGA AI SERVICES ML SERVICES FRAMEWORKS & INFRASTRUCTURE DeepGraphLibrary Amazon Rekognition Amazon Polly Amazon Transcribe +Medical Amazon Lex Amazon Personalize Amazon Forecast Amazon Comprehend +Medical Amazon Textract Amazon Kendra Amazon CodeGuru Amazon Fraud Detector Amazon Translate INDUSTRIAL AI CODE AND DEVOPS Amazon DevOps Guru Voice ID For Amazon Connect Contact Lens Amazon Monitron AWS Panorama + Appliance Amazon Lookout for Vision Amazon Lookout for Equipment Amazon HealthLake HEALTHCARE AI Amazon Lookout for Metrics ANOMOLY DETECTION Amazon Transcribe for Medical Amazon Comprehend for Medical 모든 개발자를 위한 다양한 인공 지능 도구 제공

Slide 60

Slide 60 text

References • Image Insights - Automatic Image Tagging & Analysis System: https://tinyurl.com/kxydd2wn • Visual Image Search with Amazon SageMaker and Amazon ES: https://tinyurl.com/53yubfzw • Amazon Rekognition으로 나만의 컴퓨터 비전 모델 만들기: https://youtu.be/tioJru6b_4M • SageMaker Immersion Day Workshop ✯✯✯ : https://sagemaker-immersionday.workshop.aws/ • SageMaker Examples (100+) ✯✯✯ : https://sagemaker-examples.readthedocs.io/ • Advanced SageMaker Workshop: https://sagemaker-workshop.com/ • Amazon SageMaker 데모 (2020-03-25) ✯✯✯ : https://youtu.be/miIVGlq6OUk • How Startups Deploy Pretrained Models on Amazon SageMaker: https://tinyurl.com/4nwy9bd5 • Deploy trained Keras or TensorFlow models using Amazon SageMaker: https://tinyurl.com/nf2rwrw9 • Host multiple TensorFlow computer vision models using Amazon SageMaker multi-model endpoints: https://tinyurl.com/6w8f4862