独自設計チップ AWS Inferentia と AWS Trainium による機械学習の高速化とコスト最適化 / AWS Innovate AWS Inferentia Trainium

2022年 2 月 24 日開催、 AWS Innovate – AI/ML Editionより

AWS では、広く利用され歴史もある NVIDIA GPU を搭載したインスタンスを提供する一方、より高い性能、より高いコストパフォーマンスを持つ幅広い選択肢を提供するため、機械学習ワークロード向けの独自設計カスタムチップ、AWS Inferentia 及びAWS Trainium を開発してきました。このセッションでは、AWS Inferentia を搭載した ML 推論向け Inf1インスタンス、AWS Trainium を搭載した ML学習向け Trn1インスタンス、それぞれの概要と活用方法についてご紹介します。

動画視聴はこちらから
https://resources.awscloud.com/aws-ai-and-machine-learning-japan-aws-innovate/accelerate-machine-learning-and-cost-optimization-with-proprietary-chips-aws-inferentia-and-aws-trainium-jp-aws-innovate

Transcript

1. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • • AWS does not offer binding price quotes. AWS pricing is publicly available and is subject to change in accordance with the AWS Customer Agreement available at http://aws.amazon.com/agreement/. Any pricing information included in this document is provided only as an estimate of usage charges for AWS services based on certain information that you have provided. Monthly charges will be based on your actual use of AWS services, and may vary from the estimates provided.
2. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. 2 4 F e b r u a r y 2 0 2 2
3. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
4. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
5. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • •
6. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • • • •
7. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
8. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Nitro System , , , SSD, AWS Inferentia AWS Trainium AWS Graviton
9. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS
10. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
11. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. c6gn.8xlarge ファミリー 世代 機能 サイズ
12. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
13. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Ice Lake CPU Cascade Lake CPU Habana accelerator EPYC CPU A100, A10G, T4G GPUs Graviton CPU Inferentia Chip Trainium Chip UltraScale+ FPGA C7g C6g C6i C5a M6g M6i M6a R6g R6i R5a F1 Inf1 G5g G5 P4 DL1 Trn1 Elastic Inference
14. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Ice Lake CPU Cascade Lake CPU Habana accelerator EPYC CPU A100, A10G, T4G GPUs Graviton CPU Inferentia Chip Trainium Chip UltraScale+ FPGA C7g C6g C6i C5a M6g M6i M6a R6g R6i R5a F1 Inf1 G5g G5 P4 DL1 Trn1 Elastic Inference
15. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
16. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • AWS AWS Inferentia • • GPU 2.3 70% • (TensorFlow, PyTorch, MXNet) https://aws.amazon.com/ec2/instance-types/inf1/
17. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • 4 • 1~16 Inferentia • 6xlarge 24xlarge Inferentia • 100Gbps • 2022 1 23 • EC2 • Savings Plan *2022年1月時点の米国東部 (バージニア北部)の価格
18. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • AWS • 4 Neuron / • 128 TOPS (2,000 TOPS @24xlarge) • 2 • 8GB DRAM • FP16, BF16, INT8 • FP32 BF16 • https://aws.amazon.com/machine-learning/inferentia/
19. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • •
20. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. $0.000 $0.300 $0.600 $0.900 G4dn.xl G5.xl Inf1.xl Yolov5 $0.000 $0.025 $0.050 $0.075 G4dn.xl G5.xl Inf1.xl Resnet50 $0.000 $0.100 $0.200 $0.300 G4dn.xl G5.xl Inf1.xl Bert-Base -49% -68% Bert-Base Yolov5 Resnet50 -42%
21. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. https://awsdocs-neuron.readthedocs-hosted.com/ https://github.com/aws/aws-neuron-sdk
22. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
23. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. •
24. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • • • • • AWS Deep Learning Containers AWS Deep Learning AMIs Amazon SageMaker AWS Elastic Kubernetes Service Amazon Elastic Container Service
25. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • •
26. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. 0 40 80 120 160 0.00 0.30 0.60 0.90 1.20 G4dn Inf1
27. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. https://aws.amazon.com/ec2/instance-types/inf1/#Customer_Testimonials Hotpot.ai Amazon Rekognition
28. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AMAZON ALEXA © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved.
29. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
30. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. ELMo (2018) BERT-Large (2018) GPT-2 (2019) Turing NLG (2020) GPT-3 (2020) Switch-C (2021) … 100B 1B 1T 10T 10B 100M G R O W T H I N M O D E L C O M P L E X I T Y ( # O F P A R A M E T E R S ) 1. 2. 3. ML
31. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. T H E M O S T C O S T - E F F I C I E N T D L I N S T A N C E I N T H E C L O U D B F 1 6 / F P 1 6 F P 3 2 840 TFLOPS T F 3 2 3.4 PFLOPS 3.4 PFLOPS T R A N S I S T O R S P E R C H I P 55,000,000,000 Trn1 3 GHz 512 GB 13.1 TB/sec 768 GB/sec 800 Gbps EFA
32. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. T H E M O S T C O S T - E F F I C I E N T D L I N S T A N C E I N T H E C L O U D • • • • •
33. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • • • • • • • • • https://aws.amazon.com/machine-learning/trainium/ Collective compute Neuron Neuron
34. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. FP32 P R E C I S I O N R A N G E S TF32 BF16 FP16 cFP8 UINT8 0 1 2 3 4 BF16/FP16 TF32 FP32 Normalized Performance P3dn P4d Trn1 NLP/DLRM Computer vision >5x >2.5x 1.4x
35. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • • • • https://arxiv.org/pdf/1502.02551.pdf
36. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Petabits/s throughput, billions of IOPS Trn1 10K+ Trainium Chips Trn1 Trn1 Trn1 Trn1 Trn1 Trn1 Trn1 EC2 UltraClusters Petabit non- blocking TOR E C 2 U L T R A C L U S T E R 1 T R A I N I U M
37. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. P3dn 256 GB 320 GB 512 GB P4d Trn1 P3dn 300 GB/s 600 GB/s 768 GB/s P4d Trn1 P3dn 100 Gb/s 400 Gb/s 800 Gb/s P4d Trn1
38. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. 0 0.4 0.8 1.2 1.6 16 32 64 128 256 512 1024 Relative performance Batch size Trn1.32xl P4d.24xl S T R O N G S C A L I N G Trainium Compute Compute Comm Computation Communication Time
39. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS
40. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon FSx for Lustre Amazon S3 EC2 Trn1 UltraCluster Amazon EC2 Trn1 Amazon SageMaker AWS Deep Learning AMIs Amazon EKS Amazon ECS AWS Deep Learning Containers Elastic Fabric Adapter Amazon EBS Amazon EFS Pytorch TensorFlow
41. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
42. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
43. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • • • • • • •
44. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. Hiroshi Tokoyo htokoyo@amzon.co.jp
45. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Appendix
46. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • AWS Inferentia • https://aws.amazon.com/jp/machine-learning/inferentia/ • Amazon EC2 Inf1 • https://aws.amazon.com/ec2/instance-types/inf1/ • AWS Trainium • https://aws.amazon.com/jp/machine-learning/trainium/ • Amazon EC2 Trn1 • https://aws.amazon.com/ec2/instance-types/trn1/ • Amazon EC2 Trn1 URL • https://pages.awscloud.com/EC2-Trn1-Preview.html
47. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • • • • •
48. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • • • • • •
49. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • • • • • •
50. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • • • • • •
51. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • • • • • •