独自設計チップ AWS Inferentia と AWS Trainium による機械学習の高速化とコスト最適化 / AWS Innovate AWS Inferentia Trainium

Slide 1

Slide 1 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • • AWS does not offer binding price quotes. AWS pricing is publicly available and is subject to change in accordance with the AWS Customer Agreement available at http://aws.amazon.com/agreement/. Any pricing information included in this document is provided only as an estimate of usage charges for AWS services based on certain information that you have provided. Monthly charges will be based on your actual use of AWS services, and may vary from the estimates provided.

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Ice Lake CPU Cascade Lake CPU Habana accelerator EPYC CPU A100, A10G, T4G GPUs Graviton CPU Inferentia Chip Trainium Chip UltraScale+ FPGA C7g C6g C6i C5a M6g M6i M6a R6g R6i R5a F1 Inf1 G5g G5 P4 DL1 Trn1 Elastic Inference

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Slide 18

Slide 18 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • AWS • 4 Neuron / • 128 TOPS (2,000 TOPS @24xlarge) • 2 • 8GB DRAM • FP16, BF16, INT8 • FP32 BF16 • https://aws.amazon.com/machine-learning/inferentia/

Slide 19

Slide 19 text

Slide 20

Slide 20 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. $0.000 $0.300 $0.600 $0.900 G4dn.xl G5.xl Inf1.xl Yolov5 $0.000 $0.025 $0.050 $0.075 G4dn.xl G5.xl Inf1.xl Resnet50 $0.000 $0.100 $0.200 $0.300 G4dn.xl G5.xl Inf1.xl Bert-Base -49% -68% Bert-Base Yolov5 Resnet50 -42%

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • • • • • • AWS Deep Learning Containers AWS Deep Learning AMIs Amazon SageMaker AWS Elastic Kubernetes Service Amazon Elastic Container Service

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. ELMo (2018) BERT-Large (2018) GPT-2 (2019) Turing NLG (2020) GPT-3 (2020) Switch-C (2021) … 100B 1B 1T 10T 10B 100M G R O W T H I N M O D E L C O M P L E X I T Y ( # O F P A R A M E T E R S ) 1. 2. 3. ML

Slide 31

Slide 31 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. T H E M O S T C O S T - E F F I C I E N T D L I N S T A N C E I N T H E C L O U D B F 1 6 / F P 1 6 F P 3 2 840 TFLOPS T F 3 2 3.4 PFLOPS 3.4 PFLOPS T R A N S I S T O R S P E R C H I P 55,000,000,000 Trn1 3 GHz 512 GB 13.1 TB/sec 768 GB/sec 800 Gbps EFA

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. FP32 P R E C I S I O N R A N G E S TF32 BF16 FP16 cFP8 UINT8 0 1 2 3 4 BF16/FP16 TF32 FP32 Normalized Performance P3dn P4d Trn1 NLP/DLRM Computer vision >5x >2.5x 1.4x

Slide 35

Slide 35 text

Slide 36

Slide 36 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Petabits/s throughput, billions of IOPS Trn1 10K+ Trainium Chips Trn1 Trn1 Trn1 Trn1 Trn1 Trn1 Trn1 EC2 UltraClusters Petabit non- blocking TOR E C 2 U L T R A C L U S T E R 1 T R A I N I U M

Slide 37

Slide 37 text

Slide 38

Slide 38 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. 0 0.4 0.8 1.2 1.6 16 32 64 128 256 512 1024 Relative performance Batch size Trn1.32xl P4d.24xl S T R O N G S C A L I N G Trainium Compute Compute Comm Computation Communication Time

Slide 39

Slide 39 text

Slide 40

Slide 40 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon FSx for Lustre Amazon S3 EC2 Trn1 UltraCluster Amazon EC2 Trn1 Amazon SageMaker AWS Deep Learning AMIs Amazon EKS Amazon ECS AWS Deep Learning Containers Elastic Fabric Adapter Amazon EBS Amazon EFS Pytorch TensorFlow

Slide 41

Slide 41 text

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Slide 44

Slide 44 text

Slide 45

Slide 45 text

Slide 46

Slide 46 text

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • AWS Inferentia • https://aws.amazon.com/jp/machine-learning/inferentia/ • Amazon EC2 Inf1 • https://aws.amazon.com/ec2/instance-types/inf1/ • AWS Trainium • https://aws.amazon.com/jp/machine-learning/trainium/ • Amazon EC2 Trn1 • https://aws.amazon.com/ec2/instance-types/trn1/ • Amazon EC2 Trn1 URL • https://pages.awscloud.com/EC2-Trn1-Preview.html