Share
Embed
メルカリ写真検索における Amazon EKS の活用事例と プロダクトにおけるEdgeAI technologyの展望
AWSLoftで行われた AWS Containers talk with Mercari で使用した資料です
https://awscontainertalkwithmercari.splashthat.com/
Hirofumi Nakagawa/中河 宏文
August 30, 2019
Transcript
43 slides
-
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. ϝϧΧϦࣸਅݕࡧʹ͓͚Δ Amazon EKS ͷ׆༻ࣄྫͱ ϓϩμΫτʹ͓͚ΔEdgeAI technologyͷల !1 גࣜձࣾϝϧΧϦ தՏ จ -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. தՏ จ • 20177݄ೖࣾ • ॴଐSRE→AI/MLνʔϜ • σόΠευϥΠό։ൃ͔ΒϑϩϯτΤϯυ ։ൃ·ͰΔԿͰ Twitter: hnakagawa14 GitHub: hnakagawa !2 ࣗݾհ -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Introduction !3 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. What is ࣸਅݕࡧ • ࣸਅݕࡧͱɺ͍ΘΏΔը૾ݕࡧػೳ • ΞϓϦ͔ΒࣸਅΛݩʹΛݕࡧ͢Δ • ໊ΛΒͳͯ͘ը૾͔ΒΛݕࡧ Ͱ͖Δ !4 ಈըϦϯΫ: https://youtu.be/kTni8EvOCgI -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. جຊతͳࣸਅݕࡧͷΈ !5 Deep Neural Networks (DNN)Λ༻ͯ͠ը૾ ͔ΒಛϕΫτϧΛऔಘ औಘͨ͠ಛϕΫτϧΛ Approximate Nearest Neighbor Index(ANN Index) ʹՃͯ͠ը૾indexΛߏங ݕࡧ࣌ʹಉ͘͡ը૾͔Β DNNΛհͯ͠ಛϕΫτϧΛऔ ಘ͠ɺANN Index͔Βݕࡧ 2 3 1 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. What is Kubernetes • KubernetesʢҎԼk8sʣͱΦʔϓϯιʔε ͷίϯςφɾΦʔέετϨʔγϣϯγες Ϝ • k8sʹCustom Resource Definitionͱݺ ΕΔಠࣗͷϦιʔεΛఆٛͰ͖Δػೳ͕͋ Γɺ։ൃऀͦͷػೳΛհͯ͠k8sͷػೳΛ ֦ுͰ͖Δ • Amazon Elastic Container Service for Kubernetes (Amazon EKS) ͱk8sͷϚω ʔδυɾαʔϏεɺίϯτϩʔϧϓϨʔϯ ͷཧΛߦͬͯ͘ΕΔ !6 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. What is Custom Resource Definition • Custom Resource DefinitionʢҎԼCRDʣͱ ಠࣗʹϦιʔεΛఆٛͰ͖Δk8sͷػೳ • CRDɾϦιʔεͱɺΧελϜɾίϯτϩʔ ϥͰߏ͞ΕΔ • ΧελϜɾίϯτϩʔϥ͕CRDɾϦιʔε ͷϥΠϑαΠΫϧ/ঢ়ଶʹԠͯ͡Ϋϥελͷ ঢ়ଶΛίϯτϩʔϧ͢Δ !7 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. ML Platform Lykeion ࣸਅݕࡧLykeionͱݺΕΔͷML Platform্ ʹߏங͞Ε͓ͯΓɺԼهͷػೳPlatformଆͷػೳ Λ༻͍ͯ͠Δ !8 • Training/Serving CRD & ΧελϜίϯτϩʔϥ • ίϯςφϕʔεɾύΠϓϥΠϯ • Training/Serving ίϯςφΠϝʔδɾϏϧμʔ • ϞσϧɾϨϙδτϦ -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Architecture !9 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Architecture֓ཁਤ !10 S3 EKS -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 1.TrainingɾϦιʔεͷ࡞ !11 S3 EKS -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. TrainingɾϦιʔεͷ࡞ • Training custom resourceΛCronJob͕࡞ • ΧελϜɾίϯτϩʔϥ͕CRDɾϦιʔε Ͱઃఆ͞ΕͨίϯςφϕʔεɾύΠϓϥΠ ϯΛ࣮ߦ • ࣮ߦ͢Δόον୯Ґͱͯ͠Hourly, Daily, Monthly͕ଘࡏ !12 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. ίϯςφϕʔεɾύΠϓϥΠϯ • ֤ఔΛݸผͷίϯςφɾΠϝʔδͰ࣮ߦ • ϥΠϒϥϦͷґଘؔͳͲڥφΠʔϒͳMLύΠϓϥΠϯͷΛղܾ • ύΠϓϥΠϯDAGYAMLͰهड़ • ֤ఔͷೖग़ྗPersistent VolumeʢҎԼPVʣΛհ͢ !13 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Batch Execution as Custom Resource • શͯͷόον࣮ߦใ͕CRDɾϦιʔεͱͯ͠ k8s্ʹΔ • ಉ͡ॲཧΛ࠶࣮ߦग़དྷΔͨΊɺόονͷ࠶࣮ߦ Λ͏ো෮چ࡞ۀ͕༰қ !14 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 2.ը૾ͷμϯϩʔυ !15 S3 EKS -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. ը૾ͷμϯϩʔυ • Amazon S3্ʹଘࡏ͢ΔϝϧΧϦɾΠϝʔδετΞ͔Βը૾Λμ ϯϩʔυ • ύΠϓϥΠϯ্ͬͱ͕͔͔࣌ؒΔఔʢը૾͕େͳͨΊ) • ͦͷͨΊPVʹҰఆظؒΩϟογϡ͢ΔࣄʹΑͬͯ࠶ΠϯσοΫε͕ ඞཁͳ࣌ʹૉૣ͘ύΠϓϥΠϯΛճͤΔΑ͏ʹ͍ͯ͠Δ • PVͷ࣮ମ Amazon EBS !16 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 3.ΞηοτͷΞοϓϩʔυ !17 S3 EKS -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. ΞηοτͷΞοϓϩʔυ • ύΠϓϥΠϯͷՌͰ͋ΔಛϕΫτϧͱANN IndexΛϞσϧɾϨϙδτϦʹอଘ • શͯͷՌόʔδϣϯཧ͞Εͨঢ়ଶͰอଘ͞ΕΔ • ϞσϧɾϨϙδτϦGCS্ʹߏங !18 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 4.ServingΠϝʔδͷϏϧυ !19 S3 EKS -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. ServingΠϝʔδͷϏϧυ 1. ϞσϧɾϨϙδτϦΛImage BuilderͱݺΕΔdaemon͕ࢹ 2. ৽͍͠Serving͖͢Ϧιʔε͕Ճ͞ΕΔͱࣗಈͰServingίϯςφɾΠϝʔδΛϏϧυ • ίϯςφɾΠϝʔδશͯͷANN IndexͷαʔϏϯάʹඞཁͳϦιʔεΛશؚͯΜͰ͍Δ 3. Ϗϧυ͞ΕͨίϯςφɾΠϝʔδΛίϯςφɾϨδετϦʹϓογϡ !20 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 5.ServingɾϦιʔεͷ࡞ !21 S3 EKS -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. ServingɾϦιʔεͷ࡞ • Image BuilderίϯςφɾΠϝʔδΛϏ ϧυͨ͋͠ͱɺServingΧελϜɾϦιʔ εΛ࡞ • ServingΧελϜɾίϯτϩʔϥCRDɾ ϦιʔεͷઃఆΛݩʹඞཁͳ DeploymentɺServiceΛ࡞ • ຊγεςϜͰߏங͞ΕͨANN IndexΛ ݸผͷIndexαʔϏεͱͯ͠σϓϩΠ !22 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 6.αʔϏεɾσΟεΧόϦ !23 S3 EKS -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. αʔϏεɾσΟεΧόϦ • Ϋϥελ্ʹଘࡏ͢ΔIndexαʔϏεΛ k8sΛհͯࣗ͠ಈతʹऔಘ͢Δ • ͳΔ͘େ͖ͳཻͷIndexΛ༻͢ΔΑ ͏ɺҟͳΔظؒɾཻͷIndexαʔϏε (Hourly, Daily, Monthly) ΛࣗಈతʹΈ߹ ΘͤΔ • REST <-> IndexαʔϏεؒͷϓϩτίϧ gRPCΛ༻ !24 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. ֓ཁਤͷৼΓฦΓ !25 S3 EKS -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Conclusion !26 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. ࣸਅݕࡧͷόοΫΤϯυɾΠϯϑϥ 1. ίϯςφɾϕʔεͷ࠶ݱੑͷߴ͍γεςϜ 2. k8sͷCRD/ΧελϜɾίϯτϩʔϥαʔϏεɾσΟεΧόϦͷػೳΛ׆༻ 3. Batch Execution as Custom ResourceɺML PlatformͰ࣮ݱ͞Ε͍ͯΔػೳΛ༻ ͠ɺϩόετͳγεςϜΛߏங 4. ΫϥυɾΠϯϑϥΛk8sͰநԽ͢ΔࣄʹΑͬͯɺ֤Ϋϥυɾϕϯμͷྑ͍ͱ͜औΓ Λ͍ͯ͠Δ !27 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Amazon EKSʹ͍ͭͯ • k8sͱ͍͑GCPͱ͍͏Πϝʔδ͕͋Δ͕(։ൃݩ͔ͩΒવ)ɺEKS͜ͳΕ͖ͯͯ҆ఆͨ͠ӡ ༻͕ग़དྷΔΑ͏ʹͳͬͨɻ͔͠͠Pros/Consવଘࡏ͢Δɻ • Pros • ૉͷk8sʹۙ͘ɺͦ͏͍ͬͨҙຯͰॊೈͳߏΛΈ͍͢ • Cons • ଞࣾͷϚωʔδυk8sαʔϏεʹൺɺk8sʹ͍ͭͯඞཁͳࣄલ͕ࣝଟ͍ • ͔͍͠͠উखͷྑ͍AWSͷS3RDSͷଞαʔϏεΛɺk8sͱڞʹ͍͍ͨϢʔβ͕᪳͢ Δཧ༝ͳ͍ !28 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. AWSͷྑ͍ͱ͜Ζ • AWSͷྑ͍ॴԿͱݴͬͯRDSɺAuroraʹද͞ΕΔɺϚωʔδυɾRDBMSαʔϏε • ࠷ॳRDSͰ࡞ͬͯɺύϑΥʔϚϯεʹࠔͬͨΒAuroraΛ͏ • Persistent͕ॏཁͳɺಛʹRDBΛ༻͢ΔγεςϜͰɺ࠷ॳͷબࢶʹͳΔ • ࠓճͷߏྫͱগ͠ҧ͏͕ɺk8s(EKS)ͰϏδωεɾϩδοΫ͕ͷΔΠϯϑϥ෦Λ நԽ͠PersistentʹRDSͷαʔϏεΛ༻͢Δͷ͓͢͢Ίߏͷ̍ͭ !29 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Next Future? !30 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Realtime image search • ॴҦ Edge AI TechnologyΛ༻ͯ͠ɺݕࡧʹඞཁͳਪ ॲཧͷେ෦ΛEdgeଆͰߦ͍ͬͯΔ • ϦΞϧλΠϜͳΠϯλϥΫγϣϯΛ࣮ݱ • UX্େ͖ͳϝϦοτ͕༗Δ !31 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Listing Dispacher • దͳग़ϝιουΛαδΣετͯ͘͠ΕΔ • ෳࡶͳग़ϑϩʔΛ؆ུԽ • ࠷ऴతʹ͔͚ͩ͟͢Ͱग़͕ྃ͢ΔॴΛࢦ͢!! !32 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. What must happen to make DNN work on edge • ༷ʑͳτϨʔυΦϑ͕ଘࡏ͢Δ • Accuracy • Latency • Energy consumption • Model size • తͷUXΛୡ͢ΔͨΊʹɺΞϧΰϦ ζϜɺΤϯδχΞϦϯά྆ํͰͦΕΒͷ όϥϯεΛߟྀ͢Δඞཁ͕͋Δ !33 Image credit: [1] Image credit: [1] Image credit: [2] ɾΦϖϨʔγϣϯʹΑͬͯίετ͕ҧ͏[1] ɾmobile deviceͰαϙʔτ͍ͯ͠ΔGPUΠϯλʔϑΣʔεͷγΣΞ[2]ɹ -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Landscape of execution environment Image credit: [2] Image credit: [2] ※ FacebookͷϨϙʔτ͔ΒͷҾ༻[2] ԿʹతͷUXΛ࣮ݱͰ͖ΔσόΠεͷΧόϨοδΛ͛Δ͔? !34 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Designing efficient networks: Manual efforts - Mobile Nets V1, V2 and V3([4], [5], & [6]) • Depthwise separable conv Λ༻͠ܭࢉྔΛ ݮ • Inverted residual with linear bottleneck Λ༻͠ ϝϞϦΞΫηεྔΛ ݮ Image credit: [4] Image credit: [5] Image credit: [6] • ׆ੑԽؔʹh-swishΛ༻ • squeeze & excitationΛ ༻ͨ͠channelͷAttention • ܰྔͳfinal blockͷ࠾༻ etc... !35 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Designing efficient networks: Automated ways - ௨ৗͷϞσϧͷτϨʔχϯά ΛτϨʔχϯάɾύϥϝʔλͱͨ͠Ϟσϧ Λೖྗɺ Λग़ྗͱͯ͠ɺ Λ࠷খԽ͢Δ ୳ࡧ͢Δ - ΞʔΩςΫνϟɾαʔνͰͷ୳ࡧ&τϨʔχϯά ΞʔΩςΫνϟɾύϥϝʔλ Ճ Λ࠷খԽ͢Δ Λ୳ࡧ͢Δ ͱ !36 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Two influential yet costly approaches !37 MnasNet[7] (RL-Based) FBNet[8] (Differentiable) • ୳ࡧۭ͔ؒΒઍͷmodelΛsampling͢Δ • sample͞Εͨchild modelΛεΫϥον͔ ΒτϨʔχϯά͢Δ • ڊେͳ୳ࡧۭ͔ؒΒ୳ࡧͰ͖Δ͕ɺݱ࣮త ͳΠςϨʔγϣϯΛߦ͏ҝʹɺڊେͳܭࢉ ػϦιʔε͕ඞཁʹͳΔ • DARTSϕʔεͷ୳ࡧख๏Λ࠾༻͍ͯ͠Δ • ୳ࡧۭؒͷ֤ΦϖϨʔγϣϯΛGPUϝϞϦ ʹͤΔඞཁ͕͋Δҝɺ݁ہGPUϝϞϦͷ ༻ྔ͕ͱͳΓɺsample͞Εͨproxy dataset͕ඞཁʹͳͬͨΓɺbatch sizeΛ্͛ ΒΕͳ͔ͬͨΓ͢Δ Image credit: [7] Image credit: [8] -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Our approach !38 Single-Path NAS[9] Device SoC Generation (Snapdragon) Model ImageNet Top-1 Accuracy* Latency (ms)* A 845 SPNAS 74.48 77.90 A 845 MobileNetV2 71.80 76.36 B 808 SPNAS 73.07 113.92 B 808 MobileNetV2 71.80 162.82 C 670 SPNAS 73.15 92.14 C 670 MobileNetV2 71.80 111.85 D 801 SPNAS 71.93 84.65 D 801 MobileNetV2 71.80 120.82 Image credit: [9] * All results are for float32 • superkenelͱ͍͏୳ࡧۭؒͷઃఆ ख๏ͰɺඦʙઍGPU͔͔࣌ؒ Δ୳ࡧ࣌ؒΛݮ͢Δ͜ͱ͕Ͱ͖Δ • MobileNet-V2ͱSingle-Path NAS(SPNAS)Ͱੜͨ͠Ϟσϧͱ ͷੑೳൺֱ MobileNet-V3Λϕʔεʹ୳ࡧۭؒΛઃఆ͠ɺSPNASͷϦʔζφϒϧͳNASख๏Ͱ୳ࡧ Λߦ͏ͷ͕ݱ࣮త? -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Edge function architecture !39 39 Server side Client side • αʔόଆͷDNN frameworkʹ TensorFlowΛ༻ • LykeionʹΑͬͯύΠϓϥΠϯΛ ߏங • ΫϥΠΞϯτଆͰTensorFlow Lite + MediaPipeΛ༻ • MediaPipeΛ༻͢Δ͜ͱͰલॲ ཧޙॲཧSIMDΛ༻ͯ͠ ޮԽͰ͖Δ (TF Lite) (Optional) -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Conclusion Again !40 -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Edge AI Technology !41 41 • ϦΞϧλΠϜͳΠϯλϥΫγϣϯΛ࣮ݱ͠UX্େ͖ͳϝϦοτ͕༗Δ • ͔͠͠ModelRuntimeɺ͞ΒʹBackendͱߟྀ͖͢ࣄฑ͕ଟ͘ͳΔͷࣄ࣮ • తͷUXΛ࣮ݱ͢ΔͨΊʹɺ AccuracyLatencyͷόϥϯεΛऔΔඞཁ͕͋Δ • ඞͣ͠Accuracy͕࠷༏ઌͰͳ͍ • ࠓޙҰൠԽ͞ΕΔաఔͰɺҾ͖ଓ͖RuntimeModelingख๏ͷٸܹͳਐԽ͕༧͞ΕΔ -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. References !42 [1] Lai, Liangzhen, Naveen Suda, and Vikas Chandra. "Not all ops are created equal!." arXiv preprint arXiv:1801.04326 (2018). [2] Wu, Carole-Jean, David Brooks, Kevin Chen, Douglas Chen, Sy Choudhury, Marat Dukhan, Kim Hazelwood et al. "Machine learning at facebook: Understanding inference at the edge." In 2019 IEEE International Symposium on High Performance Computer Architecture (HPCA), pp. 331-344. IEEE, 2019. [3] Howard, Andrew G., Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. "Mobilenets: Efficient convolutional neural networks for mobile vision applications." arXiv preprint arXiv:1704.04861 (2017). [4] Sandler, Mark, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. "Mobilenetv2: Inverted residuals and linear bottlenecks." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510-4520. 2018. [5] Howard, Andrew, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang et al. "Searching for mobilenetv3." arXiv preprint arXiv:1905.02244 (2019). [6] Zoph, Barret, and Quoc V. Le. "Neural architecture search with reinforcement learning." arXiv preprint arXiv:1611.01578 (2016). [7] Tan, Mingxing, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc V. Le. "Mnasnet: Platform- aware neural architecture search for mobile." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2820-2828. 2019. [8] Wu, Bichen, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer. "Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10734-10742. 2019. [9] Stamoulis, Dimitrios, Ruizhou Ding, Di Wang, Dimitrios Lymberopoulos, Bodhi Priyantha, Jie Liu, and Diana Marculescu. "Single- path nas: Designing hardware-efficient convnets in less than 4 hours." arXiv preprint arXiv:1904.02877 (2019). -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Thank you all for coming today !43