Upgrade to Pro — share decks privately, control downloads, hide ads and more …

アニーリングマシンを活用したエッジAIにおける
生成モデルの学習効率化のためのアーキテクチャ

tsurubee
June 25, 2020

 アニーリングマシンを活用したエッジAIにおける
生成モデルの学習効率化のためのアーキテクチャ

tsurubee

June 25, 2020
Tweet

More Decks by tsurubee

Other Decks in Research

Transcript

  1. ͘͞ΒΠϯλʔωοτגࣜձࣾ (C) Copyright 1996-2019 SAKURA Internet Inc ͘͞ΒΠϯλʔωοτݚڀॴ ΞχʔϦϯάϚγϯΛ׆༻ͨ͠ΤοδAIʹ͓͚Δ ੜ੒Ϟσϧͷֶशޮ཰ԽͷͨΊͷΞʔΩςΫνϟ

    2019/12/06 ϚϧνϝσΟΞɺ෼ࢄɺڠௐͱϞόΠϧʢDICOMO2020ʣγϯϙδ΢Ϝ ɹ௽ా തจɼদຊ ྄հ ͘͞ΒΠϯλʔωοτגࣜձࣾ ͘͞ΒΠϯλʔωοτݚڀॴ
  2. 5 ΤοδAI࣮ݱͷͨΊͷγεςϜܗଶ ΤοδAIͷ࣮ݱํ๏͸ɼܭࢉίετ͕ߴ͍AIͷֶशͱਪ࿦ͷϓϩηεΛͲͷܭࢉࢿݯ্Ͱ ࣮ߦ͢Δ͔ʹΑΓଟذʹ౉͓ͬͯΓɼ໨త΍ཁٻʹԠͯ͡ద੾ʹબ୒͢Δɽ Ϋϥ΢υͰॲཧ͢Δ৔߹ ΤοδAIͷγεςϜܗଶͷྫ Cloud ֶश ਪ࿦ Edge

    ֶश΍ਪ࿦ͷҰ෦ΛσόΠεΑΓ५୔ͳॲཧೳྗ Λ༗͢ΔΫϥ΢υͰߦ͏ σόΠεɾΤοδͰॲཧ͢Δ৔߹ Cloud ֶश ਪ࿦ ֶश΍ਪ࿦ͷҰ෦ΛσόΠε্ɼ͋Δ͍͸σόΠε ʹ෺ཧతʹ͍ۙωοτϫʔΫ্ͷܭࢉࢿݯͰߦ͏ ֶश ਪ࿦ ྫ͑͹ɼσόΠε͕ਓؒʹର͢ΔΠϯλʔϑΣʔεΛ୲͏΋ͷΛ૝ఆͨ͠৔߹ɼਓ͕ؒ஗ԆΛײ͡ͳ͍ ਺ेmsΦʔμʔͷԠ౴ੑ͕ཁٻ͞ΕΔͨΊɼ௨৴஗ԆͷΈͰཁٻ͞ΕΔԠ౴࣌ؒΛ௒͑Δɽ ͜ͷ৔߹ɼΫϥ΢υΛհ͞ͳ͍γεςϜܗଶ͕๬·ΕΔɽ ਺ඦmsΦʔμʔͷ ௨৴஗Ԇ͕ൃੜ ਺msΦʔμʔͷ ௨৴஗ԆΛ཈͑Δ ਪ࿦ ਪ࿦
  3. 6 ΤοδAIʹ͓͚Δੜ੒Ϟσϧ • ؍ଌσʔλ͸֬཰తʹੜ੒͞Ε͍ͯΔͱԾఆ͠ɼͦͷੜ੒աఔΛ ֬཰෼෍ͱͯ͠ϞσϧԽ͢ΔΞϓϩʔν (=֬཰తੜ੒Ϟσϧ) • ֶशޙͷϞσϧ͔Β৽ͨʹσʔλΛੜ੒Ͱ͖Δ ੜ੒Ϟσϧͱ͸ •

    ଟ͘ͷ֬཰తੜ੒Ϟσϧͷֶशɾਪ࿦ʹ͸ɼҰൠʹܭࢉίετ͕ߴ͍֬཰෼෍(໬౓ؔ਺)͔Βͷ αϯϓϦϯά͕ඞཁ • σόΠε͸Ұൠʹॲཧೳྗ͕௿͍ͨΊɼσόΠε্Ͱֶशɾਪ࿦Λ׬݁͢Δͷ͸೉͍͠ɽ ੜ੒Ϟσϧͷֶश [1] R. Salakhutdinov and G. Hinton, “Deep Boltzmann Machines”, in Proceedings of the International Conference on Artificial Intelligence and Statistics, 2009, pp. 448-455. • ΤϯυσόΠεʹը૾ɾจষੜ੒౳ͷ૑଄తλεΫ΍ҟৗݕ஌ɼ ऩूσʔλͷܽଛ஋ิ׬ɾϊΠζআڈͳͲ෯޿͍Ԡ༻ੑΛ΋ͨΒ͢ɽ ਂ૚ϘϧπϚϯϚγϯʹΑΔը૾ੜ੒[1] pθ (x) pdata (x) ਅͷ෼෍(σʔλ෼෍) ੜ੒Ϟσϧ ੜ੒Ϟσϧ Λσʔλ෼෍ ʹ͚ۙͮΔΑ͏ʹֶश͢Δ pθ (x) pdata (x)
  4. 9 ΤοδAIʹ͓͚Δੜ੒Ϟσϧͷ׆༻ • Ϋϥ΢υΛհ͞ͳ͍ੜ੒Ϟσϧͷֶशɾਪ࿦ͷͨΊͷΞϓϩʔν • σόΠε͔ΒͷΞϓϩʔν • ߴ͍ॲཧೳྗΛ༗͢ΔσόΠεͷ։ൃ ྫʣEdge TPUɼJetson

    Nano • ֶशϞσϧ͔ΒͷΞϓϩʔν • αϯϓϦϯά͕ܰྔͳֶशϞσϧɼϞσϧѹॖٕज़ͷ։ൃͳͲɽ • ֶशɾਪ࿦ख๏͔ΒͷΞϓϩʔν • ޮ཰తͳֶशɾਪ࿦ख๏ͷ։ൃͳͲ
  5. 10 ΤοδAIʹ͓͚Δੜ੒Ϟσϧͷ׆༻ • Ϋϥ΢υΛհ͞ͳ͍ੜ੒Ϟσϧͷֶशɾਪ࿦ͷͨΊͷΞϓϩʔν • σόΠε͔ΒͷΞϓϩʔν • ߴ͍ॲཧೳྗΛ༗͢ΔσόΠεͷ։ൃ ྫʣEdge TPUɼJetson

    Nano • ֶशϞσϧ͔ΒͷΞϓϩʔν • αϯϓϦϯά͕ܰྔͳֶशϞσϧɼϞσϧѹॖٕज़ͷ։ൃͳͲɽ • ֶशɾਪ࿦ख๏͔ΒͷΞϓϩʔν • ޮ཰తͳֶशɾਪ࿦ख๏ͷ։ൃͳͲ
  6. 11 ྔࢠΞχʔϦϯάͷԠ༻ઌ ྔࢠΞχʔϦϯάΛ༻͍ͨ৽ͨͳΞϓϩʔν 1ɽ૊߹ͤ࠷దԽ໰୊ 2ɽαϯϓϦϯά αϯϓϦϯάͱ͸ɼ֬཰෼෍͔Βඪຊ(αϯϓϧ)Λநग़͢Δ͜ͱɽ ֬཰෼෍ͷظ଴஋ܭࢉʹ༻͍ΒΕΔɽ D-Wave Systems͕ࣾ։ൃ͢ΔྔࢠΞχʔϦϯάϚγϯʢҎԼɼD-WaveϚγϯʣ͸ɼҎԼͷ ੜ੒Ϟσϧͷֶशɾਪ࿦ͷߴ଎Խ΍ߴਫ਼౓Խʹ੒ޭ͍ͯ͠Δɽ

    [1] S. H. Adachi and M. P. Henderson, “Application of Quantum Annealing to Training of Deep Neural Networks”, arXiv:1510.06356 2015. [2] D. Korenkevych et al., “Benchmarking Quantum Hardware for Training of Fully Visible Boltzmann Machines”, arXiv:1611.04528 2016. [3] W. Vinci et al., “A Path Towards Quantum Advantage in Training Deep Generative Models with Quantum Annealers”, arXiv:1912.02119 2019. • ϘϧπϚϯϚγϯ • ੍ݶϘϧπϚϯϚγϯ[1] • ՄࢹϊʔυͷΈͷϘϧπϚϯϚγϯ[2] • ม෼ΦʔτΤϯίʔμʢVAEʣ[3]
  7. 12 ΤοδAI΁ͷద༻ͱ՝୊ [1] http://dwavejapan.com/system/ [2] D. Korenkevych et al., “Benchmarking

    Quantum Hardware for Training of Fully Visible Boltzmann Machines”, arXiv:1611.04528 2016. [3] Z. Chen et al., “An empirical study of latency in an emerging class of edge computing applications for wearable cognitive assistance”, in Proceedings of the Second ACM/IEEE Symposium on Edge Computing, Oct. 2017, pp. 1-14. D-Wave 2000Q[1] • ྔࢠΞχʔϦϯάΛ׆༻ͨ͠ੜ੒Ϟσϧͷֶशɾਪ࿦ͷ࢓૊ΈΛΤοδAI ʹద༻͢Δ৔߹ɼD-WaveϚγϯ͸අ༻΍ۃ௿Թͷಈ࡞؀ڥΛཁ͢ΔͳͲͷ ཧ༝͔ΒΤοδྖҬ΁ͷ഑ஔ͸ࠔ೉Ͱ͋Δɽ • D-WaveϚγϯΛར༻͢Δࡍ͸ɼωοτϫʔΫΛܦ༝ͨ͠Ϋϥ΢υαʔϏε ͱͯ͠ར༻͢Δɽ • Ϋϥ΢υΛհͣ͞ΤοδଆͰֶशΛߦ͏͜ͱ͕Ͱ͖ͳ͍ͨΊɼσόΠεͱΫϥ΢υؒͷ෺ཧత ͳڑ཭ʹىҼͨ͠௨৴஗Ԇ͕ൃੜ͢Δɽ • D-WaveϚγϯͷαϯϓϦϯάʹ͔͔Δ͕࣌ؒ֓ͶμsΦʔμʔ[2]Ͱ͋Δ͜ͱɼ͓ΑͼΫϥ΢υ ͱͷ௨৴஗Ԇ͕֓Ͷ਺ඦmsΦʔμʔ[3]Ͱ͋Δ͜ͱΛߟ͑Δͱɼੜ੒Ϟσϧͷֶश΍ਪ࿦ʹ͓͍ ͯ௨৴஗Ԇ͕ϘτϧωοΫʹͳΔɽ
  8. 13 D-WaveϚγϯͱΞχʔϦϯάϚγϯ ΠδϯάϞσϧ [1] M. Yamaoka et al., “CMOS Annealing

    Machine: an In-memory Computing Accelerator to Process Combinatorial Optimization Problems”, in IEEE Custom Integrated Circuits Conference (CICC), Apr. 2019, pp. 1-8. [2] M. Aramon et al., “Physics-Inspired Optimization for Quadratic Unconstrained Problems Using a Digital Annealer”, Front. Phys., vol. 7, no. 48, Apr. 2019. [3] E. Crosson and A. W. Harrow, “Simulated Quantum Annealing Can Be Exponentially Faster Than Classical Simulated Annealing”, in IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS), Oct. 2016, pp. 714-723. • D-WaveϚγϯͷΑ͏ͳྔࢠϏοτͷৼΔ෣͍Λσδλϧճ࿏্ͰγϛϡϨʔτ͢Δ΋ͷͱͯ͠ɼ ΞχʔϦϯάϚγϯ͕͋Δɽ • ΞχʔϦϯάϚγϯ͸͜Ε·ͰʹɼݱߦͷD-WaveϚγϯͰ͸ղ͚ͳ͍໰୊αΠζΛ࣋ͭ૊߹ͤ ࠷దԽ໰୊ͷղ๏ͱͯ͠ɼͦͷ༗ޮੑ͕ࣔ͞Ε͍ͯΔɽ • ΞχʔϦϯάϚγϯ͸ɼD-WaveϚγϯͱൺ΂ͯখܕ͔ͭ௿ίετͰ͋ΓɼৗԹͰಈ࡞͢Δ ͱ͍͏ಛ௃Λ༗͍ͯ͠Δɽ ※ ຊൃදʹ͓͚ΔΞχʔϦϯάϚγϯͷఆٛ͸ɼΠδϯάϞσϧͷجఈঢ়ଶ୳ࡧΛ໨త ɹͱͨ͠΋ͷͰ͋ΓɼFPGA΍ASICΛ༻͍ͨϋʔυ΢ΣΞ࣮૷[1,2]ʹՃ͑ɼGPU΍CPU ɹ্Ͱಈ࡞͢Διϑτ΢ΣΞ࣮૷[3]΋ؚΉ
  9. 15 ΞʔΩςΫνϟ֓ཁ ੜ੒Ϟσϧͷֶशɾਪ࿦ʹඞཁͳαϯϓϦϯάॲཧͷΞΫηϥϨʔλͱͯ͠ɼ ΞχʔϦϯάϚγϯΛσόΠεͷۙ๣ʹ഑ஔ͢ΔΞʔΩςΫνϟ Edge server Cloud WAN Edge End

    devices Annealing machine AI AI AI AI AI ΤοδྖҬʹ഑ஔͨ͠ΞχʔϦϯάϚγϯ͸ɼαϯϓϦϯάॲཧͷΞΫηϥϨʔλͱͯ͠ ෳ਺୆ͷσόΠεʹڞ༗͞ΕΔɽ
  10. ࣮ݧ؀ڥ ֤ϩʔϧͷઆ໌ • σόΠε • ੜ੒Ϟσϧͷֶशɾਪ࿦Λߦ͏໾ׂɽ ֶशɾਪ࿦ͰඞཁͳαϯϓϦϯάΛΞχʔϦϯάϚγϯʹཁٻ͢ΔΫϥΠΞϯτͱͯ͠ಈ࡞ • ੜ੒Ϟσϧͱͯ͠ɼ੍ݶϘϧπϚϯϚγϯ(RBM)Λ༻͍Δɽ •

    ֶशσʔλͱͯ͠MNISTσʔληοτΛ༻͍ͯɼRBMͰը૾ͷੜ੒աఔΛֶश͢Δɽ • ΞχʔϦϯάϚγϯ • σόΠε͔ΒͷαϯϓϦϯάཁٻΛड͚ɼܭࢉ݁ՌΛฦ͢໾ׂ • αϯϓϦϯάͷॲཧʹ͸ɼSimulated Quantum Annealing(SQA)Λ༻͍Δɽ • SQAΛ࣮૷ͨ͠Sqaod[1]Λ༻͍ͯWeb APIΛߏஙͨ͠ɽ v 1 v 2 v 3 v n h 1 h 2 h m RBMͷάϥϑߏ଄ ӅΕ૚ m = 100 Մࢹ૚ n = 784 ΞχʔϦϯάϚγϯ σόΠε CPU1ίΞ ϝϞϦ1GB CPU8ίΞ ϝϞϦ32GB αϯϓϦϯάཁٻ ܭࢉ݁ՌΛฦ͢ 20 Web API ֶशσʔλ [1] https://github.com/shinmorino/sqaod
  11. 1. ΞχʔϦϯάϚγϯΛ༻ֶ͍ͨश RBMΛҎԼͷೋͭͷֶशख๏Ͱֶश͠ɼൺֱ͢Δɽ ɹطଘͷֶशख๏ɿίϯτϥεςΟϒɾμΠόʔδΣϯε(CD)๏[1] ɹఏҊख๏ɿΞχʔϦϯάϚγϯ (ΞϧΰϦζϜ͸SQA) ֶशํ๏ αϯϓϦϯάͱύϥϝʔλߋ৽͕Ұճ࣮ͣͭߦ 21 1ΤϙοΫ=600ΠςϨʔγϣϯ

    CD๏͸ɼશͯͷॲཧΛσόΠε্Ͱ࣮ߦ͢Δɽ ҰํɼSQAͰ͸αϯϓϦϯάͷॲཧͷΈΛWeb APIܦ༝ͯ͠ΞχʔϦϯάϚγϯͰ࣮ߦ͢Δɽ MNISTͷ60000ຕͷ܇࿅ը૾σʔλΛόοναΠζ100ͰϛχόονֶशΛߦ͏ ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ ɾ 1ΠςϨʔγϣϯ [1] G. Hinton, “Training Products of Experts by Minimizing Contrastive Divergence”, Neural computation, vol. 14, no. 8, pp. 1771-1800, Aug. 2002.
  12. 22 1. ΞχʔϦϯάϚγϯΛ༻ֶ͍ͨश • ޿ൣͳ໰୊ઃఆʹରͯ͠ಉ༷ͷ݁Ռ͕ಘΒΕΔ͔ɽ • ݫີʹ͸࠷খ஋୳ࡧͱظ଴஋ܭࢉ͸ҟͳΔॲཧͰ͋ΔͨΊɼ࠷খ஋୳ࡧͷฏۉ஋Λظ଴஋ ͱͯۙ͠ࣅ͢Δ͜ͱʹΑΔӨڹ͸Ͳ͏͔ɼetc. SQA͸CD๏ʹൺ΂ͯɼগͳ͍ΤϙοΫ਺Ͱ௿͍࠶ߏ੒ޡࠩʹ ౸ୡ͓ͯ͠Γɼऩଋޙͷ࠶ߏ੒ޡࠩ΋௿͍஋Λ͍ࣔͯ͠Δɽ

    ࠓճͷ৚݅ʹ͓͍ͯɼSQA͸CD๏ΑΓ΋RBMΛޮ཰తʹֶश Ͱ͖͍ͯΔͱݴ͑Δɽ ֶशޙͷϞσϧ͕σʔλͷੜ੒աఔΛͲΕ͚ͩଊ͑ͯ ͍Δ͔ͷࢦඪͱͳΓɼ௿͍஋΄Ͳྑ͍Ϟσϧͱݴ͑Δ ධՁࢦඪ ߋͳΔݕূ͕ඞཁ ΤϙοΫ͋ͨΓͷฏۉֶश࣌ؒ CD๏ɿ5.21 sɼSQAɿ282 s ΤϙοΫ਺ʹର͢Δ࠶ߏ੒ޡࠩ Reconstruction Error = ∑ i (image(i) ≠ reconstructedImage(i)) total pixels
  13. 23 2. ௨৴஗ԆͷӨڹ ҎԼͷೋͭͷ؀ڥͰRBMͷਪ࿦ʹཁ͢Δ࣌ؒΛܭଌ͠ɼൺֱ͢Δɽ ๺ւಓੴङࢢͷDC ΞϝϦΧ߹ऺࠃ όʔδχΞभͷDC ΞχʔϦϯάϚγϯ σόΠε σόΠε

    ᶃ RTT: 0.612 ms ᶄ RTT: 176 ms ᶃ ఏҊख๏ σόΠεͱΞχʔϦϯάϚγϯΛಉDC಺ʹ഑ஔ͢Δ ᶄ Ϋϥ΢υ؀ڥ σόΠεͱΞχʔϦϯάϚγϯΛ෺ཧతʹڑ཭͕ ɹ཭ΕͨDC಺ʹͦΕͧΕ഑ஔ͢Δɽ ਪ࿦ͷॲཧͷྲྀΕ σόΠε ΞχʔϦϯάϚγϯ ਪ࿦࣌ؒʹ͸ɼ ɾσόΠεͱΞχʔϦϯάϚγϯؒͷ௨৴஗Ԇ ɾΞχʔϦϯάϚγϯ্ͰͷαϯϓϦϯά࣌ؒ ɾWeb APIͷԠ౴࣌ؒ ͳͲؚ͕·Ε͍ͯΔɽ ᶃαϯϓϦϯάཁٻ ᶅܭࢉ݁ՌΛฦ͢ ᶄαϯϓϦϯά࣮ߦ ※ RTT: pingίϚϯυΛ༻͍ͯଌఆͨ͠ϥ΢ϯυτϦοϓλΠϜ Web API
  14. 24 2. ௨৴஗ԆͷӨڹ RBMͷਪ࿦࣌ؒͷܭଌ݁Ռ • ਪ࿦࣌ؒ͸ɼΫϥ΢υ؀ڥ͕ఏҊख๏ʹൺ΂ͯ350 ms ஗͍݁Ռͱͳͬͨɽ • ͜ͷࠩ෼͸ɼΫϥ΢υ؀ڥ͕ఏҊख๏ʹൺ΂ͯσόΠε

    ͱΞχʔϦϯάϚγϯؒͷ෺ཧతͳڑ཭͕ԕ͘ͳΔ͜ͱ Ͱੜ͡Δ௨৴஗ԆʹىҼ͍ͯ͠Δͱߟ͑ΒΕΔɽ ࣮ݧ݁Ռ ※ਪ࿦࣌ؒʹ͸αϯϓϦϯάؚ͕࣌ؒ·ΕΔ
  15. 25 2. ௨৴஗ԆͷӨڹ RBMͷਪ࿦࣌ؒͷܭଌ݁Ռ • ਪ࿦࣌ؒ͸ɼΫϥ΢υ؀ڥ͕ఏҊख๏ʹൺ΂ͯ350 ms ஗͍݁Ռͱͳͬͨɽ • ͜ͷࠩ෼͸ɼΫϥ΢υ؀ڥ͕ఏҊख๏ʹൺ΂ͯσόΠε

    ͱΞχʔϦϯάϚγϯؒͷ෺ཧతͳڑ཭͕ԕ͘ͳΔ͜ͱ Ͱੜ͡Δ௨৴஗ԆʹىҼ͍ͯ͠Δͱߟ͑ΒΕΔɽ • D-WaveϚγϯΛ༻͍ͨ৔߹ɼαϯϓϦϯά࣌ؒ͸ΞχʔϦϯάϚγϯΑΓߴ଎Ͱ͋Δ͜ͱ͕༧૝͞Ε Δ͕ɼྫ͑αϯϓϦϯά͕࣌ؒμsΦʔμʔͰ͋ͬͯ΋ɼ௨৴஗Ԇ͕ϘτϧωοΫͱͳΓɼਪ࿦࣌ؒ͸ఏ Ҋख๏ΑΓ΋஗͍400 msఔ౓Ͱ͋Δ͜ͱ͕༧૝͞ΕΔɽ • ҰํɼఏҊख๏Ͱ͸ɼਪ࿦࣌ؒʹରͯ͠αϯϓϦϯά͕࣌ؒࢧ഑తͰ͋ΔͨΊɼΑΓαϯϓϦϯά࣌ؒ ͕୹͍ΞχʔϦϯάϚγϯΛ༻͍Δ͜ͱͰਪ࿦࣌ؒΛେ෯ʹ୹ॖͰ͖Δɽ ྫ͑͹ɼαϯϓϦϯά࣌ؒΛ30 msఔ౓·Ͱߴ଎ԽͰ͖Ε͹ɼਪ࿦͕࣌ؒ100 msΛԼճΔɽ Ϋϥ΢υܦ༝ͰD-WaveϚγϯΛ༻͍Δ৔߹ͱͷൺֱ ࣮ݧ݁Ռ ※ਪ࿦࣌ؒʹ͸αϯϓϦϯάؚ͕࣌ؒ·ΕΔ