Deep Learningライブラリ 色々つかってみた感想まとめ

Transcript

1. Deep LearningϥΠϒϥϦ
   ৭ʑ͔ͭͬͯΈͨײ૝·ͱΊ
   @conta_
   

   View Slide

2. Self Introduction
   ॹํɹول (twitter: @conta_)
   CTO@ABEJA, Inc.
   Computer Visionͱ͔ɺMachine LearningΛ࢖ͬͨ
   ϓϩμΫτ։ൃΛ΍͍ͬͯ·͢ɻ
   

   View Slide

3. Deep Learning Library?
   

   View Slide

4. View Slide

5. ʊਓਓਓਓਓਓਓਓਓʊ
   ʼɹଟ͗ͯͭ͢Β͍ɹʻ
   ʉY^Y^Y^Y^Y^Y^Y^Yʉ
   

   View Slide

6. Dive into Deep Learning
   

   View Slide

7. ˎײ͡ํʹ͸ݸਓ͕ࠩ͋Γ·͢
   

   View Slide

8. ࠓճ঺հ͢ΔϥΠϒϥϦ
   

   View Slide

9. Caffe
   Caffe:
   UC Berkleyͷਓ͕࡞ͬͯΔɻDeepLearningք۾Ͱ͸͔ͳΓ࿝ฮͳϥΠϒϥϦͰɺ
   ޭ੷͸େ͖͍
   ݴޠ:
   ɾCore͸C++ɻPython, MatlabͷWrapper͕͋Δ
   ಛ௃:
   ɾجຊతʹProtocol BufferͰωοτϫʔΫΛهड़
   

   View Slide

10. ■͍͍ͱ͜Ζ(ݸਓతײ૝)
    ɾModel ZooʹֶशࡁΈϞσϧ͕ͨ͘͞Μެ։͞ΕͯΔ
    ʢطʹCVPR2016ͷ࿦จͷϞσϧ΋ެ։͞ΕͯΔʣ
    ɾݚڀऀׂ͕ͱ࢖ͬͯΔͷͰ࠷৽ͷݚڀ੒Ռ͕CaffeͰ࣮૷͞ΕͯͨΓ͢Δ
    ɾMulti-GPUʹରԠͨ͠ͷͰɺઃఆ̍ͭͰෳ਺ͷGPUΛར༻Մೳ
    ɾ࣮ߦ଎౓ׂ͕Γͱૣ͍
    ɾωοτϫʔΫͷύϑΥʔϚϯεςετ͕Ͱ͖Δ(caffe testίϚϯυ)
    

    View Slide

11. ■ͭΒ͍ͱ͜Ζ(ݸਓతײ૝)
    ɾΧελϚΠζ͕c++ͱProtocol BufferɻɻɻϚξͭΒ͌Ηɻɻɻʢˎ̍ʣ
    =>ਓ͕ΧελϚΠζͨ͠΋ͷ͸ɺ΋͸΍Θ͔ΒΜɻ
    ɾωοτϫʔΫΛProtocol BufferͰॻ͘ͷ͕ͭΒ͍ʢˎ̎ʣ
    =>GoogLeNet͸໿2000ߦɺResNet͸໿7000ߦɻɻɻ
    ʢProtocol Buffer৬ਓܳʣ
    ɾσʔληοτΛ࡞੒͢Δͷ͕େม
    ɾΤϥʔ͕Θ͔Γʹ͍͘
    ɾιʔείʔυΛಡΊͳ͍ͱશػೳ͸࢖͑ͳ͍ɺಈ͖͕Θ͔Βͳ͍
    ʢυΩϡϝϯτߋ৽͠Ζʂʣ
    ɾΠϯετʔϧ͕ͭΒ͍
    ʢੲʹൺ΂Δͱґଘؔ܎ͷOnOffͷΦϓγϣϯ͕෇͍ͨͨΊɺ
    ͍ͩͿϚγʣ
    ɾRNNΛѻ͏͜ͱ͸Ͱ͖ͳ͍ʢຐվ଄͞ΕͨCaffeϕʔεͷ΋ͷ͸͋Δ͚Ͳɻɻɻʣ
    

    View Slide

12. ■༨ஊʢˎ̍ʣ
    ɾ࠷ۙ͸Python Layer͕௥Ճ͞ΕͯPython͚ͩͰ΋ΧελϚΠζ
    Ͱ͖ΔΑ͏ʹຐվ଄޻෉͍ͯ͠Δ(No Documentation)
    

    View Slide

13. ■༨ஊʢˎ̎ʣ
    ɾPythonͰProtocol BufferΛੜ੒
    Ͱ͖ΔΑ͏ʹͳͬͨͨΊɺ
    ϧʔϓͨ͠هड़ׂ͕ͱ؆୯ʹͳͬͨ
    (No Documentation)
    

    View Slide

14. ■͜Μͳͻͱʹ͓͢͢Ί
    ɾ·ͣԿ͔ಈ͔͍ͨ͠ਓ
    ɾͱΓ͋͑ͣݚڀ੒ՌΛࢼ͍ͨ͠ਓ
    ɾ଎౓͕ඞཁͳਓ
    ɾC++ͱProtocol BufferΛษڧ͍ͨ͠ਓ
    ɾࠜؾڧ͘Կ͔ͱઓ͍͍ͨਓ
    

    View Slide

15. Tensorflow:
    G̋̋gle੡ͷ෼ࢄߦྻܭࢉϥΠϒϥϦɻ
    ผʹDeep͚ͩ͡Όͳ͍Μ͔ͩΒͶʂ
    ݴޠ:
    ɾCore͸C++ɻPythonͱC++ͲͪΒͰ΋ಈ͘ɻ
    ಛ௃:
    ɾ෼ࢄॲཧ͕؆୯ʹͰ͖Δ
    ɾGoogleͷϓϩμΫτͰԿ೥΋ར༻͞Ε͍ͯͯɺ҆ఆײ͕͋Δ
    

    View Slide

16. ■͍͍ͱ͜Ζ(ݸਓతײ૝)
    ɾ෼ࢄॲཧ͕ΊͬͪΌ؆୯ʹͰ͖Δ(Distributed Tensorflow)
    ɾGoogle͕MLϓϥοτϑΥʔϜΛఏڙ։࢝
    ɾ࠷ۙɺTensorflow࢖͍·ͨ͠࿦จ͕Α͘Ͱ͖͍ͯͯΔ
    ɾίΞ͕C++ͳͷͰAndroidͰ΋ಈ࡞͢Δ
    ɾDocker Container͕མͪͯΔͷͰɺDocker࢖͑ΔͳΒ
    ΠϯετʔϧʹࠔΒͳ͍
    ɾTensorboard͕ΦγϟϨ
    

    View Slide

17. ■ͭΒ͍ͱ͜Ζ(ݸਓతײ૝)
    ɾݰਓ޲͚ϥΠϒϥϦ
    =>࢓૊Έ͕ͪΐͬͱෳࡶͳͷͰཧղ͠ͳ͍ͱ࢖͍͜ͳͤͳ͍
    =>ωοτϫʔΫΛॻ͘ͷʹҰ͔Βهड़͢Δඞཁ͕͋ΔɺTheanoతͳཱͪҐஔ
    ɾιʔείʔυ͕େن໛ͳͨΊվ଄͕େมͦ͏
    ʢҰԠυΩϡϝϯτ͸͋Δ͚Ͳʣ
    ɾDistributed TensorflowΛݸਓͷࢿݯͰ׆༻͢Δͷ͸ࠔ೉ͳͷͰɺGoogleͷϓϥοτ
    ϑΥʔϜΛ࢖Θͳ͍ͱԸܙΛड͚ʹ͍͘
    =>෼ࢄίϯϐϡʔςΟϯάͷIOϘτϧωοΫɺInfiniBandΛ͍ͬͺ͍ങ͑Δ͓ۚ࣋ͪ
    ͳΒԸܙΛड͚ΒΕΔ͔΋
    

    View Slide

18. ■͜Μͳͻͱʹ͓͢͢Ί
    ɾ࢓૊Έͷ෦෼͔ΒDeep LearningΛษڧ͍ͨ͠ਓ
    ɾDeep Learningɹதʙ্ڃऀ޲͚ͷਓ
    ɾେن໛ػցֶशΛ΍ͬͯΈ͍ͨਓ
    ɾେن໛ػցֶशج൫Λ࡞Γ͍ͨਓ
    ɾMobileʹ૊ΈࠐΈ͍ͨਓ
    

    View Slide

19. Chainer:
    PFN੡ͷDeep LearningϥΠϒϥϦɻ
    ݴޠ:
    ɾPython(+Cuda)
    ಛ௃:
    ɾDefine-by-Runͱ͍͏ख๏Λͱ͍ͬͯͯɺωοτϫʔΫΛޙ͔Βղ
    ऍ
    ɾ͢͹Β͍͠
    

    View Slide

20. ■͍͍ͱ͜Ζ(ݸਓతײ૝)
    ɾωοτϫʔΫͷهड़ͷॊೈੑ͕ߴ͍
    ʢಛʹRNNܥඇৗʹॻ͖΍͍͢ʣ
    ɾ಺෦ͷಈ࡞͕Ͳ͏ͳͬͯΔ͔ඇৗʹΘ͔Γ΍͍͢
    ɾσόοΫ͠΍͍͢
    ɾφ΢͍ΞϧΰϦζϜ͕͍ͪૣ࣮͘૷͞ΕͯΔ
    ɾCupyͱ͍͏Cuda͕؆୯ʹ࢖͑ΔߦྻԋࢉϥΠϒϥϦؚ͕·Ε͍ͯ
    ͯɺࣗલͷΞϧΰϦζϜΛൺֱత؆୯ʹߴ଎ԽͰ͖Δ
    (C++Ͱॻ͍ͯϥούʔͱ͔ͭ͘Βͳ͍͍ͯ͘)
    ɾதͷਓ͕͍͢͝
    

    View Slide

21. ■ͭΒ͍ͱ͜Ζ(ݸਓతײ૝)
    ɾωοτϫʔΫҎ֎ͷهड़ྔ͕ଟ͘ͳͬͯ͠·͏ʢֶशͷίʔυͱ͔ʣ
    ɾ࣮ߦ଎౓ʢ࠷ۙ͸ͦͦ͜͜ૣ͍ͬΆ͍ʣ
    ɾDeep Learning෼͔ͬͯͳ͍ͱଟ෼࢖͍͜ͳͤͳ͍
    

    View Slide

22. ■͜Μͳͻͱʹ͓͢͢Ί
    ɾDeep LearingΛҰ͔ΒΨοπϦษڧ͍ͨ͠ਓ
    ɾDeep Learningɹதʙ্ڃऀͷਓ
    ɾݚڀͰTry and ErrorΛ܁Γฦ͠ͳ͕ΒΞϧΰϦζϜΛ։ൃ͍ͨ͠ਓ
    ɾෳࡶͳωοτϫʔΫΛهड़͍ͨ͠ਓ
    ʢωοτϫʔΫ಺Ͱ৚݅จॻ͖͍ͨɺσʔλʹΑͬͯॲཧΛ෼͚͍ͨ౳ʣ
    ɾRNNͱ͔NLPͱ͔Λॻ͖͍ͨ
    

    View Slide

23. ■MXNet:
    DMLC(Distributed (Deep) Machine Learning Community)͕࡞ͬͯ
    ΔɻXGBoostͷ࡞੒ݩͱͯ͠΋༗໊ɻ
    ■ݴޠ:
    ɾCore͸C++ɻWrapper͕ͨ͘͞Μ͋ΓɺPythonɺC++ɺScalaɺ
    RɺMatlabɺJuliaͱଟݴޠରԠɻ
    ■ಛ௃:
    ɾଟݴޠʂ
    ɾ଎౓͕͔ͳΓૣ͍ʢॴײʣ
    ɾmshadow(ߦྻԋࢉ)ɺps-lite(෼ࢄॲཧ)౳ͷϥΠϒϥϦ͕ϕʔε
    

    View Slide

24. ■͍͍ͱ͜Ζ(ݸਓతײ૝)
    ɾ෼ࢄॲཧ(1Node, Multi-GPUɺMulti-NodeɺMulti-GPUͲͪΒ
    ΋ʣ͕ΊͬͪΌ؆୯ʹͰ͖Δ(Example͋Γ)
    ɾS3΁ϞσϧσʔλΛอଘ͢Δػೳ͕͋Δ
    ɾૣ͍ʢImageNet full datasetΛGeForce GTX 980*4Ͱ8.5೔)
    ɾͳͥૣ͍͔͕υΩϡϝϯτͰྗઆ͞Ε͍ͯΔ
    ɾଟ෼࠷΋ଟݴޠ͕ਐΜͰ͍Δ
    ɾC++Ͱॻ͔ΕͯΔͷͰ
    Mobile(iOS, Android)Ͱ΋ಈ͘
    

    View Slide

25. ■ͭΒ͍ͱ͜Ζ(ݸਓతײ૝)
    ɾΤϥʔ͕Θ͔Γʹ͍͘ɺຊ౰ʹΘ͔Γʹ͍͘
    ɾυΩϡϝϯτ͕গͳ͍
    =>ಛघͳֶशσʔλΛ࡞ͬͨΓ͢Δͷ͸େม
    =>೉͍͜͠ͱΛ͠Α͏ͱ͢ΔͱιʔεΛಡ·ͳ͚Ε͹ͳΒͳ͍
    

    View Slide

26. ■͜Μͳͻͱʹ͓͢͢Ί
    ɾDeep Learningɹதʙ্ڃऀ޲͚ͷਓ
    ɾ଎౓ΛٻΊ͍ͯΔਓ
    ɾPythonɺC++Ҏ֎Ͱ΋ར༻͍ͨ͠ਓ
    

    View Slide

27. ■Keras:
    PythonͷDeep LearningϥΠϒϥϦɻ
    ࠷ۙv1.0͕ϦϦʔε͞Εͨɻ
    ■ݴޠ:
    ɾPython
    ■ಛ௃:
    ɾTorchʹࣅͨهड़ํ๏ɻ
    ɾߦྻԋࢉͷόοΫΤϯυ͸TheanoͱTensorFlowΛར༻͍ͯͯ͠ɺ
    ੾Γସ͑Δ͜ͱ͕Ͱ͖Δ
    

    View Slide

28. ■͍͍ͱ͜Ζ(ݸਓతײ૝)
    ɾωοτϫʔΫهड़͕؆୯ɺॊೈ
    ϕʔεͷAPI͕ͨ͘͞Μ४උ͞Ε͍ͯΔͨΊɺهड़ྔ΋গͳ͘ࡁ
    Ήɻ؆୯ͳωοτϫʔΫͰ͋Ε͹API૊Έ߹ΘͤͰͳΜͱ͔ͳΔɻ
    v1.0.0͔Β functional APIͳΔ΋ͷ͕ग़དྷͯɺ
    ඇৗʹ௚ײతʹωοτϫʔΫΛهड़Ͱ͖ΔΑ͏ʹͳͬͨ
    ɾֶश͕؆୯
    ScikitͷΑ͏ʹfit()ؔ਺ݺͼग़ͤ͹ΑΖ͘͠΍ͬͯ͘ΕΔ
    ɾιʔε͕ಡΈ΍͍͢
    

    View Slide

29. ■ͭΒ͍ͱ͜Ζ(ݸਓతײ૝)
    ɾMulti-GPUඇରԠ
    TheanoΛBackendͱͯ͠࢖ͬͯΔͱMulti-GPUͭΒ͍ɻ
    Tensorflowͷ͓͔͛ͰMulti-GPU͕؆୯ʹͰ͖ΔΑ͏ʹͳͬͨʁ
    ɾPython͔͠ରԠ͍ͯ͠ͳ͍
    

    View Slide

30. ■͜Μͳͻͱʹ͓͢͢Ί
    ɾDeep LearningΛ΍Γ͍ͨਓશൠ
    ɾ͋·Γࡉ͔͍͜ͱ͸ؾʹͤͣʹαΫοͱωοτϫʔΫΛ࡞
    Γ͍ͨਓ
    ˎݸਓతʹ͸Ұ൪͓͢͢Ί
    

    View Slide

31. ·ͱΊ
    ■Caffe
    ɾͱΓ͋͑ͣDeep LearingʢCNNʣ΍Γ͍ͨਓ
    ɾݚڀ੒ՌΛࢼ͍ͨ͠ਓ
    ■Tensorflow
    ɾ෼ࢄίϯϐϡʔςΟϯά΍Γ͍ͨਓ
    ■Chainer
    ɾΞϧΰϦζϜ։ൃ͍ͨ͠ਓ
    ɾຊؾͰDeep LearningΛษڧ͍ͨ͠ਓ
    ■MXNet
    ɾ଎౓͕ඞཁͳਓ
    ɾMobileͰಈ͔͍ͨ͠ਓ
    ■Keras
    ɾͱΓ͋͑ͣDeep Learingษڧ͍ͨ͠ਓ
    ɾΊΜͲ͍͘͞ͷͰ͋Δఔ౓؀ڥ͕४උ͞Ε͍ͯͯཉ͍͠ͱࢥ͏ਓ
    

    View Slide

32. We are hiring!
    
    → https://www.wantedly.com/companies/abeja
    

    View Slide