Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep Learningライブラリ 色々つかってみた感想まとめ

Deep Learningライブラリ 色々つかってみた感想まとめ

Takanori Ogata

April 17, 2016
Tweet

More Decks by Takanori Ogata

Other Decks in Technology

Transcript

  1. Deep LearningϥΠϒϥϦ
    ৭ʑ͔ͭͬͯΈͨײ૝·ͱΊ
    @conta_

    View full-size slide

  2. Self Introduction
    ॹํɹول (twitter: @conta_)
    CTO@ABEJA, Inc.
    Computer Visionͱ͔ɺMachine LearningΛ࢖ͬͨ
    ϓϩμΫτ։ൃΛ΍͍ͬͯ·͢ɻ

    View full-size slide

  3. Deep Learning Library?

    View full-size slide

  4. ʊਓਓਓਓਓਓਓਓਓʊ
    ʼɹଟ͗ͯͭ͢Β͍ɹʻ
    ʉY^Y^Y^Y^Y^Y^Y^Yʉ

    View full-size slide

  5. Dive into Deep Learning

    View full-size slide

  6. ˎײ͡ํʹ͸ݸਓ͕ࠩ͋Γ·͢

    View full-size slide

  7. ࠓճ঺հ͢ΔϥΠϒϥϦ

    View full-size slide

  8. Caffe
    Caffe:
    UC Berkleyͷਓ͕࡞ͬͯΔɻDeepLearningք۾Ͱ͸͔ͳΓ࿝ฮͳϥΠϒϥϦͰɺ
    ޭ੷͸େ͖͍
    ݴޠ:
    ɾCore͸C++ɻPython, MatlabͷWrapper͕͋Δ
    ಛ௃:
    ɾجຊతʹProtocol BufferͰωοτϫʔΫΛهड़

    View full-size slide

  9. ■͍͍ͱ͜Ζ(ݸਓతײ૝)
    ɾModel ZooʹֶशࡁΈϞσϧ͕ͨ͘͞Μެ։͞ΕͯΔ
    ʢطʹCVPR2016ͷ࿦จͷϞσϧ΋ެ։͞ΕͯΔʣ
    ɾݚڀऀׂ͕ͱ࢖ͬͯΔͷͰ࠷৽ͷݚڀ੒Ռ͕CaffeͰ࣮૷͞ΕͯͨΓ͢Δ
    ɾMulti-GPUʹରԠͨ͠ͷͰɺઃఆ̍ͭͰෳ਺ͷGPUΛར༻Մೳ
    ɾ࣮ߦ଎౓ׂ͕Γͱૣ͍
    ɾωοτϫʔΫͷύϑΥʔϚϯεςετ͕Ͱ͖Δ(caffe testίϚϯυ)

    View full-size slide

  10. ■ͭΒ͍ͱ͜Ζ(ݸਓతײ૝)
    ɾΧελϚΠζ͕c++ͱProtocol BufferɻɻɻϚξͭΒ͌Ηɻɻɻʢˎ̍ʣ
    =>ਓ͕ΧελϚΠζͨ͠΋ͷ͸ɺ΋͸΍Θ͔ΒΜɻ
    ɾωοτϫʔΫΛProtocol BufferͰॻ͘ͷ͕ͭΒ͍ʢˎ̎ʣ
    =>GoogLeNet͸໿2000ߦɺResNet͸໿7000ߦɻɻɻ
    ʢProtocol Buffer৬ਓܳʣ
    ɾσʔληοτΛ࡞੒͢Δͷ͕େม
    ɾΤϥʔ͕Θ͔Γʹ͍͘
    ɾιʔείʔυΛಡΊͳ͍ͱશػೳ͸࢖͑ͳ͍ɺಈ͖͕Θ͔Βͳ͍
    ʢυΩϡϝϯτߋ৽͠Ζʂʣ
    ɾΠϯετʔϧ͕ͭΒ͍
    ʢੲʹൺ΂Δͱґଘؔ܎ͷOnOffͷΦϓγϣϯ͕෇͍ͨͨΊɺ
    ͍ͩͿϚγʣ
    ɾRNNΛѻ͏͜ͱ͸Ͱ͖ͳ͍ʢຐվ଄͞ΕͨCaffeϕʔεͷ΋ͷ͸͋Δ͚Ͳɻɻɻʣ

    View full-size slide

  11. ■༨ஊʢˎ̍ʣ
    ɾ࠷ۙ͸Python Layer͕௥Ճ͞ΕͯPython͚ͩͰ΋ΧελϚΠζ
    Ͱ͖ΔΑ͏ʹຐվ଄޻෉͍ͯ͠Δ(No Documentation)

    View full-size slide

  12. ■༨ஊʢˎ̎ʣ
    ɾPythonͰProtocol BufferΛੜ੒
    Ͱ͖ΔΑ͏ʹͳͬͨͨΊɺ
    ϧʔϓͨ͠هड़ׂ͕ͱ؆୯ʹͳͬͨ
    (No Documentation)

    View full-size slide

  13. ■͜Μͳͻͱʹ͓͢͢Ί
    ɾ·ͣԿ͔ಈ͔͍ͨ͠ਓ
    ɾͱΓ͋͑ͣݚڀ੒ՌΛࢼ͍ͨ͠ਓ
    ɾ଎౓͕ඞཁͳਓ
    ɾC++ͱProtocol BufferΛษڧ͍ͨ͠ਓ
    ɾࠜؾڧ͘Կ͔ͱઓ͍͍ͨਓ

    View full-size slide

  14. Tensorflow:
    G̋̋gle੡ͷ෼ࢄߦྻܭࢉϥΠϒϥϦɻ
    ผʹDeep͚ͩ͡Όͳ͍Μ͔ͩΒͶʂ
    ݴޠ:
    ɾCore͸C++ɻPythonͱC++ͲͪΒͰ΋ಈ͘ɻ
    ಛ௃:
    ɾ෼ࢄॲཧ͕؆୯ʹͰ͖Δ
    ɾGoogleͷϓϩμΫτͰԿ೥΋ར༻͞Ε͍ͯͯɺ҆ఆײ͕͋Δ

    View full-size slide

  15. ■͍͍ͱ͜Ζ(ݸਓతײ૝)
    ɾ෼ࢄॲཧ͕ΊͬͪΌ؆୯ʹͰ͖Δ(Distributed Tensorflow)
    ɾGoogle͕MLϓϥοτϑΥʔϜΛఏڙ։࢝
    ɾ࠷ۙɺTensorflow࢖͍·ͨ͠࿦จ͕Α͘Ͱ͖͍ͯͯΔ
    ɾίΞ͕C++ͳͷͰAndroidͰ΋ಈ࡞͢Δ
    ɾDocker Container͕མͪͯΔͷͰɺDocker࢖͑ΔͳΒ
    ΠϯετʔϧʹࠔΒͳ͍
    ɾTensorboard͕ΦγϟϨ

    View full-size slide

  16. ■ͭΒ͍ͱ͜Ζ(ݸਓతײ૝)
    ɾݰਓ޲͚ϥΠϒϥϦ
    =>࢓૊Έ͕ͪΐͬͱෳࡶͳͷͰཧղ͠ͳ͍ͱ࢖͍͜ͳͤͳ͍
    =>ωοτϫʔΫΛॻ͘ͷʹҰ͔Βهड़͢Δඞཁ͕͋ΔɺTheanoతͳཱͪҐஔ
    ɾιʔείʔυ͕େن໛ͳͨΊվ଄͕େมͦ͏
    ʢҰԠυΩϡϝϯτ͸͋Δ͚Ͳʣ
    ɾDistributed TensorflowΛݸਓͷࢿݯͰ׆༻͢Δͷ͸ࠔ೉ͳͷͰɺGoogleͷϓϥοτ
    ϑΥʔϜΛ࢖Θͳ͍ͱԸܙΛड͚ʹ͍͘
    =>෼ࢄίϯϐϡʔςΟϯάͷIOϘτϧωοΫɺInfiniBandΛ͍ͬͺ͍ങ͑Δ͓ۚ࣋ͪ
    ͳΒԸܙΛड͚ΒΕΔ͔΋

    View full-size slide

  17. ■͜Μͳͻͱʹ͓͢͢Ί
    ɾ࢓૊Έͷ෦෼͔ΒDeep LearningΛษڧ͍ͨ͠ਓ
    ɾDeep Learningɹதʙ্ڃऀ޲͚ͷਓ
    ɾେن໛ػցֶशΛ΍ͬͯΈ͍ͨਓ
    ɾେن໛ػցֶशج൫Λ࡞Γ͍ͨਓ
    ɾMobileʹ૊ΈࠐΈ͍ͨਓ

    View full-size slide

  18. Chainer:
    PFN੡ͷDeep LearningϥΠϒϥϦɻ
    ݴޠ:
    ɾPython(+Cuda)
    ಛ௃:
    ɾDefine-by-Runͱ͍͏ख๏Λͱ͍ͬͯͯɺωοτϫʔΫΛޙ͔Βղ

    ɾ͢͹Β͍͠

    View full-size slide

  19. ■͍͍ͱ͜Ζ(ݸਓతײ૝)
    ɾωοτϫʔΫͷهड़ͷॊೈੑ͕ߴ͍
    ʢಛʹRNNܥඇৗʹॻ͖΍͍͢ʣ
    ɾ಺෦ͷಈ࡞͕Ͳ͏ͳͬͯΔ͔ඇৗʹΘ͔Γ΍͍͢
    ɾσόοΫ͠΍͍͢
    ɾφ΢͍ΞϧΰϦζϜ͕͍ͪૣ࣮͘૷͞ΕͯΔ
    ɾCupyͱ͍͏Cuda͕؆୯ʹ࢖͑ΔߦྻԋࢉϥΠϒϥϦؚ͕·Ε͍ͯ
    ͯɺࣗલͷΞϧΰϦζϜΛൺֱత؆୯ʹߴ଎ԽͰ͖Δ
    (C++Ͱॻ͍ͯϥούʔͱ͔ͭ͘Βͳ͍͍ͯ͘)
    ɾதͷਓ͕͍͢͝

    View full-size slide

  20. ■ͭΒ͍ͱ͜Ζ(ݸਓతײ૝)
    ɾωοτϫʔΫҎ֎ͷهड़ྔ͕ଟ͘ͳͬͯ͠·͏ʢֶशͷίʔυͱ͔ʣ
    ɾ࣮ߦ଎౓ʢ࠷ۙ͸ͦͦ͜͜ૣ͍ͬΆ͍ʣ
    ɾDeep Learning෼͔ͬͯͳ͍ͱଟ෼࢖͍͜ͳͤͳ͍

    View full-size slide

  21. ■͜Μͳͻͱʹ͓͢͢Ί
    ɾDeep LearingΛҰ͔ΒΨοπϦษڧ͍ͨ͠ਓ
    ɾDeep Learningɹதʙ্ڃऀͷਓ
    ɾݚڀͰTry and ErrorΛ܁Γฦ͠ͳ͕ΒΞϧΰϦζϜΛ։ൃ͍ͨ͠ਓ
    ɾෳࡶͳωοτϫʔΫΛهड़͍ͨ͠ਓ
    ʢωοτϫʔΫ಺Ͱ৚݅จॻ͖͍ͨɺσʔλʹΑͬͯॲཧΛ෼͚͍ͨ౳ʣ
    ɾRNNͱ͔NLPͱ͔Λॻ͖͍ͨ

    View full-size slide

  22. ■MXNet:
    DMLC(Distributed (Deep) Machine Learning Community)͕࡞ͬͯ
    ΔɻXGBoostͷ࡞੒ݩͱͯ͠΋༗໊ɻ
    ■ݴޠ:
    ɾCore͸C++ɻWrapper͕ͨ͘͞Μ͋ΓɺPythonɺC++ɺScalaɺ
    RɺMatlabɺJuliaͱଟݴޠରԠɻ
    ■ಛ௃:
    ɾଟݴޠʂ
    ɾ଎౓͕͔ͳΓૣ͍ʢॴײʣ
    ɾmshadow(ߦྻԋࢉ)ɺps-lite(෼ࢄॲཧ)౳ͷϥΠϒϥϦ͕ϕʔε

    View full-size slide

  23. ■͍͍ͱ͜Ζ(ݸਓతײ૝)
    ɾ෼ࢄॲཧ(1Node, Multi-GPUɺMulti-NodeɺMulti-GPUͲͪΒ
    ΋ʣ͕ΊͬͪΌ؆୯ʹͰ͖Δ(Example͋Γ)
    ɾS3΁ϞσϧσʔλΛอଘ͢Δػೳ͕͋Δ
    ɾૣ͍ʢImageNet full datasetΛGeForce GTX 980*4Ͱ8.5೔)
    ɾͳͥૣ͍͔͕υΩϡϝϯτͰྗઆ͞Ε͍ͯΔ
    ɾଟ෼࠷΋ଟݴޠ͕ਐΜͰ͍Δ
    ɾC++Ͱॻ͔ΕͯΔͷͰ
    Mobile(iOS, Android)Ͱ΋ಈ͘

    View full-size slide

  24. ■ͭΒ͍ͱ͜Ζ(ݸਓతײ૝)
    ɾΤϥʔ͕Θ͔Γʹ͍͘ɺຊ౰ʹΘ͔Γʹ͍͘
    ɾυΩϡϝϯτ͕গͳ͍
    =>ಛघͳֶशσʔλΛ࡞ͬͨΓ͢Δͷ͸େม
    =>೉͍͜͠ͱΛ͠Α͏ͱ͢ΔͱιʔεΛಡ·ͳ͚Ε͹ͳΒͳ͍

    View full-size slide

  25. ■͜Μͳͻͱʹ͓͢͢Ί
    ɾDeep Learningɹதʙ্ڃऀ޲͚ͷਓ
    ɾ଎౓ΛٻΊ͍ͯΔਓ
    ɾPythonɺC++Ҏ֎Ͱ΋ར༻͍ͨ͠ਓ

    View full-size slide

  26. ■Keras:
    PythonͷDeep LearningϥΠϒϥϦɻ
    ࠷ۙv1.0͕ϦϦʔε͞Εͨɻ
    ■ݴޠ:
    ɾPython
    ■ಛ௃:
    ɾTorchʹࣅͨهड़ํ๏ɻ
    ɾߦྻԋࢉͷόοΫΤϯυ͸TheanoͱTensorFlowΛར༻͍ͯͯ͠ɺ
    ੾Γସ͑Δ͜ͱ͕Ͱ͖Δ

    View full-size slide

  27. ■͍͍ͱ͜Ζ(ݸਓతײ૝)
    ɾωοτϫʔΫهड़͕؆୯ɺॊೈ
    ϕʔεͷAPI͕ͨ͘͞Μ४උ͞Ε͍ͯΔͨΊɺهड़ྔ΋গͳ͘ࡁ
    Ήɻ؆୯ͳωοτϫʔΫͰ͋Ε͹API૊Έ߹ΘͤͰͳΜͱ͔ͳΔɻ
    v1.0.0͔Β functional APIͳΔ΋ͷ͕ग़དྷͯɺ
    ඇৗʹ௚ײతʹωοτϫʔΫΛهड़Ͱ͖ΔΑ͏ʹͳͬͨ
    ɾֶश͕؆୯
    ScikitͷΑ͏ʹfit()ؔ਺ݺͼग़ͤ͹ΑΖ͘͠΍ͬͯ͘ΕΔ
    ɾιʔε͕ಡΈ΍͍͢

    View full-size slide

  28. ■ͭΒ͍ͱ͜Ζ(ݸਓతײ૝)
    ɾMulti-GPUඇରԠ
    TheanoΛBackendͱͯ͠࢖ͬͯΔͱMulti-GPUͭΒ͍ɻ
    Tensorflowͷ͓͔͛ͰMulti-GPU͕؆୯ʹͰ͖ΔΑ͏ʹͳͬͨʁ
    ɾPython͔͠ରԠ͍ͯ͠ͳ͍

    View full-size slide

  29. ■͜Μͳͻͱʹ͓͢͢Ί
    ɾDeep LearningΛ΍Γ͍ͨਓશൠ
    ɾ͋·Γࡉ͔͍͜ͱ͸ؾʹͤͣʹαΫοͱωοτϫʔΫΛ࡞
    Γ͍ͨਓ
    ˎݸਓతʹ͸Ұ൪͓͢͢Ί

    View full-size slide

  30. ·ͱΊ
    ■Caffe
    ɾͱΓ͋͑ͣDeep LearingʢCNNʣ΍Γ͍ͨਓ
    ɾݚڀ੒ՌΛࢼ͍ͨ͠ਓ
    ■Tensorflow
    ɾ෼ࢄίϯϐϡʔςΟϯά΍Γ͍ͨਓ
    ■Chainer
    ɾΞϧΰϦζϜ։ൃ͍ͨ͠ਓ
    ɾຊؾͰDeep LearningΛษڧ͍ͨ͠ਓ
    ■MXNet
    ɾ଎౓͕ඞཁͳਓ
    ɾMobileͰಈ͔͍ͨ͠ਓ
    ■Keras
    ɾͱΓ͋͑ͣDeep Learingษڧ͍ͨ͠ਓ
    ɾΊΜͲ͍͘͞ͷͰ͋Δఔ౓؀ڥ͕४උ͞Ε͍ͯͯཉ͍͠ͱࢥ͏ਓ

    View full-size slide

  31. We are hiring!

    → https://www.wantedly.com/companies/abeja

    View full-size slide