Deep LearningϥΠϒϥϦ৭ʑ͔ͭͬͯΈͨײ·ͱΊ@conta_
View Slide
Self Introductionॹํɹول (twitter: @conta_)[email protected], Inc.Computer Visionͱ͔ɺMachine LearningΛͬͨϓϩμΫτ։ൃΛ͍ͬͯ·͢ɻ
Deep Learning Library?
ʊਓਓਓਓਓਓਓਓਓʊʼɹଟ͗ͯͭ͢Β͍ɹʻʉY^Y^Y^Y^Y^Y^Y^Yʉ
Dive into Deep Learning
ˎײ͡ํʹݸਓ͕ࠩ͋Γ·͢
ࠓճհ͢ΔϥΠϒϥϦ
CaffeCaffe:UC Berkleyͷਓ͕࡞ͬͯΔɻDeepLearningք۾Ͱ͔ͳΓฮͳϥΠϒϥϦͰɺޭେ͖͍ݴޠ:ɾCoreC++ɻPython, MatlabͷWrapper͕͋Δಛ:ɾجຊతʹProtocol BufferͰωοτϫʔΫΛهड़
■͍͍ͱ͜Ζ(ݸਓతײ)ɾModel ZooʹֶशࡁΈϞσϧ͕ͨ͘͞Μެ։͞ΕͯΔʢطʹCVPR2016ͷจͷϞσϧެ։͞ΕͯΔʣɾݚڀऀׂ͕ͱͬͯΔͷͰ࠷৽ͷݚڀՌ͕CaffeͰ࣮͞ΕͯͨΓ͢ΔɾMulti-GPUʹରԠͨ͠ͷͰɺઃఆ̍ͭͰෳͷGPUΛར༻Մೳɾ࣮ߦׂ͕Γͱૣ͍ɾωοτϫʔΫͷύϑΥʔϚϯεςετ͕Ͱ͖Δ(caffe testίϚϯυ)
■ͭΒ͍ͱ͜Ζ(ݸਓతײ)ɾΧελϚΠζ͕c++ͱProtocol BufferɻɻɻϚξͭΒ͌Ηɻɻɻʢˎ̍ʣ=>ਓ͕ΧελϚΠζͨ͠ͷɺΘ͔ΒΜɻɾωοτϫʔΫΛProtocol BufferͰॻ͘ͷ͕ͭΒ͍ʢˎ̎ʣ=>GoogLeNet2000ߦɺResNet7000ߦɻɻɻʢProtocol Buffer৬ਓܳʣɾσʔληοτΛ࡞͢Δͷ͕େมɾΤϥʔ͕Θ͔Γʹ͍͘ɾιʔείʔυΛಡΊͳ͍ͱશػೳ͑ͳ͍ɺಈ͖͕Θ͔Βͳ͍ʢυΩϡϝϯτߋ৽͠ΖʂʣɾΠϯετʔϧ͕ͭΒ͍ʢੲʹൺΔͱґଘؔͷOnOffͷΦϓγϣϯ͕͍ͨͨΊɺ͍ͩͿϚγʣɾRNNΛѻ͏͜ͱͰ͖ͳ͍ʢຐվ͞ΕͨCaffeϕʔεͷͷ͋Δ͚Ͳɻɻɻʣ
■༨ஊʢˎ̍ʣɾ࠷ۙPython Layer͕Ճ͞ΕͯPython͚ͩͰΧελϚΠζͰ͖ΔΑ͏ʹຐվ͍ͯ͠Δ(No Documentation)
■༨ஊʢˎ̎ʣɾPythonͰProtocol BufferΛੜͰ͖ΔΑ͏ʹͳͬͨͨΊɺϧʔϓͨ͠هड़ׂ͕ͱ؆୯ʹͳͬͨ(No Documentation)
■͜Μͳͻͱʹ͓͢͢Ίɾ·ͣԿ͔ಈ͔͍ͨ͠ਓɾͱΓ͋͑ͣݚڀՌΛࢼ͍ͨ͠ਓɾ͕ඞཁͳਓɾC++ͱProtocol BufferΛษڧ͍ͨ͠ਓɾࠜؾڧ͘Կ͔ͱઓ͍͍ͨਓ
Tensorflow:G̋̋gleͷࢄߦྻܭࢉϥΠϒϥϦɻผʹDeep͚ͩ͡Όͳ͍Μ͔ͩΒͶʂݴޠ:ɾCoreC++ɻPythonͱC++ͲͪΒͰಈ͘ɻಛ:ɾࢄॲཧ͕؆୯ʹͰ͖ΔɾGoogleͷϓϩμΫτͰԿར༻͞Ε͍ͯͯɺ҆ఆײ͕͋Δ
■͍͍ͱ͜Ζ(ݸਓతײ)ɾࢄॲཧ͕ΊͬͪΌ؆୯ʹͰ͖Δ(Distributed Tensorflow)ɾGoogle͕MLϓϥοτϑΥʔϜΛఏڙ։࢝ɾ࠷ۙɺTensorflow͍·ͨ͠จ͕Α͘Ͱ͖͍ͯͯΔɾίΞ͕C++ͳͷͰAndroidͰಈ࡞͢ΔɾDocker Container͕མͪͯΔͷͰɺDocker͑ΔͳΒΠϯετʔϧʹࠔΒͳ͍ɾTensorboard͕ΦγϟϨ
■ͭΒ͍ͱ͜Ζ(ݸਓతײ)ɾݰਓ͚ϥΠϒϥϦ=>Έ͕ͪΐͬͱෳࡶͳͷͰཧղ͠ͳ͍ͱ͍͜ͳͤͳ͍=>ωοτϫʔΫΛॻ͘ͷʹҰ͔Βهड़͢Δඞཁ͕͋ΔɺTheanoతͳཱͪҐஔɾιʔείʔυ͕େنͳͨΊվ͕େมͦ͏ʢҰԠυΩϡϝϯτ͋Δ͚ͲʣɾDistributed TensorflowΛݸਓͷࢿݯͰ׆༻͢ΔͷࠔͳͷͰɺGoogleͷϓϥοτϑΥʔϜΛΘͳ͍ͱԸܙΛड͚ʹ͍͘=>ࢄίϯϐϡʔςΟϯάͷIOϘτϧωοΫɺInfiniBandΛ͍ͬͺ͍ങ͑Δ͓ۚ࣋ͪͳΒԸܙΛड͚ΒΕΔ͔
■͜Μͳͻͱʹ͓͢͢ΊɾΈͷ෦͔ΒDeep LearningΛษڧ͍ͨ͠ਓɾDeep Learningɹதʙ্ڃऀ͚ͷਓɾେنػցֶशΛͬͯΈ͍ͨਓɾେنػցֶशج൫Λ࡞Γ͍ͨਓɾMobileʹΈࠐΈ͍ͨਓ
Chainer:PFNͷDeep LearningϥΠϒϥϦɻݴޠ:ɾPython(+Cuda)ಛ:ɾDefine-by-Runͱ͍͏ख๏Λͱ͍ͬͯͯɺωοτϫʔΫΛޙ͔Βղऍɾ͢Β͍͠
■͍͍ͱ͜Ζ(ݸਓతײ)ɾωοτϫʔΫͷهड़ͷॊೈੑ͕ߴ͍ʢಛʹRNNܥඇৗʹॻ͖͍͢ʣɾ෦ͷಈ࡞͕Ͳ͏ͳͬͯΔ͔ඇৗʹΘ͔Γ͍͢ɾσόοΫ͍͢͠ɾφ͍ΞϧΰϦζϜ͕͍ͪૣ࣮͘͞ΕͯΔɾCupyͱ͍͏Cuda͕؆୯ʹ͑ΔߦྻԋࢉϥΠϒϥϦؚ͕·Ε͍ͯͯɺࣗલͷΞϧΰϦζϜΛൺֱత؆୯ʹߴԽͰ͖Δ(C++Ͱॻ͍ͯϥούʔͱ͔ͭ͘Βͳ͍͍ͯ͘)ɾதͷਓ͕͍͢͝
■ͭΒ͍ͱ͜Ζ(ݸਓతײ)ɾωοτϫʔΫҎ֎ͷهड़ྔ͕ଟ͘ͳͬͯ͠·͏ʢֶशͷίʔυͱ͔ʣɾ࣮ߦʢ࠷ۙͦͦ͜͜ૣ͍ͬΆ͍ʣɾDeep Learning͔ͬͯͳ͍ͱଟ͍͜ͳͤͳ͍
■͜Μͳͻͱʹ͓͢͢ΊɾDeep LearingΛҰ͔ΒΨοπϦษڧ͍ͨ͠ਓɾDeep Learningɹதʙ্ڃऀͷਓɾݚڀͰTry and ErrorΛ܁Γฦ͠ͳ͕ΒΞϧΰϦζϜΛ։ൃ͍ͨ͠ਓɾෳࡶͳωοτϫʔΫΛهड़͍ͨ͠ਓʢωοτϫʔΫͰ݅จॻ͖͍ͨɺσʔλʹΑͬͯॲཧΛ͚͍ͨʣɾRNNͱ͔NLPͱ͔Λॻ͖͍ͨ
■MXNet:DMLC(Distributed (Deep) Machine Learning Community)͕࡞ͬͯΔɻXGBoostͷ࡞ݩͱͯ͠༗໊ɻ■ݴޠ:ɾCoreC++ɻWrapper͕ͨ͘͞Μ͋ΓɺPythonɺC++ɺScalaɺRɺMatlabɺJuliaͱଟݴޠରԠɻ■ಛ:ɾଟݴޠʂɾ͕͔ͳΓૣ͍ʢॴײʣɾmshadow(ߦྻԋࢉ)ɺps-lite(ࢄॲཧ)ͷϥΠϒϥϦ͕ϕʔε
■͍͍ͱ͜Ζ(ݸਓతײ)ɾࢄॲཧ(1Node, Multi-GPUɺMulti-NodeɺMulti-GPUͲͪΒʣ͕ΊͬͪΌ؆୯ʹͰ͖Δ(Example͋Γ)ɾS3ϞσϧσʔλΛอଘ͢Δػೳ͕͋Δɾૣ͍ʢImageNet full datasetΛGeForce GTX 980*4Ͱ8.5)ɾͳͥૣ͍͔͕υΩϡϝϯτͰྗઆ͞Ε͍ͯΔɾଟ࠷ଟݴޠ͕ਐΜͰ͍ΔɾC++Ͱॻ͔ΕͯΔͷͰMobile(iOS, Android)Ͱಈ͘
■ͭΒ͍ͱ͜Ζ(ݸਓతײ)ɾΤϥʔ͕Θ͔Γʹ͍͘ɺຊʹΘ͔Γʹ͍͘ɾυΩϡϝϯτ͕গͳ͍=>ಛघͳֶशσʔλΛ࡞ͬͨΓ͢Δͷେม=>͍͜͠ͱΛ͠Α͏ͱ͢ΔͱιʔεΛಡ·ͳ͚ΕͳΒͳ͍
■͜Μͳͻͱʹ͓͢͢ΊɾDeep Learningɹதʙ্ڃऀ͚ͷਓɾΛٻΊ͍ͯΔਓɾPythonɺC++Ҏ֎Ͱར༻͍ͨ͠ਓ
■Keras:PythonͷDeep LearningϥΠϒϥϦɻ࠷ۙv1.0͕ϦϦʔε͞Εͨɻ■ݴޠ:ɾPython■ಛ:ɾTorchʹࣅͨهड़ํ๏ɻɾߦྻԋࢉͷόοΫΤϯυTheanoͱTensorFlowΛར༻͍ͯͯ͠ɺΓସ͑Δ͜ͱ͕Ͱ͖Δ
■͍͍ͱ͜Ζ(ݸਓతײ)ɾωοτϫʔΫهड़͕؆୯ɺॊೈϕʔεͷAPI͕ͨ͘͞Μ४උ͞Ε͍ͯΔͨΊɺهड़ྔগͳ͘ࡁΉɻ؆୯ͳωοτϫʔΫͰ͋ΕAPIΈ߹ΘͤͰͳΜͱ͔ͳΔɻv1.0.0͔Β functional APIͳΔͷ͕ग़དྷͯɺඇৗʹײతʹωοτϫʔΫΛهड़Ͱ͖ΔΑ͏ʹͳͬͨɾֶश͕؆୯ScikitͷΑ͏ʹfit()ؔݺͼग़ͤΑΖͬͯ͘͘͠ΕΔɾιʔε͕ಡΈ͍͢
■ͭΒ͍ͱ͜Ζ(ݸਓతײ)ɾMulti-GPUඇରԠTheanoΛBackendͱͯͬͯ͠ΔͱMulti-GPUͭΒ͍ɻTensorflowͷ͓͔͛ͰMulti-GPU͕؆୯ʹͰ͖ΔΑ͏ʹͳͬͨʁɾPython͔͠ରԠ͍ͯ͠ͳ͍
■͜Μͳͻͱʹ͓͢͢ΊɾDeep LearningΛΓ͍ͨਓશൠɾ͋·Γࡉ͔͍͜ͱؾʹͤͣʹαΫοͱωοτϫʔΫΛ࡞Γ͍ͨਓˎݸਓతʹҰ൪͓͢͢Ί
·ͱΊ■CaffeɾͱΓ͋͑ͣDeep LearingʢCNNʣΓ͍ͨਓɾݚڀՌΛࢼ͍ͨ͠ਓ■TensorflowɾࢄίϯϐϡʔςΟϯάΓ͍ͨਓ■ChainerɾΞϧΰϦζϜ։ൃ͍ͨ͠ਓɾຊؾͰDeep LearningΛษڧ͍ͨ͠ਓ■MXNetɾ͕ඞཁͳਓɾMobileͰಈ͔͍ͨ͠ਓ■KerasɾͱΓ͋͑ͣDeep Learingษڧ͍ͨ͠ਓɾΊΜͲ͍͘͞ͷͰ͋Δఔڥ͕४උ͞Ε͍ͯͯཉ͍͠ͱࢥ͏ਓ
We are hiring!→ https://www.wantedly.com/companies/abeja