What is Deep Learning ?

(Japanese document)
History and introductions of Neural Network,
社内ゼミでの資料です。

Transcript

1. σΟʔϓϥʔχϯάͬͯԿʁ urakarin@gmail.com 2017.02.08
2. ࿩͢͜ͱɺ࿩͞ͳ͍͜ͱ • ࿩͢͜ͱ • χϡʔϥϧωοτϫʔΫͷ਺ֶతͳ࢓૊Έ • ॳظ஋ͷܾΊํɺධՁํ๏ • ύϥϝʔλྔɺܭࢉྔͷϘϦϡʔϜײ • ϗοτͳ࿩୊ • ࿩͞ͳ͍͜ͱ • πʔϧͷ࿩ • ਺ࣜͷ࿩ • χϡʔϥϧωοτϫʔΫҎ֎ͷػցֶश • γϯΪϡϥϦςΟͳͲͷਓ޻஌ೳͷະདྷ ग़య wedge.ismedia.jp
3. Agenda • σΟʔϓϥʔχϯάͱ͸ʁ • ྺ࢙ • χϡʔϥϧωοτϫʔΫ͔ΒσΟʔϓͳχϡʔϥϧωοτϫʔΫ΁ • ୈҰ࣍AIϒʔϜ • ୈೋ࣍AIϒʔϜ • ୈࡾ࣍AIϒʔϜ • Ԡ༻ྫ • ·ͱΊ
4. • ਂ૚ֶशͱ΋ݴ͏ • ೴ʢਆܦࡉ๔ʣͷಇ͖Λ໛ֶͨ͠शΞϧΰϦζϜͰ͋Δ
 Neural Network(NN)Λ༻͍ͨਓ޻஌ೳͷߏஙٕज़ͷ૯শ • ͦͷதͰ΋ਂ͘େن໛ͳߏ଄Λ࣋ͭ͜ͱ͕ಛ௃
 
 σΟʔϓϥʔχϯάͱ͸ʁ GoogLeNet, 22Layers (ILSVRC 2014)
5. ༻ޠͷؔ܎ੑ ਓ޻஌ೳʢAIʣ ػցֶश χϡʔϥϧωοτϫʔΫ ਂ૚ֶश
6. ୅දతͳൃද Neural Networkͷ ϒϨʔΫεϧʔͱ ౙͷ࣌୅ ୅දతͳਓࡐ֫ಘ Google ͕DNN ResearchΛങऩ )JOUPO Google ͕ Deep MindΛങऩ Baidu ͕Institute of Deep LearningΛઃཱ "OESFX
   /H Facebook ͕AI Research Lab.Λઃཱ -F$VO SGD (Amari) Neocognitron (Fukushima) Boltzmann Machine (Hinton+) Conv. net (LeCun+) Sparse Coding (Olshausen&Field) 1950 1960 1970 1980 1990 2000 2010 2020 Microsoft ͕MaluubaΛങऩ #FOHJP ୈҰ࣍"*ϒʔϜ ਪ࿦ɾ୳ࡧ ୈೋ࣍"*ϒʔϜ ஌ࣝදݱ
   &YQFSU
   4ZTUFN ୈࡾ࣍"*ϒʔϜ ػցֶश ਂ૚ֶश Perceptron (Rosenblatt) Back Propagation (Rumelhart) Deep Learning (Hinton+) Big Data GPU Cloud Computing ઢܗ෼཭ෆՄೳ໰୊ YPS͕ղ͚ͳ͍ ஗͍ɺաֶशɺ 47.ਓؾ χϡʔϥϧωοτϫʔΫͷྺ࢙ NN ୈҰͷౙ NN ୈೋͷౙ
7. Torontoେֶ New Yorkେֶ Montrealେֶ
8. NN ͔Β DNN ΁ Neural Network Deep Neural Network
9. ୈҰ࣍AIϒʔϜ
10. ୯७ύʔηϓτϩϯ ʹྖҬ൑ఆثͱͯ͠ͷχϡʔϩϯ
11. NAND AND OR XOR ୯७ύʔηϓτϩϯ
12. ୈҰͷౙ • xor͕දݱͰ͖ͳ͍
13. ୈೋ࣍AIϒʔϜ
14. ڭࢣ৴߸ ޡࠩؔ਺ ೖྗ૚ ग़ྗ૚ தؒ૚ 1 1 1 x y t ⇥w 4 ⇥ w3 ⇥w2 ⇥w1 x1 x2 x3 x4 ⇥w 0 ⌃f y1 y2 y3 1. ଟ૚Խ 2. ׆ੑԽؔ਺ 3. ޡࠩؔ਺ 4. ޡࠩٯ఻ൖ๏ ଟ૚ύʔηϓτϩϯʢMLPʣ
15. ଟ૚ԽʹΑͬͯxorͷ࣮ݱ NAND OR AND s2 s1 x1 x2 y x1 x2 s1 s2 y 0 0 1 0 0 1 0 1 1 1 0 1 1 1 1 1 1 0 1 0 = 1. ଟ૚Խ
16. γάϞΠυؔ਺ɾ૒ۂઢਖ਼઀ؔ਺ ඍ෼͕Ͱ͖ͳ͍ ֶशͰ͖ͳ͍ ʢޡࠩٯ఻ൖ๏ʣ ʹೖྗ৴߸ͷ૯࿨Λग़ྗ৴߸ʹม׵͢Δؔ਺ ׆ੑԽؔ਺ 2. ׆ੑԽؔ਺ 1 ⇥w 4 ⇥ w3 ⇥w2 ⇥w1 x1 x2 x3 x4 ⇥w 0 ⌃f εςοϓؔ਺ ύʔηϓτϩϯͷ৔߹
17. 3. ଛࣦؔ਺ ޡࠩؔ਺ʢଛࣦؔ਺ʣ 1 2 N X n=1 ky tk2 N Y n=1 p(dn | x ) d=0/1ͷࣄޙ֬཰pʹରͯ͠࠷໬ਪఆΛߦ͏ ೋ৐ޡࠩͱ͢Δ ڭࢣ৴߸ ޡࠩؔ਺ ग़ྗ૚ y t y1 y2 y3 ճؼ ೋ஋෼ྨ ଟΫϥε෼ྨ ڭࢣ৴߸ΛOne-hotදݱͱ͠ɺ ࠷ऴஈͷ׆ੑԽؔ਺ΛιϑτϚοΫεؔ਺ͱ্ͨ͠Ͱ ަࠩΤϯτϩϐʔؔ਺
18. ڭࢣ৴߸ ޡࠩؔ਺ ೖྗ૚ ग़ྗ૚ தؒ૚ 1 1 1 x y t ⇥w 4 ⇥ w3 ⇥w2 ⇥w1 x1 x2 x3 x4 ⇥w 0 ⌃f y1 y2 y3 4. ޡࠩٯ఻ൖ๏ ޡࠩٯ఻ൖ๏
19. + ^2 x y t z @z @z @z @z @z @t @z @z @z @t @t @x ͨͱ͑͹ z = ( x + y )2 ͱ͍͏ࣜ͸ z = t2 t = x + y ͱ͍͏2ͭͷࣜͰߏ੒͞ΕΔɻ ࿈࠯཯ͱ͸ɺ߹੒ؔ਺ͷඍ෼ʹ͍ͭͯͷੑ࣭Ͱ͋Δ @z @x = @z @t @t @x ޡࠩٯ఻ൖ๏ 4. ޡࠩٯ఻ൖ๏
20. ޡࠩٯ఻ൖ๏ Ճࢉϊʔυͷٯ఻ൖ + x y z + @L @z @L @z · 1 @L @z · 1 ৐ࢉϊʔυͷٯ఻ൖ x y z ⇥ @L @z ⇥ @L @z · x @L @z · y 4. ޡࠩٯ఻ൖ๏ 2 100 ⇥ ⇥ 200 1.1 220 1 1.1 200 110 2.2 ΓΜ͝ͷݸ਺ ফඅ੫ ɹ۩ମྫɹ
21. ֬཰తޯ഑߱Լ๏ • ϛχόονֶश • ֶश܎਺ͷߋ৽ํ๏ • Momentum • AdaGrad • Adam • RMSProp
22. ୈೋͷౙ • ܭࢉྔ͕ଟ͗ͯ͢஗͍ • ہॴղɾաֶशʹؕΓ΍͍͢ • ޯ഑ফࣦ໰୊
23. ୈࡾ࣍AIϒʔϜ
24. Deep Belief Network vs Auto Encoder ہॴղɾաֶश ରࡦ
25. Auto Encoder Deep Belief Network v3 h2 v1 h1 v2 Visible Hidden Visible Hidden Visible Hidden RBM RBM RBM 4UBDLFE "VUP
    &ODPEFS "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث ଟஈԽ ʴϩόετੑ "VUP
    &ODPEFS ࣗݾූ߸Խث AE %FOPJTJOH "VUP
    &ODPEFS DAE SAE pre-training + fine tuning ࠾༻ ֶश Input Hidden Output %FFQ
    #FMJFG
    /FUXPSL )PQpFME /FUXPSL #PMU[NBOO
    .BDIJOF ֬཰Ϟσϧ ܭࢉྔͷ࡟ݮ ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ଟஈԽ ੍໿෇͖ #PMU[NBOO
    .BDIJOF RBM DBN pre-training + fine tuning
26. ୅දతͳൃද Neural Networkͷ ϒϨʔΫεϧʔͱ ౙͷ࣌୅ ୅දతͳਓࡐ֫ಘ Google ͕DNN ResearchΛങऩ )JOUPO Google ͕ Deep MindΛങऩ Baidu ͕Institute of Deep LearningΛઃཱ "OESFX
    /H Facebook ͕AI Research Lab.Λઃཱ -F$VO SGD (Amari) Neocognitron (Fukushima) Boltzmann Machine (Hinton+) Conv. net (LeCun+) Sparse Coding (Olshausen&Field) 1950 1960 1970 1980 1990 2000 2010 2020 Microsoft ͕MaluubaΛങऩ #FOJHO ୈҰ࣍"*ϒʔϜ ਪ࿦ɾ୳ࡧ ୈೋ࣍"*ϒʔϜ ஌ࣝදݱ
    &YQFSU
    4ZTUFN ୈࡾ࣍"*ϒʔϜ ػցֶश ਂ૚ֶश Perceptron (Rosenblatt) Back Propagation (Rumelhart) Deep Learning (Hinton+) Big Data GPU Cloud Computing ઢܗ෼཭ෆՄೳ໰୊ YPS͕ղ͚ͳ͍ ஗͍ɺաֶशɺ 47.ਓؾ χϡʔϥϧωοτϫʔΫͷྺ࢙ NN ୈҰͷౙ NN ୈೋͷౙ
27. ωο τϫʔΫͷΤωϧΪʔ͕࠷খʹͳΔΑ͏ʹঢ়ଶมԽΛ܁Γฦ͢ %FFQ
    #FMJFG
    /FUXPSL )PQpFME /FUXPSL #PMU[NBOO
    .BDIJOF ֬཰Ϟσϧ ܭࢉྔͷ࡟ݮ ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ଟஈԽ ੍໿෇͖ #PMU[NBOO
    .BDIJOF RBM DBN pre-training + fine tuning هԱ1 هԱ2 هԱΛࢥ͍ग़͢ ͍ۙ͠σʔλΛ༩͑Δͱ… ը૾ΛهԱͨ͠ωοτϫʔΫ Hopfield Networkͱ͸ هԱΛߦྻܭࢉͰγϛϡϨʔτͯ͠ΈΑ͏ http://www.gaya.jp/spiking_neuron/matrix.htm
28. %FFQ
    #FMJFG
    /FUXPSL )PQpFME /FUXPSL #PMU[NBOO
    .BDIJOF ֬཰Ϟσϧ ܭࢉྔͷ࡟ݮ ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ଟஈԽ ੍໿෇͖ #PMU[NBOO
    .BDIJOF RBM DBN pre-training + fine tuning Boltzmann Machineͱ͸ ֬཰Ϟσϧͷಋೖ Kullback LeiblerμΠόʔδΣϯε 2ͭͷۂઢʹ͍ͭͯɺॏͳΒͣʹ૬ҧʢμΠόʔδΣϯεʣ͍ͯ͠ΔྖҬʢࠩʣΛ࠷খԽ͢Δɻ ࣮ࡍͷೖྗ஋ʹ ΑΔ֬཰෼෍p ෮ݩ͞Εͨ෼෍q ࠩͷੵ෼
29. %FFQ
    #FMJFG
    /FUXPSL )PQpFME /FUXPSL #PMU[NBOO
    .BDIJOF ֬཰Ϟσϧ ܭࢉྔͷ࡟ݮ ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ଟஈԽ ੍໿෇͖ #PMU[NBOO
    .BDIJOF RBM DBN pre-training + fine tuning ੍໿෇͖Boltzmann Machine (RBN)ͱ͸ v3 h2 v1 h1 v2 Visible Hidden
30. %FFQ
    #FMJFG
    /FUXPSL )PQpFME /FUXPSL #PMU[NBOO
    .BDIJOF ֬཰Ϟσϧ ܭࢉྔͷ࡟ݮ ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ଟஈԽ ੍໿෇͖ #PMU[NBOO
    .BDIJOF RBM DBN pre-training + fine tuning Deep Belief Network (DBN)ͱ͸ Visible Hidden Visible Hidden Visible Hidden RBM RBM RBM pre-training(ڭࢣͳ͠) + fine tuning (ڭࢣ͋Γ)
31. Auto Encoder Deep Belief Network v3 h2 v1 h1 v2 Visible Hidden Visible Hidden Visible Hidden RBM RBM RBM 4UBDLFE "VUP
    &ODPEFS "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث ଟஈԽ ʴϩόετੑ "VUP
    &ODPEFS ࣗݾූ߸Խث AE %FOPJTJOH "VUP
    &ODPEFS DAE SAE pre-training + fine tuning ࠾༻ ֶश Input Hidden Output %FFQ
    #FMJFG
    /FUXPSL )PQpFME /FUXPSL #PMU[NBOO
    .BDIJOF ֬཰Ϟσϧ ܭࢉྔͷ࡟ݮ ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ଟஈԽ ੍໿෇͖ #PMU[NBOO
    .BDIJOF RBM DBN pre-training + fine tuning
32. 4UBDLFE "VUP
    &ODPEFS "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث ଟஈԽ ʴϩόετੑ "VUP
    &ODPEFS ࣗݾූ߸Խث AE %FOPJTJOH "VUP
    &ODPEFS DAE SAE pre-training + fine tuning ࠾༻ ֶश Input Hidden Output Auto Encoder (AE)ͱ͸
33. 4UBDLFE "VUP
    &ODPEFS "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث ଟஈԽ ʴϩόετੑ "VUP
    &ODPEFS ࣗݾූ߸Խث AE %FOPJTJOH "VUP
    &ODPEFS DAE SAE pre-training + fine tuning Denoising Auto Encoder (DAE)ͱ͸ ࠾༻ ֶश Input Hidden Output ϊΠζ
34. 4UBDLFE "VUP
    &ODPEFS "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث ଟஈԽ ʴϩόετੑ "VUP
    &ODPEFS ࣗݾූ߸Խث AE %FOPJTJOH "VUP
    &ODPEFS DAE SAE pre-training + fine tuning Stacked Auto Encoder (SAE)ͱ͸
35. Auto Encoder Deep Belief Network v3 h2 v1 h1 v2 Visible Hidden Visible Hidden Visible Hidden RBM RBM RBM 4UBDLFE "VUP
    &ODPEFS "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث "VUP
    &ODPEFS ࣗݾූ߸Խث ଟஈԽ ʴϩόετੑ "VUP
    &ODPEFS ࣗݾූ߸Խث AE %FOPJTJOH "VUP
    &ODPEFS DAE SAE pre-training + fine tuning ࠾༻ ֶश Input Hidden Output %FFQ
    #FMJFG
    /FUXPSL )PQpFME /FUXPSL #PMU[NBOO
    .BDIJOF ֬཰Ϟσϧ ܭࢉྔͷ࡟ݮ ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ੍໿෇͖ #PMU[NBOO
    .BDIJOF ଟஈԽ ੍໿෇͖ #PMU[NBOO
    .BDIJOF RBM DBN pre-training + fine tuning
36. γάϞΠυؔ਺ɾ૒ۂઢਖ਼઀ؔ਺ ޯ഑ফࣦ໰୊ ඍ෼஋ ωοτϫʔΫ͕ਂ͍ͱޯ഑͕ফ͑ͯ͠·͏ɻɻɻ
37. γάϞΠυؔ਺ɾ૒ۂઢਖ਼઀ؔ਺ ඍ෼஋ ωοτϫʔΫ͕ਂ͍ͱޯ഑͕ফ͑ͯ͠·͏ɻɻɻ ReLU (Rectified Linear Unit) ൃՐ͍ͯ͠ͳ͍ ൃՐ͍ͯ͠Δ ޯ഑ফࣦͳ͠ʹൃՐ͍ͯ͠Δ ࡉ๔ͷΈΛ௨ͬͯ఻೻͢Δɻ ޯ഑ফࣦ໰୊
38. • ϛχόονͷೖྗσʔλΛฏۉ0ɺ෼ࢄ1ͷσʔλʹม׵͢Δ޻෉ • ׆ੑԽؔ਺ͷલɺ΋͘͠͸ޙʹૠೖ͢Δ͜ͱͰσʔλ෼෍ͷภΓΛݮΒ͢͜ͱ ͕Մೳ • ޮՌ • ֶश܎਺Λେ͖͘͢Δ͜ͱ͕ՄೳʢֶशΛૣ͘ਐߦͤ͞Δʣ • ॳظ஋ʹͦΕ΄Ͳґଘ͠ͳ͍ • աֶशΛ཈੍͢Δ Batch Normalization
39. • DropOut (Drop Connect) • ΞϯαϯϒϧֶशʹରԠ • ਖ਼ଇԽ • Weight Decayʢޡࠩؔ਺ʹL2ϊϧϜΛՃ͑Δʣ • εύʔεਖ਼ଇԽ • σʔλ֦ுʢϊΠζɺฏߦҠಈɺճసɺ৭ʣ ͦͷଞͷ޻෉
40. ॳظ஋ͷܾΊํ
41. • 0ʹ͢ΔʁˠॏΈ͕ۉҰʹͳͬͯ͠·͍ॏෳͨ͠஋ʹͳͬͯ͠·͏ • ϥϯμϜͳॳظ஋͕ඞཁ • ׆ੑԽؔ਺ʹɺγάϞΠυؔ਺΍tanhؔ਺Λ࢖༻͢Δ৔߹ɺ
 ʮXavierͷॳظ஋ʯ͕ద౰ • ReLUΛ༻͍Δ৔߹͸ɺʮHeͷॳظ஋ʯ͕ద౰ ॏΈߦྻͷॳظ஋ • લ૚ͷϊʔυͷ਺͕ɹ ݸͷ৔߹ɺɹɹΛඪ४ภࠩͱ͢ΔΨ΢ε෼෍ n r 2 n • લ૚ͷϊʔυͷ਺͕ɹ ݸͷ৔߹ɺɹɹΛඪ४ภࠩͱ͢ΔΨ΢ε෼෍ n r 1 n
42. • ֤૚ͷχϡʔϩϯ਺ • όοναΠζ • ֶश܎਺ɺֶश܎਺ͷมԽ཰ • Weight decayʢՙॏݮਰʣ • DropOut཰ • ͳͲ ϋΠύʔύϥϝʔλ NNʹ͸ɺॏΈ΍όΠΞεύϥϝʔλͱ͸ผʹɺ ਓ͕ઃఆ͢΂͖ϋΠύʔύϥϝʔλ͕ଘࡏ͢Δɻ ύϥϝʔλܾఆʹ͸ଟ͘ͷࢼߦࡨޡ͕൐͍ɺ Ϟσϧͷੑೳʹ΋େ͖͘Өڹ͢Δɻ • ઐ༻ͷݕূσʔλΛ༻ҙ͢Δ • ܇࿅σʔλ΍ςετσʔλΛ࢖ͬͯੑೳධՁΛͯ͠͸͍͚ͳ͍ • ର਺εέʔϧͷൣғ͔ΒϥϯμϜʹαϯϓϦϯάͯ͠ධՁ͠ɺ
 ൣғΛߜΓࠐΜͰ͍͖ɺ࠷ޙʹͻͱͭΛϐοΫΞοϓ͢Δ σʔληοτ ܇࿅σʔλ ςετσʔλ ݕূσʔλ ֶश༻ ֶश݁Ռͷ ධՁ༻ ϋΠύʔύϥϝʔλͷධՁ༻
43. ༧ଌੑೳͷධՁ
44. ܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ςετσʔλ ܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ςετσʔλ ܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ςετσʔλ ܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ςετσʔλ ܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ςετσʔλ ܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ݕূσʔλ ݕূσʔλ ݕূσʔλ ݕূσʔλ ݕূσʔλ ϗʔϧυΞ΢τݕূ K෼ׂަࠩݕূ
 (Cross Validation) ੑೳධՁ
45. TP rate: ཅੑΛཅੑͱ൑அׂͨ͠߹ FP rate: ӄੑΛཅੑͱ൑அׂͨ͠߹ = = ROCۂઢͱAUC ROC:Receiver Operating Characteristic ʢड৴ऀૢ࡞ಛੑʣ AUC:Area under the curve ʢROCۂઢԼ໘ੵʣ Predicted Condition Positive Negative True Condition Positive TP FN (type II error) Negative FP (Type I error) TN True Positive True Negative False Positive False Negative AUC
46. True Positive True Negative False Positive False Negative ࠶ݱ཰: ཅੑΛཅੑͱ൑அׂͨ͠߹ ʢRecallʣ = ద߹཰: ཅੑͱ༧ଌͨ͠σʔλͷ͏ͪɼ࣮ࡍʹཅੑͰ͋Δ΋ͷͷׂ߹ = ʢPrecisionʣ F஋: F஋ͷ࠷େ஋͸͓͓ΉͶ෼ذ఺ਫ਼౓ͱҰக͢Δɻ ௐ࿨ฏۉɿٯ਺ͷฏۉͷٯ਺ http://www004.upp.so-net.ne.jp/s_honma/mean/harmony2.htm Predicted Condition Positive Negative True Condition Positive TP FN (type II error) Negative FP (Type I error) TN
47. Ԡ༻ྫ • ը૾ೝࣝ (CNN) • ࣗવݴޠॲཧɺԻ੠ೝࣝ (RNN) • ը૾ʹର͢ΔΩϟϓγϣϯੜ੒ (CNN + RNN) • ڧԽֶश (CNN + Qֶश) • ਂ૚ੜ੒Ϟσϧ (CNN)
48. ը૾ೝࣝ • Convolutional Neural Network (৞ΈࠐΈχϡʔϥϧωοτϫʔΫ) • Convolution૚ + Pooling૚
49. খ͞ͳը૾ͳΒ͜Ε·Ͱͷશ݁߹NNͰOK Convolution૚
50. ฏۉ ࠨӈʹ૸ΔΤοδ ্Լʹ૸ΔΤοδ ޲͖ʹؔ܎ͳ͘Τοδ * ϑΟϧλྫ Convolution૚
53. ը૾෼ྨ͸ਓؒΛ௒͑ͨ ILSVRC = 2010೥͔Β࢝·ͬͨେن໛ը૾ೝࣝͷڝٕձ 2012೥ͷILSVRCͰHintonઌੜͷνʔϜ͕Deep LearningͰѹউ 2015೥ʹ͸ILSVRCͷ݁ՌͰਓؒͷೝࣝੑೳΛ௒͑ͨɻ
54. ܭࢉྔ • CPUͱGPUͷੑೳͷҧ͍ • ಉ࣌ԋࢉՄೳ਺ʢ୯ਫ਼౓গ਺ʣ • CPU(Intel Core i7) : AVX256bit -> 8ݸ • nVIDIA Pascal GP100 : 114,688ݸ
55. ࣗવݴޠॲཧɺԻ੠ೝࣝ • Recurrent Neural Network (RNN)
56. ڧԽֶश • CNN + Qֶश + …
57. Prisma
58. Prisma σΟʔϓϥʔχϯάΛ࢖ͬͨΞʔτܥͷ ࿦จ͸ɺ͍Ζ͍Ζग़͍ͯΔ͕ Ұ൪ͷجૅͱͳΔͷ͸ Gatys et al. 2016 ࢖༻CNN͸VGG19ʢը૾෼ྨ༻ʹ܇࿅ࡁΈʣ͔Βશ݁߹Λ ൈ͍ͨ΋ͷ “Image Style Transfer Using Convolutional Neural Networks”
59. Prisma ίϯςϯπ ελΠϧ http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Gatys_Image_Style_Transfer_CVPR_2016_paper.pdf
60. Prisma ଛࣦؔ਺ʹίϯςϯπͷଛࣦʴελΠϧͷଛࣦ ࠷దԽʹ௨ৗ͸ೖྗ͕ݻఆͰॏΈ͕ߋ৽͞ΕΔ͕ɺٯͰॏΈ͕ݻఆͰೖྗը૾͕ߋ৽͞ΕΔ
61. Prisma ੜ੒ը૾ͷॳظ஋ A:ίϯςϯπ B:ελΠϧ C:ϗϫΠτϊΠζ4ύλʔϯ ͲΕͰ΋΄ͱΜͲมΘΒͳ͍ͱ͍͏݁࿦
62. Prisma
63. FaceApp
64. FaceApp VAE (Variational Autoencoder) CVAE (Conditional VA) Facial VAE
65. ·ͱΊ • Deep LearningͱҰޱʹݴͬͯ΋ɺٕज़΍༻్͸༷ʑ • ը૾ೝࣝʢCNNʣ, ࣗવݴޠʢRNNʣ, 
 ਂ૚ੜ੒ʢVAE, GANʣ, ڧԽֶशʢDQNʣ, … • ଞ෼໺ͷٕज़΍ͪΐͬͱͨ͠޻෉ͳͲɺΞϓϩʔνํ๏ʹؔͯ͠ϒϧʔΦʔγϟϯͳ ෼໺ • 2014-2015ͷ2೥ؒͰɺ1500΋ͷؔ࿈࿦จ • CNN + RNNͷΑ͏ͳɺֆʴԻɺݴ༿ʴֆɺηϯαʔ஋ʴจষɺͳͲɺ͜Ε·Ͱ༥߹ Ͱ͖ͳ͔ͬͨσʔλ͕༥߹͢Δ͜ͱͰ৽͍͠Ձ஋ΛੜΈग़͢༧ײ
66. ࢀߟࢿྉ • ॻ੶ • θϩ͔Β࡞ΔDeep Learning ―PythonͰֶͿσΟʔϓϥʔχϯάͷཧ࿦ͱ࣮૷ http://amzn.asia/2CTyY4U • ػցֶशͷͨΊͷ֬཰ͱ౷ܭ (ػցֶशϓϩϑΣογϣφϧγϦʔζ) http://amzn.asia/5SyEZVV • ΦϯϥΠϯػցֶश (ػցֶशϓϩϑΣογϣφϧγϦʔζ) http://amzn.asia/2kli98b • ΠϥετͰֶͿ σΟʔϓϥʔχϯά (KS৘ใՊֶઐ໳ॻ) http://amzn.asia/8Kz11LV • ΠϥετͰֶͿ ػցֶश ࠷খೋ৐๏ʹΑΔࣝผϞσϧֶशΛத৺ʹ (KS৘ใՊֶઐ໳ॻ) http://amzn.asia/6Zlo0pt • ਂ૚ֶश (ػցֶशϓϩϑΣογϣφϧγϦʔζ) http://amzn.asia/hZqrQ2w • ChainerʹΑΔ࣮ફਂ૚ֶश http://amzn.asia/5xDfvVJ • ࣮૷σΟʔϓϥʔχϯά http://amzn.asia/7YP7FPh • ͜Ε͔ΒͷڧԽֶश http://amzn.asia/gHUDp81 • ITΤϯδχΞͷͨΊͷػցֶशཧ࿦ೖ໳ http://amzn.asia/7SgiMwN • ҟৗݕ஌ͱมԽݕ஌ (ػցֶशϓϩϑΣογϣφϧγϦʔζ) http://amzn.asia/6RC0jbt • PythonʹΑΔσʔλ෼ੳೖ໳ ―NumPyɺpandasΛ࢖ͬͨσʔλॲཧ http://amzn.asia/4f2ATnL • URL / SlideShare / pdf • ʢଟ͗ͯ͢লུʣ