Tweet
Tweet

Slide 1

Slide 1 text

σΟʔϓϥʔχϯάͬͯԿʁ urakarin@gmail.com 2017.02.08

Slide 2

Slide 2 text

࿩͢͜ͱɺ࿩͞ͳ͍͜ͱ • ࿩͢͜ͱ • χϡʔϥϧωοτϫʔΫͷ਺ֶతͳ࢓૊Έ • ॳظ஋ͷܾΊํɺධՁํ๏ • ύϥϝʔλྔɺܭࢉྔͷϘϦϡʔϜײ • ϗοτͳ࿩୊ • ࿩͞ͳ͍͜ͱ • πʔϧͷ࿩ • ਺ࣜͷ࿩ • χϡʔϥϧωοτϫʔΫҎ֎ͷػցֶश • γϯΪϡϥϦςΟͳͲͷਓ޻஌ೳͷະདྷ ग़య wedge.ismedia.jp

Slide 3

Slide 3 text

Agenda • σΟʔϓϥʔχϯάͱ͸ʁ • ྺ࢙ • χϡʔϥϧωοτϫʔΫ͔ΒσΟʔϓͳχϡʔϥϧωοτϫʔΫ΁ • ୈҰ࣍AIϒʔϜ • ୈೋ࣍AIϒʔϜ • ୈࡾ࣍AIϒʔϜ • Ԡ༻ྫ • ·ͱΊ

Slide 4

Slide 4 text

• ਂ૚ֶशͱ΋ݴ͏ • ೴ʢਆܦࡉ๔ʣͷಇ͖Λ໛ֶͨ͠शΞϧΰϦζϜͰ͋Δ
 Neural Network(NN)Λ༻͍ͨਓ޻஌ೳͷߏஙٕज़ͷ૯শ • ͦͷதͰ΋ਂ͘େن໛ͳߏ଄Λ࣋ͭ͜ͱ͕ಛ௃
 
 σΟʔϓϥʔχϯάͱ͸ʁ GoogLeNet, 22Layers (ILSVRC 2014)

Slide 5

Slide 5 text

༻ޠͷؔ܎ੑ ਓ޻஌ೳʢAIʣ ػցֶश χϡʔϥϧωοτϫʔΫ ਂ૚ֶश

Slide 6

Slide 6 text

୅දతͳൃද Neural Networkͷ ϒϨʔΫεϧʔͱ ౙͷ࣌୅ ୅දతͳਓࡐ֫ಘ Google ͕DNN ResearchΛങऩ )JOUPO Google ͕ Deep MindΛങऩ Baidu ͕Institute of Deep LearningΛઃཱ "OESFX/H Facebook ͕AI Research Lab.Λઃཱ -F$VO SGD (Amari) Neocognitron (Fukushima) Boltzmann Machine (Hinton+) Conv. net (LeCun+) Sparse Coding (Olshausen&Field) 1950 1960 1970 1980 1990 2000 2010 2020 Microsoft ͕MaluubaΛങऩ #FOHJP ୈҰ࣍"*ϒʔϜ ਪ࿦ɾ୳ࡧ ୈೋ࣍"*ϒʔϜ ஌ࣝදݱ &YQFSU4ZTUFN ୈࡾ࣍"*ϒʔϜ ػցֶश ਂ૚ֶश Perceptron (Rosenblatt) Back Propagation (Rumelhart) Deep Learning (Hinton+) Big Data GPU Cloud Computing ઢܗ෼཭ෆՄೳ໰୊ YPS͕ղ͚ͳ͍ ஗͍ɺաֶशɺ 47.ਓؾ χϡʔϥϧωοτϫʔΫͷྺ࢙ NN ୈҰͷౙ NN ୈೋͷౙ

Slide 7

Slide 7 text

Torontoେֶ New Yorkେֶ Montrealେֶ

Slide 8

Slide 8 text

NN ͔Β DNN ΁ Neural Network Deep Neural Network

Slide 9

Slide 9 text

ୈҰ࣍AIϒʔϜ

Slide 10

Slide 10 text

୯७ύʔηϓτϩϯ ʹྖҬ൑ఆثͱͯ͠ͷχϡʔϩϯ

Slide 11

Slide 11 text

NAND AND OR XOR ୯७ύʔηϓτϩϯ

Slide 12

Slide 12 text

ୈҰͷౙ • xor͕දݱͰ͖ͳ͍

Slide 13

Slide 13 text

ୈೋ࣍AIϒʔϜ

Slide 14

Slide 14 text

ڭࢣ৴߸ ޡࠩؔ਺ ೖྗ૚ ग़ྗ૚ தؒ૚ 1 1 1 x y t ⇥w 4 ⇥ w3 ⇥w2 ⇥w1 x1 x2 x3 x4 ⇥w 0 ⌃f y1 y2 y3 1. ଟ૚Խ 2. ׆ੑԽؔ਺ 3. ޡࠩؔ਺ 4. ޡࠩٯ఻ൖ๏ ଟ૚ύʔηϓτϩϯʢMLPʣ

Slide 15

Slide 15 text

ଟ૚ԽʹΑͬͯxorͷ࣮ݱ NAND OR AND s2 s1 x1 x2 y x1 x2 s1 s2 y 0 0 1 0 0 1 0 1 1 1 0 1 1 1 1 1 1 0 1 0 = 1. ଟ૚Խ

Slide 16

Slide 16 text

γάϞΠυؔ਺ɾ૒ۂઢਖ਼઀ؔ਺ ඍ෼͕Ͱ͖ͳ͍ ֶशͰ͖ͳ͍ ʢޡࠩٯ఻ൖ๏ʣ ʹೖྗ৴߸ͷ૯࿨Λग़ྗ৴߸ʹม׵͢Δؔ਺ ׆ੑԽؔ਺ 2. ׆ੑԽؔ਺ 1 ⇥w 4 ⇥ w3 ⇥w2 ⇥w1 x1 x2 x3 x4 ⇥w 0 ⌃f εςοϓؔ਺ ύʔηϓτϩϯͷ৔߹

Slide 17

Slide 17 text

3. ଛࣦؔ਺ ޡࠩؔ਺ʢଛࣦؔ਺ʣ 1 2 N X n=1 ky tk2 N Y n=1 p(dn | x ) d=0/1ͷࣄޙ֬཰pʹରͯ͠࠷໬ਪఆΛߦ͏ ೋ৐ޡࠩͱ͢Δ ڭࢣ৴߸ ޡࠩؔ਺ ग़ྗ૚ y t y1 y2 y3 ճؼ ೋ஋෼ྨ ଟΫϥε෼ྨ ڭࢣ৴߸ΛOne-hotදݱͱ͠ɺ ࠷ऴஈͷ׆ੑԽؔ਺ΛιϑτϚοΫεؔ਺ͱ্ͨ͠Ͱ ަࠩΤϯτϩϐʔؔ਺

Slide 18

Slide 18 text

ڭࢣ৴߸ ޡࠩؔ਺ ೖྗ૚ ग़ྗ૚ தؒ૚ 1 1 1 x y t ⇥w 4 ⇥ w3 ⇥w2 ⇥w1 x1 x2 x3 x4 ⇥w 0 ⌃f y1 y2 y3 4. ޡࠩٯ఻ൖ๏ ޡࠩٯ఻ൖ๏

Slide 19

Slide 19 text

+ ^2 x y t z @z @z @z @z @z @t @z @z @z @t @t @x ͨͱ͑͹ z = ( x + y )2 ͱ͍͏ࣜ͸ z = t2 t = x + y ͱ͍͏2ͭͷࣜͰߏ੒͞ΕΔɻ ࿈࠯཯ͱ͸ɺ߹੒ؔ਺ͷඍ෼ʹ͍ͭͯͷੑ࣭Ͱ͋Δ @z @x = @z @t @t @x ޡࠩٯ఻ൖ๏ 4. ޡࠩٯ఻ൖ๏

Slide 20

Slide 20 text

ޡࠩٯ఻ൖ๏ Ճࢉϊʔυͷٯ఻ൖ + x y z + @L @z @L @z · 1 @L @z · 1 ৐ࢉϊʔυͷٯ఻ൖ x y z ⇥ @L @z ⇥ @L @z · x @L @z · y 4. ޡࠩٯ఻ൖ๏ 2 100 ⇥ ⇥ 200 1.1 220 1 1.1 200 110 2.2 ΓΜ͝ͷݸ਺ ফඅ੫ ɹ۩ମྫɹ

Slide 21

Slide 21 text

֬཰తޯ഑߱Լ๏ • ϛχόονֶश • ֶश܎਺ͷߋ৽ํ๏ • Momentum • AdaGrad • Adam • RMSProp

Slide 22

Slide 22 text

ୈೋͷౙ • ܭࢉྔ͕ଟ͗ͯ͢஗͍ • ہॴղɾաֶशʹؕΓ΍͍͢ • ޯ഑ফࣦ໰୊

Slide 23

Slide 23 text

ୈࡾ࣍AIϒʔϜ

Slide 24

Slide 24 text

Deep Belief Network vs Auto Encoder ہॴղɾաֶश ରࡦ

Slide 25

Slide 25 text

Auto Encoder Deep Belief Network v3 h2 v1 h1 v2 Visible Hidden Visible Hidden Visible Hidden RBM RBM RBM 4UBDLFE "VUP&ODPEFS "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث ଟஈԽ ʴϩόετੑ "VUP&ODPEFS ࣗݾූ߸Խث AE %FOPJTJOH "VUP&ODPEFS DAE SAE pre-training + fine tuning ࠾༻ ֶश Input Hidden Output %FFQ#FMJFG /FUXPSL )PQpFME /FUXPSL #PMU[NBOO .BDIJOF ֬཰Ϟσϧ ܭࢉྔͷ࡟ݮ ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ଟஈԽ ੍໿෇͖ #PMU[NBOO .BDIJOF RBM DBN pre-training + fine tuning

Slide 26

Slide 26 text

୅දతͳൃද Neural Networkͷ ϒϨʔΫεϧʔͱ ౙͷ࣌୅ ୅දతͳਓࡐ֫ಘ Google ͕DNN ResearchΛങऩ )JOUPO Google ͕ Deep MindΛങऩ Baidu ͕Institute of Deep LearningΛઃཱ "OESFX/H Facebook ͕AI Research Lab.Λઃཱ -F$VO SGD (Amari) Neocognitron (Fukushima) Boltzmann Machine (Hinton+) Conv. net (LeCun+) Sparse Coding (Olshausen&Field) 1950 1960 1970 1980 1990 2000 2010 2020 Microsoft ͕MaluubaΛങऩ #FOJHO ୈҰ࣍"*ϒʔϜ ਪ࿦ɾ୳ࡧ ୈೋ࣍"*ϒʔϜ ஌ࣝදݱ &YQFSU4ZTUFN ୈࡾ࣍"*ϒʔϜ ػցֶश ਂ૚ֶश Perceptron (Rosenblatt) Back Propagation (Rumelhart) Deep Learning (Hinton+) Big Data GPU Cloud Computing ઢܗ෼཭ෆՄೳ໰୊ YPS͕ղ͚ͳ͍ ஗͍ɺաֶशɺ 47.ਓؾ χϡʔϥϧωοτϫʔΫͷྺ࢙ NN ୈҰͷౙ NN ୈೋͷౙ

Slide 27

Slide 27 text

ωο τϫʔΫͷΤωϧΪʔ͕࠷খʹͳΔΑ͏ʹঢ়ଶมԽΛ܁Γฦ͢ %FFQ#FMJFG /FUXPSL )PQpFME /FUXPSL #PMU[NBOO .BDIJOF ֬཰Ϟσϧ ܭࢉྔͷ࡟ݮ ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ଟஈԽ ੍໿෇͖ #PMU[NBOO .BDIJOF RBM DBN pre-training + fine tuning هԱ1 هԱ2 هԱΛࢥ͍ग़͢ ͍ۙ͠σʔλΛ༩͑Δͱ… ը૾ΛهԱͨ͠ωοτϫʔΫ Hopfield Networkͱ͸ هԱΛߦྻܭࢉͰγϛϡϨʔτͯ͠ΈΑ͏ http://www.gaya.jp/spiking_neuron/matrix.htm

Slide 28

Slide 28 text

%FFQ#FMJFG /FUXPSL )PQpFME /FUXPSL #PMU[NBOO .BDIJOF ֬཰Ϟσϧ ܭࢉྔͷ࡟ݮ ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ଟஈԽ ੍໿෇͖ #PMU[NBOO .BDIJOF RBM DBN pre-training + fine tuning Boltzmann Machineͱ͸ ֬཰Ϟσϧͷಋೖ Kullback LeiblerμΠόʔδΣϯε 2ͭͷۂઢʹ͍ͭͯɺॏͳΒͣʹ૬ҧʢμΠόʔδΣϯεʣ͍ͯ͠ΔྖҬʢࠩʣΛ࠷খԽ͢Δɻ ࣮ࡍͷೖྗ஋ʹ ΑΔ֬཰෼෍p ෮ݩ͞Εͨ෼෍q ࠩͷੵ෼

Slide 29

Slide 29 text

%FFQ#FMJFG /FUXPSL )PQpFME /FUXPSL #PMU[NBOO .BDIJOF ֬཰Ϟσϧ ܭࢉྔͷ࡟ݮ ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ଟஈԽ ੍໿෇͖ #PMU[NBOO .BDIJOF RBM DBN pre-training + fine tuning ੍໿෇͖Boltzmann Machine (RBN)ͱ͸ v3 h2 v1 h1 v2 Visible Hidden

Slide 30

Slide 30 text

%FFQ#FMJFG /FUXPSL )PQpFME /FUXPSL #PMU[NBOO .BDIJOF ֬཰Ϟσϧ ܭࢉྔͷ࡟ݮ ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ଟஈԽ ੍໿෇͖ #PMU[NBOO .BDIJOF RBM DBN pre-training + fine tuning Deep Belief Network (DBN)ͱ͸ Visible Hidden Visible Hidden Visible Hidden RBM RBM RBM pre-training(ڭࢣͳ͠) + fine tuning (ڭࢣ͋Γ)

Slide 31

Slide 31 text

Auto Encoder Deep Belief Network v3 h2 v1 h1 v2 Visible Hidden Visible Hidden Visible Hidden RBM RBM RBM 4UBDLFE "VUP&ODPEFS "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث ଟஈԽ ʴϩόετੑ "VUP&ODPEFS ࣗݾූ߸Խث AE %FOPJTJOH "VUP&ODPEFS DAE SAE pre-training + fine tuning ࠾༻ ֶश Input Hidden Output %FFQ#FMJFG /FUXPSL )PQpFME /FUXPSL #PMU[NBOO .BDIJOF ֬཰Ϟσϧ ܭࢉྔͷ࡟ݮ ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ଟஈԽ ੍໿෇͖ #PMU[NBOO .BDIJOF RBM DBN pre-training + fine tuning

Slide 32

Slide 32 text

4UBDLFE "VUP&ODPEFS "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث ଟஈԽ ʴϩόετੑ "VUP&ODPEFS ࣗݾූ߸Խث AE %FOPJTJOH "VUP&ODPEFS DAE SAE pre-training + fine tuning ࠾༻ ֶश Input Hidden Output Auto Encoder (AE)ͱ͸

Slide 33

Slide 33 text

4UBDLFE "VUP&ODPEFS "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث ଟஈԽ ʴϩόετੑ "VUP&ODPEFS ࣗݾූ߸Խث AE %FOPJTJOH "VUP&ODPEFS DAE SAE pre-training + fine tuning Denoising Auto Encoder (DAE)ͱ͸ ࠾༻ ֶश Input Hidden Output ϊΠζ

Slide 34

Slide 34 text

4UBDLFE "VUP&ODPEFS "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث ଟஈԽ ʴϩόετੑ "VUP&ODPEFS ࣗݾූ߸Խث AE %FOPJTJOH "VUP&ODPEFS DAE SAE pre-training + fine tuning Stacked Auto Encoder (SAE)ͱ͸

Slide 35

Slide 35 text

Auto Encoder Deep Belief Network v3 h2 v1 h1 v2 Visible Hidden Visible Hidden Visible Hidden RBM RBM RBM 4UBDLFE "VUP&ODPEFS "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث "VUP&ODPEFS ࣗݾූ߸Խث ଟஈԽ ʴϩόετੑ "VUP&ODPEFS ࣗݾූ߸Խث AE %FOPJTJOH "VUP&ODPEFS DAE SAE pre-training + fine tuning ࠾༻ ֶश Input Hidden Output %FFQ#FMJFG /FUXPSL )PQpFME /FUXPSL #PMU[NBOO .BDIJOF ֬཰Ϟσϧ ܭࢉྔͷ࡟ݮ ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ੍໿෇͖ #PMU[NBOO .BDIJOF ଟஈԽ ੍໿෇͖ #PMU[NBOO .BDIJOF RBM DBN pre-training + fine tuning

Slide 36

Slide 36 text

γάϞΠυؔ਺ɾ૒ۂઢਖ਼઀ؔ਺ ޯ഑ফࣦ໰୊ ඍ෼஋ ωοτϫʔΫ͕ਂ͍ͱޯ഑͕ফ͑ͯ͠·͏ɻɻɻ

Slide 37

Slide 37 text

γάϞΠυؔ਺ɾ૒ۂઢਖ਼઀ؔ਺ ඍ෼஋ ωοτϫʔΫ͕ਂ͍ͱޯ഑͕ফ͑ͯ͠·͏ɻɻɻ ReLU (Rectified Linear Unit) ൃՐ͍ͯ͠ͳ͍ ൃՐ͍ͯ͠Δ ޯ഑ফࣦͳ͠ʹൃՐ͍ͯ͠Δ ࡉ๔ͷΈΛ௨ͬͯ఻೻͢Δɻ ޯ഑ফࣦ໰୊

Slide 38

Slide 38 text

• ϛχόονͷೖྗσʔλΛฏۉ0ɺ෼ࢄ1ͷσʔλʹม׵͢Δ޻෉ • ׆ੑԽؔ਺ͷલɺ΋͘͠͸ޙʹૠೖ͢Δ͜ͱͰσʔλ෼෍ͷภΓΛݮΒ͢͜ͱ ͕Մೳ • ޮՌ • ֶश܎਺Λେ͖͘͢Δ͜ͱ͕ՄೳʢֶशΛૣ͘ਐߦͤ͞Δʣ • ॳظ஋ʹͦΕ΄Ͳґଘ͠ͳ͍ • աֶशΛ཈੍͢Δ Batch Normalization

Slide 39

Slide 39 text

• DropOut (Drop Connect) • ΞϯαϯϒϧֶशʹରԠ • ਖ਼ଇԽ • Weight Decayʢޡࠩؔ਺ʹL2ϊϧϜΛՃ͑Δʣ • εύʔεਖ਼ଇԽ • σʔλ֦ுʢϊΠζɺฏߦҠಈɺճసɺ৭ʣ ͦͷଞͷ޻෉

Slide 40

Slide 40 text

ॳظ஋ͷܾΊํ

Slide 41

Slide 41 text

• 0ʹ͢ΔʁˠॏΈ͕ۉҰʹͳͬͯ͠·͍ॏෳͨ͠஋ʹͳͬͯ͠·͏ • ϥϯμϜͳॳظ஋͕ඞཁ • ׆ੑԽؔ਺ʹɺγάϞΠυؔ਺΍tanhؔ਺Λ࢖༻͢Δ৔߹ɺ
 ʮXavierͷॳظ஋ʯ͕ద౰ • ReLUΛ༻͍Δ৔߹͸ɺʮHeͷॳظ஋ʯ͕ద౰ ॏΈߦྻͷॳظ஋ • લ૚ͷϊʔυͷ਺͕ɹ ݸͷ৔߹ɺɹɹΛඪ४ภࠩͱ͢ΔΨ΢ε෼෍ n r 2 n • લ૚ͷϊʔυͷ਺͕ɹ ݸͷ৔߹ɺɹɹΛඪ४ภࠩͱ͢ΔΨ΢ε෼෍ n r 1 n

Slide 42

Slide 42 text

• ֤૚ͷχϡʔϩϯ਺ • όοναΠζ • ֶश܎਺ɺֶश܎਺ͷมԽ཰ • Weight decayʢՙॏݮਰʣ • DropOut཰ • ͳͲ ϋΠύʔύϥϝʔλ NNʹ͸ɺॏΈ΍όΠΞεύϥϝʔλͱ͸ผʹɺ ਓ͕ઃఆ͢΂͖ϋΠύʔύϥϝʔλ͕ଘࡏ͢Δɻ ύϥϝʔλܾఆʹ͸ଟ͘ͷࢼߦࡨޡ͕൐͍ɺ Ϟσϧͷੑೳʹ΋େ͖͘Өڹ͢Δɻ • ઐ༻ͷݕূσʔλΛ༻ҙ͢Δ • ܇࿅σʔλ΍ςετσʔλΛ࢖ͬͯੑೳධՁΛͯ͠͸͍͚ͳ͍ • ର਺εέʔϧͷൣғ͔ΒϥϯμϜʹαϯϓϦϯάͯ͠ධՁ͠ɺ
 ൣғΛߜΓࠐΜͰ͍͖ɺ࠷ޙʹͻͱͭΛϐοΫΞοϓ͢Δ σʔληοτ ܇࿅σʔλ ςετσʔλ ݕূσʔλ ֶश༻ ֶश݁Ռͷ ධՁ༻ ϋΠύʔύϥϝʔλͷධՁ༻

Slide 43

Slide 43 text

༧ଌੑೳͷධՁ

Slide 44

Slide 44 text

܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ςετσʔλ ܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ςετσʔλ ܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ςετσʔλ ܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ςετσʔλ ܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ςετσʔλ ܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ܇࿅σʔλ ݕূσʔλ ݕূσʔλ ݕূσʔλ ݕূσʔλ ݕূσʔλ ϗʔϧυΞ΢τݕূ K෼ׂަࠩݕূ
 (Cross Validation) ੑೳධՁ

Slide 45

Slide 45 text

TP rate: ཅੑΛཅੑͱ൑அׂͨ͠߹ FP rate: ӄੑΛཅੑͱ൑அׂͨ͠߹ = = ROCۂઢͱAUC ROC:Receiver Operating Characteristic ʢड৴ऀૢ࡞ಛੑʣ AUC:Area under the curve ʢROCۂઢԼ໘ੵʣ Predicted Condition Positive Negative True Condition Positive TP FN (type II error) Negative FP (Type I error) TN True Positive True Negative False Positive False Negative AUC

Slide 46

Slide 46 text

True Positive True Negative False Positive False Negative ࠶ݱ཰: ཅੑΛཅੑͱ൑அׂͨ͠߹ ʢRecallʣ = ద߹཰: ཅੑͱ༧ଌͨ͠σʔλͷ͏ͪɼ࣮ࡍʹཅੑͰ͋Δ΋ͷͷׂ߹ = ʢPrecisionʣ F஋: F஋ͷ࠷େ஋͸͓͓ΉͶ෼ذ఺ਫ਼౓ͱҰக͢Δɻ ௐ࿨ฏۉɿٯ਺ͷฏۉͷٯ਺ http://www004.upp.so-net.ne.jp/s_honma/mean/harmony2.htm Predicted Condition Positive Negative True Condition Positive TP FN (type II error) Negative FP (Type I error) TN

Slide 47

Slide 47 text

Ԡ༻ྫ • ը૾ೝࣝ (CNN) • ࣗવݴޠॲཧɺԻ੠ೝࣝ (RNN) • ը૾ʹର͢ΔΩϟϓγϣϯੜ੒ (CNN + RNN) • ڧԽֶश (CNN + Qֶश) • ਂ૚ੜ੒Ϟσϧ (CNN)

Slide 48

Slide 48 text

ը૾ೝࣝ • Convolutional Neural Network (৞ΈࠐΈχϡʔϥϧωοτϫʔΫ) • Convolution૚ + Pooling૚

Slide 49

Slide 49 text

খ͞ͳը૾ͳΒ͜Ε·Ͱͷશ݁߹NNͰOK Convolution૚

Slide 50

Slide 50 text

ฏۉ ࠨӈʹ૸ΔΤοδ ্Լʹ૸ΔΤοδ ޲͖ʹؔ܎ͳ͘Τοδ * ϑΟϧλྫ Convolution૚

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

ը૾෼ྨ͸ਓؒΛ௒͑ͨ ILSVRC = 2010೥͔Β࢝·ͬͨେن໛ը૾ೝࣝͷڝٕձ 2012೥ͷILSVRCͰHintonઌੜͷνʔϜ͕Deep LearningͰѹউ 2015೥ʹ͸ILSVRCͷ݁ՌͰਓؒͷೝࣝੑೳΛ௒͑ͨɻ

Slide 54

Slide 54 text

ܭࢉྔ • CPUͱGPUͷੑೳͷҧ͍ • ಉ࣌ԋࢉՄೳ਺ʢ୯ਫ਼౓গ਺ʣ • CPU(Intel Core i7) : AVX256bit -> 8ݸ • nVIDIA Pascal GP100 : 114,688ݸ

Slide 55

Slide 55 text

ࣗવݴޠॲཧɺԻ੠ೝࣝ • Recurrent Neural Network (RNN)

Slide 56

Slide 56 text

ڧԽֶश • CNN + Qֶश + …

Slide 57

Slide 57 text

Prisma

Slide 58

Slide 58 text

Prisma σΟʔϓϥʔχϯάΛ࢖ͬͨΞʔτܥͷ ࿦จ͸ɺ͍Ζ͍Ζग़͍ͯΔ͕ Ұ൪ͷجૅͱͳΔͷ͸ Gatys et al. 2016 ࢖༻CNN͸VGG19ʢը૾෼ྨ༻ʹ܇࿅ࡁΈʣ͔Βશ݁߹Λ ൈ͍ͨ΋ͷ “Image Style Transfer Using Convolutional Neural Networks”

Slide 59

Slide 59 text

Prisma ίϯςϯπ ελΠϧ http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Gatys_Image_Style_Transfer_CVPR_2016_paper.pdf

Slide 60

Slide 60 text

Prisma ଛࣦؔ਺ʹίϯςϯπͷଛࣦʴελΠϧͷଛࣦ ࠷దԽʹ௨ৗ͸ೖྗ͕ݻఆͰॏΈ͕ߋ৽͞ΕΔ͕ɺٯͰॏΈ͕ݻఆͰೖྗը૾͕ߋ৽͞ΕΔ

Slide 61

Slide 61 text

Prisma ੜ੒ը૾ͷॳظ஋ A:ίϯςϯπ B:ελΠϧ C:ϗϫΠτϊΠζ4ύλʔϯ ͲΕͰ΋΄ͱΜͲมΘΒͳ͍ͱ͍͏݁࿦

Slide 62

Slide 62 text

Prisma

Slide 63

Slide 63 text

FaceApp

Slide 64

Slide 64 text

FaceApp VAE (Variational Autoencoder) CVAE (Conditional VA) Facial VAE

Slide 65

Slide 65 text

·ͱΊ • Deep LearningͱҰޱʹݴͬͯ΋ɺٕज़΍༻్͸༷ʑ • ը૾ೝࣝʢCNNʣ, ࣗવݴޠʢRNNʣ, 
 ਂ૚ੜ੒ʢVAE, GANʣ, ڧԽֶशʢDQNʣ, … • ଞ෼໺ͷٕज़΍ͪΐͬͱͨ͠޻෉ͳͲɺΞϓϩʔνํ๏ʹؔͯ͠ϒϧʔΦʔγϟϯͳ ෼໺ • 2014-2015ͷ2೥ؒͰɺ1500΋ͷؔ࿈࿦จ • CNN + RNNͷΑ͏ͳɺֆʴԻɺݴ༿ʴֆɺηϯαʔ஋ʴจষɺͳͲɺ͜Ε·Ͱ༥߹ Ͱ͖ͳ͔ͬͨσʔλ͕༥߹͢Δ͜ͱͰ৽͍͠Ձ஋ΛੜΈग़͢༧ײ

Slide 66

Slide 66 text

ࢀߟࢿྉ • ॻ੶ • θϩ͔Β࡞ΔDeep Learning ―PythonͰֶͿσΟʔϓϥʔχϯάͷཧ࿦ͱ࣮૷ http://amzn.asia/2CTyY4U • ػցֶशͷͨΊͷ֬཰ͱ౷ܭ (ػցֶशϓϩϑΣογϣφϧγϦʔζ) http://amzn.asia/5SyEZVV • ΦϯϥΠϯػցֶश (ػցֶशϓϩϑΣογϣφϧγϦʔζ) http://amzn.asia/2kli98b • ΠϥετͰֶͿ σΟʔϓϥʔχϯά (KS৘ใՊֶઐ໳ॻ) http://amzn.asia/8Kz11LV • ΠϥετͰֶͿ ػցֶश ࠷খೋ৐๏ʹΑΔࣝผϞσϧֶशΛத৺ʹ (KS৘ใՊֶઐ໳ॻ) http://amzn.asia/6Zlo0pt • ਂ૚ֶश (ػցֶशϓϩϑΣογϣφϧγϦʔζ) http://amzn.asia/hZqrQ2w • ChainerʹΑΔ࣮ફਂ૚ֶश http://amzn.asia/5xDfvVJ • ࣮૷σΟʔϓϥʔχϯά http://amzn.asia/7YP7FPh • ͜Ε͔ΒͷڧԽֶश http://amzn.asia/gHUDp81 • ITΤϯδχΞͷͨΊͷػցֶशཧ࿦ೖ໳ http://amzn.asia/7SgiMwN • ҟৗݕ஌ͱมԽݕ஌ (ػցֶशϓϩϑΣογϣφϧγϦʔζ) http://amzn.asia/6RC0jbt • PythonʹΑΔσʔλ෼ੳೖ໳ ―NumPyɺpandasΛ࢖ͬͨσʔλॲཧ http://amzn.asia/4f2ATnL • URL / SlideShare / pdf • ʢଟ͗ͯ͢লུʣ