ひたすら楽してディープラーニング

Slide 1

Slide 1 text

Yuji Isobe ͻͨ͢Βָͯ͠  σΟʔϓϥʔχϯά NodeֶԂ20࣌ݶ໨

Slide 2

Slide 2 text

[ “Node.js”, “MongoDB”, “AngularJS”, “socket.io”, “Emotion Intelligence “  ] @yujiosaka +BWB4DSJQU

Slide 3

Slide 3 text

emin = Emotion Intelligence ؾ࣋ͪΛղ͢ΔςΫϊϩδʔͷ୳ڀ Emotion Intelligence͸ɺʮແҙࣝͷߦಈ͔Βɺ ਓͷؾ࣋ͪͷػඍΛղ͢Δ஌ੑʯΛɺਓ޻஌ೳ͓Αͼػցֶ शͷԠ༻ٕज़Λ༻͍ͯ ։ൃ͠ɺϏδωεʹԠ༻͍ͯ͠·͢ɻ

Slide 4

Slide 4 text

ZenClerk Series lϢʔβʔͷߪೖҙཉͷߴ·ΓzΛ࡯஌͢ΔγεςϜͰɺ  ΋ͬͱαΠτͷച্Λ৳͹ͯ͠Έ·ͤΜ͔ʁ [FODMFSLMJUF ײ৘Λղੳ͢Δਓ޻஌ೳʮ&NPUJPO*0ʯ͕  &$αΠτ্Ͱʮ࠷దͳλΠϛϯάͰͷൢଅʯΛ࣮ݱ͠ɺച্Λ޲্ͤ͞·͢ɻ ࠓ·͞ʹߪೖΛ໎͍ͬͯΔ͓٬༷ʹ  ͏Ε͍͠ʮ࠷ޙͷͻͱԡ͠ʯΛఏڙ͠·͢ɻ ;FODMFSL

Slide 5

Slide 5 text

ػցֶशͱݴ͑͹…

Slide 6

Slide 6 text

Lee Sedol vs. AlphaGo

Slide 7

Slide 7 text

Google Trends

Slide 8

Slide 8 text

ࠔͬͨ(´ɾωɾʆ)

Slide 9

Slide 9 text

͜ͷձࣾʹೖΔ·ͰҰ౓΋  ػցֶश΍ͬͯ͜ͳ͔ͬͨorz

Slide 10

Slide 10 text

ͱΓ͋͑ͣೖ໳ॻΛಡΉ ✓ ϐϯΫͷബ͍ຊ ✓ ௨শʮ͸͡ύλຊʯ ✓ ݟͨ໨ΑΓࠎଠͳ಺༰ ✓ ͪΌΜͱ਺͕ࣜࡌͬͯΔ  ʢͦͯͭ͠·ͮ͘ʣ

Slide 11

Slide 11 text

ୈ3ষɿϕΠζ ୈ5ষɿkNN๏ ୈ6ষɿҰൠԽઢܗϞσϧ ୈ7ষɿχϡʔϥϧωοτϫʔΫ ୈ8ষɿαϙʔτϕΫλʔϚγϯ ӽ͑ΒΕͳ͍น ← ෼͔Δ ← ෼͔Δ ← ෼͔Δ ← ͓ɺ͓͏… ← (ɾ㱼ɾ) ŜŠŘŘ!!

Slide 12

Slide 12 text

σʔλՄࢹԽ ػցֶश ਺ֶ ౷ܭֶ ܭࢉػՊֶ ίϛϡχέʔγϣϯ υϝΠϯ஌ࣝ ࣗ෼ͷεΩϧηοτ ͜͜Λ৳͹͍ͨ͠

Slide 13

Slide 13 text

Kaggle͸اۀ΍ݚڀऀ͕σʔλΛ౤ߘ͠ɺੈքதͷ ౷ܭՈ΍σʔλ෼ੳՈ͕ͦͷ࠷దϞσϧΛڝ͍߹͏ɺ ༧ଌϞσϦϯάٴͼ෼ੳख๏ؔ࿈ϓϥοτϑΥʔϜٴ ͼͦͷӡӦձࣾͰ͋Δɻ ΢ΟΩϖσΟΞ ૑ཱऀɿ ΞϯιχʔɾΰʔϧυϒϧʔϜ ઃཱɿ 2010೥4݄

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

ࠓճ͸͜Εʹ  νϟϨϯδ

Slide 16

Slide 16 text

✓ MNISTʢ਺ࣈͷखॻ͖σʔλʣͷ෼ྨ ✓ 28 x 28ϐΫηϧ ✓ 6ສ݅ͷֶशσʔλ ✓ 1ສ݅ͷςετσʔλ νϡʔτϦΞϧ՝୊

Slide 17

Slide 17 text

ਖ਼ղ཰99%Λ໨ࢦ͢ http://yann.lecun.com/exdb/mnist/

Slide 18

Slide 18 text

Ͱ΋ͳΜ͔೉ͦ͠͏…

Slide 19

Slide 19 text

Ͱ͖Ε͹ͻͨ͢Βָ͍ͨ͠

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

ͻͨ͢Βָͯ͠FF6

Slide 22

Slide 22 text

1.࠷খউརճ਺ͰΫϦΞ 2.ඞਢઓಆҎ֎Ͱ౪·ͳ͍ 3.ಓதͷΞΠςϜ͸ճऩ͠ͳ͍ 4.ฤू͸ؤுΒͳ͍ ͻͨ͢Βָͯ͠FF6

Slide 23

Slide 23 text

1.࠷ۙͷϒʔϜʹ৐͔ͬΔ 2.ۤखͳݴޠ͸࢖Θͳ͍ 3.ߴ౓ͳϥΠϒϥϦ͸࢖Θͳ͍ 4.ϓϨθϯࢿྉ͸ؤுΒͳ͍ ͻͨ͢Βָͯ͠Kaggle ͱΓ͋͑ͣσΟʔϓϥʔχϯά  ࢖ͬͯΈΔ +BWB4DSJQU͔͠࢖Θͳ͍ MPEBTIͱઢܗ୅਺ϥΠϒϥϦ  ͑͋͞Ε͹͍͍ ΤϯδχΞͩ͠ίʔυͰউෛ

Slide 24

Slide 24 text

·ͣ͸৘ใऩू͔Β

Slide 25

Slide 25 text

IUUQOFVSBMOFUXPSLTBOEEFFQMFBSOJOHDPNJOEFYIUNM

Slide 26

Slide 26 text

✓ ӳޠͷΦϯϥΠϯϒοΫʢ೔ຊޠԽ΋ਐߦதʣ ✓ χϡʔϥϧωοτϫʔΫ͔Β  σΟʔϓϥʔχϯά·ͰͷྲྀΕΛৄࡉʹղઆ ✓ Pythonͷαϯϓϧ࣮૷͕ಡΈ΍͍͢ Neural Networks and Deep Learning Φεεϝʂ

Slide 27

Slide 27 text

ઓུΛཱͯΔ

Slide 28

Slide 28 text

Python→CoffeeScript→ES2015 sed + ؾ߹ Decaf JS ੈքॳʁES2015ͰσΟʔϓϥʔχϯά npmௐ΂ JavaScript Babel

Slide 29

Slide 29 text

PythonͱCoffeeScriptͬͯࣅͯͶʁ

Slide 30

Slide 30 text

Python def update_mini_batch(self, mini_batch, eta): nabla_b = [np.zeros(b.shape) for b in self.biases] nabla_w = [np.zeros(w.shape) for w in self.weights] for x, y in mini_batch: delta_nabla_b, delta_nabla_w = self.backprop(x, y) nabla_b = [nb+dnb for nb, dnb in zip(nabla_b, delta_nabla_b)] nabla_w = [nw+dnw for nw, dnw in zip(nabla_w, delta_nabla_w)] self.weights = [w-(eta/len(mini_batch))*nw for w, nw in zip(self.weights, nabla_w)] self.biases = [b-(eta/len(mini_batch))*nb for b, nb in zip(self.biases, nabla_b)]

Slide 31

Slide 31 text

CoffeeScript updateMiniBatch: (miniBatch, eta) -> nablaB = (Matrix.zeros(b.rows, b.cols) for b in @biases) nablaW = (Matrix.zeros(w.rows, w.cols) for w in @weights) for [x, y] in miniBatch [deltaNablaB, deltaNablaW] = @backprop(x, y) nablaB = (nb.plus(dnb) for [nb, dnb] in _.zip(nablaB, deltaNablaB)) nablaW = (nw.plus(dnw) for [nw, dnw] in _.zip(nablaW, deltaNablaW)) @weights = (w.minus(nw.mulEach(eta / miniBatch.length))  for [w, nw] in _.zip(@weights, nablaW)) @biases = (b.minus(nb.mulEach(eta / miniBatch.length))  for [b, nb] in _.zip(@biases, nablaB))

Slide 32

Slide 32 text

PythonϥΠϒϥϦͷAPIΛ࣮૷͢Δ

Slide 33

Slide 33 text

numpy.nan_to_num nanToNum() { let thisData = this.data, rows = this.rows, cols = this.cols; let row, col, result = new Array(rows); for (row=0; row

Slide 34

Slide 34 text

numpy.ravel ravel() { let thisData = this.data, rows = this.rows, cols = this.cols; let a = new Array(rows * cols); for (let i = 0, jBase = 0; i

Slide 35

Slide 35 text

CoffeeScript͕ڐ͞ΕΔͷ͸2015೥·ͰͩΑͶʔ https://github.com/juliankrispel/decaf

Slide 36

Slide 36 text

…

Slide 37

Slide 37 text

ಈ͍ͨস

Slide 38

Slide 38 text

ͦΖͦΖͪΌΜͱษڧ͢Δ

Slide 39

Slide 39 text

χϡʔϥϧωοτϫʔΫ ਆܦճ࿏໢ɺӳOFVSBMOFUXPSL // ͸ɺ೴ػೳʹݟΒΕΔ͍͔ͭ͘ͷಛੑΛܭࢉػ ্ͷγϛϡϨʔγϣϯʹΑͬͯදݱ͢Δ͜ͱΛ໨ࢦͨ͠਺ֶϞσϧͰ͋Δɻ χϡʔϥϧωοτϫʔΫ8JLJQFEJB IUUQTKBXJLJQFEJBPSHXJLJχϡʔϥϧωοτϫʔΫ χϡʔϥϧωοτϫʔΫͱ͸ʁ

Slide 40

Slide 40 text

b ύʔηϓτϩϯϞσϧ x1 x2 x3 output w1 w2 w3 PVUQVU JGЄKXKYKC≤  JGЄKXKYKC

Slide 41

Slide 41 text

5 ύʔηϓτϩϯϞσϧ ఱؾ͸ྑ͍͔ʁ ൴ঁ͸ߦ͖͍͔ͨʁ ձ৔͸Ӻͷ͔ۙ͘ʁ ͓ࡇΓʹߦ͘ʁ 6 2 2 No Yes Yes No ≤

Slide 42

Slide 42 text

b γάϞΠυχϡʔϩϯϞσϧ x1 x2 x3 w1 w2 w3 PVUQVU   FYQ ЄKXKYKC output

Slide 43

Slide 43 text

εςοϓؔ਺ʢύʔηϓτϩϯʣ

Slide 44

Slide 44 text

γάϞΠυؔ਺

Slide 45

Slide 45 text

✓ 0ͱ1ͷؒͷ஋ΛදݱͰ͖ΔΑ͏ʹͳͬͨ ✓ খ͞ͳೖྗͷมԽ͕খ͞ͳग़ྗͷมԽʹରԠ͢Δ ✓ ཁ͢Δʹඍ෼Ͱ͖Δ͜ͱʹͳͬͨ Կ͕خ͍͠ͷ͔ʁ

Slide 46

Slide 46 text

χϡʔϥϧωοτϫʔΫͷߏ଄ w + Δw  b + Δb output + Δoutput

Slide 47

Slide 47 text

✓ ֤χϡʔϩϯʢɹʣͷॏΈʢwʣͱόΠΞεʢbʣ Λௐ੔͢Δ͜ͱͰ༧ଌਫ਼౓Λ޲্͢Δ ✓ ͦͷͨΊͷςΫχοΫͱͯ͠ɺ  όοΫϓϩύήʔγϣϯͳͲ͕ੜΈग़͞Εͨ χϡʔϥϧωοτϫʔΫͷֶश

Slide 48

Slide 48 text

σΟʔϓϥʔχϯάͱ͸ʁ

Slide 49

Slide 49 text

χϡʔϥϧωοτϫʔΫͷߏ଄

Slide 50

Slide 50 text

σΟʔϓϥʔχϯάͷߏ଄

Slide 51

Slide 51 text

࠷ۙͷϒʔϜͷഎܠ ✓ ࠷ۙςΫχοΫ͕͍ͭ͘΋ݟ͔ͭͬͨ ✓ ૚ΛॏͶͯ΋͏·ֶ͘शͰ͖ΔΑ͏ʹͳͬͨ ✓ ૚ΛॏͶΔ͜ͱͰදݱೳྗͱਫ਼౓͕޲্ͨ͠

Slide 52

Slide 52 text

͍࣮͟૷΁

Slide 53

Slide 53 text

৞ΈࠐΈχϡʔϥϧωοτϫʔΫ

Slide 54

Slide 54 text

໰୊ ͨͬͨ1pxͣΕΔ͚ͩͰผͷը૾ͱͯ͠ೝࣝ͞Εͯ͠·͏ 1px

Slide 55

Slide 55 text

ղܾΠϝʔδ

Slide 56

Slide 56 text

ߏ଄ convolutional layer pooling layer

Slide 57

Slide 57 text

✓ γάϞΠυؔ਺Ҏ֎ͷ׆ੑԽؔ਺ʢReLUʣ ✓ ਖ਼نԽʢL1ਖ਼نԽ/ਖ਼نԽ/υϩοϓΞ΢τʣ ✓ ΫϩεΤϯτϩϐʔίετؔ਺ ✓ ιϑτϚοΫεؔ਺ ✓ ॏΈॳظԽͷվળ ͦͷଞͷٕज़

Slide 58

Slide 58 text

σΟʔϓϥʔχϯά͸ٕज़ͷू߹ମ ʮσΟʔϓϥʔχϯάʯͱ͍͏ΞϧΰϦζϜ͸ଘࡏ͠ͳ͍ ໨తʹԠͯ͡Ϟσϧ΍ٕज़ΛύζϧͷΑ͏ʹ૊Έཱͯͯɺ  ΑΓߴ͍ਫ਼౓ΛੜΈग़͢͜ͱ͕Ͱ͖Δ

Slide 59

Slide 59 text

ϋϚͬͨ͜ͱ

Slide 60

Slide 60 text

ϋϚͬͨᶃ  ਺͕ࣜ಄ͷதʹೖͬͯ͜ͳ͍໰୊ ීஈ࢖͍׳ΕͨϓϩάϥϜʹͯ͠ΈΕ͹  ҙ֎ͱʮͳΜͩͦΜͳ͜ͱ͔ʯͰࡁΉ͜ͱ΋͋Δ function sigmoid(z) { return 1 / (1 + Math.exp(-z)); } let output = sigmoid(w.dot(a).plus(b));

Slide 61

Slide 61 text

ܭࢉํ๏͸෼͔Βͳ͍ͷͰɺภඍ෼ͨ݁͠Ռͷ਺ࣜΛ  stackoverﬂow͔ΒҾͬு͖ͬͯͨΒಈ͍ͨ costDelta(y) { this.outputDropout.minus(y); } ϋϚͬͨᶄ  ภඍ෼ͷܭࢉํ๏͕෼͔Βͳ͍໰୊

Slide 62

Slide 62 text

ڭՊॻ௨Γʹ࣮૷͢ΔͱιϑτϚοΫεؔ਺͕ܻ͋;Ε͢Δ ·ͨ΋΍stackoverﬂow͔ΒҾͬு͖ͬͯͨΒಈ͍ͨ ϋϚͬͨᶅ  ڭՊॻʹ͸ॻ͍͍ͯͳ͍໰୊ let max = _.max(vector), tmp = _.map(vector, (v) => { return Math.exp(v - max); }), sum = _.sum(tmp); return _.map(tmp, (v) => { return v / sum; });

Slide 63

Slide 63 text

PythonͰ1࣌ؒͷͱ͜Ζ͕24͔͔࣌ؒΔ ཪͰNumpy͕ੌ͍͜ͱ΍ͬͯ͘ΕͯΔΒ͍͠ ·ͩΫϥελϦϯά΋Ͱ͖ͯͳ͍͠ɺϝϞϦޮ཰΋ѱ͍ ϋϚͬͨᶆ  Զͷઢܗ୅਺ͷܭࢉͦ͘஗͍໰୊ ։ൃ࣌͸খ͞ͳσʔληοτͰରԠ

Slide 64

Slide 64 text

ࢀߟ࣮૷͕Theano΍TensorFlowͳͲͷߴػೳͳ  ϥΠϒϥϦΛ࢖༻͍ͯ͠ΔͱԿ΍ͬͯΔͷ͔෼͔Βͳ͍ ࣗಈඍ෼ͬͯͳΜ΍ͶΜ ϋϚͬͨᶇ  ϥΠϒϥϦߴػೳա͗໰୊ ؾ͍ͮͨΒPythonʹৄ͘͠ͳͬͯͨ

Slide 65

Slide 65 text

…

Slide 66

Slide 66 text

WIP

Slide 67

Slide 67 text

IUUQTHJUIVCDPNZVKJPTBLBEFFQMFBSOJOHKT ZVKJPTBLBEFFQMFBSOJOHKT %FFQ-FBSOJOHXSJUUFOJO&4

Slide 68

Slide 68 text

σϞ

Slide 69

Slide 69 text

ਖ਼ղ཰99.1% ※ ࣮ͨͩ͠ߦ࣌ؒ24࣌ؒҎ্

Slide 70

Slide 70 text

ຊ൪؀ڥͰ͸  PythonΛ࢖͍·͠ΐ͏

Slide 71

Slide 71 text

ͻͨ͢Βָͯ͠  σΟʔϓϥʔχϯά ָ͠ΜͰ

Slide 72

Slide 72 text

$BSFFST ࠾༻৘ใ &NPUJPO*0ͷςΫϊϩδʔͰɺ ʮҰาઌͷະདྷΛΑΓ๛͔ʹ͢Δ࢓૊ΈΛ࡞ΔʯϝϯόʔΛืूதͰ͢ɻ ืूதͷ৬छΛݟΔ WE ARE HIRING! https://www.emin.co.jp/careers/