ひたすら楽してディープラーニング

 ひたすら楽してディープラーニング

5d247ff63b1861db5e6a56d4990e5a4f?s=128

yujiosaka

April 05, 2016
Tweet

Transcript

  1. Yuji Isobe ͻͨ͢Βָͯ͠
 σΟʔϓϥʔχϯά NodeֶԂ20࣌ݶ໨

  2. [ “Node.js”, “MongoDB”, “AngularJS”, “socket.io”, “Emotion Intelligence “
 ] @yujiosaka

    +BWB4DSJQU
  3. emin = Emotion Intelligence ؾ࣋ͪΛղ͢ΔςΫϊϩδʔͷ୳ڀ Emotion Intelligence͸ɺʮແҙࣝͷߦಈ͔Βɺ ਓͷؾ࣋ͪͷػඍΛղ͢Δ஌ੑʯΛɺਓ޻஌ೳ͓Αͼػցֶ शͷԠ༻ٕज़Λ༻͍ͯ ։ൃ͠ɺϏδωεʹԠ༻͍ͯ͠·͢ɻ

  4. ZenClerk Series lϢʔβʔͷߪೖҙཉͷߴ·ΓzΛ࡯஌͢ΔγεςϜͰɺ
 ΋ͬͱαΠτͷച্Λ৳͹ͯ͠Έ·ͤΜ͔ʁ [FODMFSLMJUF ײ৘Λղੳ͢Δਓ޻஌ೳʮ&NPUJPO*0ʯ͕
 &$αΠτ্Ͱʮ࠷దͳλΠϛϯάͰͷൢଅʯΛ࣮ݱ͠ɺച্Λ޲্ͤ͞·͢ɻ ࠓ·͞ʹߪೖΛ໎͍ͬͯΔ͓٬༷ʹ
 ͏Ε͍͠ʮ࠷ޙͷͻͱԡ͠ʯΛఏڙ͠·͢ɻ ;FODMFSL

  5. ػցֶशͱݴ͑͹…

  6. Lee Sedol vs. AlphaGo

  7. Google Trends

  8. ࠔͬͨ(´ɾωɾʆ)

  9. ͜ͷձࣾʹೖΔ·ͰҰ౓΋
 ػցֶश΍ͬͯ͜ͳ͔ͬͨorz

  10. ͱΓ͋͑ͣೖ໳ॻΛಡΉ ✓ ϐϯΫͷബ͍ຊ ✓ ௨শʮ͸͡ύλຊʯ ✓ ݟͨ໨ΑΓࠎଠͳ಺༰ ✓ ͪΌΜͱ਺͕ࣜࡌͬͯΔ
 ʢͦͯͭ͠·ͮ͘ʣ

  11. ୈ3ষɿϕΠζ ୈ5ষɿkNN๏ ୈ6ষɿҰൠԽઢܗϞσϧ ୈ7ষɿχϡʔϥϧωοτϫʔΫ ୈ8ষɿαϙʔτϕΫλʔϚγϯ ӽ͑ΒΕͳ͍น ← ෼͔Δ ← ෼͔Δ

    ← ෼͔Δ ← ͓ɺ͓͏… ← (ɾ㱼ɾ) ŜŠŘŘ!!
  12. σʔλՄࢹԽ ػցֶश ਺ֶ ౷ܭֶ ܭࢉػՊֶ ίϛϡχέʔγϣϯ υϝΠϯ஌ࣝ ࣗ෼ͷεΩϧηοτ ͜͜Λ৳͹͍ͨ͠

  13. Kaggle͸اۀ΍ݚڀऀ͕σʔλΛ౤ߘ͠ɺੈքதͷ ౷ܭՈ΍σʔλ෼ੳՈ͕ͦͷ࠷దϞσϧΛڝ͍߹͏ɺ ༧ଌϞσϦϯάٴͼ෼ੳख๏ؔ࿈ϓϥοτϑΥʔϜٴ ͼͦͷӡӦձࣾͰ͋Δɻ ΢ΟΩϖσΟΞ ૑ཱऀɿ ΞϯιχʔɾΰʔϧυϒϧʔϜ ઃཱɿ 2010೥4݄

  14. None
  15. ࠓճ͸͜Εʹ
 νϟϨϯδ

  16. ✓ MNISTʢ਺ࣈͷखॻ͖σʔλʣͷ෼ྨ ✓ 28 x 28ϐΫηϧ ✓ 6ສ݅ͷֶशσʔλ ✓ 1ສ݅ͷςετσʔλ

    νϡʔτϦΞϧ՝୊
  17. ਖ਼ղ཰99%Λ໨ࢦ͢ http://yann.lecun.com/exdb/mnist/

  18. Ͱ΋ͳΜ͔೉ͦ͠͏…

  19. Ͱ͖Ε͹ͻͨ͢Βָ͍ͨ͠

  20. None
  21. ͻͨ͢Βָͯ͠FF6

  22. 1.࠷খউརճ਺ͰΫϦΞ 2.ඞਢઓಆҎ֎Ͱ౪·ͳ͍ 3.ಓதͷΞΠςϜ͸ճऩ͠ͳ͍ 4.ฤू͸ؤுΒͳ͍ ͻͨ͢Βָͯ͠FF6

  23. 1.࠷ۙͷϒʔϜʹ৐͔ͬΔ 2.ۤखͳݴޠ͸࢖Θͳ͍ 3.ߴ౓ͳϥΠϒϥϦ͸࢖Θͳ͍ 4.ϓϨθϯࢿྉ͸ؤுΒͳ͍ ͻͨ͢Βָͯ͠Kaggle ͱΓ͋͑ͣσΟʔϓϥʔχϯά
 ࢖ͬͯΈΔ +BWB4DSJQU͔͠࢖Θͳ͍ MPEBTIͱઢܗ୅਺ϥΠϒϥϦ
 ͑͋͞Ε͹͍͍

    ΤϯδχΞͩ͠ίʔυͰউෛ
  24. ·ͣ͸৘ใऩू͔Β

  25. IUUQOFVSBMOFUXPSLTBOEEFFQMFBSOJOHDPNJOEFYIUNM

  26. ✓ ӳޠͷΦϯϥΠϯϒοΫʢ೔ຊޠԽ΋ਐߦதʣ ✓ χϡʔϥϧωοτϫʔΫ͔Β
 σΟʔϓϥʔχϯά·ͰͷྲྀΕΛৄࡉʹղઆ ✓ Pythonͷαϯϓϧ࣮૷͕ಡΈ΍͍͢ Neural Networks and

    Deep Learning Φεεϝʂ
  27. ઓུΛཱͯΔ

  28. Python→CoffeeScript→ES2015 sed + ؾ߹ Decaf JS ੈքॳʁES2015ͰσΟʔϓϥʔχϯά npmௐ΂ JavaScript Babel

  29. PythonͱCoffeeScriptͬͯࣅͯͶʁ

  30. Python def update_mini_batch(self, mini_batch, eta): nabla_b = [np.zeros(b.shape) for b

    in self.biases] nabla_w = [np.zeros(w.shape) for w in self.weights] for x, y in mini_batch: delta_nabla_b, delta_nabla_w = self.backprop(x, y) nabla_b = [nb+dnb for nb, dnb in zip(nabla_b, delta_nabla_b)] nabla_w = [nw+dnw for nw, dnw in zip(nabla_w, delta_nabla_w)] self.weights = [w-(eta/len(mini_batch))*nw for w, nw in zip(self.weights, nabla_w)] self.biases = [b-(eta/len(mini_batch))*nb for b, nb in zip(self.biases, nabla_b)]
  31. CoffeeScript updateMiniBatch: (miniBatch, eta) -> nablaB = (Matrix.zeros(b.rows, b.cols) for

    b in @biases) nablaW = (Matrix.zeros(w.rows, w.cols) for w in @weights) for [x, y] in miniBatch [deltaNablaB, deltaNablaW] = @backprop(x, y) nablaB = (nb.plus(dnb) for [nb, dnb] in _.zip(nablaB, deltaNablaB)) nablaW = (nw.plus(dnw) for [nw, dnw] in _.zip(nablaW, deltaNablaW)) @weights = (w.minus(nw.mulEach(eta / miniBatch.length))
 for [w, nw] in _.zip(@weights, nablaW)) @biases = (b.minus(nb.mulEach(eta / miniBatch.length))
 for [b, nb] in _.zip(@biases, nablaB))
  32. PythonϥΠϒϥϦͷAPIΛ࣮૷͢Δ

  33. numpy.nan_to_num nanToNum() { let thisData = this.data, rows = this.rows,

    cols = this.cols; let row, col, result = new Array(rows); for (row=0; row<rows; ++row) { result[row] = new Array(cols); for (col=0; col<cols; ++col) { result[row][col] = n2n(thisData[row][col]); } } return new Matrix(result); };
  34. numpy.ravel ravel() { let thisData = this.data, rows = this.rows,

    cols = this.cols; let a = new Array(rows * cols); for (let i = 0, jBase = 0; i<rows; ++i, jBase += cols) { for (let j = 0; j<cols; ++j) { a[jBase + j] = thisData[i][j]; } } return a; };
  35. CoffeeScript͕ڐ͞ΕΔͷ͸2015೥·ͰͩΑͶʔ https://github.com/juliankrispel/decaf

  36. ಈ͍ͨস

  37. ͦΖͦΖͪΌΜͱษڧ͢Δ

  38. χϡʔϥϧωοτϫʔΫ ਆܦճ࿏໢ɺӳOFVSBMOFUXPSL // ͸ɺ೴ػೳʹݟΒΕΔ͍͔ͭ͘ͷಛੑΛܭࢉػ ্ͷγϛϡϨʔγϣϯʹΑͬͯදݱ͢Δ͜ͱΛ໨ࢦͨ͠਺ֶϞσϧͰ͋Δɻ χϡʔϥϧωοτϫʔΫ8JLJQFEJB IUUQTKBXJLJQFEJBPSHXJLJχϡʔϥϧωοτϫʔΫ χϡʔϥϧωοτϫʔΫͱ͸ʁ

  39. b ύʔηϓτϩϯϞσϧ x1 x2 x3 output w1 w2 w3 PVUQVU

    JGЄKXKYKC≤
 JGЄKXKYKC
  40. 5 ύʔηϓτϩϯϞσϧ ఱؾ͸ྑ͍͔ʁ ൴ঁ͸ߦ͖͍͔ͨʁ ձ৔͸Ӻͷ͔ۙ͘ʁ ͓ࡇΓʹߦ͘ʁ 6 2 2 No

    Yes Yes No   ≤
  41. b γάϞΠυχϡʔϩϯϞσϧ x1 x2 x3 w1 w2 w3 PVUQVU 


     FYQ ЄKXKYKC output
  42. εςοϓؔ਺ʢύʔηϓτϩϯʣ         

  43. γάϞΠυؔ਺         

  44. ✓ 0ͱ1ͷؒͷ஋ΛදݱͰ͖ΔΑ͏ʹͳͬͨ ✓ খ͞ͳೖྗͷมԽ͕খ͞ͳग़ྗͷมԽʹରԠ͢Δ ✓ ཁ͢Δʹඍ෼Ͱ͖Δ͜ͱʹͳͬͨ Կ͕خ͍͠ͷ͔ʁ

  45. χϡʔϥϧωοτϫʔΫͷߏ଄ w + Δw
 b + Δb output + Δoutput

  46. ✓ ֤χϡʔϩϯʢɹʣͷॏΈʢwʣͱόΠΞεʢbʣ Λௐ੔͢Δ͜ͱͰ༧ଌਫ਼౓Λ޲্͢Δ ✓ ͦͷͨΊͷςΫχοΫͱͯ͠ɺ
 όοΫϓϩύήʔγϣϯͳͲ͕ੜΈग़͞Εͨ χϡʔϥϧωοτϫʔΫͷֶश

  47. σΟʔϓϥʔχϯάͱ͸ʁ

  48. χϡʔϥϧωοτϫʔΫͷߏ଄

  49. σΟʔϓϥʔχϯάͷߏ଄

  50. ࠷ۙͷϒʔϜͷഎܠ ✓ ࠷ۙςΫχοΫ͕͍ͭ͘΋ݟ͔ͭͬͨ ✓ ૚ΛॏͶͯ΋͏·ֶ͘शͰ͖ΔΑ͏ʹͳͬͨ ✓ ૚ΛॏͶΔ͜ͱͰදݱೳྗͱਫ਼౓͕޲্ͨ͠

  51. ͍࣮͟૷΁

  52. ৞ΈࠐΈχϡʔϥϧωοτϫʔΫ

  53. ໰୊ ͨͬͨ1pxͣΕΔ͚ͩͰผͷը૾ͱͯ͠ೝࣝ͞Εͯ͠·͏ 1px

  54. ղܾΠϝʔδ

  55. ߏ଄ convolutional layer pooling layer

  56. ✓ γάϞΠυؔ਺Ҏ֎ͷ׆ੑԽؔ਺ʢReLUʣ ✓ ਖ਼نԽʢL1ਖ਼نԽ/ਖ਼نԽ/υϩοϓΞ΢τʣ ✓ ΫϩεΤϯτϩϐʔίετؔ਺ ✓ ιϑτϚοΫεؔ਺ ✓ ॏΈॳظԽͷվળ

    ͦͷଞͷٕज़
  57. σΟʔϓϥʔχϯά͸ٕज़ͷू߹ମ ʮσΟʔϓϥʔχϯάʯͱ͍͏ΞϧΰϦζϜ͸ଘࡏ͠ͳ͍ ໨తʹԠͯ͡Ϟσϧ΍ٕज़ΛύζϧͷΑ͏ʹ૊Έཱͯͯɺ
 ΑΓߴ͍ਫ਼౓ΛੜΈग़͢͜ͱ͕Ͱ͖Δ

  58. ϋϚͬͨ͜ͱ

  59. ϋϚͬͨᶃ
 ਺͕ࣜ಄ͷதʹೖͬͯ͜ͳ͍໰୊ ීஈ࢖͍׳ΕͨϓϩάϥϜʹͯ͠ΈΕ͹
 ҙ֎ͱʮͳΜͩͦΜͳ͜ͱ͔ʯͰࡁΉ͜ͱ΋͋Δ function sigmoid(z) { return 1 /

    (1 + Math.exp(-z)); } let output = sigmoid(w.dot(a).plus(b));
  60. ܭࢉํ๏͸෼͔Βͳ͍ͷͰɺภඍ෼ͨ݁͠Ռͷ਺ࣜΛ
 stackoverflow͔ΒҾͬு͖ͬͯͨΒಈ͍ͨ costDelta(y) { this.outputDropout.minus(y); } ϋϚͬͨᶄ
 ภඍ෼ͷܭࢉํ๏͕෼͔Βͳ͍໰୊

  61. ڭՊॻ௨Γʹ࣮૷͢ΔͱιϑτϚοΫεؔ਺͕ܻ͋;Ε͢Δ ·ͨ΋΍stackoverflow͔ΒҾͬு͖ͬͯͨΒಈ͍ͨ ϋϚͬͨᶅ
 ڭՊॻʹ͸ॻ͍͍ͯͳ͍໰୊ let max = _.max(vector), tmp =

    _.map(vector, (v) => { return Math.exp(v - max); }), sum = _.sum(tmp); return _.map(tmp, (v) => { return v / sum; });
  62. PythonͰ1࣌ؒͷͱ͜Ζ͕24͔͔࣌ؒΔ ཪͰNumpy͕ੌ͍͜ͱ΍ͬͯ͘ΕͯΔΒ͍͠ ·ͩΫϥελϦϯά΋Ͱ͖ͯͳ͍͠ɺϝϞϦޮ཰΋ѱ͍ ϋϚͬͨᶆ
 Զͷઢܗ୅਺ͷܭࢉͦ͘஗͍໰୊ ։ൃ࣌͸খ͞ͳσʔληοτͰରԠ

  63. ࢀߟ࣮૷͕Theano΍TensorFlowͳͲͷߴػೳͳ
 ϥΠϒϥϦΛ࢖༻͍ͯ͠ΔͱԿ΍ͬͯΔͷ͔෼͔Βͳ͍ ࣗಈඍ෼ͬͯͳΜ΍ͶΜ ϋϚͬͨᶇ
 ϥΠϒϥϦߴػೳա͗໰୊ ؾ͍ͮͨΒPythonʹৄ͘͠ͳͬͯͨ

  64. WIP

  65. IUUQTHJUIVCDPNZVKJPTBLBEFFQMFBSOJOHKT ZVKJPTBLBEFFQMFBSOJOHKT %FFQ-FBSOJOHXSJUUFOJO&4

  66. σϞ

  67. ਖ਼ղ཰99.1% ※ ࣮ͨͩ͠ߦ࣌ؒ24࣌ؒҎ্

  68. ຊ൪؀ڥͰ͸
 PythonΛ࢖͍·͠ΐ͏

  69. ͻͨ͢Βָͯ͠
 σΟʔϓϥʔχϯά ָ͠ΜͰ

  70. $BSFFST ࠾༻৘ใ &NPUJPO*0ͷςΫϊϩδʔͰɺ ʮҰาઌͷະདྷΛΑΓ๛͔ʹ͢Δ࢓૊ΈΛ࡞ΔʯϝϯόʔΛืूதͰ͢ɻ ืूதͷ৬छΛݟΔ WE ARE HIRING! https://www.emin.co.jp/careers/