ひたすら楽してディープラーニング

Transcript

1. Yuji Isobe
   ͻͨ͢Βָͯ͠

   σΟʔϓϥʔχϯά
   NodeֶԂ20࣌ݶ໨
   

   View Slide

2. [
   “Node.js”,
   “MongoDB”,
   “AngularJS”,
   “socket.io”,
   “Emotion Intelligence “

   ]
   @yujiosaka
   +BWB4DSJQU
   

   View Slide

3. emin = Emotion Intelligence
   ؾ࣋ͪΛղ͢ΔςΫϊϩδʔͷ୳ڀ
   Emotion Intelligence͸ɺʮແҙࣝͷߦಈ͔Βɺ
   ਓͷؾ࣋ͪͷػඍΛղ͢Δ஌ੑʯΛɺਓ޻஌ೳ͓Αͼػցֶ
   शͷԠ༻ٕज़Λ༻͍ͯ
   ։ൃ͠ɺϏδωεʹԠ༻͍ͯ͠·͢ɻ
   

   View Slide

4. ZenClerk Series
   lϢʔβʔͷߪೖҙཉͷߴ·ΓzΛ࡯஌͢ΔγεςϜͰɺ

   ΋ͬͱαΠτͷച্Λ৳͹ͯ͠Έ·ͤΜ͔ʁ
   [FODMFSLMJUF
   ײ৘Λղੳ͢Δਓ޻஌ೳʮ&NPUJPO*0ʯ͕

   &$αΠτ্Ͱʮ࠷దͳλΠϛϯάͰͷൢଅʯΛ࣮ݱ͠ɺച্Λ޲্ͤ͞·͢ɻ
   ࠓ·͞ʹߪೖΛ໎͍ͬͯΔ͓٬༷ʹ

   ͏Ε͍͠ʮ࠷ޙͷͻͱԡ͠ʯΛఏڙ͠·͢ɻ
   ;FODMFSL
   

   View Slide

5. ػցֶशͱݴ͑͹…
   

   View Slide

6. Lee Sedol vs. AlphaGo
   

   View Slide

7. Google Trends
   

   View Slide

8. ࠔͬͨ(´ɾωɾʆ)
   

   View Slide

9. ͜ͷձࣾʹೖΔ·ͰҰ౓΋

   ػցֶश΍ͬͯ͜ͳ͔ͬͨorz
   

   View Slide

10. ͱΓ͋͑ͣೖ໳ॻΛಡΉ
    ✓ ϐϯΫͷബ͍ຊ
    ✓ ௨শʮ͸͡ύλຊʯ
    ✓ ݟͨ໨ΑΓࠎଠͳ಺༰
    ✓ ͪΌΜͱ਺͕ࣜࡌͬͯΔ

    ʢͦͯͭ͠·ͮ͘ʣ
    

    View Slide

11. ୈ3ষɿϕΠζ
    ୈ5ষɿkNN๏
    ୈ6ষɿҰൠԽઢܗϞσϧ
    ୈ7ষɿχϡʔϥϧωοτϫʔΫ
    ୈ8ষɿαϙʔτϕΫλʔϚγϯ
    ӽ͑ΒΕͳ͍น
    ← ෼͔Δ
    ← ෼͔Δ
    ← ෼͔Δ
    ← ͓ɺ͓͏…
    ← (ɾ㱼ɾ) ŜŠŘŘ!!
    

    View Slide

12. σʔλՄࢹԽ
    ػցֶश
    ਺ֶ ౷ܭֶ
    ܭࢉػՊֶ
    ίϛϡχέʔγϣϯ
    υϝΠϯ஌ࣝ
    ࣗ෼ͷεΩϧηοτ
    ͜͜Λ৳͹͍ͨ͠
    

    View Slide

13. Kaggle͸اۀ΍ݚڀऀ͕σʔλΛ౤ߘ͠ɺੈքதͷ
    ౷ܭՈ΍σʔλ෼ੳՈ͕ͦͷ࠷దϞσϧΛڝ͍߹͏ɺ
    ༧ଌϞσϦϯάٴͼ෼ੳख๏ؔ࿈ϓϥοτϑΥʔϜٴ
    ͼͦͷӡӦձࣾͰ͋Δɻ ΢ΟΩϖσΟΞ
    ૑ཱऀɿ ΞϯιχʔɾΰʔϧυϒϧʔϜ
    ઃཱɿ 2010೥4݄
    

    View Slide

14. View Slide

15. ࠓճ͸͜Εʹ

    νϟϨϯδ
    

    View Slide

16. ✓ MNISTʢ਺ࣈͷखॻ͖σʔλʣͷ෼ྨ
    ✓ 28 x 28ϐΫηϧ
    ✓ 6ສ݅ͷֶशσʔλ
    ✓ 1ສ݅ͷςετσʔλ
    νϡʔτϦΞϧ՝୊
    

    View Slide

17. ਖ਼ղ཰99%Λ໨ࢦ͢
    http://yann.lecun.com/exdb/mnist/
    

    View Slide

18. Ͱ΋ͳΜ͔೉ͦ͠͏…
    

    View Slide

19. Ͱ͖Ε͹ͻͨ͢Βָ͍ͨ͠
    

    View Slide

20. View Slide

21. ͻͨ͢Βָͯ͠FF6
    

    View Slide

22. 1.࠷খউརճ਺ͰΫϦΞ
    2.ඞਢઓಆҎ֎Ͱ౪·ͳ͍
    3.ಓதͷΞΠςϜ͸ճऩ͠ͳ͍
    4.ฤू͸ؤுΒͳ͍
    ͻͨ͢Βָͯ͠FF6
    

    View Slide

23. 1.࠷ۙͷϒʔϜʹ৐͔ͬΔ
    2.ۤखͳݴޠ͸࢖Θͳ͍
    3.ߴ౓ͳϥΠϒϥϦ͸࢖Θͳ͍
    4.ϓϨθϯࢿྉ͸ؤுΒͳ͍
    ͻͨ͢Βָͯ͠Kaggle
    ͱΓ͋͑ͣσΟʔϓϥʔχϯά

    ࢖ͬͯΈΔ
    +BWB4DSJQU͔͠࢖Θͳ͍
    MPEBTIͱઢܗ୅਺ϥΠϒϥϦ

    ͑͋͞Ε͹͍͍
    ΤϯδχΞͩ͠ίʔυͰউෛ
    

    View Slide

24. ·ͣ͸৘ใऩू͔Β
    

    View Slide

25. IUUQOFVSBMOFUXPSLTBOEEFFQMFBSOJOHDPNJOEFYIUNM
    

    View Slide

26. ✓ ӳޠͷΦϯϥΠϯϒοΫʢ೔ຊޠԽ΋ਐߦதʣ
    ✓ χϡʔϥϧωοτϫʔΫ͔Β

    σΟʔϓϥʔχϯά·ͰͷྲྀΕΛৄࡉʹղઆ
    ✓ Pythonͷαϯϓϧ࣮૷͕ಡΈ΍͍͢
    Neural Networks and Deep Learning
    Φεεϝʂ
    

    View Slide

27. ઓུΛཱͯΔ
    

    View Slide

28. Python→CoffeeScript→ES2015
    sed + ؾ߹ Decaf JS
    ੈքॳʁES2015ͰσΟʔϓϥʔχϯά
    npmௐ΂
    JavaScript
    Babel
    

    View Slide

29. PythonͱCoffeeScriptͬͯࣅͯͶʁ
    

    View Slide

30. Python
    def update_mini_batch(self, mini_batch, eta):
    nabla_b = [np.zeros(b.shape) for b in self.biases]
    nabla_w = [np.zeros(w.shape) for w in self.weights]
    for x, y in mini_batch:
    delta_nabla_b, delta_nabla_w = self.backprop(x, y)
    nabla_b = [nb+dnb for nb, dnb in zip(nabla_b, delta_nabla_b)]
    nabla_w = [nw+dnw for nw, dnw in zip(nabla_w, delta_nabla_w)]
    self.weights = [w-(eta/len(mini_batch))*nw
    for w, nw in zip(self.weights, nabla_w)]
    self.biases = [b-(eta/len(mini_batch))*nb
    for b, nb in zip(self.biases, nabla_b)]
    

    View Slide

31. CoffeeScript
    updateMiniBatch: (miniBatch, eta) ->
    nablaB = (Matrix.zeros(b.rows, b.cols) for b in @biases)
    nablaW = (Matrix.zeros(w.rows, w.cols) for w in @weights)
    for [x, y] in miniBatch
    [deltaNablaB, deltaNablaW] = @backprop(x, y)
    nablaB = (nb.plus(dnb) for [nb, dnb] in _.zip(nablaB, deltaNablaB))
    nablaW = (nw.plus(dnw) for [nw, dnw] in _.zip(nablaW, deltaNablaW))
    @weights = (w.minus(nw.mulEach(eta / miniBatch.length))

    for [w, nw] in _.zip(@weights, nablaW))
    @biases = (b.minus(nb.mulEach(eta / miniBatch.length))

    for [b, nb] in _.zip(@biases, nablaB))
    

    View Slide

32. PythonϥΠϒϥϦͷAPIΛ࣮૷͢Δ
    

    View Slide

33. numpy.nan_to_num
    nanToNum() {
    let thisData = this.data,
    rows = this.rows,
    cols = this.cols;
    let row, col, result = new Array(rows);
    for (row=0; rowresult[row] = new Array(cols);
    for (col=0; colresult[row][col] = n2n(thisData[row][col]);
    }
    }
    return new Matrix(result);
    };
    

    View Slide

34. numpy.ravel
    ravel() {
    let thisData = this.data,
    rows = this.rows,
    cols = this.cols;
    let a = new Array(rows * cols);
    for (let i = 0, jBase = 0; ifor (let j = 0; ja[jBase + j] = thisData[i][j];
    }
    }
    return a;
    };
    

    View Slide

35. CoffeeScript͕ڐ͞ΕΔͷ͸2015೥·ͰͩΑͶʔ
    https://github.com/juliankrispel/decaf
    

    View Slide

36. …
    

    View Slide

37. ಈ͍ͨস
    

    View Slide

38. ͦΖͦΖͪΌΜͱษڧ͢Δ
    

    View Slide

39. χϡʔϥϧωοτϫʔΫ ਆܦճ࿏໢ɺӳOFVSBMOFUXPSL //
    ͸ɺ೴ػೳʹݟΒΕΔ͍͔ͭ͘ͷಛੑΛܭࢉػ
    ্ͷγϛϡϨʔγϣϯʹΑͬͯදݱ͢Δ͜ͱΛ໨ࢦͨ͠਺ֶϞσϧͰ͋Δɻ
    χϡʔϥϧωοτϫʔΫ8JLJQFEJB
    IUUQTKBXJLJQFEJBPSHXJLJχϡʔϥϧωοτϫʔΫ
    χϡʔϥϧωοτϫʔΫͱ͸ʁ
    

    View Slide

40. b
    ύʔηϓτϩϯϞσϧ
    x1
    x2
    x3
    output
    w1
    w2
    w3
    PVUQVU
    JGЄKXKYKC≤

    JGЄKXKYKC
    

    View Slide

41. 5
    ύʔηϓτϩϯϞσϧ
    ఱؾ͸ྑ͍͔ʁ
    ൴ঁ͸ߦ͖͍͔ͨʁ
    ձ৔͸Ӻͷ͔ۙ͘ʁ
    ͓ࡇΓʹߦ͘ʁ
    6
    2
    2
    No
    Yes
    Yes
    No
    ≤
    

    View Slide

42. b
    γάϞΠυχϡʔϩϯϞσϧ
    x1
    x2
    x3
    w1
    w2
    w3
    PVUQVU
    

    FYQ ЄKXKYKC
    
    output
    

    View Slide

43. εςοϓؔ਺ʢύʔηϓτϩϯʣ
    
    

    View Slide

44. γάϞΠυؔ਺
    
    

    View Slide

45. ✓ 0ͱ1ͷؒͷ஋ΛදݱͰ͖ΔΑ͏ʹͳͬͨ
    ✓ খ͞ͳೖྗͷมԽ͕খ͞ͳग़ྗͷมԽʹରԠ͢Δ
    ✓ ཁ͢Δʹඍ෼Ͱ͖Δ͜ͱʹͳͬͨ
    Կ͕خ͍͠ͷ͔ʁ
    

    View Slide

46. χϡʔϥϧωοτϫʔΫͷߏ଄
    w + Δw

    b + Δb
    output + Δoutput
    

    View Slide

47. ✓ ֤χϡʔϩϯʢɹʣͷॏΈʢwʣͱόΠΞεʢbʣ
    Λௐ੔͢Δ͜ͱͰ༧ଌਫ਼౓Λ޲্͢Δ
    ✓ ͦͷͨΊͷςΫχοΫͱͯ͠ɺ

    όοΫϓϩύήʔγϣϯͳͲ͕ੜΈग़͞Εͨ
    χϡʔϥϧωοτϫʔΫͷֶश
    

    View Slide

48. σΟʔϓϥʔχϯάͱ͸ʁ
    

    View Slide

49. χϡʔϥϧωοτϫʔΫͷߏ଄
    

    View Slide

50. σΟʔϓϥʔχϯάͷߏ଄
    

    View Slide

51. ࠷ۙͷϒʔϜͷഎܠ
    ✓ ࠷ۙςΫχοΫ͕͍ͭ͘΋ݟ͔ͭͬͨ
    ✓ ૚ΛॏͶͯ΋͏·ֶ͘शͰ͖ΔΑ͏ʹͳͬͨ
    ✓ ૚ΛॏͶΔ͜ͱͰදݱೳྗͱਫ਼౓͕޲্ͨ͠
    

    View Slide

52. ͍࣮͟૷΁
    

    View Slide

53. ৞ΈࠐΈχϡʔϥϧωοτϫʔΫ
    

    View Slide

54. ໰୊
    ͨͬͨ1pxͣΕΔ͚ͩͰผͷը૾ͱͯ͠ೝࣝ͞Εͯ͠·͏
    1px
    

    View Slide

55. ղܾΠϝʔδ
    

    View Slide

56. ߏ଄
    convolutional layer
    pooling layer
    

    View Slide

57. ✓ γάϞΠυؔ਺Ҏ֎ͷ׆ੑԽؔ਺ʢReLUʣ
    ✓ ਖ਼نԽʢL1ਖ਼نԽ/ਖ਼نԽ/υϩοϓΞ΢τʣ
    ✓ ΫϩεΤϯτϩϐʔίετؔ਺
    ✓ ιϑτϚοΫεؔ਺
    ✓ ॏΈॳظԽͷվળ
    ͦͷଞͷٕज़
    

    View Slide

58. σΟʔϓϥʔχϯά͸ٕज़ͷू߹ମ
    ʮσΟʔϓϥʔχϯάʯͱ͍͏ΞϧΰϦζϜ͸ଘࡏ͠ͳ͍
    ໨తʹԠͯ͡Ϟσϧ΍ٕज़ΛύζϧͷΑ͏ʹ૊Έཱͯͯɺ

    ΑΓߴ͍ਫ਼౓ΛੜΈग़͢͜ͱ͕Ͱ͖Δ
    

    View Slide

59. ϋϚͬͨ͜ͱ
    

    View Slide

60. ϋϚͬͨᶃ

    ਺͕ࣜ಄ͷதʹೖͬͯ͜ͳ͍໰୊
    ීஈ࢖͍׳ΕͨϓϩάϥϜʹͯ͠ΈΕ͹

    ҙ֎ͱʮͳΜͩͦΜͳ͜ͱ͔ʯͰࡁΉ͜ͱ΋͋Δ
    function sigmoid(z) {
    return 1 / (1 + Math.exp(-z));
    }
    let output = sigmoid(w.dot(a).plus(b));
    

    View Slide

61. ܭࢉํ๏͸෼͔Βͳ͍ͷͰɺภඍ෼ͨ݁͠Ռͷ਺ࣜΛ

    stackoverflow͔ΒҾͬு͖ͬͯͨΒಈ͍ͨ
    costDelta(y) {
    this.outputDropout.minus(y);
    }
    ϋϚͬͨᶄ

    ภඍ෼ͷܭࢉํ๏͕෼͔Βͳ͍໰୊
    

    View Slide

62. ڭՊॻ௨Γʹ࣮૷͢ΔͱιϑτϚοΫεؔ਺͕ܻ͋;Ε͢Δ
    ·ͨ΋΍stackoverflow͔ΒҾͬு͖ͬͯͨΒಈ͍ͨ
    ϋϚͬͨᶅ

    ڭՊॻʹ͸ॻ͍͍ͯͳ͍໰୊
    let max = _.max(vector),
    tmp = _.map(vector, (v) => { return Math.exp(v - max); }),
    sum = _.sum(tmp);
    return _.map(tmp, (v) => { return v / sum; });
    

    View Slide

63. PythonͰ1࣌ؒͷͱ͜Ζ͕24͔͔࣌ؒΔ
    ཪͰNumpy͕ੌ͍͜ͱ΍ͬͯ͘ΕͯΔΒ͍͠
    ·ͩΫϥελϦϯά΋Ͱ͖ͯͳ͍͠ɺϝϞϦޮ཰΋ѱ͍
    ϋϚͬͨᶆ

    Զͷઢܗ୅਺ͷܭࢉͦ͘஗͍໰୊
    ։ൃ࣌͸খ͞ͳσʔληοτͰରԠ
    

    View Slide

64. ࢀߟ࣮૷͕Theano΍TensorFlowͳͲͷߴػೳͳ

    ϥΠϒϥϦΛ࢖༻͍ͯ͠ΔͱԿ΍ͬͯΔͷ͔෼͔Βͳ͍
    ࣗಈඍ෼ͬͯͳΜ΍ͶΜ
    ϋϚͬͨᶇ

    ϥΠϒϥϦߴػೳա͗໰୊
    ؾ͍ͮͨΒPythonʹৄ͘͠ͳͬͯͨ
    

    View Slide

65. …
    

    View Slide

66. WIP
    

    View Slide

67. IUUQTHJUIVCDPNZVKJPTBLBEFFQMFBSOJOHKT
    ZVKJPTBLBEFFQMFBSOJOHKT
    %FFQ-FBSOJOHXSJUUFOJO&4
    

    View Slide

68. σϞ
    

    View Slide

69. ਖ਼ղ཰99.1%
    ※ ࣮ͨͩ͠ߦ࣌ؒ24࣌ؒҎ্
    

    View Slide

70. ຊ൪؀ڥͰ͸

    PythonΛ࢖͍·͠ΐ͏
    

    View Slide

71. ͻͨ͢Βָͯ͠

    σΟʔϓϥʔχϯά
    ָ͠ΜͰ
    

    View Slide

72. $BSFFST
    ࠾༻৘ใ
    &NPUJPO*0ͷςΫϊϩδʔͰɺ
    ʮҰาઌͷະདྷΛΑΓ๛͔ʹ͢Δ࢓૊ΈΛ࡞ΔʯϝϯόʔΛืूதͰ͢ɻ
    ืूதͷ৬छΛݟΔ
    WE ARE HIRING!
    https://www.emin.co.jp/careers/
    

    View Slide