$30 off During Our Annual Pro Sale. View Details »

ひたすら楽してディープラーニング

 ひたすら楽してディープラーニング

yujiosaka

April 05, 2016
Tweet

More Decks by yujiosaka

Other Decks in Programming

Transcript

  1. Yuji Isobe
    ͻͨ͢Βָͯ͠

    σΟʔϓϥʔχϯά
    NodeֶԂ20࣌ݶ໨

    View Slide

  2. [
    “Node.js”,
    “MongoDB”,
    “AngularJS”,
    “socket.io”,
    “Emotion Intelligence “

    ]
    @yujiosaka
    +BWB4DSJQU

    View Slide

  3. emin = Emotion Intelligence
    ؾ࣋ͪΛղ͢ΔςΫϊϩδʔͷ୳ڀ
    Emotion Intelligence͸ɺʮແҙࣝͷߦಈ͔Βɺ
    ਓͷؾ࣋ͪͷػඍΛղ͢Δ஌ੑʯΛɺਓ޻஌ೳ͓Αͼػցֶ
    शͷԠ༻ٕज़Λ༻͍ͯ
    ։ൃ͠ɺϏδωεʹԠ༻͍ͯ͠·͢ɻ

    View Slide

  4. ZenClerk Series
    lϢʔβʔͷߪೖҙཉͷߴ·ΓzΛ࡯஌͢ΔγεςϜͰɺ

    ΋ͬͱαΠτͷച্Λ৳͹ͯ͠Έ·ͤΜ͔ʁ
    [FODMFSLMJUF
    ײ৘Λղੳ͢Δਓ޻஌ೳʮ&NPUJPO*0ʯ͕

    &$αΠτ্Ͱʮ࠷దͳλΠϛϯάͰͷൢଅʯΛ࣮ݱ͠ɺച্Λ޲্ͤ͞·͢ɻ
    ࠓ·͞ʹߪೖΛ໎͍ͬͯΔ͓٬༷ʹ

    ͏Ε͍͠ʮ࠷ޙͷͻͱԡ͠ʯΛఏڙ͠·͢ɻ
    ;FODMFSL

    View Slide

  5. ػցֶशͱݴ͑͹…

    View Slide

  6. Lee Sedol vs. AlphaGo

    View Slide

  7. Google Trends

    View Slide

  8. ࠔͬͨ(´ɾωɾʆ)

    View Slide

  9. ͜ͷձࣾʹೖΔ·ͰҰ౓΋

    ػցֶश΍ͬͯ͜ͳ͔ͬͨorz

    View Slide

  10. ͱΓ͋͑ͣೖ໳ॻΛಡΉ
    ✓ ϐϯΫͷബ͍ຊ
    ✓ ௨শʮ͸͡ύλຊʯ
    ✓ ݟͨ໨ΑΓࠎଠͳ಺༰
    ✓ ͪΌΜͱ਺͕ࣜࡌͬͯΔ

    ʢͦͯͭ͠·ͮ͘ʣ

    View Slide

  11. ୈ3ষɿϕΠζ
    ୈ5ষɿkNN๏
    ୈ6ষɿҰൠԽઢܗϞσϧ
    ୈ7ষɿχϡʔϥϧωοτϫʔΫ
    ୈ8ষɿαϙʔτϕΫλʔϚγϯ
    ӽ͑ΒΕͳ͍น
    ← ෼͔Δ
    ← ෼͔Δ
    ← ෼͔Δ
    ← ͓ɺ͓͏…
    ← (ɾ㱼ɾ) ŜŠŘŘ!!

    View Slide

  12. σʔλՄࢹԽ
    ػցֶश
    ਺ֶ ౷ܭֶ
    ܭࢉػՊֶ
    ίϛϡχέʔγϣϯ
    υϝΠϯ஌ࣝ
    ࣗ෼ͷεΩϧηοτ
    ͜͜Λ৳͹͍ͨ͠

    View Slide

  13. Kaggle͸اۀ΍ݚڀऀ͕σʔλΛ౤ߘ͠ɺੈքதͷ
    ౷ܭՈ΍σʔλ෼ੳՈ͕ͦͷ࠷దϞσϧΛڝ͍߹͏ɺ
    ༧ଌϞσϦϯάٴͼ෼ੳख๏ؔ࿈ϓϥοτϑΥʔϜٴ
    ͼͦͷӡӦձࣾͰ͋Δɻ ΢ΟΩϖσΟΞ
    ૑ཱऀɿ ΞϯιχʔɾΰʔϧυϒϧʔϜ
    ઃཱɿ 2010೥4݄

    View Slide

  14. View Slide

  15. ࠓճ͸͜Εʹ

    νϟϨϯδ

    View Slide

  16. ✓ MNISTʢ਺ࣈͷखॻ͖σʔλʣͷ෼ྨ
    ✓ 28 x 28ϐΫηϧ
    ✓ 6ສ݅ͷֶशσʔλ
    ✓ 1ສ݅ͷςετσʔλ
    νϡʔτϦΞϧ՝୊

    View Slide

  17. ਖ਼ղ཰99%Λ໨ࢦ͢
    http://yann.lecun.com/exdb/mnist/

    View Slide

  18. Ͱ΋ͳΜ͔೉ͦ͠͏…

    View Slide

  19. Ͱ͖Ε͹ͻͨ͢Βָ͍ͨ͠

    View Slide

  20. View Slide

  21. ͻͨ͢Βָͯ͠FF6

    View Slide

  22. 1.࠷খউརճ਺ͰΫϦΞ
    2.ඞਢઓಆҎ֎Ͱ౪·ͳ͍
    3.ಓதͷΞΠςϜ͸ճऩ͠ͳ͍
    4.ฤू͸ؤுΒͳ͍
    ͻͨ͢Βָͯ͠FF6

    View Slide

  23. 1.࠷ۙͷϒʔϜʹ৐͔ͬΔ
    2.ۤखͳݴޠ͸࢖Θͳ͍
    3.ߴ౓ͳϥΠϒϥϦ͸࢖Θͳ͍
    4.ϓϨθϯࢿྉ͸ؤுΒͳ͍
    ͻͨ͢Βָͯ͠Kaggle
    ͱΓ͋͑ͣσΟʔϓϥʔχϯά

    ࢖ͬͯΈΔ
    +BWB4DSJQU͔͠࢖Θͳ͍
    MPEBTIͱઢܗ୅਺ϥΠϒϥϦ

    ͑͋͞Ε͹͍͍
    ΤϯδχΞͩ͠ίʔυͰউෛ

    View Slide

  24. ·ͣ͸৘ใऩू͔Β

    View Slide

  25. IUUQOFVSBMOFUXPSLTBOEEFFQMFBSOJOHDPNJOEFYIUNM

    View Slide

  26. ✓ ӳޠͷΦϯϥΠϯϒοΫʢ೔ຊޠԽ΋ਐߦதʣ
    ✓ χϡʔϥϧωοτϫʔΫ͔Β

    σΟʔϓϥʔχϯά·ͰͷྲྀΕΛৄࡉʹղઆ
    ✓ Pythonͷαϯϓϧ࣮૷͕ಡΈ΍͍͢
    Neural Networks and Deep Learning
    Φεεϝʂ

    View Slide

  27. ઓུΛཱͯΔ

    View Slide

  28. Python→CoffeeScript→ES2015
    sed + ؾ߹ Decaf JS
    ੈքॳʁES2015ͰσΟʔϓϥʔχϯά
    npmௐ΂
    JavaScript
    Babel

    View Slide

  29. PythonͱCoffeeScriptͬͯࣅͯͶʁ

    View Slide

  30. Python
    def update_mini_batch(self, mini_batch, eta):
    nabla_b = [np.zeros(b.shape) for b in self.biases]
    nabla_w = [np.zeros(w.shape) for w in self.weights]
    for x, y in mini_batch:
    delta_nabla_b, delta_nabla_w = self.backprop(x, y)
    nabla_b = [nb+dnb for nb, dnb in zip(nabla_b, delta_nabla_b)]
    nabla_w = [nw+dnw for nw, dnw in zip(nabla_w, delta_nabla_w)]
    self.weights = [w-(eta/len(mini_batch))*nw
    for w, nw in zip(self.weights, nabla_w)]
    self.biases = [b-(eta/len(mini_batch))*nb
    for b, nb in zip(self.biases, nabla_b)]

    View Slide

  31. CoffeeScript
    updateMiniBatch: (miniBatch, eta) ->
    nablaB = (Matrix.zeros(b.rows, b.cols) for b in @biases)
    nablaW = (Matrix.zeros(w.rows, w.cols) for w in @weights)
    for [x, y] in miniBatch
    [deltaNablaB, deltaNablaW] = @backprop(x, y)
    nablaB = (nb.plus(dnb) for [nb, dnb] in _.zip(nablaB, deltaNablaB))
    nablaW = (nw.plus(dnw) for [nw, dnw] in _.zip(nablaW, deltaNablaW))
    @weights = (w.minus(nw.mulEach(eta / miniBatch.length))

    for [w, nw] in _.zip(@weights, nablaW))
    @biases = (b.minus(nb.mulEach(eta / miniBatch.length))

    for [b, nb] in _.zip(@biases, nablaB))

    View Slide

  32. PythonϥΠϒϥϦͷAPIΛ࣮૷͢Δ

    View Slide

  33. numpy.nan_to_num
    nanToNum() {
    let thisData = this.data,
    rows = this.rows,
    cols = this.cols;
    let row, col, result = new Array(rows);
    for (row=0; rowresult[row] = new Array(cols);
    for (col=0; colresult[row][col] = n2n(thisData[row][col]);
    }
    }
    return new Matrix(result);
    };

    View Slide

  34. numpy.ravel
    ravel() {
    let thisData = this.data,
    rows = this.rows,
    cols = this.cols;
    let a = new Array(rows * cols);
    for (let i = 0, jBase = 0; ifor (let j = 0; ja[jBase + j] = thisData[i][j];
    }
    }
    return a;
    };

    View Slide

  35. CoffeeScript͕ڐ͞ΕΔͷ͸2015೥·ͰͩΑͶʔ
    https://github.com/juliankrispel/decaf

    View Slide


  36. View Slide

  37. ಈ͍ͨস

    View Slide

  38. ͦΖͦΖͪΌΜͱษڧ͢Δ

    View Slide

  39. χϡʔϥϧωοτϫʔΫ ਆܦճ࿏໢ɺӳOFVSBMOFUXPSL //
    ͸ɺ೴ػೳʹݟΒΕΔ͍͔ͭ͘ͷಛੑΛܭࢉػ
    ্ͷγϛϡϨʔγϣϯʹΑͬͯදݱ͢Δ͜ͱΛ໨ࢦͨ͠਺ֶϞσϧͰ͋Δɻ
    χϡʔϥϧωοτϫʔΫ8JLJQFEJB
    IUUQTKBXJLJQFEJBPSHXJLJχϡʔϥϧωοτϫʔΫ
    χϡʔϥϧωοτϫʔΫͱ͸ʁ

    View Slide

  40. b
    ύʔηϓτϩϯϞσϧ
    x1
    x2
    x3
    output
    w1
    w2
    w3
    PVUQVU
    JGЄKXKYKC≤

    JGЄKXKYKC

    View Slide

  41. 5
    ύʔηϓτϩϯϞσϧ
    ఱؾ͸ྑ͍͔ʁ
    ൴ঁ͸ߦ͖͍͔ͨʁ
    ձ৔͸Ӻͷ͔ۙ͘ʁ
    ͓ࡇΓʹߦ͘ʁ
    6
    2
    2
    No
    Yes
    Yes
    No

    View Slide

  42. b
    γάϞΠυχϡʔϩϯϞσϧ
    x1
    x2
    x3
    w1
    w2
    w3
    PVUQVU

    FYQ ЄKXKYKC

    output

    View Slide

  43. εςοϓؔ਺ʢύʔηϓτϩϯʣ

    View Slide

  44. γάϞΠυؔ਺

    View Slide

  45. ✓ 0ͱ1ͷؒͷ஋ΛදݱͰ͖ΔΑ͏ʹͳͬͨ
    ✓ খ͞ͳೖྗͷมԽ͕খ͞ͳग़ྗͷมԽʹରԠ͢Δ
    ✓ ཁ͢Δʹඍ෼Ͱ͖Δ͜ͱʹͳͬͨ
    Կ͕خ͍͠ͷ͔ʁ

    View Slide

  46. χϡʔϥϧωοτϫʔΫͷߏ଄
    w + Δw

    b + Δb
    output + Δoutput

    View Slide

  47. ✓ ֤χϡʔϩϯʢɹʣͷॏΈʢwʣͱόΠΞεʢbʣ
    Λௐ੔͢Δ͜ͱͰ༧ଌਫ਼౓Λ޲্͢Δ
    ✓ ͦͷͨΊͷςΫχοΫͱͯ͠ɺ

    όοΫϓϩύήʔγϣϯͳͲ͕ੜΈग़͞Εͨ
    χϡʔϥϧωοτϫʔΫͷֶश

    View Slide

  48. σΟʔϓϥʔχϯάͱ͸ʁ

    View Slide

  49. χϡʔϥϧωοτϫʔΫͷߏ଄

    View Slide

  50. σΟʔϓϥʔχϯάͷߏ଄

    View Slide

  51. ࠷ۙͷϒʔϜͷഎܠ
    ✓ ࠷ۙςΫχοΫ͕͍ͭ͘΋ݟ͔ͭͬͨ
    ✓ ૚ΛॏͶͯ΋͏·ֶ͘शͰ͖ΔΑ͏ʹͳͬͨ
    ✓ ૚ΛॏͶΔ͜ͱͰදݱೳྗͱਫ਼౓͕޲্ͨ͠

    View Slide

  52. ͍࣮͟૷΁

    View Slide

  53. ৞ΈࠐΈχϡʔϥϧωοτϫʔΫ

    View Slide

  54. ໰୊
    ͨͬͨ1pxͣΕΔ͚ͩͰผͷը૾ͱͯ͠ೝࣝ͞Εͯ͠·͏
    1px

    View Slide

  55. ղܾΠϝʔδ

    View Slide

  56. ߏ଄
    convolutional layer
    pooling layer

    View Slide

  57. ✓ γάϞΠυؔ਺Ҏ֎ͷ׆ੑԽؔ਺ʢReLUʣ
    ✓ ਖ਼نԽʢL1ਖ਼نԽ/ਖ਼نԽ/υϩοϓΞ΢τʣ
    ✓ ΫϩεΤϯτϩϐʔίετؔ਺
    ✓ ιϑτϚοΫεؔ਺
    ✓ ॏΈॳظԽͷվળ
    ͦͷଞͷٕज़

    View Slide

  58. σΟʔϓϥʔχϯά͸ٕज़ͷू߹ମ
    ʮσΟʔϓϥʔχϯάʯͱ͍͏ΞϧΰϦζϜ͸ଘࡏ͠ͳ͍
    ໨తʹԠͯ͡Ϟσϧ΍ٕज़ΛύζϧͷΑ͏ʹ૊Έཱͯͯɺ

    ΑΓߴ͍ਫ਼౓ΛੜΈग़͢͜ͱ͕Ͱ͖Δ

    View Slide

  59. ϋϚͬͨ͜ͱ

    View Slide

  60. ϋϚͬͨᶃ

    ਺͕ࣜ಄ͷதʹೖͬͯ͜ͳ͍໰୊
    ීஈ࢖͍׳ΕͨϓϩάϥϜʹͯ͠ΈΕ͹

    ҙ֎ͱʮͳΜͩͦΜͳ͜ͱ͔ʯͰࡁΉ͜ͱ΋͋Δ
    function sigmoid(z) {
    return 1 / (1 + Math.exp(-z));
    }
    let output = sigmoid(w.dot(a).plus(b));

    View Slide

  61. ܭࢉํ๏͸෼͔Βͳ͍ͷͰɺภඍ෼ͨ݁͠Ռͷ਺ࣜΛ

    stackoverflow͔ΒҾͬு͖ͬͯͨΒಈ͍ͨ
    costDelta(y) {
    this.outputDropout.minus(y);
    }
    ϋϚͬͨᶄ

    ภඍ෼ͷܭࢉํ๏͕෼͔Βͳ͍໰୊

    View Slide

  62. ڭՊॻ௨Γʹ࣮૷͢ΔͱιϑτϚοΫεؔ਺͕ܻ͋;Ε͢Δ
    ·ͨ΋΍stackoverflow͔ΒҾͬு͖ͬͯͨΒಈ͍ͨ
    ϋϚͬͨᶅ

    ڭՊॻʹ͸ॻ͍͍ͯͳ͍໰୊
    let max = _.max(vector),
    tmp = _.map(vector, (v) => { return Math.exp(v - max); }),
    sum = _.sum(tmp);
    return _.map(tmp, (v) => { return v / sum; });

    View Slide

  63. PythonͰ1࣌ؒͷͱ͜Ζ͕24͔͔࣌ؒΔ
    ཪͰNumpy͕ੌ͍͜ͱ΍ͬͯ͘ΕͯΔΒ͍͠
    ·ͩΫϥελϦϯά΋Ͱ͖ͯͳ͍͠ɺϝϞϦޮ཰΋ѱ͍
    ϋϚͬͨᶆ

    Զͷઢܗ୅਺ͷܭࢉͦ͘஗͍໰୊
    ։ൃ࣌͸খ͞ͳσʔληοτͰରԠ

    View Slide

  64. ࢀߟ࣮૷͕Theano΍TensorFlowͳͲͷߴػೳͳ

    ϥΠϒϥϦΛ࢖༻͍ͯ͠ΔͱԿ΍ͬͯΔͷ͔෼͔Βͳ͍
    ࣗಈඍ෼ͬͯͳΜ΍ͶΜ
    ϋϚͬͨᶇ

    ϥΠϒϥϦߴػೳա͗໰୊
    ؾ͍ͮͨΒPythonʹৄ͘͠ͳͬͯͨ

    View Slide


  65. View Slide

  66. WIP

    View Slide

  67. IUUQTHJUIVCDPNZVKJPTBLBEFFQMFBSOJOHKT
    ZVKJPTBLBEFFQMFBSOJOHKT
    %FFQ-FBSOJOHXSJUUFOJO&4

    View Slide

  68. σϞ

    View Slide

  69. ਖ਼ղ཰99.1%
    ※ ࣮ͨͩ͠ߦ࣌ؒ24࣌ؒҎ্

    View Slide

  70. ຊ൪؀ڥͰ͸

    PythonΛ࢖͍·͠ΐ͏

    View Slide

  71. ͻͨ͢Βָͯ͠

    σΟʔϓϥʔχϯά
    ָ͠ΜͰ

    View Slide

  72. $BSFFST
    ࠾༻৘ใ
    &NPUJPO*0ͷςΫϊϩδʔͰɺ
    ʮҰาઌͷະདྷΛΑΓ๛͔ʹ͢Δ࢓૊ΈΛ࡞ΔʯϝϯόʔΛืूதͰ͢ɻ
    ืूதͷ৬छΛݟΔ
    WE ARE HIRING!
    https://www.emin.co.jp/careers/

    View Slide