ねこでもわかる深層学習入門

オタク機械学習勉強会 #1 午前セッション資料

Transcript

1. Ͷ͜Ͱ΋Θ͔Δ
   ਂ૚ֶशೖ໳
   ܚጯٛक़େֶ੓ࡦɾϝσΟΞݚڀՊM2 ൘୩ ۏԻ
   @kotone_nyt
   

   View Slide

2. ೋ࣌ؒ͸௕͐Αʂʂʂ
   

   View Slide

3. ͜ͷ࿩ແବ͡Όͳ͍ʁ
   

   View Slide

4. ͸͍ɺແବͰ͢ʂʂʂ
   

   View Slide

5. ChainerΛ࢖͍·͢
   $ pip install chainer
   

   View Slide

6. ࣗݾ঺հ
   

   View Slide

7. DCGANͰσϨϚεͷإੜ੒
   ΦλΫػցֶशษڧձ
   @kotone_nyt
   

   View Slide

8. View Slide

9. ϓϩϑΟʔϧ
   • ൘୩ ۏԻ @kotone_nytɹ
   • Ϟσϧ෼ࢄֶशΞϧΰϦζϜͷ։ൃ
   • ܚጯٛक़େֶ੓ࡦɾϝσΟΞݚڀՊ (SFC)
   • શ೴ΞʔΩςΫνϟएखͷձ
   • શ೴ΞʔΩςΫνϟΠχγΞνϒ
   

   View Slide

10. ஫ҙࣄ߲
    • ਂ૚ֶशͷ਺ཧͳͲʹ͸৮Ε·ͤΜ
    • ֶशΞϧΰϦζϜͷ࿩͸͠·ͤΜ
    • ؆୯ͳͷͰ৸ͯͯͩͬͯ͘͞΋݁ߏͰ
    ͢
    

    View Slide

11. ࣋ͪؼͬͯཉ͍͜͠ͱ
    1. ਂ૚ֶशͷجຊͷʮΩʯ
    2. ࣮૷ͷ࠷ॳͷͱ͔͔ͬΓ
    3. εςοϓΞοϓͷώϯτ
    

    View Slide

12. ਂ૚ֶशͷͨΊͷઢܗ୅਺
    

    View Slide

13. εΧϥ
    0ͱ͔1ͱ͔3.1415923565...
    ͷΑ͏ͳ୯ମͷ਺ʢ֊਺0ͷςϯιϧͱ΋ʣ
    

    View Slide

14. ϕΫτϧ
    v = x
    1
    x
    2
    ! x
    N
    ( ) u =
    y
    1
    y
    2
    !
    y
    N
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    ཁૉ਺NͷߦϕΫτϧ ཁૉ਺NͷྻϕΫτϧ
    εΧϥΛฒ΂ͨ΋ͷʢ֊਺1ͷςϯιϧʣ
    ߦϕΫτϧͱྻϕΫτϧ͕͋Δ
    

    View Slide

15. ߦྻ
    MߦNྻ (MxN) ͷߦྻ
    ϕΫτϧͷ֓೦Λ֦ுͨ͠΋ͷʢ֊਺2ͷςϯιϧʣ
    ओʹҰ࣍ม׵ʢઢܗม׵ʣʹ࢖ΘΕΔ
    A =
    x
    11
    x
    12
    ! x
    1N
    x
    21
    x
    21
    ! x
    2N
    " " # "
    x
    M1
    x
    M2
    ! x
    MN
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    

    View Slide

16. సஔߦྻ
    MߦNྻ (MxN) ͷߦྻͷసஔߦྻ
    ߦྻͷߦͱྻΛೖΕସ͑ͨ΋ͷ
    MߦNྻͷߦྻ͸NߦMྻʹͳΔ
    AT =
    x
    11
    x
    12
    ! x
    1N
    x
    21
    x
    21
    ! x
    2N
    " " # "
    x
    M1
    x
    M2
    ! x
    MN
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    

    View Slide

17. ԋࢉͷϧʔϧ
    

    View Slide

18. Ճݮࢉ
    ಉ͡ܗͷςϯιϧಉ࢜͸ՃݮࢉͰ͖Δ
    

    View Slide

19. ϕΫτϧͷՃݮࢉ
    x
    1
    x
    2
    !
    x
    N
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    +
    y
    1
    y
    2
    !
    y
    N
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    =
    x
    1
    + y
    1
    x
    2
    + y
    2
    !
    x
    N
    + y
    N
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    ཁૉ਺NͷྻϕΫτϧͷՃࢉ
    x
    1
    x
    2
    ! x
    N
    ( )+ y
    1
    y
    2
    ! y
    N
    ( )
    = x
    1
    + y
    1
    x
    2
    + y
    2
    ! x
    N
    + y
    N
    ( )
    ཁૉ਺NͷߦϕΫτϧͷՃࢉ
    

    View Slide

20. ߦྻͷՃݮࢉ
    x
    11
    x
    12
    ! x
    1N
    x
    21
    x
    21
    ! x
    2N
    " " # "
    x
    M1
    x
    M2
    ! x
    MN
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    +
    y
    11
    y
    12
    ! y
    1N
    y
    21
    y
    21
    ! y
    2N
    " " # "
    y
    M1
    y
    M2
    ! y
    MN
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    MߦNྻ (MxN) ͷߦྻͷՃࢉ
    =
    x
    11
    + y
    11
    x
    12
    + y
    12
    ! x
    1N
    + y
    1N
    x
    21
    + y
    21
    x
    21
    + y
    21
    ! x
    2N
    + y
    2N
    " " # "
    x
    M1
    + y
    M1
    x
    M2
    + y
    M2
    ! x
    MN
    + y
    MN
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    

    View Slide

21. ৐ࢉ
    ಉ͡ܗͷςϯιϧ
    ·ͨ͸
    NྻͷςϯιϧͱNߦͷςϯιϧ
    ͸৐ࢉͰ͖Δ
    

    View Slide

22. ΞμϚʔϧੵ
    x
    11
    x
    12
    ! x
    1N
    x
    21
    x
    21
    ! x
    2N
    " " # "
    x
    M1
    x
    M2
    ! x
    MN
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    ⊙
    y
    11
    y
    12
    ! y
    1N
    y
    21
    y
    21
    ! y
    2N
    " " # "
    y
    M1
    y
    M2
    ! y
    MN
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    MߦNྻ (MxN) ͷߦྻͷՃࢉ
    =
    x
    11
    y
    11
    x
    12
    y
    12
    ! x
    1N
    y
    1N
    x
    21
    y
    21
    x
    21
    y
    21
    ! x
    2N
    y
    2N
    " " # "
    x
    M1
    y
    M1
    x
    M2
    y
    M2
    ! x
    MN
    y
    MN
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    

    View Slide

23. ϕΫτϧͷυοτੵ
    ཁૉ਺NͷߦϕΫτϧͱྻϕΫτϧͷ৐ࢉʢυοτੵʣ
    ߦϕΫτϧͷཁૉ਺͸ྻ਺
    ྻϕΫτϧͷཁૉ਺͸ߦ਺
    x
    1
    x
    2
    ! x
    N
    ( )
    y
    1
    y
    2
    "
    y
    N
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    = x
    k
    y
    k
    k
    ∑ = x
    1
    y
    1
    + x
    2
    y
    2
    +…+ x
    N
    y
    N
    

    View Slide

24. ߦྻੵ
    4ߦ3ྻͷߦྻͱ3ߦ2ྻͷߦྻͷߦྻੵ
    x
    11
    x
    21
    x
    31
    x
    41
    x
    12
    x
    22
    x
    32
    x
    42
    x
    13
    x
    23
    x
    33
    x
    43
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    ⎟
    y
    11
    y
    21
    y
    31
    y
    12
    y
    22
    y
    32
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    =
    x
    11
    y
    11
    + x
    12
    y
    21
    + x
    13
    y
    31
    x
    21
    y
    11
    + x
    22
    y
    21
    + x
    23
    y
    31
    x
    31
    y
    11
    + x
    32
    y
    21
    + x
    33
    y
    31
    x
    41
    y
    11
    + x
    42
    y
    21
    + x
    43
    y
    31
    x
    11
    y
    12
    + x
    12
    y
    22
    + x
    13
    y
    32
    x
    21
    y
    12
    + x
    22
    y
    22
    + x
    23
    y
    32
    x
    31
    y
    12
    + x
    32
    y
    22
    + x
    33
    y
    32
    x
    41
    y
    12
    + x
    42
    y
    22
    + x
    43
    y
    32
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    ⎟
    x
    11
    x
    21
    x
    31
    x
    41
    x
    12
    x
    22
    x
    32
    x
    42
    x
    13
    x
    23
    x
    33
    x
    43
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    ⎟
    y
    11
    y
    21
    y
    31
    y
    12
    y
    22
    y
    32
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    

    View Slide

25. ߦྻੵ
    4ߦ3ྻͷߦྻͱ3ߦ2ྻͷߦྻͷߦྻੵ
    =
    x
    11
    y
    11
    + x
    12
    y
    21
    + x
    13
    y
    31
    x
    21
    y
    11
    + x
    22
    y
    21
    + x
    23
    y
    31
    x
    31
    y
    11
    + x
    32
    y
    21
    + x
    33
    y
    31
    x
    41
    y
    11
    + x
    42
    y
    21
    + x
    43
    y
    31
    x
    11
    y
    12
    + x
    12
    y
    22
    + x
    13
    y
    32
    x
    21
    y
    12
    + x
    22
    y
    22
    + x
    23
    y
    32
    x
    31
    y
    12
    + x
    32
    y
    22
    + x
    33
    y
    32
    x
    41
    y
    12
    + x
    42
    y
    22
    + x
    43
    y
    32
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    ⎟
    x
    11
    x
    21
    x
    31
    x
    41
    x
    12
    x
    22
    x
    32
    x
    42
    x
    13
    x
    23
    x
    33
    x
    43
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    ⎟
    ⎟
    y
    11
    y
    21
    y
    31
    y
    12
    y
    22
    y
    32
    ⎛
    ⎝
    ⎜
    ⎜
    ⎜
    ⎞
    ⎠
    ⎟
    ⎟
    ⎟
    

    View Slide

26. ߦྻੵ
    MߦNྻͷߦྻͱNߦOྻͷߦྻͷߦྻੵ͸
    MߦOྻʹͳΔ
    

    View Slide

27. ͜Ε͚ͩΘ͔Ε͹
    ਂ૚ֶश͕Θ͔Δʂ
    

    View Slide

28. χϡʔϥϧωοτϫʔΫ #ͱ͸
    

    View Slide

29. f: x→y
    x y
    

    View Slide

30. f: x→y
    લ઒Έ͘
    ౡଜӎ݄
    

    View Slide

31. ୯७ύʔηϓτϩϯ
    w1
    w2
    w3
    w4
    x1
    x2
    x3
    x4
    a g(a) y
    

    View Slide

32. ୯७ύʔηϓτϩϯ
    w1
    w2
    w3
    w4
    x1
    x2
    x3
    x4
    a g(a) y
    ←ೖྗ
    ←݁߹ՙॏ
    ൃՐ཰ූ߸ˣ
    ↑׆ੑԽؔ਺
    

    View Slide

33. ̍χϡʔϩϯͷൃՐܭࢉ
    x → ೖྗϕΫτϧ
    w → ݁߹ՙॏϕΫτϧ
    g(...) → ׆ੑԽؔ਺
    a = wx = Σwkxkɹ
    y = g(a)ɹ
    

    View Slide

34. g(wx)
    x y
    

    View Slide

35. g(wx)
    લ઒Έ͘
    ͦΕҎ֎
    

    View Slide

36. ୯૚ύʔηϓτϩϯ
    x1
    x2
    x3
    x4
    a1 g(a1) y1
    a2 y2
    a3 g(a3) y3
    g(a2)
    

    View Slide

37. ୯૚ύʔηϓτϩϯ
    a2 y2
    a3 g(a3) y3
    g(a2)
    a1 g(a1) y1
    x1
    x2
    x3
    x4
    

    View Slide

38. ଟχϡʔϩϯͷൃՐܭࢉ
    x → ೖྗϕΫτϧ
    W → ݁߹ՙॏߦྻ
    g(...) → ׆ੑԽؔ਺
    a = Wxɹ
    y = g(a)ɹ
    

    View Slide

39. g(Wx)
    x y
    

    View Slide

40. g(Wx)
    લ઒Έ͘
    ౡଜӎ݄
    

    View Slide

41. ଟ૚΁ͷҰൠԽ
    W1 W2 W3
    x h1 h2 y x, y → ϕΫτϧ
    W, U → ߦྻ
    WT, xT → సஔ
    g(...) → ؔ਺
    Wx → ߦྻੵ
    x ⊙ y → ΞμϚʔϧੵ
    

    View Slide

42. ଟ૚΁ͷҰൠԽ
    W1 W2 W3
    x h1 h2 y
    h1 = g(W1 x)
    h2 = g(W2 h1)
    y = W3 h2
    

    View Slide

43. g(W2h1)
    h1 h2
    g(W1x)
    x h1
    W3h2
    h2 y
    

    View Slide

44. શ݁߹૚
    Linear/Fully Connected Layer
    

    View Slide

45. શ݁߹૚ͷಛ௃
    • ࠷΋୯७ͳ݁߹૚
    • ߦྻͷେ͖͕͞߹͍ͬͯΕ͹OKʂ
    • Ϟσϧ͕ਂ͗͢ΔͱֶशͰ͖ͳ͍
    • σʔλʹΑͬͯաֶशΛى͜͠΍͍͢
    

    View Slide

46. ࢖͏࣌ͷϙΠϯτ
    • MߦNྻͷߦྻΛ࢖ͬͨઢܗม׵
    • ೖྗˠཁૉ਺M
    • ग़ྗˠཁૉ਺N
    

    View Slide

47. ৞ΈࠐΈ૚
    Convolution Layer
    

    View Slide

48. View Slide

49. View Slide

50. http://colah.github.io/posts/2014-07-Understanding-Convolutions/
    

    View Slide

51. ৞ΈࠐΈ૚ͷಛ௃
    • ہॴ݁߹Λ࣋ͬͨ݁߹૚
    • ը૾౳ͰಛʹҖྗΛൃش͢Δ
    • ਂͯ͘͠΋ֶशͰ͖Δʢ100૚௒͑
    ΋ʂʣ
    • աֶशΛى͜͠ʹ͍͘
    

    View Slide

52. ৞ΈࠐΈ૚ͷಛ௃
    • ϋΠύʔύϥϝʔλ͕ଟ͍
    • ΧʔωϧαΠζ
    • ετϥΠυ
    • ύσΟϯά
    • ग़ྗνϟωϧ਺
    

    View Slide

53. ࢖͏࣌ͷϙΠϯτ
    • طଘͷ࣮૷ΛਅࣅΔͷ͕٢
    • ೖྗνϟωϧ਺ͱग़ྗνϟωϧ਺͕ॏ
    ཁ
    • ࠷ޙʹશ݁߹૚Λ͚ͬͭ͘Δ͜ͱ͕ଟ
    ͍
    • Deconvolution΋͋Δʢੜ੒Ͱ࢖͏ʣ
    

    View Slide

54. ChainerΛ࢖ͬͯΈΑ͏
    

    View Slide

55. $ pip install chainer --user
    

    View Slide

56. $ pip install chainer --upgrade --user
    

    View Slide

57. $ git clone https://github.com/ktnyt/cat_chainer.git
    

    View Slide

58. ୯૚ύʔηϓτϩϯ
    class SLP(chainer.Chain):
    def __init__(self):
    super(SLP, self).__init__(
    layer=L.Linear(None, 10),
    )
    def __call__(self, x):
    return self.layer(x)
    

    View Slide

59. $ python train.py --model SLP --out slp
    

    View Slide

60. Softmax Cross Entropy
    

    View Slide

61. Accuracy
    

    View Slide

62. ଟ૚ύʔηϓτϩϯ
    class MLP(chainer.Chain):
    def __init__(self):
    super(MLP, self).__init__(
    l1=L.Linear(None, 1000),
    l2=L.Linear(None, 1000),
    l3=L.Linear(None, 10),
    )
    def __call__(self, x):
    h1 = F.sigmoid(self.l1(x))
    h2 = F.sigmoid(self.l2(h1))
    return self.l3(h2)
    

    View Slide

63. $ python train.py --model MLP --out mlp
    

    View Slide

64. Softmax Cross Entropy
    

    View Slide

65. Accuracy
    

    View Slide

66. $ pip install matplotlib
    

    View Slide

67. ৞ΈࠐΈωοτϫʔΫ
    class CNN(chainer.Chain):
    def __init__(self):
    super(CNN, self).__init__(
    conv1=L.Convolution2D(1, 16, 3),
    conv2=L.Convolution2D(16, 32, 3),
    conv3=L.Convolution2D(32, 64, 3),
    l1=L.Linear(None, 512),
    l2=L.Linear(None, 10),
    )
    def __call__(self, x):
    x.data = x.data.reshape((len(x.data), 1, 28, 28))
    h = F.relu(self.conv1(x))
    h = F.max_pooling_2d(h, 2)
    h = F.relu(self.conv2(h))
    h = F.max_pooling_2d(h, 2)
    h = F.relu(self.conv3(h))
    h = F.max_pooling_2d(h, 2)
    h = F.dropout(F.relu(self.l1(h)))
    return self.l2(h)
    

    View Slide

68. $ python train.py --model CNN --out cnn
    

    View Slide

69. Softmax Cross Entropy
    

    View Slide

70. Accuracy
    

    View Slide

71. ࢲ͕௨ͬͨٙ໰ͨͪ
    

    View Slide

72. ಈ͍ͨʂ
    ͡Ό͋࣍͸Ͳ͏͢Ε͹ʁ
    

    View Slide

73. 1. ϞσϧΛ͍͡Ζ͏
    • slp.py/mlp.py/cnn.pyΛ͍ͬͯ͡ΈΑ͏
    • ӅΕૉࢠ਺
    • ӅΕ૚ͷ਺
    • ৞ΈࠐΈνϟωϧ਺
    • ׆ੑԽؔ਺ ͳͲͳͲ...
    

    View Slide

74. 2. ଞͷαϯϓϧΛࢼͦ͏
    • Chainerެࣜʹͨ͘͞Μྫ͕͋Δ
    • CIFAR10/CIFAR100
    • ImageNet
    • PTB
    • etc...
    

    View Slide

75. 3. ଞਓͷίʔυΛࢼͦ͏
    • DCGAN
    • https://github.com/mattya/chainer-DCGAN
    • PaintsChainer
    • https://github.com/pfnet/PaintsChainer
    • QRNN
    • http://bit.ly/jekbradbury-qrnn
    

    View Slide

76. ChainerΛ࢖͑͹
    Ϟσϧ͸؆୯ʹ૊ΊΔʂ
    

    View Slide

77. ৭ʑࢼͯ͠Έͯ
    ໘ന͍͜ͱΛ͠·͠ΐ͏
    

    View Slide

78. Thank you!
    

    View Slide