Python初心者がPyTorchをいじって機械学習の計算してみた PyCon mini Hiroshima 2018 / Python-newbies machine learning learning with PyTorch

PyCon mini Hiroshima 2018 (2018年10月6日(土)) での「Python初心者がPyTorchをいじって機械学習の計算してみた」という稚拙な発表のスライドです。

Transcript

1. Pythonॳ৺ऀ͕PyTorchΛ͍ͬͯ͡
   ػցֶशͷܭࢉΛͯ͠Έͨ
   Python-newbie’s machine learning learning with PyTorch
   ๺㝳೗๏
   PyCon mini Hiroshima 2018
   @޿ౡେֶ ະདྷ૑ੜηϯλʔ @2018೥10݄6೔(౔)
   

   View Slide

2. ࣗݾ঺հ
   

   View Slide

3. View Slide

4. ๺㝳೗๏
   ▸ ͖͍ͨͩΏ͖ͷΓ
   ▸ ʮ೗๏ʯΛԻಡΈ(༗৬ಡΈ)ͯ͠ňʹΐ΄͏ʼn
   ▸ ϋΠύɾϝσΟΞɾτϥϯεϨʔλ
   ▸ ೔༵ϓϩάϥϚ
   

   View Slide

5. iPad ޿ౡห ݕࡧ
   ৄ͘͠͸
   

   View Slide

6. ʮ͜Μͳײ͡ͷ͕ग़͖ͯ·͢ɻʯ
   

   View Slide

7. Pythonॳ৺ऀ͕PyTorchΛ
   ͍ͬͯ͡
   ػցֶशͷܭࢉΛͯ͠Έͨ
   ݁Ռwwwwwwwwww
   

   View Slide

8. Pythonॳ৺ऀ͕PyTorchΛ
   ͍ͬͯ͡
   ػցֶशͷܭࢉΛͯ͠Έͨ
   ݁ՌΛൃද͠·͢ɻ
   

   View Slide

9. ʢɾ㱼ɾʣŘŘ!!
   

   View Slide

10. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ɻ
    

    View Slide

11. ΋͏ͪΐͬͱ
    ৄ͘͠
    

    View Slide

12. ຊ೔ͷϝχϡʔ
    ػցֶश
    1Z5PSDI
    ΍ͬͯΈͨྫ
    

    View Slide

13. Pythonॳ৺ऀ?
    

    View Slide

14. Pythonॳ৺ऀ౓߹͍
    ▸ for ͱ͔ if ͱ͔ with ͱ͔ϨϕϧͰຊɾυΩϡϝϯτɾωοτΛݟͳ͕ΒͰ
    ͳ͍ͱॻ͚ͳ͍
    ▸ Ψνͷॳ৺ऀ
    

    View Slide

15. ػցֶश͸?
    

    View Slide

16. CourseraͷNgઌੜͷ
    ػցֶशͷίʔε
    

    View Slide

17. मྃ! (2016೥5݄͝Ζ)
    

    View Slide

18. ͔͠΋मྃ࣌
    ʹNgઌੜ͔Β
    

    View Slide

19. ͔͠΋मྃ࣌ʹ Andrew Ng ઌੜ͔Β
    ▸ Those were the topics of this class and if you worked all the way through this course
    you should now consider yourself an expert in machine learning.
    ▸ As you know, machine learning is a technology that's having huge impact on
    science, technology and industry.
    ▸ And you're now well qualified to use these tools of machine learning to great effect.
    ▸ I hope that many of you in this class will find ways to use machine learning to build
    cool systems and cool applications and cool products.
    ▸ And I hope that you find ways to use machine learning not only to make your life
    better but maybe someday to use it to make many other people's life better as well.
    

    View Slide

20. Ngઌੜ͔Βઐ໳Ո
    ೝఆ͞ΕͱΔ
    

    View Slide

21. ͜Γ΋͏ؒҧ͍ͳ͘Θ͠
    ػցֶशͷઐ໳Ո
    Ͱ͢(ތுදݱ)
    

    View Slide

22. 1. ػցֶश
    

    View Slide

23. ػցֶशͱ͸
    ▸ ͬ͘͟Γ
    ▸ ڭࢣ͋Γֶश (supervised learning)
    ▸ ڭࢣͳֶ͠श (unsupervised learning)
    

    View Slide

24. ػցֶशͱ͸
    ▸ ͍ΖΜͳσʔλͷഎޙʹજΉ๏ଇͱ͔ߏ଄ΛɺऔΓग़ͦ͏ͱ͢Δܭࢉ
    ▸ ͬ͘͟Γ
    ▸ ڭࢣ͋Γֶश (supervised learning)
    ▸ ڭࢣͳֶ͠श (unsupervised learning)
    ▸ ͷ2छྨ͕͋Δ
    

    View Slide

25. ڭࢣ͋Γֶश — ػցֶश
    ▸ σʔλʹରԠ͢Δσʔλ͕͋Δ
    ▸ ͱ͍͍ͩͨͳΓͦ͏ͳؔ਺ f Λ୳Γ౰͍ͯͨ
    ▸ ͔͠΋͜Ε͔ΒಘΔະ஌ͷσʔλʹ͍ͭͯ΋ͦ͏ͳͬͯཉ͍͠
    x1
    , x2
    , …, xN
    (N͸ී௨େ͖͍)
    y1
    , y2
    , …, yN xn
    ↦ yn
    yn
    = f(xn
    )
    

    View Slide

26. ྫ͑͹
    

    View Slide

27. ը૾Λೖྗͨ͠Βɺ
    ͍ࣸͬͯΔ΋ͷ(ೣɾ
    ص…)Λ౴͑Δؔ਺
    

    View Slide

28. ΍Γํ
    

    View Slide

29. શવҧ͏ؔ਺͔Β࢝Ί
    ͯɺ
    ͡Θ͡Θ͚͍ۙͮͯ͘
    

    View Slide

30. தֶ2೥਺ֶՊ
    

    View Slide

31. தֶߍ2೥ੜΛࢥ͍ग़͢
    σʔλʹҰ൪߹͏௚ઢ
    0.2 0.4 0.6 0.8 1.0
    1.1
    1.2
    1.3
    1.4
    1.5
    1.6
    1.7
    y = ax + b
    a, bΛܾΊΔ
    

    View Slide

32. ͜͏͍͏ͷ΋
    ཱ೿ͳػցֶश
    

    View Slide

33. ͞Βʹผͷ
    ؔ਺Λ߹੒
    

    View Slide

34. ▸ Կ͔ʹ൓Ԡ͢Δײ͡ͷؔ਺͕࡞ΕΔ
    ͞Βʹผͷؔ਺Λ߹੒
    -1.0 -0.5 0.5 1.0 1.5 2.0
    0.5
    1.0
    1.5
    2.0
    -10 -5 5 10
    0.2
    0.4
    0.6
    0.8
    1.0
    s(x) =
    1
    1 + e−x
    ,
    s(f(x))
    

    View Slide

35. ࡢࠓྲྀߦͬͯ
    ͍Δͷ͸
    

    View Slide

36. ਂ૚ֶश
    Deep learning
    

    View Slide

37. ͖ͬ͞ͷؔ਺Λ
    ͨ͘͞Μ༻ҙͯ͠
    ͭͳ͛·͘Δ
    

    View Slide

38. ͭͳ͛·ͬͨ͘ͷ = χϡʔϥϧɾωοτϫʔΫ
    ▸ ○1ͭ1͕͖ͭͬ͞ͷؔ਺
    ▸ ΍͡Δ͠ͷΑ͏ʹ
    ▸ ग़ྗΛ࣍ͷೖྗʹ
    

    View Slide

39. 2. PyTorch
    

    View Slide

40. View Slide

41. PyTorch
    ▸ ਂ૚ֶश༻ͷϑϨʔϜϫʔΫ
    ▸ theano, Caffe, TensorFlow, mxnet, Chainer
    ▸ ͍ͬͺ͍͋Δ͏ͪͷҰͭ
    ▸ Torch ΑΓ (Torch ͸ LuaJIT)
    ▸ ChainerΛେ͍ʹࢀߟʹ͍ͯ͠Δ
    ▸ (ʮforkͱݺͼ͚ͨΕ͹Ͳ͏ͧʯhttps://www.reddit.com/r/MachineLearning/comments/
    74md00/n_how_to_use_chainer_for_theano_users/dnzkba1/ )
    

    View Slide

42. PyTorch
    ▸ ಛ௃
    ▸ υΩϡϝϯτɾνϡʔτϦΞϧ͕݁ߏৄ͍͠
    ▸ ಈతʹܭࢉάϥϑ࡞੒
    ▸ ࣗಈඍ෼
    ▸ χϡʔϥϧɾωοτϫʔΫ͕ॻ͖΍͍͢ (Chainerͱࣅ͍ͯΔ)
    

    View Slide

43. ॻ͖΍ͦ͢͏ͳงғؾΛݟΔ
    class Net(nn.Module):
    def __init__(self):
    super(Net, self).__init__()
    self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
    self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
    self.conv2_drop = nn.Dropout2d()
    self.fc1 = nn.Linear(320, 50)
    self.fc2 = nn.Linear(50, 10)
    def forward(self, x):
    x = F.relu(F.max_pool2d(self.conv1(x), 2))
    x =
    F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
    x = x.view(-1, 320)
    x = F.relu(self.fc1(x))
    x = F.dropout(x, training=self.training)
    x = self.fc2(x)
    return F.log_softmax(x, dim=1)
    ‏͖ͬ͞ͷ˓Λ
    Կݸ༻ҙͯ͠
    ͭͳ͙͔
    ‏ͭͳ͛Δͱ͖ʹ߹੒͢Δؔ਺
    

    View Slide

44. ॻ͖΍ͦ͢͏ͳงғؾΛݟΔ
    ▸ random_split
    ▸ σʔλΛγϟοϑϧͭͭ͠෼ׂͯ͘͠ΕΔศརϝιου
    train_size = int(0.8 * len(dataset))
    test_size = len(dataset) - train_size
    trainset, testset = torch.utils.data.random_split(dataset,
    [train_size, test_size])
    NEW 0.4.1
    

    View Slide

45. PyTorchདྷͱΔ
    ▸ Frameworks mentioned @ICLR (International Conference on Learning Representations)
    2018➡2019
    ▸ TensorFlow 228 ➡ 266
    ▸ Keras: 42 ➡ 56
    ▸ PyTorch 87 ➡ 252
    ▸ https://www.reddit.com/r/MachineLearning/comments/9kys38/
    r_frameworks_mentioned_iclr_20182019_tensorflow/
    ▸ ݁ߏݚڀͷίʔυ͕PyTorchͰॻ͔Ε͍ͯΔ
    ▸ ΦϯϥΠϯߨٛ΋PyTorch͕૿͖͑ͯͨ
    

    View Slide

46. PyTorch 1.0.0.dev20181002
    (1.0 developer preview!)
    ΋͏ͪΐͬͱͰ ver.1.0
    https://code.fb.com/ai-research/facebook-accelerates-ai-development-with-new-partners-and-production-capabilities-for-pytorch-1-0/
    

    View Slide

47. 3. ΍ͬͯΈͨྫ
    

    View Slide

48. ᶃ΢Σϒϖʔδ෼ྨ
    

    View Slide

49. ΢Σϒϖʔδ෼ྨ?
    

    View Slide

50. ΢Σϒϖʔδ෼ྨ?
    ▸ ΢ΣϒϖʔδͷςΩετσʔλΛೖྗͱ͢Δ
    ▸ ͦͷϖʔδͷ಺༰ͷछྨΛਪఆ͢Δ
    ▸ ·ͣ͸࿅शͷͨΊʹʮ2஋෼ྨʯ
    

    View Slide

51. ΍Γํ
    ▸ ࣗ෼ͷʮ͸ͯͳϒοΫϚʔΫʯ4ສҎ্ϒοΫϚʔΫ͍ͯ͠Δ
    ▸ ϒοΫϚʔΫλάΛͤͬͤͱ෇͚͍ͯΔ
    ▸ ʮػցֶशʯλάΛ෇͚͍ͯΔ͔Ͳ͏͔Ͱ෼ྨ
    ▸ 1000Ҏ্͋Δ
    ▸ ࣗ෼ͷϒοΫϚʔΫ͔ΒσʔλΛूΊ͖ͯͯɺֶशɻ
    ▸ text ➡ MeCab(ܗଶૉղੳ) ➡ fasttext ➡ จॻϕΫτϧʹ͢Δ
    

    View Slide

52. ΍Γํ
    ࢈ۀٕज़૯߹ݚڀॴ͸
    2018೥5݄29೔ɺ౦ژେ
    ֶͱڞಉͰɺػցֶशͷֶ
    शॲཧʹඞཁͳԋࢉྔΛ࡟
    ݮ͢Δܭࢉํࣜͱԋࢉճ࿏
    ΛߟҊͨ͠ͱൃදͨ͠ɻ
    ࠓقͷΞχϝͷ
    ͋ͦͼ͋ͦ͹ͤͬͯ
    ໘ന͍Ͷɻ
    ػցֶशͷ͜ͱ͕ॻ͍ͯ͋Δϖʔδ͡Ό!
    ΢ΣϒϖʔδB
    ΢ΣϒϖʔδA
    ػցֶश͡Όͳ͍ϖʔδ͡Ό!
    

    View Slide

53. Demo
    

    View Slide

54. View Slide

55. View Slide

56. ͨͿΜ͏·͍ͬͨ͘!
    

    View Slide

57. ࣍
    

    View Slide

58. ᶄϒοΫϚʔΫ਺ਪఆ
    

    View Slide

59. ϒοΫϚʔΫ਺ਪఆ?
    

    View Slide

60. ϒοΫϚʔΫ਺ਪఆ?
    ▸ ࣗ෼ͷʮ͸ͯͳϒοΫϚʔΫʯ4ສҎ্ϒοΫϚʔΫ͍ͯ͠Δ
    ▸ ֤ϖʔδͷϒοΫϚʔΫ਺…ଞͷਓ΋߹ΘͤͯԿਓϒοΫϚʔΫ͔ͨ͠
    ▸ ·ͨࣗ෼ͷϒοΫϚʔΫ͔ΒσʔλΛूΊֶͯश
    ▸ text ➡ MeCab(ܗଶૉղੳ) ➡ fasttext ➡ จॻϕΫτϧʹ͢Δ
    ▸ ద౰ʹωοτϫʔΫΛֶͭͬͯ͘शͯ͠ΈΔ
    

    View Slide

61. Demo
    

    View Slide

62. ܇࿅σʔλଛࣦ⤵
    ࢼݧσʔλଛࣦ⤴
    ʮաֶश͠·͘ΔΈ͍ͨͰ͢ɻʯ
    

    View Slide

63. (´ɾωɾ`)
    શવ͏·ֶ͘श͠ͳ͍ (աֶश͹͔Γ͢Δ)
    

    View Slide

64. ϒοΫϚʔΫ਺ਪఆ͸จॻͰ͸ແཧͰ͸?
    ▸ ബʑؾ෇͍͍ͯͨ
    ▸ ͸ͯͳϒοΫϚʔΫͷ਺Λϖʔδͷ಺༰͚͔ͩΒਪଌ͢Δͷ͸ແཧͰ͸ɻ
    ▸ ༗໊ਓ͕֦ࢄͨ͠ͱ͔ɺଞͷӡͷཁૉ͕ڧ͍
    ▸ Ͱ΋७ਮʹจষ͚͔ͩΒજࡏతͳʮϒοΫϚʔΫ͞ΕྗʯΈ͍ͨͳͷ͕Ͱ
    ͳ͍͔ͳ͋ͱ୶͍ظ଴͸͋ͬͨ
    ▸ ݚڀଓߦ͢Δ΂͖! ͸ͯϒΛՔ͍͗ͨ!
    

    View Slide

65. ᶅGAN
    

    View Slide

66. GAN?
    

    View Slide

67. GAN?
    ▸ GAN = generative adversarial network (ఢରతੜ੒ωοτϫʔΫ)
    ▸ ੜ੒ऀ generator (G) v.s. ࣝผऀ discriminator (D) ͕ఢର͠ͳ͕Βֶशͯ͠
    ͍͘
    ▸ G ͱ D ͕ఢର͠ͳ͕Βֶश͍ͯ͘͠
    ▸ Կ͔Λʮੜ੒͢Δʯ
    

    View Slide

68. ը૾ੜ੒ͷ࡞ྫ
    

    View Slide

69. ͜͏͍͏ͷ͕
    ࡞Γ͍ͨ
    ▸ Ҵ৔Ѫ߳ͪΌΜͱΘ͠ͷը૾Λ
    ֶशͯ͠ɺը૾Λੜ੒
    

    View Slide

70. epoch 25
    ▸ ΄΅ϊΠζ
    

    View Slide

71. epoch 37
    ▸ ΍΍إΒ͖͠΋ͷ͕ݟ͖͑ͯͨ
    

    View Slide

72. epoch 149
    ▸ ੜ੒Ͱ͖͖ͯͨ!
    

    View Slide

73. epoch 192
    

    View Slide

74. ൺֱ
    ਖ਼ ϑΣΠΫ
    

    View Slide

75. ָ͍͠
    ✌('ω'✌ )ࡾ✌('ω')✌ࡾ( ✌'ω')✌
    

    View Slide

76. ·ͱΊ
    ▸ Pythonॳ৺ऀ͕PyTorchΛ࢖ͬͯػցֶशΛ΍ͬͯΈͨ
    ▸ ͔֬ʹॻ͖΍͍͢ؾ͕͢Δ
    ▸ ͲΜͲΜ࣮ݧͯ͠༡΂Δ
    ▸ ৽͍͠ઃܭ͕࣍ʑग़͍ͯΔͷͰ΍͍͖͍ͬͯͨ
    ͋Γ͕ͱ͏͍͟͝·ͨ͠ɻ
    

    View Slide