Slide 1
Slide 1 text
Ͷ͜ͰΘ͔Δ
ਂֶशೖ
ܚጯٛक़େֶࡦɾϝσΟΞݚڀՊM2 ൘୩ ۏԻ
@kotone_nyt
Slide 2
Slide 2 text
ೋ࣌ؒ͐Αʂʂʂ
Slide 3
Slide 3 text
͜ͷແବ͡Όͳ͍ʁ
Slide 4
Slide 4 text
͍ɺແବͰ͢ʂʂʂ
Slide 5
Slide 5 text
ChainerΛ͍·͢
$ pip install chainer
Slide 6
Slide 6 text
ࣗݾհ
Slide 7
Slide 7 text
DCGANͰσϨϚεͷإੜ
ΦλΫػցֶशษڧձ
@kotone_nyt
Slide 8
Slide 8 text
No content
Slide 9
Slide 9 text
ϓϩϑΟʔϧ
• ൘୩ ۏԻ @kotone_nytɹ
• ϞσϧࢄֶशΞϧΰϦζϜͷ։ൃ
• ܚጯٛक़େֶࡦɾϝσΟΞݚڀՊ (SFC)
• શΞʔΩςΫνϟएखͷձ
• શΞʔΩςΫνϟΠχγΞνϒ
Slide 10
Slide 10 text
ҙࣄ߲
• ਂֶशͷཧͳͲʹ৮Ε·ͤΜ
• ֶशΞϧΰϦζϜͷ͠·ͤΜ
• ؆୯ͳͷͰ৸ͯͯͩͬͯ݁͘͞ߏͰ
͢
Slide 11
Slide 11 text
࣋ͪؼͬͯཉ͍͜͠ͱ
1. ਂֶशͷجຊͷʮΩʯ
2. ࣮ͷ࠷ॳͷͱ͔͔ͬΓ
3. εςοϓΞοϓͷώϯτ
Slide 12
Slide 12 text
ਂֶशͷͨΊͷઢܗ
Slide 13
Slide 13 text
εΧϥ
0ͱ͔1ͱ͔3.1415923565...
ͷΑ͏ͳ୯ମͷʢ֊0ͷςϯιϧͱʣ
Slide 14
Slide 14 text
ϕΫτϧ
v = x
1
x
2
! x
N
( ) u =
y
1
y
2
!
y
N
⎛
⎝
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
ཁૉNͷߦϕΫτϧ ཁૉNͷྻϕΫτϧ
εΧϥΛฒͨͷʢ֊1ͷςϯιϧʣ
ߦϕΫτϧͱྻϕΫτϧ͕͋Δ
Slide 15
Slide 15 text
ߦྻ
MߦNྻ (MxN) ͷߦྻ
ϕΫτϧͷ֓೦Λ֦ுͨ͠ͷʢ֊2ͷςϯιϧʣ
ओʹҰ࣍มʢઢܗมʣʹΘΕΔ
A =
x
11
x
12
! x
1N
x
21
x
21
! x
2N
" " # "
x
M1
x
M2
! x
MN
⎛
⎝
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
Slide 16
Slide 16 text
సஔߦྻ
MߦNྻ (MxN) ͷߦྻͷసஔߦྻ
ߦྻͷߦͱྻΛೖΕସ͑ͨͷ
MߦNྻͷߦྻNߦMྻʹͳΔ
AT =
x
11
x
12
! x
1N
x
21
x
21
! x
2N
" " # "
x
M1
x
M2
! x
MN
⎛
⎝
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
Slide 17
Slide 17 text
ԋࢉͷϧʔϧ
Slide 18
Slide 18 text
Ճݮࢉ
ಉ͡ܗͷςϯιϧಉ࢜ՃݮࢉͰ͖Δ
Slide 19
Slide 19 text
ϕΫτϧͷՃݮࢉ
x
1
x
2
!
x
N
⎛
⎝
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
+
y
1
y
2
!
y
N
⎛
⎝
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
=
x
1
+ y
1
x
2
+ y
2
!
x
N
+ y
N
⎛
⎝
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
ཁૉNͷྻϕΫτϧͷՃࢉ
x
1
x
2
! x
N
( )+ y
1
y
2
! y
N
( )
= x
1
+ y
1
x
2
+ y
2
! x
N
+ y
N
( )
ཁૉNͷߦϕΫτϧͷՃࢉ
Slide 20
Slide 20 text
ߦྻͷՃݮࢉ
x
11
x
12
! x
1N
x
21
x
21
! x
2N
" " # "
x
M1
x
M2
! x
MN
⎛
⎝
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
+
y
11
y
12
! y
1N
y
21
y
21
! y
2N
" " # "
y
M1
y
M2
! y
MN
⎛
⎝
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
MߦNྻ (MxN) ͷߦྻͷՃࢉ
=
x
11
+ y
11
x
12
+ y
12
! x
1N
+ y
1N
x
21
+ y
21
x
21
+ y
21
! x
2N
+ y
2N
" " # "
x
M1
+ y
M1
x
M2
+ y
M2
! x
MN
+ y
MN
⎛
⎝
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
Slide 21
Slide 21 text
ࢉ
ಉ͡ܗͷςϯιϧ
·ͨ
NྻͷςϯιϧͱNߦͷςϯιϧ
ࢉͰ͖Δ
Slide 22
Slide 22 text
ΞμϚʔϧੵ
x
11
x
12
! x
1N
x
21
x
21
! x
2N
" " # "
x
M1
x
M2
! x
MN
⎛
⎝
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
⊙
y
11
y
12
! y
1N
y
21
y
21
! y
2N
" " # "
y
M1
y
M2
! y
MN
⎛
⎝
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
MߦNྻ (MxN) ͷߦྻͷՃࢉ
=
x
11
y
11
x
12
y
12
! x
1N
y
1N
x
21
y
21
x
21
y
21
! x
2N
y
2N
" " # "
x
M1
y
M1
x
M2
y
M2
! x
MN
y
MN
⎛
⎝
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
Slide 23
Slide 23 text
ϕΫτϧͷυοτੵ
ཁૉNͷߦϕΫτϧͱྻϕΫτϧͷࢉʢυοτੵʣ
ߦϕΫτϧͷཁૉྻ
ྻϕΫτϧͷཁૉߦ
x
1
x
2
! x
N
( )
y
1
y
2
"
y
N
⎛
⎝
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
= x
k
y
k
k
∑ = x
1
y
1
+ x
2
y
2
+…+ x
N
y
N
Slide 24
Slide 24 text
ߦྻੵ
4ߦ3ྻͷߦྻͱ3ߦ2ྻͷߦྻͷߦྻੵ
x
11
x
21
x
31
x
41
x
12
x
22
x
32
x
42
x
13
x
23
x
33
x
43
⎛
⎝
⎜
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
⎟
y
11
y
21
y
31
y
12
y
22
y
32
⎛
⎝
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
=
x
11
y
11
+ x
12
y
21
+ x
13
y
31
x
21
y
11
+ x
22
y
21
+ x
23
y
31
x
31
y
11
+ x
32
y
21
+ x
33
y
31
x
41
y
11
+ x
42
y
21
+ x
43
y
31
x
11
y
12
+ x
12
y
22
+ x
13
y
32
x
21
y
12
+ x
22
y
22
+ x
23
y
32
x
31
y
12
+ x
32
y
22
+ x
33
y
32
x
41
y
12
+ x
42
y
22
+ x
43
y
32
⎛
⎝
⎜
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
⎟
x
11
x
21
x
31
x
41
x
12
x
22
x
32
x
42
x
13
x
23
x
33
x
43
⎛
⎝
⎜
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
⎟
y
11
y
21
y
31
y
12
y
22
y
32
⎛
⎝
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
Slide 25
Slide 25 text
ߦྻੵ
4ߦ3ྻͷߦྻͱ3ߦ2ྻͷߦྻͷߦྻੵ
=
x
11
y
11
+ x
12
y
21
+ x
13
y
31
x
21
y
11
+ x
22
y
21
+ x
23
y
31
x
31
y
11
+ x
32
y
21
+ x
33
y
31
x
41
y
11
+ x
42
y
21
+ x
43
y
31
x
11
y
12
+ x
12
y
22
+ x
13
y
32
x
21
y
12
+ x
22
y
22
+ x
23
y
32
x
31
y
12
+ x
32
y
22
+ x
33
y
32
x
41
y
12
+ x
42
y
22
+ x
43
y
32
⎛
⎝
⎜
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
⎟
x
11
x
21
x
31
x
41
x
12
x
22
x
32
x
42
x
13
x
23
x
33
x
43
⎛
⎝
⎜
⎜
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
⎟
⎟
y
11
y
21
y
31
y
12
y
22
y
32
⎛
⎝
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
Slide 26
Slide 26 text
ߦྻੵ
MߦNྻͷߦྻͱNߦOྻͷߦྻͷߦྻੵ
MߦOྻʹͳΔ
Slide 27
Slide 27 text
͜Ε͚ͩΘ͔Ε
ਂֶश͕Θ͔Δʂ
Slide 28
Slide 28 text
χϡʔϥϧωοτϫʔΫ #ͱ
Slide 29
Slide 29 text
f: x→y
x y
Slide 30
Slide 30 text
f: x→y
લΈ͘
ౡଜӎ݄
Slide 31
Slide 31 text
୯७ύʔηϓτϩϯ
w1
w2
w3
w4
x1
x2
x3
x4
a g(a) y
Slide 32
Slide 32 text
୯७ύʔηϓτϩϯ
w1
w2
w3
w4
x1
x2
x3
x4
a g(a) y
←ೖྗ
←݁߹ՙॏ
ൃՐූ߸ˣ
↑׆ੑԽؔ
Slide 33
Slide 33 text
̍χϡʔϩϯͷൃՐܭࢉ
x → ೖྗϕΫτϧ
w → ݁߹ՙॏϕΫτϧ
g(...) → ׆ੑԽؔ
a = wx = Σwkxkɹ
y = g(a)ɹ
Slide 34
Slide 34 text
g(wx)
x y
Slide 35
Slide 35 text
g(wx)
લΈ͘
ͦΕҎ֎
Slide 36
Slide 36 text
୯ύʔηϓτϩϯ
x1
x2
x3
x4
a1 g(a1) y1
a2 y2
a3 g(a3) y3
g(a2)
Slide 37
Slide 37 text
୯ύʔηϓτϩϯ
a2 y2
a3 g(a3) y3
g(a2)
a1 g(a1) y1
x1
x2
x3
x4
Slide 38
Slide 38 text
ଟχϡʔϩϯͷൃՐܭࢉ
x → ೖྗϕΫτϧ
W → ݁߹ՙॏߦྻ
g(...) → ׆ੑԽؔ
a = Wxɹ
y = g(a)ɹ
Slide 39
Slide 39 text
g(Wx)
x y
Slide 40
Slide 40 text
g(Wx)
લΈ͘
ౡଜӎ݄
Slide 41
Slide 41 text
ଟͷҰൠԽ
W1 W2 W3
x h1 h2 y x, y → ϕΫτϧ
W, U → ߦྻ
WT, xT → సஔ
g(...) → ؔ
Wx → ߦྻੵ
x ⊙ y → ΞμϚʔϧੵ
Slide 42
Slide 42 text
ଟͷҰൠԽ
W1 W2 W3
x h1 h2 y
h1 = g(W1 x)
h2 = g(W2 h1)
y = W3 h2
Slide 43
Slide 43 text
g(W2h1)
h1 h2
g(W1x)
x h1
W3h2
h2 y
Slide 44
Slide 44 text
શ݁߹
Linear/Fully Connected Layer
Slide 45
Slide 45 text
શ݁߹ͷಛ
• ࠷୯७ͳ݁߹
• ߦྻͷେ͖͕͞߹͍ͬͯΕOKʂ
• Ϟσϧ͕ਂ͗͢ΔͱֶशͰ͖ͳ͍
• σʔλʹΑͬͯաֶशΛى͍͜͢͠
Slide 46
Slide 46 text
͏࣌ͷϙΠϯτ
• MߦNྻͷߦྻΛͬͨઢܗม
• ೖྗˠཁૉM
• ग़ྗˠཁૉN
Slide 47
Slide 47 text
ΈࠐΈ
Convolution Layer
Slide 48
Slide 48 text
No content
Slide 49
Slide 49 text
No content
Slide 50
Slide 50 text
http://colah.github.io/posts/2014-07-Understanding-Convolutions/
Slide 51
Slide 51 text
ΈࠐΈͷಛ
• ہॴ݁߹Λ࣋ͬͨ݁߹
• ը૾ͰಛʹҖྗΛൃش͢Δ
• ਂֶͯ͘͠शͰ͖Δʢ100͑
ʂʣ
• աֶशΛى͜͠ʹ͍͘
Slide 52
Slide 52 text
ΈࠐΈͷಛ
• ϋΠύʔύϥϝʔλ͕ଟ͍
• ΧʔωϧαΠζ
• ετϥΠυ
• ύσΟϯά
• ग़ྗνϟωϧ
Slide 53
Slide 53 text
͏࣌ͷϙΠϯτ
• طଘͷ࣮ΛਅࣅΔͷ͕٢
• ೖྗνϟωϧͱग़ྗνϟωϧ͕ॏ
ཁ
• ࠷ޙʹશ݁߹Λ͚ͬͭ͘Δ͜ͱ͕ଟ
͍
• Deconvolution͋ΔʢੜͰ͏ʣ
Slide 54
Slide 54 text
ChainerΛͬͯΈΑ͏
Slide 55
Slide 55 text
$ pip install chainer --user
Slide 56
Slide 56 text
$ pip install chainer --upgrade --user
Slide 57
Slide 57 text
$ git clone https://github.com/ktnyt/cat_chainer.git
Slide 58
Slide 58 text
୯ύʔηϓτϩϯ
class SLP(chainer.Chain):
def __init__(self):
super(SLP, self).__init__(
layer=L.Linear(None, 10),
)
def __call__(self, x):
return self.layer(x)
Slide 59
Slide 59 text
$ python train.py --model SLP --out slp
Slide 60
Slide 60 text
Softmax Cross Entropy
Slide 61
Slide 61 text
Accuracy
Slide 62
Slide 62 text
ଟύʔηϓτϩϯ
class MLP(chainer.Chain):
def __init__(self):
super(MLP, self).__init__(
l1=L.Linear(None, 1000),
l2=L.Linear(None, 1000),
l3=L.Linear(None, 10),
)
def __call__(self, x):
h1 = F.sigmoid(self.l1(x))
h2 = F.sigmoid(self.l2(h1))
return self.l3(h2)
Slide 63
Slide 63 text
$ python train.py --model MLP --out mlp
Slide 64
Slide 64 text
Softmax Cross Entropy
Slide 65
Slide 65 text
Accuracy
Slide 66
Slide 66 text
$ pip install matplotlib
Slide 67
Slide 67 text
ΈࠐΈωοτϫʔΫ
class CNN(chainer.Chain):
def __init__(self):
super(CNN, self).__init__(
conv1=L.Convolution2D(1, 16, 3),
conv2=L.Convolution2D(16, 32, 3),
conv3=L.Convolution2D(32, 64, 3),
l1=L.Linear(None, 512),
l2=L.Linear(None, 10),
)
def __call__(self, x):
x.data = x.data.reshape((len(x.data), 1, 28, 28))
h = F.relu(self.conv1(x))
h = F.max_pooling_2d(h, 2)
h = F.relu(self.conv2(h))
h = F.max_pooling_2d(h, 2)
h = F.relu(self.conv3(h))
h = F.max_pooling_2d(h, 2)
h = F.dropout(F.relu(self.l1(h)))
return self.l2(h)
Slide 68
Slide 68 text
$ python train.py --model CNN --out cnn
Slide 69
Slide 69 text
Softmax Cross Entropy
Slide 70
Slide 70 text
Accuracy
Slide 71
Slide 71 text
ࢲ͕௨ͬͨٙͨͪ
Slide 72
Slide 72 text
ಈ͍ͨʂ
͡Ό͋࣍Ͳ͏͢Εʁ
Slide 73
Slide 73 text
1. ϞσϧΛ͍͡Ζ͏
• slp.py/mlp.py/cnn.pyΛ͍ͬͯ͡ΈΑ͏
• ӅΕૉࢠ
• ӅΕͷ
• ΈࠐΈνϟωϧ
• ׆ੑԽؔ ͳͲͳͲ...
Slide 74
Slide 74 text
2. ଞͷαϯϓϧΛࢼͦ͏
• Chainerެࣜʹͨ͘͞Μྫ͕͋Δ
• CIFAR10/CIFAR100
• ImageNet
• PTB
• etc...
Slide 75
Slide 75 text
3. ଞਓͷίʔυΛࢼͦ͏
• DCGAN
• https://github.com/mattya/chainer-DCGAN
• PaintsChainer
• https://github.com/pfnet/PaintsChainer
• QRNN
• http://bit.ly/jekbradbury-qrnn
Slide 76
Slide 76 text
ChainerΛ͑
Ϟσϧ؆୯ʹΊΔʂ
Slide 77
Slide 77 text
৭ʑࢼͯ͠Έͯ
໘ന͍͜ͱΛ͠·͠ΐ͏
Slide 78
Slide 78 text
Thank you!