ظ
• ֬ม͕࿈ଓͷͱ͖ʢ֬ີؔʣ
µi =
E
{
xi
} =
Z
dxi xip
(
xi)
µi =
E
{
xi
} =
X
k
x
(k)
i P
⇣
x
(k)
i
⌘
• ֬ม͕ࢄͷͱ͖ʢ࣭֬ྔؔʣ
µ
= ¯
x
=
1
N
N
X
i=1
xi
• ؍ଌσʔλ͕ N ݸ༩͑ΒΕ͍ͯΔͱ͖
Slide 14
Slide 14 text
पล֬
• ֬ม͕ࢄͷͱ͖ʢ࣭֬ྔؔʣ
• ͍ͯ͠ΔಛྔͰͳ͍ͷͥΜͿʢੵʣ
ΛͱΔ
ϋϯόʔά͕͖͔ʁ
yes no sum
ΤϏϑϥΠ
͕͖͔ʁ
yes 60 40 100
no 30 20 50
sum 90 60
p
(
xi) =
Z
dx1
Z
dx2
· · ·
Z
dxi 1
Z
dxi+1
· · ·
Z
dxd p
(
x1, x2, . . . , xd)
Slide 15
Slide 15 text
ڞࢄߦྻ
⌃ = Var {
x
}
= E (
x µ
)(
x µ
)T
=
0
B
@
E {(
x1 µ1)(
x1 µ1)}
. . .
E {(
x1 µ1)(
xd µd)}
.
.
.
...
.
.
.
E {(
xd µd)(
x1 µ1)}
. . .
E {(
xd µd)(
xd µd)}
1
C
A
= ( ij)
Slide 16
Slide 16 text
ڞࢄߦྻ
• ؍ଌσʔλ͕ N ݸ༩͑ΒΕ͍ͯΔͱ͖
ij =
1
N
N
X
n=1
(
xni µi)(
xnj µj)
Slide 17
Slide 17 text
ڞࢄߦྻ
• ຊདྷෆภࢄΛ͏͖
• ظΛͱͬͨͱ͖ʹਅͷࢄʹ
ऩଋ͢ΔΑ͏ௐઅ͢Δ
• ඪຊ͕ଟ͍߹େࠩͳ͍ͷͰ
͜͜Ͱγϯϓϧʹ͍ͯ͠Δ
sij =
1
N
1
N
X
n=1
(
xni µi)(
xnj µj)
Slide 18
Slide 18 text
ࢄͱඪ४ภࠩ
• ࢄฏۉ͔Βͷೋޡࠩͷظ
• େ͖͚ΕΒ͍͍ͭͯΔ
• ࢄͷฏํ͕ࠜඪ४ภࠩ
• ࢄͩͱಛྔͷ୯Ґ͕มΘͬͯ͠·͏ͨΊ
ͱͷ୯Ґʹ͢
2
i
= ii = E (
xi µi)2
i =
q
2
i
Slide 19
Slide 19 text
ڞࢄͱ૬ؔ
• ڞࢄҟͳΔಛྔؒͰΒ͖ͭํ͕
ಉ͔͡Ͳ͏͔Λಛ͚ͮΔ
• ਖ਼ͷ૬͕ؔ͋Ε +ɺෛͷ૬͕ؔ͋Ε -
• ڞࢄ୯ҐΛ͍࣋ͬͯΔͨΊ
୯Ґ͕ҧ͏ڞࢄͲ͏͠ΛൺֱͰ͖ͳ͍
ij = E {(
xi µi)(
xj µj)}
Slide 20
Slide 20 text
ڞࢄͱ૬ؔ
⇢ij = ij
i j
• ڞࢄΛແ࣍ݩԽͨ͠ͷ͕૬ؔ
• ਖ਼ͷ૬͕ؔ͋Ε +ɺෛͷ૬͕ؔ͋Ε -
• ඞͣ [-1, 1] ΛͱΔ
Slide 21
Slide 21 text
૬ؔઢܗ૬͔ؔ͠ΩϟονͰ͖ͳ͍
• x = [-2, -1, 0, 1, 2], y = x^2 ͷͱ͖
૬ؔ ρ_xy θϩ
https://upload.wikimedia.org/wikipedia/commons/d/d4/Correlation_examples2.svg
Slide 22
Slide 22 text
ϕΫτϧతͳղऍ
• N ݸͷଌఆ͕͋Δͱ͖ɺ
ij =
1
N
N
X
n=1
(
xni µi)(
xnj µj)
=
1
N
N
X
n=1
dnidnj
=
1
N
di
· dj
di = (
x1i µi, x2i µi, . . . , xNi µi)T
= (
d1i, d2i, . . . , dNi)T
ͱ͓͘ͱɺڞࢄੵʢͷఆഒʣʹͳΔ
ϕΫτϧۭؒͱͯ͠
ѻ͏ͨΊʹ
֤ಛྔͰ୯Ґ͕
ἧ͍ͬͯΔඞཁ͕͋Δ
Slide 23
Slide 23 text
2
i
=
1
N
di
· di
=
1
N
|di
|2
i = =
1
p
N
|di
|
ϕΫτϧతͳղऍ
⇢ij =
ij
i j
=
(1
/N
)
di
· dj
(1
/
p
N
)
|di
|
(1
/
p
N
)
|dj
|
=
di
· dj
|di
| |dj
|
= cos
✓ij
Slide 24
Slide 24 text
ϕΫτϧతͳղऍ
• શͳਖ਼ͷ૬͕ؔ͋Δͱ͖
⇢ij = 1
cos
✓ij = 1
✓ij = 0
di =
cdj (
c >
0)
Slide 25
Slide 25 text
ϕΫτϧతͳղऍ
di
dj
⇢ij = 1 ⇢ij = 0 ⇢ij = 1
di
dj
di
dj
{
}
p
N i
p
N j
ฏۉɾࢄͱઢܗม
• ઢܗมΛߟ͑Δ
y
=
ax
+
b
E {
y
} = E {
ax
+
b
} =
a
E {
x
} +
b
=
aµ
+
b
• ฏۉͱࢄҎԼͷΑ͏ʹԠ
Var {
y
} = E (
y Ey
)2
= E [
ax
+
b
(
aµ
+
b
)]2 = E
a
2(
x µ
)2
=
a
2E (
x µ
)2 =
a
2Var {
x
}
=
a
2 2
Slide 29
Slide 29 text
ඪ४Խ
• ҎԼͷઢܗมΛ͏ͱ
ฏۉ 0 ɺࢄ 1 ͷಛྔ͕ಘΒΕΔ
z
= x µ
E {
z
} = E
⇢
x µ =
1
(E {
x
}
µ
)
= 0
Var {
z
} = Var
⇢
x µ =
1
2
Var {
x
}
= 1
ݻ༗ϕΫτϧ
• ରশߦྻʹରͯ͠ɿ
• ݻ༗࣮
• ݻ༗ϕΫτϧަ
➡ ݻ༗ϕΫτϧਖ਼نަجఈ
sT
i
sj = ij =
⇢
1 (i = j)
0 (i 6= j)
Slide 40
Slide 40 text
ճసߦྻ
• ݻ༗ϕΫτϧΛฒͯߦྻΛ࡞Δ
• ਖ਼نަجఈΛฒͨߦྻަߦྻͱͳΔ
S = (s1, s2, . . . , sd)
(ST S)ij = sT
i
sj = ij
) ST S = I
) ST = S 1
• ͜ͷ߹ճసߦྻʢ㱬ަߦྻʣͱͳΔ
Slide 41
Slide 41 text
ແ૬ؔԽ
y
= ST
x
E {
y
} = E ST
x
= ST E {
x
}
= ST
µ
Var {
y
} = E (
y
E {
y
})(
y
E {
y
})T
= E (ST
x
ST
µ
)(ST
x
ST
µ
)T
= E ST (
x µ
)[ST (
x µ
)]T
= E ST (
x µ
)(
x µ
)T S
= ST E (
x µ
)(
x µ
)T S
= ST ⌃S
Slide 42
Slide 42 text
ແ૬ؔԽ
S 1⌃S = S 1⌃(s1, s2, . . . , sd) = S 1( 1s1, 2s2, . . . , dsd)
= S 1S
0
B
B
B
@
1 0 . . . 0
0 2 . . . 0
.
.
.
...
.
.
.
0 0 . . . d
1
C
C
C
A
=
0
B
B
B
@
1 0 . . . 0
0 2 . . . 0
.
.
.
...
.
.
.
0 0 . . . d
1
C
C
C
A
= ⇤
• ͳͷͰɺແ૬ؔԽ͞Ε͍ͯΔ
• ແ૬͕ؔͩɺඪ४Խ͞Ε͍ͯͳ͍
(Var {y})ij = 0 (i 6= j)
(Var {y})ii = i
Slide 43
Slide 43 text
ന৭Խ
Slide 44
Slide 44 text
ന৭Խ
• ඪ४Խʴແ૬ؔԽ
u
= ⇤ 1/2ST (
x µ
)
(⇤ 1/2)ij =
⇢
1/
p
i (i = j)
0 (i 6= j)
Slide 45
Slide 45 text
ന৭Խ
E {
u
} = E n⇤ 1/2ST (
x µ
)o = ⇤ 1/2ST (E {
x
}
µ
)
= 0
Var {
u
} = E n⇤ 1/2ST (
x µ
)(
x µ
)T S(⇤ 1/2)T o
= ⇤ 1/2ST E (
x µ
)(
x µ
)T S(⇤ 1/2)T
= ⇤ 1/2ST ⌃S(⇤ 1/2)T
= ⇤ 1/2⇤(⇤ 1/2)T
= I
• ඪ४Խ͞Εɺ͔ͭແ૬ؔԽ͞Εͨʂ
ਖ਼ن
• ૬ؔͷ͚ͩճస͠ɺඪ४ภࠩͷ͚ͩҾ͖৳͞Ε͍ͯΔ
• ന৭Խͷٯ
(
x µ
)T ⌃ 1(
x µ
) = (
x µ
)T [S⇤S 1] 1(
x µ
)
= (
x µ
)T S⇤ 1S(
x µ
)
= [ST (
x µ
)]T ⇤ 1[ST (
x µ
)]
=
y
T ⇤ 1
y
(*
y
⌘ ST (
x µ
))
=
y
T (⇤1/2)T ⇤1/2
y
= (⇤1/2
y
)T (⇤1/2
y
)
=
z
T
z
(*
z
⌘ ⇤ 1/2
y
)
Slide 54
Slide 54 text
ϚϋϥϊϏεڑ
• ͷத৺͔ΒͲΕ͚ͩΕ͍ͯΔ͔ͷࢦඪ
• ന৭Խۭͨؒ͠Ͱͷڑ
d(
x
,
µ
) =
q
(
x µ
)T ⌃ 1(
x µ
)
=
p
z
T
z
Slide 55
Slide 55 text
ਖ਼ن͔Βಋ͔ΕΔ
ࣝผؔ
Slide 56
Slide 56 text
Ϋϥε͖݅֬
• Ϋϥε͖͕݅֬ਖ਼نͰ͋ΔͱԾఆ͢Δ
ln
P
(
Ci
|
x) =
p
(x
|Ci)
P
(
Ci)
p
(x)
/ p
(x
|Ci)
P
(
Ci)
=
P
(
Ci)
(2
⇡
)
d/2 |
⌃i
|1/2 exp
1
2
(x µi)
T
⌃
1
i (x µi)
p
(x
|Ci) =
1
(2
⇡
)
d/2 |
⌃i
|1/2 exp
1
2
(x µi)
T
⌃
1
i (x µi)
Slide 57
Slide 57 text
Ϋϥε͖݅֬
• ؔͳ͍߲ΛΦϛοτɺ ×(-2)
ln P(Ci
|
x
) = ln P(Ci)
d
2
ln(2⇡)
1
2
ln |⌃i
|1/2
1
2
(
x µi)T ⌃ 1
i
(
x µi)
gi(
x
) = (
x µi)T ⌃ 1
i
(
x µi) + ln |⌃i
| 2 ln P(Ci)
[Recognized class] = arg min
i [
gi(x)]
Slide 58
Slide 58 text
ࣝผڥք
• ΫϥεؒͷڥʢࣝผڥքʣҎԼͷํఔࣜͰ
༩͑ΒΕΔ
fij(
x
) = gi(
x
) gj(
x
) = 0
Slide 59
Slide 59 text
ࣝผڥք
fij(
x
) = gi(
x
) gj(
x
)
= (
x µi)T ⌃ 1
i
(
x µi) + ln |⌃i
| 2 ln P(Ci)
(
x µj)T ⌃ 1
j
(
x µj) ln |⌃j
| + 2 ln P(Cj)
=
x
⌃ 1
i x x
⌃ 1
i µi µi⌃ 1
i x
+
µi⌃ 1
i µi
x
⌃ 1
j x x
⌃ 1
j µj µj⌃ 1
j x
+
µj⌃ 1
j µj
+ ln
|⌃i
|
⌃j
2 ln
P(Ci)
P(Cj)
=
x
(⌃ 1
i
⌃ 1
j
)
x
+ 2(
µ
T
j
⌃ 1
j µ
T
i
⌃ 1
i
)
x
+ µT
i
⌃ 1
i
µiµT
j
⌃ 1
j
µj + ln
|⌃i
|
⌃j
2 ln
P(Ci)
P(Cj)
)
x
T S
x
+ 2
c
T
x
+ F = 0 ʢ2 ࣍ࣝผؔʣ
+µT
i
⌃ 1
i
µi µT
j
⌃ 1
j
µj + ln
|⌃i
|
|⌃j
|
2 ln
P(Ci)
P(Cj)
fij(
x
) = gi(
x
) gj(
x
)
= (
x µi)T ⌃ 1
i
(
x µi) + ln |⌃i
| 2 ln P(Ci)
(
x µj)T ⌃ 1
j
(
x µj) ln |⌃j
| + 2 ln P(Cj)
=
x
⌃ 1
i x x
⌃ 1
i µi µi⌃ 1
i x
+
µi⌃ 1
i µi
x
⌃ 1
j x x
⌃ 1
j µj µj⌃ 1
j x
+
µj⌃ 1
j µj
+ ln
|⌃i
|
⌃j
2 ln
P(Ci)
P(Cj)
=
x
(⌃ 1
i
⌃ 1
j
)
x
+ 2(
µ
T
j
⌃ 1
j µ
T
i
⌃ 1
i
)
x
+ µT
i
⌃ 1
i
µiµT
j
⌃ 1
j
µj + ln
|⌃i
|
⌃j
2 ln
P(Ci)
P(Cj)
x
⌃ 1
j x
+
x
⌃ 1
j µj +
µj⌃ 1
j x µj⌃ 1
j µj
Slide 60
Slide 60 text
ࣝผڥքʢิʣ
x
T
i ⌃
1
i µi = (Scalar) = (x
T
i ⌃
1
i µi)
T
= µ
T
i (⌃
1
i )
T
x
= µ
T
i ⌃
1
i x (
*
⌃
1
i is a symmetric matrix)
• ରশߦྻͷٯߦྻରশߦྻͰ͋Δ͜ͱʹҙ
Slide 61
Slide 61 text
ࣝผڥք
• ;ͨͭͷΫϥεͷڞࢄߦྻ͕͍͠ͱ͖
fij(
x
) = 2
c
T
x
+ F = 0 ʢઢܗࣝผؔʣ
Slide 62
Slide 62 text
ࣝผڥք
⌃i = ⌃j = I P(Ci) = P(Cj)
• ͔ͭ ͷͱ͖
fij(
x
) = 2(
µ
T
j
⌃ 1
j µ
T
j
⌃ 1
i
)
x
+
µ
T
i
⌃ 1
i µi µ
T
i
⌃ 1
i µi
= 2 (
µ
T
j µ
T
i
)
x
+
µ
T
i µi µ
T
j µj = 0
x
T
x
+ 2
µ
T
i x
+
µ
T
i µi x
T
x
+ 2
µ
T
j x µ
T
j µj = 0
(
x µi)T (
x µi) (
x µj)T (
x µj) = 0
) (
x µi)T (
x µi) = (
x µj)T (
x µj)
x
T
x
2
µ
T
i x
+
µ
T
i µi x
T
x
+ 2
µ
T
j x µ
T
j µj = 0
• ྆ล σ ͰׂΓɺ x^T x Λ͠Ҿ͖
ʢ࠷ۙ๏……ͱຊʹॻ͍ͯ͋Δ͕ઢܗผੳʢLDAʣͰʁ
ʢฏۉ͔ΒͷڑΛൺ͍ͯΔͷͰʣʣ
+µT
i
⌃ 1
i
µi µT
j
⌃ 1
j
µj
fij(
x
) = 2(
µ
T
j
⌃ 1
j µ
T
j
⌃ 1
i
)
x
+
µ
T
i
⌃ 1
i µi µ
T
i
⌃ 1
i µi
= 2 (
µ
T
j µ
T
i
)
x
+
µ
T
i µi µ
T
j µj = 0
1 มਖ਼نͷ߹
L(µ,
2
) = f(x1, x2, . . . , xN
|
µ,
2
)
=
N
Y
i=1
1
p
2⇡
2 exp
(xi µ)
2
2
2
= (2⇡
2
)
N/2
exp
"
1
2
2
N
X
i=1
(xi µ)
2
#
Lln(
µ,
2) = N
2
ln(2
⇡
) N
2
ln 2
1
2 2
N
X
i=1
(
xi µ
)2
Slide 73
Slide 73 text
1 มਖ਼نͷ߹
Lln(
µ,
2) = N
2
ln(2
⇡
) N
2
ln 2
1
2 2
N
X
i=1
(
xi µ
)2
• ରΛ֤ύϥϝʔλͰภඍͯ͠ۃΛٻΊΔ
Slide 74
Slide 74 text
1 มਖ਼نͷ߹
@Lln(ˆ
µ,
2)
@µ
= @
@µ
"
1
2 2
N
X
i=1
(
xi µ
)2
#
µ=ˆ
µ
= 0
1
2 2
N
X
i=1
2(
xi ˆ
µ
)( 1) = 0
N
X
i=1
xi
N
X
i=1
ˆ
µ
= 0
) ˆ
µ
=
1
N
N
X
i=1
xi
Slide 75
Slide 75 text
1 มਖ਼نͷ߹
@Lln(
µ,
ˆ2)
@
2
= @
@
2
"
N
2
ln 2
1
2 2
N
X
i=1
(
xi µ
)2
#
2=ˆ2
= 0
N
2
1
ˆ2
1
2
1
(ˆ2)2
( 1)
N
X
i=1
(
xi µ
)2 = 0
N
ˆ2
+
1
(ˆ2)2
N
X
i=1
(
xi µ
)2 = 0
) ˆ2 =
1
N
N
X
i=1
(
xi µ
)2