Koga Kobayashi
August 17, 2020
140

# 初等確率論の基礎

「ベイズ統計の理論と方法」勉強会の資料

August 17, 2020

## Transcript

1. ॳ౳֬཰࿦ͷجૅ
ϕΠζ౷ܭͷཧ࿦ͱํ๏ษڧձ

2. ֬཰෼෍ͱ֬཰ม਺

3. ֬཰෼෍
ϢʔΫϦουۭؒ ͷݩ ͷؔ਺ ͕
ℝN x = (x1
, …, xN
) q(x) ≥ 0

q(x)dx ≡

dx1 ∫
dx2

dxN
q(x1
, x2
, ⋯, xN
) = 1
Λຬͨ͢ͱ͖ Λ֬཰෼෍͋Δ͍͸֬཰ີ౓ؔ਺ͱ͍͏ɻ
q(x)
ू߹ ʹ͍ͭͯɺ ͷݩͰͷू߹ ͷ֬཰͸
A ⊂ ℝN q(x) A
Q(A) =

A
q(x)dx
͜ͷͱ͖ɺؔ਺ ΋֬཰෼෍ͱ͍͏ɻ
Q( ⋅ )

4. ֬཰ม਺
ϢʔΫϦουۭؒ ͷ্ʹϥϯμϜʹ஋ΛऔΔม਺ Λ
ʮ ʹ஋ΛऔΔ֬཰ม਺ʯͱ͍͏ɻ
ℝN X
ℝN
ʮ ͱͳΔ֬཰ʯ͕ Ͱ͋Δͱ͖
ʮ֬཰ม਺ ͷ֬཰෼෍͸ Ͱ͋Δʯ͋Δ͍͸
ʮ֬཰ม਺ ͷ֬཰෼෍͸ ʹै͏ʯ͋Δ͍͸
ʮ֬཰ม਺ ͷ֬཰෼෍͸ Ͱ͋Δʯͱ͍͏ɻ
X ∈ A Q(A)
X q(x)
X q(x)
X Q

5. ۩ମྫਅͷ෼෍
αϯϓϧ ͕͋Δ֬཰෼෍ ʹಠཱʹै͏
֬཰ม਺ͷ࣮ݱ஋ʢ؍ଌ஋ʣͩͱ͢Δɻ
A = xn = {x1
, …, xn
} ⊂ ℝN q(x)
͢ͳΘͪ Λ ্ͷ෼෍
xn (ℝN)n
q(xn) =
n

i=1
q(xi
) = q(x1
)q(x2
)⋯q(xn
)
Λ࣋ͭ֬཰ม਺ ͷ࣮ݱ஋Ͱ͋Δͱߟ͑Δɻ
͜ͷͱ͖֬཰෼෍ Λਅͷ෼෍ͱݺͿɻ
Xn = (X1
, X2
, …, Xn
)
q(x)

6. ฏۉͱ෼ࢄ

7. ฏۉͱ෼ࢄ
ʹ஋ΛͱΔ֬཰ม਺ ͷ֬཰෼෍Λ ͱ͢Δɻ
ℝN X q(x)
[f(X)] ≡

f(x)q(x)dx
[f(X)] ≡ [(f(X) − [f(X)])(f(X) − [f(X)])T]
= [f(X)f(X)T] − [f(X)][f(X)T]
ͱఆٛ͢Δɻ
͕༩͑ΒΕͨͱ͖ɺ֬཰ม਺ ͷฏۉΛ
f : ℝN → ℝM f(X)
·ͨ෼ࢄڞ෼ࢄΛ
ͱఆٛ͢Δɻ֬཰ม਺Λ໌ه͍ͨ͠ͱ͖͸ ͱॻ͘ɻ
X
[f(X)]

8. ۩ମྫαϯϓϧͷฏۉ஋
αϯϓϧ Λද֬͢཰ม਺Λ ͱ͢Δɻ
ͦͷؔ਺ ͕༩͑ΒΕͨͱ͖ɺͦͷฏۉ஋ΛऔΔૢ࡞ Λ
xn = {x1
, …, xn
} Xn = (X1
, X2
, …, Xn
)
f(Xn) [ ⋅ ]
ͱදه͢Δɻ

͜ͷฏۉ஋ ΛʮαϯϓϧͷݱΕํʹର͢Δฏۉ஋ʯͱݺͿɻ
[ ⋅ ]
[f(XN)] =
∫ ∫

f(x1
, …, xn
)
n

i=1
q(xi
)dxi

9. ۩ମྫਅͷ෼෍ͷฏۉ
αϯϓϧͷ֬཰ม਺Λ Λ༻͍ͯɺ
ਅͷ෼෍ ͷਪଌΛߦͬͨޙɺਅͷ෼෍ͷ֬཰ม਺ Λൃੜͤͯ͞
ਪଌ݁ՌͷΑ͞ΛධՁ͍ͨ͠ɻ
͜ͷ֬཰ม਺ ͷؔ਺ ʹ͍ͭͯͷฏۉΛ
Xn = (X1
, X2
, …, Xn
)
q(x) X
X f(X)
ͱදه͢Δɻ
[f(X)]X
=

f(x)q(x)dx

10. X
X−1
֬཰ۭؒ(Ω = ℝM, ℬ, p)
w ∈ Ω
ٯ૾X−1(A)
֬཰ີ౓ؔ਺ ֬཰෼෍
q(x) = p(X−1(x))
Մଌۭؒ(Ω′ = ℝN, ℬ′ )
A ∈ ℬ′
X(w) = X
x ∈ Ω′
֬཰෼෍Q(A) =

A
q(x)dx
f(x)
ฏۉ[f(X)] ≡

f(x)q(x)dx =

f(x)p(X−1(x))dx =

p(w)X(w)dw =

pXdw
֬཰ม਺
֬཰ม਺ͱ֬཰෼෍ɺฏۉͷؔ܎
֬཰ۭؒ(Ω′ = ℝN, ℬ′ , q)

11. ಉ࣌෼෍ͱ৚݅෇͖֬཰

12. ಉ࣌෼෍ͱ৚݅෇͖෼෍
ͭͷ֬཰ม਺ ͱ ͕͋Δͱ͖ɺͦͷ૊ ͷ֬཰෼෍͕
Ͱ͋Δͱ͖ɺ Λಉ࣌֬཰෼෍ͱ͍͏ɻ
X Y (X, Y)
p(x, y) p(x, y)
·ͨ֬཰ม਺ ͕༩͑ΒΕͨͱ͖ͷ ͷ৚݅෇͖֬཰෼෍Λ࣍ͷΑ͏
ʹఆٛ͢Δɻ
X Y
p(y|x) =
p(x, y)
p(x)
पล֬཰෼෍͸࣍ͷΑ͏ʹఆٛ͢Δɻ
p(x) =

p(x, y)dy p(y) =

p(x, y)dx

13. ճؼؔ਺
֬཰ม਺ ͷ֬཰෼෍ ʹ͍ͭͯߟ͑Δɻ
ͷͱ͖ͷ ͷฏۉ஋Λ
(X, Y) p(X, Y)
X = x Y
ͱॻ͘ɻ͜ͷؔ਺Λ ͔Β ͷճؼؔ਺ ৚݅෇͖ظ଴஋
ͱ͍͏ɻ
x y
[Y|x] =

yp(y|x)dy
ؔ਺Λ ͕༩͑ΒΕͨͱ͖ͦͷೋ৐ޡࠩΛද͢൚ؔ਺Λ
y = f(x)
[(Y − f(X))2] =
∫ ∫
(y − f(x))2p(y, x)dxdy
ͱॻ͘ͱ͜Ε͸ ͷͱ͖ʹ࠷খʹͳΔɻ
f(x) = [Y|x]

14. ΧϧόοΫɾϥΠϒϥ৘ใྔ

15. ΧϧόοΫɾϥΠϒϥ৘ใྔ

্ʹೋͭͷ֬཰෼෍ ͕͋Δͱ͖
ℝN q(x), p(x)
D(p∥q) =

q(x)log
q(x)
p(x)
dx
ͷ͜ͱΛΧϧόοΫɾϥΠϒϥ৘ใྔ͋Δ͍͸૬ରΤϯτϩϐʔͱݺͿ
ΧϧόοΫɾϥΠϒϥ৘ใྔ͸͕࣍੒Γཱͭɻ
ʹ͍ͭͯ Ͱ͋Δɻ
ͱͳΔͷ͸ ͷͱ͖ʹݶΔɻ
∀q(x), p(x) D(q∥p) ≥ 0
D(q∥p) = 0 q(x) = p(x)

16. ΧϧόοΫɾϥΠϒϥ৘ใྔ

ূ໌

ͱ͓͘ͱɺ Ͱ͋Γɺ
F(t) = 0 ⇔ t = 0
F(t) = t + et − 1 (−∞ < t < ∞)
ΑΓ Ͱ͋Δ͔Β͕ࣔ͞Εͨɻ

q(x)dx = 1

p(x)dx = 1

log
q(x)
p(x)
dx = 0
·ͨɺ ͷͱ͖ɺ Ͱ Ͱ͋Δ͜ͱΛ༻͍ͯ
q(x) ≈ p(x) t ≈ 0 F′ ′ (t) ≃ t2/e
D(p∥q) ≃

q(x)(log q(x) − log p(x))2dx
͕੒Γཱͭɻ

17. ۃݶఆཧ

18. ֬཰ม਺ͷऩଋ

19. ֬཰ऩଋ
֬཰ม਺ ͕ఆ਺ ʹ֬཰ऩଋ͢Δͱ͸
ʹର͠ɺ ʹ͓͍ͯ
{Xn
}n∈ℕ
c
∀ϵ, ∀δ > 0 ∃N ∈ ℕ
n > N ⇒ P(∥Xn
− c∥ > ϵ) < δ
⇔ P(∥Xn
− c∥ < ϵ) = 1
ͱͳΔ͜ͱͰ͋Δɻ
͜Ε͸େ਺ͷऑ๏ଇʹରԠ͍ͯ͠Δɻ
Xn
c
ϵ
ඪຊ͕े෼ʹେ͖͍ͱ͖ɺඪຊฏۉ͸฼ฏۉʹऩଋ͢Δ

20. ๏ଇ ෼෍
ऩଋ
֬཰ม਺ͷྻ ͕֬཰ม਺ ʹ๏ଇ ෼෍
ऩଋ͢Δͱ͸
ͷ֬཰෼෍͕ Ͱ ͷ֬཰෼෍͕ Ͱ͋Δͱ͖ɺ
೚ҙͷ༗ք͔ͭ࿈ଓͳؔ਺ ʹରͯ͠
{Xn
}n∈ℕ
X
Xn
qn
(x) X q(x)
F(x)
lim
n→∞

F(x)qn
(x)dx =

F(x)q(x)dx
⇔ lim
n→∞
[F(Xn
)] = [F(X)]
͕੒Γཱͭ͜ͱͰ͋Δɻ͜Ε͸த৺ۃݶఆཧʹରԠ͍ͯ͠Δɻ
ඪຊ͕े෼ʹେ͖͍ͱ͖ɺ฼ूஂͷ෼෍ʹؔΘΒͣඪຊฏۉͱ฼ฏۉͷࠩ͸ਖ਼ن෼෍ʹै͏

21. ܦݧաఔ

22. ϢʔΫϦουۭؒʹ͓͚ΔίϯύΫτੑ
ϢʔΫϦουۭؒ ͷ෦෼ू߹ ͕։ू߹ͷ଒ ʹ
͍ͭͯ ͳΒ͹ɺͦͷ༗ݶݸͷ։ू߹ Ͱ
ℝN W = {O}λ∈Λ
W ⊂ ⋃
λ∈Λ

O1
, …, On

ͱͳΔ΋ͷ͕͋Δͱ͖ɺ ͸ίϯύΫτͰ͋Δͱ͍͏
W ⊂ O1
∪ … ∪ On
W
O1
, …, On

W

23. ؔ਺্ۭؒͷେ਺ͷ๏ଇ
ϢʔΫϦουۭؒ ʹ஋ΛऔΔ ͕֬཰ม਺ ͱ
ಉ֬͡཰෼෍ʹै͏ͱ͢Δɻ
ύϥϝʔλͷू߹ ΛίϯύΫτͱ͢Δɻ
ℝN X1
, X2
, …, Xn
X
w ∈ W ∈ ℝN f(x, w) : ℝN → ℝ1
X
[ sup
w∈W
|f(X, w)|] < ∞, X
[ sup
w∈W
|∇w
f(X, w)|] < ∞
৚݅
͕੒ΓཱͭͱԾఆ͢Δɻ͜ͷͱ͖ɺ ʹ͍ͭͯ
∀ϵ > 0
P( sup
w∈W
1
n
n

i=1
f(Xi
, w) − X
[f(X, w)] < ϵ) = 1
͜ͷ͜ͱΛؔ਺্ۭؒͷେ਺ͷ๏ଇͱ͍͏

24. ਖ਼ن֬཰աఔ
ू߹ ্ͷؔ਺Ͱ֬཰తʹมಈ͢Δ΋ͷ ͕ɺ
ฏۉؔ਺ ͱ૬ؔؔ਺ Λ࣋ͭਖ਼ن֬཰աఔͰ͋Δͱ͸ɺ
֤ ͝ͱʹ ͕ਖ਼ن෼෍ʹै͏֬཰ม਺Ͱ͋Γɺ
W ξ(w)
m(w) ρ(w, w′ )
w ξ(w)

m(w) = ξ
[ξ(w)],
ρ(w, w′ ) = ξ
[ξ(w)ξ(w′ )]
͕੒Γཱͭ͜ͱͰ͋Δɻ͜͜Ͱ ͸ɺ֬཰աఔ ʹ͍ͭͯͷฏۉΛ
ද͍ͯ͠ΔɻίϯύΫτू߹্Ͱͷਖ਼ن֬཰աఔ͸ɺ
ξ
[ ⋅ ] ξ
ฏۉؔ਺ͱ૬ؔؔ਺͕ܾ·ΔͱҰҙʹఆ·Δ͜ͱ͕஌ΒΕ͍ͯΔɻ

25. ܦݧաఔ

ͭ͗ʹ
X[ sup
w∈W
|f(X, w) − X
[f(X, w)]|α
] < ∞
X[ sup
w∈W
|∇w
(f(X, w) − X
[f(X, w)])|α
] < ∞
͕ Ͱ੒ΓཱͭͱԾఆ͢Δɻ
α = 2
Yn
(w) =
1
n
n

i=1
(f(Xi
, w) − X
[f(X, w)])
͜ͷ֬཰աఔ Λܦݧաఔͱ͍͏ɻ
Yn
(w)

26. ܦݧաఔ

֬཰աఔ ܦݧաఔ
͸ฏۉ͕ Ͱ૬ؔؔ਺͕
Yn
(w) 0
ͷਖ਼ن֬཰աఔ ʹ๏ଇऩଋ͢Δɻ
Y(w)

ρ(w, w′ ) = X
[f(X, w)f(X, w′ )] − X
[f(X, w)]X
[f(X, w′ )]

27. ֬཰աఔͷ๏ଇऩଋ
֬཰աఔ ܦݧաఔ
͕֬཰աఔ ʹ๏ଇऩଋ͢Δͱ͸ɺ
༗ք࿈ଓͳ൚ؔ਺ ʹ͍ͭͯ
Yn
(w) Y(w)
F( ⋅ )
͕੒Γཱͭͱ͍͏͜ͱͰ͋Δɻͳ͓ɺ൚ؔ਺ ͕࿈ଓͰ͋Δͱ͸
F( ⋅ )

lim
n→∞
[F(Yn
)] = Y
[F(Y)]

lim
n→∞
sup
w∈W
|fn
(w) − f(w)| → 0 ⇒ lim
n→∞
F(fn
) = F(f )
͕੒Γཱͭ͜ͱͰ͋Δɻ
͜ͷΑ͏ͳܗͷఆཧΛؔ਺্ۭؒͷத৺ۃݶఆཧͱ͍͏ɻ

28. ࢀߟࢿྉ
w ֬཰࿦ೖ໳ ౉ล੅෉
w ܦݧաఔͱ͸ ౉ล੅෉
w ϕΠζ౷ܭͷཧ࿦ͱํ๏ ౉ล੅෉
w ଌ౓ɾ֬཰ɾϧϕʔάੵ෼ ݪܒհ