初等確率論の基礎

「ベイズ統計の理論と方法」勉強会の資料

Transcript

1. ॳ౳֬཰࿦ͷجૅ ϕΠζ౷ܭͷཧ࿦ͱํ๏
   ษڧձ
2. 
   ֬཰෼෍ͱ֬཰ม਺
3. ֬཰෼෍ ϢʔΫϦουۭؒ
   
   ͷݩ
   
   ͷؔ਺
   
   ͕ ℝN x = (x1 , …, xN ) q(x) ≥ 0 ∫ q(x)dx ≡ ∫ dx1 ∫ dx2 … ∫ dxN q(x1 , x2 , ⋯, xN ) = 1 Λຬͨ͢ͱ͖
   
   Λ֬཰෼෍͋Δ͍͸֬཰ີ౓ؔ਺ͱ͍͏ɻ q(x) ू߹
   
   ʹ͍ͭͯɺ
   ͷݩͰͷू߹
   
   ͷ֬཰͸ A ⊂ ℝN q(x) A Q(A) = ∫ A q(x)dx ͜ͷͱ͖ɺؔ਺
   
   ΋֬཰෼෍ͱ͍͏ɻ Q( ⋅ )
4. ֬཰ม਺ ϢʔΫϦουۭؒ
   
   ͷ্ʹϥϯμϜʹ஋ΛऔΔม਺
   
   Λ
   ʮ
   ʹ஋ΛऔΔ֬཰ม਺ʯͱ͍͏ɻ ℝN X ℝN ʮ
   ͱͳΔ֬཰ʯ͕
   
   Ͱ͋Δͱ͖
   ʮ֬཰ม਺
   
   ͷ֬཰෼෍͸
   
   Ͱ͋Δʯ͋Δ͍͸
   ʮ֬཰ม਺
   
   ͷ֬཰෼෍͸
   
   ʹै͏ʯ͋Δ͍͸
   ʮ֬཰ม਺
   
   ͷ֬཰෼෍͸
   
   Ͱ͋Δʯͱ͍͏ɻ X ∈ A Q(A) X q(x) X q(x) X Q
5. ۩ମྫ
   ਅͷ෼෍ αϯϓϧ
   
   ͕͋Δ֬཰෼෍
   
   ʹಠཱʹै͏ ֬཰ม਺ͷ࣮ݱ஋ʢ؍ଌ஋ʣͩͱ͢Δɻ A = xn = {x1 , …, xn } ⊂ ℝN q(x) ͢ͳΘͪ
   
   Λ
   
   ্ͷ෼෍ xn (ℝN)n q(xn) = n ∏ i=1 q(xi ) = q(x1 )q(x2 )⋯q(xn ) Λ࣋ͭ֬཰ม਺
   
   ͷ࣮ݱ஋Ͱ͋Δͱߟ͑Δɻ
   ͜ͷͱ͖֬཰෼෍
   
   Λਅͷ෼෍ͱݺͿɻ Xn = (X1 , X2 , …, Xn ) q(x)
6. 
   ฏۉͱ෼ࢄ
7. ฏۉͱ෼ࢄ
   ʹ஋ΛͱΔ֬཰ม਺
   
   ͷ֬཰෼෍Λ
   
   ͱ͢Δɻ ℝN X q(x) [f(X)] ≡ ∫ f(x)q(x)dx [f(X)] ≡ [(f(X) − [f(X)])(f(X) − [f(X)])T] = [f(X)f(X)T] − [f(X)][f(X)T] ͱఆٛ͢Δɻ
   ͕༩͑ΒΕͨͱ͖ɺ֬཰ม਺
   
   ͷฏۉΛ f : ℝN → ℝM f(X) ·ͨ෼ࢄڞ෼ࢄΛ ͱఆٛ͢Δɻ֬཰ม਺Λ໌ه͍ͨ͠ͱ͖͸
   
   ͱॻ͘ɻ X [f(X)]
8. ۩ମྫ
   αϯϓϧͷฏۉ஋ αϯϓϧ
   
   Λද֬͢཰ม਺Λ
   
   ͱ͢Δɻ
   ͦͷؔ਺
   
   ͕༩͑ΒΕͨͱ͖ɺͦͷฏۉ஋ΛऔΔૢ࡞ Λ xn = {x1 , …, xn } Xn = (X1 , X2 , …, Xn ) f(Xn) [ ⋅ ] ͱදه͢Δɻ
   
   ͜ͷฏۉ஋
   
   ΛʮαϯϓϧͷݱΕํʹର͢Δฏۉ஋ʯͱݺͿɻ [ ⋅ ] [f(XN)] = ∫ ∫ ⋯ ∫ f(x1 , …, xn ) n ∏ i=1 q(xi )dxi
9. ۩ମྫ
   ਅͷ෼෍ͷฏۉ αϯϓϧͷ֬཰ม਺Λ
   
   Λ༻͍ͯɺ
   ਅͷ෼෍
   
   ͷਪଌΛߦͬͨޙɺਅͷ෼෍ͷ֬཰ม਺
   
   Λൃੜͤͯ͞
   ਪଌ݁ՌͷΑ͞ΛධՁ͍ͨ͠ɻ
   ͜ͷ֬཰ม਺
   
   ͷؔ਺
   
   ʹ͍ͭͯͷฏۉΛ Xn = (X1 , X2 , …, Xn ) q(x) X X f(X) ͱදه͢Δɻ [f(X)]X = ∫ f(x)q(x)dx
10. X X−1 ֬཰ۭؒ(Ω = ℝM, ℬ, p) w ∈ Ω ٯ૾
    X−1(A) ֬཰ີ౓ؔ਺ ֬཰෼෍ 
    q(x) = p(X−1(x)) Մଌۭؒ(Ω′ = ℝN, ℬ′ ) A ∈ ℬ′ X(w) = X x ∈ Ω′ ֬཰෼෍
    Q(A) = ∫ A q(x)dx f(x) ฏۉ
    [f(X)] ≡ ∫ f(x)q(x)dx = ∫ f(x)p(X−1(x))dx = ∫ p(w)X(w)dw = ∫ pXdw ֬཰ม਺ ֬཰ม਺ͱ֬཰෼෍ɺฏۉͷؔ܎ ֬཰ۭؒ
    (Ω′ = ℝN, ℬ′ , q)
11. 
    ಉ࣌෼෍ͱ৚݅෇͖֬཰
12. ಉ࣌෼෍ͱ৚݅෇͖෼෍ ͭͷ֬཰ม਺
    
    ͱ
    
    ͕͋Δͱ͖ɺͦͷ૊
    
    ͷ֬཰෼෍͕
    
    Ͱ͋Δͱ͖ɺ
    Λಉ࣌֬཰෼෍ͱ͍͏ɻ X Y (X, Y) p(x, y) p(x, y) ·ͨ֬཰ม਺
    
    ͕༩͑ΒΕͨͱ͖ͷ
    
    ͷ৚݅෇͖֬཰෼෍Λ࣍ͷΑ͏ ʹఆٛ͢Δɻ X Y p(y|x) = p(x, y) p(x) पล֬཰෼෍͸࣍ͷΑ͏ʹఆٛ͢Δɻ p(x) = ∫ p(x, y)dy p(y) = ∫ p(x, y)dx
13. ճؼؔ਺ ֬཰ม਺
    
    ͷ֬཰෼෍
    
    ʹ͍ͭͯߟ͑Δɻ
    
    ͷͱ͖ͷ
    
    ͷฏۉ஋Λ (X, Y) p(X, Y) X = x Y ͱॻ͘ɻ͜ͷؔ਺Λ
    
    ͔Β
    
    ͷճؼؔ਺ ৚݅෇͖ظ଴஋ ͱ͍͏ɻ
    x y [Y|x] = ∫ yp(y|x)dy ؔ਺Λ
    
    ͕༩͑ΒΕͨͱ͖ͦͷೋ৐ޡࠩΛද͢൚ؔ਺Λ y = f(x) [(Y − f(X))2] = ∫ ∫ (y − f(x))2p(y, x)dxdy ͱॻ͘ͱ͜Ε͸
    
    ͷͱ͖ʹ࠷খʹͳΔɻ f(x) = [Y|x]
14. 
    ΧϧόοΫɾϥΠϒϥ৘ใྔ
15. ΧϧόοΫɾϥΠϒϥ৘ใྔ
    
    ্ʹೋͭͷ֬཰෼෍
    
    ͕͋Δͱ͖ ℝN q(x), p(x) D(p∥q) = ∫ q(x)log q(x) p(x) dx ͷ͜ͱΛΧϧόοΫɾϥΠϒϥ৘ใྔ͋Δ͍͸૬ରΤϯτϩϐʔͱݺͿ ΧϧόοΫɾϥΠϒϥ৘ใྔ͸͕࣍੒Γཱͭɻ
    
    ʹ͍ͭͯ
    
    Ͱ͋Δɻ
    
    
    ͱͳΔͷ͸
    ͷͱ͖ʹݶΔɻ ∀q(x), p(x) D(q∥p) ≥ 0 D(q∥p) = 0 q(x) = p(x)
16. ΧϧόοΫɾϥΠϒϥ৘ใྔ
     ূ໌ ͱ͓͘ͱɺ
    
    Ͱ͋Γɺ F(t) = 0 ⇔ t = 0 F(t) = t + et − 1 (−∞ < t < ∞)
    
    ΑΓ
    Ͱ͋Δ͔Β
    
    ͕ࣔ͞Εͨɻ ∫ q(x)dx = 1 ∫ p(x)dx = 1 ∫ log q(x) p(x) dx = 0 ·ͨɺ
    ͷͱ͖ɺ
    Ͱ
    
    Ͱ͋Δ͜ͱΛ༻͍ͯ
    q(x) ≈ p(x) t ≈ 0 F′ ′ (t) ≃ t2/e D(p∥q) ≃ ∫ q(x)(log q(x) − log p(x))2dx ͕੒Γཱͭɻ 
17. 
    ۃݶఆཧ
18. ֬཰ม਺ͷऩଋ
19. ֬཰ऩଋ ֬཰ม਺
    
    ͕ఆ਺
    
    ʹ֬཰ऩଋ͢Δͱ͸
    
    ʹର͠ɺ
    ʹ͓͍ͯ {Xn }n∈ℕ c ∀ϵ, ∀δ > 0 ∃N ∈ ℕ n > N ⇒ P(∥Xn − c∥ > ϵ) < δ ⇔ P(∥Xn − c∥ < ϵ) = 1 ͱͳΔ͜ͱͰ͋Δɻ
    ͜Ε͸େ਺ͷऑ๏ଇʹରԠ͍ͯ͠Δɻ Xn c ϵ ඪຊ͕े෼ʹେ͖͍ͱ͖ɺඪຊฏۉ͸฼ฏۉʹऩଋ͢Δ
20. ๏ଇ ෼෍ ऩଋ ֬཰ม਺ͷྻ
    
    ͕֬཰ม਺
    
    ʹ๏ଇ ෼෍ ऩଋ͢Δͱ͸
    
    ͷ֬཰෼෍͕
    
    Ͱ
    
    ͷ֬཰෼෍͕
    
    Ͱ͋Δͱ͖ɺ
    ೚ҙͷ༗ք͔ͭ࿈ଓͳؔ਺
    
    ʹରͯ͠ {Xn }n∈ℕ X Xn qn (x) X q(x) F(x) lim n→∞ ∫ F(x)qn (x)dx = ∫ F(x)q(x)dx ⇔ lim n→∞ [F(Xn )] = [F(X)] ͕੒Γཱͭ͜ͱͰ͋Δɻ͜Ε͸த৺ۃݶఆཧʹରԠ͍ͯ͠Δɻ ඪຊ͕े෼ʹେ͖͍ͱ͖ɺ฼ूஂͷ෼෍ʹؔΘΒͣඪຊฏۉͱ฼ฏۉͷࠩ͸ਖ਼ن෼෍ʹै͏
21. ܦݧաఔ
22. ϢʔΫϦουۭؒʹ͓͚ΔίϯύΫτੑ ϢʔΫϦουۭؒ
    
    ͷ෦෼ू߹
    
    ͕։ू߹ͷ଒
    
    ʹ
    ͍ͭͯ
    
    ͳΒ͹ɺͦͷ༗ݶݸͷ։ू߹
    
    Ͱ
    ℝN W = {O}λ∈Λ W ⊂ ⋃ λ∈Λ Oλ O1 , …, On ∈
    ͱͳΔ΋ͷ͕͋Δͱ͖ɺ
    ͸ίϯύΫτͰ͋Δͱ͍͏ W ⊂ O1 ∪ … ∪ On W O1 , …, On ∈ W
23. ؔ਺্ۭؒͷେ਺ͷ๏ଇ ϢʔΫϦουۭؒ
    
    ʹ஋ΛऔΔ
    
    ͕֬཰ม਺
    
    ͱ
    ಉ֬͡཰෼෍ʹै͏ͱ͢Δɻ
    ύϥϝʔλͷू߹
    
    ΛίϯύΫτͱ͢Δɻ
    ℝN X1 , X2 , …, Xn X w ∈ W ∈ ℝN f(x, w) : ℝN → ℝ1 X [ sup w∈W |f(X, w)|] < ∞, X [ sup w∈W |∇w f(X, w)|] < ∞ ৚݅ ͕੒ΓཱͭͱԾఆ͢Δɻ͜ͷͱ͖ɺ
    ʹ͍ͭͯ ∀ϵ > 0 P( sup w∈W 1 n n ∑ i=1 f(Xi , w) − X [f(X, w)] < ϵ) = 1 ͜ͷ͜ͱΛؔ਺্ۭؒͷେ਺ͷ๏ଇͱ͍͏
24. ਖ਼ن֬཰աఔ ू߹
    
    ্ͷؔ਺Ͱ֬཰తʹมಈ͢Δ΋ͷ
    
    ͕ɺ
    ฏۉؔ਺
    
    ͱ૬ؔؔ਺
    
    Λ࣋ͭਖ਼ن֬཰աఔͰ͋Δͱ͸ɺ
    ֤
    
    ͝ͱʹ
    
    ͕ਖ਼ن෼෍ʹै͏֬཰ม਺Ͱ͋Γɺ W ξ(w) m(w) ρ(w, w′ ) w ξ(w)
    
    m(w) = ξ [ξ(w)],
    
    ρ(w, w′ ) = ξ [ξ(w)ξ(w′ )] ͕੒Γཱͭ͜ͱͰ͋Δɻ͜͜Ͱ
    
    ͸ɺ֬཰աఔ
    
    ʹ͍ͭͯͷฏۉΛ ද͍ͯ͠ΔɻίϯύΫτू߹্Ͱͷਖ਼ن֬཰աఔ͸ɺ ξ [ ⋅ ] ξ ฏۉؔ਺ͱ૬ؔؔ਺͕ܾ·ΔͱҰҙʹఆ·Δ͜ͱ͕஌ΒΕ͍ͯΔɻ
25. ܦݧաఔ  ͭ͗ʹ X[ sup w∈W |f(X, w) − X [f(X, w)]|α ] < ∞ X[ sup w∈W |∇w (f(X, w) − X [f(X, w)])|α ] < ∞ ͕
    
    Ͱ੒ΓཱͭͱԾఆ͢Δɻ α = 2 Yn (w) = 1 n n ∑ i=1 (f(Xi , w) − X [f(X, w)]) ͜ͷ֬཰աఔ
    
    Λܦݧաఔͱ͍͏ɻ Yn (w)
26. ܦݧաఔ  ֬཰աఔ ܦݧաఔ
    
    ͸ฏۉ͕
    
    Ͱ૬ؔؔ਺͕ Yn (w) 0 ͷਖ਼ن֬཰աఔ
    
    ʹ๏ଇऩଋ͢Δɻ Y(w)
    
    ρ(w, w′ ) = X [f(X, w)f(X, w′ )] − X [f(X, w)]X [f(X, w′ )]
27. ֬཰աఔͷ๏ଇऩଋ ֬཰աఔ ܦݧաఔ
    
    ͕֬཰աఔ
    
    ʹ๏ଇऩଋ͢Δͱ͸ɺ
    ༗ք࿈ଓͳ൚ؔ਺
    
    ʹ͍ͭͯ Yn (w) Y(w) F( ⋅ ) ͕੒Γཱͭͱ͍͏͜ͱͰ͋Δɻͳ͓ɺ൚ؔ਺
    
    ͕࿈ଓͰ͋Δͱ͸ F( ⋅ )
    
    lim n→∞ [F(Yn )] = Y [F(Y)]
    
    lim n→∞ sup w∈W |fn (w) − f(w)| → 0 ⇒ lim n→∞ F(fn ) = F(f ) ͕੒Γཱͭ͜ͱͰ͋Δɻ
    ͜ͷΑ͏ͳܗͷఆཧΛؔ਺্ۭؒͷத৺ۃݶఆཧͱ͍͏ɻ
28. ࢀߟࢿྉ w ֬཰࿦ೖ໳
    ౉ล੅෉
    w ܦݧաఔͱ͸
    ౉ล੅෉
    w ϕΠζ౷ܭͷཧ࿦ͱํ๏
    ౉ล੅෉
    w ଌ౓ɾ֬཰ɾϧϕʔάੵ෼
    ݪܒհ