初等確率論の基礎

「ベイズ統計の理論と方法」勉強会の資料

Transcript

1. ॳ౳֬཰࿦ͷجૅ
   ϕΠζ౷ܭͷཧ࿦ͱํ๏ษڧձ
   

   View Slide

2. ֬཰෼෍ͱ֬཰ม਺
   

   View Slide

3. ֬཰෼෍
   ϢʔΫϦουۭؒ ͷݩ ͷؔ਺ ͕
   ℝN x = (x1
   , …, xN
   ) q(x) ≥ 0
   ∫
   q(x)dx ≡
   ∫
   dx1 ∫
   dx2
   …
   ∫
   dxN
   q(x1
   , x2
   , ⋯, xN
   ) = 1
   Λຬͨ͢ͱ͖ Λ֬཰෼෍͋Δ͍͸֬཰ີ౓ؔ਺ͱ͍͏ɻ
   q(x)
   ू߹ ʹ͍ͭͯɺ ͷݩͰͷू߹ ͷ֬཰͸
   A ⊂ ℝN q(x) A
   Q(A) =
   ∫
   A
   q(x)dx
   ͜ͷͱ͖ɺؔ਺ ΋֬཰෼෍ͱ͍͏ɻ
   Q( ⋅ )
   

   View Slide

4. ֬཰ม਺
   ϢʔΫϦουۭؒ ͷ্ʹϥϯμϜʹ஋ΛऔΔม਺ Λ
   ʮ ʹ஋ΛऔΔ֬཰ม਺ʯͱ͍͏ɻ
   ℝN X
   ℝN
   ʮ ͱͳΔ֬཰ʯ͕ Ͱ͋Δͱ͖
   ʮ֬཰ม਺ ͷ֬཰෼෍͸ Ͱ͋Δʯ͋Δ͍͸
   ʮ֬཰ม਺ ͷ֬཰෼෍͸ ʹै͏ʯ͋Δ͍͸
   ʮ֬཰ม਺ ͷ֬཰෼෍͸ Ͱ͋Δʯͱ͍͏ɻ
   X ∈ A Q(A)
   X q(x)
   X q(x)
   X Q
   

   View Slide

5. ۩ମྫਅͷ෼෍
   αϯϓϧ ͕͋Δ֬཰෼෍ ʹಠཱʹै͏
   ֬཰ม਺ͷ࣮ݱ஋ʢ؍ଌ஋ʣͩͱ͢Δɻ
   A = xn = {x1
   , …, xn
   } ⊂ ℝN q(x)
   ͢ͳΘͪ Λ ্ͷ෼෍
   xn (ℝN)n
   q(xn) =
   n
   ∏
   i=1
   q(xi
   ) = q(x1
   )q(x2
   )⋯q(xn
   )
   Λ࣋ͭ֬཰ม਺ ͷ࣮ݱ஋Ͱ͋Δͱߟ͑Δɻ
   ͜ͷͱ͖֬཰෼෍ Λਅͷ෼෍ͱݺͿɻ
   Xn = (X1
   , X2
   , …, Xn
   )
   q(x)
   

   View Slide

6. ฏۉͱ෼ࢄ
   

   View Slide

7. ฏۉͱ෼ࢄ
   ʹ஋ΛͱΔ֬཰ม਺ ͷ֬཰෼෍Λ ͱ͢Δɻ
   ℝN X q(x)
   [f(X)] ≡
   ∫
   f(x)q(x)dx
   [f(X)] ≡ [(f(X) − [f(X)])(f(X) − [f(X)])T]
   = [f(X)f(X)T] − [f(X)][f(X)T]
   ͱఆٛ͢Δɻ
   ͕༩͑ΒΕͨͱ͖ɺ֬཰ม਺ ͷฏۉΛ
   f : ℝN → ℝM f(X)
   ·ͨ෼ࢄڞ෼ࢄΛ
   ͱఆٛ͢Δɻ֬཰ม਺Λ໌ه͍ͨ͠ͱ͖͸ ͱॻ͘ɻ
   X
   [f(X)]
   

   View Slide

8. ۩ମྫαϯϓϧͷฏۉ஋
   αϯϓϧ Λද֬͢཰ม਺Λ ͱ͢Δɻ
   ͦͷؔ਺ ͕༩͑ΒΕͨͱ͖ɺͦͷฏۉ஋ΛऔΔૢ࡞ Λ
   xn = {x1
   , …, xn
   } Xn = (X1
   , X2
   , …, Xn
   )
   f(Xn) [ ⋅ ]
   ͱදه͢Δɻ
   
   ͜ͷฏۉ஋ ΛʮαϯϓϧͷݱΕํʹର͢Δฏۉ஋ʯͱݺͿɻ
   [ ⋅ ]
   [f(XN)] =
   ∫ ∫
   ⋯
   ∫
   f(x1
   , …, xn
   )
   n
   ∏
   i=1
   q(xi
   )dxi
   

   View Slide

9. ۩ମྫਅͷ෼෍ͷฏۉ
   αϯϓϧͷ֬཰ม਺Λ Λ༻͍ͯɺ
   ਅͷ෼෍ ͷਪଌΛߦͬͨޙɺਅͷ෼෍ͷ֬཰ม਺ Λൃੜͤͯ͞
   ਪଌ݁ՌͷΑ͞ΛධՁ͍ͨ͠ɻ
   ͜ͷ֬཰ม਺ ͷؔ਺ ʹ͍ͭͯͷฏۉΛ
   Xn = (X1
   , X2
   , …, Xn
   )
   q(x) X
   X f(X)
   ͱදه͢Δɻ
   [f(X)]X
   =
   ∫
   f(x)q(x)dx
   

   View Slide

10. X
    X−1
    ֬཰ۭؒ(Ω = ℝM, ℬ, p)
    w ∈ Ω
    ٯ૾X−1(A)
    ֬཰ີ౓ؔ਺ ֬཰෼෍
    q(x) = p(X−1(x))
    Մଌۭؒ(Ω′ = ℝN, ℬ′ )
    A ∈ ℬ′
    X(w) = X
    x ∈ Ω′
    ֬཰෼෍Q(A) =
    ∫
    A
    q(x)dx
    f(x)
    ฏۉ[f(X)] ≡
    ∫
    f(x)q(x)dx =
    ∫
    f(x)p(X−1(x))dx =
    ∫
    p(w)X(w)dw =
    ∫
    pXdw
    ֬཰ม਺
    ֬཰ม਺ͱ֬཰෼෍ɺฏۉͷؔ܎
    ֬཰ۭؒ(Ω′ = ℝN, ℬ′ , q)
    

    View Slide

11. ಉ࣌෼෍ͱ৚݅෇͖֬཰
    

    View Slide

12. ಉ࣌෼෍ͱ৚݅෇͖෼෍
    ͭͷ֬཰ม਺ ͱ ͕͋Δͱ͖ɺͦͷ૊ ͷ֬཰෼෍͕
    Ͱ͋Δͱ͖ɺ Λಉ࣌֬཰෼෍ͱ͍͏ɻ
    X Y (X, Y)
    p(x, y) p(x, y)
    ·ͨ֬཰ม਺ ͕༩͑ΒΕͨͱ͖ͷ ͷ৚݅෇͖֬཰෼෍Λ࣍ͷΑ͏
    ʹఆٛ͢Δɻ
    X Y
    p(y|x) =
    p(x, y)
    p(x)
    पล֬཰෼෍͸࣍ͷΑ͏ʹఆٛ͢Δɻ
    p(x) =
    ∫
    p(x, y)dy p(y) =
    ∫
    p(x, y)dx
    

    View Slide

13. ճؼؔ਺
    ֬཰ม਺ ͷ֬཰෼෍ ʹ͍ͭͯߟ͑Δɻ
    ͷͱ͖ͷ ͷฏۉ஋Λ
    (X, Y) p(X, Y)
    X = x Y
    ͱॻ͘ɻ͜ͷؔ਺Λ ͔Β ͷճؼؔ਺ ৚݅෇͖ظ଴஋
    ͱ͍͏ɻ
    x y
    [Y|x] =
    ∫
    yp(y|x)dy
    ؔ਺Λ ͕༩͑ΒΕͨͱ͖ͦͷೋ৐ޡࠩΛද͢൚ؔ਺Λ
    y = f(x)
    [(Y − f(X))2] =
    ∫ ∫
    (y − f(x))2p(y, x)dxdy
    ͱॻ͘ͱ͜Ε͸ ͷͱ͖ʹ࠷খʹͳΔɻ
    f(x) = [Y|x]
    

    View Slide

14. ΧϧόοΫɾϥΠϒϥ৘ใྔ
    

    View Slide

15. ΧϧόοΫɾϥΠϒϥ৘ใྔ
    
    ্ʹೋͭͷ֬཰෼෍ ͕͋Δͱ͖
    ℝN q(x), p(x)
    D(p∥q) =
    ∫
    q(x)log
    q(x)
    p(x)
    dx
    ͷ͜ͱΛΧϧόοΫɾϥΠϒϥ৘ใྔ͋Δ͍͸૬ରΤϯτϩϐʔͱݺͿ
    ΧϧόοΫɾϥΠϒϥ৘ใྔ͸͕࣍੒Γཱͭɻ
    ʹ͍ͭͯ Ͱ͋Δɻ
    ͱͳΔͷ͸ ͷͱ͖ʹݶΔɻ
    ∀q(x), p(x) D(q∥p) ≥ 0
    D(q∥p) = 0 q(x) = p(x)
    

    View Slide

16. ΧϧόοΫɾϥΠϒϥ৘ใྔ
    
    ূ໌
    
    ͱ͓͘ͱɺ Ͱ͋Γɺ
    F(t) = 0 ⇔ t = 0
    F(t) = t + et − 1 (−∞ < t < ∞)
    ΑΓ Ͱ͋Δ͔Β͕ࣔ͞Εͨɻ
    ∫
    q(x)dx = 1
    ∫
    p(x)dx = 1
    ∫
    log
    q(x)
    p(x)
    dx = 0
    ·ͨɺ ͷͱ͖ɺ Ͱ Ͱ͋Δ͜ͱΛ༻͍ͯ
    q(x) ≈ p(x) t ≈ 0 F′ ′ (t) ≃ t2/e
    D(p∥q) ≃
    ∫
    q(x)(log q(x) − log p(x))2dx
    ͕੒Γཱͭɻ
    
    

    View Slide

17. ۃݶఆཧ
    

    View Slide

18. ֬཰ม਺ͷऩଋ
    

    View Slide

19. ֬཰ऩଋ
    ֬཰ม਺ ͕ఆ਺ ʹ֬཰ऩଋ͢Δͱ͸
    ʹର͠ɺ ʹ͓͍ͯ
    {Xn
    }n∈ℕ
    c
    ∀ϵ, ∀δ > 0 ∃N ∈ ℕ
    n > N ⇒ P(∥Xn
    − c∥ > ϵ) < δ
    ⇔ P(∥Xn
    − c∥ < ϵ) = 1
    ͱͳΔ͜ͱͰ͋Δɻ
    ͜Ε͸େ਺ͷऑ๏ଇʹରԠ͍ͯ͠Δɻ
    Xn
    c
    ϵ
    ඪຊ͕े෼ʹେ͖͍ͱ͖ɺඪຊฏۉ͸฼ฏۉʹऩଋ͢Δ
    

    View Slide

20. ๏ଇ ෼෍
    ऩଋ
    ֬཰ม਺ͷྻ ͕֬཰ม਺ ʹ๏ଇ ෼෍
    ऩଋ͢Δͱ͸
    ͷ֬཰෼෍͕ Ͱ ͷ֬཰෼෍͕ Ͱ͋Δͱ͖ɺ
    ೚ҙͷ༗ք͔ͭ࿈ଓͳؔ਺ ʹରͯ͠
    {Xn
    }n∈ℕ
    X
    Xn
    qn
    (x) X q(x)
    F(x)
    lim
    n→∞
    ∫
    F(x)qn
    (x)dx =
    ∫
    F(x)q(x)dx
    ⇔ lim
    n→∞
    [F(Xn
    )] = [F(X)]
    ͕੒Γཱͭ͜ͱͰ͋Δɻ͜Ε͸த৺ۃݶఆཧʹରԠ͍ͯ͠Δɻ
    ඪຊ͕े෼ʹେ͖͍ͱ͖ɺ฼ूஂͷ෼෍ʹؔΘΒͣඪຊฏۉͱ฼ฏۉͷࠩ͸ਖ਼ن෼෍ʹै͏
    

    View Slide

21. ܦݧաఔ
    

    View Slide

22. ϢʔΫϦουۭؒʹ͓͚ΔίϯύΫτੑ
    ϢʔΫϦουۭؒ ͷ෦෼ू߹ ͕։ू߹ͷ଒ ʹ
    ͍ͭͯ ͳΒ͹ɺͦͷ༗ݶݸͷ։ू߹ Ͱ
    ℝN W = {O}λ∈Λ
    W ⊂ ⋃
    λ∈Λ
    Oλ
    O1
    , …, On
    ∈
    ͱͳΔ΋ͷ͕͋Δͱ͖ɺ ͸ίϯύΫτͰ͋Δͱ͍͏
    W ⊂ O1
    ∪ … ∪ On
    W
    O1
    , …, On
    ∈
    W
    

    View Slide

23. ؔ਺্ۭؒͷେ਺ͷ๏ଇ
    ϢʔΫϦουۭؒ ʹ஋ΛऔΔ ͕֬཰ม਺ ͱ
    ಉ֬͡཰෼෍ʹै͏ͱ͢Δɻ
    ύϥϝʔλͷू߹ ΛίϯύΫτͱ͢Δɻ
    ℝN X1
    , X2
    , …, Xn
    X
    w ∈ W ∈ ℝN f(x, w) : ℝN → ℝ1
    X
    [ sup
    w∈W
    |f(X, w)|] < ∞, X
    [ sup
    w∈W
    |∇w
    f(X, w)|] < ∞
    ৚݅
    ͕੒ΓཱͭͱԾఆ͢Δɻ͜ͷͱ͖ɺ ʹ͍ͭͯ
    ∀ϵ > 0
    P( sup
    w∈W
    1
    n
    n
    ∑
    i=1
    f(Xi
    , w) − X
    [f(X, w)] < ϵ) = 1
    ͜ͷ͜ͱΛؔ਺্ۭؒͷେ਺ͷ๏ଇͱ͍͏
    

    View Slide

24. ਖ਼ن֬཰աఔ
    ू߹ ্ͷؔ਺Ͱ֬཰తʹมಈ͢Δ΋ͷ ͕ɺ
    ฏۉؔ਺ ͱ૬ؔؔ਺ Λ࣋ͭਖ਼ن֬཰աఔͰ͋Δͱ͸ɺ
    ֤ ͝ͱʹ ͕ਖ਼ن෼෍ʹै͏֬཰ม਺Ͱ͋Γɺ
    W ξ(w)
    m(w) ρ(w, w′ )
    w ξ(w)
    
    m(w) = ξ
    [ξ(w)],
    ρ(w, w′ ) = ξ
    [ξ(w)ξ(w′ )]
    ͕੒Γཱͭ͜ͱͰ͋Δɻ͜͜Ͱ ͸ɺ֬཰աఔ ʹ͍ͭͯͷฏۉΛ
    ද͍ͯ͠ΔɻίϯύΫτू߹্Ͱͷਖ਼ن֬཰աఔ͸ɺ
    ξ
    [ ⋅ ] ξ
    ฏۉؔ਺ͱ૬ؔؔ਺͕ܾ·ΔͱҰҙʹఆ·Δ͜ͱ͕஌ΒΕ͍ͯΔɻ
    

    View Slide

25. ܦݧաఔ
    
    ͭ͗ʹ
    X[ sup
    w∈W
    |f(X, w) − X
    [f(X, w)]|α
    ] < ∞
    X[ sup
    w∈W
    |∇w
    (f(X, w) − X
    [f(X, w)])|α
    ] < ∞
    ͕ Ͱ੒ΓཱͭͱԾఆ͢Δɻ
    α = 2
    Yn
    (w) =
    1
    n
    n
    ∑
    i=1
    (f(Xi
    , w) − X
    [f(X, w)])
    ͜ͷ֬཰աఔ Λܦݧաఔͱ͍͏ɻ
    Yn
    (w)
    

    View Slide

26. ܦݧաఔ
    
    ֬཰աఔ ܦݧաఔ
    ͸ฏۉ͕ Ͱ૬ؔؔ਺͕
    Yn
    (w) 0
    ͷਖ਼ن֬཰աఔ ʹ๏ଇऩଋ͢Δɻ
    Y(w)
    
    ρ(w, w′ ) = X
    [f(X, w)f(X, w′ )] − X
    [f(X, w)]X
    [f(X, w′ )]
    

    View Slide

27. ֬཰աఔͷ๏ଇऩଋ
    ֬཰աఔ ܦݧաఔ
    ͕֬཰աఔ ʹ๏ଇऩଋ͢Δͱ͸ɺ
    ༗ք࿈ଓͳ൚ؔ਺ ʹ͍ͭͯ
    Yn
    (w) Y(w)
    F( ⋅ )
    ͕੒Γཱͭͱ͍͏͜ͱͰ͋Δɻͳ͓ɺ൚ؔ਺ ͕࿈ଓͰ͋Δͱ͸
    F( ⋅ )
    
    lim
    n→∞
    [F(Yn
    )] = Y
    [F(Y)]
    
    lim
    n→∞
    sup
    w∈W
    |fn
    (w) − f(w)| → 0 ⇒ lim
    n→∞
    F(fn
    ) = F(f )
    ͕੒Γཱͭ͜ͱͰ͋Δɻ
    ͜ͷΑ͏ͳܗͷఆཧΛؔ਺্ۭؒͷத৺ۃݶఆཧͱ͍͏ɻ
    

    View Slide

28. ࢀߟࢿྉ
    w ֬཰࿦ೖ໳ ౉ล੅෉
    w ܦݧաఔͱ͸ ౉ล੅෉
    w ϕΠζ౷ܭͷཧ࿦ͱํ๏ ౉ล੅෉
    w ଌ౓ɾ֬཰ɾϧϕʔάੵ෼ ݪܒհ
    

    View Slide