Upgrade to Pro — share decks privately, control downloads, hide ads and more …

確率・統計の基礎勉強会1

setten-QB
November 09, 2016

 確率・統計の基礎勉強会1

研究室M1用に行った,確率・統計の基礎勉強会(第1回)のスライドです.

setten-QB

November 09, 2016
Tweet

More Decks by setten-QB

Other Decks in Science

Transcript

  1. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ୈ 1 ճ ֬཰ɾ౷ܭͷجૅษڧձ Hiroaki Tanaka

    Augmented Human Communication Laboratory, Department of Information Schience, Nara Institute of Science and Technology November 8, 2016 1 / 42
  2. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ਺ཧ౷ܭֶͷجૅ಺༰ ֬཰ۭؒ શۭؒͷଌ౓͕ 1 ͱͳΔΑ͏ͳଌ౓ۭؒ ֬཰ม਺ɾ֬཰෼෍

    PRML ͷ 2ɾ3 ষͰग़ͯ͘Δ ֬཰෼෍ͷಛੑ஋ PRML ͷ 2ɾ3 ষͰग़ͯ͘Δ ۙࣅ๏ଇ n → ∞, d :fix ͷ৔߹͕جຊత ౷ܭతਪଌ ࢖͏͚ͩͳΒ೉͘͠ͳ͍ ౷ܭతݕఆ ࢖͏͚ͩͳΒ೉͘͠ͳ͍ 6 / 42
  3. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ͜ͷษڧձͰѻ͏಺༰ ֬཰ۭؒ ୈ 1 ճ ֬཰ม਺ɾ֬཰෼෍

    ୈ 1 ճ ֬཰෼෍ͷಛੑ஋ ୈ 2 ճ ۙࣅ๏ଇ ୈ 2 ճ ౷ܭతਪఆ ୈ 3 ճ ౷ܭతݕఆ ୈ 4 ճ ଟมྔ෼෍Λѻ͏ͨΊʹඞཁͳઢܗ୅਺ ୈ 5 ճʁ 8 / 42
  4. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ σ ू߹଒ • ҎԼɼඪຊۭؒΛ Ω Ͱද͢

    • ֬཰͸ɼΩ ্ͷಛผͳ৚݅Λຬͨ͢෦෼ू߹଒্Ͱఆٛ͞ΕΔ σ-algebra Ω ͷ෦෼ू߹଒ B ͕ҎԼͷ৚݅Λຬͨ࣌͢ɼB Λ Ω ্ͷ σ ू߹଒ͱ͍͏: Ω ∈ B; A ∈ B =⇒ Ac ∈ B; A1, . . . , An, · · · ∈ B =⇒ ∞ ∪ i=1 Ai ∈ B. • ֬཰࿦Ͱ͸ σ ू߹଒ͷཁૉΛࣄ৅ͱ͍͏ • Ω Λશࣄ৅ͱ͍͏ 11 / 42
  5. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ σ ू߹଒ • σ ू߹଒͕ຬͨ͢ੑ࣭͸ͨ͘͞Μ͋Δ͕ɼৄࡉ͸ུ •

    σ ू߹଒্Ͱɼ֬཰͸ఆٛ͞ΕΔ • Ω ͱ B ͷରΛ (Ω, B) ͱॻ͍ͯɼՄଌۭؒ (measurable space) ͱݺͿ 12 / 42
  6. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ֬཰ଌ౓ Probabilitstic Measure B ্ͷ࣮਺஋ؔ਺ P

    ͕ 0 ≤ P(A) ≤ 1, ∀A ∈ B, P(Ω) = 1, Ai ∈ B(i = 1, . . . ), Ai ∩ Aj = ∅(i ̸= j) =⇒ P ( ∞ ∪ i=1 Ai ) = ∞ ∑ i=1 P(Ai) Λຬͨ͢ͱ͖ɼP Λ (Ω, B) ্ͷ֬཰ଌ౓ (probabilistic measure) ͱ͍͍ɼP(A) Λࣄ৅ A ͷ֬཰ͱ͍͏ɽ • Ω, B, P ͷ 3 ͭ૊ (Ω, B, P) Λ֬཰ۭؒͱ͍͏ɽ 13 / 42
  7. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ֬཰ଌ౓ • ͜͏΍ͬͯ֬཰Λఆٛ͢Δ͜ͱʹΑΓɼօ͕஌͍ͬͯΔΑ͏ͳ ֬཰ͷੑ࣭ P(∅) =

    0 ΍ P(A) + P(Ac) = 1, for A ∈ B ͕੒Γ ཱͭ • ͦͷଞʹ΋ͨ͘͞Μͷॏཁͳੑ࣭͕͋Δ͕ɼৄࡉ͸ུ 14 / 42
  8. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ͦ΋ͦ΋ଌ౓ͬͯԿʁ • ू߹ͷେ͖͞ΛଌΔई౓ → ଌ౓ •

    P(A1) Ͱ A1 ͷେ͖͞Λද͢ → ֬཰ • ֬཰ۭؒΛ໌Β͔ʹ͍ͯ͠ͳ͍ͱࠔΔ͜ͱ͕͋Δ • Ͳ͏͍͏ू߹্ʹ֬཰ΛೖΕΔͷʁ → σ ू߹଒ ଌ౓ R ͔Β࣮਺ a, b Λબͼɼ۠ؒ I = [a, b], a, b ∈ [0, 1] Λߏ੒ ͢Δɽ۠ؒ I ͷ “େ͖͞” ʹ͍ͭͯߟ͑Δɽ a b x M 16 / 42
  9. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ͦ΋ͦ΋ଌ౓ͬͯԿʁ • ू߹ͷେ͖͞ΛଌΔई౓ → ଌ౓ •

    P(A1) Ͱ A1 ͷେ͖͞Λද͢ → ֬཰ • ֬཰ۭؒΛ໌Β͔ʹ͍ͯ͠ͳ͍ͱࠔΔ͜ͱ͕͋Δ • Ͳ͏͍͏ू߹্ʹ֬཰ΛೖΕΔͷʁ → σ ू߹଒ ଌ౓ R ͔Β࣮਺ a, b Λબͼɼ۠ؒ I = [a, b], a, b ∈ [0, 1] Λߏ੒ ͢Δɽ۠ؒ I ͷ “େ͖͞” ʹ͍ͭͯߟ͑Δɽ a b x M ϧϕʔάଌ౓ L([a, b]) b − a 16 / 42
  10. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ৚݅෇͖֬཰ • ৚݅෇͖֬཰΋ଌ౓ۭؒ (Ω, B) ্Ͱఆٛ͞ΕΔ

    Conditional Probability C ∈ B, P(C) > 0 ͷͱ͖ɼ P(A|C) = P(A ∩ C) P(C) , A ∈ B Λࣄ৅ C ͕༩͑ΒΕͨͱ͖ͷ A ∈ B ͷ৚݅෇͖֬཰ͱ͍ɼ P( · |C) Λ C ͕༩͑ΒΕͨͱ͖ͷ৚݅෇͖֬཰ଌ౓ͱ͍͏ɽ • ্هͷΑ͏ʹ৚݅෇͖֬཰Λఆٛ͢ΔͱɼP( · |C) ͸ (Ω, B) ্ͷ֬཰ଌ౓ʹͳ͍ͬͯΔ 17 / 42
  11. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ৚݅෇͖֬཰ • Α͘࢖͏શ֬཰ͷެࣜ શ֬཰ͷެࣜ ∞ ∪

    i=1 Ai = Ω, Ai ∩ Aj = ∅, P(Ai) > 0 ͱ͢Δɽ͜ͷͱ͖ɼ A ∈ B ʹ͍ͭͯ P(A) = ∞ ∑ i=1 P(Ai)P(A|Ai) ͕੒Γཱͭɽ 18 / 42
  12. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ϘϨϧू߹଒ • ֬཰ۭؒ (Ω, B, P)

    Λ࣮਺ۭؒ΁݁ͼ͚ͭΔ΋ͷ • ࣮਺ۭؒͰ͸ɼ(Ω, B) ͕ (R, B) ʹରԠ͢Δ Borel algebra શͯͷ൒։۠ؒ (a, b] ΛؚΉ࠷খͷ σ ू߹଒ B Λ R ্ͷϘ Ϩϧू߹଒ͱ͍͏ɽ • B ͸ (a, b] Ҏ֎ʹ΋༷ʑͳ۠ؒΛؚΉ • { a } , (a, b), [a, b], [a, b), (a, ∞), (−∞, a] ͸શͯϘϨϧू߹ 20 / 42
  13. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ֬཰ม਺ • ֬཰ม਺ͷݫີͳఆٛΛ͢Δ • ֬཰ม਺͕ɼந৅తͳ֬཰ۭؒͱ࣮਺ͷ֬཰ۭؒΛରԠ෇͚Δ ֬཰ม਺

    X : Ω → R ͕ (Ω, B) ্ͷ֬཰ม਺ def ⇐⇒∀B ∈ B, { ω | X(ω) ∈ B } = X−1(B) ∈ B. • X Λ֬཰ม਺ͱ͢Δͱ͖ɼଟ߲ࣜɼlog |X|, eX, sin X ͳͲ͸ શͯ֬཰ม਺ͱͳΔ 21 / 42
  14. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ֬཰ม਺ 1 ∈ 2 ∈ −1(1

    ) −1(2 ) Measurable function Measurable function Figure: ֬཰ม਺ͷΠϝʔδ • ͲͷΑ͏ͳ B ∈ B ΛબΜͰ΋ɼX−1(B) ∈ B Մଌؔ਺ X 22 / 42
  15. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ෼෍ؔ਺ • (Ω, B) ͷ࣮਺ۭؒ൛ͱͯ͠ (R,

    B) ͕ಋೖ͞Εͨ • ࢒Γͷ֬཰ଌ౓ P ʹରԠ͢Δ΋ͷͱͯ͠ɼ෼෍ؔ਺ F Λಋೖ ͢Δ Distribution Function F : R → R ͕ ∀x ∈ R, F(x) = P(X ≤ x) = P ({ ω | X(ω) ≤ x }) ʹΑͬͯ F Λఆٛ͢Δͱ͖ɼF Λ X ͷྦྷੵ෼෍ؔ਺ (cumu- lative distribution function; cdf) ͱ͍͏ɽ • P(X ≤ x) Λ X ≤ x ͱͳΔ֬཰ͱಡΈସ͑Δࣄ͕Ͱ͖Δ 23 / 42
  16. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ͜͜·Ͱͷ·ͱΊ • ֬཰͸Մଌ্ۭؒͰఆٛ͞ΕΔ࣮਺஋ؔ਺ • ֬཰ม਺ X

    ͕ɼ֬཰ۭؒͱ࣮਺ۭؒΛͭͳ͙ • ֬཰ม਺͸࣮͸ม਺Ͱ͸ͳ͘Մଌؔ਺ • ࣮਺ۭؒͰ͸֬཰ม਺͸ม਺ͷΑ͏ʹѻͬͯ΋ࠩ͠ࢧ͑ͳ͍ 25 / 42
  17. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ֬཰ม਺ͷλΠϓ Random Variable Discrete type X

    : (Ω, B, P) ্ͷ r.v. ͱ͢Δɽ X ͕ D (Discrete) -type def ⇐⇒∃E ⊂ R (ߴʑՃࢉͳू߹) s.t. P(X ∈ E) = 1 Continuous type FX ɿX ͷ෼෍ؔ਺ͱ͢Δɽ X ͕ C (Continuous) -type def ⇐⇒∃fX : R → R+ (Մଌؔ਺) s.t. FX (x) = ∫ x −∞ fX (u) du, ∀x ∈ R 26 / 42
  18. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ֬཰ม਺ͷλΠϓ • ؆୯ʹݴ͏ͱ . . .

    D-type औΓ͏Δ஋͕཭ࢄ஋ (E ⊂ N, Z, Q) C-type औΓ͏Δ஋͕࿈ଓ஋ (E ⊂ R) 27 / 42
  19. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ཭ࢄܕ֬཰ม਺ • X ͷऔΓ͏Δ஋͕཭ࢄ஋ͷͱ͖͸ D-type •

    E = { x1, . . . , xn, . . . } , p(xi) = P(X = xi) ͱ͢Δͱɼ ∞ ∑ i=1 p(xi) = 1, F(x) = ∑ xi ≤ xp(xi) ͕੒Γཱͭɽ͜ͷ p(·) ͷ͜ͱΛ X ͷ֬཰࣭ྔؔ਺ (p.m.f.) ͱ ݺͿ 28 / 42
  20. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ཭ࢄܕ֬཰ม਺ͱͦͷ෼෍ͷྫ 2 ߲෼෍ B(n, p) •

    0 ≤ p ≤ 1, i = 1, . . . , n p(xi) = P(X = i) = ( n i ) pi(1 − p)n−i • E = { 1, . . . , n } ͱ͢ΔͱɼP(X ∈ E) = 1 ͕੒Γཱͭ • ʮද͕ग़Δ֬཰ p ͷίΠϯΛ n ճ౤͛ͯɼද͕ i ճͰΔ֬཰Λ ද͢ʯͱ͍͏ϑϨʔζͰ֮͑Ε͹ྑ͍ • n = 1 ͷ৔߹ɼಛʹϕϧψʔΠ෼෍ͱݺ͹ΕΔ 29 / 42
  21. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ཭ࢄܕ֬཰ม਺ͱͦͷ෼෍ͷྫ ϙΞιϯ෼෍ Po(λ) p(k) = P(X

    = k) = λk k! exp(−λ) • E = { 0, 1, . . . , } ͱ͢Δͱ P(X ∈ E) = 1 ͕੒Γཱͭ • كʹى͜Δࣄ৅ͷ؍ଌϞσϧͳͲʹΑ͘࢖ΘΕΔ 30 / 42
  22. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ࿈ଓܕ֬཰ม਺ • C-type r.v. ͷఆٛʹग़ͯ͘Δ fX

    ΛɼX ͷ֬཰ີ౓ؔ਺ (p.d.f.) ͱ͍͏ • fX(x) ≥ 0, ∫ ∞ −∞ f(u) du = 1, d dx FX(x) = f(x) ͕੒Γ ཱͭ • ࣮ࡍʹ֬཰ܭࢉΛ͢Δͱ͖͸ɼ P(a < x ≤ b) =F(b) − F(a) = ∫ b −∞ f(u) du − ∫ a −∞ f(u) du = ∫ b a f(u) du ͱ͢Δ 31 / 42
  23. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ࿈ଓܕ֬཰ม਺ͱͦͷ෼෍ ࿈ଓܕ֬཰ม਺ͷ෼෍ Ұ༷෼෍ U(a, b) fX(x)

    =    1 b − a , x ∈ (a, b) 0, otherwise ਖ਼ن෼෍ N ( µ, σ2 ) fX (x) = 1 √ 2πσ2 exp [ − (x − µ)2 2σ2 ] , −∞ < µ < ∞, σ > 0 32 / 42
  24. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ਖ਼ن෼෍ • ௒ॏཁ • c.d.f. ͕ॳ౳ؔ਺Ͱཅʹॻ͚ͳ͍

    • X ∼ N ( µ, σ2 ) ͷͱ͖ɼY = (x − µ)/σ ∼ N (0, 1) • X ∼ N (0, 1) ͷͱ͖ɼP(X ≥ 1.282) ≒ 0.10, P(X ≥ 1.645) ≒ 0.05, P(X ≥ 2.326) ≒ 0.01 ͋ͨΓ͸ਪఆɾݕఆͰ සग़ 33 / 42
  25. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ֬཰ม਺ͷม׵ Y = aX + b

    ͷ෼෍ a, b ∈ R Λ༻͍ͯɼC-type ֬཰ม਺ X ͷม׵ Y = aX + b Λߟ͑Δɽ͜ͷͱ͖ɼY ͷ p.d.f. ΛٻΊΑɽ FY (y) = P(Y ≤ y) = P(aX+b ≤ y) =        P ( X ≤ y − b a ) , a ≥ 0 P ( X > y − b a ) , a < 0 Ͱ͋Δ͔Βɼ FY (y) =        FX ( y − b a ) , a ≥ 0 1 − FX ( y − b a ) , a < 0 34 / 42
  26. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ֬཰ม਺ͷม׵ ͕ͨͬͯ͠ɼ fY (y) = d

    dy FY (y) =        1 a fX ( y − b a ) , a ≥ 0 − 1 a fX ( y − b a ) , a < 0 = 1 |a| fX ( y − b a ) . 35 / 42
  27. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ଟ࣍ݩ෼෍ • ͜Ε·Ͱ͸ 1 ͭͷ֬཰ม਺͚ͩΛߟ͍͕͑ͯͨɼ͜ΕΛଟ࣍ݩ ֬཰ม਺ϕΫτϧʹ֦ு͢Δ

    • X1, . . . , Xn Λ (Ω, B, P) ্ͷ֬཰ม਺ͱͯ͠ɼ֬཰ม਺ϕΫ τϧ X = [X1, . . . , Xn] Ͱఆٛ͢Δ • n ࣍ݩϘϨϧू߹଒΋ Bn = ∏ n i=1 (ai, bi] Ͱఆٛ͞ΕΔ • PRML ΛಡΉ্Ͱ͸ɼ ʮ֬཰ม਺͕ϕΫτϧʹͳͬͨʯͱ͍͏ ఔ౓ͷཧղͰ໰୊ͳ͍ (ଟ෼) • ଟ࣍ݩ෼෍΋ɼ1 ࣍ݩͷ৔߹ͱಉ༷ʹ཭ࢄܕͱ࿈ଓܕʹ෼͚Β ΕΔ 36 / 42
  28. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ಉ࣌ྦྷੵ෼෍ؔ਺ Joint Cumulative Distribution Function X

    = [X1, . . . , Xn]ɿ֬཰ม਺ϕΫτϧͱͯ͠ɼX ͷಉ࣌ྦྷ ੵ෼෍ؔ਺ (j.c.d.f.) Λ FX(x1, . . . , xn) = P (X1 ≤ x1, . . . , Xn ≤ xn) Ͱఆٛ͢Δɽ·ͨɼX ͷ෦෼֬཰ϕΫτϧͷ෼෍ؔ਺Λपล ෼෍ؔ਺ (m.d.f.) ͱ͍͏ɽ 37 / 42
  29. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ ଟ࣍ݩ෼෍ͷܭࢉ • C-type Ͱͷ j.c.d.f. ͷٻΊํΛݟ͓ͯ͘

    • X := [X1, . . . , Xn] • fXɿX ͷ j.p.d.f. FX(x1, . . . , xn) = ∫ xn −∞ ∫ xn−1 −∞ · · · ∫ x1 −∞ fX(u1, . . . , un) du1du2 . . . dun, fX(x1, . . . , xn) = ∂n ∂x1 · · · ∂xn FX(x1, . . . , xn) 38 / 42
  30. Introduction ֬཰ ֬཰ม਺ ࣍ճ༧ࠂ 2 ࣍ݩਖ਼ن෼෍ 2 ࣍ݩਖ਼ن෼෍ [X, Y

    ] ͷ j.p.d.f. ͕ fXY (x, y) = 1 √ |det(2πΣ)| exp [ − 1 2 (z − µ)′ Σ−1 (z − µ) ] ͷͱ͖ɼ[X, Y ] ͸ 2 ࣍ݩਖ਼ن෼෍ʹै͏ͱ͍͍ɼ[X, Y ] ∼ N (µ, Σ) Ͱද͢ɽ͜͜Ͱɼ−∞ < µX, µY < ∞, σX, σY > 0, |ρ| < 1 Ͱ͋Γɼ z = [ x y ] , µ = [ µX µY ] , Σ = [ σ2 X ρσXσY ρσXσY ρ2 Y ] Ͱ͋Δɽ • fX(x) = ∫ R fXY (x, y) dy = 1 √ 2πσX exp [ − (x − µX)2 2σ2 X ] 39 / 42