Share
Embed
Transcript
43 slides
-
Introduction ֬ ֬ม ࣍ճ༧ࠂ ୈ 1 ճ ֬ɾ౷ܭͷجૅษڧձ Hiroaki Tanaka
Augmented Human Communication Laboratory, Department of Information Schience, Nara Institute of Science and Technology November 8, 2016 1 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ 1 Introduction 2 ֬ 3 ֬ม
4 ࣍ճ༧ࠂ 2 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ͡Ίʹ • εϥΠυͦͷଞͷࢿྉ https://setten-qb.amebaownd.com/ ʹ͓͍ͯ͋Γ·͢ •
Twitter: @setten QB 3 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ֬ͱ౷ܭֶ • ֬ԋ៷త • ౷ܭֶؼೲత •
֬ͷํ͕நతͳٞΛ͢Δࣄ͕ଟ͍ 4 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ཧ౷ܭֶͷجૅ༰ ۭ֬ؒ શۭؒͷଌ͕ 1 ͱͳΔΑ͏ͳଌۭؒ ֬มɾ֬
PRML ͷ 2ɾ3 ষͰग़ͯ͘Δ ֬ͷಛੑ PRML ͷ 2ɾ3 ষͰग़ͯ͘Δ ۙࣅ๏ଇ n → ∞, d :fix ͷ߹͕جຊత ౷ܭతਪଌ ͏͚ͩͳΒ͘͠ͳ͍ ౷ܭతݕఆ ͏͚ͩͳΒ͘͠ͳ͍ 6 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ͜ͷษڧձͷ౸ୡඪ • ౷ܭֶͷجૅΛཧղ͢Δ • PRML2ɾ3 ষͷࣜΛ͍͍ͩͨ͑ΔΑ͏ʹ͢Δ
• ༨༟ͱχʔζ͕͋Εɼֶशཧ·Ͱͬͯྑ͍͔ 7 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ͜ͷษڧձͰѻ͏༰ ۭ֬ؒ ୈ 1 ճ ֬มɾ֬
ୈ 1 ճ ֬ͷಛੑ ୈ 2 ճ ۙࣅ๏ଇ ୈ 2 ճ ౷ܭతਪఆ ୈ 3 ճ ౷ܭతݕఆ ୈ 4 ճ ଟมྔΛѻ͏ͨΊʹඞཁͳઢܗ ୈ 5 ճʁ 8 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ 1 Introduction 2 ֬ 3 ֬ม
4 ࣍ճ༧ࠂ 9 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ جຊࣄͱඪຊۭؒ • ະͳݱΛཧղ͢ΔͨΊͷ࣮ݧௐࠪΛࢼߦͱݺͼɼࢼߦͷ ݁ՌΛجຊࣄͱݺͿ • ಘΒΕΔͰ͋Ζ͏݁ՌʢجຊࣄʣશମΛநతʹඪຊۭؒͱ
ݺͿ Example αΠίϩΛ 1 ճ͛Δͱ͍͏ࢼߦΛߟ͑Δɽ͜ͷͱ͖ɼඪຊ ۭؒ { 1 ͷ͕ग़Δ, . . . , 2 ͷ͕ग़Δ } ʹͳΔɽ 10 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ σ ू߹ • ҎԼɼඪຊۭؒΛ Ω Ͱද͢
• ֬ɼΩ ্ͷಛผͳ݅Λຬͨ͢෦ू߹্Ͱఆٛ͞ΕΔ σ-algebra Ω ͷ෦ू߹ B ͕ҎԼͷ݅Λຬͨ࣌͢ɼB Λ Ω ্ͷ σ ू߹ͱ͍͏: Ω ∈ B; A ∈ B =⇒ Ac ∈ B; A1, . . . , An, · · · ∈ B =⇒ ∞ ∪ i=1 Ai ∈ B. • ֬Ͱ σ ू߹ͷཁૉΛࣄͱ͍͏ • Ω Λશࣄͱ͍͏ 11 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ σ ू߹ • σ ू߹͕ຬͨ͢ੑ࣭ͨ͘͞Μ͋Δ͕ɼৄࡉུ •
σ ू߹্Ͱɼ֬ఆٛ͞ΕΔ • Ω ͱ B ͷରΛ (Ω, B) ͱॻ͍ͯɼՄଌۭؒ (measurable space) ͱݺͿ 12 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ֬ଌ Probabilitstic Measure B ্ͷ࣮ؔ P
͕ 0 ≤ P(A) ≤ 1, ∀A ∈ B, P(Ω) = 1, Ai ∈ B(i = 1, . . . ), Ai ∩ Aj = ∅(i ̸= j) =⇒ P ( ∞ ∪ i=1 Ai ) = ∞ ∑ i=1 P(Ai) Λຬͨ͢ͱ͖ɼP Λ (Ω, B) ্ͷ֬ଌ (probabilistic measure) ͱ͍͍ɼP(A) Λࣄ A ͷ֬ͱ͍͏ɽ • Ω, B, P ͷ 3 ͭ (Ω, B, P) Λۭ֬ؒͱ͍͏ɽ 13 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ֬ଌ • ͜͏ͬͯ֬Λఆٛ͢Δ͜ͱʹΑΓɼօ͕͍ͬͯΔΑ͏ͳ ֬ͷੑ࣭ P(∅) =
0 P(A) + P(Ac) = 1, for A ∈ B ͕Γ ཱͭ • ͦͷଞʹͨ͘͞Μͷॏཁͳੑ࣭͕͋Δ͕ɼৄࡉུ 14 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ͦͦଌͬͯԿʁ • ू߹ͷେ͖͞ΛଌΔई → ଌ •
P(A1) Ͱ A1 ͷେ͖͞Λද͢ → ֬ • ۭ֬ؒΛ໌Β͔ʹ͍ͯ͠ͳ͍ͱࠔΔ͜ͱ͕͋Δ • Ͳ͏͍͏ू߹্ʹ֬ΛೖΕΔͷʁ → σ ू߹ ଌ R ͔Β࣮ a, b Λબͼɼ۠ؒ I = [a, b], a, b ∈ [0, 1] Λߏ ͢Δɽ۠ؒ I ͷ “େ͖͞” ʹ͍ͭͯߟ͑Δɽ a b x M 16 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ͦͦଌͬͯԿʁ • ू߹ͷେ͖͞ΛଌΔई → ଌ •
P(A1) Ͱ A1 ͷେ͖͞Λද͢ → ֬ • ۭ֬ؒΛ໌Β͔ʹ͍ͯ͠ͳ͍ͱࠔΔ͜ͱ͕͋Δ • Ͳ͏͍͏ू߹্ʹ֬ΛೖΕΔͷʁ → σ ू߹ ଌ R ͔Β࣮ a, b Λબͼɼ۠ؒ I = [a, b], a, b ∈ [0, 1] Λߏ ͢Δɽ۠ؒ I ͷ “େ͖͞” ʹ͍ͭͯߟ͑Δɽ a b x M ϧϕʔάଌ L([a, b]) b − a 16 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ͖݅֬ • ͖݅֬ଌۭؒ (Ω, B) ্Ͱఆٛ͞ΕΔ
Conditional Probability C ∈ B, P(C) > 0 ͷͱ͖ɼ P(A|C) = P(A ∩ C) P(C) , A ∈ B Λࣄ C ͕༩͑ΒΕͨͱ͖ͷ A ∈ B ͷ͖݅֬ͱ͍ɼ P( · |C) Λ C ͕༩͑ΒΕͨͱ͖ͷ͖݅֬ଌͱ͍͏ɽ • ্هͷΑ͏ʹ͖݅֬Λఆٛ͢ΔͱɼP( · |C) (Ω, B) ্ͷ֬ଌʹͳ͍ͬͯΔ 17 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ͖݅֬ • Α͘͏શ֬ͷެࣜ શ֬ͷެࣜ ∞ ∪
i=1 Ai = Ω, Ai ∩ Aj = ∅, P(Ai) > 0 ͱ͢Δɽ͜ͷͱ͖ɼ A ∈ B ʹ͍ͭͯ P(A) = ∞ ∑ i=1 P(Ai)P(A|Ai) ͕Γཱͭɽ 18 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ 1 Introduction 2 ֬ 3 ֬ม
4 ࣍ճ༧ࠂ 19 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ϘϨϧू߹ • ۭ֬ؒ (Ω, B, P)
Λ࣮ۭؒ݁ͼ͚ͭΔͷ • ࣮ۭؒͰɼ(Ω, B) ͕ (R, B) ʹରԠ͢Δ Borel algebra શͯͷ։۠ؒ (a, b] ΛؚΉ࠷খͷ σ ू߹ B Λ R ্ͷϘ Ϩϧू߹ͱ͍͏ɽ • B (a, b] Ҏ֎ʹ༷ʑͳ۠ؒΛؚΉ • { a } , (a, b), [a, b], [a, b), (a, ∞), (−∞, a] શͯϘϨϧू߹ 20 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ֬ม • ֬มͷݫີͳఆٛΛ͢Δ • ֬ม͕ɼநతͳۭ֬ؒͱ࣮ͷۭ֬ؒΛରԠ͚Δ ֬ม
X : Ω → R ͕ (Ω, B) ্ͷ֬ม def ⇐⇒∀B ∈ B, { ω | X(ω) ∈ B } = X−1(B) ∈ B. • X Λ֬มͱ͢Δͱ͖ɼଟ߲ࣜɼlog |X|, eX, sin X ͳͲ શͯ֬มͱͳΔ 21 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ֬ม 1 ∈ 2 ∈ −1(1
) −1(2 ) Measurable function Measurable function Figure: ֬มͷΠϝʔδ • ͲͷΑ͏ͳ B ∈ B ΛબΜͰɼX−1(B) ∈ B Մଌؔ X 22 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ؔ • (Ω, B) ͷ࣮ۭؒ൛ͱͯ͠ (R,
B) ͕ಋೖ͞Εͨ • Γͷ֬ଌ P ʹରԠ͢Δͷͱͯ͠ɼؔ F Λಋೖ ͢Δ Distribution Function F : R → R ͕ ∀x ∈ R, F(x) = P(X ≤ x) = P ({ ω | X(ω) ≤ x }) ʹΑͬͯ F Λఆٛ͢Δͱ͖ɼF Λ X ͷྦྷੵؔ (cumu- lative distribution function; cdf) ͱ͍͏ɽ • P(X ≤ x) Λ X ≤ x ͱͳΔ֬ͱಡΈସ͑Δࣄ͕Ͱ͖Δ 23 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ؔʹΑΔ֬ܭࢉ ֬ܭࢉ ֬ม X ʹର͠ɼ P(a
≤ X ≤ b) = P ( X−1 ((a, b]) ) = FX(b) − FX(a) 24 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ͜͜·Ͱͷ·ͱΊ • ֬Մଌ্ۭؒͰఆٛ͞ΕΔ࣮ؔ • ֬ม X
͕ɼۭ֬ؒͱ࣮ۭؒΛͭͳ͙ • ֬ม࣮มͰͳ͘Մଌؔ • ࣮ۭؒͰ֬มมͷΑ͏ʹѻͬͯࠩ͠ࢧ͑ͳ͍ 25 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ֬มͷλΠϓ Random Variable Discrete type X
: (Ω, B, P) ্ͷ r.v. ͱ͢Δɽ X ͕ D (Discrete) -type def ⇐⇒∃E ⊂ R (ߴʑՃࢉͳू߹) s.t. P(X ∈ E) = 1 Continuous type FX ɿX ͷؔͱ͢Δɽ X ͕ C (Continuous) -type def ⇐⇒∃fX : R → R+ (Մଌؔ) s.t. FX (x) = ∫ x −∞ fX (u) du, ∀x ∈ R 26 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ֬มͷλΠϓ • ؆୯ʹݴ͏ͱ . . .
D-type औΓ͏Δ͕ࢄ (E ⊂ N, Z, Q) C-type औΓ͏Δ͕࿈ଓ (E ⊂ R) 27 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ࢄܕ֬ม • X ͷऔΓ͏Δ͕ࢄͷͱ͖ D-type •
E = { x1, . . . , xn, . . . } , p(xi) = P(X = xi) ͱ͢Δͱɼ ∞ ∑ i=1 p(xi) = 1, F(x) = ∑ xi ≤ xp(xi) ͕Γཱͭɽ͜ͷ p(·) ͷ͜ͱΛ X ͷ࣭֬ྔؔ (p.m.f.) ͱ ݺͿ 28 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ࢄܕ֬มͱͦͷͷྫ 2 ߲ B(n, p) •
0 ≤ p ≤ 1, i = 1, . . . , n p(xi) = P(X = i) = ( n i ) pi(1 − p)n−i • E = { 1, . . . , n } ͱ͢ΔͱɼP(X ∈ E) = 1 ͕Γཱͭ • ʮද͕ग़Δ֬ p ͷίΠϯΛ n ճ͛ͯɼද͕ i ճͰΔ֬Λ ද͢ʯͱ͍͏ϑϨʔζͰ֮͑Εྑ͍ • n = 1 ͷ߹ɼಛʹϕϧψʔΠͱݺΕΔ 29 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ࢄܕ֬มͱͦͷͷྫ ϙΞιϯ Po(λ) p(k) = P(X
= k) = λk k! exp(−λ) • E = { 0, 1, . . . , } ͱ͢Δͱ P(X ∈ E) = 1 ͕Γཱͭ • كʹى͜Δࣄͷ؍ଌϞσϧͳͲʹΑ͘ΘΕΔ 30 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ࿈ଓܕ֬ม • C-type r.v. ͷఆٛʹग़ͯ͘Δ fX
ΛɼX ͷ֬ີؔ (p.d.f.) ͱ͍͏ • fX(x) ≥ 0, ∫ ∞ −∞ f(u) du = 1, d dx FX(x) = f(x) ͕Γ ཱͭ • ࣮ࡍʹ֬ܭࢉΛ͢Δͱ͖ɼ P(a < x ≤ b) =F(b) − F(a) = ∫ b −∞ f(u) du − ∫ a −∞ f(u) du = ∫ b a f(u) du ͱ͢Δ 31 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ࿈ଓܕ֬มͱͦͷ ࿈ଓܕ֬มͷ Ұ༷ U(a, b) fX(x)
= 1 b − a , x ∈ (a, b) 0, otherwise ਖ਼ن N ( µ, σ2 ) fX (x) = 1 √ 2πσ2 exp [ − (x − µ)2 2σ2 ] , −∞ < µ < ∞, σ > 0 32 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ਖ਼ن • ॏཁ • c.d.f. ͕ॳؔͰཅʹॻ͚ͳ͍
• X ∼ N ( µ, σ2 ) ͷͱ͖ɼY = (x − µ)/σ ∼ N (0, 1) • X ∼ N (0, 1) ͷͱ͖ɼP(X ≥ 1.282) ≒ 0.10, P(X ≥ 1.645) ≒ 0.05, P(X ≥ 2.326) ≒ 0.01 ͋ͨΓਪఆɾݕఆͰ සग़ 33 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ֬มͷม Y = aX + b
ͷ a, b ∈ R Λ༻͍ͯɼC-type ֬ม X ͷม Y = aX + b Λߟ͑Δɽ͜ͷͱ͖ɼY ͷ p.d.f. ΛٻΊΑɽ FY (y) = P(Y ≤ y) = P(aX+b ≤ y) = P ( X ≤ y − b a ) , a ≥ 0 P ( X > y − b a ) , a < 0 Ͱ͋Δ͔Βɼ FY (y) = FX ( y − b a ) , a ≥ 0 1 − FX ( y − b a ) , a < 0 34 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ֬มͷม ͕ͨͬͯ͠ɼ fY (y) = d
dy FY (y) = 1 a fX ( y − b a ) , a ≥ 0 − 1 a fX ( y − b a ) , a < 0 = 1 |a| fX ( y − b a ) . 35 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ଟ࣍ݩ • ͜Ε·Ͱ 1 ͭͷ֬ม͚ͩΛߟ͍͕͑ͯͨɼ͜ΕΛଟ࣍ݩ ֬มϕΫτϧʹ֦ு͢Δ
• X1, . . . , Xn Λ (Ω, B, P) ্ͷ֬มͱͯ͠ɼ֬มϕΫ τϧ X = [X1, . . . , Xn] Ͱఆٛ͢Δ • n ࣍ݩϘϨϧू߹ Bn = ∏ n i=1 (ai, bi] Ͱఆٛ͞ΕΔ • PRML ΛಡΉ্Ͱɼ ʮ֬ม͕ϕΫτϧʹͳͬͨʯͱ͍͏ ఔͷཧղͰͳ͍ (ଟ) • ଟ࣍ݩɼ1 ࣍ݩͷ߹ͱಉ༷ʹࢄܕͱ࿈ଓܕʹ͚Β ΕΔ 36 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ಉ࣌ྦྷੵؔ Joint Cumulative Distribution Function X
= [X1, . . . , Xn]ɿ֬มϕΫτϧͱͯ͠ɼX ͷಉ࣌ྦྷ ੵؔ (j.c.d.f.) Λ FX(x1, . . . , xn) = P (X1 ≤ x1, . . . , Xn ≤ xn) Ͱఆٛ͢Δɽ·ͨɼX ͷ෦֬ϕΫτϧͷؔΛपล ؔ (m.d.f.) ͱ͍͏ɽ 37 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ ଟ࣍ݩͷܭࢉ • C-type Ͱͷ j.c.d.f. ͷٻΊํΛݟ͓ͯ͘
• X := [X1, . . . , Xn] • fXɿX ͷ j.p.d.f. FX(x1, . . . , xn) = ∫ xn −∞ ∫ xn−1 −∞ · · · ∫ x1 −∞ fX(u1, . . . , un) du1du2 . . . dun, fX(x1, . . . , xn) = ∂n ∂x1 · · · ∂xn FX(x1, . . . , xn) 38 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ 2 ࣍ݩਖ਼ن 2 ࣍ݩਖ਼ن [X, Y
] ͷ j.p.d.f. ͕ fXY (x, y) = 1 √ |det(2πΣ)| exp [ − 1 2 (z − µ)′ Σ−1 (z − µ) ] ͷͱ͖ɼ[X, Y ] 2 ࣍ݩਖ਼نʹै͏ͱ͍͍ɼ[X, Y ] ∼ N (µ, Σ) Ͱද͢ɽ͜͜Ͱɼ−∞ < µX, µY < ∞, σX, σY > 0, |ρ| < 1 Ͱ͋Γɼ z = [ x y ] , µ = [ µX µY ] , Σ = [ σ2 X ρσXσY ρσXσY ρ2 Y ] Ͱ͋Δɽ • fX(x) = ∫ R fXY (x, y) dy = 1 √ 2πσX exp [ − (x − µX)2 2σ2 X ] 39 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ 1 Introduction 2 ֬ 3 ֬ม
4 ࣍ճ༧ࠂ 40 / 42 -
Introduction ֬ ֬ม ࣍ճ༧ࠂ Reference • ฏণจ, ౷ܭղੳೖ, ग़൛גࣜձࣾ, 2007
• ౻؏ଠ, ཧ౷ܭֶ II ߨٛϊʔτ, 2014 42 / 42