言語処理のための機械学習入門 1.1〜1.4

社内の勉強会で発表した内容です

Transcript

1. ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 @fhiyo February 27, 2020 @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 1 / 33
2. Content 1.1 ४උͱຊॻʹ͓͚Δ໿ଋࣄ 1.2 ࠷దԽ໰୊ 1.3 ֬཰ 1.4 ࿈ଓ֬཰ม਺ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 2 / 33
3. 1.1 ४උͱຊॻʹ͓͚Δ໿ଋࣄ ݴޠॲཧʹ͓͚ΔओͳλεΫ ୯ޠ෼ׂʢword segmentationʣViterbi Ͱ࠷໬ύεΛٻΊΔ ඼ࢺλά෇͚ʢpart-of-speech taggingʣHMM Ͱ඼ࢺྻΛਪఆ ߏจղੳ (܎Γड͚ղੳɼ۟ߏ଄ղੳ)ʢsyntactic parsingʣPrim ๏ͳ ͲͰ spanning tree Λߏ੒ or shift-reduce จॻ෼ྨʢtext classificationʣLR, SVM ͳͲɽϞσϧͷબ୒ࢶଟ͍ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 3 / 33
4. 1.1 ४උͱຊॻʹ͓͚Δ໿ଋࣄ ༻ޠɾه๏ ࣄྫʢinstanceʣ: λεΫͷॲཧ୯Ґʢྫ: จॻ෼ྨͷࣄྫˠจॻʣ ίʔύεʢcorpusʣ: ݴޠͷ͞·͟·ͳܗͷ༻ྫͷू·Γ x(1) : σʔλͱͯ͠༩͑ΒΕͨࣄྫ n(w, d), nw,d : จॻ d ʹ͓͚Δ୯ޠ w ͷग़ݱճ਺ n(w, s), nw,s : จ s ʹ͓͚Δ୯ޠ w ͷग़ݱճ਺ n(w, c), nw,c : Ϋϥε c ʹଐ͢Δจॻ܈ʹ͓͚Δ୯ޠ w ͷग़ݱճ਺ N(w, c), Nw,c : Ϋϥε c ʹଐ͢Δจॻͷ͏ͪ w ͕ग़ݱ͢ΔΑ͏ͳจ ॻͷ਺ δ(w, d), δw,d : จॻ d ʹ͓͍ͯ୯ޠ w ͕ग़ݱͨ͠ͱ͖ 1, ग़ݱ͠ͳ ͔ͬͨͱ͖ 0 @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 4 / 33
5. 1.2 ࠷దԽ໰୊ ࠷దԽ໰୊ʢoptimization problemʣ ͋Δ੍໿ͷ΋ͱͰؔ਺Λ࠷దԽ͢Δม਺஋ͱɼͦͷͱ͖ͷؔ਺஋ΛٻΊ Δ໰୊ɽ Ұൠܗ max. f (x) s.t. g(x) ≥ 0 h(x) = 0 f (x): ໨తؔ਺ g(x) >= 0: ෆ౳੍ࣜ໿ h(x) = 0: ౳੍ࣜ໿ ੍໿Λຬͨ͢ղ: ࣮ߦՄೳղ ʢfeasible solutionʣ ࣮ߦՄೳղͷू߹: ࣮ߦՄೳ ྖҬʢfeasible regionʣ ຊॻͰ͸ɼತ࠷దԽʹ͍ͭͯड़΂ ͍ͯΔɽ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 5 / 33
6. 1.2 ࠷దԽ໰୊ ྫ୊ 1.1 ྫ୊ 1.1 ͭ͗ͷ࠷େԽ໰୊Λղ͚ʢa ͸ఆ਺ʣ ɽ max. − x1x2 s.t. x1 − x2 − a = 0 @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 6 / 33
7. 1.2 ࠷దԽ໰୊ ྫ୊ 1.1 ྫ୊ 1.1 ͭ͗ͷ࠷େԽ໰୊Λղ͚ʢa ͸ఆ਺ʣ ɽ max. − x1x2 s.t. x1 − x2 − a = 0 ੍໿͔Β x2 = x1 − a ͱ͓͍ͯ໨ తؔ਺ʹ୅ೖ͠ɼx1 Ͱภඍ෼ͯ͠ 0 ͱ͓͚͹౴͕͑ग़Δɽ ౴͑: (x1, x2) = (a/2, −a/2) ͜ͷΑ͏ͳ x1=?? ͷܗͷղͷද ͠ํΛดܗࣜʢclosed formʣͱ͍ ͏ɽ??͸Ճݮ৐আ΍ॳ౳ؔ਺ͷ߹ ੒ؔ਺Ͱද͞ΕΔ ดܗࣜͷղ͕ಘΒΕΔ໰୊Λɼղ ੳతʹղ͚Δʢanalyticall solvableʣ໰୊ͱ͍͏ɽ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 7 / 33
8. 1.2.1 ತू߹ͱತؔ਺ ತू߹ ू߹ A ⊆ Rd ͕ತू߹ʢconvex setʣ Ͱ͋Δͱ͸ɼ೚ҙͷ x(1) ∈ A ͱ x(2) ∈ A ͕͋ͬͨͱ͖ɼ೚ҙͷ t ∈ [0, 1] ʹରͯ͠ tx(1) + (1 − t)x(2) ∈ A ͕੒Γཱͭ͜ͱͰ͋Δɽ ௚ײతʹ͍͑͹ɼtx(1) + (1 − t)x(2) ͸ x(1) ͱ x(2) Λ݁Ϳઢ෼Λද͢ͷͰɼ ͦͷઢ෼͕ू߹͔Β͸Έग़Δ͜ͱ͕ͳ͍ɼͱ͍͏ঢ়ଶɽͭ·Γɼू߹ʹ ʮ΁͜Έʯ͕ͳ͍ɽ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 8 / 33
9. 1.2.1 ತू߹ͱತؔ਺ ྫ୊ 1.2 ͋Δ༩͑ΒΕͨฏ໘্ʹ͋Δͱ͍͏੍໿Λຬͨ͢ϕΫτϧͷू߹ A = {x|mx + b = 0, x ∈ Rd } ͸ತू߹Ͱ͋Δ͜ͱΛࣔͤɽ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 9 / 33
10. 1.2.1 ತू߹ͱತؔ਺ ྫ୊ 1.2 ͋Δ༩͑ΒΕͨฏ໘্ʹ͋Δͱ͍͏੍໿Λຬͨ͢ϕΫτϧͷू߹ A = {x|mx + b = 0, x ∈ Rd } ͸ತू߹Ͱ͋Δ͜ͱΛࣔͤɽ A ͷཁૉ x(1), x(2) Λߟ͑Δɽ೚ҙͷ࣮਺ t ∈ [0, 1] ʹରͯ͠ x = tx(1) + (1 − t)x(2) ͱ͢Δͱɼ mx + b = m(tx(1) + (1 − t)x(2)) + b = t(mx(1) + b) + (1 − t)(mx(2) + b) = 0 Αͬͯತू߹Ͱ͋Δɽ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 10 / 33
11. 1.2.1 ತू߹ͱತؔ਺ ষ຤໰୊ [3] ྖҬ A ͱ B ͷؒͷۭؒ͸ A ∪ B Ͱͳ͍ྖҬɽ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 11 / 33
12. 1.2.1 ತू߹ͱತؔ਺ ತؔ਺ ؔ਺ f (x) ্͕ʹತͰ͋Δͱ͸ɼ೚ҙͷ x(1), x(2) ∈ Rd ɼ೚ҙͷ t ∈ [0, 1] ʹର͠ɼf (tx(1) + (1 − t)x(2)) ≥ tf (x(1)) + (1 − t)f (x(2)) ͕੒ཱ͢Δ͜ͱ Ͱ͋Δɽ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 12 / 33
13. 1.2.1 ತू߹ͱತؔ਺ ྫ୊ 1.4 (i)’ ্ʹತͳؔ਺ f1(x), · · · , fn(x) ͷ࿨͸্ʹತͳؔ਺Ͱ͋Δ͜ͱΛࣔͤɽ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 13 / 33
14. 1.2.1 ತू߹ͱತؔ਺ ྫ୊ 1.4 (i)’ ্ʹತͳؔ਺ f1(x), · · · , fn(x) ͷ࿨͸্ʹತͳؔ਺Ͱ͋Δ͜ͱΛࣔͤɽ h(x) = ∑ n i=1 fi (x) ͱ͓͘ɽ h(t(x(1)) + (1 − t)(x(2))) = n ∑ i=1 fi (t(x(1)) + (1 − t)(x(2))) ≥ n ∑ i=1 tfi (x(1)) + (1 − t)fi (x(2)) = t( n ∑ i=1 fi )(x(1)) + (1 − t)( n ∑ i=1 fi )(x(2)) = th(x(1)) + (1 − t)h(x(2)) @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 14 / 33
15. 1.2.1 ತू߹ͱತؔ਺ ತؔ਺Ͱ͋ΔͨΊͷ 1 ࣍ͷ৚݅ ؔ਺ f ্͕ʹತͰ͋Δͱ͖ɼ೚ҙͷ x(1) ∈ R, x(2) ∈ R ʹ͍ͭͯɼ f (x(2)) − f (x(1)) ≤ ∂f (x(1)) ∂x (x(2) − x(1)) ͕੒Γཱͭɽ ತؔ਺Ͱ͋ΔͨΊͷ 2 ࣍ͷ৚݅ 1 ม਺ؔ਺ f (x) ্͕ʹತͰ 2 ֊ඍ෼Մೳͳͱ͖ɼ2 ֊ඍ෼ f ′′(x) ͸ͭͶʹ ෛ·ͨ͸ 0 ͕੒Γཱͭɽ ತؔ਺Ͱ͋ΔͨΊͷ 2 ࣍ͷ৚݅ʢଟม਺ؔ਺൛ʣ ଟม਺ؔ਺ f (x) ্͕ʹತͰ 2 ֊ඍ෼ՄೳͰ͋Δͱ͖ɼϔοηߦྻ Hf (x) ͸൒ෛఆ஋Ͱ͋Δɽ ϔοηߦྻ H ͕൒ෛఆ஋ ⇐⇒ ೚ҙͷ n ࣍ݩϕΫτϧ x ʹ͍ͭͯ xTHx ≤ 0 ͕੒Γཱͭ ⇐⇒ ϔοηߦྻͷݻ༗஋͕͢΂ͯෛ·ͨ͸ 0 @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 15 / 33
16. 1.2.2 ತܭը໰୊ ತܭը໰୊ ͋Δ࠷దԽ໰୊͕ತܭը໰୊Ͱ͋Δͱ͸ɼͦͷ໨తؔ਺͕ತؔ਺Ͱ͋ͬ ͯɼ͔࣮ͭߦՄೳྖҬ͕ತू߹Ͱ͋Δ͜ͱΛ͍͏ɽ ྫ. ͭ͗ͷತܭը໰୊Λղ͚ɽ max. − x2 1 − x2 2 s.t. x1 − x2 − a = 0 ತܭը໰୊ͷղ͕ɼ໨తม਺Λඍ෼ͯ͠ 0 ʹͳΔ఺ͱͯ͠ٻΊΒΕΔ৔ ߹͸ͦΕͰ OKɽ ٻΊΒΕͳ͍৔߹ͷྫ: f (x) = 1 2 x2 + exp(x) ˠॳظ஋͔Βগͣͭ͠஋Λߋ৽͢Δ਺஋ղ๏ʢnumerical methodʣ ʹΑΓ ۙࣅղΛٻΊΔɽ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 16 / 33
17. 1.2.2 ತܭը໰୊ ਺஋ղ๏ͷղ͖ํͷྫ: 1. ࠷ٸ߱Լ๏ʢgradient descent methodʣ xnew = xold − ϵ∇x f (xold) 2. χϡʔτϯ๏ʢNewton’s methodʣ xnew = xold − H−1 xold ∇x f (xold) @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 17 / 33
18. 1.2.3 ౳੍ࣜ໿෇ತܭը໰୊ max. f (x) s.t. g(x) = 0 Ұൠʹ g(x) = 0 Λຬͨ͢ղ͕ภඍ෼ͯ͠ 0 ͱͳΔͱ͸ݶΒͳ͍ͷͰɼࠓ ·Ͱͷख๏͸࢖͑ͳ͍ ˠϥάϥϯδϡؔ਺ʢLagrangianʣ L(x, λ) Λಋೖ͢Δ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 18 / 33
19. ϥάϥϯδϡͷະఆ৐਺๏ʢLagrange Multipliersʣ ϥάϥϯδϡͷະఆ৐਺๏ʹΑΔղ͖ํ: L(x, λ) = f (x) + λg(x) Ͱ͋Δϥάϥϯδϡؔ਺ L Λಋೖ͢Δɽ ͜ͷͱ͖ɼ      ∂L ∂x = 0 (1) ∂L ∂λ = 0 (2) ͱ͍͏࿈ཱํఔࣜΛղ͘͜ͱʹΑΓɼ࠷దͳղ͕ಘΒΕΔɽ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 19 / 33
20. ϥάϥϯδϡͷະఆ৐਺๏ʢLagrange Multipliersʣ 1 1Pattern Recognition and Machine Learning, Christopher M. Bishop, Springer, p.708 @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 20 / 33
21. 1.2.4 ෆ౳੍ࣜ໿෇ತܭը໰୊ max. f (x) s.t. g(x) ≥ 0 Ҍ఺ L(x, λ∗) ≤ L(x∗, λ∗) ≤ L(x∗, λ) ͕ղΛຬͨ͢఺ͱͳΔ͕ɺͦͷূ໌ ͕Ͱ͖ͳ͔ͬͨɽ ɽҎԼʹ໰୊Λղ͘खॱ͚ͩॻ͖·͢ɽ 1 L(x, λ) = f (x) + λg(x) (λ ≥ 0) Ͱ͋Δϥάϥϯδϡؔ਺Λߏ੒͢Δ 2 L(x, λ) Λ࠷େԽ͢Δ x = x∗(λ) ΛٻΊΔ 3 L(x∗(λ), λ) Λ λ ʹ͍ͭͯ࠷খԽ͢Δ (૒ର໰୊Λղ͘͜ͱʹΑΓղΛٻ ΊΔ) 2 2 https://ja.wolframalpha.com/input/?i=%E9%9E%8D%E7%82%B9+x%5E2+-+y%5E2 @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 21 / 33
22. 1.2.4 ෆ౳੍ࣜ໿෇ತܭը໰୊ ྫ୊ 1.8 ͭ͗ͷ࠷େԽ໰୊Λղ͚ɽ max. − x2 1 − x2 2 s.t. x1 + x2 − 1 ≥ 0 1 L(x, λ) = f (x) + λg(x) (λ ≥ 0) Ͱ͋Δϥάϥϯδϡؔ਺Λߏ੒͢Δ L(x1, x2, λ) = −x2 1 − x2 2 + λ(x1 + x2 − 1), λ ≥ 0 2 L(x, λ) Λ࠷େԽ͢Δ x = x∗(λ) ΛٻΊΔ L Λ x1, x2 Ͱඍ෼ͯ͠ 0 ͱ͓͘ɽ x∗ 1 (λ) = x∗ 2 (λ) = λ/2 3 L(x∗(λ), λ) Λ λ ʹ͍ͭͯ࠷খԽ͢Δ (૒ର໰୊Λղ͘͜ͱʹΑΓղ ΛٻΊΔ) L(x1 = λ/2, x2 = λ/2, λ) = 1 2 λ2 − λ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 22 / 33
23. 1.2.4 ෆ౳੍ࣜ໿෇ತܭը໰୊ ষ຤໰୊ 6 g(x) ≥ 0 ͳΔ੍໿ͷ΋ͱͰ f (x) Λ࠷େԽ͢Δ໰୊Λߟ͑Δɽ͜Ε͕ತܭ ը໰୊Ͱ͋Δͱ͖ɼ(x∗, λ∗) ͕ L(x, λ) = f (x) + λg(x) ͷҌ఺ͳΒ͹ɼx∗ ͸࠷దղͰ͋Δ͜ͱΛূ໌ͤΑɽ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 23 / 33
24. 1.2.4 ෆ౳੍ࣜ໿෇ತܭը໰୊ ষ຤໰୊ 6 g(x) ≥ 0 ͳΔ੍໿ͷ΋ͱͰ f (x) Λ࠷େԽ͢Δ໰୊Λߟ͑Δɽ͜Ε͕ತܭ ը໰୊Ͱ͋Δͱ͖ɼ(x∗, λ∗) ͕ L(x, λ) = f (x) + λg(x) ͷҌ఺ͳΒ͹ɼx∗ ͸࠷దղͰ͋Δ͜ͱΛূ໌ͤΑɽ (x∗, λ∗) ͕Ҍ఺ ⇐⇒ L(x, λ∗) ≤ L(x∗, λ∗) ≤ L(x∗, λ) ͕੒ཱ͢Δɽ L(x∗, λ∗) ≤ L(x∗, λ) ΑΓɼλ∗g(x∗) ≤ λg(x∗)ɽ͜Ε͕೚ҙͷ λ ʹ͍ͭͯ੒Γཱ ͭɽg(x∗) = 0 ͷͱ͖͸ λ∗ ͸೚ҙɽg(x∗) > 0 ͷͱ͖ɼλ∗ ≤ λɽλ ≥ 0 ͳͷͰ λ∗ = 0ɽΑͬͯ λ∗g(x∗) = 0 ͱͳΔɽ ͯ͞ɼತܭը໰୊Ͱ͋Δ͜ͱ͔Βɼ L(x, λ∗) ≤ L(x∗, λ∗) + ∇x L(x∗, λ)(x − x∗) = f (x∗) + λ∗g(x∗) + ∇x L(x∗, λ)(x − x∗) = f (x∗) ·ͨɼL(x, λ∗) = f (x) + λ∗g(x) ≥ f (x)ɽΑͬͯ f (x) ≤ L(x, λ∗) ≤ f (x∗) f (x) ≤ f (x∗) ೚ҙͷ x ʹ͍ͭͯ੒ΓཱͭͷͰɼ୊ҙ੒ཱ. (ূ໌ऴ) @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 24 / 33
25. 1.3 ֬཰ ʢ͓ͦΒ͘ຊڭՊॻͷ໨తʹԊ͏ͨΊʹɼ͜ͷลΓͷఆٛ͸ׂͱᐆດʹ ॻ͔Ε͍ͯ·ͨ͠ɽͬ͘͟Γͱͨ͠Πϝʔδ͕Ͱ͖Ε͹͍͍ͱࢥ͏ͷͰ ͜ͷลΓ͸ඈ͹͠·͢) ࣄ৅ ࣄ৅ۭؒ ֬཰ม਺ ֬཰ؔ਺ ... @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 25 / 33
26. 1.3 ֬཰ ষ຤໰୊ʲ9ʳ ֬཰ม਺ X1, X2, X3, X4 ͷ೚ҙͷ஋ x1, x2, x3, x4 ʹର͠ɼͭ͗ͷ౳͕ࣜ ੒Γཱͭ͜ͱΛࣔͤ ʢP(x2|x3, x4) ̸= 0, P(x3, x4) ̸= 0 ͱ͢Δʣ P(x1|x2, x3, x4) = P(x1, x2|x3, x4) P(x2|x3, x4) @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 26 / 33
27. 1.3 ֬཰ ষ຤໰୊ʲ9ʳ ֬཰ม਺ X1, X2, X3, X4 ͷ೚ҙͷ஋ x1, x2, x3, x4 ʹର͠ɼͭ͗ͷ౳͕ࣜ ੒Γཱͭ͜ͱΛࣔͤ ʢP(x2|x3, x4) ̸= 0, P(x3, x4) ̸= 0 ͱ͢Δʣ P(x1|x2, x3, x4) = P(x1, x2|x3, x4) P(x2|x3, x4) P(x1, x2|x3, x4) = P(x1, x2, x3, x4) P(x3, x4) P(x2|x3, x4) = P(x2, x3, x4) P(x3, x4) P(x1, x2|x3, x4) P(x2|x3, x4) = P(x1, x2, x3, x4) P(x2, x3, x4) = P(x1|x2, x3, x4) @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 27 / 33
28. 1.3 ֬཰ ྫ୊ 1.14ʢϕΠζͷఆཧʣ x ͸֬཰ม਺ X ͷ೚ҙͷ஋Ͱ͋Γɼy ͸֬཰ม਺ Y ͷ೚ҙͷ஋Ͱ͋Δͱ ͢ΔɽP(x) ̸= 0 ͷͱ͖ɼͭ͗ͷ౳͕ࣜ੒ཱ͢Δ͜ͱΛࣔͤɽ P(y|x) = P(y)P(x|y) P(x) P(y|x) = P(y, x) P(x) = P(y)P(x|y) P(x) ϕΠζͷఆཧ͸ҎԼͷΑ͏ʹɼ͋Δ֬཰ม਺ x3 ʹ৚݅෇͚ΒΕͨ৔߹Ͱ΋ಉ༷ ʹ੒Γཱͭɽ P(x2|x1, x3 ) = P(x2|x3 )P(x1|x2, x3 ) P(x1, x3 ) @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 28 / 33
29. 1.3 ֬཰ पล֬཰ʢmerginal probabilityʣ ಉ࣌෼෍ P(x, y) ͕͋ͬͨͱ͖ɼͨͱ͑͹ͦΕΛ y ʹ͍ͭͯ଍͠߹ΘͤΔ ͜ͱͰɼP(x) ͕ಘΒΕΔɽͭ·Γɼ ∑ y P(x, y) = P(x) Ͱ͋Δɽ͜ͷΑ ͏ʹͯ͠ P(x) ΛٻΊͨͱ͖ɼP(x) Λ x ͷपล֬཰ͱ͍͏ɽ ಠཱʢindependentʣ 2 ͭͷ֬཰ม਺ XɼY ʹ͍ͭͯɼP(X = x, Y = y) = P(X = x)P(Y = y) ͕೚ҙͷ x,y ʹ͍ͭͯ੒ཱ͢Δͱ͖ɼX ͱ Y ͸ಠཱͰ͋Δͱ͍͏ɽ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 29 / 33
30. 1.3 ֬཰ ৚݅෇͖ಠཱʢconditionally independentʣ 3 ͭͷ֬཰ม਺ X1, X2, X3 ʹ͍ͭͯߟ͑ΔɽX3 ͷ͋Δ࣮ݱ஋ x3 ʹର͠ɼ X1,X2 ͷ೚ҙͷ࣮ݱ஋ x1,x2 ʹ͍ͭͯ P(X1 = x1, X2 = x2|x3) = P(X1 = x1|x3)P(X2 = x2|x3) ͕੒ཱ͢Δͱ͖ɼX1 ͱ X2 ͸ɼX3 = x3 ͱ͍͏৚݅ͷ΋ͱͰ৚݅෇͖ಠ ཱͰ͋Δͱ͍͏ɽ ·ͨɼಛఆͷ஋ x3 ͚ͩͰͳ͘ɼX3 ͷ೚ҙͷ஋ʹ͍ͭͯ X1,X2 ͕৚݅෇͖ ಠཱͰ͋Δͱ͖ɼX1 ͱ X2 ͸ɼX3 ͕༩͑ΒΕͨ৔߹ʹ৚݅෇͖ಠཱͰ͋ Δͱ͍͏ɽ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 30 / 33
31. 1.3 ֬཰ ྫ୊ 3 ͭͷ֬཰ม਺ X1, X2, X3 ʹ͍ͭͯߟ͑ΔɽX1 ͱ X2 ͕ X3 ͷ΋ͱͰ৚݅ ෇͖ಠཱ Ͱ͋Δͱ͖ɼҎԼ͕੒Γཱͭ͜ͱΛূ໌ͤΑɽ P(x1|x2, x3) = P(x1|x3) P(x1|x2, x3) = P(x1, x2|x3) P(x1|x3) = P(x1|x3)P(x2|x3) P(x1|x3) = P(x1|x3) @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 31 / 33
32. 1.3 ֬཰ ϕϧψʔΠ෼෍: ίΠϯ౤͛ͷ 1 ճͷࢼߦͷ֬཰෼෍ ϚϧνψʔΠ෼෍: αΠίϩ౤͛ͷ 1 ճͷࢼߦͷ֬཰෼෍ ೋ߲෼෍: ίΠϯ౤͛ͷ n ճͷࢼߦͷ֬཰෼෍ ଟ߲෼෍: αΠίϩ౤͛ͷ n ճͷࢼߦͷ֬཰෼෍ ϙΞιϯ෼෍: ද͕ग़Δ֬཰͕ඇৗʹ௿͍ίΠϯΛඇৗʹଟ͘౤͛ ͨͱ͖ͷೋ߲෼෍ ((ίΠϯ౤͛ͷࢼߦճ਺)×(ද͕ग़Δ֬཰) ͕ύϥ ϝʔλ λ ʹͳΔ) @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 32 / 33
33. 1.4 ࿈ଓ֬཰ม਺ Ψ΢ε෼෍: ฏۉ͕࠷΋ग़΍͘͢ɼࠨӈରশͰແݶʹଓ͘෼෍ σΟϦΫϨ෼෍: αΠίϩ౤͛ʹ͓͚ΔɼαΠίϩͷ֤໘͕ग़Δ֬ ཰ͷ෼෍ @fhiyo ݴޠॲཧͷͨΊͷػցֶशೖ໳ 1.1ʙ1.4 February 27, 2020 33 / 33