PRMLセミナー

セミナー情報 : https://ml-for-experts.connpass.com/event/124722/

1. ୈ 3 ճ PRML ηϛφʔ 2019/04/22 ࡔޱ ྒี 1 / 74
2. 0. ࠓճͷηϛφʔʹ͍ͭͯ ࠓճͷηϛφʔͰ͸ɺPRML ͷୈ 8 ষͷάϥϑΟΧϧϞσϧΛ͓࿩͠ ͍ͨ͠ͱࢥ͍·͢ɻ ·ͨɺ͜ΕΒͷ࿩୊Λઆ໌͢ΔͨΊʹඞཁͳ༧උ஌ࣝΛղઆ͠·͢ɻ (PRML 1.2 ֬཰࿦ʹରԠ) ͳ͓஫ҙ఺ͱͯ͠ɺຊεϥΠυͷࣜ൪߸ͱ PRML ͷࣜ൪߸͸ҟͳΓ· ͢ͷͰɺ͝஫ҙ͍ͩ͘͞ɻ 2 / 74
3. ໨࣍ 1. ༧උ஌ࣝ 1-1. ֬཰࿦ͱϕΠζͷఆཧ (PRML 1.2) 2. άϥϑΟΧϧϞσϧ 2-1. ϕΠδΞϯωοτϫʔΫ (PRML 8.1) 2-2. ৚݅෇͖ಠཱੑ (PRML 8.2) 2-2-1. 3 ͭͷάϥϑͷྫ (PRML 8.2.1) 2-2-2. ༗޲෼཭ (D ෼཭)(PRML 8.2.2) 2-3. Ϛϧίϑ֬཰৔ (PRML 8.3) 2-3-1. ৚݅෇͖ಠཱੑ (PRML 8.3.1) 2-3-2. ෼ղಛੑ (PRML 8.3.2) 2-3-3. ༗޲άϥϑͱͷؔ܎ (PRML 8.3.4) 2-4. άϥϑΟΧϧϞσϧʹ͓͚Δਪ࿦ (PRML 8.4) 2-4-1. ࿈࠯ʹ͓͚Δਪ࿦ (PRML 8.4.1) 2-4-2. ɹ໦ (PRML 8.4.2) 2-4-3. Ҽࢠάϥϑ (PRML 8.4.3) 2-4-4. ੵ࿨ΞϧΰϦζϜ (PRML 8.4.4) 3 / 74
4. 1-1. ֬཰࿦ͱϕΠζͷఆཧ ·ͣ͸༧උ஌ࣝͱͯ͠ɺ֬཰࿦ (ಛʹ֬཰ͷՃ๏ఆཧɺ֬཰ͷ৐๏ఆཧɺ ϕΠζͷఆཧ) Λ؆୯ʹ͓͞Β͍͢Δɻ ·ͣ཭ࢄతͳ֬཰ม਺ X, Y Λߟ͑ɺ͜ΕΒ͸ X = xi (i = 1, 2, · · · , M)ɺY = yj (j = 1, 2, · · · , L) ΛͱΔͱ͢Δɻ ·ͨɺX, Y ͷ྆ํʹ͍ͭͯαϯϓϧΛऔΔ͜ͱΛશ෦Ͱ N ճߦ͏ɻ ͦͷ͏ͪɺX = xi , Y = yj ͳΔࢼߦͷ਺Λ nij ͱ͠ɺ(Y ʹ͸ؔ܎ͳ ͘)X = xi ͳΔࢼߦͷ਺Λ ci ͱ͠ɺ(X ʹ͸ؔ܎ͳ͘)Y = yj ͳΔࢼߦ ͷ਺Λ rj ͱ͢Δɻ(ҎԼͷਤ) 4 / 74
5. 1-1. ֬཰࿦ͱϕΠζͷఆཧ Ҏ্ͷઃఆΑΓɺX = xi , Y = yj ͱͳΔ֬཰ (ಉ࣌֬ ཰)p(X = xi , Y = yj ) ͸ҎԼͷΑ͏ʹͳΔɻ p(X = xi , Y = yj ) = nij N (1.1) ͜͜Ͱɺ૯ࢼߦճ਺ N ͸҉ʹແݶେͷۃݶΛԾఆ͍ͯ͠Δɻ ·ͨɺ(Y ʹ͸ؔ܎ͳ͘)X = xi ͳΔ֬཰ (पล֬཰)p(X = xi ) ͸ҎԼ ͷΑ͏ʹͳΔɻ p(X = xi ) = ci N (1.2) ·ͨɺci ͱ nij ʹ͸ ci = L ∑ j=1 nij (1.3) ͳΔؔ܎͕੒ཱ͢Δɻ 5 / 74
6. 1-1. ֬཰࿦ͱϕΠζͷఆཧ ͕ͨͬͯ͠ɺपล֬཰ p(X = xi ) ͱಉ࣌֬཰ p(X = xi , Y = yj ) ʹ͸Ҏ Լͷؔ܎͕੒ཱ͢Δɻ p(X = xi ) = ci N = L ∑ j=1 nij N = L ∑ j=1 p(X = xi , Y = yj ) (1.4) ͜ΕΛ֬཰ͷՃ๏ఆཧͱ͍͏ɻ ࣍ʹ X = xi ͱͳΔࣄ৅͚ͩΛߟ͑ɺͦͷ಺ Y = yj ͱͳΔ֬཰͸ p(Y = yj |X = xi ) ͱॻ͔ΕΔɻ ͜ͷ֬཰Λ৚݅෇͖֬཰ͱ͍͍ɺX = xi ͕༩͑ΒΕ͍ͯΔ্Ͱ Y = yj ͱͳΔ֬཰Ͱ͋Δɻ ۩ମతʹ͸ p(Y = yj |X = xi ) ͸ p(Y = yj |X = xi ) = nij ci (1.5) ͱॻ͚Δɻ 6 / 74
7. 1-1. ֬཰࿦ͱϕΠζͷఆཧ Ҏ্ΑΓɺಉ࣌֬཰ p(X = xi , Y = yj ) ͸पล֬཰ p(X = xi ) ͱ৚݅෇ ͖֬཰ p(Y = yj |X = xi ) Λ༻͍ͯҎԼͷΑ͏ʹॻ͚Δɻ p(X = xi , Y = yj ) = nij N = nij ci · ci N = p(Y = yj |X = xi )p(X = xi ) (1.6) ͜ΕΛ֬཰ͷ৐๏ఆཧͱ͍͏ɻ ͜ΕΒͷ 2 ͭͷఆཧΛ·ͱΊΔͱ p(X) = ∑ Y p(X, Y ) (1.7) p(X, Y ) =p(Y |X)p(X) (1.8) ͱͳΔɻ ͜͜ͰɺදهΛ؆ུԽ͢ΔͨΊʹ֬཰ม਺͕Ͳͷ஋ΛऔΔͷ͔Λলུ ͨ͠ɻ ࠓޙ͸ޡղ͕ͳ͍ݶΓɺ͜ͷදهΛར༻͢Δɻ 7 / 74
8. 1-1. ֬཰࿦ͱϕΠζͷఆཧ ͞Βʹɺಉ࣌෼෍ͷରশੑ p(X, Y ) = p(Y, X) ͱ৐๏ఆཧ (1.8) Λ༻͍ ΔͱɺҎԼͷϕΠζͷఆཧ͕ಋ͚Δɻ p(Y |X)p(X) = p(X|Y )p(Y ) → p(Y |X) = p(X|Y )p(Y ) p(X) (1.9) Ҏ্ͷఆཧΛ༻͍ͯɺҎԼͷάϥϑΟΧϧϞσϧΛղઆ͢Δɻ 8 / 74
9. 2. άϥϑΟΧϧϞσϧ ༷ʑͳ֬཰Ϟσϧͷܭࢉ͸਺ࣜΛมܗ͠ɺղ͘͜ͱ͕Ͱ͖Δɻ ͜ͷষͰ͸ɺ֬཰෼෍ΛਤͰදݱ͢Δ͜ͱʹΑΓɺͦͷΑ͏ͳ֬཰Ϟσ ϧͷܭࢉΛਤࣜతʹߦ͏ํ๏Λಋೖ͢Δɻ(֬཰తάϥϑΟΧϧϞσϧ) άϥϑ͸ҎԼͷਤͷΑ͏ʹϦϯΫ (ઢͷ෦෼) ͱϊʔυ (ԁͷ෦෼) Ͱߏ ੒͞ΕɺҎԼͷΑ͏ͳϦϯΫʹ໼ҹ (޲͖) ͕͋ΔϞσϧ (ϕΠδΞϯ ωοτϫʔΫɺ΋͘͠͸༗޲άϥϑΟΧϧϞσϧ) ͱϦϯΫ͕ํ޲ੑΛ ࣋ͨͳ͍Ϟσϧ (Ϛϧίϑ֬཰৔ɺ΋͘͠͸ແ޲άϥϑΟΧϧϞσϧ) ͕͋Δɻ 9 / 74
10. 2-1. ϕΠδΞϯωοτϫʔΫ ·ͣɺϕΠδΞϯωοτϫʔΫ (༗޲άϥϑΟΧϧϞσϧ) ʹ͍ͭͯઆ ໌͢Δɻ ͦͷͨΊʹ 3 ͭͷ֬཰ม਺ a, b, c ͷಉ࣌෼෍ؔ਺ p(a, b, c) Λߟ͑Δɻ (a, b, c ͸࿈ଓม਺Ͱ΋཭ࢄม਺Ͱ΋ྑ͍ɻ) ֬཰࿦ͷ৐๏ఆཧ (1.8) Λ༻͍ͯɺ͜ͷ֬཰෼෍ p(a, b, c) Λ༗޲άϥϑ Ͱදݱ͢ΔͨΊʹҎԼͷࣜมܗΛ͢Δɻ p(a, b, c) = p(c|a, b)p(a, b) = p(c|a, b)p(b|a)p(a) (2.1) ࣍ʹ (2.1) ͷ࠷ӈลΛάϥϑΛ࢖ͬͯදݱ͢Δɻ 10 / 74
11. 2-1. ϕΠδΞϯωοτϫʔΫ ·ͣɺ֬཰ม਺ a, b, c ͦΕͧΕʹରԠ͢ΔϊʔυΛඳ͖ɺͦΕͧΕͷ ϊʔυม਺ (a, b, c) ͱ (2.1) ͷ࠷ӈลͷҼࢠͰ͋Δ৚݅෇͖֬཰ΛରԠ ͤ͞Δɻ(a → p(a), b → p(b|a), c → p(c|a, b)) ͦͯ͠ɺͦΕͧΕͷ৚݅෇͖֬཰ʹରͯ͠ɺ৚݅ͱͯ͠༩͑ΒΕ͍ͯΔ ม਺ϊʔυ͔Β֬཰ม਺ͱͳΔม਺ϊʔυ΁༗޲ϦϯΫΛҾ͘ɻ ͨͱ͑͹ɺp(c|a, b) ʹରԠ͢Δ c ϊʔυʹ͸ɺa, b ͷϊʔυ͔Β༗޲Ϧ ϯΫ͕Ҿ͔Εɺp(a) ʹରԠ͢Δ a ϊʔυʹ͸Ͳ͔͜Β΋ a ϊʔυ΁ͷ ༗޲ϦϯΫ͸Ҿ͔Εͳ͍ɻ ͜ͷ݁Ռ͕ҎԼͷάϥϑͰ͋Γɺ(2.1) ͷ࠷ӈลΛද͢ɻ ͜͜Ͱɺa ͸ b ͷ਌ϊʔυͰ͋Γɺb ͸ a ͷࢠϊʔυͰ͋Δͱ͍͏ɻ 11 / 74
12. 2-1. ϕΠδΞϯωοτϫʔΫ (2.1) ͷ࠷ӈลʹରԠ͢Δάϥϑ͸શͯͷϊʔυ͕ࣗ෼Ҏ֎ͷશͯͷ ϊʔυͱͭͳ͕͍ͬͯΔɻ ͜ͷΑ͏ͳάϥϑΛશ݁߹Ͱ͋Δͱ͍͏ɻ ·ͣ͸ಉ࣌෼෍ͷࣜ (2.1) ͔ΒରԠ͢ΔάϥϑΛඳ͘͜ͱΛߦ͕ͬͨɺ ࣍͸άϥϑ͔ΒରԠ͢Δಉ࣌෼෍ͷࣜΛॻ͖Լ͢͜ͱΛߦ͏ɻ ۩ମతʹ͸ҎԼͷάϥϑʹରԠ͢Δಉ࣌෼෍ͷࣜΛॻ͖Լ͢ɻ 12 / 74
13. 2-1. ϕΠδΞϯωοτϫʔΫ ஫ҙਂ͘؍࡯͢Ε͹ɺಉ࣌෼෍͸ҎԼͷΑ͏ʹ͔͚Δɻ p(x1 )p(x2 )p(x3 )p(x4 |x1 , x2 , x3 )p(x5 |x1 , x3 )p(x6 |x4 )p(x7 |x4 , x5 ) (2.2) ͜ͷΑ͏ʹಉ࣌෼෍͔ΒରԠ͢Δάϥϑ͕ඳ͚ɺ͞Βʹٯʹάϥϑ͔Β ରԠ͢Δಉ࣌෼෍͕ॻ͚Δɻ(ͨͩ͠ɺಉ࣌෼෍ͱάϥϑ͸ҰରҰରԠ ͸͓ͯ͠ΒͣɺҰͭͷಉ࣌෼෍ʹରͯ͠ରԠ͢Δάϥϑ͸ෳ਺͋Δ͜ͱ ͕͋Δɻ) ͪͳΈʹҎԼͷάϥϑ͸શ݁߹Ͱ͸ͳ͍ɻ 13 / 74
14. 2-1. ϕΠδΞϯωοτϫʔΫ Ұൠతͳಉ࣌෼෍Λ p(x) ͱ͢Δɻ(͜͜Ͱɺx = (x1 , · · · , xK )T Ͱ ͋Δɻ) ·ͨɺϊʔυ xk ͷ਌ϊʔυͷू߹Λ pak ͱ͢Δͱɺಉ࣌෼෍ p(x) ͸ ҎԼͷΑ͏ʹॻ͚Δɻ p(x) = K ∏ k=1 p(xk |pak ) (2.3) ྫ͑͹ɺҎԼͷάϥϑͰ͸ɺpa5 = {x1 , x3 } Ͱ͋Δɻ 14 / 74
15. 2-2. ৚݅෇͖ಠཱੑ ͜͜Ͱ͸ɺ৚݅෇͖ಠཱੑͱάϥϑͷؔ܎Λઆ໌͢Δɻ ·ͣɺԾఆͱͯ͠ 3 ͭͷ֬཰ม਺ a, b, c ͷ಺ɺb, c ͕༩͑ΒΕͨ࣌ͷ a ͷ৚݅෇͖෼෍ p(a|b, c) ͕ b ʹґଘ͠ͳ͍ͱ͢Δɻ ͭ·ΓɺࣜͰॻ͘ͱ p(a|b, c) = p(a|c) (2.4) Ͱ͋Δ࣌ɺc ͕༩͑ΒΕͨԼͰɺa ͸ b ʹରͯ͠৚݅෇͖ಠཱͰ͋Δͱ ͍͏ɻ Ұํɺ৐๏ఆཧ (1.8) Λ༻͍Δͱɺ৚݅෇͖ಠཱ͸ҎԼͷΑ͏ʹ (2.4) ͱ͸ผͷදݱ͕Ͱ͖Δɻ p(a, b|c) = p(a, b, c) p(c) = p(a|b, c) · p(b, c) p(c) =p(a|b, c)p(b|c) = p(a|c)p(b|c) (2.5) ࠷ޙͷΠίʔϧͰ (2.4) Λ࢖༻ͨ͠ɻ 15 / 74
16. 2-2. ৚݅෇͖ಠཱੑ (2.5) ΛݟΔͱɺc ͕༩͑ΒΕͨ࣌ͷ a ͱ b ͷಉ࣌෼෍ p(a, b|c) ͕ c ͕ ༩͑ΒΕͨ࣌ͷ a ͷपล෼෍ p(a|c) ͱ c ͕༩͑ΒΕͨ࣌ͷ b ͷपล෼ ෍ p(b|c) ͷੵʹ෼ղͰ͖Δ͜ͱ͕Θ͔Δɻ ͭ·Γɺc ͕༩͑ΒΕͨͱ͖ a ͱ b ͸ಠཱͰɺ͜ͷΑ͏ͳঢ়گΛҎԼͷ Α͏ͳه๏Ͱද͢ɻ a |= b | c (2.6) ࣍ʹɺ͜ͷ৚݅෇͖ಠཱੑͱάϥϑͷؔ܎Λݟ͍ͯ͘ɻ 16 / 74
17. 2-2-1. 3 ͭͷάϥϑͷྫ ͜͜Ͱ͸ 3 ͭͷ؆୯ͳάϥϑʹରԠ͢Δಉ࣌෼෍ͷಠཱੑΛௐ΂Δɻ ·ͣ͸ҎԼͷάϥϑΛߟ͑Δɻ ͜ͷάϥϑʹରԠ͢Δಉ࣌෼෍͸ p(a, b, c) = p(a|c)p(b|c)p(c) (2.7) ͱͳΔɻ ·ͣ͸ a ͱ b ͷಠཱੑ (p(a, b) ? = p(a)p(b)) Λௐ΂Δɻ 17 / 74
18. 2-2-1. 3 ͭͷάϥϑͷྫ (2.7) ͷ྆ลΛ c ͰपลԽ (1.7) ͢Δͱ p(a, b) = ∑ c p(a|c)p(b|c)p(c) ̸= p(a)p(b) (2.8) ͱͳΓɺp(a, b) ͸ p(a) ͱ p(b) ͷੵʹ෼ղͰ͖ͳ͍ͷͰಠཱͰͳ͍ɻ a  |= b | ∅ (2.9) ͜͜Ͱɺ∅ ͸ۭू߹Λද͢ɻ Ұํɺc Λ৚͚݅ͭΔͨΊʹ (2.7) ͷ྆ลΛ p(c) ͰׂΔͱ p(a, b|c) = p(a, b, c) p(c) = p(a|c)p(b|c) (2.10) ͱͳΓɺ৚݅෇͖ಠཱੑ a |= b | c (2.11) Λຬͨ͢͜ͱ͕Θ͔Δɻ 18 / 74
19. 2-2-1. 3 ͭͷάϥϑͷྫ (2.9) ͱ (2.11) ͷ 2 ͭͷੑ࣭ΛάϥϑΛ࢖ͬͯߟ࡯͢Δͱɺc ͕༩͑Β Ε͍ͯͳ͍ͱ͖͸ a ͱ b ͷϊʔυ͕ܨ͕͍ͬͯͯɺa ͱ b ͸ಠཱͰͳ͍ɻ ͨͩ͠ɺc ͕༩͑ΒΕΔͱɺa ͱ b ͷϊʔυ͕ c ͷϊʔυʹःஅ͞Εͯ a ͱ b ͸ಠཱʹͳΔɻ ͜ͷΑ͏ͳϊʔυ c ͸ a ͔Β b ΁ͷܦ࿏ʹؔͯ͠ tail-to-tail Ͱ͋Δͱݴ ΘΕΔɻ 19 / 74
20. 2-2-1. 3 ͭͷάϥϑͷྫ ࣍ʹҎԼͷάϥϑΛߟ͑Δɻ ͜ͷάϥϑʹରԠ͢Δಉ࣌෼෍͸ p(a, b, c) = p(a)p(c|a)p(b|c) (2.12) ͱͳΔɻ ͜ͷಉ࣌෼෍Ͱ͸ɺ྆ล p(a) ͰׂΓɺc Ͱ࿨ΛͱΔͱɺҎԼͷੑ࣭͕੒ ཱ͢Δɻ ∑ c p(a, b, c) p(a) = ∑ c p(c|a)p(b|c) → p(a, b) p(a) = ∑ c p(c|a)p(b|c) →p(b|a) = ∑ c p(c|a)p(b|c) (2.13) 20 / 74
21. 2-2-1. 3 ͭͷάϥϑͷྫ લͷྫͱಉ༷ʹɺ·ͣ͸ a ͱ b ͷಠཱੑΛௐ΂Δɻ (2.12) ͷ྆ลΛ c ͰपลԽ͢Δͱ ((2.13) Λ༻͍Δ) p(a, b) = p(a) ∑ c p(c|a)p(b|c) = p(a)p(b|a) ̸= p(a)p(b) (2.14) ͱͳΓɺp(a, b) ͸ p(a) ͱ p(b) ͷੵʹ෼ղͰ͖ͳ͍ͷͰಠཱͰͳ͍ɻ a  |= b | ∅ (2.15) ͦΕͰ͸࣍ʹɺc Λ৚͚݅ͭΔͨΊʹ (2.12) ͷ྆ลΛ p(c) ͰׂΔͱ p(a, b|c) = p(a, b, c) p(c) = p(a)p(c|a)p(b|c) p(c) = p(a|c)p(b|c) (2.16) ͱͳΔɻ ͜͜ͰɺϕΠζͷఆཧ (1.9) p(a|c) = p(a)p(c|a) p(c) (2.17) Λ΋͍ͪͨɻ 21 / 74
22. 2-2-1. 3 ͭͷάϥϑͷྫ (2.16) ΑΓɺ৚݅෇͖ಠཱੑ a |= b | c (2.18) Λຬͨ͢͜ͱ͕Θ͔Δɻ (2.15) ͱ (2.18) ͷ 2 ͭͷੑ࣭ΛάϥϑΛ࢖ͬͯߟ࡯͢Δͱɺલͷྫͱ ಉ༷ʹɺc ͕༩͑ΒΕ͍ͯͳ͍ͱ͖͸ a ͱ b ͷϊʔυ͕ܨ͕͍ͬͯͯɺ a ͱ b ͸ಠཱͰͳ͍ɻ ͨͩ͠ɺc ͕༩͑ΒΕΔͱɺa ͱ b ͷϊʔυ͕ c ͷϊʔυʹःஅ͞Εͯ a ͱ b ͸ಠཱʹͳΔɻ ͜ͷΑ͏ͳϊʔυ c ͸ a ͔Β b ΁ͷܦ࿏ʹؔͯ͠ head-to-tail Ͱ͋Δͱ ݴΘΕΔɻ 22 / 74
23. 2-2-1. 3 ͭͷάϥϑͷྫ ͦΕͰ͸࠷ޙʹɺҎԼͷάϥϑΛߟ͑Δɻ ͜ͷάϥϑʹରԠ͢Δಉ࣌෼෍͸ p(a, b, c) = p(a)p(b)p(c|a, b) (2.19) ͱͳΔɻ 23 / 74
24. 2-2-1. 3 ͭͷάϥϑͷྫ ·ͣ͸ a ͱ b ͷಠཱੑΛௐ΂Δɻ (2.19) ͷ྆ลΛ c ͰपลԽ͢Δͱ p(a, b) = p(a)p(b) ∑ c p(c|a, b) = p(a)p(b) (2.20) ͱͳΓɺp(a, b) ͸ p(a) ͱ p(b) ͷੵʹ෼ղͰ͖ΔͷͰಠཱͰ͋Δɻ a |= b | ∅ (2.21) ͦΕͰ͸࣍ʹɺc Λ৚͚݅ͭΔͨΊʹ (2.19) ͷ྆ลΛ p(c) ͰׂΔͱ p(a, b|c) = p(a, b, c) p(c) = p(a)p(b)p(c|a, b) p(c) ̸= p(a|c)p(b|c) (2.22) ͱͳΓɺ৚݅෇͖ಠཱੑΛຬͨ͞ͳ͍ɻ a  |= b | c (2.23) 24 / 74
25. 2-2-1. 3 ͭͷάϥϑͷྫ ࠷ޙͷྫ͸લͷ 2 ྫͱ൓ରͷৼΔ෣͍Λ͢Δɻ ͭ·Γɺc ͕༩͑ΒΕ͍ͯͳ͍ͱ͖͸ a ͱ b ͷϊʔυΛ݁Ϳܦ࿏͕ःஅ ͞Ε͍ͯͯɺa ͱ b ͸ಠཱͰ͋Δɻ c ͕༩͑ΒΕΔͱɺa ͱ b ͷϊʔυΛ݁Ϳܦ࿏ͷ c ͷϊʔυʹΑΔःஅ ͕ղ͔Εɺa ͱ b ͷؒʹґଘؔ܎͕Ͱ͖Δɻ ͜ͷΑ͏ͳϊʔυ c ͸ a ͔Β b ΁ͷܦ࿏ʹؔͯ͠ head-to-head Ͱ͋Δ ͱݴΘΕΔɻ 25 / 74
26. 2-2-1. 3 ͭͷάϥϑͷྫ ·ͨɺҎԼͷάϥϑͰϊʔυ e ͸ head-to-head Ͱ͋Γɺe ͕༩͑ΒΕͨ Β a ͱ f ͸ಠཱͰͳ͘ͳΔɻ ͜͜Ͱॏཁͳੑ࣭ͱͯ͠ɺe ͕༩͑ΒΕ͍ͯͳͯ͘΋ɺͦͷࢠϊʔυͰ ͋Δϊʔυ c(a ΍ f ͷࢠଙϊʔυͱݺͿɻ) ͕༩͑ΒΕ͍ͯͨͱͯ͠΋ a ͱ f ͸ಠཱͰͳ͘ͳΔ͜ͱ͕Θ͔Δɻ(PRML ԋश 8.10) 26 / 74
27. 2-2-2. ༗޲෼཭ (D ෼཭) 3 छྨͷάϥϑಠཱੑΛ༻͍ͯɺάϥϑͷ༗޲෼཭ (D ෼཭) Λಋೖ ͢Δɻ A, B ͓Αͼ C Λॏෳ͠ͳ͍೚ҙͷϊʔυͷू߹ͱ͢Δɻ A ʹଐ͢Δ೚ҙͷϊʔυ͔Β B ʹଐ͢Δ೚ҙͷϊʔυ΁ͷܦ࿏͕ҎԼ ͷͲͪΒ͔ͷ৚݅Λຬͨ͢ͱ͖ɺͦͷܦ࿏͸ःஅ͞Ε͍ͯΔɻ (a) ू߹ C ʹؚ·ΕΔϊʔυͰ͋Γɺܦ࿏ʹؚ·ΕΔ໼ҹ͕ ͦ͜Ͱ head-to-tail ΋͘͠͸ tail-to-tail Ͱ͋Δ (b) ܦ࿏ʹؚ·ΕΔ໼ҹ͕ͦͷϊʔυͰ head-to-head Ͱ͋ Γɺͦͷϊʔυ΍ͦͷࢠଙ͕ू߹ C ʹؚ·Εͳ͍ ͢΂ͯͷܦ࿏͕ःஅ͞Ε͍ͯΕ͹ɺA ͸ C ʹΑΓ B ͔Β༗ޮ෼཭͞Ε ͍ͯΔͱ͍͍ɺ A |= B | C (2.24) ͱද͢ɻ 27 / 74
28. 2-2-2. ༗޲෼཭ (D ෼཭) ͨͱ͑͹ɺҎԼͷਤͰϊʔυͷू߹ A = {a}ɺB = {b, f}ɺC = {c, e} ͱ͢Δͱɺ A  |= B | C (2.25) Ͱ͋Δɻ head-to-head ϊʔυͰ͋Δ e ͱࢠଙϊʔυ c ͕ϊʔυͷू߹ C ʹؚ· ΕΔ͔ΒͰ͋Δɻ 28 / 74
29. 2-2-2. ༗޲෼཭ (D ෼཭) ༗޲άϥϑͷ࠷ޙʹϚϧίϑϒϥϯέοτͷ֓೦ʹ͍ͭͯߟ͑Δɻ ·ͣઃఆͱͯ͠ɺD ݸͷϊʔυΛ࣋ͭ༗ޮάϥϑͰදݱ͞ΕΔಉ࣌෼ ෍ p(x1 , · · · , xD ) Λ༻ҙͯ͠ɺ͜ͷಉ࣌෼෍͔Β xi Ҏ֎ͷม਺ xj̸=i ͕ ༩͑ΒΕͨͱ͖ͷ xi ͷ৚݅෇͖෼෍ p(xi |{xj }j̸=i ) Λܭࢉ͢Δͱɺ p(xi |{xj }j̸=i ) = p(x1 , · · · , xD ) ∫ p(x1 , · · · , xD ) dxi = ∏ k=1 p(xk |pak ) ∫ ∏ k=1 p(xk |pak ) dxi (2.26) ͱͳΔɻ 29 / 74
30. 2-2-2. ༗޲෼཭ (D ෼཭) p(xk |pak ) ͷதͰɺk = i ΋͘͠͸ϊʔυͷू߹ pak ͷதʹ xi ؚΉΑ͏ ͳ k Ҏ֎ͷҼࢠ͸ɺ(2.26) ͷ෼฼ͷੵ෼ͷ֎ʹग़ͯɺ෼ࢠͱΩϟϯηϧ ͞ΕΔɻ ͨͱ͑͹ྫͱͯ͠ɺҎԼͷάϥϑͷಉ࣌෼෍͸ (2.2) p(x1 )p(x2 )p(x3 )p(x4 |x1 , x2 , x3 )p(x5 |x1 , x3 )p(x6 |x4 )p(x7 |x4 , x5 ) (2.27) Ͱ͋Δɻ 30 / 74
31. 2-2-2. ༗޲෼཭ (D ෼཭) x1 ͷ৚݅෇͖֬཰ p(x1 |x2 , · · · , x7 ) ͸ p(x1|x2, · · · , x7) = p(x1)p(x2)p(x3)p(x4|x1, x2, x3)p(x5|x1, x3)p(x6|x4)p(x7|x4, x5) ∫ p(x1)p(x2)p(x3)p(x4|x1, x2, x3)p(x5|x1, x3)p(x6|x4)p(x7|x4, x5) dx1 = p(x1)p(x2)p(x3)p(x4|x1, x2, x3)p(x5|x1, x3)p(x6|x4)p(x7|x4, x5) p(x2)p(x3)p(x6|x4)p(x7|x4, x5) ∫ p(x1)p(x4|x1, x2, x3)p(x5|x1, x3) dx1 = p(x1)p(x4|x1, x2, x3)p(x5|x1, x3) ∫ p(x1)p(x4|x1, x2, x3)p(x5|x1, x3) dx1 (2.28) ͱͳΔɻ ͭ·Γɺk = i(= 1) ͷҼࢠ p(x1 ) ͱϊʔυͷू߹ pak ͷதʹ xi (= x1 ) ؚΉΑ͏ͳ k(= 4, 5) Ҏ֎ͷҼࢠ͸෼฼ͷੵ෼ͷ֎ʹग़ͯɺ෼ࢠͱΩϟ ϯηϧ͞ΕΔ͜ͱ͕Θ͔Δɻ 31 / 74
32. 2-2-2. ༗޲෼཭ (D ෼཭) ͭ·Γ͜ͷྫͰ͸ɺ(2.28) ΑΓɺ৚݅෇͖֬཰ p(x1 |x2 , · · · , x7 ) ͷܭࢉ ʹ͸ x1 ࣗ਎ɺx1 ͷࢠϊʔυ x4 , x5 ɺͦͷࢠϊʔυ x4 , x5 ͷڞಉ਌ (x4 , x5 ͷ਌ϊʔυͰ x1 Ҏ֎ͷϊʔυ) Ͱ͋Δ x2 , x3 ͷΈ͕ؔ܎͢Δɻ ڞಉ਌ x3 ͕د༩͢Δ͜ͱ͸ɺࢠϊʔυ x5 ͕ x1 ͔Β x3 ΁ͷܦ࿏Ͱ head-to-head Ͱ͋Δ͜ͱ͕ཧ༝Ͱ͋Δɻ ΋͠ɺhead-to-head Ͱͳ͍৔߹͸ (tail-to-tail ͷͱ͖ͳͲ) ৚݅෇͖֬཰ p(x1 |x2 , · · · , x7 ) ͷܭࢉʹ x3 ͸د༩͠ͳ͍͜ͱ͕Θ͔Δɻ 32 / 74
33. 2-2-2. ༗޲෼཭ (D ෼཭) Ұൠ࿦ʹ໭Δͱɺ(2.26) ΑΓɺxi ͷ৚݅෇͖෼෍ p(xi |{xj }j̸=i ) ʹ͸Ҏ ԼͷਤͷϊʔυͷΈ͕د༩͢Δɻ ͜ͷϊʔυͷू߹ΛϚϧίϑϒϥϯέοτͱ͍͏ɻ 33 / 74
34. 2-3. Ϛϧίϑ֬཰৔ ͜͜·Ͱ͸ϦϯΫʹ޲͖͕͋ΓɺͦͷϦϯΫͰ݁͹ΕͨΒ਌ϊʔυͱࢠ ϊʔυͱ͍͏ඇରশͳؔ܎ੑ͕ੜ੒͞ΕΔɺ༗޲άϥϑΛߟ͖͑ͯͨɻ ͜ͷ༗޲ੑʹΑΓɺhead-to-head ͷΑ͏ͳಛघͳݱ৅͕ൃੜ͠ɺෳࡶͳ ৚݅෇͖ಠཱੑΛҾ͖ى͍ͯͨ͜͠ɻ ͜͜Ͱ͸ɺϦϯΫʹ޲͖Λͳ͘͠ɺͦͷϦϯΫͰ݁͹ΕͨϊʔυΛฏ౳ ʹѻ͏͜ͱʹΑΓɺ༗޲άϥϑΑΓγϯϓϧͳ৚݅෇͖ಠཱੑΛ΋ͭϚ ϧίϑ֬཰৔ (ແ޲άϥϑΟΧϧϞσϧ) ʹ͍ͭͯٞ࿦͢Δɻ 34 / 74
35. 2-3-1. ৚݅෇͖ಠཱੑ ༗޲άϥϑͰ͸ɺ֬཰෼෍ͷ৚݅෇͖ಠཱੑ͔ΒରԠ͢Δάϥϑͷ৚݅ ෇͖ಠཱੑΛಋग़͕ͨ͠ɺແ޲άϥϑͰ͸άϥϑͷ৚݅෇͖ಠཱੑΛఆ ٛ͢Δͱ͜Ζ͔Β࢝ΊΔɻ ແ޲άϥϑͰɺA, B ͓Αͼ C Λॏෳ͠ͳ͍೚ҙͷϊʔυͷू߹ͱ ͢Δɻ A ʹଐ͢Δ೚ҙͷϊʔυ͔Β B ʹଐ͢Δ೚ҙͷϊʔυ΁ͷ͢΂ͯͷܦ ࿏͕৚݅ ▶ C ʹؚ·ΕΔϊʔυͷগͳ͘ͱ΋ 1 ͭΛඞͣ௨Δ Λຬͨ͢ͱ͖ɺҎԼͷ৚݅෇͖ಠཱੑ͸ຬͨ͞ΕΔͱ͢Δɻ A |= B | C (2.29) 35 / 74
36. 2-3-1. ৚݅෇͖ಠཱੑ ͨͱ͑͹ɺҎԼͷΑ͏ͳάϥϑ͸ (2.29) Λຬͨ͢ɻ ༗޲άϥϑͰ͸ɺA ͱ B ͷϊʔυ͔Β C ͷϊʔυ΁ͷϦϯΫ͕ head-to-head Ͱଘࡏ͍ͯ͠Δͱ͖͸ɺ(2.29) ͸੒ཱ͠ͳ͔ͬͨɻ 36 / 74
37. 2-3-1. ৚݅෇͖ಠཱੑ ·ͨɺϚϧίϑϒϥϯέοτ΋ඇৗʹ୯७ͰɺҎԼͷਤͷ xi (നͷϊʔ υ) ͷ৚݅෇͖֬཰͸ྡ઀ͨ͠ϊʔυͷΈد༩͢Δɻ 37 / 74
38. 2-3-2. ෼ղಛੑ Ҏ্ͷ৚݅෇͖ಠཱੑΑΓɺάϥϑʹରԠ͢Δ֬཰෼෍ͷ෼ղͷੑ࣭ʹ ͍ͭͯߟ͑Δɻ ·ͣɺ௚઀ϦϯΫͰܨ͕͍ͬͯͳ͍ 2 ͭͷϊʔυ xi ͱ xj ʹ͍ͭͯߟ ͑Δɻ ϊʔυ xi ͷΈΛཁૉʹ࣋ͭϊʔυू߹Λ A ͱ͠ɺϊʔυ xj ͷΈΛཁ ૉʹ࣋ͭϊʔυू߹Λ B ͱ͠ɺଞͷ͢΂ͯͷϊʔυΛཁૉʹ࣋ͭϊʔ υू߹Λ C ͱ͢Δͱɺxi ͱ xj ͸௚઀ϦϯΫͰܨ͕͍ͬͯͳ͍ͷͰɺxi ͔Β xj ΁ͷܦ࿏͸ C ʹؚ·ΕΔϊʔυΛܦ༝͢Δ͔͠ͳ͍ɻɹ Αͬͯɺ৚݅෇͖ಠཱ A |= B | C (2.30) Λຬͨ͠ɺରԠ͢Δ৚݅෇͖֬཰͸ p(xi , xj |{xl }l̸=i,j ) = p(xi |{xl }l̸=i,j )p(xj |{xl }l̸=i,j ) (2.31) Λຬͨ͢ɻ ͭ·Γɺ௚઀ϦϯΫͰܨ͕͍ͬͯͳ͍ϊʔυม਺ಉ࢜͸ಉ͡Ҽࢠʹؚ· Εͳ͍Α͏ʹҼ਺෼ղ͞ΕΔɻ 38 / 74
39. 2-3-2. ෼ղಛੑ ͜͜ͰɺΫϦʔΫͱ͍͏֓೦Λಋೖ͢ΔͱศརͰ͋Δɻ ΫϦʔΫͱ͸ɺ͢΂ͯͷϊʔυͷ૊ʹϦϯΫ͕ଘࡏ͢ΔΑ͏ͳάϥϑͷ ෦෼ू߹Ͱ͋Δɻ ͭ·ΓɺΫϦʔΫͷϊʔυू߹͸શ݁߹Ͱ͋Δɻ ·ͨɺ೚ҙͷϊʔυΛҰͭͰ΋Ճ͑ͨΒɺΫϦʔΫͰ͸ͳ͘ͳͬͯ͠· ͏Α͏ͳΫϦʔΫΛۃେΫϦʔΫͱݺͿɻ 39 / 74
40. 2-3-2. ෼ղಛੑ ۩ମྫͱͯ͠ɺҎԼͷແ޲άϥϑΛߟ͑Δɻ ͜ͷάϥϑ͸ 5 ͭͷΫϦʔΫ ({x1 , x2 }, {x1 , x3 }, {x2 , x3 }, {x2 , x4 }, {x3 , x4 }) ͱ 2 ͭͷۃେΫϦʔΫ ({x1 , x2 , x3 }, {x2 , x3 , x4 },) Λ࣋ͭɻ 40 / 74
41. 2-3-2. ෼ղಛੑ Ҏ্ͷٞ࿦ΑΓɺແ޲άϥϑͷಉ࣌෼෍͸ۃେΫϦʔΫʹଐ͢Δม਺ू ߹ͷؔ਺ͷੵͰ͔͚Δɻ ͨͱ͑͹ɺҎԼͷྫͰ͸ p(x1 , x2 , x3 , x4 ) = f(x1 , x2 , x3 )g(x2 , x3 , x4 ) (2.32) ͱॻ͚Δɻ 41 / 74
42. 2-3-2. ෼ղಛੑ ҰൠతʹɺۃେΫϦʔΫΛ C ͱॻ͖ɺͦͷۃେΫϦʔΫʹଐ͢Δม਺ ू߹Λ xC ͱॻ͘ͱɺಉ࣌෼෍ؔ਺ p(x) ͸ p(x) = 1 Z ∏ C ψC (xC ) (2.33) ͱͳΔɻ ͜͜Ͱɺ ∏ C ͸͢΂ͯͷۃେΫϦʔΫͰͷੵΛද͠ɺZ ͸ن֨Խఆ਺Ͱ Z = ∑ x ∏ C ψC (xC ) (2.34) Ͱఆٛ͞ΕΔɻ 42 / 74
43. 2-3-3. ༗޲άϥϑͱͷؔ܎ ༗޲άϥϑͱແ޲άϥϑͷ 2 छྨͷάϥϑΛಋೖͨ͠ͷͰɺ͜ΕΒͷά ϥϑͷؔ܎Λߟ͑Δɻ ༗޲άϥϑͰهड़͞ΕͨϞσϧΛແ޲άϥϑʹม׵͢Δ໰୊Λߟ͑Δɻ ·ͣɺ۩ମྫͱͯ͠ҎԼͷ༗޲άϥϑʹରԠ͢Δಉ࣌෼෍ؔ਺͸ҎԼͷ Α͏ʹͳΔɻ p(x) = p(x1 )p(x2 |x1 ) · · · p(xN−1 |xN−2 )p(xN |xN−1 ) (2.35) 43 / 74
44. 2-3-3. ༗޲άϥϑͱͷؔ܎ (2.35) ͷӈลΛɺ(2.33) ͷΑ͏ʹۃେΫϦʔΫͷੵͰද͢ͱɺແ޲άϥ ϑͷಉ࣌෼෍ؔ਺͸ p(x) = 1 Z ψ1,2 (x1 , x2 )ψ2,3 (x2 , x3 ) · · · ψN,N−1 (xN , xN−1 ) (2.36) ψ1,2 (x1 , x2 ) =p(x1 )p(x2 |x1 ) ψ2,3 (x2 , x3 ) =p(x3 |x2 ) . . . ψN,N−1 (xN , xN−1 ) =p(xN |xN−1 ) (2.37) ͱͳΔɻ͜͜ͰɺZ = 1 Ͱ͋Δ͜ͱʹ஫ҙɻ (2.36) ʹରԠ͢Δແ޲άϥϑ͸ҎԼͰ͋Γɺ༗޲άϥϑͱܗ͸มΘΒͳ ͍ɻ(ਤͷ xN ͱ xN−1 ͷҐஔ͸ٯͰ͢) 44 / 74
45. 2-3-3. ༗޲άϥϑͱͷؔ܎ ࣍ʹ΋͏গ͠ෳࡶͳྫͱͯ͠ɺҎԼͷΑ͏ͳ༗޲άϥϑͱ౳Ձͳແ޲ά ϥϑΛ୳͢ɻ ͜ͷ༗޲άϥϑʹରԠ͢Δ֬཰෼෍͸ p(x) = p(x1 )p(x2 )p(x3 )p(x4 |x1 , x2 , x3 ) (2.38) Ͱ͋Δɻ (2.38) ͷӈลΛɺ(2.33) ͷΑ͏ʹۃେΫϦʔΫͷੵͰදͦ͏ͱ͢Δͱɺ Ҽࢠ p(x4 |x1 , x2 , x3 ) ͕ଘࡏ͢Δ͍ͤͰۃେΫϦʔΫ͸ {x1 , x2 , x3 , x4 } ͷ 1 ͭͰ͋Δ͜ͱ͕Θ͔Δɻ 45 / 74
46. 2-3-3. ༗޲άϥϑͱͷؔ܎ ΑͬͯɺۃେΫϦʔΫ͸શϊʔυͰ͋ΓɺۃେΫϦʔΫ͕શ݁߹ͳͷ Ͱɺແ޲άϥϑ͸ҎԼͷΑ͏ͳશ݁߹άϥϑͰ͋Δ͜ͱ͕Θ͔Δɻ ͭ·Γɺϊʔυ x4 ͷ਌ಉ࢜Λ݁߹ͤ͞Ε͹ྑ͍ɻ(ϞϥϧԽ) 46 / 74
47. 2-4. άϥϑΟΧϧϞσϧʹ͓͚Δਪ࿦ Ҏ্Ͱड़΂͖ͯͨ஌ࣝΛ༻͍ͯɺάϥϑΟΧϧϞσϧΛ༻͍ͨਪ࿦Λ ߦ͏ɻ ۩ମతʹ͸ɺಉ࣌෼෍ؔ਺͔Βपล෼෍ΛٻΊΔͨΊʹάϥϑͷߏ଄Λ ༻͍Δɻ 47 / 74
48. 2-4-1. ࿈࠯ʹ͓͚Δਪ࿦ ·ͣɺ࠷΋؆୯ͳҎԼͷ࿈࠯ʹ͍ͭͯߟ͑Δɻ ͜ͷάϥϑʹରԠ͢Δಉ࣌෼෍ؔ਺͸ (2.36) ΑΓɺ p(x) = 1 Z ψ1,2 (x1 , x2 )ψ2,3 (x2 , x3 ) · · · ψN,N−1 (xN , xN−1 ) (2.39) ͱͳΔɻ ͜͜Ͱɺxn ͸ K ঢ়ଶม਺Ͱ͋Δɻ ͜ͷಉ࣌෼෍ؔ਺͔Βɺ͋Δϊʔυ xn ͷपล෼෍ p(xn ) ΛٻΊΔ͜ͱ Λߟ͑Δɻ 48 / 74
49. 2-4-1. ࿈࠯ʹ͓͚Δਪ࿦ ࠷΋۪௚ͳํ๏͸ɺ(1.7) ͷՃ๏ఆཧΑΓɺҎԼͷΑ͏ʹ xn Ҏ֎ͷঢ়ଶ ม਺ͷ࿨ΛͱΔ͜ͱͰ͋Δɻ p(xn ) = ∑ x1 · · · ∑ xn−1 ∑ xn+1 · · · ∑ xN p(x) (2.40) ͜Εͷܭࢉྔ͸ɺ·ͣ x ͷऔΓ͏Δ͢΂ͯͷ஋ʹର͢Δ p(x) Λ༻ҙ͢ Δɻ(x ͷऔΓ͏Δ͢΂ͯͷ஋͸ KN ݸ͋Δɻ) ͦΕͧΕͷ xn (K ݸ) ʹର͠ɺxn Ҏ֎ͷঢ়ଶม਺ͷ࿨ΛͱΔɻ(࿨Λͱ Δճ਺͸େମ KN−1 ճ) Αͬͯɺ͢΂ͯ xn ͷ஋ʹର͢Δ p(xn ) ΛٻΊΔʹ͸ɺେମ K × KN−1 = KN ճͷ࿨ΛͱΔඞཁ͕͋Δɻ ͳͷͰɺ͜ͷ۪௚ͳํ๏Ͱͷपล෼෍ͷܭࢉྔ͸ O(KN ) Ͱɺঢ়ଶม਺ ͷ਺ N ʹରͯ͠ࢦ਺తʹ૿Ճ͢Δɻ 49 / 74
50. 2-4-1. ࿈࠯ʹ͓͚Δਪ࿦ ࣍ʹɺάϥϑͷੑ࣭Λར༻ͨ͠पล෼෍ͷܭࢉΛߦ͏ɻ άϥϑΑΓɺಉ࣌෼෍͸ p(x) = 1 Z ψ1,2 (x1 , x2 )ψ2,3 (x2 , x3 ) · · · ψN,N−1 (xN , xN−1 ) (2.41) ͱॻ͚͍ͯͨɻ ͜ͷӈลΛ (2.40) ʹ୅ೖ͢Δͱɺ p(x) = 1 Z [ ∑ x1 · · · ∑ xn−1 ψ1,2 (x1 , x2 ) · · · ψn−1,n (xn−1 , xn ) ] =µα(xn) × [ ∑ xn+1 · · · ∑ xN ψn,n+1 (xn , xn+1 ) · · · ψN−1,N (xN−1 , xN ) ] =µβ (xn) (2.42) ͱͳΔɻ 50 / 74
51. 2-4-1. ࿈࠯ʹ͓͚Δਪ࿦ ͞Βʹ µα (xn ) ʹରͯࣜ͠มܗΛߦ͏ͱɺ µα(xn) = ∑ x1 · · · ∑ xn−1 [ ψ1,2(x1, x2) · · · ψn−1,n(xn−1, xn) ] = ∑ xn−1 [ ψn−1,n(xn−1, xn) · · · ∑ x2 [ ψ2,3(x2, x3) ∑ x1 [ ψ1,2(x1, x2) ] =f(x2) ] =g(x3) ] (2.43) ͱͳΓɺx1 ͷ࿨͔Β xn−1 ͷ࿨Λॱ൪ʹ࣮ߦ͍͚ͯ͠͹ܭࢉͰ͖Δɻ ͨͱ͑͹ɺx1 ͷ࿨͸ ψ1,2 (x1 , x2 ) ͷ͢΂ͯͷ x2 (K ݸ) ʹରͯ͠ɺx1 ͷ ࿨ (K ճͷ࿨) ΛऔΕ͹͍͍ɻ ͢΂ͯͷ x2 ʹରͯ͠ɺ࿨ΛऔΒͳͯ͘͸͍͚ͳ͍ཧ༝͸ɺ࣍ʹ x2 ͷ࿨ ΛͱΔͱ͖ʹ͢΂ͯͷ x2 ʹର͢Δ f(x2 ) ͷ஋͕ඞཁ͔ͩΒͰ͋Δɻ ͜ΕΑΓɺx1 ͷ࿨͚ͩͰେମ K2 ճͷ଍͠ࢉΛߦ͏ඞཁ͕͋Δɻ 51 / 74
52. 2-4-1. ࿈࠯ʹ͓͚Δਪ࿦ ͦΕʹΑΓಘΒΕͨ f(x2 ) ͷ஋Λ ψ2,3 (x2 , x3 ) ʹ͔͚ͯɺx2 ͷ࿨ΛͱΔ ͜ͱΛ͢΂ͯͷ x3 ʹରͯ͠ߦ͏ɻ(ܭࢉྔ K2) ͜ΕΛ܁Γฦ͢ͱɺµα (xn ) ͷܭࢉʹ͸ O((n − 1)K2) ͷܭࢉ͕ඞཁͰ ͋Δɻ ಉ༷ʹ µβ (xn ) ͷܭࢉʹ͸ O((N − n)K2) ͷܭࢉ͕ඞཁͰ͋ΔͷͰɺ߹ ܭͰ O(NK2) ͷܭࢉ͕ඞཁɻ(N ≫ 1 ΛԾఆ) ͜Ε͸ܭࢉྔ͕ N ʹରͯ͠ɺઢܗʹ૿͍͚͑ͯͩ͘ͳͷͰɺҎલʹ࿩ ͨ͠΋ͬͱ΋۪௚ͳํ๏ͷͱ͖ͷࢦ਺తͳ૿Ճʹൺ΂Δͱɺඇৗʹେ͖ ͳ N ͷͱ͖ʹάϥϑΛ༻͍ͨपลԽ͕ܭࢉίετͷ໘Ͱ༗ޮͰ͋Δ͜ ͱ͕Θ͔Δɻ 52 / 74
53. 2-4-1. ࿈࠯ʹ͓͚Δਪ࿦ µα (xn ) ΍ µβ (xn ) ͸ϝοηʔδͱݺ͹ΕΔྔͰ͋Δɻ ͜ͷΑ͏ʹݺ͹ΕΔཧ༝Λղઆ͢Δɻ ͜͜ͰࠞཚΛ๷͙ͨΊʹɺ͜Ε·Ͱग़͖ͯͨ µα (xn ) ΍ µβ (xn ) Λ µ(n) α (xn ) ΍ µ(n) β (xn ) ͱॻ͘͜ͱʹ͢Δɻ ·ͣɺµ(n) α (xn ) ͸ µ(n) α (xn) = ∑ xn−1 [ ψn−1,n(xn−1, xn) · · · ∑ x2 [ ψ2,3(x2, x3) ∑ x1 [ ψ1,2(x1, x2) ]]] = ∑ xn−1 [ ψn−1,n(xn−1, xn) ∑ xn−2 [ ψn−2,n−1(xn−2, xn−1) · · · ∑ x1 [ ψ1,2(x1, x2) ]]] = ∑ xn−1 ψn−1,n(xn−1, xn)µ(n−1) α (xn−1) (2.44) ͱ͔͚Δɻ 53 / 74
54. 2-4-1. ࿈࠯ʹ͓͚Δਪ࿦ ͭ·Γɺ͸͡Ίʹ µ(2) α (x2 ) = ∑ x1 ψ1,2 (x1 , x2 ) (2.45) Λܭࢉ͢Ε͹ɺ(2.44) ΑΓ࠶ؼతʹ µ(n) α (xn ) ΛٻΊΔ͜ͱ͕Ͱ͖Δɻ ಉ༷ʹɺµ(n) β (xn ) ʹ͍ͭͯ΋ µ(n) β (xn ) = ∑ xn+1 ψn,n+1 (xn , xn+1 )µ(n+1) β (xn+1 ) (2.46) Λຬͨ͢ͷͰɺ͸͡Ίʹ µ(N−1) β (xN−1 ) = ∑ xN ψN−1,N (xN−1 , xN ) (2.47) Λܭࢉ͢Ε͹ɺ(2.46) ΑΓ࠶ؼతʹ µ(n) β (xn ) ΛٻΊΔ͜ͱ͕Ͱ͖Δɻ 54 / 74
55. 2-4-1. ࿈࠯ʹ͓͚Δਪ࿦ ਤͰද͢ͱɺҎԼͷΑ͏ʹɺµα ͸ӈ޲͖ʹ µβ ͸ࠨ޲͖ʹ࠶ؼతʹܭ ࢉ͞Εɺ͜Ε͕ϊʔυ͔Βϊʔυʹϝοηʔδ͕ૹΒΕ͍ͯΔΑ͏ͳඳ ૾ͳͷͰɺµ(n) α (xn ) ΍ µ(n) β (xn ) ͸ϝοηʔδͱݺ͹ΕΔɻ 55 / 74
56. 2-4-2. ໦ Ҏ্ٞ࿦ͨ͠࿈࠯ͷ࣍ʹෳࡶͳ໦ߏ଄Λ࣋ͭάϥϑʹ͍ͭͯɺपล෼෍ ΛٻΊΔͱ͖ʹάϥϑߏ଄Λར༻͢ΔͱศརͰ͋Δ͜ͱΛղઆ͢Δɻ ·ͣɺ͜͜Ͱ͸໦ߏ଄Λ࣋ͭάϥϑΛಋೖ͢Δɻ ແ޲άϥϑͷ৔߹ɺҎԼͷਤͷ (a) ͷΑ͏ʹɺ೚ҙͷϊʔυͷ૊ͷؒͷ ܦ࿏͕།ҰʹͳΔΑ͏ͳάϥϑͱఆٛ͞ΕΔɻ(ͭ·Γɺϧʔϓ͸࣋ͨ ͳ͍ɻ) ҰํͰɺ༗޲άϥϑͷ৔߹ɺ໦ߏ଄͸ਤͷ (b) ͷΑ͏ʹ਌Λ࣋ͨͳ͍ ϊʔυ͕Ұͭ͋Γ (ࠜϊʔυͱ͍͏)ɺଞͷϊʔυ͸͢΂ͯ਌ΛҰͭͣͭ ࣋ͭάϥϑͰ͋Δɻ ਤ (c) ͷΑ͏ʹ਌Λ 2 ͭ΋ͭϊʔυ͕͋Δ͕ɺ೚ҙͷϊʔυͷ૊ͷؒͷ ܦ࿏͕།ҰͰ͋ΔΑ͏ͳάϥϑΛଟॏ໦ͱ͍͏ɻ 56 / 74
57. 2-4-3. Ҽࢠάϥϑ ࿈࠯άϥϑΑΓ΋͏গ͠ෳࡶʹͳͬͨແ޲໦ɺ༗޲໦ɺଟॏ໦ʹର͠ ͯɺपล෼෍ΛάϥϑΛ༻͍ͯܭࢉ͢Δͱ͖ʹҼࢠάϥϑΛಋೖ͢Δͱ ศརͰ͋Δɻ ·ͣɺ༗ޮάϥϑ΍ແ޲άϥϑʹରԠ͢Δಉ࣌෼෍͸ p(x) ͸ม਺ͷ෦ ෼ू߹ͷΈʹґଘ͢Δؔ਺ͷੵͰ͔͚Δɻ p(x) = ∏ s fs (xs ) (2.48) ͜͜Ͱɺs ͸ม਺ͷ෦෼ू߹Ͱ͋Δɻ ༗޲άϥϑʹ͓͚Δ (2.3) ΍ແ޲άϥϑͷ (2.33) ͸ (2.48) ͷಛผͳ৔߹ Ͱ͋Δɻ 57 / 74
58. 2-4-3. Ҽࢠάϥϑ Ҽࢠάϥϑʹ͓͍ͯ΋ɺ௨ৗͷάϥϑͱಉ༷ʹม਺ϊʔυ͸ؙ͍ԁͰඳ ͔ΕΔɻ ҧ͍͸·ͣɺҼࢠϊʔυͱݺ͹ΕΔϊʔυ͕ଘࡏ͠ɺͦΕ͸ (2.48) ͷ Ҽࢠ fs (xs ) ʹରԠ͢ΔϊʔυͰ͋Δɻ ͞Βʹ௨ৗͷάϥϑͷϦϯΫ͕औΓআ͔ΕɺҼࢠϊʔυͱͦͷҼࢠʹଐ ͢Δ͢΂ͯͷม਺ϊʔυ͕ແ޲ϦϯΫͰ݁͹ΕΔɻ ྫͱͯ͠ɺҎԼͷΑ͏ͳ෼෍Λߟ͑Δɻ p(x) = fa (x1 , x2 )fb (x1 , x2 )fc (x2 , x3 )fd (x3 ) (2.49) ͜ͷ෼෍ʹରԠ͢ΔҼࢠάϥϑ͸ҎԼͰ͋Δɻ 58 / 74
59. 2-4-3. Ҽࢠάϥϑ ·ͨٯʹҼࢠάϥϑ͔ΒରԠ͢Δ෼෍Λ୳͠ग़͢ʹ͸ɺ·ͣҼࢠ͕ fa , fb , fc , fd ͷ 4 ͭଘࡏ͠ɺͦΕͧΕͷҼࢠ͸ରԠ͢ΔҼࢠϊʔυͱϦ ϯΫΛ࣋ͭม਺ʹͷΈґଘ͠ɺ p(x) = fa (x1 , x2 )fb (x1 , x2 )fc (x2 , x3 )fd (x3 ) (2.50) ͱॻ͘ɻ ·ͨɺม਺ϊʔυ (ྫ͑͹ x1 ) ࢹ఺Ͱ͸ɺx1 ͱϦϯΫΛ࣋ͭҼࢠϊʔυ ʹରԠ͢ΔҼࢠ fa , fb ʹͷΈ x1 ͸ґଘ͠ɺx1 ʹґଘ͢Δ fa ͱ fb ͷੵ ͱ x1 ʹґଘ͠ͳ͍Ҽࢠ (fc (x2 , x3 )fd (x3 )) ͷੵͰॻ͚Δɻ 59 / 74
60. 2-4-3. Ҽࢠάϥϑ ࣍ʹҎԼͷ༗޲άϥϑ (a) ΛҼࢠάϥϑʹม׵͢Δͱ͖͸ɺಉ࣌෼෍͸ p(x1 , x2 , x3 ) = p(x1 )p(x2 )p(x3 |x1 , x2 ) (2.51) ͱͳΓɺҼࢠ͸ p(x1 )p(x2 )p(x3 |x1 , x2 ) = f(x1 , x2 , x3 ) ͱॻ͚ɺҼࢠά ϥϑ͸ (b) ͱͳΔɻ ·ͨɺҼࢠ͸ p(x1 )p(x2 )p(x3 |x1 , x2 ) = fa (x1 )fb (x2 )fc (x1 , x2 , x3 ) ͱ΋ ॻ͚ΔͷͰɺҼࢠάϥϑ͸ (c) ͱ΋ͳΔɻ(༗޲άϥϑ΋ରԠ͢ΔҼࢠ άϥϑ͸ҰҙͰ͸ͳ͍) 60 / 74
61. 2-4-3. Ҽࢠάϥϑ ·ͨɺ໦ߏ଄ͷҼࢠάϥϑͰ͍ͭͯɺ༗޲໦΍ແ޲໦͸Ҽࢠάϥϑʹ͢ Δͱ໦ߏ଄ΛอͭΑ͏ʹม׵Ͱ͖Δɻ ଟॏ໦ (a) ͷ৔߹ɺϞϥϧԽ͠ (b)ɺͦΕΛҼࢠάϥϑʹ͢Δͱɺ(c) ͷ Α͏ʹ໦ߏ଄ʹͳΔɻ 61 / 74
62. 2-4-4. ੵ࿨ΞϧΰϦζϜ ͦΕͰ͸ҼࢠάϥϑΛ༻͍ͯɺΑΓޮ཰తʹपลԽΛߦ͏͜ͱ͕Ͱ͖ Δੵ࿨ΞϧΰϦζϜΛಋग़͢Δɻ ͜͜Ͱ͸ɺάϥϑ͸ແ޲໦ɺ༗޲໦ɺଟॏ໦ͷ͍ͣΕ͔Ͱ͋ΔͱԾఆ͢ Δɻ(͜ΕΒͷάϥϑ͸Ҽࢠάϥϑʹม׵͢Δͱɺ໦ߏ଄Ͱ͋Δ͔Β) ͍ͭ΋ͷΑ͏ʹҎԼͷܭࢉͰಉ࣌෼෍ؔ਺ p(x) ͔Βɺx ͷपล෼෍ p(x) ΛٻΊΔ͜ͱΛߟ͑Δɻ p(x) = ∑ x\x p(x) (2.52) ͜͜Ͱɺx\x ͸ x ͷதͰ x Ҏ֎ͷ֬཰ม਺ͷू߹Λද͢ɻ 62 / 74
63. 2-4-4. ੵ࿨ΞϧΰϦζϜ ࠓճ͸ҎԼͷਤͰදͤΔΑ͏ͳάϥϑͷҰ෦ʹ͍ͭͯߟ͑Δɻ άϥϑ͸໦ߏ଄ͳͷͰɺಉ࣌෼෍͸ม਺ϊʔυ x ʹྡ઀͢Δ֤Ҽࢠ ϊʔυ͝ͱʹάϧʔϓ෼͚Ͱ͖ɺҎԼͷΑ͏ʹॻ͚Δɻ p(x) = ∏ s∈ne(x) Fs (x, Xs ) (2.53) ͜͜Ͱɺne(x) ͸ x ʹྡ઀͢Δ͢΂ͯͷҼࢠϊʔυͷू߹ͰɺXs ͸Ҽ ࢠϊʔυ fs Λ௨ͯ͠ x ʹ઀ଓ͞ΕΔ෦෼໦ʹଐ͢Δ͢΂ͯͷม਺ू߹ Ͱ͋Δɻ 63 / 74
64. 2-4-4. ੵ࿨ΞϧΰϦζϜ ͦ͜Ͱɺx ʹྡ઀͢Δ͢΂ͯͷҼࢠϊʔυ s Λ s = 1, · · · , S ͱϥϕϧ෇ ͚͠ɺ(2.53) ͷӈลΛ (2.52) ʹ୅ೖ͢Δͱɺ p(x) = ∑ x\x ∏ s∈ne(x) Fs (x, Xs ) = ∑ X1 · · · ∑ XS S ∏ s=1 Fs (x, Xs ) = S ∏ s=1 [ ∑ Xs Fs (x, Xs ) ] = ∏ s∈ne(x) [ ∑ Xs Fs (x, Xs ) ] = ∏ s∈ne(x) µfs→x (x) (2.54) ͱͳΔɻ ͜͜Ͱɺ µfs→x (x) = ∑ Xs Fs (x, Xs ) (2.55) ͱఆٛͨ͠ɻ 64 / 74
65. 2-4-4. ੵ࿨ΞϧΰϦζϜ ͜͜Ͱɺµfs→x (x) ͸Ҽࢠϊʔυ fs ͔Βϊʔυ x ΁ͷϝοηʔδͱղऍ Ͱ͖Δɻ 65 / 74
66. 2-4-4. ੵ࿨ΞϧΰϦζϜ ·ͨɺFs (x, Xs ) ࣗମ΋ҼࢠάϥϑʹΑͬͯهड़͞ΕΔͷͰɺ(2.48) Α ΓɺҎԼͷΑ͏ͳܗ΁ͷ෼ղ͕ՄೳͰ͋Δɻ Fs (x, Xs ) = fs (x, x1 , · · · , xM )G1 (x1 , Xs1 ) · · · GM (xM , XsM ) (2.56) ͜͜Ͱɺ{x, x1 , · · · , xM } ͸Ҽࢠϊʔυ fs ʹܨ͕Δม਺ϊʔυͰ͋Δɻ ·ͨɺXsm ͸ xm ͕ܨ͕͍ͬͯΔҼࢠϊʔυ fsm ʹܨ͕͍ͬͯΔ xm Ҏ֎ͷม਺ͷू߹Ͱ͋Δɻ 66 / 74
67. 2-4-4. ੵ࿨ΞϧΰϦζϜ ͜ΕΒͷه߸ͷఆٛΑΓɺ(2.56) Λ (2.55) ʹ୅ೖ͢Δͱ µfs→x(x) = ∑ x1 · · · ∑ xM ∑ Xs1 · · · ∑ XsM = ∑ Xs fs(x, x1, · · · , xM )G1(x1, Xs1) · · · GM (xM , XsM ) = ∑ x1 · · · ∑ xM fs(x, x1, · · · , xM ) ∏ m∈ne(fs)\x [ ∑ Xsm Gm(xm, Xsm) ] = ∑ x1 · · · ∑ xM fs(x, x1, · · · , xM ) ∏ m∈ne(fs)\x µxm→fs (xm) (2.57) ͱͳΔɻ ͜͜Ͱɺfs (x, x1 , · · · , xM ) ͸Ҽࢠɺne(fs )\x ͸Ҽࢠϊʔυ fs ʹྡ઀͢ Δ͢΂ͯͷม਺ϊʔυͷத͔Β x Λআ͍ͨ΋ͷͰ͋Γɺ·ͨɺ µxm→fs (xm ) = ∑ Xsm Gm (xm , Xsm ) (2.58) ͱఆٛͨ͠ɻ 67 / 74
68. 2-4-4. ੵ࿨ΞϧΰϦζϜ ͜͜Ͱɺµxm→fs (xm ) ͸ม਺ϊʔυ xm ͔ΒҼࢠϊʔυ fs ΁ͷϝο ηʔδͱղऍͰ͖Δɻ 68 / 74
69. 2-4-4. ੵ࿨ΞϧΰϦζϜ ·ͣ۩ମྫͱͯ͠ɺҎԼͷҼࢠάϥϑͷपล෼෍ p(x2 ) Λߟ͑Δɻ ͜ͷάϥϑͷಉ࣌෼෍͸ p(x) = f(x1 , x2 , x3 ) (2.59) ͱॻ͚Δɻ (2.54) ΑΓɺp(x2 ) ͸ p(x2 ) = µfs→x2 (x2 ) (2.60) ͱͳΔɻ 69 / 74
70. 2-4-4. ੵ࿨ΞϧΰϦζϜ ·ͨ (2.57) ΑΓɺµfs→x2 (x2 ) ͸ µfs→x2 (x2 ) = ∑ x1 ∑ x3 f(x1 , x2 , x3 )µx1→f (x1 )µx3→f (x3 ) (2.61) ͱͳΔɻ Αͬͯɺp(x2 ) ͸ p(x2 ) = ∑ x1 ∑ x3 f(x1 , x2 , x3 )µx1→f (x1 )µx3→f (x3 ) (2.62) Ͱ͋Δ͕ɺ۪௚ʹ (2.59) ͔Βܭࢉ͞Εͨपล෼෍͸ҎԼͷΑ͏ʹͳΔɻ p(x2 ) = ∑ x1 ∑ x3 f(x1 , x2 , x3 ) (2.63) 70 / 74
71. 2-4-4. ੵ࿨ΞϧΰϦζϜ Αͬͯɺ༿ϊʔυ͔ΒҼࢠϊʔυ΁ͷϝοηʔδ͸ µx1→f (x1 ) = µx3→f (x3 ) = 1 (2.64) ͱऔΕ͹ྑ͍͜ͱ͕Θ͔Δɻ 71 / 74
72. 2-4-4. ੵ࿨ΞϧΰϦζϜ ࠷ޙʹҎԼͷάϥϑͷ x2 ͷपล෼෍ p(x2 ) Λੵ࿨ΞϧΰϦζϜΛ༻͍ ͯܭࢉ͢Δɻ ·ͣɺ͜ͷάϥϑͷಉ࣌෼෍͸ p(x) = fa (x1 , x2 )fb (x2 , x3 )fc (x2 , x4 ) (2.65) ͱ͢Δɻ ͜͜Ͱɺp(x) ͸ن֨Խ͞Ε͍ͯͳ͍͕ɺٻΊ͍ͨͷ͸पล֬཰ p(x2 ) Ͱ ͋Γɺಉ࣌෼෍ͷஈ֊Ͱن֨Խ͢Δ (4 ม਺Ͱͷੵ෼Λ͢Δඞཁ͋Γ) ΑΓ΋पลԽ͔ͯ͠Βن֨Խ͢Δ (1 ม਺Ͱͷੵ෼ͰΑ͍) ํ͕ޮ཰త Ͱ͋Δ͔Βɺ·ͣ͸ن֨Խ͞Ε͍ͯͳ͍पล෼෍ p(x2 ) ΛٻΊΔͱ͜Ζ ͔Β࢝ΊΔɻ 72 / 74
73. 2-4-4. ੵ࿨ΞϧΰϦζϜ (2.54) ΑΓɺp(x2 ) ͸ p(x2 ) = µfa→x2 (x2 )µfb→x2 (x2 )µfc→x2 (x2 ) (2.66) ͱͳΔɻ ·ͨɺ(2.57) ΑΓɺҼࢠάϥϑ͔Βม਺ϊʔυ΁ͷϝοηʔδ͸ҎԼͷ Α͏ʹͳΔɻ µfa→x2 (x2 ) = ∑ x1 [ fa (x1 , x2 )µx1→fa (x1 ) ] µfb→x2 (x2 ) = ∑ x3 [ fb (x2 , x3 )µx3→fb (x3 ) ] µfc→x2 (x2 ) = ∑ x4 [ fc (x2 , x4 )µx4→fc (x4 ) ] (2.67) 73 / 74
74. 2-4-4. ੵ࿨ΞϧΰϦζϜ (2.64) ΑΓɺ µx1→fa (x1 ) = 1 µx3→fb (x3 ) = 1 µx4→fc (x4 ) = 1 (2.68) ͱͱΔͱɺ(2.66) ΑΓ p(x2 ) = ∑ x1 [ fa (x1 , x2 ) ] ∑ x3 [ fb (x2 , x3 ) ] ∑ x4 [ fc (x2 , x4 ) ] (2.69) ͱͳΔɻ 74 / 74