非負値行列因子分解 (ベイズ推論による機械学習)

須山さんのベイズ緑本（ベイズ推論による機械学習）の"5-2節 非負値行列因子分解"の輪読資料

Transcript

1. ඇෛ஋ߦྻҼࢠ෼ղ
   /POOFHBUJWF
   .BUSJY
   'BDUPSJ[BUJPO UBOJNV
2. ඇෛ஋ߦྻҼࢠ෼ղͱ͸ w ඇෛ஋Ͱߏ੒͞ΕͨσʔλߦྻΛɼ௿࣍ݩͷͭͷඇෛ஋ߦྻͷੵͰ ۙࣅ͢Δख๏
   w σʔλͷ࣍ݩѹॖ͕Մೳ
   w ॏཁͳಛ௃ͷΈΛநग़͢Δ͜ͱ͕Մೳ
   w ඇෛ஋ߦྻͷͨΊɼղऍੑʹ༏Ε͍ͯΔʢ஋͕େ͖͍΄ͲӨڹ͕ڧ͍ʣ
   w Ԡ༻ઌʢඇෛͰ͋Δσʔλશൠʹద༻Մೳʣ
   w ઢܗ࣍ݩ࡟ݮʹ͋ͬͨΑ͏ͳը૾σʔλͷѹॖ΍อ؅
   w Ի੠σʔλͷ෼཭
   w ߦྻิ׬ʹΑΔਪનΞϧΰϦζϜ΍ɼࣗવݴޠॲཧ 2
3. Ϟσϧ
   .PEFM
4. ඇෛ஋ߦྻ෼ղ /.' ͷϞσϧ 4 X 2 ND⇥N <latexit sha1_base64="z1/IOhquVJrH/6gLv0iX91m7q7Q=">AAACD3icbVC7TsMwFHV4lvIKMLJYVCCmKgEkGCtgYKqKRB9SEyrHdVqrjhPZN0hVlD9g4VdYGECIlZWNv8F9DNByJEvH59yre+8JEsE1OM63tbC4tLyyWlgrrm9sbm3bO7sNHaeKsjqNRaxaAdFMcMnqwEGwVqIYiQLBmsHgauQ3H5jSPJZ3MEyYH5Ge5CGnBIzUsY+8iEA/CLNWjj0u8eQbZNX8PrvGHvCIaVzNO3bJKTtj4HniTkkJTVHr2F9eN6ZpxCRQQbRuu04CfkYUcCpYXvRSzRJCB6TH2oZKYub42fieHB8apYvDWJknAY/V3x0ZibQeRoGpHK2rZ72R+J/XTiG88DMukxSYpJNBYSowxHgUDu5yxSiIoSGEKm52xbRPFKFgIiyaENzZk+dJ46Tsnpad27NS5XIaRwHtowN0jFx0jiroBtVQHVH0iJ7RK3qznqwX6936mJQuWNOePfQH1ucPttGccA==</latexit> %
   
   .
   BOE
   /
   
   . / % % / . . -BUFOU
   'FBUVSF -BUFOU
   
   'FBUVSF 8 )
5. w Ұൠతʹ༻͍ΒΕΔͷ͸ɼϙΞιϯ෼෍ͱΨϯϚ෼෍ʹΑͬͯԾ ఆ͞Εͨ/.'
   w Q 9c8
   ) ͱ΍Δͱɼ8ͱ)ʹΑΔਪ࿦݁ՌʹରԠ͢Δม਺ΛϞσ ϧʹಋೖ͢Δ͜ͱ͕Ͱ͖ͳ͍ͨΊɼ48)ͱͳΔม਺4Λ໌ࣔతʹ Ϟσϧʹಋೖ͢Δ ϕΠζਪ࿦ʹΑΔ/.' 5 p(X, S, W, H) = p(X|S)p(S|W, H)p(W)p(H) <latexit sha1_base64="WTnSCircDJhmCBHV/l4mnIa7Ghs=">AAACenicbZFdS8MwFIbT+jXn19RLEcKGuqGMVgd6Iwy92eVkbh1sZaRZuoWlaUlSYdT9CP+ad/4Sb7ww+xC76YHAk/OeNzk58SJGpbKsD8NcW9/Y3MpsZ3d29/YPcodHLRnGApMmDlko2h6ShFFOmooqRtqRICjwGHG80eNUd16IkDTkz2ocETdAA059ipHSqV7uLSp2A6SGnp+0J1fwhxspdlJcm5TgPUx74GvKVEpJjbS0esZvmbO001ovV7DK1izgX7AXUACLqPdy791+iOOAcIUZkrJjW5FyEyQUxYxMst1YkgjhERqQjkaOAiLdZDa6CTzTmT70Q6EXV3CWTTsSFEg5DjxdOW1RrmrT5H9aJ1b+nZtQHsWKcDy/yI8ZVCGc/gPsU0GwYmMNCAuqe4V4iATCSv9WVg/BXn3yX2hdl+2bsvVUKVQfFuPIgBOQB0Vgg1tQBTVQB02AwadxapwbF8aXmTdL5uW81DQWnmOwFGblG/H+v7A=</latexit> p(S, W, H|X) / p(X|S)p(S|W, H)p(W)p(H) <latexit sha1_base64="YvT2ujV6NM3YGJfzCq1KDgzPu0s=">AAACgXicbVFdT8IwFO2GH4hfqI++NKIJJEo2NNHEF6IvPGIQtgQI6UqHDd3WtJ0Jmfsf/i7f/DPG8qEO8CZNzj3n3NvbW48zKpVlfRpmbmNzazu/U9jd2z84LB4dd2QUC0zaOGKRcD0kCaMhaSuqGHG5ICjwGHG88eNUd16JkDQKn9WEk36ARiH1KUZKU4PiOy/3AqRePD9ppZfwBzsZ3Ejh22/iphXY4yLiKoJ/pW7W0tKWbNeMtNw2a3OWMq0NiiWras0CrgN7AUpgEc1B8aM3jHAckFBhhqTs2hZX/QQJRTEjaaEXS8IRHqMR6WoYooDIfjLbYAovNDOEfiT0CRWcsdmKBAVSTgJPO6cjylVtSv6ndWPl3/UTGvJYkRDPL/JjBvX6pt8Bh1QQrNhEA4QF1bNC/IIEwkp/WkEvwV598jro1Kr2ddV6uinVHxbryINTcAbKwAa3oA4aoAnaAIMv49y4MqpmzqyYllmbW01jUXMClsK8/wZnusIq</latexit> 
6. ิॿม਺4ͷఆٛ w 8<E
   >
   YN
   )<
   O>
   NY
   ͷཁૉੵ
   NY ͷ݁ՌΛ4ͱஔ͘
   w ͭ·Γ8ͱ)ͷߦྻͷཁૉੵͷू߹Λ4ͱஔ͍͍ͯΔ
   w 4ʹରͯ͠ϙΞιϯ෼෍ΛԾఆ͢Δ
   w ·ͨɼ8ͱ)ͷ֤ཁૉʹରͯ͠͸ɼڞ໾ࣄલ෼෍Ͱ͋ΔΨϯϚ෼෍ ΛԾఆ 6 9 ֤ཁૉʹରͯ͠
   ΨϯϚ෼෍ΛԾఆ X I Tʹର͠Ͱ͸ͳ͘
   ֤X
   
   Iʹରͯ͠
   ϙΞιϯ෼෍ΛԾఆ Y
7. 4ͱ8
   )ͷ෼෍ͷԾఆ w 4ͷੜ੒෼෍ΛϙΞιϯ෼෍ͰԾఆ
   w 8
   )ͷੜ੒෼෍ΛΨϯϚ෼෍ͰԾఆ 7
8. 9ͷ໬౓෼෍ w 4ͷ໬౓෼෍ΛҎԼͷσϧλ෼෍ͰԾఆ͢Δ
   w TO
   JO
   4
   ͱ
   YO
   JO
   9͕Ұக͢Ε͹ɼ֬཰ɼͦ͏Ͱͳ͚Ε͹ͷ෼෍ 8 p(S, W, H|X) / p(X|S)p(S|W, H)p(W)p(H) <latexit sha1_base64="YvT2ujV6NM3YGJfzCq1KDgzPu0s=">AAACgXicbVFdT8IwFO2GH4hfqI++NKIJJEo2NNHEF6IvPGIQtgQI6UqHDd3WtJ0Jmfsf/i7f/DPG8qEO8CZNzj3n3NvbW48zKpVlfRpmbmNzazu/U9jd2z84LB4dd2QUC0zaOGKRcD0kCaMhaSuqGHG5ICjwGHG88eNUd16JkDQKn9WEk36ARiH1KUZKU4PiOy/3AqRePD9ppZfwBzsZ3Ejh22/iphXY4yLiKoJ/pW7W0tKWbNeMtNw2a3OWMq0NiiWras0CrgN7AUpgEc1B8aM3jHAckFBhhqTs2hZX/QQJRTEjaaEXS8IRHqMR6WoYooDIfjLbYAovNDOEfiT0CRWcsdmKBAVSTgJPO6cjylVtSv6ndWPl3/UTGvJYkRDPL/JjBvX6pt8Bh1QQrNhEA4QF1bNC/IIEwkp/WkEvwV598jro1Kr2ddV6uinVHxbryINTcAbKwAa3oA4aoAnaAIMv49y4MqpmzqyYllmbW01jUXMClsK8/wZnusIq</latexit>
9. άϥϑΟΧϧϞσϧ w 9
   ؍ଌσʔλɹ4
   9ͷύϥϝʔλʢิॿม਺ʣ
   w 8
   )
   4ͷύϥϝʔλʢ࣮࣭ੜ੒෼෍ͷओͳύϥϝʔλʣ
   w %
   ࣍ݩ਺ɹ/
   σʔλ਺ɹ.
   .'ͷϥϯΫʢ.࣍ݩͰ֤ಛ௃Λදݱʣ 9
10. Ի੠σʔλʹର͢Δ/.'ͷ݁Ռ 10 σʔλ9
    ࣌ؒ
    Y
    प೾਺
    Y
    ڧ͞ ߴप೾ ௿प೾ ߴप೾ͷύλʔϯ
    ௿प೾ͷύλʔϯ εϖΫτϩάϥϜͰද͞ΕͨԻ੠σʔλʹରͯ͠.Ͱ/.'Λద༻ 8
    प೾਺
    Y
    . )
    .
    Y
    प೾਺
11. ม෼ਪ࿦
    7BSJBUJPOBM
    *OGFSFODF
12. ม෼ਪ࿦ʹΑΔࣄޙ෼෍ w ֤ม਺͕ಠཱʹͳΔͱԾఆ্ͨ͠Ͱͷࣄޙ෼෍ΛٻΊΔ
    w ͦΕͧΕͷRΛม෼ਪ࿦ͷެࣜΛ΋ͱʹٻΊΔ
    w ม෼ਪ࿦ͷެࣜ
    
    
    ಉ࣌෼෍ͷظ଴஋
    w ಉ࣌෼෍ 12 p(S, W, H|X) ⇡ q(S)q(W)q(H) <latexit sha1_base64="gvhGC0Vc19Bx0gYHx2Tu/3qY8Jc=">AAACTXicbZFLSwMxEMez9dFaX1WPXoJFaEHKrgp6LHrpsaJ9QLuUbDbbBrO7McmKZd0v6EXw5rfw4kERMdtW6cOBwG/+M0Mm/zicUalM89XILC2vrGZza/n1jc2t7cLOblOGkcCkgUMWiraDJGE0IA1FFSNtLgjyHUZazu1lWm/dEyFpGNyoISe2j/oB9ShGSku9gstLXR+pgePF18kR/OXWFNcS+PiXtJMy7CLORfgA76ZGy1NZayarJeVeoWhWzFHARbAmUASTqPcKL103xJFPAoUZkrJjmVzZMRKKYkaSfDeShCN8i/qkozFAPpF2PHIjgYdacaEXCn0CBUfq9ESMfCmHvqM70xXlfC0V/6t1IuWd2zENeKRIgMcXeRGDKoSptdClgmDFhhoQFlTvCvEACYSV/oC8NsGaf/IiNI8r1knFujotVi8mduTAPjgAJWCBM1AFNVAHDYDBE3gDH+DTeDbejS/je9yaMSYze2AmMtkffNO0Dg==</latexit> p(S, W, H, X) = p(X|S)p(S|W, H)p(W)p(H) <latexit sha1_base64="nWezdERLWFZiHRJWDJ2nPNIiiHY=">AAACenicbZFLSwMxEMez66vW16pHEUKL2qKUXS3oRSh66bGi7S60pWTTbBuafZBkhbLuh/CrefOTePFg+pBuqwOBX+Y/M5nMuBGjQprmp6avrW9sbuW28zu7e/sHxuFRS4Qxx6SJQxZyx0WCMBqQpqSSESfiBPkuI7Y7epzo9ivhgobBixxHpOujQUA9ipFUrp7xHpU6PpJD10ue0yv4y3aG6xl20jK8h4scJ4VvcFGgDLPlMtJyvWyYvXRTWs8omhVzavAvWHMogrk1esZHpx/i2CeBxAwJ0bbMSHYTxCXFjKT5TixIhPAIDUhbYYB8IrrJdHQpPFOePvRCrk4g4dSbzUiQL8TYd1XkpEWxqk2c/2ntWHp33YQGUSxJgGcPeTGDMoSTPcA+5QRLNlaAMKeqV4iHiCMs1bbyagjW6pf/Quu6Yt1UrKdqsfYwH0cOnIACKAEL3IIaqIMGaAIMvrRT7Vy70L71gl7WL2ehujbPOQZLpld/APFIv7E=</latexit>
13. 8ͷࣄޙ෼෍ 13 ୯७ʹ୅ೖ ෆཁͳ߲ΛDPOTU΁ ର਺࿨ʹมܗ ظ଴஋Ͱల։ʁ ཁૉ͝ͱͷܗʹมܗ
    ཁૉ͝ͱʹಠཱͰ͋ΔͨΊ
14. શͯͷࣄޙ෼෍ w ಉ༷ʹܭࢉ͍ͯ͘͠ͱ 14
15. 8ͷࣄޙ෼෍͔Βܭࢉ w ཁૉ͝ͱʹಠཱͰ͋ΔͨΊɼ͋Δཁૉ8E Nʹؔͯ͠ͷΈߟ͑Δ
    w ϙΞιϯ෼෍ͷ߲
    w ΨϯϚ෼෍ͷ߲ 15 lnq(Wd,m) = N X n=1 hlnp(Sd,m,n |Wd,m, Hm,n)iq(S)q(H) + lnp(Wd,m) + const. <latexit sha1_base64="wRd5eY8he/PPRg/xqBowrD6maEQ=">AAAClnicbVFba9swFJa9W5ddmq0vg70cFgYJDcHeBmsfWkrLWJ5GRpemEGdGVuRUVJJdSS4ETT9pf2Zv+zeT44x07Q4IPn3nfOealZxpE0W/g/De/QcPH209bj15+uz5dvvFyzNdVIrQMSl4oc4zrClnko4NM5yel4pikXE6yS5Pav/kmirNCvnNLEs6E3ghWc4INp5K2z9togRwCQ6uupPUzvsgXA8OINGVSK08iN13+8VBwrFccAqb8LJ72oT3QTr4AX/FfRimtiF7kKiVLLVX3URgc5Hl9tSn3/yGrudau2DhRt5NG7DbFCSF1GbgWmm7Ew2ilcFdEK9BB61tlLZ/JfOCVIJKQzjWehpHpZlZrAwjnLpWUmlaYnKJF3TqocSC6pldrdXBW8/MIS+Uf9LAir2psFhovRSZj6yn0bd9Nfk/37Qy+d7MMllWhkrSFMorDqaA+kYwZ4oSw5ceYKKY7xXIBVaYGH/Jegnx7ZHvgrN3g/j9IP76oXN0vF7HFnqN3qAuitFHdISGaITGiAQ7wX5wHJyEr8LD8FP4uQkNg7VmB/1j4egP6W3DsQ==</latexit>
16. ϙΞιϯ෼෍ͷର਺ w ϙΞιϯ෼෍
    w ϙΞιϯ෼෍ͷର਺
    w YΛ4ɼЕΛ)8ʹஔ͖׵͑Ε͹Α͍ 16 Poi(x| ) = x x! e <latexit sha1_base64="hOpCwWDG94WA8rAJPDWQ5fyAQ50=">AAACJXicbVDLSsNAFJ3UV62vqEs3o0WoC0uigi4Uim5cVrAPaNIymUzaoZMHMxNpifkZN/6KGxcWEVz5K07bCNp6YOBwzrncuceJGBXSMD613MLi0vJKfrWwtr6xuaVv79RFGHNMajhkIW86SBBGA1KTVDLSjDhBvsNIw+nfjP3GA+GChsG9HEbE9lE3oB7FSCqpo18mFvdhNaQpLA3gI7SYmnXREbyClscRTjKhPUiTwX4KSTs5/gmlHb1olI0J4DwxM1IEGaodfWS5IY59EkjMkBAt04iknSAuKWYkLVixIBHCfdQlLUUD5BNhJ5MrU3ioFBd6IVcvkHCi/p5IkC/E0HdU0keyJ2a9sfif14qld2EnNIhiSQI8XeTFDMoQjiuDLuUESzZUBGFO1V8h7iFVjlTFFlQJ5uzJ86R+UjZPy+bdWbFyndWRB3vgAJSACc5BBdyCKqgBDJ7AC3gDI+1Ze9XetY9pNKdlM7vgD7Svb/f0pFk=</latexit> lnPoi(x| ) = xln lnx! <latexit sha1_base64="lJngUpMm+3AcX6paJr7B5NpLMGo=">AAACMHicbZDLSgMxFIYzXmu9jbp0c7QIdWGZUUE3QtGFLivYC7SlZNK0Dc1khiQjLWMfyY2PohsFRdz6FKbtiNp6IPDn+88hOb8Xcqa047xYM7Nz8wuLqaX08srq2rq9sVlSQSQJLZKAB7LiYUU5E7Somea0EkqKfY/Tste9GPrlWyoVC8SN7oe07uO2YC1GsDaoYV/GNekDF1AI2ACyPbiDGjfjTbwPZ9CDxB58Uzj4Qb0dc0t4w844OWdUMC3cRGRQUoWG/VhrBiTyqdCEY6WqrhPqeoylZoTTQboWKRpi0sVtWjVSYJ+qejxaeAB7hjShFUhzhIYR/T0RY1+pvu+ZTh/rjpr0hvA/rxrp1mk9ZiKMNBVk/FAr4qADGKYHTSYp0bxvBCaSmb8C6WCJiTYZp00I7uTK06J0mHOPcu71cSZ/nsSRQttoF2WRi05QHl2hAioigu7RE3pFb9aD9Wy9Wx/j1hkrmdlCf8r6/AKiPKbn</latexit>
17. ϙΞιϯ෼෍ͷ߲ͷܭࢉ 17 N X n=1 hlnp(Sd,m,n |Wd,m, Hm,n)iq(S)q(H) <latexit sha1_base64="ngHs6MQ1lKKa+2slCv2LipvELtI=">AAACVXicbVFda9swFJW9rB/Z2qbrY18uDYUEQrDbQvtSKOtLnkZHlqQQZ0ZW5FRUkl1JLgRNf7Ivpf9kL4PZSWBr2guCc8+5B10dJTln2gTBi+d/qH3c2Nzarn/6vLO719j/MtRZoQgdkIxn6jbBmnIm6cAww+ltrigWCaej5P660kePVGmWyR9mntOJwDPJUkawKam4wSNdiNjKy9D9tN8cRBzLGadgIyWAS3CQt/qxnXZAdEA6+AWjZec60IvtkmxDpBa22D60IoHNXZLavmvDv67n2q4eN5pBN1gUvAXhCjTRqm7ixlM0zUghqDSEY63HYZCbicXKMMKpq0eFpjkm93hGxyWUWFA9sYtUHByXzBTSTJVHGliw/zssFlrPRVJOVkvqda0i39PGhUkvJpbJvDBUkuVFacHBZFBFDFOmKDF8XgJMFCt3BXKHFSam/IgqhHD9yW/B8KQbnnbD72fNq6+rOLbQITpCLRSic3SFeugGDRBBT+i353m+9+z98Wv+xnLU91aeA/Sq/L2/zTCxCw==</latexit> lnPoi(x| ) = xln lnx! <latexit sha1_base64="lJngUpMm+3AcX6paJr7B5NpLMGo=">AAACMHicbZDLSgMxFIYzXmu9jbp0c7QIdWGZUUE3QtGFLivYC7SlZNK0Dc1khiQjLWMfyY2PohsFRdz6FKbtiNp6IPDn+88hOb8Xcqa047xYM7Nz8wuLqaX08srq2rq9sVlSQSQJLZKAB7LiYUU5E7Somea0EkqKfY/Tste9GPrlWyoVC8SN7oe07uO2YC1GsDaoYV/GNekDF1AI2ACyPbiDGjfjTbwPZ9CDxB58Uzj4Qb0dc0t4w844OWdUMC3cRGRQUoWG/VhrBiTyqdCEY6WqrhPqeoylZoTTQboWKRpi0sVtWjVSYJ+qejxaeAB7hjShFUhzhIYR/T0RY1+pvu+ZTh/rjpr0hvA/rxrp1mk9ZiKMNBVk/FAr4qADGKYHTSYp0bxvBCaSmb8C6WCJiTYZp00I7uTK06J0mHOPcu71cSZ/nsSRQttoF2WRi05QHl2hAioigu7RE3pFb9aD9Wy9Wx/j1hkrmdlCf8r6/AKiPKbn</latexit> ΑΓɼ Yʹ4ɼЕʹ)8Λ୅ೖ ର਺࿨ʹมܗ ෆཁͳ߲ΛDPOTU΁ ظ଴஋ల։
18. 8ͷࣄޙ෼෍͔Βܭࢉ w ཁૉ͝ͱʹಠཱͰ͋ΔͨΊɼ͋Δཁૉ8E Nʹؔͯ͠ͷΈߟ͑Δ
    w ϙΞιϯ෼෍ͷ߲
    w ΨϯϚ෼෍ͷ߲ 18 lnq(Wd,m) = N X n=1 hlnp(Sd,m,n |Wd,m, Hm,n)iq(S)q(H) + lnp(Wd,m) + const. <latexit sha1_base64="wRd5eY8he/PPRg/xqBowrD6maEQ=">AAAClnicbVFba9swFJa9W5ddmq0vg70cFgYJDcHeBmsfWkrLWJ5GRpemEGdGVuRUVJJdSS4ETT9pf2Zv+zeT44x07Q4IPn3nfOealZxpE0W/g/De/QcPH209bj15+uz5dvvFyzNdVIrQMSl4oc4zrClnko4NM5yel4pikXE6yS5Pav/kmirNCvnNLEs6E3ghWc4INp5K2z9togRwCQ6uupPUzvsgXA8OINGVSK08iN13+8VBwrFccAqb8LJ72oT3QTr4AX/FfRimtiF7kKiVLLVX3URgc5Hl9tSn3/yGrudau2DhRt5NG7DbFCSF1GbgWmm7Ew2ilcFdEK9BB61tlLZ/JfOCVIJKQzjWehpHpZlZrAwjnLpWUmlaYnKJF3TqocSC6pldrdXBW8/MIS+Uf9LAir2psFhovRSZj6yn0bd9Nfk/37Qy+d7MMllWhkrSFMorDqaA+kYwZ4oSw5ceYKKY7xXIBVaYGH/Jegnx7ZHvgrN3g/j9IP76oXN0vF7HFnqN3qAuitFHdISGaITGiAQ7wX5wHJyEr8LD8FP4uQkNg7VmB/1j4egP6W3DsQ==</latexit>
19. 8ͷࣄޙ෼෍͔Βܭࢉ w ཁૉ͝ͱʹಠཱͰ͋ΔͨΊɼ͋Δཁૉ8E Nʹؔͯ͠ͷΈߟ͑Δ
    w ϙΞιϯ෼෍ͷ߲
    w ΨϯϚ෼෍ͷ߲ 19 lnq(Wd,m) = N X n=1 hlnp(Sd,m,n |Wd,m, Hm,n)iq(S)q(H) + lnp(Wd,m) + const. <latexit sha1_base64="wRd5eY8he/PPRg/xqBowrD6maEQ=">AAAClnicbVFba9swFJa9W5ddmq0vg70cFgYJDcHeBmsfWkrLWJ5GRpemEGdGVuRUVJJdSS4ETT9pf2Zv+zeT44x07Q4IPn3nfOealZxpE0W/g/De/QcPH209bj15+uz5dvvFyzNdVIrQMSl4oc4zrClnko4NM5yel4pikXE6yS5Pav/kmirNCvnNLEs6E3ghWc4INp5K2z9togRwCQ6uupPUzvsgXA8OINGVSK08iN13+8VBwrFccAqb8LJ72oT3QTr4AX/FfRimtiF7kKiVLLVX3URgc5Hl9tSn3/yGrudau2DhRt5NG7DbFCSF1GbgWmm7Ew2ilcFdEK9BB61tlLZ/JfOCVIJKQzjWehpHpZlZrAwjnLpWUmlaYnKJF3TqocSC6pldrdXBW8/MIS+Uf9LAir2psFhovRSZj6yn0bd9Nfk/37Qy+d7MMllWhkrSFMorDqaA+kYwZ4oSw5ceYKKY7xXIBVaYGH/Jegnx7ZHvgrN3g/j9IP76oXN0vF7HFnqN3qAuitFHdISGaITGiAQ7wX5wHJyEr8LD8FP4uQkNg7VmB/1j4egP6W3DsQ==</latexit> Є Є
20. ΨϯϚ෼෍ͷର਺ w ΨϯϚ෼෍
    w ΨϯϚ෼෍ͷର਺
    w ЕΛ8E
    Nʹஔ͖׵͑Ε͹Α͍ 20 Gam( |a, b) = CG(a, b) a 1e b <latexit sha1_base64="OQ/Ku0vgxd48Wei1H815ytiOh7s=">AAACNHicbVDLSgMxFM34tr6qLt1cLEIFW2ZU0I0gulBwo2BV6Is7aWqDycyQZIQyzke58UPciOBCEbd+g2k7gloPBA7nnMvNPX4kuDau++yMjI6NT0xOTedmZufmF/KLSxc6jBVlFRqKUF35qJngAasYbgS7ihRD6Qt26d8c9vzLW6Y0D4Nz041YXeJ1wNucorFSM3+S1JSEI5QpFGvCzrUQ7gA3wF+HPThsDmxrDqQs0kiw5KXAGkkJ/G8xbeYLbtntA4aJl5ECyXDazD/WWiGNJQsMFah11XMjU09QGU4FS3O1WLMI6Q1es6qlAUqm60n/6BTWrNKCdqjsCwz01Z8TCUqtu9K3SYmmo/96PfE/rxqb9m494UEUGxbQwaJ2LMCE0GsQWlwxakTXEqSK278C7aBCamzPOVuC9/fkYXKxWfa2yt7ZdmH/IKtjiqyQVVIkHtkh++SYnJIKoeSePJFX8uY8OC/Ou/MxiI442cwy+QXn8wsqaKeP</latexit> lnGam( |a, b) = (a 1)ln b + lnCG(a, b) <latexit sha1_base64="uspTueiQkGOOvKXKs1f0zCTNyfY=">AAACSXicbVBNSwMxFMzW7/pV9ejlYREqatlVQS+C6EGPClaFttS3aarBJLskWaGs/XtevHnzP3jxoIgn03b9diAwzMx7SSaMBTfW9x+83MDg0PDI6Fh+fGJyarowM3tiokRTVqGRiPRZiIYJrljFcivYWawZylCw0/Bqr+ufXjNteKSObTtmdYkXirc4ReukRuE8rWkJQsE+yg6UasKNNhFuAFcgXIJtKCGsQrAEWa4DH5HV8JMuf7l7jR7dd7v6GxqFol/2e4C/JMhIkWQ4bBTua82IJpIpSwUaUw382NZT1JZTwTr5WmJYjPQKL1jVUYWSmXraa6IDi05pQivS7igLPfX7RIrSmLYMXVKivTS/va74n1dNbGurnnIVJ5Yp2r+olQiwEXRrhSbXjFrRdgSp5u6tQC9RI7Wu/LwrIfj95b/kZK0crJeDo43izm5WxyiZJwukRAKySXbIATkkFULJLXkkz+TFu/OevFfvrR/NednMHPmB3MA7sies4Q==</latexit>
21. ΨϯϚ෼෍ͷ߲ 21 lnp(Wd,m) <latexit sha1_base64="Ava8Lkwf9PzeoU/TvC3acbNRNRk=">AAACAHicbVDLSsNAFJ34rPUVdeHCzWARKkhJVNBl0Y3LCvYBbQiTyaQdOjMJMxOhhGz8FTcuFHHrZ7jzb5y0WWjrgQuHc+7l3nuChFGlHefbWlpeWV1br2xUN7e2d3btvf2OilOJSRvHLJa9ACnCqCBtTTUjvUQSxANGusH4tvC7j0QqGosHPUmIx9FQ0IhipI3k24cZHEgOmYA5TOpdPwvPIM9Pq75dcxrOFHCRuCWpgRIt3/4ahDFOOREaM6RU33US7WVIaooZyauDVJEE4TEakr6hAnGivGz6QA5PjBLCKJamhIZT9fdEhrhSEx6YTo70SM17hfif1091dO1lVCSpJgLPFkUpgzqGRRowpJJgzSaGICypuRXiEZIIa5NZEYI7//Ii6Zw33IuGe39Za96UcVTAETgGdeCCK9AEd6AF2gCDHDyDV/BmPVkv1rv1MWtdssqZA/AH1ucP/JWUtw==</latexit> lnGam( |a, b) = (a 1)ln b + lnCG(a, b) <latexit sha1_base64="uspTueiQkGOOvKXKs1f0zCTNyfY=">AAACSXicbVBNSwMxFMzW7/pV9ejlYREqatlVQS+C6EGPClaFttS3aarBJLskWaGs/XtevHnzP3jxoIgn03b9diAwzMx7SSaMBTfW9x+83MDg0PDI6Fh+fGJyarowM3tiokRTVqGRiPRZiIYJrljFcivYWawZylCw0/Bqr+ufXjNteKSObTtmdYkXirc4ReukRuE8rWkJQsE+yg6UasKNNhFuAFcgXIJtKCGsQrAEWa4DH5HV8JMuf7l7jR7dd7v6GxqFol/2e4C/JMhIkWQ4bBTua82IJpIpSwUaUw382NZT1JZTwTr5WmJYjPQKL1jVUYWSmXraa6IDi05pQivS7igLPfX7RIrSmLYMXVKivTS/va74n1dNbGurnnIVJ5Yp2r+olQiwEXRrhSbXjFrRdgSp5u6tQC9RI7Wu/LwrIfj95b/kZK0crJeDo43izm5WxyiZJwukRAKySXbIATkkFULJLXkkz+TFu/OevFfvrR/NednMHPmB3MA7sies4Q==</latexit> ΑΓ ෆཁͳ߲ΛDPOTU΁
22. 8ͷࣄޙ෼෍͔Βܭࢉ w ཁૉ͝ͱʹಠཱͰ͋ΔͨΊɼ͋Δཁૉ8E Nʹؔͯ͠ͷΈߟ͑Δ
    w ϙΞιϯ෼෍ͷ߲
    w ΨϯϚ෼෍ͷ߲ 22 lnq(Wd,m) = N X n=1 hlnp(Sd,m,n |Wd,m, Hm,n)iq(S)q(H) + lnp(Wd,m) + const. <latexit sha1_base64="wRd5eY8he/PPRg/xqBowrD6maEQ=">AAAClnicbVFba9swFJa9W5ddmq0vg70cFgYJDcHeBmsfWkrLWJ5GRpemEGdGVuRUVJJdSS4ETT9pf2Zv+zeT44x07Q4IPn3nfOealZxpE0W/g/De/QcPH209bj15+uz5dvvFyzNdVIrQMSl4oc4zrClnko4NM5yel4pikXE6yS5Pav/kmirNCvnNLEs6E3ghWc4INp5K2z9togRwCQ6uupPUzvsgXA8OINGVSK08iN13+8VBwrFccAqb8LJ72oT3QTr4AX/FfRimtiF7kKiVLLVX3URgc5Hl9tSn3/yGrudau2DhRt5NG7DbFCSF1GbgWmm7Ew2ilcFdEK9BB61tlLZ/JfOCVIJKQzjWehpHpZlZrAwjnLpWUmlaYnKJF3TqocSC6pldrdXBW8/MIS+Uf9LAir2psFhovRSZj6yn0bd9Nfk/37Qy+d7MMllWhkrSFMorDqaA+kYwZ4oSw5ceYKKY7xXIBVaYGH/Jegnx7ZHvgrN3g/j9IP76oXN0vF7HFnqN3qAuitFHdISGaITGiAQ7wX5wHJyEr8LD8FP4uQkNg7VmB/1j4egP6W3DsQ==</latexit>
23. 8ͷࣄޙ෼෍͔Βܭࢉ w ཁૉ͝ͱʹಠཱͰ͋ΔͨΊɼ͋Δཁૉ8E Nʹؔͯ͠ͷΈߟ͑Δ
    w ϙΞιϯ෼෍ͷ߲
    w ΨϯϚ෼෍ͷ߲ 23 lnq(Wd,m) = N X n=1 hlnp(Sd,m,n |Wd,m, Hm,n)iq(S)q(H) + lnp(Wd,m) + const. <latexit sha1_base64="wRd5eY8he/PPRg/xqBowrD6maEQ=">AAAClnicbVFba9swFJa9W5ddmq0vg70cFgYJDcHeBmsfWkrLWJ5GRpemEGdGVuRUVJJdSS4ETT9pf2Zv+zeT44x07Q4IPn3nfOealZxpE0W/g/De/QcPH209bj15+uz5dvvFyzNdVIrQMSl4oc4zrClnko4NM5yel4pikXE6yS5Pav/kmirNCvnNLEs6E3ghWc4INp5K2z9togRwCQ6uupPUzvsgXA8OINGVSK08iN13+8VBwrFccAqb8LJ72oT3QTr4AX/FfRimtiF7kKiVLLVX3URgc5Hl9tSn3/yGrudau2DhRt5NG7DbFCSF1GbgWmm7Ew2ilcFdEK9BB61tlLZ/JfOCVIJKQzjWehpHpZlZrAwjnLpWUmlaYnKJF3TqocSC6pldrdXBW8/MIS+Uf9LAir2psFhovRSZj6yn0bd9Nfk/37Qy+d7MMllWhkrSFMorDqaA+kYwZ4oSw5ceYKKY7xXIBVaYGH/Jegnx7ZHvgrN3g/j9IP76oXN0vF7HFnqN3qAuitFHdISGaITGiAQ7wX5wHJyEr8LD8FP4uQkNg7VmB/1j4egP6W3DsQ==</latexit> Є Є
24. 8ͷࣄޙ෼෍ͷܭࢉ w ͭͷ߲Λ·ͱΊΔͱɼΨϯϚ෼෍ͷܗʹͳΔ 24 lnGam( |a, b) = (a 1)ln b + lnCG(a, b) <latexit sha1_base64="uspTueiQkGOOvKXKs1f0zCTNyfY=">AAACSXicbVBNSwMxFMzW7/pV9ejlYREqatlVQS+C6EGPClaFttS3aarBJLskWaGs/XtevHnzP3jxoIgn03b9diAwzMx7SSaMBTfW9x+83MDg0PDI6Fh+fGJyarowM3tiokRTVqGRiPRZiIYJrljFcivYWawZylCw0/Bqr+ufXjNteKSObTtmdYkXirc4ReukRuE8rWkJQsE+yg6UasKNNhFuAFcgXIJtKCGsQrAEWa4DH5HV8JMuf7l7jR7dd7v6GxqFol/2e4C/JMhIkWQ4bBTua82IJpIpSwUaUw382NZT1JZTwTr5WmJYjPQKL1jVUYWSmXraa6IDi05pQivS7igLPfX7RIrSmLYMXVKivTS/va74n1dNbGurnnIVJ5Yp2r+olQiwEXRrhSbXjFrRdgSp5u6tQC9RI7Wu/LwrIfj95b/kZK0crJeDo43izm5WxyiZJwukRAKySXbIATkkFULJLXkkz+TFu/OevFfvrR/NednMHPmB3MA7sies4Q==</latexit> ΨϯϚ෼෍ͷର਺
25. 8ͱ)ͷࣄޙ෼෍ w 8ͱ)͸ରশͰ͋ΔͨΊɼ)͸8ͱಉ༷ʹٻΊΔ͜ͱ͕Մೳ 25 8ͷࣄޙ෼෍ )ͷࣄޙ෼෍
26. 4ͷࣄޙ෼෍ w σϧλ෼෍ͷ߲
    w ϙΞιϯ෼෍ͷ߲ 26
27. ϙΞιϯ෼෍ͷର਺ w ϙΞιϯ෼෍
    w ϙΞιϯ෼෍ͷର਺
    w YΛ4ɼЕΛ)8ʹஔ͖׵͑Ε͹Α͍ 27 Poi(x| ) = x x! e <latexit sha1_base64="hOpCwWDG94WA8rAJPDWQ5fyAQ50=">AAACJXicbVDLSsNAFJ3UV62vqEs3o0WoC0uigi4Uim5cVrAPaNIymUzaoZMHMxNpifkZN/6KGxcWEVz5K07bCNp6YOBwzrncuceJGBXSMD613MLi0vJKfrWwtr6xuaVv79RFGHNMajhkIW86SBBGA1KTVDLSjDhBvsNIw+nfjP3GA+GChsG9HEbE9lE3oB7FSCqpo18mFvdhNaQpLA3gI7SYmnXREbyClscRTjKhPUiTwX4KSTs5/gmlHb1olI0J4DwxM1IEGaodfWS5IY59EkjMkBAt04iknSAuKWYkLVixIBHCfdQlLUUD5BNhJ5MrU3ioFBd6IVcvkHCi/p5IkC/E0HdU0keyJ2a9sfif14qld2EnNIhiSQI8XeTFDMoQjiuDLuUESzZUBGFO1V8h7iFVjlTFFlQJ5uzJ86R+UjZPy+bdWbFyndWRB3vgAJSACc5BBdyCKqgBDJ7AC3gDI+1Ze9XetY9pNKdlM7vgD7Svb/f0pFk=</latexit> lnPoi(x| ) = xln lnx! <latexit sha1_base64="lJngUpMm+3AcX6paJr7B5NpLMGo=">AAACMHicbZDLSgMxFIYzXmu9jbp0c7QIdWGZUUE3QtGFLivYC7SlZNK0Dc1khiQjLWMfyY2PohsFRdz6FKbtiNp6IPDn+88hOb8Xcqa047xYM7Nz8wuLqaX08srq2rq9sVlSQSQJLZKAB7LiYUU5E7Somea0EkqKfY/Tste9GPrlWyoVC8SN7oe07uO2YC1GsDaoYV/GNekDF1AI2ACyPbiDGjfjTbwPZ9CDxB58Uzj4Qb0dc0t4w844OWdUMC3cRGRQUoWG/VhrBiTyqdCEY6WqrhPqeoylZoTTQboWKRpi0sVtWjVSYJ+qejxaeAB7hjShFUhzhIYR/T0RY1+pvu+ZTh/rjpr0hvA/rxrp1mk9ZiKMNBVk/FAr4qADGKYHTSYp0bxvBCaSmb8C6WCJiTYZp00I7uTK06J0mHOPcu71cSZ/nsSRQttoF2WRi05QHl2hAioigu7RE3pFb9aD9Wy9Wx/j1hkrmdlCf8r6/AKiPKbn</latexit>
28. ϙΞιϯ෼෍ͷ߲ͷܭࢉ 28 lnPoi(x| ) = xln lnx! <latexit sha1_base64="lJngUpMm+3AcX6paJr7B5NpLMGo=">AAACMHicbZDLSgMxFIYzXmu9jbp0c7QIdWGZUUE3QtGFLivYC7SlZNK0Dc1khiQjLWMfyY2PohsFRdz6FKbtiNp6IPDn+88hOb8Xcqa047xYM7Nz8wuLqaX08srq2rq9sVlSQSQJLZKAB7LiYUU5E7Somea0EkqKfY/Tste9GPrlWyoVC8SN7oe07uO2YC1GsDaoYV/GNekDF1AI2ACyPbiDGjfjTbwPZ9CDxB58Uzj4Qb0dc0t4w844OWdUMC3cRGRQUoWG/VhrBiTyqdCEY6WqrhPqeoylZoTTQboWKRpi0sVtWjVSYJ+qejxaeAB7hjShFUhzhIYR/T0RY1+pvu+ZTh/rjpr0hvA/rxrp1mk9ZiKMNBVk/FAr4qADGKYHTSYp0bxvBCaSmb8C6WCJiTYZp00I7uTK06J0mHOPcu71cSZ/nsSRQttoF2WRi05QHl2hAioigu7RE3pFb9aD9Wy9Wx/j1hkrmdlCf8r6/AKiPKbn</latexit> ΑΓɼ Yʹ4ɼЕʹ)8Λ୅ೖ MPHల։ͯ͠ɼظ଴஋ల։
    ෆཁͳ߲͸DPOTU΁
29. ଟ߲෼෍ w ଟ߲෼෍
    w ଟ߲෼෍ͷର਺
    w ϙΞιϯ෼෍ͷର਺ͷܭࢉ݁Ռ 29
30. 4ͷࣄޙ෼෍ w σϧλ෼෍ͷ߲
    w ϙΞιϯ෼෍ͷ߲ 30
31. ଟ߲෼෍ w ଟ߲෼෍
    w ଟ߲෼෍ͷର਺
    w ϙΞιϯ෼෍ͷର਺ͷܭࢉ݁Ռ 31 Λ9ʹ͢Δ
32. σϧλ෼෍ͷର਺ w σϧλ෼෍
    w σϧλ෼෍ͷର਺ͷܭࢉ 32
33. 4ͷۙࣅࣄޙ෼෍ w 4ͷۙࣅࣄޙ෼෍͸ҎԼͷଟ߲෼෍ʹै͏ 33
34. ֤ظ଴஋ w ষͷ֤෼෍ͷظ଴஋ͷܭࢉ݁Ռʹ౰ͯ͸ΊΔͱ 34
35. ߋ৽ͷखॱ w 4ͷߋ৽ʹ͸ɼ8ͱ)ͷظ଴஋͕ඞཁ
    w 8ͷߋ৽ʹ͸ɼͦΕͧΕ4ͱ8ͷظ଴஋͕ඞཁʢ)΋ಉ༷ʣ
    w ॳظԽͨ͠8ͱ)ͷߦྻੵ͔Β4Λܭࢉʢ͜ΕͰߋ৽͞ΕΔʣ
    w 8ͱ)͔Βଟ߲෼෍Λར༻ͯ͠4ͷظ଴஋Λܭࢉ
    w ಘΒΕͨ4ͷظ଴஋ͱ8ͷॳظ஋Λݩʹɼ8Λߋ৽ʢ)΋ಉ༷ʣ
    w -PPQ
    w 8ͱ)ͷߦྻੵ͔Β4Λܭࢉɼߋ৽͠ɼଟ߲෼෍͔Β4ͷظ଴஋Λܭࢉ
    w ಘΒΕͨ4ͷظ଴஋ͱͭલͷ8Ͱɼ8Λߋ৽ʢ)΋ಉ༷ʣ 35