「異常検知と変化検知」輪読会 第3章発表資料

「異常検知と変化検知」輪読会#2,#3で使用した資料です。
http://connpass.com/event/28584/
http://connpass.com/event/29197/

内容は、以下書籍の第3章　単純ベイズ法による異常検知をまとめたものです。

・井手剛, 杉山将, "異常検知と変化検知(機械学習プロフェッショナルシリーズ)", 講談社
http://www.kspub.co.jp/book/detail/1529083.html

Transcript

1. ҟৗݕ஌ͱมԽݕ஌ ྠߨձ
   @functionalaho
   2016-3-17
   2016-3-31
   @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 1 / 32
   

   View Slide

2. ୈ 3 ষɹ୯७ϕΠζ๏ʹΑΔҟৗݕ஌
   1 3 ষ֓ཁ
   2 3.1 ଟ࣍ݩͷ໰୊Λ 1 ࣍ݩʹؼண͢Δ
   ର਺໬౓ͷදࣜ
   3 3.2 ಠཱม਺Ϟσϧͷ΋ͱͰͷϗςϦϯά T2 ๏
   4 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ
   3.3.1 ଟ߲෼෍: ස౓ʹ͍ͭͯͷ෼෍
   3.3.2 ଟ߲෼෍ͷ࠷໬ਪఆ
   3.3.3 ໎࿭ϝʔϧͷ෼ྨ
   5 3.4 ࠷େࣄޙ֬཰ਪఆͱଟ߲෼෍ͷεϜʔδϯά
   6 3.5 ೋ஋෼ྨͱҟৗݕ஌ͷؔ܎
   ϕΠζܾఆଇ
   @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 2 / 32
   

   View Slide

3. ͸͡Ίʹ
   ͜ͷࢿྉʹ͍ͭͯ
   ຊࢿྉ͸ҎԼͷʮҟৗݕ஌ͱมԽݕ஌ɹྠಡձʯ༻ͷൃද༻ͱͯ͠Ұಡऀ͕࡞੒ͨ͠΋ͷͰ͢ɻ
   http://connpass.com/event/29197/
   ຊࢿྉʹؔ͢Δ࣭͝໰͸ɺશͯ@functionalaho ΁͓ئ͍͠·͢ɻஶऀɾग़൛͓ࣾΑͼษڧձओ࠵
   ऀ΁ͷ࣭͝໰͸͝ԕྀ͍ͩ͘͞ɻ
   @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 3 / 32
   

   View Slide

4. 3 ষ֓ཁ
   1 3 ষ֓ཁ
   2 3.1 ଟ࣍ݩͷ໰୊Λ 1 ࣍ݩʹؼண͢Δ
   3 3.2 ಠཱม਺Ϟσϧͷ΋ͱͰͷϗςϦϯά T2 ๏
   4 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ
   5 3.4 ࠷େࣄޙ֬཰ਪఆͱଟ߲෼෍ͷεϜʔδϯά
   6 3.5 ೋ஋෼ྨͱҟৗݕ஌ͷؔ܎
   @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 4 / 32
   

   View Slide

5. 3 ষ֓ཁ
   3 ষ֓ཁ
   ֓ཁ
   ଟ࣍ݩσʔλʹର͢Δҟৗݕ஌ͷ໰୊Λ̍࣍ݩͷ໰୊ʹؼண͢Δํ๏Λಋೖ͢Δ
   ଟ߲෼෍ͱଟ࣍ݩਖ਼ن෼෍ʹର͠ɺҟৗ౓Λಋग़͢Δ
   ໎࿭ϝʔϧ෼ྨثʹԠ༻ͯ͠ΈΔ
   @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 5 / 32
   

   View Slide

6. 3.1 ଟ࣍ݩͷ໰୊Λ 1 ࣍ݩʹؼண͢Δ
   1 3 ষ֓ཁ
   2 3.1 ଟ࣍ݩͷ໰୊Λ 1 ࣍ݩʹؼண͢Δ
   3 3.2 ಠཱม਺Ϟσϧͷ΋ͱͰͷϗςϦϯά T2 ๏
   4 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ
   5 3.4 ࠷େࣄޙ֬཰ਪఆͱଟ߲෼෍ͷεϜʔδϯά
   6 3.5 ೋ஋෼ྨͱҟৗݕ஌ͷؔ܎
   @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 6 / 32
   

   View Slide

7. 3.1 ଟ࣍ݩͷ໰୊Λ 1 ࣍ݩʹؼண͢Δ
   3.1 ଟ࣍ݩͷ໰୊Λ 1 ࣍ݩʹؼண͢Δ
   ҟৗݕ஌ͷ໰୊Λ೉͘͢͠ΔཁૉͰɺ࠷΋ॏཁͳ΋ͷ͸ม਺͕ͨ͘͞Μ͋Δ͜ͱɻ
   ୯७ϕΠζ๏ (naive bayse) ͱ͍͏ํ๏Ͱɺม਺͝ͱʹ໰୊Λ੾Γ෼͚Δ͜ͱ͕Ͱ͖Δɻ
   ୯७ϕΠζ๏
   p(x | y) = p(x1 | y)p(x2 | y) · · · p(xM | y) =
   M
   ∏
   i=1
   p(xi | y) (3.1)
   y ݻఆͷ΋ͱͰɺϕΫτϧ x ͷ֤ཁૉ xi
   ͕ͦΕͧΕಠཱͰ͋ΔͱԾఆ͢Δɻ
   @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 7 / 32
   

   View Slide

8. 3.1 ଟ࣍ݩͷ໰୊Λ 1 ࣍ݩʹؼண͢Δ ର਺໬౓ͷදࣜ
   xi
   ͷ৚݅෇͖෼෍ p(xi | θy
   i
   , y) ͷΑ͏ʹະ஌ύϥϝʔλ θy
   i
   Ͱ͔͚͍ͯΔͱ͢Δɻθy
   i
   ͸ޙड़ͷଟ߲
   ෼෍ͷྫͰ͋Ε͹ɺ໎࿭ϝʔϧ෼ྨͷ i ൪໨ͷޠͷग़ݱ֬཰ʹ૬౰͢Δɻ
   ର਺໬౓
   log ͷதͷੵ͕ͦΕͧΕͷ࿨Ͱॻ͚ͯɺ͞Βʹ n ʹ͍ͭͯ·ͱΊ͋͛ΔͱҎԼʹͳΔɻ
   L(Θ | D) =
   N
   ∑
   n=1
   M
   ∑
   i=1
   ln p(x(n)
   i
   | θy(n)
   i
   , y(n))
   =
   M
   ∑
   i=1
   { ∑
   n∈D1
   ln p(x(n)
   i
   | θ1
   i
   , y = 1) +
   ∑
   n∈D0
   ln p(x(n)
   i
   | θ0
   i
   , y = 0)
   }
   (3.2)
   ͨͩ͠ɺD0 ͸ y(n) = 0 ͱͳΔඪຊͷू߹,D1 ͸ y(n) = 1 ͱͳΔඪຊͷू߹.
   ͜ΕΑΓ θ1
   i
   ͷ࠷໬ղΛ༩͑Δ৚݅͸ҎԼɻ
   0 =
   ∂L
   ∂θ1
   i
   =
   ∂
   ∂θ1
   i
   ∑
   n∈D1
   ln p(x(n)
   i
   | θ1
   i
   , y = 1)
   ղऍ
   ࠷ӈลʹ͸ɺม਺ͷఴࣈ i ͱ y = 1 ʹରԠ͢Δ߲͔͠࢒Βͳ͍ɻ
   ͜Ε͸ɺ3.1 ͕੒Γཱͭͱ͖ʹɺม਺ i ͝ͱ,y ͝ͱʹ੾Γ෼͚ΒΕͨͱ͍͏͜ͱΛҙຯ͢Δɻ
   @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 8 / 32
   

   View Slide

9. 3.1 ଟ࣍ݩͷ໰୊Λ 1 ࣍ݩʹؼண͢Δ ର਺໬౓ͷදࣜ
   3.1 ͷ·ͱΊ
   ఆཧ 3.1 ม਺͕౷ܭతʹಠཱͳ৔߹ͷ࠷໬ਪఆ
   ౷ܭతʹಠཱͷͱ͖ʹ͸ɺࣜ (3.1) ͷΑ͏ʹੵͷܗͰ͔͚Δɻ͜ͷͱ͖ɺM ม਺ͷͦΕͧΕʹରͯ͠
   ผʑʹ࠷໬ਪఆ͢Δ͜ͱͰϞσϧͷύϥϝʔλΛٻΊΔ͜ͱ͕Ͱ͖Δɻ
   @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 9 / 32
   

   View Slide

10. 3.2 ಠཱม਺Ϟσϧͷ΋ͱͰͷϗςϦϯά T 2 ๏
    1 3 ষ֓ཁ
    2 3.1 ଟ࣍ݩͷ໰୊Λ 1 ࣍ݩʹؼண͢Δ
    3 3.2 ಠཱม਺Ϟσϧͷ΋ͱͰͷϗςϦϯά T2 ๏
    4 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ
    5 3.4 ࠷େࣄޙ֬཰ਪఆͱଟ߲෼෍ͷεϜʔδϯά
    6 3.5 ೋ஋෼ྨͱҟৗݕ஌ͷؔ܎
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 10 / 32
    

    View Slide

11. 3.2 ಠཱม਺Ϟσϧͷ΋ͱͰͷϗςϦϯά T 2 ๏
    3.2 ͷ·ͱΊ
    3.2 ͷ·ͱΊ
    ม਺͕ಠཱͰ͋Ε͹ɺM ࣍ݩϕΫτϧͷ؍ଌ஋ x′ ͷҟৗ౓͸ 1 ࣍ݩͷҟৗ౓ͷ࿨ͱͳΔ͜ͱΛࣔ͢ɻ
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 11 / 32
    

    View Slide

12. 3.2 ಠཱม਺Ϟσϧͷ΋ͱͰͷϗςϦϯά T 2 ๏
    ม਺ͷಠཱੑΛԾఆ͢ΔϞσϧ͸ϥϕϧͳ͠σʔλͰ΋ద༻Ͱ͖Δɻ
    ͜͜Ͱ͸ϗςϦϯά T2 ๏Ͱ࢖ͬͯΈΔɻ
    ·ͣɺσʔλͱͯ͠ɺM ࣍ݩͷ N ݸͷ؍ଌ஋ D = {x(1), x(2), · · · , x(N)} Λߟ͑Δɻ
    ؍ଌ஋ͷ֬཰෼෍ͷϞσϧ͸ଟมྔਖ਼ن෼෍ͷڞ෼ࢄߦྻͷඇର֯੒෼Λ 0 ͱͨ͠΋ͷ
    p(x)=
    ∏
    M
    i=1
    N (xi|µi,Σii)=
    ∏
    M
    i=1
    1
    √
    2πΣii
    exp(− 1
    2Σii
    (xi−µi)2) (3.3)
    ͜ͷ৔߹ɺม਺͝ͱͷੵʹ෼཭͞Ε͍ͯΔͷͰɺର਺໬౓Λ µi
    ͓Αͼ Σ−1
    ii
    ͷඍ෼ = 0 Λղ͘ͱɺฏ
    ۉͱ෼ࢄ͸ҎԼͱͳΓɺ1 ࣍ݩਖ਼ن෼෍ͱҰக͢Δ͜ͱ͕Θ͔Δɻ
    ˆ
    µi= 1
    N
    ∑
    N
    n=1
    x(n)
    i
    , ˆ
    Σii=ˆ
    σ2
    i
    ≡ 1
    N
    ∑
    N
    n=1
    (x(n)
    i
    −ˆ
    µi)2 (3.4)
    ҟৗ౓ (2.9) ͸ɺ
    a(x′)=
    ∑
    M
    n=1
    (
    x′
    i−ˆ
    µi
    ˆ
    σi
    )
    2
    (3.5)
    ؍ଌ஋͕ M ࣍ݩϕΫτϧͷ৔߹ɺM ݸͷม਺ͦΕͧΕಠཱʹܭࢉ͞Εͨҟৗ౓ͷ࿨ͱͳΔɻ
    (3.5) ͸ม਺ؒʹґଘؔ܎͕͋Δͱ͖ʹ͸੒Γཱͨͳ͍͕ɺͦ͏͍͏ͱ͖Ͱ΋ͬ͘͟Γͭҟৗ౓Λݟੵ
    ΋Δͱ͖ʹ༗༻ɻ
    ͳ͓ɺ(3.5) ࣜ͸ M = 2 ͰପԁͷࣜʹͳΔ͕ɺਤ 3.1 தͰࣔ͞Ε͍ͯΔҟৗ൑ఆڥք (੺࿮) ͸ପԁ
    ʹ͸ͳ͍ͬͯͳ͍ɻ
    ਤ 3.1 ͷ੺࿮͸ (3.5) ࣜ͸ M = 2 ͕ࣔ͢ൣғͰ͸ͳ͘ɺ࿨ΛͱΒͣม਺ͦΕͧΕʹ͍ͭͯൣғΛܾ
    ఆͨ͠ͱ͖ͷڥքઢΛ͍ࣔͯ͠Δ͜ͱʹ஫ҙɻ
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 12 / 32
    

    View Slide

13. 3.2 ಠཱม਺Ϟσϧͷ΋ͱͰͷϗςϦϯά T 2 ๏
    ਤ 3.1:ม਺ؒͷ૬͕ؔҟৗݕ஌ͲͷΑ͏ʹޮ͔͘
    ެ։࣌࡟আ
    ਤ 3.5
    ࣜ 3.5 ΑΓɺม਺͝ͱʹಠཱʹਖ਼ৗ࣌ͷൣғ͸ႊఆ͢ΔͨΊɺਖ਼ৗൣғ͸ඞͣਖ਼ํܗʹͳΔ (M = 2
    ͷ৔߹). ͕ͨͬͯ͠ɺຊ౰͸૬͕ؔ͋Δͱ͖ (b) ͸ɺҟৗͱ൑ఆ͢΂͖ൣғͰਖ਼ৗͱ൑ఆͯ͠͠·͏ɻ
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 13 / 32
    

    View Slide

14. 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ
    1 3 ষ֓ཁ
    2 3.1 ଟ࣍ݩͷ໰୊Λ 1 ࣍ݩʹؼண͢Δ
    3 3.2 ಠཱม਺Ϟσϧͷ΋ͱͰͷϗςϦϯά T2 ๏
    4 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ
    5 3.4 ࠷େࣄޙ֬཰ਪఆͱଟ߲෼෍ͷεϜʔδϯά
    6 3.5 ೋ஋෼ྨͱҟৗݕ஌ͷؔ܎
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 14 / 32
    

    View Slide

15. 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ
    3.3 ͷ֓ཁ
    3.3 ͷ໨ඪ
    ୯७ϕΠζ๏Λಋೖ͠ɺଟ߲෼෍ʹద༻͢Δɻ
    ͜ΕΛ໎࿭ϝʔϧ൑ఆʹ࢖ͬͯΈΔ
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 15 / 32
    

    View Slide

16. 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ 3.3.1 ଟ߲෼෍: ස౓ʹ͍ͭͯͷ෼෍
    ස౓Λදݱ͢Δ෼෍
    ස౓ͷͨΊͷ෼෍
    1 ௨ൢαΠτʹ͓͚Δ঎඼ͷӾཡճ਺
    2 ਤॻؗʹ͓͚Δδϟϯϧผͷିग़࡭਺
    ͳͲͷස౓Λදݱ͢ΔͨΊͷ֬཰෼෍͕ଟ߲෼෍ɻ
    ໎࿭ϝʔϧͷৼΓ෼͚໰୊
    ϝʔϧதͷ୯ޠͷग़ݱස౓Λϝʔϧͷಛ௃ྔͱͯ͠ར༻͢Δ͜ͱͰɺ໎࿭ϝʔϧৼΓ෼͚໰୊΋ස౓
    ͷ໰୊ʹম͖ͳ͓͢͜ͱ͕Ͱ͖Δɻ
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 16 / 32
    

    View Slide

17. 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ 3.3.1 ଟ߲෼෍: ස౓ʹ͍ͭͯͷ෼෍
    ୯ޠା٧Ί (Bag-of-words) Ϟσϧ
    ϝʔϧʹ࢖ΘΕͦ͏ͳ୯ޠΛ༧Ίྻڍ͓ͯ͘͠ɻҎ߱ɺྻڍͨ͠୯ޠ਺Λ M ͱ͢Δɻ
    ·ͨɺ1 ͭͷϝʔϧ͸ͦΕΛߏ੒͢Δ୯ޠͷස౓Λ୯ޠ͝ͱʹͳΒ΂ͨ M ࣍ݩϕΫτϧ x Ͱදݱ͞
    ΕΔɻྫ͑͹ҎԼͷΑ͏ͳײ͡ɻ
    x = (0, 0, 1, 0, 0, 2, ...)T
    ͜Ε͸ҰൠʹૄͳϕΫτϧɻ͜ͷΑ͏ͳߟ͑ํΛɺ୯ޠା٧Ί (Bag-of-words) Ϟσϧͱ͍͏ɻ
    ໰୊͸ x ͷग़ํΛද֬͢཰෼෍͕Ͳ͏ͳ͍ͬͯΔͷ͔Ͱ͋Δɻ͜ͷϞσϧ͸୯ޠಉ࢜ͷؔ܎Λແࢹ͠
    ͯݸผʹΧ΢ϯτ͢ΔϞσϧͳͷͰɺx ͷ෼෍͸֤୯ޠͷग़ݱ֬཰ θ1, · · · , θM
    ͰදݱͰ͖Δɻ͜͏
    ͍͏৔߹͸ଟ߲෼෍͕༗༻ɻ
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 17 / 32
    

    View Slide

18. 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ 3.3.1 ଟ߲෼෍: ස౓ʹ͍ͭͯͷ෼෍
    ଟ߲෼෍
    Mult(x | θ) =
    (x1 + · · · + xM )!
    x1!x2! · · · xM !
    θx1
    1
    · · · θxM
    M
    (3.6)
    M
    ∑
    i=1
    θi = 1
    M = 2 ͷͱ͖͸ೋ߲෼෍ɻ
    ଟ߲෼෍͸ස౓Λදݱ͢ΔͨΊͷ࠷΋ࣗવͳ෼෍Ͱ͋Δɻ
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 18 / 32
    

    View Slide

19. 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ 3.3.2 ଟ߲෼෍ͷ࠷໬ਪఆ
    ଟ߲෼෍ͷ࠷໬ਪఆ
    ໎࿭ϝʔϧ෼ྨ໰୊Λྫʹɺଟ߲෼෍ͷ࠷໬ਪఆΛߟ͑Δɻ
    લఏͱͯ͠͸ɺաڈϝʔϧ͕ N ௨ɺ໎࿭ϝʔϧ (y = 1) ͔ී௨ϝʔϧ (y = 0) ͔Ͱϥϕϧ͚ͮ͞Ε
    ͍ͯΔ.
    ͜ͷঢ়گͰҟৗ౓ (ࣜ 1.2) ΑΓ໎࿭ϝʔϧͷ൑ఆΛ࣮ࢪ͢Δ͜ͱΛߟ͑Δɻ
    a(x′)=ln p(x′|y=1,D)
    p(x′|y=0,D)
    (1.2)
    y = 0, y = 1 ʹରͯ͠ɺMult(x | θ0), Mult(x | θ1) ͱ͍͏ 2 ͭͷϞσϧΛԾఆ͠ɺͦΕͧΕͷະ஌
    ύϥϝλʔ θ0, θ1 Λ࠷໬ਪఆ͢Δɻ
    Ϟσϧύϥϝʔλͷର਺໬౓͸ҎԼͱͳΔɻ
    L(θ0,θ1|D)=
    ∑
    n∈D1
    ϝʔϧํ޲
    M
    ∑
    i=1
    ୯ޠํ޲
    x(n)
    i
    ln θ1
    i
    +
    ∑
    n∈D0
    ∑
    M
    i=1
    x(n)
    i
    ln θ0
    i
    +Const. (3.7)
    ໰୊͸ɺ(3.7) ࣜΛ
    ∑
    M
    i=1
    θ1
    i
    = 1,
    ∑
    M
    i=1
    θ0
    i
    = 1 ͱ͍͏੍໿ͷ΋ͱͰ࠷େԽ͢Δ͜ͱɻ
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 19 / 32
    

    View Slide

20. 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ 3.3.2 ଟ߲෼෍ͷ࠷໬ਪఆ
    ଟ߲෼෍ͷ࠷໬ਪఆ
    ϥάϥϯδϡ৐਺Λ λ0, λ1 ͱ͢Δͱɺϥάϥϯδϡؔ਺͸
    L − λ0
    M
    ∑
    j=1
    θ0
    j
    − λ1
    M
    ∑
    j=1
    θ1
    j
    ͜ΕΛ্ه੍໿ͷ΋ͱͰ࠷େԽ͢Δʹ͸ϥάϥϯδϡؔ਺ͷඍ෼ = 0 Λղ͘ɻ
    ྫ͑͹ θ0 ͷୈ i ੒෼ʹ͍ͭͯ͸ɺ
    0 =
    ∂
    ∂θ0
    i
    (
    L − λ0
    M
    ∑
    j=1
    θ0
    j
    − λ1
    M
    ∑
    j=1
    θ1
    j
    )
    =
    1
    θ0
    i
    ∑
    n∈D0
    x(n)
    i
    − λ0
    θ1 ʹ͍ͭͯ΋ಉ༷ʹ͢Δͱɺ݁ہ θ0, θ1 ͷ i ੒෼͸ҎԼͱͳΔɻ
    ˆ
    θ0
    i
    =
    ∑
    n∈D0 x
    (n)
    i
    ∑M
    j=1
    ∑
    n∈D0 x
    (n)
    j
    = D0ʹ͓͚Δ୯ޠ i ͷग़ݱ૯਺
    D0ʹ͓͚Δશ୯ޠͷग़ݱ૯਺
    ˆ
    θ0
    i
    =
    ∑
    n∈D1 x
    (n)
    i
    ∑M
    j=1
    ∑
    n∈D1 x
    (n)
    j
    = D1ʹ͓͚Δ୯ޠ i ͷग़ݱ૯਺
    D1ʹ͓͚Δશ୯ޠͷग़ݱ૯਺
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 20 / 32
    

    View Slide

21. 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ 3.3.2 ଟ߲෼෍ͷ࠷໬ਪఆ
    ଟ߲෼෍ͷ࠷໬ਪఆ
    ཁ͢Δʹɾɾɾ
    
    ୯ޠͷग़ݱ֬཰͸ग़ݱස౓ʹൺྫ͢Δͱ͍͏Θ͔Γ΍͍݁͢ՌͱͳΔ
    
    ໰୊఺
    ܇࿅σʔλʹͳ͍୯ޠͷ֬཰͕θϩͱͳΓܭࢉͰ͖ͳ͍
    ରॲ๏
    εϜʔδϯά (smoothing) ͱ͍͏ԼବΛཤ͔ͤΔख๏Λ࢖͏ɻ
    ͨͱ͑͹ɺN(y)
    i
    → N(y)
    i
    + γ(γ > 0) (ςΩετޡ২?)
    ͱஔ͖׵͑ͯɺ
    ˆ
    θy
    i
    =
    Ny
    i
    + γ
    |Dy| + Mγ
    (3.8)
    ͨͩ͠,
    ∑
    n
    x(n)
    i
    = Ny
    i
    ,
    ∑
    i
    N(y)
    i
    = |Dy|.
    શ୯ޠͰੵ෼͢ΔͱͪΌΜͱ 1 ʹͳ͍ͬͯΔ.
    M
    ∑
    i=1
    θy
    i
    =
    ∑
    i
    Ny
    i
    + Mγ
    |Dy| + Mγ
    =
    |Dy| + Mγ
    |Dy| + Mγ
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 21 / 32
    

    View Slide

22. 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ 3.3.3 ໎࿭ϝʔϧͷ෼ྨ
    ໎࿭ϝʔϧΛ෼ྨ͢Δ
    ଟ߲෼෍ (Ϟσϧ) ͷύϥϝʔλ θy
    i
    ͕ٻ·ͬͨɻ
    ͍Α͍Αະ஌ϝʔϧ x′ Λ໎࿭ϝʔϧ (y = 1) ͔ͦ͏Ͱͳ͍ (y = 0) ͔ʹ෼ྨ͢Δɻ
    ҟৗ౓ (1.2) ࣜΛ࢖͏ͱɺ೚ҙϝʔϧͷ൑ఆείΞͷࣜͱͯ͠ҎԼΛಘΔɻ
    a(x′) =
    M
    ∑
    i=1
    x′
    i
    ln
    ˆ
    θ1
    i
    ˆ
    θ0
    i
    ≡αi
    (3.9)
    = αT x′
    ͷΑ͏ͳઢܗ݁߹Ͱ͔͚Δɻ
    ଟ߲෼෍ʹجͮ͘୯७ϕΠζ෼ྨ͸ઢܗ෼ྨثͰ͋Δ͜ͱΛҙຯ͢Δɻ
    ͜͜·Ͱͷ݁ՌΑΓɺ໎࿭ϝʔϧ൑ఆͷखଓ͖͸ΞϧΰϦζϜ 3.1(P.35) ͷΑ͏ʹ·ͱΊΔ͜ͱ͕Ͱ
    ͖Δɻ
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 22 / 32
    

    View Slide

23. 3.4 ࠷େࣄޙ֬཰ਪఆͱଟ߲෼෍ͷεϜʔδϯά
    1 3 ষ֓ཁ
    2 3.1 ଟ࣍ݩͷ໰୊Λ 1 ࣍ݩʹؼண͢Δ
    3 3.2 ಠཱม਺Ϟσϧͷ΋ͱͰͷϗςϦϯά T2 ๏
    4 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ
    5 3.4 ࠷େࣄޙ֬཰ਪఆͱଟ߲෼෍ͷεϜʔδϯά
    6 3.5 ೋ஋෼ྨͱҟৗݕ஌ͷؔ܎
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 23 / 32
    

    View Slide

24. 3.4 ࠷େࣄޙ֬཰ਪఆͱଟ߲෼෍ͷεϜʔδϯά
    σΟϦΫϨ෼෍
    3.3.2 ʹ͓͍ͯɺ܇࿅σʔλʹ஧࣮ʹ࠷໬ਪఆ͢ΔͱࢧোΛ͖ͨͨ͢ΊɺԼବ γ Λ͸͔ͤΔૢ࡞Λ࣮
    ࢪͨ͠ɻϕΠζ౷ܭͷ࿮૊ΈͰ͸͜ͷૢ࡞͸ࣄલ෼෍ΛܾΊΔ͜ͱʹ૬౰͠ɺD ʹओ؍Λ৫ΓަͥΔ
    ͜ͱΛҙຯ͢Δɻ
    ଟ߲෼෍ͷڞ໾ࣄલ෼෍
    ଟ߲෼෍ͷࣄલ෼෍ͱͯ͠ɺڞ໾ͳσΟϦΫϨ෼෍͕਺ֶతʹѻ͍΍͘͢Α͘࢖ΘΕΔɻύϥϝʔλ
    ϕΫτϧΛ α ͱ͢ΔͱɺσΟϦΫϨ෼෍ͷີ౓ؔ਺͸ҎԼͱͳΔɻ
    Dir(θ | α) ≡
    Γ(α1)Γ(α2) · · · Γ(αM )
    Γ(α1 + · · · + αM )
    θα1−1
    1
    · · · θαM −1
    M
    (3.10)
    θͷ෼ࢄ͸ɺ
    Var [θm] =
    αm(ˆ
    α − αm)
    ˆ
    α2(ˆ
    α + 1)
    ,
    ˆ
    α =
    M
    ∑
    m=1
    αm
    ˜
    α → େͰ෼ࢄ → খ
    αi
    ͸֬཰ (θi) ͷॏΈͷΑ͏ʹ࡞༻͢Δɻ
    ͨͩ͠ࢦ਺ͳͷͰ࣍εϥΠυͷάϥϑͷΑ͏ʹৼΔ෣͏ɻ
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 24 / 32
    

    View Slide

25. 3.4 ࠷େࣄޙ֬཰ਪఆͱଟ߲෼෍ͷεϜʔδϯά
    σΟϦΫϨ෼෍
    άϥϑͷղऍ͸࣍εϥΠυɻ
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 25 / 32
    

    View Slide

26. 3.4 ࠷େࣄޙ֬཰ਪఆͱଟ߲෼෍ͷεϜʔδϯά
    σΟϦΫϨ෼෍
    (্ࠨ) α0, α1, α2 < 1 ͷͱ͖ɺθ ͷ֤ཁૉʹ͓͍ͯϜϥ͕େ͖͍ θ Ͱ֬཰͕େ͖͘ͳΔ.
    (্த) α0 = α1 = α2 = 1 ͷͱ͖ɺ͢΂ͯͷ θ Ͱ౳֬཰ͱͳΔ.
    (্ӈ, Լӈ) α0, α1, α2 > 1 ͰภΓ͕͋Δ৔߹ɺθ ͷ੒෼ͷ͏ͪɺେͳΔ α ʹରԠ͢Δ੒෼͕େ
    ͖͘ͳΔΑ͏ͳ θ Ͱ֬཰࠷େͱͳΔɻ͜ͷͱ͖
    ∑
    i
    θi = 1 ΑΓଞͷ θ ͸খ͘͞ͳΔ͜ͱʹ஫ҙ.
    (Լࠨ) α0 = α1 = α2 > 1 ͷͱ͖ɺ֬཰࠷େͱͳΔ θ ͸ θ ͷ֤੒෼͕౳͍͠ θ ͱͳΔɽ
    (Լத) Լࠨͷঢ়ଶͰɺα Λ౳ͨ͘͠͠··େ͖͘͢Δͱɺ֬཰࠷େͷ θ ͸ͦͷ··Ͱ෼ࢄ͕খ͞
    ͘ͳΔ.
    ͜ΕΒΑΓҎԼͷ͜ͱ͕ݴ͑Δ (ςΩετͱ͸एׯҟͳΔओு).
    ”Ѫ”ͱ͍͏୯ޠʹରԠ͢Δ α0
    Λ ∞ ʹͱ͹ͨ͠ͱ͖
    ͦͷσΟϦΫϨ෼෍͸ θ = (1, 0, 0, · · · , 0) Ͱൃࢄ͢Δ δ ؔ਺ͱͳΔɻ͜Ε͸ɺ࣍ͷจষʹ͸”Ѫ”ͱ
    ͍͏ݴ༿ؚ͔͠·ͳ͍ͱ͍͏৴೦Λࣄલ෼෍ͱͯ͠༩͑Δ͜ͱΛҙຯ͢Δ.
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 26 / 32
    

    View Slide

27. 3.4 ࠷େࣄޙ֬཰ਪఆͱଟ߲෼෍ͷεϜʔδϯά
    ϕΠζͷఆཧͱ MAP ਪఆ
    Ұൠతͳදࣜʹ͢ΔͨΊʹɺMulti(x | θ) → p(D | θ) , ࣄલ෼෍Λ p(θ) ͱॻ͘ɻσʔλ؍ଌʹΑͬ
    ͯ֬཰෼෍ͷमਖ਼͕ඞཁʹͳͬͨͱߟ͑ΔɻͱΓ΋ͳ͓ͣ͜͞Ε͕ϕΠζͷఆཧ͕ҙຯ͢Δͱ͜ΖͰ
    ͋ΓҎԼͷΑ͏ʹදݱ͞ΕΔɻ
    ϕΠζͷఆཧ
    p(θ | D)
    ࣄޙ෼෍
    =
    ໬౓
    p(D | θ)
    ࣄલ෼෍
    p(θ)
    ∫
    dθ′p(D | θ′)p(θ′)
    (3.11)
    ࠷େࣄޙ֬཰ਪఆ (MAP ਪఆ)
    ϕΠζͷఆཧʹΑͬͯɺਪఆ͍ͨ͠ྔͷ෼෍͕ಘΒΕΔɻ෼෍͍ͯ͠ΔͷͰͲΕΛબ΂͹͍͍ͷ͔ͱ
    ͍͏ࢦඪ͕ඞཁͰ͋Δɻ͍͔ͭ͘બͼํ͕͋Δ͕ɺ͜͜Ͱ͸ࣄޙ֬཰Λ࠷େԽ͢Δ஋ θ∗ ΛબͿ͜ͱʹ
    ͢Δɻ͜ͷਪఆํ๏Λ࠷େࣄޙ֬཰ਪఆ (MAP ਪఆ) ͱݺͼɺਪఆ஋ θ∗ Λ MAP ਪఆྔͳͲͱݺͿɻ
    θ∗ = arg max
    θ
    ln[p(D | θ)p(θ)] (3.12)
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 27 / 32
    

    View Slide

28. 3.4 ࠷େࣄޙ֬཰ਪఆͱଟ߲෼෍ͷεϜʔδϯά
    MAP ਪఆ͔ΒεϜʔδϯάͷࣜͷಋग़
    ଟ߲෼෍ͱσΟϦΫϨ෼෍͔ΒɺεϜʔδϯάͷࣜ (3.8) Λಋग़͢Δɻࠓͷ৔߹ະ஌ύϥϝʔλ͸
    θ0, θ1ɻ͜ΕΒ͸ޓ͍ʹಠཱͳͷͰɺࣄલ෼෍͸ͦΕͧΕͷσΟϦΫϨ෼෍ͷੵʹͳΔ. ݁ہࣜ 3.12
    ͷ argmax(·) ͷத͸ɺࣜ (3.13) ͱͳΔɻఆ਺߲͸σΟϦΫϨ෼෍ͷ ln Λͱͬͨͱ͖ʹ௥͍ग़͞Εͨ
    ఆ਺܎਺͕͘Δɻ
    M
    ∑
    i=1
    (
    α1
    i
    − 1 +
    ∑
    n∈D1
    x(n)
    i
    )
    +
    M
    ∑
    i=1
    (
    α0
    i
    − 1 +
    ∑
    n∈D0
    x(n)
    i
    )
    + (ఆ਺) (3.13)
    ϥάϥϯδϡ৐਺๏Ͱࣜ (3.13) ͷ argmax ΛٻΊΔͱɺεϜʔδϯάʹΑΔਪఆࣜ (3.8) ͷܗ͕ग़ͯ
    ͘Δɻ
    ˆ
    θy
    i
    =
    Ny
    i
    + αy
    i
    − 1
    |Dy| +
    ∑
    i
    (αy
    i
    − 1)
    (3.14)
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 28 / 32
    

    View Slide

29. 3.5 ೋ஋෼ྨͱҟৗݕ஌ͷؔ܎
    1 3 ষ֓ཁ
    2 3.1 ଟ࣍ݩͷ໰୊Λ 1 ࣍ݩʹؼண͢Δ
    3 3.2 ಠཱม਺Ϟσϧͷ΋ͱͰͷϗςϦϯά T2 ๏
    4 3.3 ଟ߲෼෍ʹΑΔ୯७ϕΠζ෼ྨ
    5 3.4 ࠷େࣄޙ֬཰ਪఆͱଟ߲෼෍ͷεϜʔδϯά
    6 3.5 ೋ஋෼ྨͱҟৗݕ஌ͷؔ܎
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 29 / 32
    

    View Slide

30. 3.5 ೋ஋෼ྨͱҟৗݕ஌ͷؔ܎ ϕΠζܾఆଇ
    ຊઅͷ໨ඪ
    ೋ஋෼ྨʹҰൠతʹ࢖ΘΕΔϕΠζႊఆଇͱɺωΠϚϯɾϐΞιϯႊఆଇͷؔ܎Λߟ͑Δ͜ͱͰɺ௨
    ৗͷ෼ྨͱҟৗݕ஌ͷ૬ҧΛߟ࡯͢Δɻ
    ఆٛ 3.2 ϕΠζႊఆଇ
    ΋͠ ln
    p(y = 1 | x)
    p(y = 0 | x)
    > 0 ͳΒ͹ y = 1 ͱ൑ఆ. (3.15)
    [࠶ܝ] ఆٛ 1.1 ωΠϚϯɾϐΞιϯႊఆଇ
    ln
    p(x′ | y = 1, D)
    p(x′ | y = 0, D)
    ͕ॴఆͷᮢ஋Λ௒͑ͨΒ y = 1 ͱ൑ఆ.
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 30 / 32
    

    View Slide

31. 3.5 ೋ஋෼ྨͱҟৗݕ஌ͷؔ܎ ϕΠζܾఆଇ
    ࠷దͳ൑ผنଇ
    ϕΠζႊఆଇ (3.15) ͕࠷దͳ൑ผنଇͰ͋Δ͜ͱΛࣔ͢ɻ೚ҙඪຊ x′ ͕༩͑ΒΕͨͱ͖ʹɺy = 0
    ·ͨ͸ y = 1 ʹ෼ྨ͢Δͱͯ͠ɺͦͷ൑ผنଇΛҎԼͷΑ͏ʹॻ͘ɻ
    ˜
    a(x′) ≥ τͳΒ͹ y = 1 (⋆)
    x ͷ෼෍ p(x) ͱɺx ͷ΋ͱͰͷ৚݅෇͖֬཰ p(y | x) ͕ط஌Ͱ͋Δͱ͢Δɻ͜ͷͱ͖ɺ͜ͷ෼෍ͷ
    ΋ͱͰͷ෼ྨͷޡΓ֬཰͸ҎԼͰ༩͑ΒΕΔɻ
    ޡΓ֬཰ =
    ∫
    dx I[˜
    a(x) ≥ τ]p(y = 0 | x)p(x) +
    ∫
    dx {1 − I[˜
    a(x) ≥ τ]}p(y = 1 | x)p(x)
    (3.16)
    = p(y = 1) +
    ∫
    dx I[˜
    a(x) ≥ τ]p(y = 0 | x)p(x)
    {
    1 −
    p(y = 1 | x)
    p(y = 0 | x)
    }
    (3.17)
    ͜ΕΛ࠷খʹ͢Δ ˜
    a ͱ τ ͕஌Γ͍ͨɻͦͷͨΊʹ͸ {·} ͷத਎ͰෛͷྖҬΛ͢΂ͯर͑ΔΑ͏ʹ ˜
    a
    ΛܾΊͯ͋͛Ε͹Α͍ͷͰɺ
    1 −
    p(y = 1 | x)
    p(y = 0 | x)
    ≤ 0 ΑΓ 0 ≤ ln
    p(y = 1 | x)
    p(y = 0 | x)
    (⋆) ͱൺֱͯ͠ɺ ˜
    a(x) = ln
    p(y = 1 | x)
    p(y = 0 | x)
    , τ = 0 (3.18)
    ͱͳΓɺϕΠζႊఆଇ (3.15) ͕ޡΓ֬཰Λ࠷খʹ͢Δ৚݅ͱͯ͠ಋ͔ΕΔɻ
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 31 / 32
    

    View Slide

32. 3.5 ೋ஋෼ྨͱҟৗݕ஌ͷؔ܎ ϕΠζܾఆଇ
    ϕΠζႊఆଈͱωΠϚϯɾϐΞιϯႊఆଇͷҧ͍
    ࣅͯඇͳΔ΋ͷ
    ϕΠζܾఆଇ͸ɺͦ΋ͦ΋ͷҟৗͷ֬཰ p(y = 1) ͱਖ਼ৗͷ֬཰ p(y = 0) ͷൺʹґଘ͢Δ֨޷ͱ
    ͳΔɻ
    ωΠϚϯɾϐΞιϯႊఆଇ
    p(x|y=1)
    p(x|y=0)
    > ᮢ஋Ͱҟৗͱ൑ఆ.
    ϕΠζႊఆଈ
    (3.15) ͱ p(x | y)p(y) ∝ p(y | x) ΑΓɺ p(x|y=1)p(y=1)
    p(x|y=0)p(y=0)
    > 1 Ͱҟৗͱ൑ఆ
    ҟৗݕ஌໰୊ͷ৔߹͸ҧ͍͕ݦஶʹͳΔ
    ҟৗݕ஌໰୊ͷ৔߹͸ɺ্هͷࠩҟ͕ݦஶʹݱΕΔɻͳͥͳΒɺ໰୊ͷੑ্࣭΄ͱΜͲͷ৔߹ɺ
    ҟৗͷ֬཰ (p(y = 1)) ≪ ਖ਼ৗͷ֬཰ (p(y = 0))
    ͱͳΔ͔ΒͰ͋Δɻ͕ͨͬͯ͠ɺطଘ࣮૷ͷ෼ྨثΛҟৗݕ஌໰୊ʹద༻͢Δͱ͖ʹ͸஫ҙ͕ඞཁͰ
    ͋Δɻ
    @functionalaho ҟৗݕ஌ͱมԽݕ஌ ྠߨձ 2016-3-17 2016-3-31 32 / 32
    

    View Slide