Upgrade to Pro — share decks privately, control downloads, hide ads and more …

「はじめてのパターン認識」読書会 第 4 章

horiem
November 09, 2017
1.1k

「はじめてのパターン認識」読書会 第 4 章

horiem

November 09, 2017
Tweet

More Decks by horiem

Transcript

  1. Πϯτϩ: 100 ԁۄͷਅآ൑ఆ ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] …

    ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … …
  2. Πϯτϩ: 100 ԁۄͷਅآ൑ఆ ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] …

    ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … ૬ؔ͋ΔͷͰ͸ʁ
  3. Πϯτϩ: 100 ԁۄͷਅآ൑ఆ ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] …

    ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … ૬ؔ͋ΔͷͰ͸ʁ
  4. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ 4.801 22601

    11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … Πϯτϩ: 100 ԁۄͷਅآ൑ఆ 100 ԁۄͷฏۉʢ4.8 gʣΑΓܰͦ͏͕ͩ
 ࠩ͸ 0.05 [g]
  5. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ 4.801 22601

    11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … Πϯτϩ: 100 ԁۄͷਅآ൑ఆ 100 ԁۄͷฏۉʢ22600 μmʣͱಉ͡Α͏͕ͩ
 ࠩ͸ 2 [μm] >> 0.05
  6. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ 4.801 22601

    11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … Πϯτϩ: 100 ԁۄͷਅآ൑ఆ • ಛ௃ؒͷ૬ؔΛͳ͍ͨ͘͠ • ୯Ґ͕ҧ͍ͬͯͯ΋౷ܭతʹൺֱ͍ͨ͠
  7. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ 4.801 22601

    11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … Πϯτϩ: 100 ԁۄͷਅآ൑ఆ • ಛ௃ؒͷ૬ؔΛͳ͍ͨ͘͠ • ୯Ґ͕ҧ͍ͬͯͯ΋౷ܭతʹൺֱ͍ͨ͠ ➡ ؍ଌσʔλΛม׵͠Α͏ʂ
  8. ฏۉϕΫτϧ • ֤ಛ௃ྔʢશ෦Ͱ d ݸʣͷฏۉΛฒ΂ͨ΋ͷ • ྫ͑͹ɿ µ = (

    µ1, µ2, . . . , µd)T = ( E { x1 } , E { x2 } , . . . , E { xd })T µ = (µweight, µdiameter, µradius)T = (4.80[g], 2260[µm], 1130[µm])T
  9. ظ଴஋ • ֬཰ม਺͕࿈ଓͷͱ͖ʢ֬཰ີ౓ؔ਺ʣ µi = E { xi } =

    Z dxi xip ( xi) µi = E { xi } = X k x (k) i P ⇣ x (k) i ⌘ • ֬཰ม਺͕཭ࢄͷͱ͖ʢ֬཰࣭ྔؔ਺ʣ µ = ¯ x = 1 N N X i=1 xi • ؍ଌσʔλ͕ N ݸ༩͑ΒΕ͍ͯΔͱ͖
  10. पล֬཰ • ֬཰ม਺͕཭ࢄͷͱ͖ʢ֬཰࣭ྔؔ਺ʣ • ஫໨͍ͯ͠Δಛ௃ྔͰͳ͍΋ͷ͸ͥΜͿ࿨ʢੵ෼ʣ
 ΛͱΔ ϋϯόʔά͕޷͖͔ʁ yes no sum

    ΤϏϑϥΠ ͕޷͖͔ʁ yes 60 40 100 no 30 20 50 sum 90 60 p ( xi) = Z dx1 Z dx2 · · · Z dxi 1 Z dxi+1 · · · Z dxd p ( x1, x2, . . . , xd)
  11. ڞ෼ࢄߦྻ ⌃ = Var { x } = E (

    x µ )( x µ )T = 0 B @ E {( x1 µ1)( x1 µ1)} . . . E {( x1 µ1)( xd µd)} . . . ... . . . E {( xd µd)( x1 µ1)} . . . E {( xd µd)( xd µd)} 1 C A = ( ij)
  12. ૬ؔ܎਺͸ઢܗ૬͔ؔ͠ΩϟονͰ͖ͳ͍ • x = [-2, -1, 0, 1, 2], y

    = x^2 ͷͱ͖
 ૬ؔ܎਺ ρ_xy ͸θϩ https://upload.wikimedia.org/wikipedia/commons/d/d4/Correlation_examples2.svg
  13. ϕΫτϧతͳղऍ • N ݸͷଌఆ͕͋Δͱ͖ɺ ij = 1 N N X

    n=1 ( xni µi)( xnj µj) = 1 N N X n=1 dnidnj = 1 N di · dj di = ( x1i µi, x2i µi, . . . , xNi µi)T = ( d1i, d2i, . . . , dNi)T ͱ͓͘ͱɺڞ෼ࢄ͸಺ੵʢͷఆ਺ഒʣʹͳΔ ϕΫτϧۭؒͱͯ͠
 ѻ͏ͨΊʹ͸
 ֤ಛ௃ྔͰ୯Ґ͕
 ἧ͍ͬͯΔඞཁ͕͋Δ
  14. 2 i = 1 N di · di = 1

    N |di |2 i = = 1 p N |di | ϕΫτϧతͳղऍ ⇢ij = ij i j = (1 /N ) di · dj (1 / p N ) |di | (1 / p N ) |dj | = di · dj |di | |dj | = cos ✓ij
  15. ϕΫτϧతͳղऍ di dj ⇢ij = 1 ⇢ij = 0 ⇢ij

    = 1 di dj di dj { } p N i p N j
  16. • ಛ௃ؒͷ૬ؔΛͳ͍ͨ͘͠ • ୯Ґ͕ҧ͍ͬͯͯ΋౷ܭతʹൺֱ͍ͨ͠ ➡ ؍ଌσʔλΛม׵͠Α͏ʂ ॏ͞ [g] ௚ܘ [μm]

    ൒ܘ [μm] … ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … Πϯτϩʢ࠶๚ʣ
  17. ฏۉɾ෼ࢄͱઢܗม׵ • ઢܗม׵Λߟ͑Δ y = ax + b E {

    y } = E { ax + b } = a E { x } + b = aµ + b • ฏۉͱ෼ࢄ͸ҎԼͷΑ͏ʹԠ౴ Var { y } = E ( y Ey )2 = E [ ax + b ( aµ + b )]2 = E a 2( x µ )2 = a 2E ( x µ )2 = a 2Var { x } = a 2 2
  18. ඪ४Խ • ҎԼͷઢܗม׵Λ࢖͏ͱ
 ฏۉ 0 ɺ෼ࢄ 1 ͷಛ௃ྔ͕ಘΒΕΔ z =

    x µ E { z } = E ⇢ x µ = 1 (E { x } µ ) = 0 Var { z } = Var ⇢ x µ = 1 2 Var { x } = 1
  19. ඪ४Խ x1 x2 µ1 µ2 1 2 1 z1 z2

    1 • σʔλͷฏۉΛ 0 ɺ෼ࢄʢඪ४ภࠩʣΛ 1 ʹ • ແ࣍ݩԽ͞Ε͍ͯΔͷͰɺ୯Ґͷҧ͍΋ٵऩͰ͖Δ
  20. ճసߦྻ • ݻ༗ϕΫτϧΛฒ΂ͯߦྻΛ࡞Δ • ਖ਼ن௚ަجఈΛฒ΂ͨߦྻ͸௚ަߦྻͱͳΔ S = (s1, s2, .

    . . , sd) (ST S)ij = sT i sj = ij ) ST S = I ) ST = S 1 • ͜ͷ৔߹͸ճసߦྻʢ㱬௚ަߦྻʣͱͳΔ
  21. ແ૬ؔԽ y = ST x E { y } =

    E ST x = ST E { x } = ST µ Var { y } = E ( y E { y })( y E { y })T = E (ST x ST µ )(ST x ST µ )T = E ST ( x µ )[ST ( x µ )]T = E ST ( x µ )( x µ )T S = ST E ( x µ )( x µ )T S = ST ⌃S
  22. ແ૬ؔԽ S 1⌃S = S 1⌃(s1, s2, . . .

    , sd) = S 1( 1s1, 2s2, . . . , dsd) = S 1S 0 B B B @ 1 0 . . . 0 0 2 . . . 0 . . . ... . . . 0 0 . . . d 1 C C C A = 0 B B B @ 1 0 . . . 0 0 2 . . . 0 . . . ... . . . 0 0 . . . d 1 C C C A = ⇤ • ͳͷͰɺແ૬ؔԽ͞Ε͍ͯΔ • ແ૬͕ؔͩɺඪ४Խ͸͞Ε͍ͯͳ͍ (Var {y})ij = 0 (i 6= j) (Var {y})ii = i
  23. ന৭Խ • ඪ४Խʴແ૬ؔԽ u = ⇤ 1/2ST ( x µ

    ) (⇤ 1/2)ij = ⇢ 1/ p i (i = j) 0 (i 6= j)
  24. ന৭Խ E { u } = E n⇤ 1/2ST (

    x µ )o = ⇤ 1/2ST (E { x } µ ) = 0 Var { u } = E n⇤ 1/2ST ( x µ )( x µ )T S(⇤ 1/2)T o = ⇤ 1/2ST E ( x µ )( x µ )T S(⇤ 1/2)T = ⇤ 1/2ST ⌃S(⇤ 1/2)T = ⇤ 1/2⇤(⇤ 1/2)T = I • ඪ४Խ͞Εɺ͔ͭແ૬ؔԽ͞Εͨʂ
  25. 4.1 ͷ·ͱΊ • ඪ४Խ • ୯ҐΛͦΖ͑ɺฏۉΛ 0 ɺ෼ࢄΛ 1 ʹ͢Δ

    • ແ૬ؔԽ • ૬͕ؔͳ͘ͳΔΑ͏ʹۭؒΛճసͤ͞Δ • ඪ४Խ͸͞Εͯͳ͍ • ന৭Խ • ඪ४Խ ʴ ແ૬ؔԽ • ୯ҐΛͦΖ͑ɺฏۉΛ 0 ɺ෼ࢄΛ 1 ʹ͠ɺ
 ૬͕ؔͳ͘ͳΔΑ͏ʹۭؒΛճసͤ͞Δ
  26. ֬཰Ϟσϧ͋Ε͜Ε • ύϥϝτϦοΫϞσϧ • ֬཰ม਺͕཭ࢄʢ֬཰࣭ྔؔ਺ʣ • ೋ߲෼෍ɺଟ߲෼෍ɺϙΞιϯ෼෍ͳͲ • ֬཰ม਺͕࿈ଓʢ֬཰ີ౓ؔ਺ʣ •

    ਖ਼ن෼෍ɺΧΠೋ৐෼෍ɺίʔγʔ෼෍ͳͲ • ϊϯύϥϝτϦοΫϞσϧ • ώετάϥϜ๏ɺkNN ๏ɺύϧπΣϯີ౓ਪఆ๏ ͳͲ
  27. ਖ਼ن෼෍ͷੑ࣭ʢൈਮʣ • ղੳతʹΑ͘ௐ΂ΒΕ͍ͯΔ • ඇਖ਼ن෼෍ʹ͕ͨ͠͏σʔλ΋
 ඪຊฏۉͷ෼෍͸ਖ਼ن෼෍ʹͳΔʢத৺ۃݶఆཧʣ • ਖ਼ن෼෍ʹ͕ͨ͠͏σʔλͷઢܗม׵͸
 ਖ਼ن෼෍ʹ͕ͨ͠͏ •

    ਖ਼ن෼෍ʹ͕ͨ͠͏ෳ਺ͷ֬཰ม਺ͷઢܗ݁߹͸
 ਖ਼ن෼෍ͱ͍͏ʢ࠶ੜੑʣ • ແ૬ؔͰ͋Δ͜ͱͱ౷ܭతʹಠཱͰ͋Δ͜ͱ͕౳Ձ
 ʢʮਖ਼ن෼෍ʹݶΓʯͷ෦෼ˠ ʮ਺ֶηϛφʔʯʹࡌͬͯΔ͔΋ʁʣ
  28. ਖ਼ن෼෍ • 1 ࣍ݩਖ਼ن෼෍ N(x | µ, 2 ) =

    1 p 2⇡ 2 exp  (x µ) 2 2 2 • ଟ࣍ݩਖ਼ن෼෍ N (x | µ , ⌃) = 1 (2 ⇡ ) d/2 | ⌃ |1/2 exp  1 2 (x µ) T ⌃ 1 (x µ)
  29. ਖ਼ن෼෍ • ૬ؔͷ෼͚ͩճస͠ɺඪ४ภࠩͷ෼͚ͩҾ͖৳͹͞Ε͍ͯΔ • ന৭Խͷٯ ( x µ )T ⌃

    1( x µ ) = ( x µ )T [S⇤S 1] 1( x µ ) = ( x µ )T S⇤ 1S( x µ ) = [ST ( x µ )]T ⇤ 1[ST ( x µ )] = y T ⇤ 1 y (* y ⌘ ST ( x µ )) = y T (⇤1/2)T ⇤1/2 y = (⇤1/2 y )T (⇤1/2 y ) = z T z (* z ⌘ ⇤ 1/2 y )
  30. Ϋϥε৚݅෇͖֬཰ • Ϋϥε৚݅෇͖֬཰͕ਖ਼ن෼෍Ͱ͋ΔͱԾఆ͢Δ ln P ( Ci | x) =

    p (x |Ci) P ( Ci) p (x) / p (x |Ci) P ( Ci) = P ( Ci) (2 ⇡ ) d/2 | ⌃i |1/2 exp  1 2 (x µi) T ⌃ 1 i (x µi) p (x |Ci) = 1 (2 ⇡ ) d/2 | ⌃i |1/2 exp  1 2 (x µi) T ⌃ 1 i (x µi)
  31. Ϋϥε৚݅෇͖֬཰ • ؔ܎ͳ͍߲ΛΦϛοτɺ ×(-2) ln P(Ci | x ) =

    ln P(Ci) d 2 ln(2⇡) 1 2 ln |⌃i |1/2 1 2 ( x µi)T ⌃ 1 i ( x µi) gi( x ) = ( x µi)T ⌃ 1 i ( x µi) + ln |⌃i | 2 ln P(Ci) [Recognized class] = arg min i [ gi(x)]
  32. ࣝผڥք fij( x ) = gi( x ) gj( x

    ) = ( x µi)T ⌃ 1 i ( x µi) + ln |⌃i | 2 ln P(Ci) ( x µj)T ⌃ 1 j ( x µj) ln |⌃j | + 2 ln P(Cj) = x ⌃ 1 i x x ⌃ 1 i µi µi⌃ 1 i x + µi⌃ 1 i µi x ⌃ 1 j x x ⌃ 1 j µj µj⌃ 1 j x + µj⌃ 1 j µj + ln |⌃i | ⌃j 2 ln P(Ci) P(Cj) = x (⌃ 1 i ⌃ 1 j ) x + 2( µ T j ⌃ 1 j µ T i ⌃ 1 i ) x + µT i ⌃ 1 i µiµT j ⌃ 1 j µj + ln |⌃i | ⌃j 2 ln P(Ci) P(Cj) ) x T S x + 2 c T x + F = 0 ʢ2 ࣍ࣝผؔ਺ʣ +µT i ⌃ 1 i µi µT j ⌃ 1 j µj + ln |⌃i | |⌃j | 2 ln P(Ci) P(Cj) fij( x ) = gi( x ) gj( x ) = ( x µi)T ⌃ 1 i ( x µi) + ln |⌃i | 2 ln P(Ci) ( x µj)T ⌃ 1 j ( x µj) ln |⌃j | + 2 ln P(Cj) = x ⌃ 1 i x x ⌃ 1 i µi µi⌃ 1 i x + µi⌃ 1 i µi x ⌃ 1 j x x ⌃ 1 j µj µj⌃ 1 j x + µj⌃ 1 j µj + ln |⌃i | ⌃j 2 ln P(Ci) P(Cj) = x (⌃ 1 i ⌃ 1 j ) x + 2( µ T j ⌃ 1 j µ T i ⌃ 1 i ) x + µT i ⌃ 1 i µiµT j ⌃ 1 j µj + ln |⌃i | ⌃j 2 ln P(Ci) P(Cj) x ⌃ 1 j x + x ⌃ 1 j µj + µj⌃ 1 j x µj⌃ 1 j µj
  33. ࣝผڥքʢิ୊ʣ x T i ⌃ 1 i µi = (Scalar)

    = (x T i ⌃ 1 i µi) T = µ T i (⌃ 1 i ) T x = µ T i ⌃ 1 i x ( * ⌃ 1 i is a symmetric matrix) • ରশߦྻͷٯߦྻ͸ରশߦྻͰ͋Δ͜ͱʹ஫ҙ
  34. ࣝผڥք ⌃i = ⌃j = I P(Ci) = P(Cj) •

    ͔ͭ ͷͱ͖ fij( x ) = 2( µ T j ⌃ 1 j µ T j ⌃ 1 i ) x + µ T i ⌃ 1 i µi µ T i ⌃ 1 i µi = 2 ( µ T j µ T i ) x + µ T i µi µ T j µj = 0 x T x + 2 µ T i x + µ T i µi x T x + 2 µ T j x µ T j µj = 0 ( x µi)T ( x µi) ( x µj)T ( x µj) = 0 ) ( x µi)T ( x µi) = ( x µj)T ( x µj) x T x 2 µ T i x + µ T i µi x T x + 2 µ T j x µ T j µj = 0 • ྆ล σ ͰׂΓɺ x^T x Λ଍͠Ҿ͖ ʢ࠷ۙ๣๏……ͱຊʹ͸ॻ͍ͯ͋Δ͕ઢܗ൑ผ෼ੳʢLDAʣͰ͸ʁ ʢฏۉ͔Βͷڑ཭Λൺ΂͍ͯΔͷͰʣʣ +µT i ⌃ 1 i µi µT j ⌃ 1 j µj fij( x ) = 2( µ T j ⌃ 1 j µ T j ⌃ 1 i ) x + µ T i ⌃ 1 i µi µ T i ⌃ 1 i µi = 2 ( µ T j µ T i ) x + µ T i µi µ T j µj = 0
  35. ࠷໬ਪఆ๏ • ࣮༻্͸σʔλ͕༩͑ΒΕ͍ͯͯύϥϝʔλ͕ະ஌ • ύϥϝʔλΛม਺ͱͯ͠ಉ࣌෼෍ΛͱΒ͑Δ L( ✓ ) = f(

    x1, x2, . . . , xN | ✓ ) ʢ໬౓ؔ਺ʣ • ໬౓ؔ਺Λ࠷େʹ͢ΔύϥϝʔλΛٻΊΔ
 ʢ࠷໬ਪఆ๏ʣ • ର਺Λͱͬͯ΋ۃ஋ͷҐஔ͸มΘΒͳ͍ͷͰ
 ໬౓ؔ਺ͷର਺ΛͱͬͯܭࢉΛ؆୯ʹͰ͖Δ͜ͱ͕͋Δ
  36. 1 ม਺ਖ਼ن෼෍ͷ৔߹ L(µ, 2 ) = f(x1, x2, . .

    . , xN | µ, 2 ) = N Y i=1 1 p 2⇡ 2 exp  (xi µ) 2 2 2 = (2⇡ 2 ) N/2 exp " 1 2 2 N X i=1 (xi µ) 2 # Lln( µ, 2) = N 2 ln(2 ⇡ ) N 2 ln 2 1 2 2 N X i=1 ( xi µ )2
  37. 1 ม਺ਖ਼ن෼෍ͷ৔߹ Lln( µ, 2) = N 2 ln(2 ⇡

    ) N 2 ln 2 1 2 2 N X i=1 ( xi µ )2 • ର਺໬౓Λ֤ύϥϝʔλͰภඍ෼ͯ͠ۃ஋ΛٻΊΔ
  38. 1 ม਺ਖ਼ن෼෍ͷ৔߹ @Lln(ˆ µ, 2) @µ = @ @µ "

    1 2 2 N X i=1 ( xi µ )2 # µ=ˆ µ = 0 1 2 2 N X i=1 2( xi ˆ µ )( 1) = 0 N X i=1 xi N X i=1 ˆ µ = 0 ) ˆ µ = 1 N N X i=1 xi
  39. 1 ม਺ਖ਼ن෼෍ͷ৔߹ @Lln( µ, ˆ2) @ 2 = @ @

    2 " N 2 ln 2 1 2 2 N X i=1 ( xi µ )2 # 2=ˆ2 = 0 N 2 1 ˆ2 1 2 1 (ˆ2)2 ( 1) N X i=1 ( xi µ )2 = 0 N ˆ2 + 1 (ˆ2)2 N X i=1 ( xi µ )2 = 0 ) ˆ2 = 1 N N X i=1 ( xi µ )2