Upgrade to Pro — share decks privately, control downloads, hide ads and more …

「はじめてのパターン認識」読書会 第 4 章

horiem
November 09, 2017
920

「はじめてのパターン認識」読書会 第 4 章

horiem

November 09, 2017
Tweet

Transcript

  1. ʮ͸͡Ίͯͷύλʔϯೝࣝʯಡॻձ
    ୈ 4 ষ
    2017 ೥ 11 ݄ 9 ೔

    horiem

    View Slide

  2. Πϯτϩ: 100 ԁۄͷਅآ൑ఆ
    ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …

    View Slide

  3. Πϯτϩ: 100 ԁۄͷਅآ൑ఆ
    ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …
    ૬ؔ͋ΔͷͰ͸ʁ

    View Slide

  4. Πϯτϩ: 100 ԁۄͷਅآ൑ఆ
    ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …
    ૬ؔ͋ΔͷͰ͸ʁ

    View Slide

  5. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …
    Πϯτϩ: 100 ԁۄͷਅآ൑ఆ
    100 ԁۄͷฏۉʢ4.8 gʣΑΓܰͦ͏͕ͩ

    ࠩ͸ 0.05 [g]

    View Slide

  6. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …
    Πϯτϩ: 100 ԁۄͷਅآ൑ఆ
    100 ԁۄͷฏۉʢ22600 μmʣͱಉ͡Α͏͕ͩ

    ࠩ͸ 2 [μm] >> 0.05

    View Slide

  7. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …
    Πϯτϩ: 100 ԁۄͷਅآ൑ఆ
    • ಛ௃ؒͷ૬ؔΛͳ͍ͨ͘͠
    • ୯Ґ͕ҧ͍ͬͯͯ΋౷ܭతʹൺֱ͍ͨ͠

    View Slide

  8. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …
    Πϯτϩ: 100 ԁۄͷਅآ൑ఆ
    • ಛ௃ؒͷ૬ؔΛͳ͍ͨ͘͠
    • ୯Ґ͕ҧ͍ͬͯͯ΋౷ܭతʹൺֱ͍ͨ͠
    ➡ ؍ଌσʔλΛม׵͠Α͏ʂ

    View Slide

  9. 4 ষͷ΋͘͡
    4. ֬཰Ϟσϧͱࣝผؔ਺
    1. ؍ଌσʔλͷઢܗม׵
    2. ֬཰Ϟσϧ
    3. ֬཰Ϟσϧύϥϝʔλͷ࠷໬ਪఆ

    View Slide

  10. 4 ষͷ΋͘͡
    4. ֬཰Ϟσϧͱࣝผؔ਺
    1. ؍ଌσʔλͷઢܗม׵
    2. ֬཰Ϟσϧ
    3. ֬཰Ϟσϧύϥϝʔλͷ࠷໬ਪఆ

    View Slide

  11. ฏۉϕΫτϧͱڞ෼ࢄߦྻ

    View Slide

  12. ฏۉϕΫτϧ
    • ֤ಛ௃ྔʢશ෦Ͱ d ݸʣͷฏۉΛฒ΂ͨ΋ͷ
    • ྫ͑͹ɿ
    µ = (
    µ1, µ2, . . . , µd)T = (
    E
    {
    x1
    }
    , E
    {
    x2
    }
    , . . . , E
    {
    xd
    })T
    µ = (µweight, µdiameter, µradius)T
    = (4.80[g], 2260[µm], 1130[µm])T

    View Slide

  13. ظ଴஋
    • ֬཰ม਺͕࿈ଓͷͱ͖ʢ֬཰ີ౓ؔ਺ʣ
    µi =
    E
    {
    xi
    } =
    Z
    dxi xip
    (
    xi)
    µi =
    E
    {
    xi
    } =
    X
    k
    x
    (k)
    i P

    x
    (k)
    i

    • ֬཰ม਺͕཭ࢄͷͱ͖ʢ֬཰࣭ྔؔ਺ʣ
    µ
    = ¯
    x
    =
    1
    N
    N
    X
    i=1
    xi
    • ؍ଌσʔλ͕ N ݸ༩͑ΒΕ͍ͯΔͱ͖

    View Slide

  14. पล֬཰
    • ֬཰ม਺͕཭ࢄͷͱ͖ʢ֬཰࣭ྔؔ਺ʣ
    • ஫໨͍ͯ͠Δಛ௃ྔͰͳ͍΋ͷ͸ͥΜͿ࿨ʢੵ෼ʣ

    ΛͱΔ
    ϋϯόʔά͕޷͖͔ʁ
    yes no sum
    ΤϏϑϥΠ
    ͕޷͖͔ʁ
    yes 60 40 100
    no 30 20 50
    sum 90 60
    p
    (
    xi) =
    Z
    dx1
    Z
    dx2
    · · ·
    Z
    dxi 1
    Z
    dxi+1
    · · ·
    Z
    dxd p
    (
    x1, x2, . . . , xd)

    View Slide

  15. ڞ෼ࢄߦྻ
    ⌃ = Var {
    x
    }
    = E (
    x µ
    )(
    x µ
    )T
    =
    0
    B
    @
    E {(
    x1 µ1)(
    x1 µ1)}
    . . .
    E {(
    x1 µ1)(
    xd µd)}
    .
    .
    .
    ...
    .
    .
    .
    E {(
    xd µd)(
    x1 µ1)}
    . . .
    E {(
    xd µd)(
    xd µd)}
    1
    C
    A
    = ( ij)

    View Slide

  16. ڞ෼ࢄߦྻ
    • ؍ଌσʔλ͕ N ݸ༩͑ΒΕ͍ͯΔͱ͖
    ij =
    1
    N
    N
    X
    n=1
    (
    xni µi)(
    xnj µj)

    View Slide

  17. ڞ෼ࢄߦྻ
    • ຊདྷ͸ෆภ෼ࢄΛ࢖͏΂͖
    • ظ଴஋Λͱͬͨͱ͖ʹਅͷ෼ࢄʹ

    ऩଋ͢ΔΑ͏ௐઅ͢Δ
    • ඪຊ਺͕ଟ͍৔߹͸େࠩͳ͍ͷͰ

    ͜͜Ͱ͸γϯϓϧʹ͍ͯ͠Δ
    sij =
    1
    N
    1
    N
    X
    n=1
    (
    xni µi)(
    xnj µj)

    View Slide

  18. ෼ࢄͱඪ४ภࠩ
    • ෼ࢄ͸ฏۉ͔Βͷೋ৐ޡࠩͷظ଴஋
    • େ͖͚Ε͹͹Β͍͍ͭͯΔ
    • ෼ࢄͷฏํ͕ࠜඪ४ภࠩ
    • ෼ࢄͩͱಛ௃ྔͷ୯Ґ͕มΘͬͯ͠·͏ͨΊ

    ΋ͱͷ୯Ґʹ໭͢
    2
    i
    = ii = E (
    xi µi)2
    i =
    q
    2
    i

    View Slide

  19. ڞ෼ࢄͱ૬ؔ܎਺
    • ڞ෼ࢄ͸ҟͳΔಛ௃ྔؒͰ͹Β͖ͭํ͕

    ಉ͔͡Ͳ͏͔Λಛ௃͚ͮΔ
    • ਖ਼ͷ૬͕ؔ͋Ε͹ +ɺෛͷ૬͕ؔ͋Ε͹ -
    • ڞ෼ࢄ͸୯ҐΛ͍࣋ͬͯΔͨΊ

    ୯Ґ͕ҧ͏ڞ෼ࢄͲ͏͠ΛൺֱͰ͖ͳ͍
    ij = E {(
    xi µi)(
    xj µj)}

    View Slide

  20. ڞ෼ࢄͱ૬ؔ܎਺
    ⇢ij = ij
    i j
    • ڞ෼ࢄΛແ࣍ݩԽͨ͠ͷ͕૬ؔ܎਺
    • ਖ਼ͷ૬͕ؔ͋Ε͹ +ɺෛͷ૬͕ؔ͋Ε͹ -
    • ඞͣ [-1, 1] ΛͱΔ

    View Slide

  21. ૬ؔ܎਺͸ઢܗ૬͔ؔ͠ΩϟονͰ͖ͳ͍
    • x = [-2, -1, 0, 1, 2], y = x^2 ͷͱ͖

    ૬ؔ܎਺ ρ_xy ͸θϩ
    https://upload.wikimedia.org/wikipedia/commons/d/d4/Correlation_examples2.svg

    View Slide

  22. ϕΫτϧతͳղऍ
    • N ݸͷଌఆ͕͋Δͱ͖ɺ
    ij =
    1
    N
    N
    X
    n=1
    (
    xni µi)(
    xnj µj)
    =
    1
    N
    N
    X
    n=1
    dnidnj
    =
    1
    N
    di
    · dj
    di = (
    x1i µi, x2i µi, . . . , xNi µi)T
    = (
    d1i, d2i, . . . , dNi)T
    ͱ͓͘ͱɺڞ෼ࢄ͸಺ੵʢͷఆ਺ഒʣʹͳΔ
    ϕΫτϧۭؒͱͯ͠

    ѻ͏ͨΊʹ͸

    ֤ಛ௃ྔͰ୯Ґ͕

    ἧ͍ͬͯΔඞཁ͕͋Δ

    View Slide

  23. 2
    i
    =
    1
    N
    di
    · di
    =
    1
    N
    |di
    |2
    i = =
    1
    p
    N
    |di
    |
    ϕΫτϧతͳղऍ
    ⇢ij =
    ij
    i j
    =
    (1
    /N
    )
    di
    · dj
    (1
    /
    p
    N
    )
    |di
    |
    (1
    /
    p
    N
    )
    |dj
    |
    =
    di
    · dj
    |di
    | |dj
    |
    = cos
    ✓ij

    View Slide

  24. ϕΫτϧతͳղऍ
    • ׬શͳਖ਼ͷ૬͕ؔ͋Δͱ͖
    ⇢ij = 1
    cos
    ✓ij = 1
    ✓ij = 0
    di =
    cdj (
    c >
    0)

    View Slide

  25. ϕΫτϧతͳղऍ
    di
    dj
    ⇢ij = 1 ⇢ij = 0 ⇢ij = 1
    di
    dj
    di
    dj
    {
    }
    p
    N i
    p
    N j

    View Slide

  26. ؍ଌσʔλͷඪ४Խ

    View Slide

  27. • ಛ௃ؒͷ૬ؔΛͳ͍ͨ͘͠
    • ୯Ґ͕ҧ͍ͬͯͯ΋౷ܭతʹൺֱ͍ͨ͠
    ➡ ؍ଌσʔλΛม׵͠Α͏ʂ
    ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …
    Πϯτϩʢ࠶๚ʣ

    View Slide

  28. ฏۉɾ෼ࢄͱઢܗม׵
    • ઢܗม׵Λߟ͑Δ
    y
    =
    ax
    +
    b
    E {
    y
    } = E {
    ax
    +
    b
    } =
    a
    E {
    x
    } +
    b
    =

    +
    b
    • ฏۉͱ෼ࢄ͸ҎԼͷΑ͏ʹԠ౴
    Var {
    y
    } = E (
    y Ey
    )2
    = E [
    ax
    +
    b
    (

    +
    b
    )]2 = E
    a
    2(
    x µ
    )2
    =
    a
    2E (
    x µ
    )2 =
    a
    2Var {
    x
    }
    =
    a
    2 2

    View Slide

  29. ඪ४Խ
    • ҎԼͷઢܗม׵Λ࢖͏ͱ

    ฏۉ 0 ɺ෼ࢄ 1 ͷಛ௃ྔ͕ಘΒΕΔ
    z
    = x µ
    E {
    z
    } = E

    x µ =
    1
    (E {
    x
    }
    µ
    )
    = 0
    Var {
    z
    } = Var

    x µ =
    1
    2
    Var {
    x
    }
    = 1

    View Slide

  30. ඪ४Խ
    x1
    x2

    View Slide

  31. ඪ४Խ
    x1
    x2

    View Slide

  32. ඪ४Խ
    x1
    x2

    View Slide

  33. ඪ४Խ
    x1
    x2
    µ1
    µ2

    View Slide

  34. ඪ४Խ
    x1
    x2
    µ1
    µ2
    1
    2

    View Slide

  35. ඪ४Խ
    x1
    x2
    µ1
    µ2
    1
    2
    1
    z1
    z2
    1

    View Slide

  36. ඪ४Խ
    x1
    x2
    µ1
    µ2
    1
    2
    1
    z1
    z2
    1
    • σʔλͷฏۉΛ 0 ɺ෼ࢄʢඪ४ภࠩʣΛ 1 ʹ
    • ແ࣍ݩԽ͞Ε͍ͯΔͷͰɺ୯Ґͷҧ͍΋ٵऩͰ͖Δ

    View Slide

  37. ؍ଌσʔλͷແ૬ؔԽ

    View Slide

  38. ݻ༗ϕΫτϧ
    • ڞ෼ࢄߦྻͷݻ༗஋໰୊Λղ͘ͱ

    d ຊͷϕΫτϧ͕ಘΒΕΔ
    • ڞ෼ࢄߦྻ͸ [d, d] ͷରশߦྻ
    ⌃si = isi

    View Slide

  39. ݻ༗ϕΫτϧ
    • ରশߦྻʹରͯ͠ɿ
    • ݻ༗஋͸࣮਺
    • ݻ༗ϕΫτϧ͸௚ަ
    ➡ ݻ༗ϕΫτϧ͸ਖ਼ن௚ަجఈ
    sT
    i
    sj = ij =

    1 (i = j)
    0 (i 6= j)

    View Slide

  40. ճసߦྻ
    • ݻ༗ϕΫτϧΛฒ΂ͯߦྻΛ࡞Δ
    • ਖ਼ن௚ަجఈΛฒ΂ͨߦྻ͸௚ަߦྻͱͳΔ
    S = (s1, s2, . . . , sd)
    (ST S)ij = sT
    i
    sj = ij
    ) ST S = I
    ) ST = S 1
    • ͜ͷ৔߹͸ճసߦྻʢ㱬௚ަߦྻʣͱͳΔ

    View Slide

  41. ແ૬ؔԽ
    y
    = ST
    x
    E {
    y
    } = E ST
    x
    = ST E {
    x
    }
    = ST
    µ
    Var {
    y
    } = E (
    y
    E {
    y
    })(
    y
    E {
    y
    })T
    = E (ST
    x
    ST
    µ
    )(ST
    x
    ST
    µ
    )T
    = E ST (
    x µ
    )[ST (
    x µ
    )]T
    = E ST (
    x µ
    )(
    x µ
    )T S
    = ST E (
    x µ
    )(
    x µ
    )T S
    = ST ⌃S

    View Slide

  42. ແ૬ؔԽ
    S 1⌃S = S 1⌃(s1, s2, . . . , sd) = S 1( 1s1, 2s2, . . . , dsd)
    = S 1S
    0
    B
    B
    B
    @
    1 0 . . . 0
    0 2 . . . 0
    .
    .
    .
    ...
    .
    .
    .
    0 0 . . . d
    1
    C
    C
    C
    A
    =
    0
    B
    B
    B
    @
    1 0 . . . 0
    0 2 . . . 0
    .
    .
    .
    ...
    .
    .
    .
    0 0 . . . d
    1
    C
    C
    C
    A
    = ⇤
    • ͳͷͰɺແ૬ؔԽ͞Ε͍ͯΔ
    • ແ૬͕ؔͩɺඪ४Խ͸͞Ε͍ͯͳ͍
    (Var {y})ij = 0 (i 6= j)
    (Var {y})ii = i

    View Slide

  43. ന৭Խ

    View Slide

  44. ന৭Խ
    • ඪ४Խʴແ૬ؔԽ
    u
    = ⇤ 1/2ST (
    x µ
    )
    (⇤ 1/2)ij =

    1/
    p
    i (i = j)
    0 (i 6= j)

    View Slide

  45. ന৭Խ
    E {
    u
    } = E n⇤ 1/2ST (
    x µ
    )o = ⇤ 1/2ST (E {
    x
    }
    µ
    )
    = 0
    Var {
    u
    } = E n⇤ 1/2ST (
    x µ
    )(
    x µ
    )T S(⇤ 1/2)T o
    = ⇤ 1/2ST E (
    x µ
    )(
    x µ
    )T S(⇤ 1/2)T
    = ⇤ 1/2ST ⌃S(⇤ 1/2)T
    = ⇤ 1/2⇤(⇤ 1/2)T
    = I
    • ඪ४Խ͞Εɺ͔ͭແ૬ؔԽ͞Εͨʂ

    View Slide

  46. 4.1 ͷ·ͱΊ
    • ඪ४Խ
    • ୯ҐΛͦΖ͑ɺฏۉΛ 0 ɺ෼ࢄΛ 1 ʹ͢Δ
    • ແ૬ؔԽ
    • ૬͕ؔͳ͘ͳΔΑ͏ʹۭؒΛճసͤ͞Δ
    • ඪ४Խ͸͞Εͯͳ͍
    • ന৭Խ
    • ඪ४Խ ʴ ແ૬ؔԽ
    • ୯ҐΛͦΖ͑ɺฏۉΛ 0 ɺ෼ࢄΛ 1 ʹ͠ɺ

    ૬͕ؔͳ͘ͳΔΑ͏ʹۭؒΛճసͤ͞Δ

    View Slide

  47. 4 ষͷ΋͘͡
    4. ֬཰Ϟσϧͱࣝผؔ਺
    1. ؍ଌσʔλͷઢܗม׵
    2. ֬཰Ϟσϧ
    3. ֬཰Ϟσϧύϥϝʔλͷ࠷໬ਪఆ

    View Slide

  48. ֬཰Ϟσϧ
    • σʔλͷ෼෍ͷ਺ཧϞσϧ
    • ύϥϝτϦοΫϞσϧ
    • ෼෍ؔ਺ΛԾఆ͠ɺύϥϝʔλΛܾఆͯ͠

    ϞσϧԽ͢Δ
    • ϊϯύϥϝτϦοΫϞσϧ
    • ಛఆͷ෼෍ΛԾఆͤͣɺσʔλͦͷ΋ͷ͔Β

    ෼෍ͷදݱΛಘΔ

    View Slide

  49. ֬཰Ϟσϧ͋Ε͜Ε
    • ύϥϝτϦοΫϞσϧ
    • ֬཰ม਺͕཭ࢄʢ֬཰࣭ྔؔ਺ʣ
    • ೋ߲෼෍ɺଟ߲෼෍ɺϙΞιϯ෼෍ͳͲ
    • ֬཰ม਺͕࿈ଓʢ֬཰ີ౓ؔ਺ʣ
    • ਖ਼ن෼෍ɺΧΠೋ৐෼෍ɺίʔγʔ෼෍ͳͲ
    • ϊϯύϥϝτϦοΫϞσϧ
    • ώετάϥϜ๏ɺkNN ๏ɺύϧπΣϯີ౓ਪఆ๏
    ͳͲ

    View Slide

  50. ਖ਼ن෼෍ؔ਺

    View Slide

  51. ਖ਼ن෼෍ͷੑ࣭ʢൈਮʣ
    • ղੳతʹΑ͘ௐ΂ΒΕ͍ͯΔ
    • ඇਖ਼ن෼෍ʹ͕ͨ͠͏σʔλ΋

    ඪຊฏۉͷ෼෍͸ਖ਼ن෼෍ʹͳΔʢத৺ۃݶఆཧʣ
    • ਖ਼ن෼෍ʹ͕ͨ͠͏σʔλͷઢܗม׵͸

    ਖ਼ن෼෍ʹ͕ͨ͠͏
    • ਖ਼ن෼෍ʹ͕ͨ͠͏ෳ਺ͷ֬཰ม਺ͷઢܗ݁߹͸

    ਖ਼ن෼෍ͱ͍͏ʢ࠶ੜੑʣ
    • ແ૬ؔͰ͋Δ͜ͱͱ౷ܭతʹಠཱͰ͋Δ͜ͱ͕౳Ձ

    ʢʮਖ਼ن෼෍ʹݶΓʯͷ෦෼ˠ ʮ਺ֶηϛφʔʯʹࡌͬͯΔ͔΋ʁʣ

    View Slide

  52. ਖ਼ن෼෍
    • 1 ࣍ݩਖ਼ن෼෍
    N(x
    |
    µ,
    2
    ) =
    1
    p
    2⇡
    2 exp

    (x µ)
    2
    2
    2
    • ଟ࣍ݩਖ਼ن෼෍
    N
    (x
    |
    µ
    ,
    ⌃) =
    1
    (2

    )
    d/2 |

    |1/2 exp

    1
    2
    (x µ)
    T

    1
    (x µ)

    View Slide

  53. ਖ਼ن෼෍
    • ૬ؔͷ෼͚ͩճస͠ɺඪ४ภࠩͷ෼͚ͩҾ͖৳͹͞Ε͍ͯΔ
    • ന৭Խͷٯ
    (
    x µ
    )T ⌃ 1(
    x µ
    ) = (
    x µ
    )T [S⇤S 1] 1(
    x µ
    )
    = (
    x µ
    )T S⇤ 1S(
    x µ
    )
    = [ST (
    x µ
    )]T ⇤ 1[ST (
    x µ
    )]
    =
    y
    T ⇤ 1
    y
    (*
    y
    ⌘ ST (
    x µ
    ))
    =
    y
    T (⇤1/2)T ⇤1/2
    y
    = (⇤1/2
    y
    )T (⇤1/2
    y
    )
    =
    z
    T
    z
    (*
    z
    ⌘ ⇤ 1/2
    y
    )

    View Slide

  54. ϚϋϥϊϏεڑ཭
    • ෼෍ͷத৺͔ΒͲΕ͚ͩ཭Ε͍ͯΔ͔ͷࢦඪ
    • ന৭Խۭͨؒ͠Ͱͷڑ཭
    d(
    x
    ,
    µ
    ) =
    q
    (
    x µ
    )T ⌃ 1(
    x µ
    )
    =
    p
    z
    T
    z

    View Slide

  55. ਖ਼ن෼෍͔Βಋ͔ΕΔ

    ࣝผؔ਺

    View Slide

  56. Ϋϥε৚݅෇͖֬཰
    • Ϋϥε৚݅෇͖֬཰͕ਖ਼ن෼෍Ͱ͋ΔͱԾఆ͢Δ
    ln
    P
    (
    Ci
    |
    x) =
    p
    (x
    |Ci)
    P
    (
    Ci)
    p
    (x)
    / p
    (x
    |Ci)
    P
    (
    Ci)
    =
    P
    (
    Ci)
    (2

    )
    d/2 |
    ⌃i
    |1/2 exp

    1
    2
    (x µi)
    T

    1
    i (x µi)
    p
    (x
    |Ci) =
    1
    (2

    )
    d/2 |
    ⌃i
    |1/2 exp

    1
    2
    (x µi)
    T

    1
    i (x µi)

    View Slide

  57. Ϋϥε৚݅෇͖֬཰
    • ؔ܎ͳ͍߲ΛΦϛοτɺ ×(-2)
    ln P(Ci
    |
    x
    ) = ln P(Ci)
    d
    2
    ln(2⇡)
    1
    2
    ln |⌃i
    |1/2
    1
    2
    (
    x µi)T ⌃ 1
    i
    (
    x µi)
    gi(
    x
    ) = (
    x µi)T ⌃ 1
    i
    (
    x µi) + ln |⌃i
    | 2 ln P(Ci)
    [Recognized class] = arg min
    i [
    gi(x)]

    View Slide

  58. ࣝผڥք
    • Ϋϥεؒͷڥ໨ʢࣝผڥքʣ͸ҎԼͷํఔࣜͰ

    ༩͑ΒΕΔ
    fij(
    x
    ) = gi(
    x
    ) gj(
    x
    ) = 0

    View Slide

  59. ࣝผڥք
    fij(
    x
    ) = gi(
    x
    ) gj(
    x
    )
    = (
    x µi)T ⌃ 1
    i
    (
    x µi) + ln |⌃i
    | 2 ln P(Ci)
    (
    x µj)T ⌃ 1
    j
    (
    x µj) ln |⌃j
    | + 2 ln P(Cj)
    =
    x
    ⌃ 1
    i x x
    ⌃ 1
    i µi µi⌃ 1
    i x
    +
    µi⌃ 1
    i µi
    x
    ⌃ 1
    j x x
    ⌃ 1
    j µj µj⌃ 1
    j x
    +
    µj⌃ 1
    j µj
    + ln
    |⌃i
    |
    ⌃j
    2 ln
    P(Ci)
    P(Cj)
    =
    x
    (⌃ 1
    i
    ⌃ 1
    j
    )
    x
    + 2(
    µ
    T
    j
    ⌃ 1
    j µ
    T
    i
    ⌃ 1
    i
    )
    x
    + µT
    i
    ⌃ 1
    i
    µiµT
    j
    ⌃ 1
    j
    µj + ln
    |⌃i
    |
    ⌃j
    2 ln
    P(Ci)
    P(Cj)
    )
    x
    T S
    x
    + 2
    c
    T
    x
    + F = 0 ʢ2 ࣍ࣝผؔ਺ʣ
    +µT
    i
    ⌃ 1
    i
    µi µT
    j
    ⌃ 1
    j
    µj + ln
    |⌃i
    |
    |⌃j
    |
    2 ln
    P(Ci)
    P(Cj)
    fij(
    x
    ) = gi(
    x
    ) gj(
    x
    )
    = (
    x µi)T ⌃ 1
    i
    (
    x µi) + ln |⌃i
    | 2 ln P(Ci)
    (
    x µj)T ⌃ 1
    j
    (
    x µj) ln |⌃j
    | + 2 ln P(Cj)
    =
    x
    ⌃ 1
    i x x
    ⌃ 1
    i µi µi⌃ 1
    i x
    +
    µi⌃ 1
    i µi
    x
    ⌃ 1
    j x x
    ⌃ 1
    j µj µj⌃ 1
    j x
    +
    µj⌃ 1
    j µj
    + ln
    |⌃i
    |
    ⌃j
    2 ln
    P(Ci)
    P(Cj)
    =
    x
    (⌃ 1
    i
    ⌃ 1
    j
    )
    x
    + 2(
    µ
    T
    j
    ⌃ 1
    j µ
    T
    i
    ⌃ 1
    i
    )
    x
    + µT
    i
    ⌃ 1
    i
    µiµT
    j
    ⌃ 1
    j
    µj + ln
    |⌃i
    |
    ⌃j
    2 ln
    P(Ci)
    P(Cj)
    x
    ⌃ 1
    j x
    +
    x
    ⌃ 1
    j µj +
    µj⌃ 1
    j x µj⌃ 1
    j µj

    View Slide

  60. ࣝผڥքʢิ୊ʣ
    x
    T
    i ⌃
    1
    i µi = (Scalar) = (x
    T
    i ⌃
    1
    i µi)
    T
    = µ
    T
    i (⌃
    1
    i )
    T
    x
    = µ
    T
    i ⌃
    1
    i x (
    *

    1
    i is a symmetric matrix)
    • ରশߦྻͷٯߦྻ͸ରশߦྻͰ͋Δ͜ͱʹ஫ҙ

    View Slide

  61. ࣝผڥք
    • ;ͨͭͷΫϥεͷڞ෼ࢄߦྻ͕౳͍͠ͱ͖
    fij(
    x
    ) = 2
    c
    T
    x
    + F = 0 ʢઢܗࣝผؔ਺ʣ

    View Slide

  62. ࣝผڥք
    ⌃i = ⌃j = I P(Ci) = P(Cj)
    • ͔ͭ ͷͱ͖
    fij(
    x
    ) = 2(
    µ
    T
    j
    ⌃ 1
    j µ
    T
    j
    ⌃ 1
    i
    )
    x
    +
    µ
    T
    i
    ⌃ 1
    i µi µ
    T
    i
    ⌃ 1
    i µi
    = 2 (
    µ
    T
    j µ
    T
    i
    )
    x
    +
    µ
    T
    i µi µ
    T
    j µj = 0
    x
    T
    x
    + 2
    µ
    T
    i x
    +
    µ
    T
    i µi x
    T
    x
    + 2
    µ
    T
    j x µ
    T
    j µj = 0
    (
    x µi)T (
    x µi) (
    x µj)T (
    x µj) = 0
    ) (
    x µi)T (
    x µi) = (
    x µj)T (
    x µj)
    x
    T
    x
    2
    µ
    T
    i x
    +
    µ
    T
    i µi x
    T
    x
    + 2
    µ
    T
    j x µ
    T
    j µj = 0
    • ྆ล σ ͰׂΓɺ x^T x Λ଍͠Ҿ͖
    ʢ࠷ۙ๣๏……ͱຊʹ͸ॻ͍ͯ͋Δ͕ઢܗ൑ผ෼ੳʢLDAʣͰ͸ʁ
    ʢฏۉ͔Βͷڑ཭Λൺ΂͍ͯΔͷͰʣʣ
    +µT
    i
    ⌃ 1
    i
    µi µT
    j
    ⌃ 1
    j
    µj
    fij(
    x
    ) = 2(
    µ
    T
    j
    ⌃ 1
    j µ
    T
    j
    ⌃ 1
    i
    )
    x
    +
    µ
    T
    i
    ⌃ 1
    i µi µ
    T
    i
    ⌃ 1
    i µi
    = 2 (
    µ
    T
    j µ
    T
    i
    )
    x
    +
    µ
    T
    i µi µ
    T
    j µj = 0

    View Slide

  63. ࣝผڥք
    µi
    µj

    View Slide

  64. ࣝผڥք
    ܾఆڥք
    µi
    µj

    View Slide

  65. ࣝผڥք
    ܾఆڥք
    µi
    µj
    µk

    View Slide

  66. ࣝผڥք
    ܾఆڥք
    µi
    µj
    µk

    View Slide

  67. ࣝผڥք
    ܾఆڥք
    cf. ϘϩϊΠਤ
    µi
    µj
    µk

    View Slide

  68. 4.2 ͷ·ͱΊ
    • ਖ਼ن෼෍͸͍Ζ͍Ζੑ࣭͕͍͍
    • ଟ࣍ݩਖ਼ن෼෍͸

    ૬ؔɾ෼ࢄʹΑΔճసɾऩॖ͕ߟྀ͞Ε͍ͯΔࣜ
    • 2 Ϋϥεؒͷࣄޙ֬཰͕౳͘͠ͳΔ఺ͷي੻͸

    ࣝผڥքΛ༩͑Δ

    View Slide

  69. 4 ষͷ΋͘͡
    4. ֬཰Ϟσϧͱࣝผؔ਺
    1. ؍ଌσʔλͷઢܗม׵
    2. ֬཰Ϟσϧ
    3. ֬཰Ϟσϧύϥϝʔλͷ࠷໬ਪఆ

    View Slide

  70. ಠཱಉ෼෍ͱಉ࣌෼෍
    • ಉҰͷʢਅͷʣ෼෍͔Βಠཱʹαϯϓϧ͞Εͨ΋ͷΛ

    i.i.d. ʢಠཱಉ෼෍: independently and identically distributedʣඪຊ

    ͱ͍͏
    • i.i.d. ͷͱ͖ɺ N ݸͷαϯϓϧͷಉ࣌෼෍͸ҎԼɿ
    f(
    x1,
    x2, . . . ,
    xN
    |

    ) =
    N
    Y
    i=1
    f(
    xi
    |

    )

    View Slide

  71. ࠷໬ਪఆ๏
    • ࣮༻্͸σʔλ͕༩͑ΒΕ͍ͯͯύϥϝʔλ͕ະ஌
    • ύϥϝʔλΛม਺ͱͯ͠ಉ࣌෼෍ΛͱΒ͑Δ
    L(

    ) = f(
    x1,
    x2, . . . ,
    xN
    |

    ) ʢ໬౓ؔ਺ʣ
    • ໬౓ؔ਺Λ࠷େʹ͢ΔύϥϝʔλΛٻΊΔ

    ʢ࠷໬ਪఆ๏ʣ
    • ର਺Λͱͬͯ΋ۃ஋ͷҐஔ͸มΘΒͳ͍ͷͰ

    ໬౓ؔ਺ͷର਺ΛͱͬͯܭࢉΛ؆୯ʹͰ͖Δ͜ͱ͕͋Δ

    View Slide

  72. 1 ม਺ਖ਼ن෼෍ͷ৔߹
    L(µ,
    2
    ) = f(x1, x2, . . . , xN
    |
    µ,
    2
    )
    =
    N
    Y
    i=1
    1
    p
    2⇡
    2 exp

    (xi µ)
    2
    2
    2
    = (2⇡
    2
    )
    N/2
    exp
    "
    1
    2
    2
    N
    X
    i=1
    (xi µ)
    2
    #
    Lln(
    µ,
    2) = N
    2
    ln(2

    ) N
    2
    ln 2
    1
    2 2
    N
    X
    i=1
    (
    xi µ
    )2

    View Slide

  73. 1 ม਺ਖ਼ن෼෍ͷ৔߹
    Lln(
    µ,
    2) = N
    2
    ln(2

    ) N
    2
    ln 2
    1
    2 2
    N
    X
    i=1
    (
    xi µ
    )2
    • ର਺໬౓Λ֤ύϥϝʔλͰภඍ෼ͯ͠ۃ஋ΛٻΊΔ

    View Slide

  74. 1 ม਺ਖ਼ن෼෍ͷ৔߹
    @Lln(ˆ
    µ,
    2)

    = @

    "
    1
    2 2
    N
    X
    i=1
    (
    xi µ
    )2
    #
    µ=ˆ
    µ
    = 0
    1
    2 2
    N
    X
    i=1
    2(
    xi ˆ
    µ
    )( 1) = 0
    N
    X
    i=1
    xi
    N
    X
    i=1
    ˆ
    µ
    = 0
    ) ˆ
    µ
    =
    1
    N
    N
    X
    i=1
    xi

    View Slide

  75. 1 ม਺ਖ਼ن෼෍ͷ৔߹
    @Lln(
    µ,
    ˆ2)
    @
    2
    = @
    @
    2
    "
    N
    2
    ln 2
    1
    2 2
    N
    X
    i=1
    (
    xi µ
    )2
    #
    2=ˆ2
    = 0
    N
    2
    1
    ˆ2
    1
    2
    1
    (ˆ2)2
    ( 1)
    N
    X
    i=1
    (
    xi µ
    )2 = 0
    N
    ˆ2
    +
    1
    (ˆ2)2
    N
    X
    i=1
    (
    xi µ
    )2 = 0
    ) ˆ2 =
    1
    N
    N
    X
    i=1
    (
    xi µ
    )2

    View Slide

  76. 4.3 ͷ·ͱΊ
    • ໬౓ؔ਺ʢͷର਺ʣΛύϥϝʔλͰภඍ෼ͯ͠

    ໬౓͕࠷େͷͱ͜ΖΛٻΊΔ

    View Slide