Upgrade to Pro — share decks privately, control downloads, hide ads and more …

「はじめてのパターン認識」読書会 第 4 章

horiem
November 09, 2017
980

「はじめてのパターン認識」読書会 第 4 章

horiem

November 09, 2017
Tweet

Transcript

  1. ʮ͸͡Ίͯͷύλʔϯೝࣝʯಡॻձ
    ୈ 4 ষ
    2017 ೥ 11 ݄ 9 ೔

    horiem

    View full-size slide

  2. Πϯτϩ: 100 ԁۄͷਅآ൑ఆ
    ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …

    View full-size slide

  3. Πϯτϩ: 100 ԁۄͷਅآ൑ఆ
    ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …
    ૬ؔ͋ΔͷͰ͸ʁ

    View full-size slide

  4. Πϯτϩ: 100 ԁۄͷਅآ൑ఆ
    ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …
    ૬ؔ͋ΔͷͰ͸ʁ

    View full-size slide

  5. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …
    Πϯτϩ: 100 ԁۄͷਅآ൑ఆ
    100 ԁۄͷฏۉʢ4.8 gʣΑΓܰͦ͏͕ͩ

    ࠩ͸ 0.05 [g]

    View full-size slide

  6. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …
    Πϯτϩ: 100 ԁۄͷਅآ൑ఆ
    100 ԁۄͷฏۉʢ22600 μmʣͱಉ͡Α͏͕ͩ

    ࠩ͸ 2 [μm] >> 0.05

    View full-size slide

  7. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …
    Πϯτϩ: 100 ԁۄͷਅآ൑ఆ
    • ಛ௃ؒͷ૬ؔΛͳ͍ͨ͘͠
    • ୯Ґ͕ҧ͍ͬͯͯ΋౷ܭతʹൺֱ͍ͨ͠

    View full-size slide

  8. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …
    Πϯτϩ: 100 ԁۄͷਅآ൑ఆ
    • ಛ௃ؒͷ૬ؔΛͳ͍ͨ͘͠
    • ୯Ґ͕ҧ͍ͬͯͯ΋౷ܭతʹൺֱ͍ͨ͠
    ➡ ؍ଌσʔλΛม׵͠Α͏ʂ

    View full-size slide

  9. 4 ষͷ΋͘͡
    4. ֬཰Ϟσϧͱࣝผؔ਺
    1. ؍ଌσʔλͷઢܗม׵
    2. ֬཰Ϟσϧ
    3. ֬཰Ϟσϧύϥϝʔλͷ࠷໬ਪఆ

    View full-size slide

  10. 4 ষͷ΋͘͡
    4. ֬཰Ϟσϧͱࣝผؔ਺
    1. ؍ଌσʔλͷઢܗม׵
    2. ֬཰Ϟσϧ
    3. ֬཰Ϟσϧύϥϝʔλͷ࠷໬ਪఆ

    View full-size slide

  11. ฏۉϕΫτϧͱڞ෼ࢄߦྻ

    View full-size slide

  12. ฏۉϕΫτϧ
    • ֤ಛ௃ྔʢશ෦Ͱ d ݸʣͷฏۉΛฒ΂ͨ΋ͷ
    • ྫ͑͹ɿ
    µ = (
    µ1, µ2, . . . , µd)T = (
    E
    {
    x1
    }
    , E
    {
    x2
    }
    , . . . , E
    {
    xd
    })T
    µ = (µweight, µdiameter, µradius)T
    = (4.80[g], 2260[µm], 1130[µm])T

    View full-size slide

  13. ظ଴஋
    • ֬཰ม਺͕࿈ଓͷͱ͖ʢ֬཰ີ౓ؔ਺ʣ
    µi =
    E
    {
    xi
    } =
    Z
    dxi xip
    (
    xi)
    µi =
    E
    {
    xi
    } =
    X
    k
    x
    (k)
    i P

    x
    (k)
    i

    • ֬཰ม਺͕཭ࢄͷͱ͖ʢ֬཰࣭ྔؔ਺ʣ
    µ
    = ¯
    x
    =
    1
    N
    N
    X
    i=1
    xi
    • ؍ଌσʔλ͕ N ݸ༩͑ΒΕ͍ͯΔͱ͖

    View full-size slide

  14. पล֬཰
    • ֬཰ม਺͕཭ࢄͷͱ͖ʢ֬཰࣭ྔؔ਺ʣ
    • ஫໨͍ͯ͠Δಛ௃ྔͰͳ͍΋ͷ͸ͥΜͿ࿨ʢੵ෼ʣ

    ΛͱΔ
    ϋϯόʔά͕޷͖͔ʁ
    yes no sum
    ΤϏϑϥΠ
    ͕޷͖͔ʁ
    yes 60 40 100
    no 30 20 50
    sum 90 60
    p
    (
    xi) =
    Z
    dx1
    Z
    dx2
    · · ·
    Z
    dxi 1
    Z
    dxi+1
    · · ·
    Z
    dxd p
    (
    x1, x2, . . . , xd)

    View full-size slide

  15. ڞ෼ࢄߦྻ
    ⌃ = Var {
    x
    }
    = E (
    x µ
    )(
    x µ
    )T
    =
    0
    B
    @
    E {(
    x1 µ1)(
    x1 µ1)}
    . . .
    E {(
    x1 µ1)(
    xd µd)}
    .
    .
    .
    ...
    .
    .
    .
    E {(
    xd µd)(
    x1 µ1)}
    . . .
    E {(
    xd µd)(
    xd µd)}
    1
    C
    A
    = ( ij)

    View full-size slide

  16. ڞ෼ࢄߦྻ
    • ؍ଌσʔλ͕ N ݸ༩͑ΒΕ͍ͯΔͱ͖
    ij =
    1
    N
    N
    X
    n=1
    (
    xni µi)(
    xnj µj)

    View full-size slide

  17. ڞ෼ࢄߦྻ
    • ຊདྷ͸ෆภ෼ࢄΛ࢖͏΂͖
    • ظ଴஋Λͱͬͨͱ͖ʹਅͷ෼ࢄʹ

    ऩଋ͢ΔΑ͏ௐઅ͢Δ
    • ඪຊ਺͕ଟ͍৔߹͸େࠩͳ͍ͷͰ

    ͜͜Ͱ͸γϯϓϧʹ͍ͯ͠Δ
    sij =
    1
    N
    1
    N
    X
    n=1
    (
    xni µi)(
    xnj µj)

    View full-size slide

  18. ෼ࢄͱඪ४ภࠩ
    • ෼ࢄ͸ฏۉ͔Βͷೋ৐ޡࠩͷظ଴஋
    • େ͖͚Ε͹͹Β͍͍ͭͯΔ
    • ෼ࢄͷฏํ͕ࠜඪ४ภࠩ
    • ෼ࢄͩͱಛ௃ྔͷ୯Ґ͕มΘͬͯ͠·͏ͨΊ

    ΋ͱͷ୯Ґʹ໭͢
    2
    i
    = ii = E (
    xi µi)2
    i =
    q
    2
    i

    View full-size slide

  19. ڞ෼ࢄͱ૬ؔ܎਺
    • ڞ෼ࢄ͸ҟͳΔಛ௃ྔؒͰ͹Β͖ͭํ͕

    ಉ͔͡Ͳ͏͔Λಛ௃͚ͮΔ
    • ਖ਼ͷ૬͕ؔ͋Ε͹ +ɺෛͷ૬͕ؔ͋Ε͹ -
    • ڞ෼ࢄ͸୯ҐΛ͍࣋ͬͯΔͨΊ

    ୯Ґ͕ҧ͏ڞ෼ࢄͲ͏͠ΛൺֱͰ͖ͳ͍
    ij = E {(
    xi µi)(
    xj µj)}

    View full-size slide

  20. ڞ෼ࢄͱ૬ؔ܎਺
    ⇢ij = ij
    i j
    • ڞ෼ࢄΛແ࣍ݩԽͨ͠ͷ͕૬ؔ܎਺
    • ਖ਼ͷ૬͕ؔ͋Ε͹ +ɺෛͷ૬͕ؔ͋Ε͹ -
    • ඞͣ [-1, 1] ΛͱΔ

    View full-size slide

  21. ૬ؔ܎਺͸ઢܗ૬͔ؔ͠ΩϟονͰ͖ͳ͍
    • x = [-2, -1, 0, 1, 2], y = x^2 ͷͱ͖

    ૬ؔ܎਺ ρ_xy ͸θϩ
    https://upload.wikimedia.org/wikipedia/commons/d/d4/Correlation_examples2.svg

    View full-size slide

  22. ϕΫτϧతͳղऍ
    • N ݸͷଌఆ͕͋Δͱ͖ɺ
    ij =
    1
    N
    N
    X
    n=1
    (
    xni µi)(
    xnj µj)
    =
    1
    N
    N
    X
    n=1
    dnidnj
    =
    1
    N
    di
    · dj
    di = (
    x1i µi, x2i µi, . . . , xNi µi)T
    = (
    d1i, d2i, . . . , dNi)T
    ͱ͓͘ͱɺڞ෼ࢄ͸಺ੵʢͷఆ਺ഒʣʹͳΔ
    ϕΫτϧۭؒͱͯ͠

    ѻ͏ͨΊʹ͸

    ֤ಛ௃ྔͰ୯Ґ͕

    ἧ͍ͬͯΔඞཁ͕͋Δ

    View full-size slide

  23. 2
    i
    =
    1
    N
    di
    · di
    =
    1
    N
    |di
    |2
    i = =
    1
    p
    N
    |di
    |
    ϕΫτϧతͳղऍ
    ⇢ij =
    ij
    i j
    =
    (1
    /N
    )
    di
    · dj
    (1
    /
    p
    N
    )
    |di
    |
    (1
    /
    p
    N
    )
    |dj
    |
    =
    di
    · dj
    |di
    | |dj
    |
    = cos
    ✓ij

    View full-size slide

  24. ϕΫτϧతͳղऍ
    • ׬શͳਖ਼ͷ૬͕ؔ͋Δͱ͖
    ⇢ij = 1
    cos
    ✓ij = 1
    ✓ij = 0
    di =
    cdj (
    c >
    0)

    View full-size slide

  25. ϕΫτϧతͳղऍ
    di
    dj
    ⇢ij = 1 ⇢ij = 0 ⇢ij = 1
    di
    dj
    di
    dj
    {
    }
    p
    N i
    p
    N j

    View full-size slide

  26. ؍ଌσʔλͷඪ४Խ

    View full-size slide

  27. • ಛ௃ؒͷ૬ؔΛͳ͍ͨ͘͠
    • ୯Ґ͕ҧ͍ͬͯͯ΋౷ܭతʹൺֱ͍ͨ͠
    ➡ ؍ଌσʔλΛม׵͠Α͏ʂ
    ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ
    4.801 22601 11301 … ਅ
    4.751 22599 11300 … آ
    4.799 22602 11301 … ਅ
    … … … … …
    Πϯτϩʢ࠶๚ʣ

    View full-size slide

  28. ฏۉɾ෼ࢄͱઢܗม׵
    • ઢܗม׵Λߟ͑Δ
    y
    =
    ax
    +
    b
    E {
    y
    } = E {
    ax
    +
    b
    } =
    a
    E {
    x
    } +
    b
    =

    +
    b
    • ฏۉͱ෼ࢄ͸ҎԼͷΑ͏ʹԠ౴
    Var {
    y
    } = E (
    y Ey
    )2
    = E [
    ax
    +
    b
    (

    +
    b
    )]2 = E
    a
    2(
    x µ
    )2
    =
    a
    2E (
    x µ
    )2 =
    a
    2Var {
    x
    }
    =
    a
    2 2

    View full-size slide

  29. ඪ४Խ
    • ҎԼͷઢܗม׵Λ࢖͏ͱ

    ฏۉ 0 ɺ෼ࢄ 1 ͷಛ௃ྔ͕ಘΒΕΔ
    z
    = x µ
    E {
    z
    } = E

    x µ =
    1
    (E {
    x
    }
    µ
    )
    = 0
    Var {
    z
    } = Var

    x µ =
    1
    2
    Var {
    x
    }
    = 1

    View full-size slide

  30. ඪ४Խ
    x1
    x2

    View full-size slide

  31. ඪ४Խ
    x1
    x2

    View full-size slide

  32. ඪ४Խ
    x1
    x2

    View full-size slide

  33. ඪ४Խ
    x1
    x2
    µ1
    µ2

    View full-size slide

  34. ඪ४Խ
    x1
    x2
    µ1
    µ2
    1
    2

    View full-size slide

  35. ඪ४Խ
    x1
    x2
    µ1
    µ2
    1
    2
    1
    z1
    z2
    1

    View full-size slide

  36. ඪ४Խ
    x1
    x2
    µ1
    µ2
    1
    2
    1
    z1
    z2
    1
    • σʔλͷฏۉΛ 0 ɺ෼ࢄʢඪ४ภࠩʣΛ 1 ʹ
    • ແ࣍ݩԽ͞Ε͍ͯΔͷͰɺ୯Ґͷҧ͍΋ٵऩͰ͖Δ

    View full-size slide

  37. ؍ଌσʔλͷແ૬ؔԽ

    View full-size slide

  38. ݻ༗ϕΫτϧ
    • ڞ෼ࢄߦྻͷݻ༗஋໰୊Λղ͘ͱ

    d ຊͷϕΫτϧ͕ಘΒΕΔ
    • ڞ෼ࢄߦྻ͸ [d, d] ͷରশߦྻ
    ⌃si = isi

    View full-size slide

  39. ݻ༗ϕΫτϧ
    • ରশߦྻʹରͯ͠ɿ
    • ݻ༗஋͸࣮਺
    • ݻ༗ϕΫτϧ͸௚ަ
    ➡ ݻ༗ϕΫτϧ͸ਖ਼ن௚ަجఈ
    sT
    i
    sj = ij =

    1 (i = j)
    0 (i 6= j)

    View full-size slide

  40. ճసߦྻ
    • ݻ༗ϕΫτϧΛฒ΂ͯߦྻΛ࡞Δ
    • ਖ਼ن௚ަجఈΛฒ΂ͨߦྻ͸௚ަߦྻͱͳΔ
    S = (s1, s2, . . . , sd)
    (ST S)ij = sT
    i
    sj = ij
    ) ST S = I
    ) ST = S 1
    • ͜ͷ৔߹͸ճసߦྻʢ㱬௚ަߦྻʣͱͳΔ

    View full-size slide

  41. ແ૬ؔԽ
    y
    = ST
    x
    E {
    y
    } = E ST
    x
    = ST E {
    x
    }
    = ST
    µ
    Var {
    y
    } = E (
    y
    E {
    y
    })(
    y
    E {
    y
    })T
    = E (ST
    x
    ST
    µ
    )(ST
    x
    ST
    µ
    )T
    = E ST (
    x µ
    )[ST (
    x µ
    )]T
    = E ST (
    x µ
    )(
    x µ
    )T S
    = ST E (
    x µ
    )(
    x µ
    )T S
    = ST ⌃S

    View full-size slide

  42. ແ૬ؔԽ
    S 1⌃S = S 1⌃(s1, s2, . . . , sd) = S 1( 1s1, 2s2, . . . , dsd)
    = S 1S
    0
    B
    B
    B
    @
    1 0 . . . 0
    0 2 . . . 0
    .
    .
    .
    ...
    .
    .
    .
    0 0 . . . d
    1
    C
    C
    C
    A
    =
    0
    B
    B
    B
    @
    1 0 . . . 0
    0 2 . . . 0
    .
    .
    .
    ...
    .
    .
    .
    0 0 . . . d
    1
    C
    C
    C
    A
    = ⇤
    • ͳͷͰɺແ૬ؔԽ͞Ε͍ͯΔ
    • ແ૬͕ؔͩɺඪ४Խ͸͞Ε͍ͯͳ͍
    (Var {y})ij = 0 (i 6= j)
    (Var {y})ii = i

    View full-size slide

  43. ന৭Խ
    • ඪ४Խʴແ૬ؔԽ
    u
    = ⇤ 1/2ST (
    x µ
    )
    (⇤ 1/2)ij =

    1/
    p
    i (i = j)
    0 (i 6= j)

    View full-size slide

  44. ന৭Խ
    E {
    u
    } = E n⇤ 1/2ST (
    x µ
    )o = ⇤ 1/2ST (E {
    x
    }
    µ
    )
    = 0
    Var {
    u
    } = E n⇤ 1/2ST (
    x µ
    )(
    x µ
    )T S(⇤ 1/2)T o
    = ⇤ 1/2ST E (
    x µ
    )(
    x µ
    )T S(⇤ 1/2)T
    = ⇤ 1/2ST ⌃S(⇤ 1/2)T
    = ⇤ 1/2⇤(⇤ 1/2)T
    = I
    • ඪ४Խ͞Εɺ͔ͭແ૬ؔԽ͞Εͨʂ

    View full-size slide

  45. 4.1 ͷ·ͱΊ
    • ඪ४Խ
    • ୯ҐΛͦΖ͑ɺฏۉΛ 0 ɺ෼ࢄΛ 1 ʹ͢Δ
    • ແ૬ؔԽ
    • ૬͕ؔͳ͘ͳΔΑ͏ʹۭؒΛճసͤ͞Δ
    • ඪ४Խ͸͞Εͯͳ͍
    • ന৭Խ
    • ඪ४Խ ʴ ແ૬ؔԽ
    • ୯ҐΛͦΖ͑ɺฏۉΛ 0 ɺ෼ࢄΛ 1 ʹ͠ɺ

    ૬͕ؔͳ͘ͳΔΑ͏ʹۭؒΛճసͤ͞Δ

    View full-size slide

  46. 4 ষͷ΋͘͡
    4. ֬཰Ϟσϧͱࣝผؔ਺
    1. ؍ଌσʔλͷઢܗม׵
    2. ֬཰Ϟσϧ
    3. ֬཰Ϟσϧύϥϝʔλͷ࠷໬ਪఆ

    View full-size slide

  47. ֬཰Ϟσϧ
    • σʔλͷ෼෍ͷ਺ཧϞσϧ
    • ύϥϝτϦοΫϞσϧ
    • ෼෍ؔ਺ΛԾఆ͠ɺύϥϝʔλΛܾఆͯ͠

    ϞσϧԽ͢Δ
    • ϊϯύϥϝτϦοΫϞσϧ
    • ಛఆͷ෼෍ΛԾఆͤͣɺσʔλͦͷ΋ͷ͔Β

    ෼෍ͷදݱΛಘΔ

    View full-size slide

  48. ֬཰Ϟσϧ͋Ε͜Ε
    • ύϥϝτϦοΫϞσϧ
    • ֬཰ม਺͕཭ࢄʢ֬཰࣭ྔؔ਺ʣ
    • ೋ߲෼෍ɺଟ߲෼෍ɺϙΞιϯ෼෍ͳͲ
    • ֬཰ม਺͕࿈ଓʢ֬཰ີ౓ؔ਺ʣ
    • ਖ਼ن෼෍ɺΧΠೋ৐෼෍ɺίʔγʔ෼෍ͳͲ
    • ϊϯύϥϝτϦοΫϞσϧ
    • ώετάϥϜ๏ɺkNN ๏ɺύϧπΣϯີ౓ਪఆ๏
    ͳͲ

    View full-size slide

  49. ਖ਼ن෼෍ؔ਺

    View full-size slide

  50. ਖ਼ن෼෍ͷੑ࣭ʢൈਮʣ
    • ղੳతʹΑ͘ௐ΂ΒΕ͍ͯΔ
    • ඇਖ਼ن෼෍ʹ͕ͨ͠͏σʔλ΋

    ඪຊฏۉͷ෼෍͸ਖ਼ن෼෍ʹͳΔʢத৺ۃݶఆཧʣ
    • ਖ਼ن෼෍ʹ͕ͨ͠͏σʔλͷઢܗม׵͸

    ਖ਼ن෼෍ʹ͕ͨ͠͏
    • ਖ਼ن෼෍ʹ͕ͨ͠͏ෳ਺ͷ֬཰ม਺ͷઢܗ݁߹͸

    ਖ਼ن෼෍ͱ͍͏ʢ࠶ੜੑʣ
    • ແ૬ؔͰ͋Δ͜ͱͱ౷ܭతʹಠཱͰ͋Δ͜ͱ͕౳Ձ

    ʢʮਖ਼ن෼෍ʹݶΓʯͷ෦෼ˠ ʮ਺ֶηϛφʔʯʹࡌͬͯΔ͔΋ʁʣ

    View full-size slide

  51. ਖ਼ن෼෍
    • 1 ࣍ݩਖ਼ن෼෍
    N(x
    |
    µ,
    2
    ) =
    1
    p
    2⇡
    2 exp

    (x µ)
    2
    2
    2
    • ଟ࣍ݩਖ਼ن෼෍
    N
    (x
    |
    µ
    ,
    ⌃) =
    1
    (2

    )
    d/2 |

    |1/2 exp

    1
    2
    (x µ)
    T

    1
    (x µ)

    View full-size slide

  52. ਖ਼ن෼෍
    • ૬ؔͷ෼͚ͩճస͠ɺඪ४ภࠩͷ෼͚ͩҾ͖৳͹͞Ε͍ͯΔ
    • ന৭Խͷٯ
    (
    x µ
    )T ⌃ 1(
    x µ
    ) = (
    x µ
    )T [S⇤S 1] 1(
    x µ
    )
    = (
    x µ
    )T S⇤ 1S(
    x µ
    )
    = [ST (
    x µ
    )]T ⇤ 1[ST (
    x µ
    )]
    =
    y
    T ⇤ 1
    y
    (*
    y
    ⌘ ST (
    x µ
    ))
    =
    y
    T (⇤1/2)T ⇤1/2
    y
    = (⇤1/2
    y
    )T (⇤1/2
    y
    )
    =
    z
    T
    z
    (*
    z
    ⌘ ⇤ 1/2
    y
    )

    View full-size slide

  53. ϚϋϥϊϏεڑ཭
    • ෼෍ͷத৺͔ΒͲΕ͚ͩ཭Ε͍ͯΔ͔ͷࢦඪ
    • ന৭Խۭͨؒ͠Ͱͷڑ཭
    d(
    x
    ,
    µ
    ) =
    q
    (
    x µ
    )T ⌃ 1(
    x µ
    )
    =
    p
    z
    T
    z

    View full-size slide

  54. ਖ਼ن෼෍͔Βಋ͔ΕΔ

    ࣝผؔ਺

    View full-size slide

  55. Ϋϥε৚݅෇͖֬཰
    • Ϋϥε৚݅෇͖֬཰͕ਖ਼ن෼෍Ͱ͋ΔͱԾఆ͢Δ
    ln
    P
    (
    Ci
    |
    x) =
    p
    (x
    |Ci)
    P
    (
    Ci)
    p
    (x)
    / p
    (x
    |Ci)
    P
    (
    Ci)
    =
    P
    (
    Ci)
    (2

    )
    d/2 |
    ⌃i
    |1/2 exp

    1
    2
    (x µi)
    T

    1
    i (x µi)
    p
    (x
    |Ci) =
    1
    (2

    )
    d/2 |
    ⌃i
    |1/2 exp

    1
    2
    (x µi)
    T

    1
    i (x µi)

    View full-size slide

  56. Ϋϥε৚݅෇͖֬཰
    • ؔ܎ͳ͍߲ΛΦϛοτɺ ×(-2)
    ln P(Ci
    |
    x
    ) = ln P(Ci)
    d
    2
    ln(2⇡)
    1
    2
    ln |⌃i
    |1/2
    1
    2
    (
    x µi)T ⌃ 1
    i
    (
    x µi)
    gi(
    x
    ) = (
    x µi)T ⌃ 1
    i
    (
    x µi) + ln |⌃i
    | 2 ln P(Ci)
    [Recognized class] = arg min
    i [
    gi(x)]

    View full-size slide

  57. ࣝผڥք
    • Ϋϥεؒͷڥ໨ʢࣝผڥքʣ͸ҎԼͷํఔࣜͰ

    ༩͑ΒΕΔ
    fij(
    x
    ) = gi(
    x
    ) gj(
    x
    ) = 0

    View full-size slide

  58. ࣝผڥք
    fij(
    x
    ) = gi(
    x
    ) gj(
    x
    )
    = (
    x µi)T ⌃ 1
    i
    (
    x µi) + ln |⌃i
    | 2 ln P(Ci)
    (
    x µj)T ⌃ 1
    j
    (
    x µj) ln |⌃j
    | + 2 ln P(Cj)
    =
    x
    ⌃ 1
    i x x
    ⌃ 1
    i µi µi⌃ 1
    i x
    +
    µi⌃ 1
    i µi
    x
    ⌃ 1
    j x x
    ⌃ 1
    j µj µj⌃ 1
    j x
    +
    µj⌃ 1
    j µj
    + ln
    |⌃i
    |
    ⌃j
    2 ln
    P(Ci)
    P(Cj)
    =
    x
    (⌃ 1
    i
    ⌃ 1
    j
    )
    x
    + 2(
    µ
    T
    j
    ⌃ 1
    j µ
    T
    i
    ⌃ 1
    i
    )
    x
    + µT
    i
    ⌃ 1
    i
    µiµT
    j
    ⌃ 1
    j
    µj + ln
    |⌃i
    |
    ⌃j
    2 ln
    P(Ci)
    P(Cj)
    )
    x
    T S
    x
    + 2
    c
    T
    x
    + F = 0 ʢ2 ࣍ࣝผؔ਺ʣ
    +µT
    i
    ⌃ 1
    i
    µi µT
    j
    ⌃ 1
    j
    µj + ln
    |⌃i
    |
    |⌃j
    |
    2 ln
    P(Ci)
    P(Cj)
    fij(
    x
    ) = gi(
    x
    ) gj(
    x
    )
    = (
    x µi)T ⌃ 1
    i
    (
    x µi) + ln |⌃i
    | 2 ln P(Ci)
    (
    x µj)T ⌃ 1
    j
    (
    x µj) ln |⌃j
    | + 2 ln P(Cj)
    =
    x
    ⌃ 1
    i x x
    ⌃ 1
    i µi µi⌃ 1
    i x
    +
    µi⌃ 1
    i µi
    x
    ⌃ 1
    j x x
    ⌃ 1
    j µj µj⌃ 1
    j x
    +
    µj⌃ 1
    j µj
    + ln
    |⌃i
    |
    ⌃j
    2 ln
    P(Ci)
    P(Cj)
    =
    x
    (⌃ 1
    i
    ⌃ 1
    j
    )
    x
    + 2(
    µ
    T
    j
    ⌃ 1
    j µ
    T
    i
    ⌃ 1
    i
    )
    x
    + µT
    i
    ⌃ 1
    i
    µiµT
    j
    ⌃ 1
    j
    µj + ln
    |⌃i
    |
    ⌃j
    2 ln
    P(Ci)
    P(Cj)
    x
    ⌃ 1
    j x
    +
    x
    ⌃ 1
    j µj +
    µj⌃ 1
    j x µj⌃ 1
    j µj

    View full-size slide

  59. ࣝผڥքʢิ୊ʣ
    x
    T
    i ⌃
    1
    i µi = (Scalar) = (x
    T
    i ⌃
    1
    i µi)
    T
    = µ
    T
    i (⌃
    1
    i )
    T
    x
    = µ
    T
    i ⌃
    1
    i x (
    *

    1
    i is a symmetric matrix)
    • ରশߦྻͷٯߦྻ͸ରশߦྻͰ͋Δ͜ͱʹ஫ҙ

    View full-size slide

  60. ࣝผڥք
    • ;ͨͭͷΫϥεͷڞ෼ࢄߦྻ͕౳͍͠ͱ͖
    fij(
    x
    ) = 2
    c
    T
    x
    + F = 0 ʢઢܗࣝผؔ਺ʣ

    View full-size slide

  61. ࣝผڥք
    ⌃i = ⌃j = I P(Ci) = P(Cj)
    • ͔ͭ ͷͱ͖
    fij(
    x
    ) = 2(
    µ
    T
    j
    ⌃ 1
    j µ
    T
    j
    ⌃ 1
    i
    )
    x
    +
    µ
    T
    i
    ⌃ 1
    i µi µ
    T
    i
    ⌃ 1
    i µi
    = 2 (
    µ
    T
    j µ
    T
    i
    )
    x
    +
    µ
    T
    i µi µ
    T
    j µj = 0
    x
    T
    x
    + 2
    µ
    T
    i x
    +
    µ
    T
    i µi x
    T
    x
    + 2
    µ
    T
    j x µ
    T
    j µj = 0
    (
    x µi)T (
    x µi) (
    x µj)T (
    x µj) = 0
    ) (
    x µi)T (
    x µi) = (
    x µj)T (
    x µj)
    x
    T
    x
    2
    µ
    T
    i x
    +
    µ
    T
    i µi x
    T
    x
    + 2
    µ
    T
    j x µ
    T
    j µj = 0
    • ྆ล σ ͰׂΓɺ x^T x Λ଍͠Ҿ͖
    ʢ࠷ۙ๣๏……ͱຊʹ͸ॻ͍ͯ͋Δ͕ઢܗ൑ผ෼ੳʢLDAʣͰ͸ʁ
    ʢฏۉ͔Βͷڑ཭Λൺ΂͍ͯΔͷͰʣʣ
    +µT
    i
    ⌃ 1
    i
    µi µT
    j
    ⌃ 1
    j
    µj
    fij(
    x
    ) = 2(
    µ
    T
    j
    ⌃ 1
    j µ
    T
    j
    ⌃ 1
    i
    )
    x
    +
    µ
    T
    i
    ⌃ 1
    i µi µ
    T
    i
    ⌃ 1
    i µi
    = 2 (
    µ
    T
    j µ
    T
    i
    )
    x
    +
    µ
    T
    i µi µ
    T
    j µj = 0

    View full-size slide

  62. ࣝผڥք
    µi
    µj

    View full-size slide

  63. ࣝผڥք
    ܾఆڥք
    µi
    µj

    View full-size slide

  64. ࣝผڥք
    ܾఆڥք
    µi
    µj
    µk

    View full-size slide

  65. ࣝผڥք
    ܾఆڥք
    µi
    µj
    µk

    View full-size slide

  66. ࣝผڥք
    ܾఆڥք
    cf. ϘϩϊΠਤ
    µi
    µj
    µk

    View full-size slide

  67. 4.2 ͷ·ͱΊ
    • ਖ਼ن෼෍͸͍Ζ͍Ζੑ࣭͕͍͍
    • ଟ࣍ݩਖ਼ن෼෍͸

    ૬ؔɾ෼ࢄʹΑΔճసɾऩॖ͕ߟྀ͞Ε͍ͯΔࣜ
    • 2 Ϋϥεؒͷࣄޙ֬཰͕౳͘͠ͳΔ఺ͷي੻͸

    ࣝผڥքΛ༩͑Δ

    View full-size slide

  68. 4 ষͷ΋͘͡
    4. ֬཰Ϟσϧͱࣝผؔ਺
    1. ؍ଌσʔλͷઢܗม׵
    2. ֬཰Ϟσϧ
    3. ֬཰Ϟσϧύϥϝʔλͷ࠷໬ਪఆ

    View full-size slide

  69. ಠཱಉ෼෍ͱಉ࣌෼෍
    • ಉҰͷʢਅͷʣ෼෍͔Βಠཱʹαϯϓϧ͞Εͨ΋ͷΛ

    i.i.d. ʢಠཱಉ෼෍: independently and identically distributedʣඪຊ

    ͱ͍͏
    • i.i.d. ͷͱ͖ɺ N ݸͷαϯϓϧͷಉ࣌෼෍͸ҎԼɿ
    f(
    x1,
    x2, . . . ,
    xN
    |

    ) =
    N
    Y
    i=1
    f(
    xi
    |

    )

    View full-size slide

  70. ࠷໬ਪఆ๏
    • ࣮༻্͸σʔλ͕༩͑ΒΕ͍ͯͯύϥϝʔλ͕ະ஌
    • ύϥϝʔλΛม਺ͱͯ͠ಉ࣌෼෍ΛͱΒ͑Δ
    L(

    ) = f(
    x1,
    x2, . . . ,
    xN
    |

    ) ʢ໬౓ؔ਺ʣ
    • ໬౓ؔ਺Λ࠷େʹ͢ΔύϥϝʔλΛٻΊΔ

    ʢ࠷໬ਪఆ๏ʣ
    • ର਺Λͱͬͯ΋ۃ஋ͷҐஔ͸มΘΒͳ͍ͷͰ

    ໬౓ؔ਺ͷର਺ΛͱͬͯܭࢉΛ؆୯ʹͰ͖Δ͜ͱ͕͋Δ

    View full-size slide

  71. 1 ม਺ਖ਼ن෼෍ͷ৔߹
    L(µ,
    2
    ) = f(x1, x2, . . . , xN
    |
    µ,
    2
    )
    =
    N
    Y
    i=1
    1
    p
    2⇡
    2 exp

    (xi µ)
    2
    2
    2
    = (2⇡
    2
    )
    N/2
    exp
    "
    1
    2
    2
    N
    X
    i=1
    (xi µ)
    2
    #
    Lln(
    µ,
    2) = N
    2
    ln(2

    ) N
    2
    ln 2
    1
    2 2
    N
    X
    i=1
    (
    xi µ
    )2

    View full-size slide

  72. 1 ม਺ਖ਼ن෼෍ͷ৔߹
    Lln(
    µ,
    2) = N
    2
    ln(2

    ) N
    2
    ln 2
    1
    2 2
    N
    X
    i=1
    (
    xi µ
    )2
    • ର਺໬౓Λ֤ύϥϝʔλͰภඍ෼ͯ͠ۃ஋ΛٻΊΔ

    View full-size slide

  73. 1 ม਺ਖ਼ن෼෍ͷ৔߹
    @Lln(ˆ
    µ,
    2)

    = @

    "
    1
    2 2
    N
    X
    i=1
    (
    xi µ
    )2
    #
    µ=ˆ
    µ
    = 0
    1
    2 2
    N
    X
    i=1
    2(
    xi ˆ
    µ
    )( 1) = 0
    N
    X
    i=1
    xi
    N
    X
    i=1
    ˆ
    µ
    = 0
    ) ˆ
    µ
    =
    1
    N
    N
    X
    i=1
    xi

    View full-size slide

  74. 1 ม਺ਖ਼ن෼෍ͷ৔߹
    @Lln(
    µ,
    ˆ2)
    @
    2
    = @
    @
    2
    "
    N
    2
    ln 2
    1
    2 2
    N
    X
    i=1
    (
    xi µ
    )2
    #
    2=ˆ2
    = 0
    N
    2
    1
    ˆ2
    1
    2
    1
    (ˆ2)2
    ( 1)
    N
    X
    i=1
    (
    xi µ
    )2 = 0
    N
    ˆ2
    +
    1
    (ˆ2)2
    N
    X
    i=1
    (
    xi µ
    )2 = 0
    ) ˆ2 =
    1
    N
    N
    X
    i=1
    (
    xi µ
    )2

    View full-size slide

  75. 4.3 ͷ·ͱΊ
    • ໬౓ؔ਺ʢͷର਺ʣΛύϥϝʔλͰภඍ෼ͯ͠

    ໬౓͕࠷େͷͱ͜ΖΛٻΊΔ

    View full-size slide