Share
Embed
Support Vector Machine (SVM)
はじめての『はじめてのパターン認識』第8章
This is a general introduction of support vector machine (SVM) for machine learning, which includes hard-margin SVM, soft-margin SVM and Kernel trick method.
Shoh-kudo
June 28, 2019
Transcript
115 slides
-
065-*/& ఆٛ ઢܗࣝผฏ໘ Ϛʔδϯ࠷େԽ݅ ϥάϥϯδϡͷະఆ๏ ର ྫ ຕ ຕ ຕ
ຕ ຕ ຕ ઢܗՄೳͳ߹ͷ ϋʔυϚʔδϯ47. ·ͱΊ ຕ -
ͱઢͷڑͷެࣜ ઢ w1 x1 + w2 x2 + b =
0ͱɼ (x1i , x2i ) ͷڑ d ࣍ͷࣜͰॻ͖දͤΔɽ wT = (w1 w2 ) xi T = (x1i x2i ) ઢܗࣝผฏ໘ D D Ϛʔδϯ ઢܗࣝผฏ໘ʹ ࠷͍ۙσʔλͱͷڑ% ඞͣ Ҏ্ʹͳΔ d D d = |w1 x1i + w2 x2i + b| w2 1 + w2 2 = |wT xi + b| ∥w∥ d = |wT xi + b| ∥w∥ ≥ D ϋʔυϚʔδϯ47. -
ͱઢͷڑͷެࣜ ઢ w1 x1 + w2 x2 + b =
0ͱɼ (x1i , x2i ) ͷڑ d ࣍ͷࣜͰॻ͖දͤΔɽ d = |w1 x1i + w2 x2i + b| w2 1 + w2 2 = |wT xi + b| ∥w∥ wT = (w1 w2 ) xi T = (x1i x2i ) d = |wT xi + b| ∥w∥ ≥ D ඞͣ Ҏ্ʹͳΔ d D ∥w∥ = w2 1 + w2 2 ≥ 0 d∥w∥ = |wT xi + b| ≥ κ wT = (w1 w2 ) ͷͦΕͧΕҰఆͳͷͰ ઢܗࣝผฏ໘ݻఆ͞ΕΔͱ͢Δͱ ҰఆͷͣͰ͋ΔɽͦͷͨΊɼڑͷࣜʹ͜ΕΛ͔͚ɼ ࠷ऴతͳϚʔδϯͱઢܗࣝผฏ໘͕Ұͭʹఆ·Δͱ͢Δͱ D∥w∥ = κ ͱஔ͍ͯ ڑͷ݅ ڑͷ݅ ઢܗࣝผڥքʹ࠷͍ۙɼͰ͋ΔϕΫτϧ͕αϙʔτϕΫτϧ |wT xi + b| = κ ϋʔυϚʔδϯ47. -
|wT xi + b| ≥ κ |wT xi + b|
κ ≥ κ κ ⟹ |wT xi + b| ≥ 1 (wT = wT κ = ( w1 κ w2 κ ), b = b κ ) શମΛ κ ͰׂΓɼॏΈΛ࠶ఆٛ͢Δͱɽɽɽ ⟺ |wT xi + b| = { (wT xi + b) ≥ 1 (wT xi + b ≥ 0) −(wT xi + b) ≥ 1 (wT xi + b < 0) = { wT xi + b ≥ 1 (wT xi + b ≥ 0) wT xi + b ≤ − 1 (wT xi + b < 0) ڑͷ݅ ઢܗࣝผڥքʹ࠷͍ۙɼͰ͋ΔϕΫτϧ͕αϙʔτϕΫτϧ |wT xi + b| = 1 ͱઢͷڑͷެࣜ ઢ w1 x1 + w2 x2 + b = 0ͱɼ (x1i , x2i ) ͷڑ d ࣍ͷࣜͰॻ͖දͤΔɽ d = |w1 x1i + w2 x2i + b| w2 1 + w2 2 = |wT xi + b| ∥w∥ wT = (w1 w2 ) xi T = (x1i x2i ) ϋʔυϚʔδϯ47. -
|wT xi + b| = { wT xi + b
≥ 1 (wT xi + b ≥ 0) wT xi + b ≤ − 1 (wT xi + b < 0) ࣝผਖ਼ղͷ݅ ͱઢͷڑͷެࣜ ઢ w1 x1 + w2 x2 + b = 0ͱɼ (x1i , x2i ) ͷڑ d ࣍ͷࣜͰॻ͖දͤΔɽ d = |w1 x1i + w2 x2i + b| w2 1 + w2 2 = |wT xi + b| ∥w∥ wT = (w1 w2 ) xi T = (x1i x2i ) D D C1 C2 ti = 1 Ϋϥεͷਖ਼ղϥϕϧ Ϋϥεʹ͓͍ͯ wT xi + b ≥ 1 ti (wT xi + b) ≥ 1 ti (wT xi + b) ≥ 1 ti = − 1 Ϋϥεͷਖ਼ղϥϕϧ Ϋϥεʹ͓͍ͯ wT xi + b ≤ 1 ϋʔυϚʔδϯ47. -
ͱઢͷڑͷެࣜ ઢ w1 x1 + w2 x2 + b =
0ͱɼ (x1i , x2i ) ͷڑ d ࣍ͷࣜͰॻ͖දͤΔɽ d = |w1 x1i + w2 x2i + b| w2 1 + w2 2 = |wT xi + b| ∥w∥ wT = (w1 w2 ) xi T = (x1i x2i ) ti (wT xi + b) ≥ 1 D D C1 C2 ti = 1 Ϋϥεͷਖ਼ղϥϕϧ Ϋϥεʹ͓͍ͯ wT xi + b ≥ 1 ti (wT xi + b) ≥ 1 ti (wT xi + b) ≥ 1 ti = − 1 Ϋϥεͷਖ਼ղϥϕϧ Ϋϥεʹ͓͍ͯ wT xi + b ≤ 1 ϋʔυϚʔδϯ47. ࣝผਖ਼ղͷ݅ -
ͱઢͷڑͷެࣜ ઢ w1 x1 + w2 x2 + b =
0ͱɼ (x1i , x2i ) ͷڑ d ࣍ͷࣜͰॻ͖දͤΔɽ d = |w1 x1i + w2 x2i + b| w2 1 + w2 2 = |wT xi + b| ∥w∥ wT = (w1 w2 ) xi T = (x1i x2i ) ti (wT xi + b) − 1 ≥ 0 D D C1 C2 ti = 1 Ϋϥεͷਖ਼ղϥϕϧ Ϋϥεʹ͓͍ͯ wT xi + b ≥ 1 ti (wT xi + b) ≥ 1 ti (wT xi + b) ≥ 1 ti = − 1 Ϋϥεͷਖ਼ղϥϕϧ Ϋϥεʹ͓͍ͯ wT xi + b ≤ 1 ϋʔυϚʔδϯ47. ࣝผਖ਼ղͷ݅ -
ઢܗࣝผฏ໘ͱ֤σʔλͷڑ͔Βಋ͍ͨࣝผΫϥε͕ਖ਼͍݅͠ ͱઢͷڑͷެࣜ ઢ w1 x1 + w2 x2 + b
= 0ͱɼ (x1i , x2i ) ͷڑ d ࣍ͷࣜͰॻ͖දͤΔɽ d = |w1 x1i + w2 x2i + b| w2 1 + w2 2 = |wT xi + b| ∥w∥ wT = (w1 w2 ) xi T = (x1i x2i ) ϋʔυϚʔδϯ47. ti (wT xi + b) − 1 ≥ 0 -
ti (wT xi + b) − 1 ≥ 0 47.ͷͨΊʹೋͭͷ݅Λநग़͢Δ
ઢܗࣝผฏ໘ͱ֤σʔλͷڑͷ݅ Ϛʔδϯ࠷େԽ݅Λຬͨؔ͢ ϋʔυϚʔδϯ47. -
Ϛʔδϯɿઢܗࣝผฏ໘ͱͦΕʹ࠷͍ۙσʔλͱͷڑ% ࠷େͷϚʔδϯ Л Λୡ͢ΔΑ͏ͳ αϙʔτϕΫτϧ TW Λݟ͚ͭΔ ઢܗࣝผฏ໘ D1 D2
Ϛʔδϯ C1 C2 ρ(w, b) = D1 + D2 = min d1i + min d2i ʜϚʔδϯͷఆٛ i i ͱઢͷڑͷެࣜ ઢ w1 x1 + w2 x2 + b = 0ͱɼ (x1i , x2i ) ͷڑ d ࣍ͷࣜͰॻ͖දͤΔɽ wT = (w1 w2 ) xi T = (x1i x2i ) d = |w1 x1i + w2 x2i + b| w2 1 + w2 2 = |wT xi + b| ∥w∥ ϋʔυϚʔδϯ47. -
Ϛʔδϯɿઢܗࣝผฏ໘ͱͦΕʹ࠷͍ۙσʔλͱͷڑ% ρ(w, b) = min d1i + min d2i =
|wT xsv1 + b| ∥w∥ + |wT xsv2 + b| ∥w∥ Ϛʔδϯͷఆٛ = 1 ∥w∥ + 1 ∥w∥ = 2 ∥w∥ ρ(w0 , b0 ) = max ρ(w, b) = max 2 ∥w∥ ∝ max 1 ∥w∥ Ϛʔδϯ࠷େԽ݅ xsv1 |sv ∈ C1 xsv2 |sv ∈ C2 i i w, b w w Ϛʔδϯ࠷େ࣌ͷมɹɹΛɹɹͱஔ͍ͨ w, b w0 , b0 ͱઢͷڑͷެࣜ ઢ w1 x1 + w2 x2 + b = 0ͱɼ (x1i , x2i ) ͷڑ d ࣍ͷࣜͰॻ͖දͤΔɽ wT = (w1 w2 ) xi T = (x1i x2i ) d = |w1 x1i + w2 x2i + b| w2 1 + w2 2 = |wT xi + b| ∥w∥ ϋʔυϚʔδϯ47. -
Ϛʔδϯɿઢܗࣝผฏ໘ͱͦΕʹ࠷͍ۙσʔλͱͷڑ% ρ(w0 , b0 ) ⟹ ρ(w0 ) = max
1 ∥w∥ Ϛʔδϯ࠷େԽ݅ w ⟹ min ∥w∥ w ⟹ min 1 2 ∥w∥2 = min 1 2 wTw w min 1 2 wTw Ϛʔδϯ࠷େԽ݅ w ͱઢͷڑͷެࣜ ઢ w1 x1 + w2 x2 + b = 0ͱɼ (x1i , x2i ) ͷڑ d ࣍ͷࣜͰॻ͖දͤΔɽ wT = (w1 w2 ) xi T = (x1i x2i ) d = |w1 x1i + w2 x2i + b| w2 1 + w2 2 = |wT xi + b| ∥w∥ w ϋʔυϚʔδϯ47. -
ͱઢͷڑͷެࣜ ઢ w1 x1 + w2 x2 + b =
0ͱɼ (x1i , x2i ) ͷڑ d ࣍ͷࣜͰॻ͖දͤΔɽ wT = (w1 w2 ) xi T = (x1i x2i ) d = |w1 x1i + w2 x2i + b| w2 1 + w2 2 = |wT xi + b| ∥w∥ Ϛʔδϯɿઢܗࣝผฏ໘ͱͦΕʹ࠷͍ۙσʔλͱͷڑ% Ϛʔδϯ࠷େԽ݅Λຬͨؔ͢ʢධՁؔʣ min f(w) = min 1 2 wTw w w ϋʔυϚʔδϯ47. -
ti (wT xi + b) − 1 ≥ 0 47.ͷͨΊʹೋͭͷ݅Λநग़͢Δ
ઢܗࣝผฏ໘ͱ֤σʔλͷڑͷ݅ Ϛʔδϯ࠷େԽ݅Λຬͨؔ͢ min f(w) = min 1 2 wTw w w ϋʔυϚʔδϯ47. -
47.ͷͨΊʹೋͭͷ݅Λநग़͢Δ ઢܗࣝผฏ໘ͱ֤σʔλͷڑͷ݅ Ϛʔδϯ࠷େԽ݅Λຬͨؔ͢ min f(w) = min 1 2 wTw
w w ϋʔυϚʔδϯ47. ti (wT xi + b) − 1 ≥ 0 -
ओ gi (w, b) = ti (wT xi + b)
− 1 ≥ 0 f(w) = 1 2 wTw ධՁؔʢ࠷খԽʣ ෆ੍ࣜ݅ ্هͷෆ੍ࣜ݅ԼͰධՁؔΛղ͘ʢ࠷খԽ͢Δʣ͜ͱΛओͱ͢Δɽ ͜͜Ͱɼෆ੍ࣜ݅ͱධՁؔΛಉ࣌ʹຬͨ͢৽͍ؔ͠Λߟ͑Δͱʜ ˜ Lp (w, b, α) = 1 2 wTw − N ∑ i=1 αi (ti (wT xi + b) − 1) ϥάϥϯδϡؔ ͜Ε͔ΒͷܭࢉΛ؆୯ʹ͢ΔͨΊʹ·ͣϥάϥϯδϡؔΛల։͍ͯ͘͠ʜ ৽͍͠จࣈɹͦ͜ͰͰઆ໌͠·͢ʜ αi ϋʔυϚʔδϯ47. -
˜ Lp (w, b, α) = 1 2 wTw −
∑N i=1 αi (ti (wT xi + b) − 1) = 1 2 wTw − ∑N i=1 {αi ti (wT xi + b) − αi } = 1 2 wTw − ∑N i=1 {αi ti wT xi + αi ti b − αi } = 1 2 wTw − ∑N i=1 αi ti wT xi − ∑N i=1 αi ti b + ∑N i=1 αi = 1 2 wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ల։ͨ͠ϥάϥϯδϡؔ ϥάϥϯδϡؔͷల։ ࣍ʹओͷ݅ͱҰக͢Δ݅ΛʢఱԼΓతʹʣఆ͍ٛͯ͘͠ʜ ϋʔυϚʔδϯ47. -
ී௨ͷ࠷খԽͱಉ༷ʹɼมʹΑΔภඍΛͬͯؔͷఈΛ୳͍ͯ͠Δɽ ඍͨ͠ޙʹ࠷దԽͷղͰ͋ΔɹɹɹΛೖͨ͠ܗɽ ϥάϥϯδϡؔ ݅ᶃ ∂ ˜ Lp (w, b, α)
∂w w=w0 = w0 − ∑N i=1 αi ti xi = 0 w0 = ∑N i=1 αi ti xi = 1 2 wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) w = w0 ϋʔυϚʔδϯ47. -
ෆ੍ࣜ݅Λߏ͢Δؔͷޯͷઢܕ݁߹ʹΑͬͯ ධՁؔͷޯΛදݱͰ͖Δ͜ͱΛ͍ࣔͯ͠Δɽ ϥάϥϯδϡؔ ݅ᶃ w0 = ∑N i=1 αi ti
xi ධՁؔͷඍ ∂f ∂w ෆ੍ࣜ݅ͷඍ ∂gi ∂w ∂f ∂w = ∑N i=1 αi ∂gi ∂w f(w) = 1 2 wTw gi (w, b) = ti (wT xi + b) ඍͷܗʹॻ͖͑Δͱʜ = 1 2 wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) ϋʔυϚʔδϯ47. -
ෆ੍ࣜ݅Λߏ͢Δؔͷޯͷઢܕ݁߹ʹΑͬͯ ධՁؔͷޯΛදݱͰ͖Δ͜ͱΛ͍ࣔͯ͠Δɽ ϥάϥϯδϡؔ ݅ᶃ = 1 2 wTw − ∑N
i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) ∂f ∂w = ∑N i=1 αi ∂gi ∂w ॏΈۭؒ W f ∑N i=1 αi gi ЋͷʹΑͬͯ ؔͷܗ͕มΘΔ ϋʔυϚʔδϯ47. -
݅ᶄ ∂ ˜ Lp (w, b, α) ∂b = ∑N
i=1 αi ti = 0 ී௨ͷ࠷খԽͱಉ༷ʹɼมʹΑΔภඍΛͬͯؔͷఈΛ୳͍ͯ͠Δɽ ੍݅ʹΑͬͯੜ·ΕͨมʹΑΔޯʹ͢Δɽ ϥάϥϯδϡؔ = 1 2 wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) ϋʔυϚʔδϯ47. -
݅ᶅ ti (wT xi + b) − 1 ≥ 0
੍݅ͦͷͷͰ͋Δɽ͜ͷ݅σʔλͷ͚ͩଘࡏ͢Δɽ ϥάϥϯδϡؔ = 1 2 wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) ʜ੍݅ ϋʔυϚʔδϯ47. -
݅ᶆ ϥάϥϯδϡͷະఆͱݺΕΔɹͷ݅Ͱ͋Δɽ ͜ͷʹΑ֤ͬͯσʔλʹ͓͚Δෆ੍ࣜ݅ͷॏΈ͕ܾ·Δɽ αi ≥ 0 ϥάϥϯδϡؔ = 1 2
wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) αi ϋʔυϚʔδϯ47. -
݅ᶆ ϥάϥϯδϡͷະఆͱݺΕΔɹͷ݅Ͱ͋Δɽ ͜ͷʹΑ֤ͬͯσʔλʹ͓͚Δෆ੍ࣜ݅ͷॏΈ͕ܾ·Δɽ Ճ͑ͯɼࠨਤʹ͓͚Δ͍ؔͷܗΛੜΈग़͢͜ͱ͕Ͱ͖Δɽ αi ≥ 0 ϥάϥϯδϡؔ f ∑N
i=1 αi gi = 1 2 wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) αi ϋʔυϚʔδϯ47. -
݅ᶇ ૬ิੑ݅ɽ ͜ΕʹΑͬͯɼϥάϥϯδϡؔඍ͢Δ·ͰݩͷධՁؔͱಉ͡ܗΛอ͓ͬͯΓɼ ࠷খԽͷͨΊʹඍͨ࣌͠ʹͷΈෆ੍͕ࣜ݅͏·͘ݱΕΔΑ͏ʹͳͬͯΔʜ αi (ti (wT xi + b)
− 1) = 0 ϥάϥϯδϡؔ = 1 2 wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) ϋʔυϚʔδϯ47. -
݅ᶇ αi (ti (wT xi + b) − 1) =
0 ϥάϥϯδϡؔ = 1 2 wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) ૬ิੑ݅ɽ ·ͨɼɹɹͷ࣌ʹɹɹɹɹɹɹɹɹ͕Γཱͪɼٯಉ༷ʹΓཱͭɽ ͢ͳΘͪɼɹɹɹ͕αϙʔτϕΫτϧͰ͋Δɹɹɹɹɹɹɹɹɹͷ࣌ʹཱ͠ ࠷খԽʹ͓͍ͯɼαϙʔτϕΫτϧ͕ͦͷଋറʹڧ͘د༩͍ͯ͠Δ͜ͱΛ͍ࣔͯ͠Δɽ αi ≠ 0 ti (wT xi + b) − 1 = 0 αi ≠ 0 xi ti (wT xi + b) − 1 = 0 ϋʔυϚʔδϯ47. -
Ld (w, b, α) = 1 2 wTw − ∑N
i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi Ҏ্Λ·ͱΊͯ47.࠷దԽʹ͓͚Δ,,5 ,BSVTI,VIO5VDLFS ݅ ∑N i=1 αi ti = 0 ti (wT xi + b) − 1 ≥ 0 αi ≥ 0 αi (ti (wT xi + b) − 1) = 0 w0 = ∑N i=1 αi ti xi ͕ಘΒΕͨͷͰɼݩͷϥάϥϯδϡؔΛ͜ΕΒͷ݅Ͱஔ͖͑ͯΈΔͱʜ ᶃ ᶄ ᶅ ᶆ ᶇ = 0 ∑N i=1 αi ti ᶄΛద༻ͯ͠ʜ ϥάϥϯδϡؔ = 1 2 wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) ϋʔυϚʔδϯ47. -
Ld (w, α) = 1 2 wTw − ∑N i=1
αi ti wT xi + ∑N i=1 αi = 1 2 wTw − wT(∑N i=1 αi ti xi ) + ∑N i=1 αi Ҏ্Λ·ͱΊͯ47.࠷దԽʹ͓͚Δ,,5 ,BSVTI,VIO5VDLFS ݅ ͕ಘΒΕͨͷͰɼݩͷϥάϥϯδϡؔΛ͜ΕΒͷ݅Ͱஔ͖͑ͯΈΔͱʜ ɹͱؔͷͳ͍ɹΛ૯ͷ֎ʹग़ͯ͠ʜ i wT ∑N i=1 αi ti = 0 ti (wT xi + b) − 1 ≥ 0 αi ≥ 0 αi (ti (wT xi + b) − 1) = 0 w0 = ∑N i=1 αi ti xi ᶃ ᶄ ᶅ ᶆ ᶇ ϥάϥϯδϡؔ = 1 2 wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) ϋʔυϚʔδϯ47. -
Ld (α) = 1 2 wT 0 w0 − wT
0 (∑N i=1 αi ti xi ) + ∑N i=1 αi Ҏ্Λ·ͱΊͯ47.࠷దԽʹ͓͚Δ,,5 ,BSVTI,VIO5VDLFS ݅ ͕ಘΒΕͨͷͰɼݩͷϥάϥϯδϡؔΛ͜ΕΒͷ݅Ͱஔ͖͑ͯΈΔͱʜ ࠷దԽ݅ʹ͓͍ͯɹɹɹͳͷͰॻ͖͑ͯʜ w = w0 ∑N i=1 αi ti = 0 ti (wT xi + b) − 1 ≥ 0 αi ≥ 0 αi (ti (wT xi + b) − 1) = 0 w0 = ∑N i=1 αi ti xi ᶃ ᶄ ᶅ ᶆ ᶇ ϥάϥϯδϡؔ = 1 2 wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) ϋʔυϚʔδϯ47. -
Ld (α) = 1 2 (∑N i=1 αi ti xi
)T(∑N i=1 αi ti xi ) − (∑N i=1 αi ti xi )T(∑N i=1 αi ti xi ) + ∑N i=1 αi Ҏ্Λ·ͱΊͯ47.࠷దԽʹ͓͚Δ,,5 ,BSVTI,VIO5VDLFS ݅ ͕ಘΒΕͨͷͰɼݩͷϥάϥϯδϡؔΛ͜ΕΒͷ݅Ͱஔ͖͑ͯΈΔͱʜ ᶃΛద༻ͯ͠ʜ ∑N i=1 αi ti = 0 ti (wT xi + b) − 1 ≥ 0 αi ≥ 0 αi (ti (wT xi + b) − 1) = 0 w0 = ∑N i=1 αi ti xi ᶃ ᶄ ᶅ ᶆ ᶇ ϥάϥϯδϡؔ = 1 2 wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) w0 = ∑N i=1 αi ti xi ϋʔυϚʔδϯ47. -
Ld (α) = 1 2 (∑N i=1 ∑N j=1 αi
αj ti tj xT i xj ) − (∑N i=1 ∑N j=1 αi αj ti tj xT i xj ) + ∑N i=1 αi Ҏ্Λ·ͱΊͯ47.࠷దԽʹ͓͚Δ,,5 ,BSVTI,VIO5VDLFS ݅ ͕ಘΒΕͨͷͰɼݩͷϥάϥϯδϡؔΛ͜ΕΒͷ݅Ͱஔ͖͑ͯΈΔͱʜ ͱॻ͚ͯɼ (∑N i=1 αi ti xi )T(∑N i=1 αi ti xi ) = (∑N i=1 ∑N j=1 αi ti xT i αj tj xj ) = (∑N i=1 ∑N j=1 αi αj ti tj xT i xj ) ∑N i=1 αi ti = 0 ti (wT xi + b) − 1 ≥ 0 αi ≥ 0 αi (ti (wT xi + b) − 1) = 0 w0 = ∑N i=1 αi ti xi ᶃ ᶄ ᶅ ᶆ ᶇ ϥάϥϯδϡؔ = 1 2 wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) w0 = ∑N i=1 αi ti xi ϋʔυϚʔδϯ47. -
Ld (α) = − 1 2 ∑N i=1 ∑N j=1
αi αj ti tj xT i xj + ∑N i=1 αi Ҏ্Λ·ͱΊͯ47.࠷దԽʹ͓͚Δ,,5 ,BSVTI,VIO5VDLFS ݅ ͕ಘΒΕͨͷͰɼݩͷϥάϥϯδϡؔΛ͜ΕΒͷ݅Ͱஔ͖͑ͯΈΔͱʜ ϥάϥϯδϡؔ = 1 2 wTw − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) ∑N i=1 αi ti = 0 ti (wT xi + b) − 1 ≥ 0 αi ≥ 0 αi (ti (wT xi + b) − 1) = 0 w0 = ∑N i=1 αi ti xi ᶃ ᶄ ᶅ ᶆ ᶇ ϋʔυϚʔδϯ47. -
ϥάϥϯδϡؔ = 1 2 wTw − ∑N i=1 αi ti
wT xi − b∑N i=1 αi ti + ∑N i=1 αi ˜ Lp (w, b, α) = 1 2 wTw − ∑N i=1 αi (ti (wT xi + b) − 1) Ld (α) = − 1 2 ∑N i=1 ∑N j=1 αi αj ti tj xT i xj + ∑N i=1 αi Ҏ্Λ·ͱΊͯ47.࠷దԽʹ͓͚Δ,,5 ,BSVTI,VIO5VDLFS ݅ ∑N i=1 αi ti = 0 ti (wT xi + b) − 1 ≥ 0 αi ≥ 0 αi (ti (wT xi + b) − 1) = 0 w0 = ∑N i=1 αi ti xi ͕ಘΒΕͨͷͰɼݩͷϥάϥϯδϡؔΛ͜ΕΒͷ݅Ͱஔ͖͑ͯΈΔͱʜ ᶃ ᶄ ᶅ ᶆ ᶇ ਖ਼ղϥϕϧɹͱσʔλɹطͳͷͰɹະఆɹͷΈͷؔʹͳΔɽ ti xi Ld αi ϋʔυϚʔδϯ47. -
Ld (α) = − 1 2 ∑N i=1 ∑N j=1
αi αj ti tj xT i xj + ∑N i=1 αi ϥάϥϯδϡؔ α = (α1 α2 . . . αN )T, t = (t1 t2 . . . tN )T, H = (Hij = ti tj xT i xj ), 1 = (1 1...1)T ͱ͢Δͱɼ Ld (α) = − 1 2 αT Hα + αT1 ͜Εʹ,,5݅Λߟྀ͢Δͱʜ αi (ti (wT xi + b) − 1) { ti (wT xi + b) − 1 = 0 ⟹ αi > 0 ti (wT xi + b) − 1 ≠ 0 ⟹ αi = 0 ᶅɼᶆɼᶇ ∑N i=1 αi ti = αT t = 0 ᶄ ͱͳΔͷͰɼ͜ΕΒΛ༻͍ͯ৽ͨͳΛΈཱͯΔͱ࣍ͷରʹͳΔ ϋʔυϚʔδϯ47. -
α w ͜ΕʹΑΓɼओͷ࠷খԽ͕ରͷ࠷େԽʹஔ͖Θͬͨʜ ओͰ͠Α͏ͱ͍ͯͨ͜͠ͱ ରͰ͠Α͏ͱ͍ͯ͠Δ͜ͱ ϥάϥϯδϡͷະఆ๏Ͱ ϥάϥϯδϡؔͷҌΛٻΊ͍ͯΔʜ ධՁؔʢ࠷େԽʣ ੍݅ ର
Ld (α) = − 1 2 αT Hα + αT1 αT t = 0 ϋʔυϚʔδϯ47. -
͜ΕʹΑΓɼओͷ࠷খԽ͕ରͷ࠷େԽʹஔ͖Θͬͨʜ α w ͜ͷ࠷దԽΛ࣮ࡍʹղ͘ʹ 4.0ͱݺΕΔ ߴͳΞϧΰϦζϜ͕༻͍ΒΕΔ ධՁؔʢ࠷େԽʣ ੍݅ ର Ld
(α) = − 1 2 αT Hα + αT1 αT t = 0 ϋʔυϚʔδϯ47. -
w0 = ∑N i=1 αi ti xi 4.0Λ༻͍ͯରΛղ͖ɼ࣮ࡍʹɹΛશͯٻΊऴ͑ͨΒʜ α ओʹ͓͚Δ,,5݅ͷᶃΛ༻͍ͯɹΛٻΊʜ
w0 tsv (wT 0 xsv + b0 ) − 1 = 0 b0 = 1 tsv − wT 0 xsv αϙʔτϕΫτϧ TW ͢ͳΘͪɹɹɹɹɹɹɹɹͱͳ ɹɹɹ͓ΑͼɹɹɹΛ༻͍Εɼ࠷దͳόΠΞεɹ ti (wT xi + b) − 1 = 0 x = xsv t = tsv b0 ͔ΒٻΊΔɽ αϙʔτϕΫτϧෳ͋ΔͷͰͦΕΒͷฏۉͰٻΊΔ͜ͱ͋Δɽ ධՁؔʢ࠷େԽʣ ੍݅ ର Ld (α) = − 1 2 αT Hα + αT1 αT t = 0 ϋʔυϚʔδϯ47. -
ධՁؔʢ࠷େԽʣ ੍݅ ର Ld (α) = − 1 2 αT
Hα + αT1 αT t = 0 wT 0 w0 = ∑N i=1 ˜ αi ti xT i w0 4.0Λ༻͍ͯରΛղ͖ɼ࣮ࡍʹɹΛશͯٻΊऴ͑ͨΒʜ α ϥάϥϯδϡͷ࠷దղΛɹɹɹɹɹɹɹͷΑ͏ʹ͢Δͱ ˜ α = ( ˜ α1 , . . . , ˜ αN )T ,,5ᶃͷٯ ͱͯ͠ΔͷTW༝དྷͷͷ͚ͩͳͷͰ ˜ αi ti (wT xi + b0 ) − 1 = 0 ͱͯ͠ܭࢉʜ = ∑N i=1 ˜ αi − b0 ∑N i=1 ti αi = ∑N i=1 ˜ αi (1 − ti b0 ) = ∑N i=1 ˜ αi ,,5ᶄ w0 = ∑N i=1 αi ti xi ∑N i=1 αi ti = 0 Dmax = 1 ∥w0 ∥ = 1 wT 0 w0 = 1 ∑N i=1 αi Αͬͯ࠷େϚʔδϯ ϋʔυϚʔδϯ47. -
ධՁؔʢ࠷খԽʣ ෆ੍ࣜ݅ f(x) = x2 { g1 (x) = x
− 2 ≤ 0 g2 (x) = x − 3 ≤ 0 L(x, λ1 , λ2 ) = x2 − λ1 g1 (x) − λ2 g2 (x) = x2 − λ1 (x − 2) − λ2 (x − 3) ,,5݅ʜ ∂L ∂x = 2x − λ1 − λ2 = 0 ∂L ∂λ1 = − (x − 2) ≥ 0 ⟹ x − 2 ≤ 0 ∂L ∂λ2 = − (x − 3) ≥ 0 ⟹ x − 3 ≤ 0 λ1 , λ2 ≥ 0 λ1 (x − 2) = 0 λ2 (x − 3) = 0 λ1 , λ2 ≠ 0 ͷ࣌ x = 2 ⟹ g2 (2) < 0 x = 3 ⟹ g1 (3) > 0 ༗ޮ ੍͕݅ෳ༩͑ΒΕͨ࣌ɼʹཱ͍ͬͯΔ੍Λ۩ମతʹܭࢉ͢ΔͰ͢ɽ 47.తʹαϙʔτϕΫτϧʹؔ͢Δ੍͕ʹཱ͍ͬͯΔʹͨΓ·͢ɽ ओͷϥάϥϯδϡؔ ྫओ ϋʔυϚʔδϯ47. -
ධՁؔʢ࠷খԽʣ ෆ੍ࣜ݅ f(x) = x2 { g1 (x) = x
− 2 ≤ 0 g2 (x) = x − 3 ≤ 0 ੍͕݅ෳ༩͑ΒΕͨ࣌ɼʹཱ͍ͬͯΔ੍Λ۩ମతʹܭࢉ͢ΔͰ͢ɽ 47.తʹαϙʔτϕΫτϧʹؔ͢Δ੍͕ʹཱ͍ͬͯΔʹͨΓ·͢ɽ ओͷϥάϥϯδϡؔ L(x, λ1 , λ2 ) = x2 − λ1 g1 (x) − λ2 g2 (x) = x2 − λ1 (x − 2) − λ2 (x − 3) ,,5݅ʜ ∂L ∂x = 2x − λ1 − λ2 = 0 ∂L ∂λ1 = − (x − 2) ≥ 0 ⟹ x − 2 ≤ 0 ∂L ∂λ2 = − (x − 3) ≥ 0 ⟹ x − 3 ≤ 0 λ1 , λ2 ≥ 0 λ1 (x − 2) = 0 λ2 (x − 3) = 0 ༗ޮ λ1 ≠ 0 λ2 = 0 ྫओ ϋʔυϚʔδϯ47. -
ઢܗࣝผฏ໘ͱ֤σʔλͷڑͷ݅ Ϛʔδϯ࠷େԽ݅Λຬͨؔ͢ ti (wT xi + b) ≥ 1 min
f(w) = min 1 2 wTw w w α w ϥάϥϯδϡͷະఆ๏Λ༻͍ͯ ओΛରʹམͱ͠ࠐΜͰ ઢܗࣝผฏ໘ͷࣜΛٻΊΔ D D C1 C2 ·ͱΊ ϋʔυϚʔδϯࣝผث ઢܗࣝผՄೳͰ͋Δ߹ʹ࠷େϚʔδϯΛ ࣮ݱ͢Δઢܗࣝผฏ໘Λج४ʹ̎Ϋϥε ͷྨΛߦ͏ํ๏ ϋʔυϚʔδϯ47. -
065-*/& ఆٛ ੍݅ͷ࠶ઃఆ ϥάϥϯδϡͷະఆ๏ ର ຕ ຕ ຕ ຕ ઢܗՄೳͰͳ͍߹ͷ
·ͱΊ ຕ ιϑτϚʔδϯ47. -
࠷େϚʔδϯΛ࣮ݱ͢Δઢܗࣝผฏ໘Λج४ʹ̎ΫϥεͷྨΛߦ͏ํ๏ ఆٛ ೋछྨͷσʔλ܈Λ ࠷ྑ͍ײ͡ʹ͚Δڥքઢ ઢܗࣝผฏ໘ D D Ϛʔδϯ ઢܗՄೳͰͳ͍ྫ ࣝผฏ໘Λ͑ͯ
σʔλ͕ଘࡏ͢Δ ઢܗͰ͖ͳ͍σʔλͱʜ ιϑτϚʔδϯ47. -
࠷େϚʔδϯΛ࣮ݱ͢Δઢܗࣝผฏ໘Λج४ʹ̎ΫϥεͷྨΛߦ͏ํ๏ ఆٛ ೋछྨͷσʔλ܈Λ ࠷ྑ͍ײ͡ʹ͚Δڥքઢ ઢܗࣝผฏ໘ D D Ϛʔδϯ ઢܗՄೳͰͳ͍ྫ ࣝผฏ໘Λ͍͑ͯͳ͍͕
TWͷଆʹσʔλ͕ଘࡏ͢Δ ઢܗͰ͖ͳ͍σʔλͱʜ ιϑτϚʔδϯ47. -
࠷େϚʔδϯΛ࣮ݱ͢Δઢܗࣝผฏ໘Λج४ʹ̎ΫϥεͷྨΛߦ͏ํ๏ ఆٛ ೋछྨͷσʔλ܈Λ ࠷ྑ͍ײ͡ʹ͚Δڥքઢ ઢܗࣝผฏ໘ D D Ϛʔδϯ ઢܗՄೳͰͳ͍ྫ ϊΠζ࣍ୈͰΫϥε͕
ͿΕΔ͜ͱ͕͋Δͷͨͪ ઢܗͰ͖ͳ͍σʔλͱʜ ιϑτϚʔδϯ47. -
࠷େϚʔδϯΛ࣮ݱ͢Δઢܗࣝผฏ໘Λج४ʹ̎ΫϥεͷྨΛߦ͏ํ๏ ఆٛ ೋछྨͷσʔλ܈Λ ࠷ྑ͍ײ͡ʹ͚Δڥքઢ ઢܗࣝผฏ໘ D D Ϛʔδϯ ઢܗՄೳͰͳ͍ྫ ͜ΕΒͷσʔλΛߟྀͨ͠
ࣝผؔͷ࠶ઃఆ͕ඞཁ ઢܗͰ͖ͳ͍σʔλͱʜ ιϑτϚʔδϯ47. -
࠷େϚʔδϯΛ࣮ݱ͢Δઢܗࣝผฏ໘Λج४ʹ̎ΫϥεͷྨΛߦ͏ํ๏ ఆٛ ೋछྨͷσʔλ܈Λ ࠷ྑ͍ײ͡ʹ͚Δڥքઢ ઢܗࣝผฏ໘ D D Ϛʔδϯ ઢܗՄೳͰͳ͍ྫ ͜ΕΒͷσʔλΛߟྀͨ͠
੍݅ͷ࠶ઃఆ͕ඞཁ ઢܗͰ͖ͳ͍σʔλͱʜ ιϑτϚʔδϯ47. -
੍݅ͷ࠶ઃఆ ઢܗՄೳͳ߹ gi (w, b) = ti (wT xi +
b) ≥ 1 ઢܗՄೳͰͳ͍߹ gi (w, b) = ti (wT xi + b) + ξi ≥ 1 D D ξi = 0 0 < ξi ≤ 1 ξi < 1 ξi = 0 +ξi σʔλʹΑͬͯҟͳΔΛ࣋ͭεϥοΫมɹΛಋೖͯ͠ʜ ξi ιϑτϚʔδϯ47. -
੍݅ͷ࠶ઃఆ ઢܗՄೳͳ߹ gi (w, b) = ti (wT xi +
b) ≥ 1 ઢܗՄೳͰͳ͍߹ gi (w, b) = ti (wT xi + b) + ξi ≥ 1 D D +ξi ͱͳΔΑ͏ʹ֤σʔλʹରͯ͠ɹΛܾΊΔ ξi ιϑτϚʔδϯ47. ξi = 0 0 < ξi ≤ 1 ξi < 1 ξi = 0 -
ઢܗՄೳͰͳ͍߹ gi (w, b) = ti (wT xi + b)
+ ξi ≥ 1 D D +ξi ੍݅ͷ࠶ઃఆ ξi = max[0,1 − ti (wT xi + b)] εϥοΫมɹͱʜTWͱࣝผڥքͱͷڑΛͱͨ࣌͠ʹ ྨ͕ਖ਼͘͠TWΑΓࣝผڥք͔Βԕ͍ͷʹରͯ͠Λฦ͠ ྨਖ਼͍͕͠TWΑΓࣝผڥքʹ͍ۙͷʹରͯͦ͠ͷڑΛฦ͠ ྨ͕ਖ਼͘͠ͳ͍ͷʹରͯ͠ಉ༷ʹͦͷڑΛฦ͢Α͏ͳมͰ͋Δ ξi ιϑτϚʔδϯ47. ξi = 0 0 < ξi ≤ 1 ξi < 1 ξi = 0 -
ઢܗՄೳͰͳ͍߹ gi (w, b) = ti (wT xi + b)
+ ξi ≥ 1 D D ξi = 0 0 < ξi ≤ 1 ξi < 1 ξi = 0 +ξi ੍݅ͷ࠶ઃఆ ξi = max[0,1 − ti (wT xi + b)] গ͠ڧҾʹݴ͍͑Δͱ εϥοΫมɹͱʜ47.ͷఆٛʹ্ख͘Βͳ͔ͬͨσʔλΛແཧΓ TWʹԡ͠ࠐΉΑ͏ͳม ξi ιϑτϚʔδϯ47. -
ઢܗՄೳͰͳ͍߹ gi (w, b) = ti (wT xi + b)
+ ξi ≥ 1 D D ξi = 0 0 < ξi ≤ 1 ξi < 1 ξi = 0 +ξi ੍݅ͷ࠶ઃఆ ξi = max[0,1 − ti (wT xi + b)] ٙɿTWʹԡ͠ࠐΜͰେৎͳͷ͔ʜ ͋͘·Ͱઢܗࣝผฏ໘Λಋग़͢ΔͨΊͷ֦ுͳͷͰͳ͍ ࣝผ͢ΔࡍٻΊͨɹͱɹΛͬͯΫϥεΛܾΊΔɽ w0 b0 ιϑτϚʔδϯ47. -
ઢܗՄೳͰͳ͍߹ gi (w, b) = ti (wT xi + b)
+ ξi ≥ 1 D D ξi = 0 0 < ξi ≤ 1 ξi < 1 ξi = 0 +ξi ੍݅ͷ࠶ઃఆ ξi = max[0,1 − ti (wT xi + b)] ͱͳΔͷͰɼ͜ͷ͕,ҎԼͰ͋ΔͳΒޡࣝผ,ݸະຬʹͳΔɽ ξi > 1 ޡࣝผͨ͠ͷͷεϥοΫม Λ,ूΊΔͱ∑ i∈{1,2,...n} ξi > K ͜͜·ͰͷใͰϥάϥϯδϡະఆ๏ʹ͓͚ΔओΛߏஙͯ͠ΈΔʜ ιϑτϚʔδϯ47. -
ઢܗՄೳͰͳ͍߹ gi (w, b) = ti (wT xi + b)
+ ξi ≥ 1 D D ξi = 0 0 < ξi ≤ 1 ξi < 1 ξi = 0 +ξi ੍݅ͷ࠶ઃఆ ξi = max[0,1 − ti (wT xi + b)] ξi = f+ (1 − ti (wT xi + b)) ͱॻ͚ΔɹɹҰൠతʹ f+ (x) = { x (x > 0) 0 (x ≤ 0) f+ ͱॻ͘͜ͱ͕Ͱ͖ɼͦͷάϥϑͷܗ͔ΒIJOHFؔͱݺΕΔ ιϑτϚʔδϯ47. -
f(w, ξ) = 1 2 wTw + C∑N i=1 ξi
ओ ti (wT xi + b) + ξi ≥ 1, ξi ≥ 0 ධՁؔʢ࠷খԽʣ ෆ੍ࣜ݅ ෆ੍ࣜ݅ʹθϩҎ্ͷɹ͕ଘࡏ͢ΔͷͰ͜ͷ࣍ୈͰͲΜͳɹͰ ۃʹݴ͑ɹɹɹʹ͕ͯ݅͠Γཱͬͯ͠·͏͜ͱ͕ى͜Γ͏Δʜ ͦ͜ͰɼධՁؔɹɹɹʹɹʹ༝དྷ͢ΔϖφϧςΟɹɹɹɹΛ՝͢͜ͱͰ ɹ͕খ͘͞ͳΓ͗͢Δ͜ͱΛ͍Ͱ͍Δʜ f(w, ξ) ξi C∑N i=1 ξi w w = 0 w ξi ͜ͷ࣌ϖφϧςΟ߲ʹ͋Δɹ՝͢ΔϖφϧςΟͷڧ͞ΛܾΊΔͰ͋Γ ਓ͕ؒҙʹܾΊΔύϥϝʔλ ϋΠύʔύϥϝʔλ Ͱ͋Δɽ C C∑N i=1 ξi ιϑτϚʔδϯ47. -
f(w, ξ) = 1 2 wTw + C∑N i=1 ξi
ओ ti (wT xi + b) − 1 + ξi ≥ 0, ξi ≥ 0 ධՁؔʢ࠷খԽʣ ෆ੍ࣜ݅ ্هͷෆ੍ࣜ݅ԼͰධՁؔΛղ͘ʢ࠷খԽ͢Δʣ͜ͱΛओͱ͢Δɽ ઢܗՄೳͳ߹ͱಉ༷ʹϥάϥϯδϡؔΛߟ͑Δͱʜ ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi ϥάϥϯδϡؔ ͜ΕઢܗՄೳͳ߹ͱಉ༷ʹ·ͣϥάϥϯδϡؔΛల։͍ͯ͘͠ʜ ιϑτϚʔδϯ47. -
ల։ͨ͠ϥάϥϯδϡؔ ϥάϥϯδϡؔͷల։ ࣍ʹओͷ݅ͱҰக͢Δ݅ΛʢఱԼΓతʹʣఆ͍ٛͯ͘͠ʜ ˜ Lp (w, b, α, ξ, μ)
= 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 {αi ti (wT xi + b) − αi + αi ξi } − ∑N i=1 μi ξi = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 {αi ti wT xi + αi ti b − αi + αi ξi } − ∑N i=1 μi ξi = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi ti wT xi − ∑N i=1 αi ti b + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi −∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ιϑτϚʔδϯ47. -
݅ᶃɼᶄઢܗࣝผՄೳͳ߹ͱಉ͡ܗʹͳΔɽ ϥάϥϯδϡؔ ݅ᶃ ∂ ˜ Lp (w, b, α, ξ,
μ) ∂w w=w0 = w0 − ∑N i=1 αi ti xi = 0 w0 = ∑N i=1 αi ti xi = 1 2 wTw + C∑N i=1 ξi −∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ݅ᶄ ∂ ˜ Lp (w, b, α, ξ, μ) ∂b = ∑N i=1 αi ti = 0 1 2 wTw −∑N i=1 αi ti wT xi −b∑N i=1 αi ti ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi ιϑτϚʔδϯ47. -
ϥάϥϯδϡະఆͷɹͱɹͷ݅ ݅ᶇ ͔ΒϖφϧςΟͷڧ͞ΛܾΊΔɹ͕ ɹͷൣғΛԡ͚͍͑ͭͯ͞Δ݅Λಋ͖ग़ͤΔɽ ϥάϥϯδϡؔ ݅ᶅ ∂ ˜ Lp (w,
b, α, ξ, μ) ∂ξi = C − αi − μi = 0 C − αi = μi ≥ 0 ⟹ C − αi ≥ 0 C ≥ αi ≥ 0 αi μi C αi = 1 2 wTw + C∑N i=1 ξi −∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi C∑N i=1 ξi ∑N i=1 αi ξi − ∑N i=1 μi ξi ιϑτϚʔδϯ47. -
ϥάϥϯδϡؔ ݅ᶆ = 1 2 wTw + C∑N i=1 ξi
−∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi ti (wT xi + b) − 1 + ξi ʜ੍݅ ti (wT xi + b) − 1 + ξi ≥ 0 ੍݅ͦͷͷͰ͋Δɽ͜ͷ݅σʔλͷ͚ͩଘࡏ͢Δɽ ιϑτϚʔδϯ47. -
ϥάϥϯδϡؔ ݅ᶇ = 1 2 wTw + C∑N i=1 ξi
−∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi ξi αi μi ξi ξi ξi ξi ξi αi μi ϥάϥϯδϡͷະఆͰ͋Δɹɼ͓ΑͼεϥοΫมɹͷ݅Ͱ͋Δɽ ͜ͷʹΑ֤ͬͯσʔλʹ͓͚Δෆ੍ࣜ݅ͷॏΈ͕ܾ·Δɽ ξi ≥ 0, αi ≥ 0, μi ≥ 0 αi μi ξi αi αi αi ιϑτϚʔδϯ47. -
ϥάϥϯδϡؔ ݅ᶈ = 1 2 wTw + C∑N i=1 ξi
−∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi αi (ti (wT xi + b) − 1 + ξi ) = 0 αi (ti (wT xi + b) − 1 + ξi ) ૬ิੑ݅ɽ ·ͨɼɹɹͷ࣌ʹɹɹɹɹɹɹɹɹɹɹ͕Γཱͪɼٯಉ༷ʹΓཱͭɽ ͢ͳΘͪɼɹɹɹ͕αϙʔτϕΫτϧͰ͋Δɹɹɹɹɹɹɹɹɹɹͷ࣌ʹཱ͠ ࠷খԽʹ͓͍ͯɼαϙʔτϕΫτϧ͕ͦͷଋറʹڧ͘د༩͍ͯ͠Δ͜ͱΛ͍ࣔͯ͠Δɽ αi ≠ 0 ti (wT xi + b) − 1 + ξi = 0 αi ≠ 0 xi ti (wT xi + b) − 1 + ξi = 0 ti (wT xi + b) − 1 + ξi = 0 ιϑτϚʔδϯ47. -
ϥάϥϯδϡؔ ݅ᶉ = 1 2 wTw + C∑N i=1 ξi
−∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi μi ξi = 0 αi (ti (wT xi + b) − 1 + ξi ) ૬ิੑ݅ɽ ·ͨɼɹɹͷ࣌ʹɹɹɹ͕Γཱͪɼٯಉ༷ʹΓཱͭɽ μi ≠ 0 ξi = 0 ιϑτϚʔδϯ47. -
ϥάϥϯδϡؔ = 1 2 wTw + C∑N i=1 ξi −∑N
i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi αi (ti (wT xi + b) − 1 + ξi ) ݅ᶅ C − αi = μi ≥ 0 ⟹ C − αi ≥ 0 ⟺ C ≥ αi ≥ 0 Λ͏ͱ݅ᶉʜ μi ξi = 0 { μi = 0 (ξi > 0) ⟹ C − αi = μi = 0 ⟹ C − αi = 0 ⟺ C = αi ≥ 0 ξi = 0 (μi > 0) ⟹ C − αi = μi > 0 ⟹ C − αi > 0 ⟺ C > αi ≥ 0 C = αi C > αi μi = 0 μi > 0 ιϑτϚʔδϯ47. -
ϥάϥϯδϡؔ = 1 2 wTw + C∑N i=1 ξi −∑N
i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi αi (ti (wT xi + b) − 1 + ξi ) ݅ᶉ D D μi ξi = 0 { μi = 0 (ξi > 0) ⟹ C − αi = μi = 0 ⟹ C − αi = 0 ⟺ C = αi ≥ 0 ξi = 0 (μi > 0) ⟹ C − αi = μi > 0 ⟹ C − αi > 0 ⟺ C > αi ≥ 0 μi = 0 (ξi > 0) ⟹ C − αi = μi = 0 ⟹ C − αi = 0 ⟺ C = αi ≥ 0 ιϑτϚʔδϯ47. -
ϥάϥϯδϡؔ = 1 2 wTw + C∑N i=1 ξi −∑N
i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi αi (ti (wT xi + b) − 1 + ξi ) ݅ᶉ D D +ξi D D μi ξi = 0 { μi = 0 (ξi > 0) ⟹ C − αi = μi = 0 ⟹ C − αi = 0 ⟺ C = αi ≥ 0 ξi = 0 (μi > 0) ⟹ C − αi = μi > 0 ⟹ C − αi > 0 ⟺ C > αi ≥ 0 μi = 0 (ξi > 0) ⟹ C − αi = μi = 0 ⟹ C − αi = 0 ⟺ C = αi ≥ 0 ͕ ্ݶαϙʔτϕΫτϧͨͪ αi ιϑτϚʔδϯ47. -
ϥάϥϯδϡؔ = 1 2 wTw + C∑N i=1 ξi −∑N
i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi αi (ti (wT xi + b) − 1 + ξi ) ݅ᶉ D D μi ξi = 0 { μi = 0 (ξi > 0) ⟹ C − αi = μi = 0 ⟹ C − αi = 0 ⟺ C = αi ≥ 0 ξi = 0 (μi > 0) ⟹ C − αi = μi > 0 ⟹ C − αi > 0 ⟺ C > αi ≥ 0 ξi = 0 (μi > 0) ⟹ C − αi = μi > 0 ⟹ C − αi > 0 ⟺ C > αi ≥ 0 ιϑτϚʔδϯ47. -
ϥάϥϯδϡؔ = 1 2 wTw + C∑N i=1 ξi −∑N
i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi αi (ti (wT xi + b) − 1 + ξi ) ݅ᶉ D D μi ξi = 0 { μi = 0 (ξi > 0) ⟹ C − αi = μi = 0 ⟹ C − αi = 0 ⟺ C = αi ≥ 0 ξi = 0 (μi > 0) ⟹ C − αi = μi > 0 ⟹ C − αi > 0 ⟺ C > αi ≥ 0 ξi = 0 (μi > 0) ⟹ C − αi = μi > 0 ⟹ C − αi > 0 ⟺ C > αi ≥ 0 D D αi > 0 +ξi ͕ ࣗ༝αϙʔτϕΫτϧͨͪ αi ιϑτϚʔδϯ47. -
ઢܗࣝผՄೳͰͳ͍47.࠷దԽʹ͓͚Δ,,5 ,BSVTI,VIO5VDLFS ݅ ͕ಘΒΕͨͷͰɼݩͷϥάϥϯδϡؔΛ͜ΕΒͷ݅Ͱஔ͖͑ͯΈΔͱʜ w0 = ∑N i=1 αi ti
xi ᶃ ∑N i=1 αi ti = 0 ᶄ ᶄΛద༻ͯ͠ʜ ϥάϥϯδϡؔ = 1 2 wTw + C∑N i=1 ξi −∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi ᶆ ti (wT xi + b) − 1 + ξi ≥ 0 C − αi = μi ᶅ ᶇ ξi ≥ 0, αi ≥ 0, μi ≥ 0 ᶇ αi (ti (wT xi + b) − 1 + ξi ) = 0 ᶈ μi ξi = 0 ᶉ Ld (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi −∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ∑N i=1 αi ti = 0 ιϑτϚʔδϯ47. -
ઢܗࣝผՄೳͰͳ͍47.࠷దԽʹ͓͚Δ,,5 ,BSVTI,VIO5VDLFS ݅ ͕ಘΒΕͨͷͰɼݩͷϥάϥϯδϡؔΛ͜ΕΒͷ݅Ͱஔ͖͑ͯΈΔͱʜ w0 = ∑N i=1 αi ti
xi ᶃ ∑N i=1 αi ti = 0 ᶄ ϥάϥϯδϡؔ = 1 2 wTw + C∑N i=1 ξi −∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi ᶆ ti (wT xi + b) − 1 + ξi ≥ 0 C − αi = μi ᶅ ᶇ ξi ≥ 0, αi ≥ 0, μi ≥ 0 ᶇ αi (ti (wT xi + b) − 1 + ξi ) = 0 ᶈ μi ξi = 0 ᶉ Ld (w, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi −∑N i=1 αi ti wT xi + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ∑N i=1 αi ξi + ∑N i=1 μi ξi = ∑N i=1 (αi + μi )ξi = ∑N i=1 Cξi = C∑N i=1 ξi ᶅ͔Β −∑N i=1 αi ξi − ∑N i=1 μi ξi ͳͷͰ ιϑτϚʔδϯ47. -
ઢܗࣝผՄೳͰͳ͍47.࠷దԽʹ͓͚Δ,,5 ,BSVTI,VIO5VDLFS ݅ ͕ಘΒΕͨͷͰɼݩͷϥάϥϯδϡؔΛ͜ΕΒͷ݅Ͱஔ͖͑ͯΈΔͱʜ w0 = ∑N i=1 αi ti
xi ᶃ ∑N i=1 αi ti = 0 ᶄ ϥάϥϯδϡؔ = 1 2 wTw + C∑N i=1 ξi −∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi ᶆ ti (wT xi + b) − 1 + ξi ≥ 0 C − αi = μi ᶅ ᶇ ξi ≥ 0, αi ≥ 0, μi ≥ 0 ᶇ αi (ti (wT xi + b) − 1 + ξi ) = 0 ᶈ μi ξi = 0 ᶉ −∑N i=1 αi ti wT xi + ∑N i=1 αi − C∑N i=1 ξi ∑N i=1 αi ξi + ∑N i=1 μi ξi = ∑N i=1 (αi + μi )ξi = ∑N i=1 Cξi = C∑N i=1 ξi ᶅ͔Β ͳͷͰ −C∑N i=1 ξi Ld (w, α, ξ) = 1 2 wTw + C∑N i=1 ξi C∑N i=1 ξi ιϑτϚʔδϯ47. -
ઢܗࣝผՄೳͰͳ͍47.࠷దԽʹ͓͚Δ,,5 ,BSVTI,VIO5VDLFS ݅ ͕ಘΒΕͨͷͰɼݩͷϥάϥϯδϡؔΛ͜ΕΒͷ݅Ͱஔ͖͑ͯΈΔͱʜ w0 = ∑N i=1 αi ti
xi ᶃ ∑N i=1 αi ti = 0 ᶄ ϥάϥϯδϡؔ = 1 2 wTw + C∑N i=1 ξi −∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi ᶆ ti (wT xi + b) − 1 + ξi ≥ 0 C − αi = μi ᶅ ᶇ ξi ≥ 0, αi ≥ 0, μi ≥ 0 ᶇ αi (ti (wT xi + b) − 1 + ξi ) = 0 ᶈ μi ξi = 0 ᶉ Ld (w, b, α) = 1 2 wTw − ∑N i=1 αi ti wT xi + ∑N i=1 αi ∑N i=1 αi ξi + ∑N i=1 μi ξi = ∑N i=1 (αi − μi )ξi = ∑N i=1 Cξi = C∑N i=1 ξi ᶅ͔Β ͳͷͰ ιϑτϚʔδϯ47. -
ઢܗࣝผՄೳͰͳ͍47.࠷దԽʹ͓͚Δ,,5 ,BSVTI,VIO5VDLFS ݅ ͕ಘΒΕͨͷͰɼݩͷϥάϥϯδϡؔΛ͜ΕΒͷ݅Ͱஔ͖͑ͯΈΔͱʜ w0 = ∑N i=1 αi ti
xi ᶃ ∑N i=1 αi ti = 0 ᶄ ϥάϥϯδϡؔ = 1 2 wTw + C∑N i=1 ξi −∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi ᶆ ti (wT xi + b) − 1 + ξi ≥ 0 C − αi = μi ᶅ ᶇ ξi ≥ 0, αi ≥ 0, μi ≥ 0 ᶇ αi (ti (wT xi + b) − 1 + ξi ) = 0 ᶈ μi ξi = 0 ᶉ Ld (w, b, α) = 1 2 wTw − ∑N i=1 αi ti wT xi + ∑N i=1 αi ઢܗࣝผՄೳͳ߹ͷϥάϥϯδϡؔͱಉ͡ܗ͕ग़ͯ͘ΔͷͰʜ ιϑτϚʔδϯ47. -
ઢܗࣝผՄೳͰͳ͍47.࠷దԽʹ͓͚Δ,,5 ,BSVTI,VIO5VDLFS ݅ ͕ಘΒΕͨͷͰɼݩͷϥάϥϯδϡؔΛ͜ΕΒͷ݅Ͱஔ͖͑ͯΈΔͱʜ w0 = ∑N i=1 αi ti
xi ᶃ ∑N i=1 αi ti = 0 ᶄ ϥάϥϯδϡؔ = 1 2 wTw + C∑N i=1 ξi −∑N i=1 αi ti wT xi − b∑N i=1 αi ti + ∑N i=1 αi − ∑N i=1 αi ξi − ∑N i=1 μi ξi ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi − ∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi ᶆ ti (wT xi + b) − 1 + ξi ≥ 0 C − αi = μi ᶅ ᶇ ξi ≥ 0, αi ≥ 0, μi ≥ 0 ᶇ αi (ti (wT xi + b) − 1 + ξi ) = 0 ᶈ μi ξi = 0 ᶉ ઢܗࣝผՄೳͳ߹ͱશ͘ಉ͡มܗͰҎԼͷ͕ࣜಘΒΕΔ Ld (α) = − 1 2 αT Hα + αT1 ιϑτϚʔδϯ47. -
α w ͋ͱɹͷɹʹΑΔൣғࢦఆͷ݅ΛΓࠐΜͩରʹམͱ͠ࠐΜͰ 4.0ͱ͍͏ϥΠϒϥϦΛ༻͍ͯ࠷దԽΛղ͚ྑ͍ ධՁؔʢ࠷େԽʣ ੍݅ ର Ld (α) =
− 1 2 αT Hα + αT1 αT t = 0, C ≥ αi ≥ 0 αi C ιϑτϚʔδϯ47. -
ධՁؔʢ࠷େԽʣ ੍݅ ର Ld (α) = − 1 2 αT
Hα + αT1 αT t = 0, C ≥ αi ≥ 0 4.0Λ༻͍ͯରΛղ͖ɼ࣮ࡍʹɹΛશͯٻΊऴ͑ͨΒʜ α w0 = ∑N i=1 αi ti xi ઢܗࣝผՄೳͳͷͱಉ༷ʹओʹ͓͚Δ,,5݅ͷᶃΛ༻͍ͯɹΛٻΊʜ w0 ιϑτϚʔδϯ47. -
ධՁؔʢ࠷େԽʣ ੍݅ ର Ld (α) = − 1 2 αT
Hα + αT1 αT t = 0, C ≥ αi ≥ 0 ࣗ༝αϙʔτϕΫτϧ TW ͢ͳΘͪɹɹɹɹɹɹɹɹͱͳΔ ɹɹɹ͓ΑͼɹɹɹΛ༻͍Εɼ࠷దͳόΠΞεɹ ti (wT xi + b) − 1 = 0 xi = xsv ti = tsv b0 ti (wT xi + b) − 1 = 0 ti (wT xi + b) − 1 = 0 ti (wT xi + b) − 1 = 0 ti (wT xi + b) − 1 = 0 ti (wT 0 xi + b) − 1 = 0 tsv (wT 0 xsv + b0 ) − 1 = 0 b0 = 1 tsv − wT 0 xsv ˠྫ ͔ΒٻΊΔɽ ࣗ༝αϙʔτϕΫτϧΛ༻͍ͨཧ༝σʔλຖʹҟͳΔɹ͕ͳ͍͍ͨ͘͢Ί ɹɹɹɹɹɹɹɹɹɹͱॻ͚Δ্ݶαϙʔτϕΫτϧ͍ʹ͍͘ ξi ti (wT xi + b) − 1 + ξi = 0 4.0Λ༻͍ͯରΛղ͖ɼ࣮ࡍʹɹΛશͯٻΊऴ͑ͨΒʜ ιϑτϚʔδϯ47. -
ධՁؔʢ࠷େԽʣ ੍݅ ର Ld (α) = − 1 2 αT
Hα + αT1 αT t = 0, C ≥ αi ≥ 0 wT 0 w0 = ∑N i=1 ˜ αi ti xT i w0 4.0Λ༻͍ͯରΛղ͖ɼ࣮ࡍʹɹΛશͯٻΊऴ͑ͨΒʜ α ϥάϥϯδϡͷ࠷దղΛɹɹɹɹɹɹɹͷΑ͏ʹ͢Δͱ ˜ α = ( ˜ α1 , . . . , ˜ αN )T ,,5ᶃͷٯ ͱͯ͠ΔͷTW༝དྷͷͷ͚ͩͳͷͰ ˜ αi ti (wT 0 xi + b0 ) − 1 = 0 ͱͯ͠ܭࢉʜ = ∑N i=1 ˜ αi − b0 ∑N i=1 ti αi = ∑N i=1 ˜ αi (1 − ti b0 ) = ∑N i=1 ˜ αi ,,5ᶄ w0 = ∑N i=1 αi ti xi ∑N i=1 αi ti = 0 Dmax = 1 ∥w0 ∥ = 1 wT 0 w0 = 1 ∑N i=1 αi Αͬͯ࠷େϚʔδϯ ιϑτϚʔδϯ47. -
D D ιϑτϚʔδϯ47. ઢܗࣝผՄೳͰͳ͍߹ʹ࠷େϚʔδϯΛ ࣮ݱ͢Δઢܗࣝผฏ໘Λج४ʹ̎Ϋϥε ͷྨΛߦ͏ํ๏ ϥάϥϯδϡͷະఆ๏Λ༻͍ͯ ओΛରʹམͱ͠ࠐΜͰ ઢܗࣝผฏ໘ͷࣜΛٻΊΔ ιϑτϚʔδϯࣝผث
D D +ξi εϥοΫมΛಋೖͯؒ͠ҧͬͨͷ ؒҧ͍ͦ͏ͳͷΛTWʹԡ͠ࠐΉ ϖφϧςΟΛઃ͚ͨϥάϥϯδϡؔΛ࡞Δ ˜ Lp (w, b, α, ξ, μ) = 1 2 wTw + C∑N i=1 ξi −∑N i=1 αi (ti (wT xi + b) − 1 + ξi ) − ∑N i=1 μi ξi ·ͱΊ -
࠷େϚʔδϯΛ࣮ݱ͢Δઢܗࣝผฏ໘Λج४ʹ̎ΫϥεͷྨΛߦ͏ํ๏ ఆٛ ߴ࣍ݩຒΊࠐΈ ඇઢܗಛࣸ૾ ઢܗແཧͦ͏͚ͩͲɼͳΜ͔ͩڥքઢҾ͚ͦ͏ʜ ೋ࣍ݩ͔Βࡾ࣍ݩʹಛྔΛ૿͢ͱ ઢܗࣝผฏ໘Λॻ͚ͯͦ͏ʜʁ x1 x2 x2
1 + x2 2 x2 x1 ࠨਤͷத৺Λʹͨ͠ σʔλ·Ͱͷڑͷ࣠࡞Δ -
࠷େϚʔδϯΛ࣮ݱ͢Δઢܗࣝผฏ໘Λج४ʹ̎ΫϥεͷྨΛߦ͏ํ๏ ఆٛ ߴ࣍ݩຒΊࠐΈ ඇઢܗಛࣸ૾ Ͱ͜ͷߴ࣍ݩͷࣸ૾ʜ x1 x2 x2 1 +
x2 2 ݩͷਤΛ࣮ࡍʹ࡞ͬͯΈͯ σʔλͷͷํ͕Θ͔ͬͯͳ͍ͱ ͲΜͳ৽͍͠ಛΛ࡞Ε͍͍͔Θ͔Βͳ͍ʜ Ͳ͏͠Α͏ͳ͍͠ɼํͳ͍ͷͰ Ұ୴ͲΜͳಛྔΛ࡞Δ͔Ͳ͏͔ஔ͍ͱ͍ͯ ಛྔͷඇઢܗͷมͷҰൠతͳܗΛߟ͑Δʜ -
࠷େϚʔδϯΛ࣮ݱ͢Δઢܗࣝผฏ໘Λج४ʹ̎ΫϥεͷྨΛߦ͏ํ๏ ఆٛ ߴ࣍ݩຒΊࠐΈ ඇઢܗಛࣸ૾ ಛྔͷඇઢܗมͷҰൠతͳܗ x1 x2 x2 1 +
x2 2 x = (x1 x2 ) ඇઢܗม φ φ(x) = (φ0 (x) = 1, φ1 (x), . . . , φM (x))T = (1, x1 , x2 , x2 1 , x2 2 , x1 x2 , . . . )T ྫ -
࠷େϚʔδϯΛ࣮ݱ͢Δઢܗࣝผฏ໘Λج४ʹ̎ΫϥεͷྨΛߦ͏ํ๏ ఆٛ ߴ࣍ݩຒΊࠐΈ ඇઢܗಛࣸ૾ ಛྔͷඇઢܗมͷҰൠతͳܗ φ x = (x1 x2
) ⟹ φ(x) = (φ0 (x) = 1, φ1 (x), . . . , φM (x))T ಛྔΛूΊͨϕΫτϧͳͷͰ͜Ε·Ͱಉ༷ʹઢܗࣝผฏ໘ͷࣜ φ(x) w0 φ0 + w1 φ1 + w2 φ2 + . . . + wM φM = wTφ ͜͜ͰɼιϑτϚʔδϯ47.ʹ͓͚ΔରΛࢥ͍ग़͢ͱʜ max Ld (α) = − 1 2 ∑N i=1 ∑N j=1 αi αj ti tj xT i xj + ∑N i=1 αi s . t . αT t = 0, C ≥ αi ≥ 0 α -
ߴ࣍ݩຒΊࠐΈ ඇઢܗಛࣸ૾ ධՁؔʢ࠷େԽʣ ੍݅ ର αT t = 0, C
≥ αi ≥ 0 Ld (α) = − 1 2 ∑N i=1 ∑N j=1 αi αj ti tj xT i xj + ∑N i=1 αi σʔλɹɹ͕ιϑτϚʔδϯ47.ͰઢܗՄೳͰ͋ΔͱԾఆ͢Ε ্هͷධՁؔʹɹɹɹɹΛೖ͢Δ͜ͱ͕Ͱ͖ͯ xi = φ(xi ) Ld (α) = − 1 2 ∑N i=1 ∑N j=1 αi αj ti tj φ(xi )Tφ(xj ) + ∑N i=1 αi ͔͠͠ɼಛྔͷมͷํɹ͕Θ͔Βͳ͚Ε͜ͷؔ͑ͳ͍͜ͱʹͳΔʜ ͦ͜Ͱɼɹɹɹɹɹͷܗʹ͠ɼ͜ͷੵΛදݱͰ͖ΔΧʔωϧؔ ͱݺΕΔؔɹɹɹɹΛɹɹɹɹɹͱஔ͢Δ͜ͱͰಉ͡ܭࢉΛදݱ͢Δͱʜ φ(xi )Tφ(xj ) φ(xi )Tφ(xj ) K(xi , xj ) φ(x) φ Ld (α) = − 1 2 ∑N i=1 ∑N j=1 αi αj ti tj K(xi , xj ) + ∑N i=1 αi -
ߴ࣍ݩຒΊࠐΈ ඇઢܗಛࣸ૾ ධՁؔʢ࠷େԽʣ ੍݅ ର αT t = 0, C
≥ αi ≥ 0 Ld (α) = − 1 2 ∑N i=1 ∑N j=1 αi αj ti tj K(xi , xj ) + ∑N i=1 αi ্هͷΑ͏ͳϥάϥϯδϡؔΛ࡞Δ͜ͱ͕Ͱ͖Δ ۩ମతͳΧʔωϧؔʹҎԼͷΑ͏ͳͷ͕ڍ͛ΒΕΔ ଟ߲ࣜΧʔωϧ ಈܘجఈؔ Ψε Χʔωϧ Kp (xi , xj ) = (α + xT i xj )p Kσ (xi , xj ) = exp(− 1 2σ2 ∥xi − xj ∥2) -
ߴ࣍ݩຒΊࠐΈ ඇઢܗಛࣸ૾ ධՁؔʢ࠷େԽʣ ੍݅ ର αT t = 0, C
≥ αi ≥ 0 Ld (α) = − 1 2 ∑N i=1 ∑N j=1 αi αj ti tj K(xi , xj ) + ∑N i=1 αi ্هͷΑ͏ͳϥάϥϯδϡؔΛ࡞Δ͜ͱ͕Ͱ͖Δ ۩ମతͳΧʔωϧؔʹҎԼͷΑ͏ͳͷ͕ڍ͛ΒΕΔ ଟ߲ࣜΧʔωϧ ಈܘجఈؔ Ψε Χʔωϧ Kp (xi , xj ) = (α + xT i xj ) Kσ (xi , xj ) = exp(− 1 2σ2 ∥xi − xj ∥2) = exp(−γ∥xi − xj ∥2) -
ߴ࣍ݩຒΊࠐΈ ඇઢܗಛࣸ૾ ಈܘجఈؔ Ψε Χʔωϧ ݩੵͳͷͰσʔλಉ࢜ͷྨࣅ ࣅ͍ͯΕ ࣅ͍ͯͳ͚Ε Λ දݱ͍ͯ͠ΔΘ͚͕ͩɼͦΕΛσʔλಉ࢜ͷڑͰ࠶ݱͯ͠Δؔɽ
ɹਓؒͷखͰઃఆ͢ΔϋΠύʔύϥϝʔλͰ͋Δɽ Kσ (xi , xj ) = exp(− 1 2σ2 ∥xi − xj ∥2) = exp(−γ∥xi − xj ∥2) γ -
ߴ࣍ݩຒΊࠐΈ ඇઢܗಛࣸ૾ ಈܘجఈؔ Ψε Χʔωϧ Kσ (xi , xj )
= exp(− 1 2σ2 ∥xi − xj ∥2) = exp(−γ∥xi − xj ∥2) ΨεΧʔωϧΛల։͢Δͱʜ Kσ (xi , xj ) = exp(− 1 2σ2 ∥xi − xj ∥2) = exp(− 1 2σ2 (xi − xj )T(xi − xj )) = exp(− 1 2σ2 xT i xi − 1 2σ2 xT j xj + 1 σ2 xi xj )) = exp(− 1 2σ2 xT i xi )exp(− 1 2σ2 xT j xj )exp( 1 σ2 xT i xj ) = g(xi )g(xj )exp( K1 (xi , xj ) σ2 ) g(xi ) = exp(− 1 2σ2 xT i xi ) g(xj ) = exp(− 1 2σ2 xT j xj ) K1 (xi , xj ) = xT i xj -
ߴ࣍ݩຒΊࠐΈ ඇઢܗಛࣸ૾ ಈܘجఈؔ Ψε Χʔωϧ Kσ (xi , xj )
= exp(− 1 2σ2 ∥xi − xj ∥2) = exp(−γ∥xi − xj ∥2) ΨεΧʔωϧΛల։͢Δͱʜ Kσ (xi , xj ) = exp(− 1 2σ2 ∥xi − xj ∥2) = exp(− 1 2σ2 (xi − xj )T(xi − xj )) = exp(− 1 2σ2 xT i xi − 1 2σ2 xT j xj + 1 σ2 xi xj )) = exp(− 1 2σ2 xT i xi )exp(− 1 2σ2 xT j xj )exp( 1 σ2 xT i xj ) = g(xi )g(xj )exp( K1 (xi , xj ) σ2 ) exp( K1 (xi , xj ) σ2 ) ଟ߲ࣜΧʔωϧ෦ʹ͍ͭͯ ͏গ͠۷ΓԼ͛ͯΈΔ -
ߴ࣍ݩຒΊࠐΈ ඇઢܗಛࣸ૾ ಈܘجఈؔ Ψε Χʔωϧ Kσ (xi , xj )
= exp(− 1 2σ2 ∥xi − xj ∥2) = exp(−γ∥xi − xj ∥2) ΨεΧʔωϧͷଟ߲ࣜΧʔωϧ෦Λల։͢Δͱʜ exp( K1 (xi , xj ) σ2 ) = exp( xT i xj σ2 ) = exp( ∑d k=1 xik xjk σ2 ) = exp(∑d k=1 xik σ xjk σ ) = ∏d k=1 exp( xik σ xjk σ ) exp( xik σ xjk σ ) f(x) = 1 0! f(0) + 1 1! f′(0)x + 1 2! f′′(0)x2 + 1 3! f′′′(0)x3 + . . . = ∑∞ n=0 1 n! f(n)(0)xn Λ͏͢͜͠มܗ͢ΔͨΊʹϚΫϩʔϦϯల։Λಋೖ͢Δ ∑∞ n=0 1 n! f(n)(0)xn f(x) = exp(x) ͱ͢Δͱʜ exp(x) = 1 0! exp(0) + 1 1! exp(0)x + 1 2! exp(0)x2 + 1 3! exp(0)x3 + . . . = ∑∞ n=0 1 n! xn -
ߴ࣍ݩຒΊࠐΈ ඇઢܗಛࣸ૾ ಈܘجఈؔ Ψε Χʔωϧ Kσ (xi , xj )
= exp(− 1 2σ2 ∥xi − xj ∥2) = exp(−γ∥xi − xj ∥2) ΨεΧʔωϧͷଟ߲ࣜΧʔωϧ෦Λల։͢Δͱʜ exp( K1 (xi , xj ) σ2 ) = exp( xT i xj σ2 ) = exp( ∑d k=1 xik xjk σ2 ) = exp(∑d k=1 xik σ xjk σ ) = ∏d k=1 exp( xik σ xjk σ ) exp(x) = 1 0! exp(0) + 1 1! exp(0)x + 1 2! exp(0)x2 + 1 3! exp(0)x3 + . . . = ∑∞ n=0 1 n! xn exp( xik σ xjk σ ) = ∑∞ n=0 1 n! ( xik σ xjk σ )n = 1 + xik σ xjk σ + 1 2 ( xik σ xjk σ )2 + 1 6 ( xik σ xjk σ )3 + 1 24 ( xik σ xjk σ )4 + . . . = (1 xik σ 1 2 ( xik σ )2 1 6 ( xik σ )3 1 2 6 ( xik σ )4 . . . )(1 xjk σ 1 2 ( xjk σ )2 1 6 ( xjk σ )3 1 2 6 ( xjk σ )4 . . . )T = xT ik xjk ಛྔɼɹͱɹʹ༝དྷ͢Δ্ʹ࣍ݩ͕ɹ͔Βɹ·ͰͷແݶେͷੵʹͳΔɽ xi xj 0 ∞ (1 xik σ 1 2 ( xik σ )2 1 6 ( xik σ )3 1 2 6 ( xik σ )4 . . . )(1 xjk σ 1 2 ( xjk σ )2 1 6 ( xjk σ )3 1 2 6 ( xjk σ )4 . . . )T ແݶݸͷಛ -
ߴ࣍ݩຒΊࠐΈ ඇઢܗಛࣸ૾ ઢܗࣝผՄೳͰͳ͍߹ʹ࠷େϚʔδϯΛ ࣮ݱ͢Δઢܗࣝผฏ໘Λج४ʹ̎Ϋϥε ͷྨΛߦ͏ํ๏ ಈܘجఈؔΧʔωϧΛ༻͍Δͱ ແݶ࣍ݩͷಛϕΫτϧʹͳΔͨΊ ඞͣઢܗՄೳͳσʔλʹͳΔʁ Χʔωϧ๏ ߴ࣍ݩʹσʔλΛ֦ு͢Δ͜ͱͰ
ઢܗՄೳʹ͢Δ ΧʔωϧؔͰੵΛදݱͨ͠ ϥάϥϯδϡؔΛ࡞Δ ·ͱΊ x2 x1 x1 x2 x2 1 + x2 2 φ Ld (α) = − 1 2 ∑N i=1 ∑N j=1 αi αj ti tj K(xi xi ) + ∑N i=1 αi