Share
Embed
Transcript
29 slides
-
ࠓճt+1·ͨ-1Ͱ ͳ͘ɺ1·ͨ0ͱ͢Δ x ʢ৺ഥ ૿ݮʣ y ʢάοζ ࠩʣ t ʢ1:පؾ
0:͡Όͳ͍ʣ 0 5.604765 -0.837603 0 1 5.093028 -1.098183 0 2 -2.595448 1.348614 0 3 -0.662749 -5.056531 0 4 15.573566 10.073330 1 5 7.084038 2.165339 0 6 6.204333 -2.945187 0 7 13.349965 12.577250 1 8 16.487809 6.629031 1 9 0.060047 -2.900301 0 τϨʔχϯάηοτ50ݸத ͷઌ಄10ݸͷ පؾάϧʔϓ t=1ͱ͢Δ පؾ͡Όͳ͍ άϧʔϓ t=0ͱ͢Δ -
લճͱಉ͡Α͏ʹɺׂ ͢ΔઢΛఆٛ͢Δ (x,y)ฏ໘্ͷσʔλΛྨ ͢ΔઢΛࣜͰදݱ͢ Δ ͜ͷ࣌ɺ(x,y)ฏ໘Λׂ͢ Δઢ࣍ࣜͰද͞ΕΔ f(x, y) =
w0 + w1 x + w2 y f(x, y) = 0 ͬͪ͜පؾ ͬͪ͜පؾ ͡Όͳ͍ -
ڥքઢ͔ΒΕΔ΄Ͳf(x,y)ͷ ઈରେ͖͘ͳΔͷͩͬͨ f(x,y)Ͱׂ͞ΕΔྖҬɺf(x,y)ͷූ߸ͰผͰ͖Δ ڥքઢ͔ΒΕΔ΄Ͳɺf(x,y)ͷઈର͕େ͖͘ͳΔ f(x, y) > 0 f(x, y)
< 0 f(x, y) = 0 (15,20) (2,4) ∟ f(x, y) = 49 f(x, y) = − 22 -
f(x,y)ฏ໘্ͷҙͷʹ͓ ͍ͯɺt=1Ͱ͋Δ֬ʁ f(x, y) > 0 f(x, y) < 0
∟ ׂઢͷӈ্ͷํ ʹΕΔ΄Ͳt=1 Ͱ͋Δ֬ߴ͍ ͣͩ ׂઢ্ͳΒt=1Ͱ ͋Δ֬0.5 ׂઢͷࠨԼͷํ ʹΕΔ΄Ͳt=1 Ͱ͋Δ͍֬ -
͕ͨͬͯ͠ࠓճϞσϧ Λ͜͏ఆٛ͢Δ (x,y)ͰಘΒΕͨσʔλͷଐੑ͕t=1Ͱ͋Δ֬࣍ࣜͰ ද͞ΕΔ ରʹɺt=0Ͱ͋Δ֬࣍ࣜʹͳΔ ͜Ε͕ҰൠతͳύϥϝτϦοΫϞσϧͷ3ͭͷεςοϓͷ͏ ͪͷεςοϓ̍ɺʮύϥϝʔλΛؚΉϞσϧʢࣜʣΛઃ ఆ͢Δʯ P(x,
y) = σ(w0 + w1 x + w2 y) 1 − P(x, y) -
·ͣಛఆͷ1ͭͷσʔλ Ͱߟ͑Δ̍ ̍ͭͷσʔλɹɹɹɹɹ͕ಘΒΕΔ֬ ֶతٕΛ༻͍ͯ̍ͭͷࣜʹ͢Δͱ ͜ΕʹϞσϧͷࣜΛೖ͢Δͱ tn = 1ͷ߹ɿP(xn , yn
) tn = 0ͷ߹ɿ1 − P(xn , yn ) Pn = σ(w0 + w1 xn + w2 yn )tn(1 − σ(w0 + w1 xn + w2 yn ))1−tn (xn , yn , tn ) Pn = P(xn , yn )tn(1 − P(xn , yn ))1−tn -
·ͣಛఆͷ1ͭͷσʔλ Ͱߟ͑Δ̎ ѻ͍͘͢͢ΔͨΊʹɺϞσϧͷࣜ෦Λzͱஔ͖͑Δ ͜͜Ͱzn࣍ࣜͰఆٛͨ͠ͷͰɺʮn൪ͷσʔλͷଐੑ ͕t=1Ͱ͋Δ֬ʯΛද͢ wͱПύʔηϓτϩϯͰఆٛͨ͠ͷͱಉ͡ Pn = ztn n
(1 − zn )1−tn zn = σ(wTϕn ) w = w0 w1 w2 ϕn = ( 1 xn yn ) όΠΞε߲ -
ʢࢀߟʣχϡʔτϯɾϥ ϑιϯ๏Λ༻͍ͨIRLS๏ αϯϓϧϓϩάϥϜ05-logistic_vs_perceptronͰ༻ ͍ͯ͠ΔͷͰհɻҎԼͷΑ͏ͳΞϧΰϦζϜɻ ͜ΕΛ܁Γฦ͢ͱ֬Pͷ͕େ͖͘ͳ͍͖ͬͯɺ࠷ ऴతʹ࠷େʹୡ͢Δɻ͜Ε͕IRLS๏ɻ wnew = wold −
(ΦTRΦ)−1ΦT(z − t) z = z1 ⋮ zN R = diag[z1 (1 − zn ), …, zN (1 − zN )] t = t1 ⋮ tN Φn = 1 x1 y1 1 x2 y2 ⋮ ⋮ ⋮ 1 xN yN ର֯ߦྻ -
ฏۉަࠩΤϯτϩϐʔޡ ࠩ ؔੵͷܗΛ͍ͯ͠ΔͷͰɺѻ͍͘͢͢ΔͨΊʹର ʢlogͷ͜ͱʣΛͱΔ ରؔ୯ௐ૿ՃؔͰ͋ΔͨΊɺରΛͱͬͯؔ Λ࠷େԽ͢ΔύϥϝʔλมΘΒͳ͍ ͞ΒʹϚΠφε̍Λֻ͚ͯɺNͰׂΔɻ͜ΕΛฏۉަࠩΤϯτϩ ϐʔޡࠩͱͯ͠ఆٛ͢Δɻ͜ͷޡࠩؔΛ࠷খԽ͢Δύϥϝʔλ wΛٻΊΔ͜ͱʹͳΔɻ͜Ε͕εςοϓ̎ͷతؔʹ૬ɻ log
P = N ∑ n=1 {tn log zn + (1 − tn )log(1 − zn )} E(w) = − 1 N log P = − 1 N N ∑ n=1 {tn log zn + (1 − tn )log(1 − zn )} -
ฏۉަࠩΤϯτϩϐʔޡ ࠩؔE(w)Λภඍ͢Δ εςοϓ̏ͱͯ͠ޯ๏Ͱ࠷ྑͷධՁΛ༩͑ΔύϥϝʔλΛܾఆ͢Δ͜ͱʹ͢Δ ಋग़লུ ֶ͕ͨͬͯ͠शଇ ∂E ∂w0 = 1 N
N ∑ n=1 (zn − tn ) ∂E ∂w1 = 1 N N ∑ n=1 (zn − tn )xn ∂E ∂w2 = 1 N N ∑ n=1 (zn − tn )yn w0 := w0 − ∂E ∂w0 w1 := w1 − ∂E ∂w1 w2 := w2 − ∂E ∂w2 -
scipy.optimizeͷ minimize()ͷ͍ํώϯτ ୈ̍ҾʹతؔʢࠓճͩͱฏۉަࠩΤϯτϩϐʔޡࠩؔʣΛࢦఆ͢Δ ୈ̎ҾʹύϥϝʔλͷॳظΛྻͰ͢ argsʹతؔʹ͢Ҿͷ͏ͪɺw_initͰ͍ͯ͠ͳ͍ΛλϓϧͰ͢ jacʹతؔΛภඍͨؔ͠Λ͢ɻҾతؔͱಉ͡ͰɺΓޯϕΫτϧʢ[w0ͷ ภඍ, w1ͷภඍ, w2ͷภඍ]Έ͍ͨͳײ͡ʣΛฦؔ͢ methodʹ࠷దԽख๏Λࢦఆ͢Δɻ”CG”ڞޯ๏ɻ
minimize()ͷΓͷதͷxϓϩύςΟʹɺ࠷దԽ͞Εͨύϥϝʔλ͕ೖ͍ͬͯΔ from scipy.optimize import minimize def fit_logistic(w_init, train_set): res = minimize(cee_logistic, w_init, args=(train_set), jac=dcee_logistic, method="CG") return res.x -
05-logistic_vs_perceptron.ipynbΛ ϕʔεʹͯ͠՝ʹνϟϨϯδͯ͠ ΈΑ͏ z࣠Λt=1Ͱ͋Δ֬ʢP(x,y)ͷʣͱͯ͠ɺڥք໘Λ3DϓϩοτͰදݱͯ͠Έ Δ ฏۉަࠩΤϯτϩϐʔޡࠩؔͷภඍΛखͰͬͯΈΔ ͜Ε݁ߏ͍͠Ͱ͢ɻ࿈ɺlogͷඍͷެࣜɺγάϞΠυؔͷඍ ͷެࣜͳͲΛۦ͠·͠ΐ͏ ޯ๏Λ࣮ͯ͠ΈΔ ͱΓ͋͑ͣɺϩδεςΟοΫճؼϞσϧΛؔͱ࣮ͯͯ͠͠ΈΔͱ͜Ζ
͔Β࢝ΊΔͱ͍͍͔ɻdef logistic(w, x, y):Έ͍ͨͳײ͡Ͱɻ ៉ྷʹઢܗͰ͖ͳ͍ࠞͬͨ͟ײ͡ͷτϨʔχϯάηοτΛ༻ҙͯ͠ɺIRLS ๏·ͨޯ๏Ͱͯ͠ΈΔ