機械学習勉強会06 ロジスティック回帰/MLStudy06

機械学習勉強会06 ロジスティック回帰

Transcript

1. ϩδεςΟοΫճ ؼ ֬཰Λ࢖ͬͨ෼ྨ @hachiilcane
2. લճ·Ͱ͸ ύʔηϓτϩϯͱ͍͏෼ྨΞϧΰϦζϜ ͷجૅΛֶΜͩ
3. ࠓճֶͿ಺༰͸ ෼ྨΞϧΰϦζϜͱͯ͠ύʔηϓτϩϯΑΓ΋൚༻ ੑ͕ߴ͍ɺϩδεςΟοΫճؼΛֶͿʢʮճؼʯͬ ໊ͯલ͕͍͍ͭͯΔ͕ɺҰൠతʹ͸ʮ෼ྨʯʹ࢖Θ Ε·͢ʣ ֬཰ͷߟ͑ํΛऔΓࠐΜͰ͍ΔͨΊɺʮ͜ͷݕ਍݁ ՌͳΒපؾͰ͋Δ֬཰͸80ˋʯͱ͍͏ݴ͍ํ͕Ͱ͖ Δ γάϞΠυؔ਺ͱ͍͏ศརͳ΋ͷʹ׳ΕΔ
4. લճͱಉ͡໰୊Λߟ͑Δ ۙ೥ɺʮ͍͵͞Μͷ͜ͱ͕޷͖͗ͯ͢පʯʹ͔͔Δਓ͕૿͑ ͯࣾձ໰୊ʹͳ͍ͬͯ·͢ɻ͜ͷපؾʹ͔͔ͬͨਓ͸ಛผͳ ࣏ྍ͕ඞཁʹͳΓ·͕͢ɺපؾʹ͔͔͔ͬͨͲ͏͔ͷ൑ఆ͸ ೉͍͜͠ͱ͕Θ͔͍ͬͯ·͢ ஍ಓͳݚڀͷ݁Ռɺʮ͍͵͞Μͷը૾Λݟͤͨ࣌ͷ৺ഥ਺ͷ ૿ݮʯͱʮ͍͵͞Μάοζॴ༗਺ͷҰൠਓฏۉͱͷࠩʯ͕ප ؾͷ൑ఆʹେ͖͘໾ཱͭ͜ͱ͕Θ͔͖ͬͯ·ͨ͠ ʮ৺ഥ਺ͷ૿ݮʯͱʮάοζॴ༗਺ͷࠩʯΛܭଌ͢Δ͜ͱʹ Αͬͯɺͦͷਓ͕පؾʹ͔͔͍ͬͯΔ͔൱͔ΛҰ࣍൑ఆͨ͠ ͍Ͱ͢
5. ࠓճ͸t͸+1·ͨ͸-1Ͱ ͸ͳ͘ɺ1·ͨ͸0ͱ͢Δ x ʢ৺ഥ਺ ૿ݮʣ y ʢάοζ ਺ࠩʣ t ʢ1:පؾ 0:͡Όͳ͍ʣ 0 5.604765 -0.837603 0 1 5.093028 -1.098183 0 2 -2.595448 1.348614 0 3 -0.662749 -5.056531 0 4 15.573566 10.073330 1 5 7.084038 2.165339 0 6 6.204333 -2.945187 0 7 13.349965 12.577250 1 8 16.487809 6.629031 1 9 0.060047 -2.900301 0 τϨʔχϯάηοτ50ݸத ͷઌ಄10ݸͷ஋ පؾάϧʔϓ t=1ͱ͢Δ පؾ͡Όͳ͍ άϧʔϓ t=0ͱ͢Δ
6. લճͱಉ͡Α͏ʹɺ෼ׂ ͢Δ௚ઢΛఆٛ͢Δ (x,y)ฏ໘্ͷσʔλΛ෼ྨ ͢Δ௚ઢΛ਺ࣜͰදݱ͢ Δ ͜ͷ࣌ɺ(x,y)ฏ໘Λ෼ׂ͢ Δ௚ઢ͸࣍ࣜͰද͞ΕΔ f(x, y) = w0 + w1 x + w2 y f(x, y) = 0 ͬͪ͜පؾ ͬͪ͜පؾ ͡Όͳ͍
7. ڥքઢ͔Β཭ΕΔ΄Ͳf(x,y)ͷ ઈର஋͸େ͖͘ͳΔͷͩͬͨ f(x,y)Ͱ෼ׂ͞ΕΔྖҬ͸ɺf(x,y)ͷූ߸Ͱ൑ผͰ͖Δ ڥքઢ͔Β཭ΕΔ΄Ͳɺf(x,y)ͷઈର஋͕େ͖͘ͳΔ f(x, y) > 0 f(x, y) < 0 f(x, y) = 0 (15,20) (2,4) ∟ f(x, y) = 49 f(x, y) = − 22
8. f(x,y)ฏ໘্ͷ೚ҙͷ఺ʹ͓ ͍ͯɺt=1Ͱ͋Δ֬཰͸ʁ f(x, y) > 0 f(x, y) < 0 ∟ ෼ׂઢͷӈ্ͷํ ޲ʹ཭ΕΔ΄Ͳt=1 Ͱ͋Δ֬཰͸ߴ͍ ͸ͣͩ ෼ׂઢ্ͳΒt=1Ͱ ͋Δ֬཰͸0.5 ෼ׂઢͷࠨԼͷํ ޲ʹ཭ΕΔ΄Ͳt=1 Ͱ͋Δ֬཰͸௿͍
9. t=1Ͱ͋Δ֬཰Λਤʹͯ͠ ΈΔͱ͜Μͳײ͡ʹͳΔ σʔλͷଐੑ͕ t=1Ͱ͋Δ֬཰ f(x, y) f(x,y)=0ͳΒ ʢ෼ׂઢ্ͳ Βʣt=1Ͱ͋Δ ֬཰͸0.5
10. ͜Ε͸γάϞΠυؔ਺Ͱ දݱͰ͖Δ aͷ஋Λ-∞͔Β∞ʹมԽͤ͞ΔͱɺМ(a) ͷ஋͸0͔Β1΁׈Β͔ʹมԽ͢Δ γάϞΠυؔ਺ɿσ(a) = 1 1 + e−a
11. γάϞΠυؔ਺ͷಛ௃ σ(a)ͷaͷͱ͜Ζʹ௚ઢͷࣜΛಥͬࠐΉ ͱɺ௚ઢͷେ͖͍ਖ਼ͷ஋͸1ʹ͍ۙ஋ ʹɺઈର஋ͷେ͖͍ෛͷ஋͸0ʹ͍ۙ஋ ʹม׵͞ΕΔ ௚ઢ͕0͔Β1ͷൣғʹ௵͞ΕΔΑ͏ͳ Πϝʔδɻ͜ΕΛਖ਼نԽͱݴ͏ɻ
12. ෼ׂઢf(x,y)Λ௚ઢͷ··Ͱ ͸ͳ͘МͰแΉͱ͍͏͜ͱ ௚ઢϞσϧʹγάϞΠυؔ਺Λ͔·͠ ͨͷ͕ϩδεςΟοΫճؼϞσϧ 0͔Β1ͷ஋ΛͱΔΑ͏ʹͳΔͷͰ֬཰ Λද͢ͷʹศརͩ͠ɺεςοϓؔ਺Α Γ΋ඍ෼͠΍͍͢
13. ͕ͨͬͯ͠ࠓճ͸Ϟσϧ Λ͜͏ఆٛ͢Δ ఺ (x,y)ͰಘΒΕͨσʔλͷଐੑ͕t=1Ͱ͋Δ֬཰͸࣍ࣜͰ ද͞ΕΔ ൓ରʹɺt=0Ͱ͋Δ֬཰͸࣍ࣜʹͳΔ ͜Ε͕ҰൠతͳύϥϝτϦοΫϞσϧͷ3ͭͷεςοϓͷ͏ ͪͷεςοϓ̍ɺʮύϥϝʔλΛؚΉϞσϧʢ਺ࣜʣΛઃ ఆ͢Δʯ P(x, y) = σ(w0 + w1 x + w2 y) 1 − P(x, y)
14. Ϟσϧ͸ύʔηϓτϩϯ ͱͲ͏ҧ͏ʁ ύʔηϓτϩϯ ௚ઢͰ۠੾Δ f(x,y)ͷ஋ͷਖ਼ෛͰපؾ͔පؾͰͳ͍͔ͷ۠ผʢೋ୒ʣ ͭ·Γ۠੾Γ͕֊ஈঢ়ͷεςοϓؔ਺ ϩδεςΟοΫճؼ ௚ઢͰ۠੾Δͱ͍͏ߟ͑ํ͸ࠜͬ͜͸ಉ͡ P(x,y)ͷ஋ʢ0ʙ1ͷ஋ʣ͕පؾͰ͋Δ֬཰Λࣔ͢ ͭ·Γ۠੾Γ͕ͳΊΒͳۂઢ
15. εςοϓ2ɿύϥϝʔλ ͷධՁج४ΛఆΊΔ ύʔηϓτϩϯͰ͸ਖ਼͘͠෼ྨͰ͖ͳ ͔ͬͨ఺Λޡࠩͱͯ͠ߟ͑ͯޡࠩؔ਺Λ ϕʔεʹ͕ͨ͠…… ϩδεςΟοΫճؼͰ͸໬౓ؔ਺Λ༻ ͍ͯɺ࠷໬ਪఆ๏Ͱ࠷దͳύϥϝʔλΛ ٻΊΔ
16. ໬౓ͱ͸ ʮ࠷΋Β͍͠౓߹͍ʯͱ͍͏ҙຯ ౜ಥʹɺʮτϨʔχϯάηοτ͕ಘΒΕΔ֬཰ʯΛߟ͑ ͯΈΔ ࣮ࡍʹ͸τϨʔχϯάηοτ͸͢ͰʹಘΒΕ͍ͯΔ ͷͰɺޙ෇͚Ͱ֬཰Λߟ͍͑ͯΔΑ͏ͳ΋ͷ ྫ͑͹ɺαΠίϩΛ2ճ߱ͬͯ1ͷͧΖ໨͕ग़ͨͱ͠ ͯɺͦΕΛޙ͔ΒʮͲΕ͘Β͍௝͍͜͠ͱ͕ى͖ͨ ͷͩΖ͏ʁʯͱߟ͍͑ͯΔΑ͏ͳΠϝʔδ
17. ·ͣಛఆͷ1ͭͷσʔλ Ͱߟ͑Δ̍ ̍ͭͷσʔλɹɹɹɹɹ͕ಘΒΕΔ֬཰ ਺ֶతٕ޼Λ༻͍ͯ̍ͭͷࣜʹ͢Δͱ ͜ΕʹϞσϧͷࣜΛ୅ೖ͢Δͱ tn = 1ͷ৔߹ɿP(xn , yn ) tn = 0ͷ৔߹ɿ1 − P(xn , yn ) Pn = σ(w0 + w1 xn + w2 yn )tn(1 − σ(w0 + w1 xn + w2 yn ))1−tn (xn , yn , tn ) Pn = P(xn , yn )tn(1 − P(xn , yn ))1−tn
18. ·ͣಛఆͷ1ͭͷσʔλ Ͱߟ͑Δ̎ ѻ͍΍͘͢͢ΔͨΊʹɺϞσϧͷࣜ෦෼Λzͱஔ͖׵͑Δ ͜͜Ͱzn͸࣍ࣜͰఆٛͨ͠΋ͷͰɺʮn൪໨ͷσʔλͷଐੑ ͕t=1Ͱ͋Δ֬཰ʯΛද͢ wͱП͸ύʔηϓτϩϯͰఆٛͨ͠΋ͷͱಉ͡ Pn = ztn n (1 − zn )1−tn zn = σ(wTϕn ) w = w0 w1 w2 ϕn = ( 1 xn yn ) όΠΞε߲
19. Ͱ͸τϨʔχϯάηοτʹؚ· ΕΔσʔλΛ·ͱΊͯߟ͑Δ શ෦ͷσʔλΛ·ͱΊͯߟ͑Δͱɺ͜ΕΒͷσʔλ͕ ಘΒΕΔ֬཰͸ɺ֤σʔλ͕ಘΒΕΔ֬཰ͷੵʹͳΔ ͜ͷ֬཰͸ύϥϝʔλwʹΑͬͯ஋͕มΘΔͷͰɺύ ϥϝʔλwͷؔ਺ΛݟΔ͜ͱ͕Ͱ͖Δ ʮτϨʔχϯάηοτͷσʔλ͕ಘΒΕΔ֬཰ʯΛύ ϥϝʔλͷؔ਺ͱΈͳͨ͠΋ͷΛ໬౓ؔ਺ͱݺͿ P = N ∏ n=1 Pn = N ∏ n=1 ztn n (1 − zn )1−tn
20. ʮ͓લͷ࣍ͷηϦϑ͸ʯ ʮ؍ଌ͞ΕͨσʔλʢτϨʔχϯάηοτʣ͸ɺ࠷ ΋ൃੜ֬཰͕ߴ͍σʔλʹҧ͍ͳ͍ʯ ͜ͷԾઆ͕ਖ਼͍͠΋ͷͱͯ͠ɺ֬཰P͕࠷େʹͳΔ Α͏ʹύϥϝʔλΛܾఆ͢Δख๏Λ࠷໬ਪఆ๏ͱݴ ͏ ʮτϨʔχϯάηοτͱͯ͠ɺൃੜ֬཰͕௿͍ ʢͱͯ΋௝͍͠ʣσʔλ͕ಘΒΕΔ΄Ͳࣗ෼͸ ӡ͕͍͍͸͕ͣͳ͍ʯͱߟ͍͑ͯΔΑ͏ͳ΋ͷ
21. ۩ମతʹͲ͏΍ͬͯ֬཰PΛ࠷େԽ ͢ΔύϥϝʔλΛݟ͚ͭΔ͔ʢε ςοϓ̏ͷํ๏ʣ ͦͷ̍ɿχϡʔτϯɾϥϑιϯ๏Λ༻͍ ͨΞϧΰϦζϜ IRLS๏ʢ൓෮࠶ॏΈ෇ ͚࠷খೋ৐๏ʣΛ࢖͏ํ๏ ͦͷ̎ɿલճಉ༷ʹภඍ෼ͯ͠ޯ഑๏ ʹ࣋ͪࠐΉํ๏
22. ʢࢀߟʣχϡʔτϯɾϥ ϑιϯ๏Λ༻͍ͨIRLS๏ αϯϓϧϓϩάϥϜ05-logistic_vs_perceptronͰ࢖༻ ͍ͯ͠ΔͷͰ঺հɻҎԼͷΑ͏ͳΞϧΰϦζϜɻ ͜ΕΛ܁Γฦ͢ͱ֬཰Pͷ஋͕େ͖͘ͳ͍͖ͬͯɺ࠷ ऴతʹ͸࠷େ஋ʹୡ͢Δɻ͜Ε͕IRLS๏ɻ wnew = wold − (ΦTRΦ)−1ΦT(z − t) z = z1 ⋮ zN R = diag[z1 (1 − zn ), …, zN (1 − zN )] t = t1 ⋮ tN Φn = 1 x1 y1 1 x2 y2 ⋮ ⋮ ⋮ 1 xN yN ର֯ߦྻ
23. ޯ഑๏Ͱ࠷దͳύϥϝʔλ ΛٻΊΔ͜ͱΛ໨ࢦͯ͠ ҎԼͷྲྀΕ ໬౓ؔ਺ΛฏۉަࠩΤϯτϩϐʔޡࠩؔ਺ͱͯ͠࠶ ఆٛ͢Δ ޡࠩؔ਺Λภඍ෼ֶͯ͠शଇʹ࢖͏ภඍ෼ΛٻΊΔ ޯ഑๏Ͱগͣͭ͠ύϥϝʔλwΛม͍͖͑ͯʢޯ഑ ϕΫτϧͷ൓ରํ޲ͷࡔΛԼ͍͖ͬͯʣɺ࠷దͳύ ϥϝʔλwΛٻΊΔ
24. ฏۉަࠩΤϯτϩϐʔޡ ࠩ ໬౓ؔ਺͸ੵͷܗΛ͍ͯ͠ΔͷͰɺѻ͍΍͘͢͢ΔͨΊʹର਺ ʢlogͷ͜ͱʣΛͱΔ ର਺ؔ਺͸୯ௐ૿Ճؔ਺Ͱ͋ΔͨΊɺର਺Λͱͬͯ΋໬౓ؔ ਺Λ࠷େԽ͢Δύϥϝʔλ͸มΘΒͳ͍ ͞ΒʹϚΠφε̍Λֻ͚ͯɺNͰׂΔɻ͜ΕΛฏۉަࠩΤϯτϩ ϐʔޡࠩͱͯ͠ఆٛ͢Δɻ͜ͷޡࠩؔ਺Λ࠷খԽ͢Δύϥϝʔλ wΛٻΊΔ͜ͱʹͳΔɻ͜Ε͕εςοϓ̎ͷ໨తؔ਺ʹ૬౰ɻ log P = N ∑ n=1 {tn log zn + (1 − tn )log(1 − zn )} E(w) = − 1 N log P = − 1 N N ∑ n=1 {tn log zn + (1 − tn )log(1 − zn )}
25. ฏۉަࠩΤϯτϩϐʔޡ ࠩؔ਺E(w)Λภඍ෼͢Δ εςοϓ̏ͱͯ͠ޯ഑๏Ͱ࠷ྑͷධՁΛ༩͑ΔύϥϝʔλΛܾఆ͢Δ͜ͱʹ͢Δ ಋग़͸লུ ֶ͕ͨͬͯ͠शଇ͸ ∂E ∂w0 = 1 N N ∑ n=1 (zn − tn ) ∂E ∂w1 = 1 N N ∑ n=1 (zn − tn )xn ∂E ∂w2 = 1 N N ∑ n=1 (zn − tn )yn w0 := w0 − ∂E ∂w0 w1 := w1 − ∂E ∂w1 w2 := w2 − ∂E ∂w2
26. ޯ഑๏ͷ࣮૷ͰָΛ͢Δ scipy.optimizeϥΠϒϥϦʹؚ·ΕΔ minimize()ͱ͍͏ؔ਺Ͱޯ഑๏͕ߦ͑Δ ֶश཰Λ಺෦Ͱࣗಈతʹઃఆͯ͘͠Ε ΔͷͰศར ܭࢉճ਺΋ͳΜ͔উखʹ΍ͬͯ͘ΕΔ
27. scipy.optimizeͷ minimize()ͷ࢖͍ํώϯτ ୈ̍Ҿ਺ʹ͸໨తؔ਺ʢࠓճͩͱฏۉަࠩΤϯτϩϐʔޡࠩؔ਺ʣΛࢦఆ͢Δ ୈ̎Ҿ਺ʹ͸ύϥϝʔλͷॳظ஋Λ഑ྻͰ౉͢ argsʹ͸໨తؔ਺ʹ౉͢Ҿ਺ͷ͏ͪɺw_initͰ౉͍ͯ͠ͳ͍෺ΛλϓϧͰ౉͢ jacʹ͸໨తؔ਺Λภඍ෼ͨؔ͠਺Λ౉͢ɻҾ਺͸໨తؔ਺ͱಉ͡Ͱɺ໭Γ஋͸ޯ഑ϕΫτϧʢ[w0ͷ ภඍ෼஋, w1ͷภඍ෼஋, w2ͷภඍ෼஋]Έ͍ͨͳײ͡ʣΛฦؔ͢਺ methodʹ͸࠷దԽख๏Λࢦఆ͢Δɻ”CG”͸ڞ໾ޯ഑๏ɻ minimize()ͷ໭Γ஋ͷதͷxϓϩύςΟʹ͸ɺ࠷దԽ͞Εͨύϥϝʔλ͕ೖ͍ͬͯΔ from scipy.optimize import minimize def fit_logistic(w_init, train_set): res = minimize(cee_logistic, w_init, args=(train_set), jac=dcee_logistic, method="CG") return res.x
28. 05-logistic_vs_perceptron.ipynbΛ ϕʔεʹͯ͠՝୊ʹνϟϨϯδͯ͠ ΈΑ͏ z࣠Λt=1Ͱ͋Δ֬཰ʢP(x,y)ͷ஋ʣͱͯ͠ɺڥք໘Λ3DϓϩοτͰදݱͯ͠Έ Δ ฏۉަࠩΤϯτϩϐʔޡࠩؔ਺ͷภඍ෼ΛखͰ΍ͬͯΈΔ ͜Ε͸݁ߏ೉͍͠Ͱ͢ɻ࿈࠯཯ɺlogͷඍ෼ͷެࣜɺγάϞΠυؔ਺ͷඍ෼ ͷެࣜͳͲΛۦ࢖͠·͠ΐ͏ ޯ഑๏Λ࣮૷ͯ͠ΈΔ ͱΓ͋͑ͣɺϩδεςΟοΫճؼϞσϧΛؔ਺ͱ࣮ͯ͠૷ͯ͠ΈΔͱ͜Ζ ͔Β࢝ΊΔͱ͍͍͔΋ɻdef logistic(w, x, y):Έ͍ͨͳײ͡Ͱɻ ៉ྷʹઢܗ෼཭Ͱ͖ͳ͍ࠞͬͨ͟ײ͡ͷτϨʔχϯάηοτΛ༻ҙͯ͠ɺIRLS ๏·ͨ͸ޯ഑๏Ͱ෼཭ͯ͠ΈΔ
29. ࢀߟจݙ தҪ ӻ࢘ʮITΤϯδχΞͷͨΊͷػց ֶशཧ࿦ೖ໳ʯٕज़ධ࿦ࣾ, 2015 ҏ౻ ਅʮPythonͰಈֶ͔ͯ͠Ϳʂ͋ͨ Β͍͠ػցֶशͷڭՊॻʯᠳӭࣾ, 2018