機械学習勉強会07 ROC曲線/MLStudy07

機械学習勉強会07 ROC曲線

Transcript

1. ROCۂઢ ֶशϞσϧͷධՁํ๏ @hachiilcane
2. લճ͸ ϩδεςΟοΫճؼʹΑͬͯɺฏ໘্ͷ֤఺(x,y)ʹ͓͚Δ ʮͦ͜ͰಘΒΕΔσʔλ͕t=1Ͱ͋Δ֬཰ʯΛߟ͑Δ͜ͱ Ͱɺ෼ׂઢΛܾఆͨ͠ ࠷ऴతʹf(x,y)=0Ͱ༩͑ΒΕΔ෼ׂઢ͸ɺ֬཰͕ͪΐ͏Ͳ1/2 Ͱ͋Δ఺ʹରԠ͢Δ
3. ࠓճ͸ ෼ྨ໰୊Ͱ֬཰1/2Ͱͳ͍ͱ͜Ζʹ൑அ ج४Λஔ͖͍ͨ৔߹ɺROCۂઢΛར༻ ͢Δ͜ͱΛֶͿ ਅཅੑ཰ɾِཅੑ཰͸෼ྨ໰୊ʹ͓͚ ΔϞσϧͷධՁࢦඪͱͯ͠΋࢖͑Δ͜ͱ ΛֶͿ
4. ෼ׂઢ͸֬཰1/2Ͱྑ͍ ͔ʁ පؾͷਫ਼ີݕࠪΛड͚Δʗड͚ͳ͍ͷ൑அʹ࢖͏ͳΒɺ ΋ͬͱݫ͠໨ʹ͍ͨ͠ͷͰ͋͑ͯ֬཰20ˋΛ෼ׂઢʹ͢ Δɺͱ͔͋Δ ԿΛج४ʹܾΊͨΒ͍͍ͩΖ͏ʁ ֬཰50% ֬཰80% ֬཰20%
5. ཅੑɺӄੑͱ͍͏ݴ༿ t=1Ͱ͋ΔσʔλΛཅੑͱݺͿ t=1Ͱ͋Δ֬཰ʢපؾͰ͋Δ֬཰ʣΛ ཅੑͰ͋Δ֬཰ͱ΋ݴ͑Δ
6. ਅཅੑɺِཅੑ ਅཅੑʢTP: True Positiveʣ ཅੑͩͱ൑அͨ͠σʔλʹ͍ͭͯɺͦΕ͕ຊ౰ʹཅੑͩͬͨ΋ͷ ِཅੑʢFP: False Positiveʣ ཅੑͩͱ൑அͨ͠σʔλʹ͍ͭͯɺͦΕ͕࣮͸ӄੑͩͬͨ΋ͷ ෼ྨ݁Ռ͕ཅੑ ෼ྨ݁Ռ͕ӄੑ ࣮ࡍ͸ཅੑ ʢt=1ʣ TP: True Positive ʢਅཅੑʣ FN: False Negative ࣮ࡍ͸ӄੑ ʢt=0ʣ FP: False Positive ʢِཅੑʣ TN: True Negative
7. ͯ͞Ͳ͏ͳΔͱྑ͍͔ ਅཅੑͷ֬཰͸Ͱ͖Δ͚ͩߴͯ͘͠ɺِ ཅੑͷ֬཰͸Ͱ͖Δ͚ͩ௿͍ͨ͘͠ ͔͠͠ݱ࣮ʹ͸ਅཅੑ཰ͱِཅੑ཰ ͸τϨʔυΦϑ
8. τϨʔυΦϑͰ͋Δ͜ͱΛτ ϨʔχϯάηοτͰߟ͑ͯΈΔ ཅੑͱ൑அ͢Δج४Λ ֬཰P͕͍ͭ͘Ҏ্ͱ͢ Δ͔ʢͲ͜ʹઢΛҾ͘ ͔ʣͰɺਅཅੑ཰ͱِ ཅੑ཰͸มΘ͍ͬͯ͘ No. x y type probability 17 3.112981 18.365505 1 0.974877 5 9.565425 14.490813 1 0.970444 4 16.518876 10.829925 1 0.968636 15 4.683143 14.311496 1 0.947828 0 -2.179448 17.027400 1 0.940079 19 6.697921 11.162725 1 0.916815 1 -4.503331 10.331480 0 0.720126 7 3.096324 5.841293 0 0.682496 11 -4.251525 8.810845 1 0.650768 8 12.351163 -2.448305 1 0.473750 13 0.135944 2.193228 1 0.398745 9 2.832298 0.529935 0 0.379728 18 12.742703 -6.120525 0 0.287259 12 -6.851055 2.156915 1 0.230266 6 -8.006545 2.658170 0 0.227698 16 4.473311 -8.369764 0 0.086250 2 -3.540915 -7.057080 0 0.049229 14 -1.728843 -8.918192 0 0.039690 10 -14.774465 -3.970796 0 0.028895 3 -4.767995 -9.256136 0 0.026421 ͜͜ʹઢΛҾ͚͹ਅཅੑ཰͸6/10ɺ ِཅੑ཰͸0͕ͩ……
9. ROCۂઢʢReceiver Operating Characteristicۂઢʣ ൑அج४ͷมߋͰਅཅੑ཰ͱِཅ ੑ཰͕ͲͷΑ͏ʹมԽ͢Δ͔Λද ͨ͠ͷ͕ROCۂઢ ͜ΕΛݟͳ͕ΒͲ͜ʹ൑அج ४ΛҾ͔͘Λߟ͑Δͱ͍͍ ͜Ε͸ΧΫΧΫ͍ͯ͠Δ͚Ͳɺτ Ϩʔχϯάηοτ͕͋Δఔ౓ଟ͚ Ε͹ࠨ্ʹுΓग़ͨ͠ۂઢʹͳΔ ROCۂઢ ϩδεςΟοΫճؼʹΑΔ෼ׂ
10. ͞ΒʹROCۂઢʹ͍ͭͯ ߟ࡯͢Δ ࠨ্ʹுΓ෇͍͍ͯΔͳΒͦΕ͸ཧ૝ͷ൑ఆํ๏ ࠨԼΛج४ʹऔΔͳΒશͯΛӄੑͱ൑அ͢Δɺӈ ্Λج४ʹऔΔͳΒશͯΛཅੑͱ൑அ͢Δʢແ ஌ʣ ର֯ઢ্Ͱ͋Ε͹ɺҰఆ֬཰ͰϥϯμϜʹཅੑͱ ൑அ͢Δʢແ஌ʣ ӈԼͷํʹுΓग़͍ͯ͠ΔͳΒɺҙਤతʹޡͬͨ ൑அΛ͢Δʢѱҙʣ ݁ہɺਅཅੑ཰ͱِཅੑ཰ͷͲͪΒ͔͚ͩΛݟͯ ͍ͯ΋ΞϧΰϦζϜͷྑ͠ѱ͠͸Θ͔Βͳ͍ ROCۂઢ ϩδεςΟοΫճؼʹΑΔ෼ׂ
11. ෼ྨ໰୊ʹ͓͚ΔֶशϞσϧ ͷධՁࢦඪʹ΋࢖͑Δ਺஋ ਖ਼ղ཰ʢ·ͨ͸ਫ਼౓ʣʢaccuracyʣ ෼ྨ݁Ռ͕ཅੑ ෼ྨ݁Ռ͕ӄੑ ࣮ࡍ͸ཅੑ TP FN ࣮ࡍ͸ӄੑ FP TN Accuracy = TP+TN TP+FP+FN+TN શମͷσʔλͷதͰਖ਼ ͘͠෼ྨͰ͖ͨσʔλ ͸ͲΕ͘Β͍͔ Ͱ΋ྫ͑͹100ݸͷσʔληοτͷ͏ͪ95ݸ͕ཅੑͩͬͨ ͱ͢Δͱɺશ෦ཅੑͱ൑அ͢ΔֶशϞσϧʹͪ͠Ό͑͹ Accuracy͸0.95ʹͳͬͪΌ͏ͷͰɺAccuracy͚ͩͰࢦඪ͸ े෼ͱ͸ݴ͑ͳ͍ΜͩΑͶ…
12. ଞʹ΋ࢦඪ͸͋Δ ద߹཰ʢprecisionʣɺ࠶ݱ཰ʢrecallʣ Precision = TP TP+FP Recall = TP TP+FN ෼ྨ݁Ռ͕ཅੑ ෼ྨ݁Ռ͕ӄੑ ࣮ࡍ͸ཅੑ TP FN ࣮ࡍ͸ӄੑ FP TN ʢPositiveΛج४ʹߟ͑ͨͱ͖ʹʣ ͲΕ͘Β͍ؒҧͬͨ෼ྨ͕গͳ͍͔ ʢPositiveΛج४ʹߟ͑ͨͱ͖ʹʣ ͲΕ͘Β͍औΓ͜΅͠ͳ͘ਖ਼͘͠෼ ྨ͍ͯ͠Δ͔ Ͱ΋Ұൠతʹ͸ͲͪΒ͔͕ߴ͘ͳΔͱͲͪΒ͔͕ ௿͘ͳΔΑ͏ͳτϨʔυΦϑͷؔ܎ʹ͋ΔͷͰɺ ͜ΕΒ୯ಠͰݟͯ΋े෼ͱ͸ݴ͑ͳ͍ΜͩΑͶ…
13. ΋͏গ͠૯߹తͳੑೳΛ ਤΔࢦඪ F1஋ʢFmeasureʣ ଞʹ΋ࢦඪ͸͋Δ͕ɺ݁ہԿΛͲͷΑ ͏ʹ࢖͏͔͸ࣗ෼࣍ୈ Fmeasure = 2 1 Precision + 1 Recall = 2 ⋅ Precision ⋅ Recall Precision + Recall ਺ֶతʹ͸ద߹཰ͱ࠶ݱ཰ ͷௐ࿨ฏۉͷ஋ɻόϥϯε ͕͍͍ͱߴ͍஋ʹͳΔ
14. ՝୊ʹνϟϨϯδͯ͠Έ Α͏ 05-roc_curve.ipynbΛ࣮ߦͯ͠ΈΑ͏ 05-roc_curve.ipynbΛ࣮ߦ͢Δͱ͍͔ͭ͘ܯࠂ͕ग़ ΔͷͰ௚ͯ͠ΈΑ͏ ୯७ʹద߹཰(Precision)ͱ࠶ݱ཰(Recall)ͷฏۉΛ ͱͬͨͷͰ͸͋·Γྑ͘ͳ͍ྫΛߟ͑ͯΈΑ͏ Accuracy, Precision, Recall, F1ͷ஋Λࢉग़ͯ͠ΈΑ͏
15. ࢀߟจݙ தҪ ӻ࢘ʮITΤϯδχΞͷͨΊͷػցֶश ཧ࿦ೖ໳ʯٕज़ධ࿦ࣾ, 2015 ҏ౻ ਅʮPythonͰಈֶ͔ͯ͠Ϳʂ͋ͨΒ͠ ͍ػցֶशͷڭՊॻʯᠳӭࣾ, 2018 ཱੴݡޗʮ΍ֶ͘͞͠Ϳ ػցֶशΛཧղ͢ ΔͨΊͷ਺ֶͷ͖΄Μ ʯϚΠφϏग़൛, 2017