Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ゾンビでわかる分類評価指標

F0e950c6bf20815f3d169f063d3c5d4f?s=47 Aipa
March 09, 2019

 ゾンビでわかる分類評価指標

PythonBeginners沖縄(ゼロからDeep1&DjangoでWebアプリ)@琉球大学 で発表してきたLT

https://python-beginners-okinawa.connpass.com/event/121317/

F0e950c6bf20815f3d169f063d3c5d4f?s=128

Aipa

March 09, 2019
Tweet

More Decks by Aipa

Other Decks in Technology

Transcript

  1. Python Beginnersԭೄ LT ΞΠύʔୂ௕

  2. ࣗݾ঺հ

  3. ࣗݾ঺հ • ΞΠύʔୂ௕(ࠓ೥ 29) • ͪΎΒσʔλגࣜձࣾ • PyData.OkinawaڞಉΦʔΨφΠβʔ • Pythonͱո्ͱΫιΈ͍ͨͳөը͕޷͖Ͱ͢

  4. ࠂ஌

  5. PyData.Okinawa #38 LTେձ

  6. ঎඼͕͋Γ·͢ʂʂʢશһ౤ථʣ

  7. ͳΜͱࢀՃऀ͕গͳ͍ʢདྷͯ͘ΕཔΉʣ

  8. ຊ೔͓࿩͢Δ͜ͱ

  9. κϯϏͰΘ͔Δ෼ྨධՁࢦඪ

  10. ͜ͷલΈͨөը

  11. ͜ͷલΈͨөը ৽ײછ

  12. ͜ͷલΈͨөը ৽ײછ ˺৽װઢ

  13. Կ͜ͷ๜୊ɾɾɾ https://twitter.com/geri_otto_mrs/status/811131042723598336

  14. Կ͜ͷ๜୊ɾɾɾ өը͸໘ന͔ͬͨͷͰ҆৺͍ͯͩ͘͠͞

  15. ͯ͞ɺ͜ͷөըͷ͝঺հ ฼͕͍Δ佂ࢁʹ͍͖͍ͨ෕ͱ່͕ ৐͍ͬͯΔ৽װઢͷ৐٬Ұਓ͕ ಥવκϯϏԽͯ͠ɺӺһΛऻ͏ ऻΘΕͨӺһɾ৐٬͸਺ඵޙκϯϏʹͳΓ ػ಺͸ύϯσϛοΫɾେύχοΫ Ռͨͯ͠ɺ෕ͱ່͸佂ࢁʹ͍Δ฼ͷݩʹ ແࣄʹͨͲΓண͚Δͷ͔ ݂ΈͲΖͳඳࣸͰ৺Թ·ΔɺؖࠃκϯϏөը

  16. ͯ͞ɺ͜ͷөըͷ͝঺հ ฼͕͍Δ佂ࢁʹ͍͖͍ͨ෕ͱ່͕ ৐͍ͬͯΔ৽װઢͷ৐٬Ұਓ͕ ಥવκϯϏԽͯ͠ɺӺһΛऻ͏ ऻΘΕͨӺһɾ৐٬͸਺ඵޙκϯϏʹͳΓ ػ಺͸ύϯσϛοΫɾେύχοΫ Ռͨͯ͠ɺ෕ͱ່͸佂ࢁʹ͍Δ฼ͷݩʹ ແࣄʹͨͲΓண͚Δͷ͔ ݂ΈͲΖͳඳࣸͰ৺Թ·ΔɺؖࠃκϯϏөը

  17. ύϯσϛοΫ ΋͠ݱ࣮ͩͬͨΒɾɾɾ

  18. ύϯσϛοΫ ΋͠ݱ࣮ͩͬͨΒɾɾɾ ւΛഎʹ

  19. ύϯσϛοΫ ΋͠ݱ࣮ͩͬͨΒɾɾɾ ւΛഎʹόϦέʔυ࡞ͬͯɺ

  20. ύϯσϛοΫ ΋͠ݱ࣮ͩͬͨΒɾɾɾ ւΛഎʹόϦέʔυ࡞ͬͯɺདྷΔ΋ͷશһ΍ͬͯ͠·͑͹ྑ͍

  21. ύϯσϛοΫ ΋͠ݱ࣮ͩͬͨΒɾɾɾ ւΛഎʹόϦέʔυ࡞ͬͯɺདྷΔ΋ͷશһ΍ͬͯ͠·͑͹ྑ͍

  22. ͍͚·ͤΜ

  23. ؒҧ͑ͯੜଘऀΛࣹࡴͯ͠͠·͏έʔε κϯϏөը͋Δ͋Δ

  24. ؒҧ͑ͯੜଘऀΛࣹࡴͯ͠͠·͏έʔε κϯϏөը͋Δ͋Δ

  25. ؒҧ͑ͯੜଘऀΛࣹࡴͯ͠͠·͏έʔε κϯϏөը͋Δ͋Δ

  26. Α͘ͳ͍

  27. ෼ྨ͠·͠ΐ͏ κϯϏΛ1ɺੜଘऀΛ0ͱ͍͏ϥϕϧΛషΓ෇͚Δ 0 0 1 1 1

  28. 1Λ΍Δ ෼ྨ͠·͠ΐ͏

  29. ʢͻͱΓؒҧ͑ͯ΍ͬͯ͠·ͬͨͬΆ͍ͧɾɾɾʣ ෼ྨͨ݁͠ՌΛ֬ೝ͠·͠ΐ͏

  30. ධՁࢦඪ

  31. ࠞಉߦྻʢconfusion matrixʣ • ෼ྨ໰୊ͷ݁ՌͷੑೳΛ໌Β͔ʹ͢Δߦྻ • ਅཅੑ(true positive → TP) •

    ਅӄੑ(true negative → TN) • ِཅੑ(false positive → FP) • ِӄੑ(false negative → FN)
  32. ࠞಉߦྻʢconfusion matrixʣ ༧ଌ͞ΕͨΫϥε ࣮ࡍͷΫϥε 1 / 1 51 '/ /

    '1 5/
  33. κϯϏ͕50ਓ(?) ੜଘऀ͕50໊͖ͨ ༧ଌ͞ΕͨΫϥε ࣮ࡍͷΫϥε κϯϏͱ༧ଌ κϯϏ ͡Όͳ͍ͱ ༧ଌ κϯϏ 

    51  '/ κϯϏ ͡Όͳ͍  '1  5/
  34. ׬ᘳʹ෼ྨͰ͖ΔͱͲ͏ͳΔʁ ༧ଌ͞ΕͨΫϥε ࣮ࡍͷΫϥε κϯϏͱ༧ଌ κϯϏ ͡Όͳ͍ͱ ༧ଌ κϯϏ  51

     '/ κϯϏ ͡Όͳ͍  '1  5/
  35. ධՁࢦඪ

  36. ਖ਼ղ཰ʢAccuracyʣ • ༧ଌͨ݁͠Ռͱ࣮ࡍͷ஋͕Ұகׂͨ͠߹ • ʢTP + TNʣ / ʢTP +

    TN + FP + FNʣ • FPͱFN͕0ͳΒ1.0ʹͳΔʢ100%ʣ
  37. ʢ࠶ܝʣκϯϏ͕50ਓ(?) ੜଘऀ͕50໊͖ͨ ༧ଌ͞ΕͨΫϥε ࣮ࡍͷΫϥε κϯϏͱ༧ଌ κϯϏ ͡Όͳ͍ͱ ༧ଌ κϯϏ 

    51  '/ κϯϏ ͡Όͳ͍  '1  5/ • (27 + 45) / (27 + 45 + 5 + 23) = 72% • ਖ਼ղ཰͸72%
  38. Α͠ʂ72%ͷਫ਼౓ͩʂ

  39. ͱ͜Ζ͕͗ͬͪΐΜ

  40. ੈք͕ฏ࿨ʹͳ͖ͬͯͨ → κϯϏ͕গͳ͘ͳ͖ͬͯͨ

  41. κϯϏ͕10ਓ(?) ੜଘऀ͕90໊͖ͨ ༧ଌ͞ΕͨΫϥε ࣮ࡍͷΫϥε κϯϏͱ༧ଌ κϯϏ ͡Όͳ͍ͱ ༧ଌ κϯϏ 

    51  '/ κϯϏ ͡Όͳ͍  '1  5/ • (0 + 90) / (0 + 90 + 0 + 10) = 90% • ਖ਼ղ཰͸90%
  42. Α͠ʂ90%ͷਫ਼౓ͩʂ ҆৺Ͱ͖Δʂʂ

  43. κϯϏ͕10ਓ(?) ੜଘऀ͕90໊͖ͨ ༧ଌ͞ΕͨΫϥε ࣮ࡍͷΫϥε κϯϏͱ༧ଌ κϯϏ ͡Όͳ͍ͱ ༧ଌ κϯϏ 

    51  '/ κϯϏ ͡Όͳ͍  '1  5/ • (0 + 90) / (0 + 90 + 0 + 10) = 90% • ਖ਼ղ཰͸90% ͓Θ͔Γ͍͚ͨͩͨͩΖ͏͔
  44. None
  45. κϯϏ 10ਓ ༧ଌʹࣦഊ͍ͯ͠Δʂʂʂ ʢ৵ೖ͞Εͨʣ

  46. κϯϏ 10ਓ ༧ଌʹࣦഊ͍ͯ͠Δʂʂʂ ʢ৵ೖ͞Εͨʣ

  47. Accuracyͷ໰୊఺ • Ξϯόϥϯεʢෆۉߧʣͳσʔλʹ͸దͯ͠ ͍ͳ͍ • গͳ͍σʔλͷ༧ଌ͕શ͋ͨ͘Βͳͯ͘΋ɺ ׂ߹͸ߴ͘ͳΔ܏޲ʹͳΔ • TPɺTNɺFPɺFN͕ἧΘͳ͍ͱࢉग़Ͱ͖ͳ͍

  48. κϯϏΛඞͣݟ෼͚ΔϞσϧ or κϯϏΛඞͣಀ͞ͳ͍Ϟσϧ • લऀ͸ద߹཰ʢPrecisionʣ • ޙऀ͸࠶ݱ཰ʢRecallʣ • ͜ͷ̎ͭΛ͍͍ײ͡ʹ
 ௐ੔ͨ͠ͷ͕F1-scoreʢF஋ʣ

  49. ద߹཰ʢPrecisionʣ • Ϟσϧ͕κϯϏͱ༧ଌͨ݁͠ՌʹɺͲΕ͚ͩ κϯϏؚ͕·Ε͍͔ͯͨ • TP / (TP + FP)

    • 0 / (0 + 0) = 0% ༧ଌ͞ΕͨΫϥε ࣮ࡍͷΫϥε κϯϏͱ༧ଌ κϯϏ ͡Όͳ͍ͱ ༧ଌ κϯϏ  51  '/ κϯϏ ͡Όͳ͍  '1  5/
  50. ద߹཰ʢPrecisionʣ • ؒҧ͑ͯੜଘऀΛ΍ͬͯ͠·͍ͨ͘ͳ͍ͳΒɺ PrecisionΛਫ਼౓ͱ͢Ε͹ྑ͍ • Precision 100%ͳΒੜଘऀΛܸͪؒҧ͑Δ
 ͜ͱ͸ͳ͍ • ݟۃΊతͳࢦඪ

  51. ࠶ݱ཰ʢRecallʣ ༧ଌ͞ΕͨΫϥε ࣮ࡍͷΫϥε κϯϏͱ༧ଌ κϯϏ ͡Όͳ͍ͱ ༧ଌ κϯϏ  51

     '/ κϯϏ ͡Όͳ͍  '1  5/ • κϯϏʹର͠Ϟσϧ͕κϯϏͱ൑அׂͨ͠߹ • TP / (TP + FN) • 0 / (0 + 10) = 0%
  52. ࠶ݱ཰ʢRecallʣ • όϦέʔυલ·ͰκϯϏΛ౸ୡͤͨ͘͞ͳ͚ Ε͹ɺશһ΍ͬͯ͠·͑͹ྑ͍ʢήεإʣ • Recall 100%ͳΒκϯϏΛಀ͠࿙Ε͕ͳ͍ • ໢ཏతͳࢦඪ

  53. F஋ʢF1-scoreʣ • PrecisionͱRecallΛ૊Έ߹Θͤͯௐ੔ͨ͠஋ • 2 * Precision * Recall /

    Precision + Recall • 2 * 0 * 0 / 0 + 0 = 0% ༧ଌ͞ΕͨΫϥε ࣮ࡍͷΫϥε κϯϏͱ༧ଌ κϯϏ ͡Όͳ͍ͱ ༧ଌ κϯϏ  51  '/ κϯϏ ͡Όͳ͍  '1  5/
  54. ࡶײ • ෼ྨͷධՁࢦඪ͸ࠞಉߦྻΛݩʹࢉग़͞ΕΔ • AccuracyͰ͸͏·͘ධՁͰ͖ͳ͍৔߹͕͋Δ • ϒϩάͱ͔࿦จʹ͋Δʮਫ਼౓ʯ͸ԿΛࢦ͍ͯ͠Δͷ ͔ҙࣝͯ͠ಡΉͱ͍͍͔΋ • ͜Ε͕Θ͔Ε͹ROCۂઢɺAUCͱ͔ཧղͰ͖Δ

  55. None