目的関数と評価指標 / Objective Function and Evaluation Metrics

This presentation explains goodness of modeling in machine learning.
After that, it shows differences between Objective Function and Evaluation Metrics.

Transcript

1. ໨తؔ਺ͱධՁࢦඪ OBJECTIVE FUNCTION AND EVALUATION METRICS
2. ྑ͍Ϟσϧͬͯ ͳΜͰ͔͢ʁ
3. ͏͓͓͓͓͓͓͓ʂʂʂʂʂ
   ਫ਼౓ʂ·͞ʹ̔̕ɽ̕ˋʂ
   ͨ͗ΔʂԶͷମʂʂ
   ΄ͱ͹͠Εʂ3.4&ʂʂʂʂ AIΤϯδχΞ
4. それって現場で ちゃんと使えるの？ Ϗδωε୲౰ऀ
5. モデルの”良さ”は ⼈・場⾯によって 異なる
6. Ϟσϧͷ ྑ͞ ͱ͸
7. σʔλ෼ੳͷϑΣʔζʹΑͬͯҟͳΔ ”ྑ͞” ࣮૷ ▸ ໨తؔ਺ɿϞσϧͷֶशʹ͓͍ͯ࠷దԽ͞ΕΔؔ਺ɺ༧ଌ஋ͱ࣮ ଌ஋͕ဃ཭͠ͳ͍ͷ͕”ྑ͞”ʢRMSE΍LoglossͳͲʣ ▸ ධՁࢦඪɿϞσϧ΍༧ଌ஋ͷੑೳͷྑ͠ѱ͠ΛଌΔࢦඪɺ෼ੳ՝ ୊΁ͷޮՌͷߴ͕͞”ྑ͞”ʢRMSE΍AccuracyɺAUCͳͲʣ ▸ KPIɿϏδωε্Ͱͷ՝୊΁ͷޮՌͷߴ͕͞”ྑ͞” ▸ ROIɿಋೖඅ༻ʹର͢ΔϞσϧར༻ޮՌͷߴ͕͞”ྑ͞” Ϗδωε
8. ࣮૷Ͱ͸Ͳ͜·Ͱҙࣝ͢΂͖ʁ ▸ Ϟσϧ͸Ϗδωεͷݱ৔Ͱ࢖ΘΕΔ΋ͷͰ͋ΔͨΊɺROI΍KPIͱ͍ͬͨϏδωε্ ͷࢦඪΛϞσϧͷ࠷େԽࢦඪͱ͢Δͷ͕ཧ૝త ▸ ͔͠͠ɺͦΕ͸ݱ࣮తͰ͸ͳ͍ ▸ ͳͥͳΒɺϏδωε্ͷࢦඪ͸ϞσϧΛར༻ͯ͠ҙࢥܾఆͨ݁͠ՌͰ͋Δ͠ɺϞσ ϧͷ݁ՌΛ୭͔Ͳ͏ҙࢥܾఆͯ͠Ϗδωε্ͷ݁Ռ͕΋ͨΒ͞Ε͔ͨͱ͍ͬͨσʔ λ͸ଘࡏ͠ͳ͍ ▸ ͕ͨͬͯ͠ɺϞσϧߏஙͷϑΣʔζͰ͸ɺҙࢥܾఆͷ໾ʹཱͭ݁ՌΛग़͢ͱ͍͏ϙ Πϯτʹয఺ΛߜͬͯϞσϧͷྑ͞Λఆٛͨ͠΄͏͕ྑ͍ ▸ ͭ·ΓɺϞσϧߏஙͷࡍʹ͸ɺධՁࢦඪΛҙࣝ͢Δͷ͕ݱ࣮త
9. ໨తؔ਺ͱ ධՁࢦඪͷ ҧ͍
10. ໨తؔ਺ͱධՁࢦඪͷҧ͍ લड़ͷྑ͞ͷϨϕϧͷҧ͍ʹՃ͑ͯɺଟ͘ͷػցֶशϞσϧʹ͓͍ͯ͸ɺ ໨తؔ਺͸ඍ෼ՄೳͰͳ͚Ε͹ͳΒͳ͍ͷʹର͠ɺධՁࢦඪʹ͸੍໿͕ͳ͍ ▸ Q. ͳͥ໨తؔ਺͸ඍ෼ՄೳͰͳ͚Ε͹ͳΒͳ͍͔ ▸ A. ໨తؔ਺Λ࠷దԽ͢Δ͜ͱ͕ϞσϧΛ࡞Δ͜ͱͷҰ෦Ͱ͋Δ͔Β ͦͯ͠ɺ࠷దԽͱ͸ɺ͋Δ஋͕͍͍ײ͡ʹͳΔͱ͜ΖΛ୳͢ख๏ͷ͜ͱΛࢦ͔͢Β ͦͯͦͯ͠͠ɺ͍͍͔Μ͡ʹͳΔͱ͜ΖΛ୳͢ख๏Ͱ͸ɺۃ஋΍มԽ཰ͷܭࢉ͕ඞཁͱͳΔ
11. ࠷దԽख๏ͱඍ෼ ▸ ࠷খೋ৐๏ શ࣮ଌ஋ʹ͓͍ͯ༧ଌ஋ͱͷ࢒ࠩͷ૯࿨͕࠷খʹͳΔɺϞσϧࣜͷύϥϝʔλΛٻΊΔ ʹޡࠩؔ਺ͷۃখ஋Λ୳͢ͱ͍͏λεΫʹͳΔ ʹ֤ύϥϝʔλʹ͍ͭͯภඍ෼͠ɺ࿈ཱํఔࣜΛ࡞ͬͯղ͘ʢਖ਼نํఔࣜΛղ͘ʣ ʹඍ෼͠ͳ͚Ε͹ͳΒͳ͍ ▸ ࠷໬๏ શ࣮ଌ஋ʹ͓͍ͯͦΕΒͷ஋ΛऔΔ֬཰ͷ૬৐ʢ໬౓ʣ͕࠷େͱͳΔɺϞσϧࣜͷύϥϝʔλΛٻΊΔ ʹ໬౓ؔ਺ͷۃେ஋Λ୳͢ͱ͍͏λεΫʹͳΔ ʹ֤ύϥϝʔλʹ͍ͭͯภඍ෼͠ɺ࿈ཱํఔࣜΛ࡞ͬͯղ͘ ʹඍ෼͠ͳ͚Ε͹ͳΒͳ͍ ▸ ͦͷଞɺޯ഑߱Լ๏ɺ४χϡʔτϯ๏ͳͲ΋ಉ͡ ※ඍ෼Λ༻͍ͳ͍࠷దԽख๏΋͋Δ
12. ධՁࢦඪΛ ࠷దԽ͢Δࡍͷ Ξϓϩʔν
13. ධՁࢦඪΛ࠷దԽ͢ΔࡍͷΞϓϩʔν 1. ୯ʹਖ਼͘͠ϞσϦϯάΛߦ͏ ‣ ໨తؔ਺ͱධՁࢦඪʹಉ͡΋ͷΛࢦఆͰ͖Δ࣌ 2. ֶशσʔλͷલॲཧΛͯ͠ɺҟͳΔධՁࢦඪΛ࠷దԽ͢Δ ‣ ධՁࢦඪ͕RMSLEͷͱ͖ɺ໨తؔ਺ʹRMSEΛ࢖༻͠ɺ݁ՌΛࢦ਺ม׵͢ΔͳͲ 3. ҟͳΔධՁࢦඪͷ࠷దԽΛߦ͍ɺޙॲཧΛߦ͏ ‣ ධՁࢦඪͷੑ࣭ʹैͬͯσʔλΛॲཧͨ͠Γɺᮢ஋Λ࠷దԽ͢ΔͳͲ 4. ΧελϜ໨తؔ਺Λ࢖༻͢Δ ‣ ධՁࢦඪʹ͍ۙ໨తؔ਺Λ࢖༻͢Δ 5. ҟͳΔධՁࢦඪΛ࠷దԽ͠ɺΞʔϦʔετοϐϯάΛߦ͏
14. ۩ମྫᶃ 3. F1-score͕ධՁࢦඪͷ৔߹ͷɺᮢ஋ͷ࠷దԽ ▸ F1-score͸precisionͱrecallͷௐ࿨ฏۉͷͨΊɺਖ਼ྫෛྫͷෆۉߧσʔλͷ৔߹͸ɺ୯७ʹ0.5Λ ᮢ஋ͱ͠ͳ͍΄͏͕ྑ͍৔߹͕͋Δ ▸ ͜ͷ৔߹͸ɺᮢ஋Λ࠷దԽΞϧΰϦζϜΛ༻͍ͯ࠷దԽ͢Δඞཁ͕͋Δ
15. ۩ମྫᶄ 4. MAE͕ධՁࢦඪͷ৔߹ͷɺ໨తؔ਺ͷඍ෼Մೳͳۙࣅؔ਺ͷར༻ ▸ MAE͸ඍ෼ෆՄೳͰ͋ΔͨΊɺ໨తؔ਺ʹͦͷ··ར༻͢Δ͜ͱ͸Ͱ͖ͳ͍ ▸ ͜ͷ৔߹ɺඍ෼ՄೳͳMAEͷۙࣅؔ਺Λ༻͍ΔࣄʹΑͬͯɺධՁࢦඪʹରͯ͠ΑΓ࠷దԽͰ͖Δ Մೳੑ͕͋Δ
16. ࢀߟจݙ ▸ ໳࿬େีɼࡕాོ࢘ɼอࡔܡ༎ɼฏদ༤࢘ ʢ̎̌̍̕ʣ, KaggleͰউͭσʔλ෼ ੳͷٕज़, ٕज़ධ࿦ࣾ, ୈೋষ λεΫͱධՁࢦඪ ▸ Datarobot blog Ϟσϧ࠷దԽࢦඪ-ධՁࢦඪͷબͼํ ▸ https://www.datarobot.com/jp/blog/Ϟσϧ࠷దԽࢦඪ-ධՁࢦඪͷબͼํ/