Slide 1

Slide 1 text

Evaluation for classification dataitgirls3 Instructor Sunmi Yoon

Slide 2

Slide 2 text

Confusion Matrix

Slide 3

Slide 3 text

https://sumniya.tistory.com/26

Slide 4

Slide 4 text

Evaluation Metrics from Confusion Matrix

Slide 5

Slide 5 text

https://towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62

Slide 6

Slide 6 text

Precision(੿޻ب), PPV(Positive Predictive Value) ݽ؛੉ TrueۄҊ ࠙ܨೠ Ѫ ઺ী, पઁ Trueੋ Ѫ੄ ࠺ਯ Recall(੤അਯ), Sensitivity, hit rate पઁ True ઺ী ݽ؛੉ True۽ ࠙ܨೠ ࠺ਯ “Precision݅ न҃ਸ ॳݶ ݽ؛੉ ੋ࢝೧૑Ҋ, Recall݅ न҃ॳݶ ݽ؛੉ ೻ಌ૓׮” ੄ ੄޷ܳ ࢤп೧ࠁࣁਃ.

Slide 7

Slide 7 text

Accuracy TP, TNਸ ݽف Ҋ۰ೞח ૑಴. Label ࠛӐഋ੉ बೡ ٸী ࢎਊਸ ઱੄೧ঠ ೤פ׮. F1 Score Precisionҗ Recall੄ ઑചಣӐ Label ࠛӐഋ੉ बೡ ٸী ݽ؛੄ ࢿמਸ ੿ഛೞѱ ಣоೡ ࣻ ੓णפ׮. Label ࠛӐഋ੉ बೡ ٸী, Accuracyח ૑಴۽ࢲ न܉ࢿਸ ੏णפ׮. ੉ਬܳ ࢤп ೧ ࠁࣁਃ.

Slide 8

Slide 8 text

https://sumniya.tistory.com/26 ৵ ࢑ࣿಣӐ੉ ইפҊ ઑചಣӐੋо?

Slide 9

Slide 9 text

ઑӘ݅ ؊ о ࠇद׮

Slide 10

Slide 10 text

https://towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62 ׮द ੉ Ӓܿਵ۽ جই৬ࢲ, ଘ ফܳ ઺बਵ۽ ࢤп೮਺

Slide 11

Slide 11 text

https://towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62 ׮द ੉ Ӓܿਵ۽ جই৬ࢲ, ੉ߣূ ফب э੉ ࢤпೞݶࢲ ࠇद׮ (૑Әࠗఠ ഁтܾ ࣻ ੓਺)

Slide 12

Slide 12 text

੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN

Slide 13

Slide 13 text

੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN Precision Positive Predictive Value ࠙ܨ Ѿҗ(ݽ؛)ਸ ઺बਵ۽

Slide 14

Slide 14 text

੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN Negative Predictive Value ࠙ܨ Ѿҗ(ݽ؛)ਸ ઺बਵ۽

Slide 15

Slide 15 text

੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN Recall Sensitivity True Positive Rate ੿׹ਸ ઺बਵ۽

Slide 16

Slide 16 text

੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN ੿׹ਸ ઺बਵ۽ False Positive Rate

Slide 17

Slide 17 text

੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN ੿׹ਸ ઺बਵ۽ Specificity True Negative Rate

Slide 18

Slide 18 text

੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN ੿׹ਸ ઺बਵ۽ Fall-out rate False Positive Rate

Slide 19

Slide 19 text

https://towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62 ੉Ѧ ೞ੗Ҋ ೮঻ભ. ੉ߣূ ফب э੉ ࢤпೞݶࢲ ࠇद׮ (૑Әࠗఠ ഁтܾ ࣻ ੓਺)

Slide 20

Slide 20 text

੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN ? TP ઺ब ૑಴ٜ ੿ܻೞ੗ݶ, ?

Slide 21

Slide 21 text

੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN TN ઺ब ૑಴ٜ ? ੿ܻೞ੗ݶ, ?

Slide 22

Slide 22 text

ഁтܻભ? ਗې Ӓ۠Ѣ৘ਃ

Slide 23

Slide 23 text

ӝୡח ೮ਵפө ઑӘ݅ ؊ ೧ ࠇद׮.

Slide 24

Slide 24 text

Confusion Matrix with Histogram

Slide 25

Slide 25 text

https://www.medcalc.org/manual/roc-curves.php Criterion, Threshold য়ܲଃ Distribution਷ Actual True, ৽ଃ਷ Actual False. Threshold ਤ۽ח ݽف True۽ ৘ஏೞח ݽ؛੉ ੓׮Ҋ о੿೮ਸ ٸ,

Slide 26

Slide 26 text

https://www.medcalc.org/manual/roc-curves.php Thresholdܳ ӓױ੸ਵ۽ ਋ஏ ੉ز दெࠇद׮. যڃ ੌ੉ ੌযաաਃ? Precision: Recall: Specificity: Fall-out:

Slide 27

Slide 27 text

https://www.medcalc.org/manual/roc-curves.php Thresholdܳ ӓױ੸ਵ۽ ਋ஏ ੉ز दெࠇद׮. যڃ ੌ੉ ੌযաաਃ? True positive rate: True negative rate:

Slide 28

Slide 28 text

https://www.medcalc.org/manual/roc-curves.php ੉ߣূ ߈؀۽ ੉ز दெࠇद׮. যڃ ੌ੉ ੌযաաਃ? True positive rate: True negative rate:

Slide 29

Slide 29 text

Specificity৬ Sensitivity੄ ҙ҅ https://www.medcalc.org/manual/roc-curves.php

Slide 30

Slide 30 text

ROC(Receiver Operating Characteristic) curve

Slide 31

Slide 31 text

рױೞѱח, Sensitivity৬ 1-Specificityܳ п ୷ਵ۽ ೞח 2ରਗ Ӓې೐ https://www.medcalc.org/manual/roc-curves.php AUC (Area Under Curve)

Slide 32

Slide 32 text

рױೞѱח, Sensitivity৬ 1-Specificityܳ п ୷ਵ۽ ೞח 2ରਗ Ӓې೐ https://www.medcalc.org/manual/roc-curves.php Actual True৬ Actual False distribution੉ ৮߷ೞѱ эਸ ٸ (feature੄ class ߸߹מ۱ হ਺) ROC curveח 45ب пب ૒ࢶ

Slide 33

Slide 33 text

рױೞѱח, Sensitivity৬ 1-Specificityܳ п ୷ਵ۽ ೞח 2ରਗ Ӓې೐ https://www.medcalc.org/manual/roc-curves.php Actual True৬ Actual False distribution੉ Ҁ஖ח ৔৉ হ੉ ৮߷ೞѱ ܻ࠙ ؼ ٸ ROC ழ࠳ (feature੄ class ߸߹ מ۱੉ ৮߷) ROC ழ࠳о ઝ࢚ױী оө਎ࣻ۾ feature੄ class ߸߹ מ۱੉ જ׮Ҋ ೡ ࣻ ੓׮.

Slide 34

Slide 34 text

ROC(Receiver Operating Characteristic) curve with Machine Learning

Slide 35

Slide 35 text

Classifierܳ ݅ٚ׮ח Ѥ, ف ѐ੄ histogramਸ ӒܻҊ Thresholdܳ ੿੄ೞח Ѫ https://www.medcalc.org/manual/roc-curves.php

Slide 36

Slide 36 text

https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html#sphx-glr-auto-examples-model-selection-plot-roc-py Histogramਸ Ӓ۷׮ח Ѥ ROC ழ࠳ܳ Ӓܾ ࣻ ੓׮ח Ѫ!

Slide 37

Slide 37 text

https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html#sphx-glr-auto-examples-model-selection-plot-roc-py ROC ழ࠳ܳ Ӓܾ ࣻ ੓׮ח Ѥ ৈ۞ ROC ழ࠳ р ࠺Үܳ ా೧ જ਷ ࢿמ੄ ݽ؛ਸ ଺ইյ ࣻ ੓׮ח Ѫ!

Slide 38

Slide 38 text

AUCо ௼׮ = ݽ؛੉ ҅࢑ೠ probabilityܳ ߄ఔਵ۽ Ӓܽ histogramٜ੉ ੜ ܻ࠙غয ੓׮. = ݽ؛੉ Threshold(Decision BoundaryۄҊب ೠ׮)ী ؏ ޹хೞ׮. = উ੿੸ੋ ৘ஏਸ ೠ׮.

Slide 39

Slide 39 text

ݽ؛ ࢶఖী ROC ழ࠳ܳ ഝਊೠ׮ = Decision Boundaryী ࢚ҙহ੉ ؊ જ਷ ݽ؛ਸ ଺ח׮. = ganziо դ׮.

Slide 40

Slide 40 text

૒੽ Ӓ۰ࠇद׮. ؘ੉ఠ: titanic ݽ؛ - sklearn.linear_model.LinearRegression - sklearn.linear_model.LogisticRegression - sklearn.tree.DecisionTreeClassifier - sklearn.ensemble.RandomForestClassifier ١ whatever you want ౲ - Tree ҅ৌ੄ ݽ؛੄ ҃਋ model੄ predict_proba() ݫࣗ٘ܳ ࢎਊೞݶ ഛܫ ҅࢑੉ ؾ פ׮. - ୊਺ীח Thresholdܳ a ݅ఀ ੉ز೧оݴ Sensitivity, Specificityܳ ҅࢑೧ ઝ಴ܳ ҳೞ ࣁਃ. - যڌѱ ೞݶ Thresholdܳ ੜ ੉زदఃݶࢲ ROC ઝ಴ܳ ନਸ ࣻ ੓ਸөਃ? - ઝ಴ٜਸ ಣݶ࢚ী ନযࠁࣁਃ.

Slide 41

Slide 41 text

sklearn.metrics.roc_curve ܳ ഝਊ ೧ ࠇद׮. ؘ੉ఠ: titanic ݽ؛ - sklearn.linear_model.LinearRegression - sklearn.linear_model.LogisticRegression - sklearn.tree.DecisionTreeClassifier - sklearn.ensemble.RandomForestClassifier ١ whatever you want ؊ աইоࢲ, - sklearnਸ ੉ਊ೧ AUCب ҅࢑ ೧ࠇद׮. - ৈ۞ ݽ؛ٜ੄ ࢿמਸ ࠺Ү ೧ ࠇद׮. - DecisionTreeClassifierܳ ࢎਊ೮؊ۄب, ࢎਊೠ featureо ׮ܰ׮ݶ ӒѤ ׮ܲ ݽ؛ੑפ ׮. - ఋ੉ఋץ ݈Ҋ, ׮ܲ classification ޙઁীب ഝਊ೧ ࠁࣁਃ.

Slide 42

Slide 42 text

՘