Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Confusion matrix
Search
Sunmi Yoon
November 03, 2019
Technology
0
150
Confusion matrix
Confusion matrix 기초부터 머신러닝 응용까지 for dataitgirls3
Sunmi Yoon
November 03, 2019
Tweet
Share
More Decks by Sunmi Yoon
See All by Sunmi Yoon
데이터 분석가 채용 공고 읽는 방법
ysunmi0427
1
330
Deep down in classification 0.5 magic number
ysunmi0427
0
92
Tree Methods
ysunmi0427
0
120
심슨의 역설
ysunmi0427
0
2.2k
회사는 어떤 사람을 데이터 분석가로 채용하고 싶어하는 것일까?
ysunmi0427
0
2.3k
Other Decks in Technology
See All in Technology
Serverless Meetup #21
yoshidashingo
1
110
ビジネス文書に特化した基盤モデル開発 / SaaSxML_Session_2
sansan_randd
0
280
薬屋のひとりごとにみるトラブルシューティング
tomokusaba
0
200
React Server ComponentsでAPI不要の開発体験
polidog
PRO
0
130
AIのグローバルトレンド 2025 / ai global trend 2025
kyonmm
PRO
1
130
Claude Codeが働くAI中心の業務システム構築の挑戦―AIエージェント中心の働き方を目指して
os1ma
9
2.2k
プロダクトエンジニアリングで開発の楽しさを拡張する話
barometrica
0
120
猫でもわかるQ_CLI(CDK開発編)+ちょっとだけKiro
kentapapa
0
3.4k
反脆弱性(アンチフラジャイル)とデータ基盤構築
cuebic9bic
3
170
VLMサービスを用いた請求書データ化検証 / SaaSxML_Session_1
sansan_randd
0
240
Amazon Q と『音楽』-ゲーム音楽もAmazonQで作成してみた感想-
senseofunity129
0
130
o11yツールを乗り換えた話
tak0x00
2
550
Featured
See All Featured
Embracing the Ebb and Flow
colly
86
4.8k
A designer walks into a library…
pauljervisheath
207
24k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.4k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
8
750
StorybookのUI Testing Handbookを読んだ
zakiyama
30
6k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
234
17k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Faster Mobile Websites
deanohume
308
31k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
46
7.6k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
183
54k
Transcript
Evaluation for classification dataitgirls3 Instructor Sunmi Yoon
Confusion Matrix
https://sumniya.tistory.com/26
Evaluation Metrics from Confusion Matrix
https://towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62
Precision(ب), PPV(Positive Predictive Value) ݽ؛ TrueۄҊ ࠙ܨೠ Ѫ ী, पઁ
Trueੋ Ѫ ࠺ਯ Recall(അਯ), Sensitivity, hit rate पઁ True ী ݽ؛ True۽ ࠙ܨೠ ࠺ਯ “Precision݅ न҃ਸ ॳݶ ݽ؛ ੋ࢝೧Ҋ, Recall݅ न҃ॳݶ ݽ؛ ಌ” ܳ ࢤп೧ࠁࣁਃ.
Accuracy TP, TNਸ ݽف Ҋ۰ೞח . Label ࠛӐഋ बೡ ٸী
ࢎਊਸ ೧ঠ פ. F1 Score Precisionҗ Recall ઑചಣӐ Label ࠛӐഋ बೡ ٸী ݽ؛ ࢿמਸ ഛೞѱ ಣоೡ ࣻ णפ. Label ࠛӐഋ बೡ ٸী, Accuracyח ۽ࢲ न܉ࢿਸ णפ. ਬܳ ࢤп ೧ ࠁࣁਃ.
https://sumniya.tistory.com/26 ৵ ࣿಣӐ ইפҊ ઑചಣӐੋо?
ઑӘ݅ ؊ о ࠇद
https://towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62 द Ӓܿਵ۽ جই৬ࢲ, ଘ ফܳ बਵ۽ ࢤп೮
https://towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62 द Ӓܿਵ۽ جই৬ࢲ, ߣূ ফب э ࢤпೞݶࢲ ࠇद
(Әࠗఠ ഁтܾ ࣻ )
TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN
TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN
Precision Positive Predictive Value ࠙ܨ Ѿҗ(ݽ؛)ਸ बਵ۽
TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN
Negative Predictive Value ࠙ܨ Ѿҗ(ݽ؛)ਸ बਵ۽
TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN
Recall Sensitivity True Positive Rate ਸ बਵ۽
TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN
ਸ बਵ۽ False Positive Rate
TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN
ਸ बਵ۽ Specificity True Negative Rate
TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN
ਸ बਵ۽ Fall-out rate False Positive Rate
https://towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62 Ѧ ೞҊ ೮ભ. ߣূ ফب э ࢤпೞݶࢲ ࠇद (Әࠗఠ
ഁтܾ ࣻ )
TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN
? TP ब ٜ ܻೞݶ, ?
TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN
TN ब ٜ ? ܻೞݶ, ?
ഁтܻભ? ਗې Ӓ۠Ѣਃ
ӝୡח ೮ਵפө ઑӘ݅ ؊ ೧ ࠇद.
Confusion Matrix with Histogram
https://www.medcalc.org/manual/roc-curves.php Criterion, Threshold য়ܲଃ Distribution Actual True, ৽ଃ Actual False.
Threshold ਤ۽ח ݽف True۽ ஏೞח ݽ؛ Ҋ о೮ਸ ٸ,
https://www.medcalc.org/manual/roc-curves.php Thresholdܳ ӓױਵ۽ ஏ ز दெࠇद. যڃ ੌ ੌযաաਃ? Precision:
Recall: Specificity: Fall-out:
https://www.medcalc.org/manual/roc-curves.php Thresholdܳ ӓױਵ۽ ஏ ز दெࠇद. যڃ ੌ ੌযաաਃ? True
positive rate: True negative rate:
https://www.medcalc.org/manual/roc-curves.php ߣূ ߈۽ ز दெࠇद. যڃ ੌ ੌযաաਃ? True positive
rate: True negative rate:
Specificity৬ Sensitivity ҙ҅ https://www.medcalc.org/manual/roc-curves.php
ROC(Receiver Operating Characteristic) curve
рױೞѱח, Sensitivity৬ 1-Specificityܳ п ୷ਵ۽ ೞח 2ରਗ Ӓې https://www.medcalc.org/manual/roc-curves.php AUC
(Area Under Curve)
рױೞѱח, Sensitivity৬ 1-Specificityܳ п ୷ਵ۽ ೞח 2ରਗ Ӓې https://www.medcalc.org/manual/roc-curves.php Actual
True৬ Actual False distribution ৮߷ೞѱ эਸ ٸ (feature class ߸߹מ۱ হ) ROC curveח 45ب пب ࢶ
рױೞѱח, Sensitivity৬ 1-Specificityܳ п ୷ਵ۽ ೞח 2ରਗ Ӓې https://www.medcalc.org/manual/roc-curves.php Actual
True৬ Actual False distribution Ҁח হ ৮߷ೞѱ ܻ࠙ ؼ ٸ ROC ழ࠳ (feature class ߸߹ מ۱ ৮߷) ROC ழ࠳о ઝ࢚ױী оөࣻ۾ feature class ߸߹ מ۱ જҊ ೡ ࣻ .
ROC(Receiver Operating Characteristic) curve with Machine Learning
Classifierܳ ݅ٚח Ѥ, ف ѐ histogramਸ ӒܻҊ Thresholdܳ ೞח Ѫ
https://www.medcalc.org/manual/roc-curves.php
https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html#sphx-glr-auto-examples-model-selection-plot-roc-py Histogramਸ Ӓ۷ח Ѥ ROC ழ࠳ܳ Ӓܾ ࣻ ח Ѫ!
https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html#sphx-glr-auto-examples-model-selection-plot-roc-py ROC ழ࠳ܳ Ӓܾ ࣻ ח Ѥ ৈ۞ ROC ழ࠳
р ࠺Үܳ ా೧ જ ࢿמ ݽ؛ਸ ইյ ࣻ ח Ѫ!
AUCо = ݽ؛ ҅ೠ probabilityܳ ߄ఔਵ۽ Ӓܽ histogramٜ ੜ
ܻ࠙غয . = ݽ؛ Threshold(Decision BoundaryۄҊب ೠ)ী ؏ хೞ. = উੋ ஏਸ ೠ.
ݽ؛ ࢶఖী ROC ழ࠳ܳ ഝਊೠ = Decision Boundaryী ࢚ҙহ ؊
જ ݽ؛ਸ ח. = ganziо դ.
Ӓ۰ࠇद. ؘఠ: titanic ݽ؛ - sklearn.linear_model.LinearRegression - sklearn.linear_model.LogisticRegression -
sklearn.tree.DecisionTreeClassifier - sklearn.ensemble.RandomForestClassifier ١ whatever you want - Tree ҅ৌ ݽ؛ ҃ model predict_proba() ݫࣗ٘ܳ ࢎਊೞݶ ഛܫ ҅ ؾ פ. - ীח Thresholdܳ a ݅ఀ ز೧оݴ Sensitivity, Specificityܳ ҅೧ ઝܳ ҳೞ ࣁਃ. - যڌѱ ೞݶ Thresholdܳ ੜ زदఃݶࢲ ROC ઝܳ ନਸ ࣻ ਸөਃ? - ઝٜਸ ಣݶ࢚ী ନযࠁࣁਃ.
sklearn.metrics.roc_curve ܳ ഝਊ ೧ ࠇद. ؘఠ: titanic ݽ؛ - sklearn.linear_model.LinearRegression
- sklearn.linear_model.LogisticRegression - sklearn.tree.DecisionTreeClassifier - sklearn.ensemble.RandomForestClassifier ١ whatever you want ؊ աইоࢲ, - sklearnਸ ਊ೧ AUCب ҅ ೧ࠇद. - ৈ۞ ݽ؛ٜ ࢿמਸ ࠺Ү ೧ ࠇद. - DecisionTreeClassifierܳ ࢎਊ೮؊ۄب, ࢎਊೠ featureо ܰݶ ӒѤ ܲ ݽ؛ੑפ . - ఋఋץ ݈Ҋ, ܲ classification ޙઁীب ഝਊ೧ ࠁࣁਃ.