Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making. Carrie J. Cai and Emily Reif and Narayan Hegde and Jason Hipp and Been Kim and Daniel Smilkov and Martin Wattenberg and Fernanda Viegas and Greg S. Corrado and Martin C. Stumpe and Michael Terry
1. Introduction <- 2. Related Works 3. User Needs 4. User Interface and System Design 5. Tool Evaluation Study 6. User Study 7. User Study Results 8. Tool Use and Navigation Patterns 9. Decision Making and Coping With Black-box ML 10. Discussion 11. Conclusion TOC 7
1. Introduction 2. Related Works <- 3. User Needs 4. User Interface and System Design 5. Tool Evaluation Study 6. User Study 7. User Study Results 8. Tool Use and Navigation Patterns 9. Decision Making and Coping With Black-box ML 10. Discussion 11. Conclusion TOC 11
1. Introduction 2. Related Works 3. User Needs <- 4. User Interface and System Design 5. Tool Evaluation Study 6. User Study 7. User Study Results 8. Tool Use and Navigation Patterns 9. Decision Making and Coping With Black-box ML 10. Discussion 11. Conclusion TOC 13
病理学的診断において、さまざまな仮説をエビデンスの元で比較して、もっとも確 からしい診断を下す 診断は次のように行われる i. 仮説と、対立仮説を立てる ii. 生検の結果や過去の診断と照らし合わせ、どちらが妥当か判断する iii. はっきりしないときは、類似の画像を探したり、セカンドオピニオンを求めた り、再検査を要求する 画像を探すときには、異なるカテゴリの診断を横断して、もっとも類似するように 見える画像を探し、見落としがないようにする 仮説はある、でも常にセーフティーネットを求めている…他にも似ているも のがあったりしないだろうか？ Needs During Clinical Decision-Making 15
1. Introduction 2. Related Works 3. User Needs 4. User Interface and System Design <- 5. Tool Evaluation Study 6. User Study 7. User Study Results 8. Tool Use and Navigation Patterns 9. Decision Making and Coping With Black-box ML 10. Discussion 11. Conclusion TOC 17
SMILY では入力画像に対して、学習済みの深層学習モデルを用いて高次元の Embedding を計算する データベースの中から、Embedding space での近傍画像を出力する ユーザーのフィードバックの結果、1 ページにおおよそ 15 枚の画像を出力するよ うに調整した User Interface and System Design 18
1. Introduction 2. Related Works 3. User Needs 4. User Interface and System Design 5. Tool Evaluation Study <- 6. User Study 7. User Study Results 8. Tool Use and Navigation Patterns 9. Decision Making and Coping With Black-box ML 10. Discussion 11. Conclusion TOC 28
1. Introduction 2. Related Works 3. User Needs 4. User Interface and System Design 5. Tool Evaluation Study 6. User Study <- 7. User Study Results <- 8. Tool Use and Navigation Patterns 9. Decision Making and Coping With Black-box ML 10. Discussion 11. Conclusion TOC 31
既存の画像検索システムと今回作成した SMILE とのどちらのほうがユーザーにと って良かったのかアンケート調査 1. Diagnostic utility: 診断を下すために見つけた画像が役立っただった度合い 2. Mental support for decision-making: 診断や考えをまとめるのに役立った度合い 3. Workload: 使いこなすのに必要だった労力や、使うときに感じた苛立ち度合い 4. Trust: システムの能力と、その振る舞いについての信頼度合い 5. Future use: 業務でこの先使いたいと思う度合い 6. Overall preference between the two interfaces: 総合評価 User Study 32
1. Introduction 2. Related Works 3. User Needs 4. User Interface and System Design 5. Tool Evaluation Study 6. User Study 7. User Study Results 8. Tool Use and Navigation Patterns <- 9. Decision Making and Coping With Black-box ML 10. Discussion 11. Conclusion TOC 34
1. Introduction 2. Related Works 3. User Needs 4. User Interface and System Design 5. Tool Evaluation Study 6. User Study 7. User Study Results 8. Tool Use and Navigation Patterns 9. Decision Making and Coping With Black-box ML <- 10. Discussion 11. Conclusion TOC 41
病理医が仮説を立案し、比較し、確からしさを判断するプロセスについて考える まず、意思決定のために仮説をより良いものにしていく戦略について述べる 次に、機械学習モデルをうまく取り扱いながら仮説をより良いものにする戦略につ いて述べる Decision Making and Coping With Black-box ML 42
1. Introduction 2. Related Works 3. User Needs 4. User Interface and System Design 5. Tool Evaluation Study 6. User Study 7. User Study Results 8. Tool Use and Navigation Patterns 9. Decision Making and Coping With Black-box ML 10. Discussion <- 11. Conclusion TOC 46
1. Introduction 2. Related Works 3. User Needs 4. User Interface and System Design 5. Tool Evaluation Study 6. User Study 7. User Study Results 8. Tool Use and Navigation Patterns 9. Decision Making and Coping With Black-box ML 10. Discussion <- 11. Conclusion TOC 48
A Human-Centered Evaluation of a Deep Learning System Deployed in Clinics for the Detection of Diabetic Retinopathy Emma Beede, Elizabeth Baylor, Fred Hersch, Anna Iurchenko, Lauren Wilcox, Paisan Ruamviboonsuk, Laura M. Vardoulaki
from How Google does Machine Learning (coursera) 1. 業務が未定義 2. 業務が定義され、手順書を作る <- 医療現場はここだった 3. 手順が明確になり、システム化を行う 4. 情報を可視化し、意思決定を行う 5. 意思決定の自動化 or 補助 <- 機械学習はここ 逆説的にこれが正しいと示す結果かもしれない 機械学習までのステップ by Google 54
Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making | Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems Google AI Blog: Building SMILY, a Human-Centric, Similar-Image Search Tool for Pathology A Human-Centered Evaluation of a Deep Learning System Deployed in Clinics for the Detection of Diabetic Retinopathy | Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems Healthcare AI systems that put people at the center Googleの失敗から学ぶ、AIツールを医療現場へ適用することの難しさ How Google does Machine Learning 日本語版 | Coursera Reference 55
asei
tomuro
keio_smilab
takatsunobuhiro
yoshiitaka
nktamago
th0x0472
bengo4com
mochico
keiji
line_developers
takuyay0ne
prathamesh
palkan
colly
trallard
eileencodes
moore
mclloyd
jonrohan
dougneiner
sonatard
jeffersonlam
keithpitt
jonyablonski