Slide 11
Slide 11 text
● Interpretation of Cost and Benefit: Pauker
1975
○ Benefit of Treating diseased vs
○ Cost of Treating non-diseased
● Interpretation of phrase “Net Benefit”
○ Pauker 1975 version
○ Vickers Net Benefit version (normalized)
● {Cost,Benefit} pair have 2 separate but
related relationships
○ Ratio: Cost/Benefit ratio on x-axis of DCA plot
○ Difference: ( Benefit - Cost )/Benefit Net
Benefit (normalized) on y-axis of DCA plot
● Estimating Cost/Benefit from clinical policy
○ clinician willing to conduct 20 biopsies to find a
high grade cancer, the probability threshold
would be 5%
■ – Net benefit approaches to the evaluation of
prediction models, molecular markers, and
diagnostic tests, Vickers et al 2015
Things that were unclear to me when first learning DCA
● Equivalence of:
○ Cost / Benefit ratio for clinical procedure ⇔ Model threshold :
“Exchange rate”
○ Both shown on same x-axis
● Why are we so into varying threshold probability?
○ “a) there are insufficient data on which to calculate a rational
threshold, or
b) patients can reasonably disagree about the appropriate
threshold, due to different preferences for alternative health states”
– Vickers in “Decision curve analysis: a discussion”, Steyerberg et
al 2008, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2577563/
● What data is needed to construct DCA plot
○ Y_true: True binary labels
○ For models: Y_score: Estimated probability from model (thresholded
to determine treatment decision)
○ For rules: Treatment decision [0/1]
● Interpretation of x axis:
○ x=0.0 : no costs only benefits, x=1.0 : no benefits only costs
○ X is both Cost / Benefit ratio ⇔ Model threshold
● Interpretation of y axis:
○ y=0.0 : do nothing implies no net benefit (no treatment)
○ Y axis units: expectation of true-positive result, 1.0 is benefit of
treating 1 diseased patient
○ Max y value: prevalence; ie. mean benefit of treating all diseased
patients