Corey Chivers
June 14, 2018
450

# From Predictions to Decisions

Bringing Decision Theory to Healthcare Data Science

June 14, 2018

## Transcript

1. From Predictions to
Decisions
Bringing Decision Theory to Healthcare Data Science
Corey Chivers, PhD
bayesianbiologist.com
@cjbayesian

2. Is this a good model?
(1 – Specificity)
(Sensitivity)

3. Is this a good model?
FP
FN TP
TN

4. "Good" is only meaningful relative to
the "goodness" of the events being
predicted and our ability to do
something to change them.

5. All models are
wrong, but some
are useful.
- George Box

6. FP
FN TP
TN

7. How good tho?
How good tho?
FP
FN TP
TN

8. Goodness can be measured in any units

9. The best decision is the one that leads to the
most goodness

10. When outcomes are uncertain, the best decision is
the one that has the highest expected goodness.

11. When outcomes are uncertain, the best decision is
the one that has the highest expected goodness.
Machine Learning
with this part!

12. When outcomes are uncertain, the best decision is
the one that has the highest expected goodness.
Machine Learning
with this part!

13. Healthcare Example
Predicted Sepsis
Treated a true case
(Potential to avoid
Predicted Sepsis
Treated a false case
(unnecessary)
Predicted No Sepsis
Didn’t treat
(all good)
Predicted No Sepsis
Failed to treat

14. Healthcare Example

15. Healthcare Example

16. Healthcare Example
• How does that help us decide?
• The key is to compare against the alternatives.
• Treat everyone? <==> set decision threshold to zero
• Treat no one? <==> set decision threshold to one
• Some other strategy? <==> Random? Status quo?
• Choose the alternative that is least bad
• Or, if you prefer the J perspective, multiply everything by -1 and choose the
one that’s the most good!

17. Healthcare Example
• How does that help us decide?
• The key is to compare against the alternatives.
• Treat everyone? <==> set decision threshold to zero
• Treat no one? <==> set decision threshold to one
• Some other strategy? <==> Random? Status quo?
• Choose the alternative that is least bad
• Or, if you prefer the J perspective, multiply everything by -1 and choose the
one that’s the most good!

18. So, like, should we use this model or what?
• There are still things we don’t know,
like what all these C terms actually are.
• In what range would they need to be for
us to choose the model over an
alternative strategy?
• How much better would the model need
to be before we would choose to use it?

19. Treat none
Cost of Intervention (\$)
Cost of event (\$)
Treat all
Pregnancy Related Hypertension (PRH) is the leading
cause of maternal morbidity and mortality in the U.S.
High-risk patients à remote blood pressure monitoring
https://healthcareinnovation.upenn.edu/projects/heart-safe-motherhood

20. Assume, evaluate, refine, repeat
• Decision theory requires making assumptions, as we’ve done here
• Nothing is stopping us from making different assumptions:
• “What if I have other priorities, like fairness?”
• Decision theory analysis is not infallible, but it allows us to:
• to the most precisely formulated questions,
• when starting from a given set of assumptions

21. Thanks!
bayesianbiologist.com
@cjbayesian