Healthcare Example Predicted Sepsis Treated a true case (Potential to avoid bad outcome) Predicted Sepsis Treated a false case (unnecessary) Predicted No Sepsis Didn’t treat (all good) Predicted No Sepsis Failed to treat (Bad outcome)
Healthcare Example • But everything just adds to the ‘badness’ total. • How does that help us decide? • The key is to compare against the alternatives. • Treat everyone? <==> set decision threshold to zero • Treat no one? <==> set decision threshold to one • Some other strategy? <==> Random? Status quo? • Choose the alternative that is least bad • Or, if you prefer the J perspective, multiply everything by -1 and choose the one that’s the most good!
Healthcare Example • But everything just adds to the ‘badness’ total. • How does that help us decide? • The key is to compare against the alternatives. • Treat everyone? <==> set decision threshold to zero • Treat no one? <==> set decision threshold to one • Some other strategy? <==> Random? Status quo? • Choose the alternative that is least bad • Or, if you prefer the J perspective, multiply everything by -1 and choose the one that’s the most good!
So, like, should we use this model or what? • There are still things we don’t know, like what all these C terms actually are. • Work backwards and ask: • In what range would they need to be for us to choose the model over an alternative strategy? • How much better would the model need to be before we would choose to use it?
Treat none Cost of Intervention ($) Cost of event ($) Treat all Pregnancy Related Hypertension (PRH) is the leading cause of maternal morbidity and mortality in the U.S. High-risk patients à remote blood pressure monitoring https://healthcareinnovation.upenn.edu/projects/heart-safe-motherhood
Assume, evaluate, refine, repeat • Decision theory requires making assumptions, as we’ve done here • Nothing is stopping us from making different assumptions: • “What if the false positives lead to additional harm?” • “What if I have other priorities, like fairness?” • Decision theory analysis is not infallible, but it allows us to: • Get the best possible answers • to the most precisely formulated questions, • when starting from a given set of assumptions