Decisions: Week 1

Transcript

1. D S D M William Lowe Data Science Lab, Hertie School Week : . .
2. S Course logistics Evaluation Visiting speakers Readings Course expectations, fond hopes, fears Decision theory and its (apparent) alternatives Week sequence
3. C L Note: Week is cancelled (we’ll move the speaker to the makeup week)
4. C L Note: Week is cancelled (we’ll move the speaker to the makeup week) Otherwise, every week from - on Tuesdays
5. C L Note: Week is cancelled (we’ll move the speaker to the makeup week) Otherwise, every week from - on Tuesdays Live lectures, so please sign the recording consent form
6. E Assignment : Session summary Assignment : Policy brief Assignment : Discussant
7. ‘S ’ is is a brand new course → We are making it up as we go along → Roughly half the sessions are invited speakers We will rely on your → engagement with us and the speakers → willingness to share professional and/or personal experiences → patience Your instructor, shown here driving the train
8. P Patrick Doupe. Senior Economist, Zalando Christopher Gandrud. Head of Experiments and Economics, Zalando
9. P Jonathan Robinson. Head of Analytics, Catalist Stefan Heeke. People Analytics and Data Science Management
10. P James Lo. University of Southern California Caroline King. Global Head, Business Development, Government A airs, SAP
11. R In general readings will be incrementally → Uploaded to Moodle → Linked from the reference section of the slides → Expansions on lecture material → ‘Optional’
12. R In general readings will be incrementally → Uploaded to Moodle → Linked from the reference section of the slides → Expansions on lecture material → ‘Optional’ ‘Optional’ means → ere will o en be more readings than you can (or should) read → It is unlikely the course will go well if you don’t read any of them → ey will usually overlap, e.g. this week’s reading, di ering in focus → We’ll be adding them as we go along In special cases there may be some required reading
13. W ’ An understanding and appreciation of → formal models of decision making, their strengths and limitations → how data science tools succeed (or fail) to realize this kind of decision making → e challenges of decision making for individuals, organizations (and individuals in organizations) → How this works in organizations from people who have tried to make it work
14. W ’ A data science / ML / AI course → We assume you already know enough about data science / ML/ AI and will concentrate on interpretation and implementation → So ware tools to take away and use in your next organization A collection of foolproof strategies for → embedding data science in decision making contexts → deciding how to make big life decisions
15. W ... Charles Darwin, April
16. D : Decision making problems are ubiquitous Using decision theory to make decisions can nevertheless be controversial
17. D : Decision making problems are ubiquitous Using decision theory to make decisions can nevertheless be controversial How to decide: → Given actions {A}, consequences {C}, and a utility function U() choose the A with highest value of E[U(A)] = i U(Ci , A)P(Ci A) Moving parts: → e uncertainty part → e utility / loss function part → e expectation part
18. D , What kind of a theory is this? → Normative? → Descriptive? → Kinda both?
19. D , What kind of a theory is this? → Normative? → Descriptive? → Kinda both? Two connected senses of normative: → you should decide this way / ‘obligation to shareholders’ / ‘national interest’ → if you don’t decide this way we don’t really understand what you’re doing (Davidson, ; Dennett, ; Quine, ) Descriptively, consider stated vs ‘revealed’ beliefs and utility/loss functions → Religious beliefs and consumption behaviour → Company strategy statements vs actions
20. D , inking di erently about the uncertainty part → Ignorance vs uncertainty (a.k.a. risk) → Qualitative approaches, e.g. ‘beyond reasonable doubt’, ‘precedent’ International Criminal Court,
21. D , inking di erently about the utility / loss part → Refusing to quantify outcomes, e.g. ‘no price on human life’ → Procedures vs consequences, e.g. ‘inadmissible evidence’ Protests around UC v. Bakke,
22. D , inking di erently about the utility / loss part → Refusing to quantify outcomes, e.g. ‘no price on human life’ → Procedures vs consequences, e.g. ‘inadmissible evidence’ Protests around UC v. Bakke, But let’s run with this ‘maximize your expected utility idea’ at least for the length of the course...
23. W → e uncertainty part → e consequences / utility / loss function part → Deciding with a machine → Deciding with a brain → Deciding in groups
24. T In a newspaper said that the US Republican party candidate had a . change of winning the election → What does . mean here?
25. T In a newspaper said that the US Republican party candidate had a . change of winning the election → What does . mean here? What is the probability that Iran re-enters the Nuclear Agreement in the next year?
26. T In a newspaper said that the US Republican party candidate had a . change of winning the election → What does . mean here? What is the probability that Iran re-enters the Nuclear Agreement in the next year? Consider one such probability estimate → What does it mean?
27. T In a newspaper said that the US Republican party candidate had a . change of winning the election → What does . mean here? What is the probability that Iran re-enters the Nuclear Agreement in the next year? Consider one such probability estimate → What does it mean? Are you sure it should not be . higher?
28. T In a newspaper said that the US Republican party candidate had a . change of winning the election → What does . mean here? What is the probability that Iran re-enters the Nuclear Agreement in the next year? Consider one such probability estimate → What does it mean? Are you sure it should not be . higher? In the light of everyone else’s estimates, do you want to change yours?
29. T In a newspaper said that the US Republican party candidate had a . change of winning the election → What does . mean here? What is the probability that Iran re-enters the Nuclear Agreement in the next year? Consider one such probability estimate → What does it mean? Are you sure it should not be . higher? In the light of everyone else’s estimates, do you want to change yours? Why?
30. W : T Motivation → Why quantify uncertainty? → Why use probability to do it? eory: → Bayesian inference → Bayesian rationality → Expectations vs E[xpectations] Justi cation: → Consistency → ‘Dutch books’ and other arguments from embarrassment I’m all about the billiards
31. W : T Inevitability: → Representation theorems → Axiomatic treatments of uncertainty measures Trouble: → Psychological realism → Computational tractability Alternatives: → Frequentism, e.g. Neyman → Abuse of power comes as no surprise. But it is still disappointing, Ronald
32. T You have a classi er that (or an expert who) makes predictions about state failure. → How much worse are false negatives than false positives?
33. T You have a classi er that (or an expert who) makes predictions about state failure. → How much worse are false negatives than false positives? Consider the utility/loss from following outcomes: → Global sea levels will rise cm by → Global sea levels will rise m by → Global sea levels will rise cm by
34. T You have a classi er that (or an expert who) makes predictions about state failure. → How much worse are false negatives than false positives? Consider the utility/loss from following outcomes: → Global sea levels will rise cm by → Global sea levels will rise m by → Global sea levels will rise cm by → What, if anything, do we owe future generations? → Some actions would reduce their size. Does this matter? If so, how?
35. W : T eory: → of loss and utility → dominance, the ‘precautionary principle’, and other heuristics → How to discount the future Justi cation → Consistency (again) → Representation theorems (again) Implications → Political theory: Rawls vs Harsanyi → Mixing up probability and loss for good (and bad) reasons Savage in a bowtie
36. W : T Social media companies make extensive use of engagement-maximizing algorithms. → Do you approve of this strategy? Why? (in decision theory-relevant terms)
37. W : T Social media companies make extensive use of engagement-maximizing algorithms. → Do you approve of this strategy? Why? (in decision theory-relevant terms) Would it be desirable to automate decision theoretically a lot of state bureaucracy? → Why?
38. W : T Social media companies make extensive use of engagement-maximizing algorithms. → Do you approve of this strategy? Why? (in decision theory-relevant terms) Would it be desirable to automate decision theoretically a lot of state bureaucracy? → Why? Would your answer change if the automation was not decision theoretical?
39. W : T Machine learning → for probability estimation and predictions → forecast evaluation, calibration, and scoring Loss functions → fake ones, e.g. ‘log loss’ → real ones, e.g. reinforcement learning and Markov Decision Processes Really real losses → fairness and bias Computers exist. Probability doesn’t
40. W : M ‘Implementation’ details → Rationality in theory...and people → How is probability and loss represented in brains? → Risk aversion, non-exponential discounting → Cognitive heuristics and cognitive biases Performance → Forecasters and super-forecasters → Human decision making with machine learning in the mix
41. G What is the proper role of interpersonal deliberation, if any, in an expected utility framework? How to deal with organizational complications? → Individual vs organizational beliefs and preferences → Principal agent problems How to adjust the theory when the consequences are other people and their actions?
42. W : G eory: → Group decisions and preference aggregation in humans and machines → Condorcet, Arrow, and the necessity of structure → Deliberation is great. Discuss. Practice: → Deliberation, ‘group think’, ‘ lter bubbles’ → Information design in bureaucracies and companies e Afghanisdag
43. W : P Mixed human and machine decision making Implementation, risks, bene ts Whatever else we’d like to talk about, or that our speakers brought up. See you all next week!
44. R Davidson, D. ( ). ‘Inquiries into truth and interpretation’. Clarendon Press. Dennett, D. C. ( ). ‘ e intentional stance’. MIT Press. Quine, W. v. O. ( ). ‘From a logical point of view’ (Second). Harvard University Press.