Time-Inconsistent Planning

90-minute presentation at the InfoCloud (cloud.kaust.edu.sa) group meeting, on "Time-Inconsistent Planning: A Computational Problem in Behavioral Economics" by Jon Kleinberg, published in EC '14.

Transcript

1. Time-Inconsistent Planning A Computational Problem in Behavioral Economics Jon Kleinberg, Sigal Oren. EC 2014 Presented by Emaad Ahmed Manzoor May 20, 2014
2. Time-Inconsistent Planning A Computational Problem in Behavioral Economics Jon Kleinberg, Sigal Oren. EC 2014 Presented by Emaad Ahmed Manzoor May 20, 2014 Individual stick figures from xkcd.com
3. Abandonment
4. Procrastination
5. A Recurring Pattern?
6. t = t 0 Exercise
7. t = t 0 Exercise t = t 1 Exercise?
8. t = t 0 Exercise t = t 1 Exercise? t = t 1 Exercise...
9. t = t 0 Exercise t = t 1 Exercise? t = t 1 Exercise... Objective function is not consistent over time
10. t = t 0 Exercise t = t 1 Exercise? t = t 1 Exercise... Objective function is not consistent over time George A. Akerlof. Procrastination and Obedience. American Economic Review, 1991.
11. George
12. Ship a package in the next n days, decide on which day t to do so t = 1, 2, …, n
13. Ship a package in the next n days, decide on which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day)
14. Ship a package in the next n days, decide on which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day) h (constant) days in transit
15. If shipped on day t, the total cost is (t + h)x Ship a package in the next n days, decide on which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day) h (constant) days in transit
16. If shipped on day t, the total cost is (t + h)x Ship a package in the next n days, decide on which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day) h (constant) days in transit c (one-time) packing cost
17. If shipped on day t, the total cost is c + (t + h)x Ship a package in the next n days, decide on which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day) h (constant) days in transit c (one-time) packing cost
18. If shipped on day t, the total cost is c + (t + h)x Ship a package in the next n days, decide on which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day) h (constant) days in transit c (one-time) packing cost What is the optimal plan?
19. What is the optimal plan?
20. What is the optimal plan?
21. George:
22. George: Procrastinate!
23. George: Procrastinate! A natural way to model procrastination?
24. Modeling Procrastination
25. Present bias
26. Agent is on day t, considering sending the package
27. Agent is on day t, considering sending the package Evaluated cost of sending it on day t
28. Agent is on day t, considering sending the package Evaluated cost of sending it on day t Evaluated cost of sending it on day t + 1
29. Agent is on day t, considering sending the package Evaluated cost of sending it on day t Evaluated cost of sending it on day t + 1 Difference in evaluated costs
30. Agent is on day t, considering sending the package Evaluated cost of sending it on day t Evaluated cost of sending it on day t + 1 Difference in evaluated costs Wait till day (t + 1): Procrastinate!
31. Quasi-Hyperbolic Discounting
32. Agent is on day t-1, considering sending the package
33. Agent is on day t-1, considering sending the package Actual cost of sending it on day t
34. Agent is on day t-1, considering sending the package Actual cost of sending it on day t Evaluated cost of sending it on day t
35. Agent is on day t-1, considering sending the package Actual cost of sending it on day t Evaluated cost of sending it on day t Present bias
36. Qualitative Predictions
37. Procrastination George A. Akerlof. Procrastination and Obedience. American Economic Review, 1991.
38. Procrastination Abandonment George A. Akerlof. Procrastination and Obedience. American Economic Review, 1991. O' Donoghue et al. Procrastination on long- term projects. Journal of Economic Behaviour and Organization. 2008.
39. Procrastination Abandonment George A. Akerlof. Procrastination and Obedience. American Economic Review, 1991. O' Donoghue et al. Procrastination on long- term projects. Journal of Economic Behaviour and Organization. 2008. Choice-Reduction Ariely et al. Procrastination, deadlines and performance: self-control by precommitment. Psychological Science. 2002.
40. Is there a single framework representing tasks and goals, capturing these effects?
41. This Paper
42. The Task Graph s a c b d e t 16 2 2 8 8 8 2 16 The Agent
43. The Task Graph s a c b d e t 16 2 2 8 8 8 2 16 The Agent
44. 8 8 8 2 The Task Graph s a c b d e t 16 2 2 16 The Agent
45. 2 2 8 2 16 The Task Graph s a c b d e t 16 8 8 The Agent
46. The Task Graph s a c b d e t 16 2 2 8 8 8 2 16 The Agent
47. 2 2 The Task Graph s a c b d e t 16 8 8 8 2 16 The Agent
48. 8 2 2 2 The Task Graph s a c b d e t 16 8 8 16 The Agent
49. 8 8 2 2 The Task Graph s a c b d e t 16 8 2 16 The Agent
50. 8 8 2 2 The Task Graph s a c b d e t 16 8 2 16 The Agent
51. 8 8 2 2 The Task Graph s a c b d e t 16 8 2 16 The Agent
52. 8 8 2 2 The Task Graph s a c b d e t 16 8 2 16 The Agent Time-consistent agent path s-c-d-t length = 24
53. 8 8 2 2 The Task Graph s a c b d e t 16 8 2 16 The Agent Time-consistent agent path s-c-d-t length = 24 Time-inconsistent agent path s-c-e-t length = 26
54. 8 8 2 2 The Task Graph s a c b d e t 16 8 2 16 The Agent Time-consistent agent path s-c-d-t length = 24 Time-inconsistent agent path s-c-e-t length = 26 Cost Ratio = 26/24
55. Can formally answer questions like:
56. Can formally answer questions like: In which scenario will a time-inconsistent agent waste the most effort?
57. Can formally answer questions like: In which scenario will a time-inconsistent agent waste the most effort? How do agents with different levels of bias proceed towards the same goal?
58. Can formally answer questions like: In which scenario will a time-inconsistent agent waste the most effort? How do agents with different levels of bias proceed towards the same goal? How do we design a complex task to improve a time- inconsistent agent's efficiency?
59. Can formally answer questions like: In which scenario will a time-inconsistent agent waste the most effort? How do agents with different levels of bias proceed towards the same goal? How do we design a complex task to improve a time- inconsistent agent's efficiency? Worst-case task graph = graph minors Cost ratio = price of irrationality
60. Can formally answer questions like: In which scenario will a time-inconsistent agent waste the most effort? How do agents with different levels of bias proceed towards the same goal? How do we design a complex task to improve a time- inconsistent agent's efficiency? Parametric shortest path problem; how many distinct paths? Bias is continuous in [0, 1]
61. Can formally answer questions like: In which scenario will a time-inconsistent agent waste the most effort? How do agents with different levels of bias proceed towards the same goal? How do we design a complex task to improve a time- inconsistent agent's efficiency? Remove nodes/edges from the graph to improve efficiency. What possible subgraphs support efficient traversal?
62. Captures a variety of phenomena studied in connection with time-inconsistent behaviour
63. Examples of Behaviour
64. The Akerlof Example h = 0, n = 5 Procrastination
65. Abandonment Agent at v, reward r at target t Abandon midway if, for all v-t paths P.
66. Abandonment Agent at v, reward r at target t Abandon midway if, for all v-t paths P. Real cost of the first edge of P
67. Abandonment Agent at v, reward r at target t Abandon midway if, for all v-t paths P. Evaluated costs of the rest of the path
68. Abandonment Agent at v, reward r at target t Abandon midway if, for all v-t paths P. Evaluated reward
69. Choice-Reduction Student takes a 3-week course, has to complete 2 projects by the end of the course. Week with no project work Cost = 1 Week with work on 1 project Cost = 4 Week with work on 2 projects Cost = 9 For completing the course Reward = 16 Present bias
70. Choice-Reduction
71. Choice-Reduction 1 4 9
72. Choice-Reduction
73. Choice-Reduction
74. Choice-Reduction Both projects in the last week of the course
75. Choice-Reduction
76. Drop course Choice-Reduction
77. Choice-Reduction A simple intervention:
78. Choice-Reduction A simple intervention: Complete first project by the end of week 2
79. Choice-Reduction A simple intervention: Complete first project by the end of week 2
80. Choice-Reduction
81. Choice-Reduction
82. Choice-Reduction
83. It is difficult to reason about the space of possible "stories"
84. It is tractable to reason about the space of possible graphs
85. Bad Cost Ratios
86. Cost-Ratio
87. The cost-ratio can be exponential in the number of nodes n.
88. The Akerlof Example h = 0, n = 5
89. The Akerlof Example
90. The Akerlof Example
91. The Akerlof Example
92. The Akerlof Example
93. The Akerlof Example
94. The Akerlof Example
95. On what kind of graphs can the cost- ratio can be exponential in the number of nodes n?
96. The undirected version of the graph must contain a large copy of the Akerlof example as a minor (proof) An unavoidable signature of any graph with an exponentially large cost ratio
97. Graph Minors
98. is a minor of Edge deletion Edge contraction Isolated vertex deletion
99. Motivating Subgraphs
100. Week with no project work Cost = 1 Week with work on 1 project Cost = 4 Week with work on 2 projects Cost = 9 For completing the course Reward = 16
101. Week with no project work Cost = 1 Week with work on 1 project Cost = 4 Week with work on 2 projects Cost = 9 For completing the course Reward = 16 Does not affect the agents choice of path, only whether it decides to continue down that path or not.
102. Week with no project work Cost = 1 Week with work on 1 project Cost = 4 Week with work on 2 projects Cost = 9 For completing the course Reward = 16 Actual value is outside of our control
103. A subgraph G' of G motivates the agent if in G' with reward r, the agent reaches the goal node t G' is a motivating subgraph
104. A motivating subgraph G' contains an s-t path P followed by the agent
105. A motivating subgraph G' contains an s-t path P followed by the agent Conjecture: P is also a motivating subgraph
115. Abandon
117. Abandon
118. No proper subgraph of G is motivating
119. Crucial for the agent to initially believe that the upper path is an option
120. Crucial for the agent to initially believe that the upper path is an option Real-life analogues?
121. Minimal Motivating Subgraph G*
122. How rich a subgraph G' do we necessarily need to motivate the agent?
123. What must a minimal motivating subgraph look like?
124. What must a minimal motivating subgraph look like? Must be sparse (proof)
125. Open Problems
126. This Paper
127. This Paper Graph-theoretic model of tasks
128. This Paper Graph-theoretic model of tasks Agent constructs a path through this graph
129. This Paper Graph-theoretic model of tasks Agent constructs a path through this graph Time-inconsistent behavior of agents in this graph reproduces: procrastination, abandonment, benefits of reduced options
130. This Paper Graph-theoretic model of tasks Agent constructs a path through this graph Time-inconsistent behavior of agents in this graph reproduces: procrastination, abandonment, benefits of reduced options Characterizations of: - Graphs with the worst cost-ratios - Minimal motivation subgraphs
131. Open Questions
132. Open Questions Upper bound on the cost-ratio of a given graph that does not contain an Akerlof minor?
133. Open Questions Upper bound on the cost-ratio of a given graph that does not contain an Akerlof minor? How hard is it computationally to find minimal motivating subgraphs?
134. Open Questions Upper bound on the cost-ratio of a given graph that does not contain an Akerlof minor? How hard is it computationally to find minimal motivating subgraphs? Can we motivate agents by placing intermediate rewards on specific tasks? "Exploitive" task design; minimize payout Negative rewards
135. Open Questions Upper bound on the cost-ratio of a given graph that does not contain an Akerlof minor? How hard is it computationally to find minimal motivating subgraphs? Can we motivate agents by placing intermediate rewards on specific tasks? "Exploitive" task design; minimize payout Negative rewards Multiple heterogenous present-biased agents?
136. .