May 20, 2014
150

Time-Inconsistent Planning

90-minute presentation at the InfoCloud (cloud.kaust.edu.sa) group meeting, on "Time-Inconsistent Planning: A Computational Problem in Behavioral Economics" by Jon Kleinberg, published in EC '14.

May 20, 2014

Transcript

1. Time-Inconsistent Planning A Computational Problem in Behavioral Economics Jon Kleinberg,

Sigal Oren. EC 2014 Presented by Emaad Ahmed Manzoor May 20, 2014
2. Time-Inconsistent Planning A Computational Problem in Behavioral Economics Jon Kleinberg,

Sigal Oren. EC 2014 Presented by Emaad Ahmed Manzoor May 20, 2014 Individual stick figures from xkcd.com

8. t = t 0 Exercise t = t 1 Exercise?

t = t 1 Exercise...
9. t = t 0 Exercise t = t 1 Exercise?

t = t 1 Exercise... Objective function is not consistent over time
10. t = t 0 Exercise t = t 1 Exercise?

t = t 1 Exercise... Objective function is not consistent over time George A. Akerlof. Procrastination and Obedience. American Economic Review, 1991.

12. Ship a package in the next n days, decide on

which day t to do so t = 1, 2, …, n
13. Ship a package in the next n days, decide on

which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day)
14. Ship a package in the next n days, decide on

which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day) h (constant) days in transit
15. If shipped on day t, the total cost is (t

+ h)x Ship a package in the next n days, decide on which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day) h (constant) days in transit
16. If shipped on day t, the total cost is (t

+ h)x Ship a package in the next n days, decide on which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day) h (constant) days in transit c (one-time) packing cost
17. If shipped on day t, the total cost is c

+ (t + h)x Ship a package in the next n days, decide on which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day) h (constant) days in transit c (one-time) packing cost
18. If shipped on day t, the total cost is c

+ (t + h)x Ship a package in the next n days, decide on which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day) h (constant) days in transit c (one-time) packing cost What is the optimal plan?

27. Agent is on day t, considering sending the package Evaluated

cost of sending it on day t
28. Agent is on day t, considering sending the package Evaluated

cost of sending it on day t Evaluated cost of sending it on day t + 1
29. Agent is on day t, considering sending the package Evaluated

cost of sending it on day t Evaluated cost of sending it on day t + 1 Difference in evaluated costs
30. Agent is on day t, considering sending the package Evaluated

cost of sending it on day t Evaluated cost of sending it on day t + 1 Difference in evaluated costs Wait till day (t + 1): Procrastinate!

33. Agent is on day t-1, considering sending the package Actual

cost of sending it on day t
34. Agent is on day t-1, considering sending the package Actual

cost of sending it on day t Evaluated cost of sending it on day t
35. Agent is on day t-1, considering sending the package Actual

cost of sending it on day t Evaluated cost of sending it on day t Present bias

1991.
38. Procrastination Abandonment George A. Akerlof. Procrastination and Obedience. American Economic

Review, 1991. O' Donoghue et al. Procrastination on long- term projects. Journal of Economic Behaviour and Organization. 2008.
39. Procrastination Abandonment George A. Akerlof. Procrastination and Obedience. American Economic

Review, 1991. O' Donoghue et al. Procrastination on long- term projects. Journal of Economic Behaviour and Organization. 2008. Choice-Reduction Ariely et al. Procrastination, deadlines and performance: self-control by precommitment. Psychological Science. 2002.
40. Is there a single framework representing tasks and goals, capturing

these effects?

42. The Task Graph s a c b d e t

16 2 2 8 8 8 2 16 The Agent
43. The Task Graph s a c b d e t

16 2 2 8 8 8 2 16 The Agent
44. 8 8 8 2 The Task Graph s a c

b d e t 16 2 2 16 The Agent
45. 2 2 8 2 16 The Task Graph s a

c b d e t 16 8 8 The Agent
46. The Task Graph s a c b d e t

16 2 2 8 8 8 2 16 The Agent
47. 2 2 The Task Graph s a c b d

e t 16 8 8 8 2 16 The Agent
48. 8 2 2 2 The Task Graph s a c

b d e t 16 8 8 16 The Agent
49. 8 8 2 2 The Task Graph s a c

b d e t 16 8 2 16 The Agent
50. 8 8 2 2 The Task Graph s a c

b d e t 16 8 2 16 The Agent
51. 8 8 2 2 The Task Graph s a c

b d e t 16 8 2 16 The Agent
52. 8 8 2 2 The Task Graph s a c

b d e t 16 8 2 16 The Agent Time-consistent agent path s-c-d-t length = 24
53. 8 8 2 2 The Task Graph s a c

b d e t 16 8 2 16 The Agent Time-consistent agent path s-c-d-t length = 24 Time-inconsistent agent path s-c-e-t length = 26
54. 8 8 2 2 The Task Graph s a c

b d e t 16 8 2 16 The Agent Time-consistent agent path s-c-d-t length = 24 Time-inconsistent agent path s-c-e-t length = 26 Cost Ratio = 26/24

56. Can formally answer questions like: In which scenario will a

time-inconsistent agent waste the most effort?
57. Can formally answer questions like: In which scenario will a

time-inconsistent agent waste the most effort? How do agents with different levels of bias proceed towards the same goal?
58. Can formally answer questions like: In which scenario will a

time-inconsistent agent waste the most effort? How do agents with different levels of bias proceed towards the same goal? How do we design a complex task to improve a time- inconsistent agent's efficiency?
59. Can formally answer questions like: In which scenario will a

time-inconsistent agent waste the most effort? How do agents with different levels of bias proceed towards the same goal? How do we design a complex task to improve a time- inconsistent agent's efficiency? Worst-case task graph = graph minors Cost ratio = price of irrationality
60. Can formally answer questions like: In which scenario will a

time-inconsistent agent waste the most effort? How do agents with different levels of bias proceed towards the same goal? How do we design a complex task to improve a time- inconsistent agent's efficiency? Parametric shortest path problem; how many distinct paths? Bias is continuous in [0, 1]
61. Can formally answer questions like: In which scenario will a

time-inconsistent agent waste the most effort? How do agents with different levels of bias proceed towards the same goal? How do we design a complex task to improve a time- inconsistent agent's efficiency? Remove nodes/edges from the graph to improve efficiency. What possible subgraphs support efficient traversal?

behaviour

65. Abandonment Agent at v, reward r at target t Abandon

midway if, for all v-t paths P.
66. Abandonment Agent at v, reward r at target t Abandon

midway if, for all v-t paths P. Real cost of the first edge of P
67. Abandonment Agent at v, reward r at target t Abandon

midway if, for all v-t paths P. Evaluated costs of the rest of the path
68. Abandonment Agent at v, reward r at target t Abandon

midway if, for all v-t paths P. Evaluated reward
69. Choice-Reduction Student takes a 3-week course, has to complete 2

projects by the end of the course. Week with no project work Cost = 1 Week with work on 1 project Cost = 4 Week with work on 2 projects Cost = 9 For completing the course Reward = 16 Present bias

of week 2

of week 2

"stories"

graphs

n.

95. On what kind of graphs can the cost- ratio can

be exponential in the number of nodes n?
96. The undirected version of the graph must contain a large

copy of the Akerlof example as a minor (proof) An unavoidable signature of any graph with an exponentially large cost ratio

deletion

100. Week with no project work Cost = 1 Week with

work on 1 project Cost = 4 Week with work on 2 projects Cost = 9 For completing the course Reward = 16
101. Week with no project work Cost = 1 Week with

work on 1 project Cost = 4 Week with work on 2 projects Cost = 9 For completing the course Reward = 16 Does not affect the agents choice of path, only whether it decides to continue down that path or not.
102. Week with no project work Cost = 1 Week with

work on 1 project Cost = 4 Week with work on 2 projects Cost = 9 For completing the course Reward = 16 Actual value is outside of our control
103. A subgraph G' of G motivates the agent if in

G' with reward r, the agent reaches the goal node t G' is a motivating subgraph

by the agent
105. A motivating subgraph G' contains an s-t path P followed

by the agent Conjecture: P is also a motivating subgraph

109. Crucial for the agent to initially believe that the upper

path is an option
110. Crucial for the agent to initially believe that the upper

path is an option Real-life analogues?

112. How rich a subgraph G' do we necessarily need to

motivate the agent?

114. What must a minimal motivating subgraph look like? Must be

sparse (proof)

118. This Paper Graph-theoretic model of tasks Agent constructs a path

through this graph
119. This Paper Graph-theoretic model of tasks Agent constructs a path

through this graph Time-inconsistent behavior of agents in this graph reproduces: procrastination, abandonment, benefits of reduced options
120. This Paper Graph-theoretic model of tasks Agent constructs a path

through this graph Time-inconsistent behavior of agents in this graph reproduces: procrastination, abandonment, benefits of reduced options Characterizations of: - Graphs with the worst cost-ratios - Minimal motivation subgraphs

122. Open Questions Upper bound on the cost-ratio of a given

graph that does not contain an Akerlof minor?
123. Open Questions Upper bound on the cost-ratio of a given

graph that does not contain an Akerlof minor? How hard is it computationally to find minimal motivating subgraphs?
124. Open Questions Upper bound on the cost-ratio of a given

graph that does not contain an Akerlof minor? How hard is it computationally to find minimal motivating subgraphs? Can we motivate agents by placing intermediate rewards on specific tasks? "Exploitive" task design; minimize payout Negative rewards
125. Open Questions Upper bound on the cost-ratio of a given

graph that does not contain an Akerlof minor? How hard is it computationally to find minimal motivating subgraphs? Can we motivate agents by placing intermediate rewards on specific tasks? "Exploitive" task design; minimize payout Negative rewards Multiple heterogenous present-biased agents?