Time-Inconsistent Planning

Time-Inconsistent Planning

90-minute presentation at the InfoCloud (cloud.kaust.edu.sa) group meeting, on "Time-Inconsistent Planning: A Computational Problem in Behavioral Economics" by Jon Kleinberg, published in EC '14.

Ed09e933a899fcae158439f11f66fed0?s=128

Emaad Manzoor

May 20, 2014
Tweet

Transcript

  1. Time-Inconsistent Planning A Computational Problem in Behavioral Economics Jon Kleinberg,

    Sigal Oren. EC 2014 Presented by Emaad Ahmed Manzoor May 20, 2014
  2. Time-Inconsistent Planning A Computational Problem in Behavioral Economics Jon Kleinberg,

    Sigal Oren. EC 2014 Presented by Emaad Ahmed Manzoor May 20, 2014 Individual stick figures from xkcd.com
  3. Abandonment

  4. Procrastination

  5. A Recurring Pattern?

  6. t = t 0 Exercise

  7. t = t 0 Exercise t = t 1 Exercise?

  8. t = t 0 Exercise t = t 1 Exercise?

    t = t 1 Exercise...
  9. t = t 0 Exercise t = t 1 Exercise?

    t = t 1 Exercise... Objective function is not consistent over time
  10. t = t 0 Exercise t = t 1 Exercise?

    t = t 1 Exercise... Objective function is not consistent over time George A. Akerlof. Procrastination and Obedience. American Economic Review, 1991.
  11. George

  12. Ship a package in the next n days, decide on

    which day t to do so t = 1, 2, …, n
  13. Ship a package in the next n days, decide on

    which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day)
  14. Ship a package in the next n days, decide on

    which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day) h (constant) days in transit
  15. If shipped on day t, the total cost is (t

    + h)x Ship a package in the next n days, decide on which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day) h (constant) days in transit
  16. If shipped on day t, the total cost is (t

    + h)x Ship a package in the next n days, decide on which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day) h (constant) days in transit c (one-time) packing cost
  17. If shipped on day t, the total cost is c

    + (t + h)x Ship a package in the next n days, decide on which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day) h (constant) days in transit c (one-time) packing cost
  18. If shipped on day t, the total cost is c

    + (t + h)x Ship a package in the next n days, decide on which day t to do so t = 1, 2, …, n Each day that the package does not reach incurs a cost x (per day) h (constant) days in transit c (one-time) packing cost What is the optimal plan?
  19. What is the optimal plan?

  20. What is the optimal plan?

  21. George:

  22. George: Procrastinate!

  23. George: Procrastinate! A natural way to model procrastination?

  24. Modeling Procrastination

  25. Present bias

  26. Agent is on day t, considering sending the package

  27. Agent is on day t, considering sending the package Evaluated

    cost of sending it on day t
  28. Agent is on day t, considering sending the package Evaluated

    cost of sending it on day t Evaluated cost of sending it on day t + 1
  29. Agent is on day t, considering sending the package Evaluated

    cost of sending it on day t Evaluated cost of sending it on day t + 1 Difference in evaluated costs
  30. Agent is on day t, considering sending the package Evaluated

    cost of sending it on day t Evaluated cost of sending it on day t + 1 Difference in evaluated costs Wait till day (t + 1): Procrastinate!
  31. Quasi-Hyperbolic Discounting

  32. Agent is on day t-1, considering sending the package

  33. Agent is on day t-1, considering sending the package Actual

    cost of sending it on day t
  34. Agent is on day t-1, considering sending the package Actual

    cost of sending it on day t Evaluated cost of sending it on day t
  35. Agent is on day t-1, considering sending the package Actual

    cost of sending it on day t Evaluated cost of sending it on day t Present bias
  36. Qualitative Predictions

  37. Procrastination George A. Akerlof. Procrastination and Obedience. American Economic Review,

    1991.
  38. Procrastination Abandonment George A. Akerlof. Procrastination and Obedience. American Economic

    Review, 1991. O' Donoghue et al. Procrastination on long- term projects. Journal of Economic Behaviour and Organization. 2008.
  39. Procrastination Abandonment George A. Akerlof. Procrastination and Obedience. American Economic

    Review, 1991. O' Donoghue et al. Procrastination on long- term projects. Journal of Economic Behaviour and Organization. 2008. Choice-Reduction Ariely et al. Procrastination, deadlines and performance: self-control by precommitment. Psychological Science. 2002.
  40. Is there a single framework representing tasks and goals, capturing

    these effects?
  41. This Paper

  42. The Task Graph s a c b d e t

    16 2 2 8 8 8 2 16 The Agent
  43. The Task Graph s a c b d e t

    16 2 2 8 8 8 2 16 The Agent
  44. 8 8 8 2 The Task Graph s a c

    b d e t 16 2 2 16 The Agent
  45. 2 2 8 2 16 The Task Graph s a

    c b d e t 16 8 8 The Agent
  46. The Task Graph s a c b d e t

    16 2 2 8 8 8 2 16 The Agent
  47. 2 2 The Task Graph s a c b d

    e t 16 8 8 8 2 16 The Agent
  48. 8 2 2 2 The Task Graph s a c

    b d e t 16 8 8 16 The Agent
  49. 8 8 2 2 The Task Graph s a c

    b d e t 16 8 2 16 The Agent
  50. 8 8 2 2 The Task Graph s a c

    b d e t 16 8 2 16 The Agent
  51. 8 8 2 2 The Task Graph s a c

    b d e t 16 8 2 16 The Agent
  52. 8 8 2 2 The Task Graph s a c

    b d e t 16 8 2 16 The Agent Time-consistent agent path s-c-d-t length = 24
  53. 8 8 2 2 The Task Graph s a c

    b d e t 16 8 2 16 The Agent Time-consistent agent path s-c-d-t length = 24 Time-inconsistent agent path s-c-e-t length = 26
  54. 8 8 2 2 The Task Graph s a c

    b d e t 16 8 2 16 The Agent Time-consistent agent path s-c-d-t length = 24 Time-inconsistent agent path s-c-e-t length = 26 Cost Ratio = 26/24
  55. Can formally answer questions like:

  56. Can formally answer questions like: In which scenario will a

    time-inconsistent agent waste the most effort?
  57. Can formally answer questions like: In which scenario will a

    time-inconsistent agent waste the most effort? How do agents with different levels of bias proceed towards the same goal?
  58. Can formally answer questions like: In which scenario will a

    time-inconsistent agent waste the most effort? How do agents with different levels of bias proceed towards the same goal? How do we design a complex task to improve a time- inconsistent agent's efficiency?
  59. Can formally answer questions like: In which scenario will a

    time-inconsistent agent waste the most effort? How do agents with different levels of bias proceed towards the same goal? How do we design a complex task to improve a time- inconsistent agent's efficiency? Worst-case task graph = graph minors Cost ratio = price of irrationality
  60. Can formally answer questions like: In which scenario will a

    time-inconsistent agent waste the most effort? How do agents with different levels of bias proceed towards the same goal? How do we design a complex task to improve a time- inconsistent agent's efficiency? Parametric shortest path problem; how many distinct paths? Bias is continuous in [0, 1]
  61. Can formally answer questions like: In which scenario will a

    time-inconsistent agent waste the most effort? How do agents with different levels of bias proceed towards the same goal? How do we design a complex task to improve a time- inconsistent agent's efficiency? Remove nodes/edges from the graph to improve efficiency. What possible subgraphs support efficient traversal?
  62. Captures a variety of phenomena studied in connection with time-inconsistent

    behaviour
  63. Examples of Behaviour

  64. The Akerlof Example h = 0, n = 5 Procrastination

  65. Abandonment Agent at v, reward r at target t Abandon

    midway if, for all v-t paths P.
  66. Abandonment Agent at v, reward r at target t Abandon

    midway if, for all v-t paths P. Real cost of the first edge of P
  67. Abandonment Agent at v, reward r at target t Abandon

    midway if, for all v-t paths P. Evaluated costs of the rest of the path
  68. Abandonment Agent at v, reward r at target t Abandon

    midway if, for all v-t paths P. Evaluated reward
  69. Choice-Reduction Student takes a 3-week course, has to complete 2

    projects by the end of the course. Week with no project work Cost = 1 Week with work on 1 project Cost = 4 Week with work on 2 projects Cost = 9 For completing the course Reward = 16 Present bias
  70. Choice-Reduction

  71. Choice-Reduction 1 4 9

  72. Choice-Reduction

  73. Choice-Reduction

  74. Choice-Reduction Both projects in the last week of the course

  75. Choice-Reduction

  76. Drop course Choice-Reduction

  77. Choice-Reduction A simple intervention:

  78. Choice-Reduction A simple intervention: Complete first project by the end

    of week 2
  79. Choice-Reduction A simple intervention: Complete first project by the end

    of week 2
  80. Choice-Reduction

  81. Choice-Reduction

  82. Choice-Reduction

  83. It is difficult to reason about the space of possible

    "stories"
  84. It is tractable to reason about the space of possible

    graphs
  85. Bad Cost Ratios

  86. Cost-Ratio

  87. The cost-ratio can be exponential in the number of nodes

    n.
  88. The Akerlof Example h = 0, n = 5

  89. The Akerlof Example

  90. The Akerlof Example

  91. The Akerlof Example

  92. The Akerlof Example

  93. The Akerlof Example

  94. The Akerlof Example

  95. On what kind of graphs can the cost- ratio can

    be exponential in the number of nodes n?
  96. The undirected version of the graph must contain a large

    copy of the Akerlof example as a minor (proof) An unavoidable signature of any graph with an exponentially large cost ratio
  97. Graph Minors

  98. is a minor of Edge deletion Edge contraction Isolated vertex

    deletion
  99. Motivating Subgraphs

  100. Week with no project work Cost = 1 Week with

    work on 1 project Cost = 4 Week with work on 2 projects Cost = 9 For completing the course Reward = 16
  101. Week with no project work Cost = 1 Week with

    work on 1 project Cost = 4 Week with work on 2 projects Cost = 9 For completing the course Reward = 16 Does not affect the agents choice of path, only whether it decides to continue down that path or not.
  102. Week with no project work Cost = 1 Week with

    work on 1 project Cost = 4 Week with work on 2 projects Cost = 9 For completing the course Reward = 16 Actual value is outside of our control
  103. A subgraph G' of G motivates the agent if in

    G' with reward r, the agent reaches the goal node t G' is a motivating subgraph
  104. A motivating subgraph G' contains an s-t path P followed

    by the agent
  105. A motivating subgraph G' contains an s-t path P followed

    by the agent Conjecture: P is also a motivating subgraph
  106. None
  107. None
  108. None
  109. None
  110. None
  111. None
  112. None
  113. None
  114. None
  115. Abandon

  116. None
  117. Abandon

  118. No proper subgraph of G is motivating

  119. Crucial for the agent to initially believe that the upper

    path is an option
  120. Crucial for the agent to initially believe that the upper

    path is an option Real-life analogues?
  121. Minimal Motivating Subgraph G*

  122. How rich a subgraph G' do we necessarily need to

    motivate the agent?
  123. What must a minimal motivating subgraph look like?

  124. What must a minimal motivating subgraph look like? Must be

    sparse (proof)
  125. Open Problems

  126. This Paper

  127. This Paper Graph-theoretic model of tasks

  128. This Paper Graph-theoretic model of tasks Agent constructs a path

    through this graph
  129. This Paper Graph-theoretic model of tasks Agent constructs a path

    through this graph Time-inconsistent behavior of agents in this graph reproduces: procrastination, abandonment, benefits of reduced options
  130. This Paper Graph-theoretic model of tasks Agent constructs a path

    through this graph Time-inconsistent behavior of agents in this graph reproduces: procrastination, abandonment, benefits of reduced options Characterizations of: - Graphs with the worst cost-ratios - Minimal motivation subgraphs
  131. Open Questions

  132. Open Questions Upper bound on the cost-ratio of a given

    graph that does not contain an Akerlof minor?
  133. Open Questions Upper bound on the cost-ratio of a given

    graph that does not contain an Akerlof minor? How hard is it computationally to find minimal motivating subgraphs?
  134. Open Questions Upper bound on the cost-ratio of a given

    graph that does not contain an Akerlof minor? How hard is it computationally to find minimal motivating subgraphs? Can we motivate agents by placing intermediate rewards on specific tasks? "Exploitive" task design; minimize payout Negative rewards
  135. Open Questions Upper bound on the cost-ratio of a given

    graph that does not contain an Akerlof minor? How hard is it computationally to find minimal motivating subgraphs? Can we motivate agents by placing intermediate rewards on specific tasks? "Exploitive" task design; minimize payout Negative rewards Multiple heterogenous present-biased agents?
  136. .