Share
Embed
Transcript
120 slides
-
Sebastian Musslick Princeton Neuroscience Institute, Princeton University 15.03.19 - Psychiatrie
und Psychotherapie III, Universitätsklinikum Ulm -
Cognitive control – reconfigure information processing away from default (automatic)
settings (Cohen et al., 1990; Botvinick & Cohen, 2015) read email follow talk -
follow talk Constraints on Control Allocation to a Single Task
¡ Costs attached to increases in control signal intensity (Shenhav, Botvinick & Cohen, 2013; Shenhav et al., 2017) -
Cognitive Control is Costly § We make cognitively frugal choices
in choosing how to solve problems (e.g., Simon, 1956; Stanovich & West, 1998; Kahneman, 2003; Todd & Gigerenzer, 2012; Fiske & Taylor, 1991; Gilbert & Hixon, 1991; Petty & Wegener, 1999) cocktail sort merge sort vs. (Lieder et al., 2014) -
Cognitive Control is Costly § We make cognitively frugal choices
in choosing how to solve problems (e.g., Simon, 1956; Stanovich & West, 1998; Kahneman, 2003; Todd & Gigerenzer, 2012; Fiske & Taylor, 1991; Gilbert & Hixon, 1991; Petty & Wegener, 1999) § All else equal, we tend to choose easier over harder tasks (Kool et al., 2010; 2013) (Kool et al., 2010) -
Cognitive Control is Costly § We make cognitively frugal choices
in choosing how to solve problems (e.g., Simon, 1956; Stanovich & West, 1998; Kahneman, 2003; Todd & Gigerenzer, 2012; Fiske & Taylor, 1991; Gilbert & Hixon, 1991; Petty & Wegener, 1999) § All else equal, we tend to choose easier over harder tasks (Kool et al., 2010; 2013) § We are willing to forgo rewards to avoid investing cognitive effort (Westbrook & Braver,2015) Task Difficulty (Westbrook & Braver,2015) -
Why is Cognitive Control Costly? Shenhav et al. (2017) Posner
& Snyder, (1975); Shiffrin & Schneider (1977) -
Why is Cognitive Control Costly? Shenhav et al. (2017) Baumeister
& Heatherton (1996); Holroyd (2015) Posner & Snyder, (1975); Shiffrin & Schneider (1977) -
Why is Cognitive Control Costly? Shenhav et al. (2017) Feng
et al. (2014); Musslick et al., (2016, 2017) Baumeister & Heatherton (1996); Holroyd (2015) Posner & Snyder, (1975); Shiffrin & Schneider (1977) -
How do we weigh the costs and benefits when allocating
cognitive control? -
Weighing the Costs and Benefits of Mental Effort A. Expected
Value of Control Theory 1. The Theory 2. The Model 3. Simulations & Predictions B. Decomposing Individual Differences in Cognitive Control C. Motivation and Cognitive Control in Depression D. Estimating the Cost of Mental Effort from Behavior -
Weighing the Costs and Benefits of Mental Effort A. Expected
Value of Control Theory 1. The Theory 2. The Model 3. Simulations & Predictions B. Decomposing Individual Differences in Cognitive Control C. Motivation and Cognitive Control in Depression D. Estimating Mental Effort from Behavior -
A Theory of Control Allocation: Expected Value of Control GREEN
EVC(signal,state) = Pr(outcome i i ∑ | signal,state)⋅Value(outcome i ) # $ % & ' (−Cost(signal) Shenhav, Botvinick, & Cohen (2013) -
A Theory of Control Allocation: Expected Value of Control Control
Signal Intensity -
A Theory of Control Allocation: Expected Value of Control Control
Signal Intensity Payoff -
A Theory of Control Allocation: Expected Value of Control Control
Signal Intensity Payoff Cost -
A Theory of Control Allocation: Expected Value of Control Control
Signal Intensity Payoff Cost EVC -
A Theory of Control Allocation: Expected Value of Control Control
Signal Intensity Payoff Cost EVC -
Weighing the Costs and Benefits of Mental Effort A. Expected
Value of Control Theory 1. The Theory 2. The Model 3. Simulations & Predictions B. Decomposing Individual Differences in Cognitive Control C. Motivation and Cognitive Control in Depression D. Estimating the Cost of Cognitive Control from Behavior -
Computational Model How is Control Implemented? Agent Task Environment Drift
Diffusion Model (DDM; Ratcliff, 1978; Bogacz; 2006) t 0 “red” “green” accumulated evidence drift = driftCONTROL + driftAUTOMATIC -
Computational Model How is Control Implemented? Agent Task Environment Drift
Diffusion Model (DDM; Ratcliff, 1978; Bogacz; 2006) t 0 “red” “green” accumulated evidence drift = driftCONTROL + driftAUTOMATIC -
Computational Model How is Control Implemented? Agent Task Environment Drift
Diffusion Model (DDM; Ratcliff, 1978; Bogacz; 2006) t 0 “red” “green” accumulated evidence AUTOMATIC drift = driftCONTROL + ! WORD + ! COLOR -
Computational Model How is Control Implemented? Agent Task Environment Drift
Diffusion Model (DDM; Ratcliff, 1978; Bogacz; 2006) t 0 “red” “green” accumulated evidence AUTOMATIC drift = driftCONTROL + ! WORD + ! COLOR -
Computational Model How is Control Implemented? Agent Task Environment Drift
Diffusion Model (DDM; Ratcliff, 1978; Bogacz; 2006) t 0 “red” “green” accumulated evidence AUTOMATIC drift = ! WORD ·" WORD + ! COLOR ·" COLOR + " WORD + " COLOR CONTROL -
Computational Model How is Control Implemented? Agent Task Environment Drift
Diffusion Model (DDM; Ratcliff, 1978; Bogacz; 2006) t 0 “red” “green” accumulated evidence drift = ! WORD ·" WORD + ! COLOR ·" COLOR + " WORD + " COLOR drift = · + · + + -
drift = ! WORD ·" WORD + ! COLOR ·"
COLOR + " WORD + " COLOR drift = · + · 1 + + 1 Computational Model How is Control Implemented? Agent Task Environment Drift Diffusion Model (DDM; Ratcliff, 1978; Bogacz; 2006) t 0 “red” “green” accumulated evidence -
drift = ! WORD ·" WORD + ! COLOR ·"
COLOR + " WORD + " COLOR drift = · + · 1 + + 1 Computational Model How is Control Implemented? Agent Task Environment Drift Diffusion Model (DDM; Ratcliff, 1978; Bogacz; 2006) t 0 “red” “green” accumulated evidence -
drift = ! WORD ·" WORD + ! COLOR ·"
COLOR + " WORD + " COLOR drift = ·(−10)+ · 1 + (−10) + 1 Computational Model How is Control Implemented? Agent Task Environment Drift Diffusion Model (DDM; Ratcliff, 1978; Bogacz; 2006) t 0 “red” “green” accumulated evidence -
drift = ! WORD ·" WORD + ! COLOR ·"
COLOR + " WORD + " COLOR drift = 0 ·(−10)+ 0 · 1 + (−10) + 1 Computational Model How is Control Implemented? Agent Task Environment Drift Diffusion Model (DDM; Ratcliff, 1978; Bogacz; 2006) t 0 “red” “green” accumulated evidence -
drift = ! WORD ·" WORD + ! COLOR ·"
COLOR + " WORD + " COLOR drift = 0 ·(−10)+ 0 · 1 + (−10) + 1 Computational Model How is Control Implemented? Agent Task Environment Drift Diffusion Model (DDM; Ratcliff, 1978; Bogacz; 2006) t 0 “red” “green” accumulated evidence drift = −9 t 0 -
drift = ! WORD ·" WORD + ! COLOR ·"
COLOR + " WORD + " COLOR drift = 0 ·(−10)+ 0 · 1 + (−10) + 1 Computational Model How is Control Implemented? Agent Task Environment Drift Diffusion Model (DDM; Ratcliff, 1978; Bogacz; 2006) t 0 “red” “green” accumulated evidence drift = -
drift = ! WORD ·" WORD + ! COLOR ·"
COLOR + " WORD + " COLOR drift = 0 ·(−10)+ 15 · 1 + (−10) + 1 Computational Model How is Control Implemented? Agent Task Environment Drift Diffusion Model (DDM; Ratcliff, 1978; Bogacz; 2006) t 0 “red” “green” accumulated evidence drift = -
drift = ! WORD ·" WORD + ! COLOR ·"
COLOR + " WORD + " COLOR drift = 0 ·(−10)+ 15 · 1 + (−10) + 1 Computational Model How is Control Implemented? Agent Task Environment Drift Diffusion Model (DDM; Ratcliff, 1978; Bogacz; 2006) t 0 “red” “green” accumulated evidence drift = 6 -
drift = ! WORD ·" WORD + ! COLOR ·"
COLOR + " WORD + " COLOR Computational Model How is Control Implemented? Agent Task Environment Drift Diffusion Model (DDM; Ratcliff, 1978; Bogacz; 2006) t 0 “red” “green” accumulated evidence #(%&''()*|,-., 0) 23(%&''()*|,-., 0) drift = 0 ·(−10)+ 15 · 1 + (−10) + 1 drift = 6 -
Computational Model How is Control Allocated? Agent Task Environment t
0 “red” “green” accumulated evidence Control Implementation Musslick, Shenhav, Botvinick & Cohen (2015, RLDM) Control Allocation -
Computational Model How is Control Allocated? Agent Task Environment t
0 “red” “green” accumulated evidence Control Implementation Musslick, Shenhav, Botvinick & Cohen (2015, RLDM) ! WORD = 0 ! COLOR = 15 t 0 1. Simulate performance Control Allocation -
Computational Model How is Control Allocated? Agent Task Environment t
0 “red” “green” accumulated evidence Control Implementation Musslick, Shenhav, Botvinick & Cohen (2015, RLDM) ! WORD = 0 ! COLOR = 15 t 0 2. Compute Expected Value of Control (EVC) 1. Simulate performance &'( !, * +,- = .(0122345| * +,-, 7) · :3;<2= − 01?5(7) Control Allocation -
Computational Model How is Control Allocated? Agent Task Environment t
0 “red” “green” accumulated evidence Control Implementation Musslick, Shenhav, Botvinick & Cohen (2015, RLDM) ! WORD = 0 ! COLOR = 15 t 0 2. Compute Expected Value of Control (EVC) 1. Simulate performance &'( !, * +,- = .(0122345| * +,-, 7) · :3;<2= − 01?5(7) Control Allocation -
Computational Model How is Control Allocated? Agent Task Environment t
0 “red” “green” accumulated evidence Control Implementation Musslick, Shenhav, Botvinick & Cohen (2015, RLDM) ! WORD = 0 ! COLOR = 15 t 0 2. Compute Expected Value of Control (EVC) 1. Simulate performance Control Allocation &'( !, * +,- = .(0122345| * +,-, 7) · :3;<2= − 01?5(7) -
Computational Model How is Control Allocated? Agent Task Environment t
0 “red” “green” accumulated evidence Control Implementation Musslick, Shenhav, Botvinick & Cohen (2015, RLDM) !WORD = 0 !COLOR = 15 t 0 2. Compute Expected Value of Control (EVC) 1. Simulate performance Control Allocation &'()*'*+,-,./+ 0/1,(!) ! 4*5/+6.7!8-,./+ 0/1,(!) 9! + ;<= !, ? @+A = B(0/88*5,| ? @+A, D) · 4*F-8G − 0/1,(D) -
Computational Model How is Control Allocated? Agent Task Environment t
0 “red” “green” accumulated evidence Control Implementation Musslick, Shenhav, Botvinick & Cohen (2015, RLDM) ! WORD = 0 ! COLOR = 15 t 0 2. Compute Expected Value of Control (EVC) 1. Simulate performance Control Allocation &'( !, * +,- = .(0122345| * +,-, 7) · :3;<2= − 01?5(7) 3. Select control signal that maximizes EVC 7∗ = max D &'( * +,-, 7 -
Computational Model How is Control Allocated? Agent Task Environment t
0 “red” “green” accumulated evidence Control Implementation Musslick, Shenhav, Botvinick & Cohen (2015, RLDM) ! WORD = 0 ! COLOR = 15 t 0 2. Compute Expected Value of Control (EVC) 1. Simulate performance Control Allocation &'( !, * +,- = .(0122345| * +,-, 7) · :3;<2= − 01?5(7) 3. Select control signal that maximizes EVC 7∗ = max D &'( * +,-, 7 -
Weighing the Costs and Benefits of Mental Effort A. Expected
Value of Control Theory 1. The Theory 2. The Model 3. Simulations & Predictions B. Decomposing Individual Differences in Cognitive Control C. Motivation and Cognitive Control in Depression D. Estimating the Cost of Cognitive Control from Behavior -
Cognitive Control Phenomena Basic Phenomena Incongruency Costs (Stroop, 1935) Switch
Costs (Allport, 1994) Effects of Incentives on Task Performance Distractor Interference/Facilitation (Padmala & Pessoa) Reward & Switch Costs (Umemoto & Holroyd, 2015) Effects of Incentives on Task Choice Demand Avoidance (Kool et al., 2010) Cognitive Effort Discounting (Westbrook & Braver, 2015) Reward and Voluntary Task Switching (Arrington & Braun, 2017) Adaptation to Task Difficulty Congruency Sequence Effect (Gratton, 1992) Proportion Congruency Effect (Logan & Zbrodoff, 1979) Non-Monotonicity in Task Engagement (Gilzenrat, 2010) Stability-Flexibility Tradeoff (Goschke, 2000) -
Cognitive Control Phenomena Basic Phenomena Incongruency Costs (Stroop, 1935) Switch
Costs (Allport, 1994) Effects of Incentives on Task Performance Distractor Interference/Facilitation (Padmala & Pessoa) Reward & Switch Costs (Umemoto & Holroyd, 2015) Effects of Incentives on Task Choice Demand Avoidance (Kool et al., 2010) Cognitive Effort Discounting (Westbrook & Braver, 2015) Reward and Voluntary Task Switching (Arrington & Braun, 2017) Adaptation to Task Difficulty Congruency Sequence Effect (Gratton, 1992) Proportion Congruency Effect (Logan & Zbrodoff, 1979) Non-Monotonicity in Task Engagement (Gilzenrat, 2010) Stability-Flexibility Tradeoff (Goschke, 2000) -
Effects of Incentives on Task Performance Padmala & Pessoa (2011)
litation Reward ward Interference Fascilitation -0.1 -0.05 0 0.05 0.1 Reaction time (s) No Reward Reward litation Reward ward Interference Fascilitation -0.15 -0.1 -0.05 0 0.05 0.1 P(Error) No Reward Reward No Reward Reward 0 2 4 6 Control Signal Intensity Picture Control Signal Word Control Signal Picture Word 0 0.5 1 1.5 Reward-driven Change in Control Signal Intensity Picture Control Signal Word Control Signal Data -
Effects of Incentives on Task Performance Padmala & Pessoa (2011)
litation Reward ward Interference Fascilitation -0.1 -0.05 0 0.05 0.1 Reaction time (s) No Reward Reward litation Reward ward Interference Fascilitation -0.15 -0.1 -0.05 0 0.05 0.1 P(Error) No Reward Reward No Reward Reward 0 2 4 6 Control Signal Intensity Picture Control Signal Word Control Signal Picture Word 0 0.5 1 1.5 Reward-driven Change in Control Signal Intensity Picture Control Signal Word Control Signal Data -
Effects of Incentives on Task Performance Padmala & Pessoa (2011)
litation Reward ward Interference Fascilitation -0.1 -0.05 0 0.05 0.1 Reaction time (s) No Reward Reward litation Reward ward Interference Fascilitation -0.15 -0.1 -0.05 0 0.05 0.1 P(Error) No Reward Reward No Reward Reward 0 2 4 6 Control Signal Intensity Picture Control Signal Word Control Signal Picture Word 0 0.5 1 1.5 Reward-driven Change in Control Signal Intensity Picture Control Signal Word Control Signal Data Interference Fascilitation -0.1 -0.05 0 0.05 0.1 Reaction time (s) No Reward Reward Interference Fascilitation -0.1 -0.05 0 0.05 0.1 Reaction time (s) No Reward Reward No Reward Re 0 2 4 6 Control Signal Intensity Picture Contr Word Contro EVC Model -
Effects of Incentives on Task Performance Padmala & Pessoa (2011)
litation Reward ward Interference Fascilitation -0.1 -0.05 0 0.05 0.1 Reaction time (s) No Reward Reward litation Reward ward Interference Fascilitation -0.15 -0.1 -0.05 0 0.05 0.1 P(Error) No Reward Reward No Reward Reward 0 2 4 6 Control Signal Intensity Picture Control Signal Word Control Signal Picture Word 0 0.5 1 1.5 Reward-driven Change in Control Signal Intensity Picture Control Signal Word Control Signal Data Interference Fascilitation -0.1 -0.05 0 0.05 0.1 Reaction time (s) No Reward Reward Interference Fascilitation -0.1 -0.05 0 0.05 0.1 Reaction time (s) No Reward Reward No Reward Re 0 2 4 6 Control Signal Intensity Picture Contr Word Contro EVC Model rference Fascilitation No Reward Reward Interference Fascilitation -0.1 -0.05 0 0.05 0.1 Reaction time (s) No Reward Reward rference Fascilitation No Reward Reward Interference Fascilitation -0.15 -0.1 -0.05 0 0.05 0.1 P(Error) No Reward Reward No Reward Reward 0 2 4 6 Control Signal Intensity Picture Control Signal Word Control Signal Picture Word 0 0.5 1 1.5 Reward-driven Change in Control Signal Intensity -
Effects of Incentives on Task Choice Arrington & Braun (2018)
Braun & Arrington (2018) Current Value Constant Current Value Decrease 0 0.2 0.4 0.6 0.8 1 Probability of Task Switches Other Value Constant Other Value Increase Data -
Effects of Incentives on Task Choice Arrington & Braun (2018)
Braun & Arrington (2018) Current Value Constant Current Value Decrease 0 0.2 0.4 0.6 0.8 1 Probability of Task Switches Other Value Constant Other Value Increase Data -
Effects of Incentives on Task Choice Arrington & Braun (2018)
Braun & Arrington (2018) Current Value Constant Current Value Decrease 0 0.2 0.4 0.6 0.8 1 Probability of Task Switches Other Value Constant Other Value Increase Data -
Effects of Incentives on Task Choice Arrington & Braun (2018)
Braun & Arrington (2018) Current Value Constant Current Value Decrease 0 0.2 0.4 0.6 0.8 1 Probability of Task Switches Other Value Constant Other Value Increase Data EVC Model Current Value Constant Current Value Decrease 0 0.2 0.4 0.6 0.8 1 Probability of Task Switches Other Value Constant Other Value Increase Braun & Arrington (2018) Current Value Constant Current Value Decrease 0 0.2 0.4 0.6 0.8 1 Probability of Task Switches Other Value Constant Other Value Increase EVC Model -
Effects of Incentives on Task Choice Arrington & Braun (2018)
Braun & Arrington (2018) Current Value Constant Current Value Decrease 0 0.2 0.4 0.6 0.8 1 Probability of Task Switches Other Value Constant Other Value Increase EVC Model Current Value Constant Current Value Decrease 0 0.2 0.4 0.6 0.8 1 Probability of Task Switches Other Value Constant Other Value Increase Braun & Arrington (2018) Current Value Constant Current Value Decrease 0 0.2 0.4 0.6 0.8 1 Probability of Task Switches Other Value Constant Other Value Increase Data EVC Model EVC Model Current Value Constant Current Value Decrease 0.2 0.4 0.6 0.8 1 Expected Value of Switching Other Value Constant Other Value Increase -
0 1 Trial Relative To Escape 0 0.5 1 Accuracy
(%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Accuracy (%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity Adaptation to Task Difficulty Gilzenrat et al. (2010) Task Difficulty Trial Escape -
0 1 Trial Relative To Escape 0 0.5 1 Accuracy
(%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Accuracy (%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity Adaptation to Task Difficulty Gilzenrat et al. (2010) Task Difficulty Trial Escape Data -
0 1 Trial Relative To Escape 0 0.5 1 Accuracy
(%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Accuracy (%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity Adaptation to Task Difficulty Gilzenrat et al. (2010) Task Difficulty Trial Escape -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Accuracy (%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity Data EVC Model -
0 1 Trial Relative To Escape 0 0.5 1 Accuracy
(%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Accuracy (%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity Adaptation to Task Difficulty Gilzenrat et al. (2010) Task Difficulty Trial Escape -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Accuracy (%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity 3 4 ape -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points 3 4 ape -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity -
0 1 Trial Relative To Escape 0 0.5 1 Accuracy
(%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Accuracy (%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity Adaptation to Task Difficulty Gilzenrat et al. (2010) Task Difficulty Trial Escape -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 Accuracy -4 0 5 10 15 Point -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 0 0.5 1 Control Signal Intensity -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Accuracy (%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity 3 4 ape -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points 3 4 ape -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity Human Participants -
0 1 Trial Relative To Escape 0 0.5 1 Accuracy
(%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Accuracy (%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity Adaptation to Task Difficulty Gilzenrat et al. (2010) Task Difficulty Trial Escape -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 Accuracy -4 0 5 10 15 Point -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 0 0.5 1 Control Signal Intensity -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Accuracy (%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0.45 0.5 0.55 Pupil Dilation (mm) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity 3 4 ape -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 10 20 Points 3 4 ape -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Control Signal Intensity -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 0.5 1 Accuracy (%) -4 -3 -2 -1 0 1 2 3 4 Trial Relative To Escape 0 5 10 15 20 25 Expected Value of Control 0.55 ) 1 EVC Model Human Participants -
Weighing the Costs and Benefits of Mental Effort A. Expected
Value of Control Theory 1. The Theory 2. The Model 3. Simulations & Predictions B. Motivation and Cognitive Control in Depression C. Decomposing Individual Differences in Cognitive Control D. Estimating the Cost of Cognitive Control from Behavior -
Cognitive Control and Depression § Depression is characterized by impairments
in attention, memory, and cognitive control (Millan et al., 2012, Snyder, 2013 ) § Origins of cognitive control deficits remains poorly understood § Most of the existing models view cognitive control deficits in depression as the reduced ability to exert control (for a review see: Grahek et al., 2018) § Existing accounts are descriptive, lacking a mechanistic understanding -
Cognitive Control and Depression Grahek, Shenhav, Musslick, Krebs & Koster
(under review) Control Intensity -
Cognitive Control and Depression Grahek, Shenhav, Musslick, Krebs & Koster
(under review) Control Intensity -
Cognitive Control and Depression Grahek, Shenhav, Musslick, Krebs & Koster
(under review) Control Intensity -
Cognitive Control and Depression Grahek, Shenhav, Musslick, Krebs & Koster
(under review) Control Intensity -
Cognitive Control and Depression Grahek, Shenhav, Musslick, Krebs & Koster
(under review) Control Intensity -
Cognitive Control and Depression Grahek, Shenhav, Musslick, Krebs & Koster
(under review) Control Intensity -
Cognitive Control and Depression Grahek, Shenhav, Musslick, Krebs & Koster
(under review) Control Intensity -
Cognitive Control and Depression Grahek, Shenhav, Musslick, Krebs & Koster
(under review) Control Intensity -
Cognitive Control and Depression - Control Efficacy - Grahek, Shenhav,
Musslick, Krebs & Koster (under review) BLUE BLUE vs. Cognitive Effort Discounting (Wesbtoork & Braver, 2015) -
Cognitive Control and Depression Grahek, Shenhav, Musslick, Krebs & Koster
(under review) Control Intensity -
Cognitive Control and Depression - Reward Sensitivity - Grahek, Shenhav,
Musslick, Krebs & Koster (under review) BLUE BLUE vs. Cognitive Effort Discounting (Wesbtoork & Braver, 2015) -
Cognitive Control and Depression Grahek, Shenhav, Musslick, Krebs & Koster
(under review) Control Intensity -
Cognitive Control and Depression - Control Cost - Grahek, Shenhav,
Musslick, Krebs & Koster (under review) BLUE BLUE vs. Cognitive Effort Discounting (Wesbtoork & Braver, 2015) -
Weighing the Costs and Benefits of Mental Effort A. Expected
Value of Control Theory 1. The Theory 2. The Model 3. Simulations & Predictions B. Motivation and Cognitive Control in Depression C. Decomposing Individual Differences in Cognitive Control D. Estimating the Cost of Cognitive Control from Behavior -
Decomposing Individual Differences in Cognitive Control Can we reliably measure
a person’s capacity to exert control? -
Decomposing Individual Differences in Cognitive Control Musslick, Cohen & Shenhav
(under review) EVC Agents Cognitive Control Capacity Task Automaticity Control Cost Learning Rate Reward Sensitivity -
Decomposing Individual Differences in Cognitive Control Musslick, Cohen & Shenhav
(under review) Task Performance -
Decomposing Individual Differences in Cognitive Control Musslick, Cohen & Shenhav
(under review) (Sequential) Adaptation -
Decomposing Individual Differences in Cognitive Control Musslick, Cohen & Shenhav
(under review) BLUE BLUE vs. Stroop Task Overall Error Rate -
Decomposing Individual Differences in Cognitive Control Musslick, Cohen & Shenhav
(under review) BLUE BLUE vs. Stroop Task Overall Error Rate -
Decomposing Individual Differences in Cognitive Control Musslick, Cohen & Shenhav
(under review) BLUE BLUE vs. Stroop Task Overall Error Rate -
Decomposing Individual Differences in Cognitive Control Musslick, Cohen & Shenhav
(under review) BLUE BLUE vs. Stroop Task Overall Error Rate -
Decomposing Individual Differences in Cognitive Control Musslick, Cohen & Shenhav
(under review) BLUE BLUE vs. Stroop Task Overall Error Rate -
Decomposing Individual Differences in Cognitive Control Musslick, Cohen & Shenhav
(under review) BLUE BLUE vs. Overall Error Rate Stroop Task -
Decomposing Individual Differences in Cognitive Control Musslick, Cohen & Shenhav
(under review) BLUE BLUE vs. Overall Error Rate Stroop Task -
Decomposing Individual Differences in Cognitive Control Musslick, Cohen & Shenhav
(under review) BLUE BLUE vs. Stroop Task Congruency Effect (Stroop Effect) Stroop (1935) -
Decomposing Individual Differences in Cognitive Control Musslick, Cohen & Shenhav
(under review) BLUE BLUE vs. Stroop Task Congruency Sequence Effect Gratton (1992) -
Decomposing Individual Differences in Cognitive Control Musslick, Cohen & Shenhav
(under review) BLUE BLUE vs. Stroop Task Logan & Zbrodoff (1979) Proportion Congruency Effect -
Decomposing Individual Differences in Cognitive Control Can we reliably measure
a person’s capacity to exert control? Not really. -
Decomposing Individual Differences in Cognitive Control Which effects best index
the capacity for cognitive control? Explained by amount of exerted control -
Weighing the Costs and Benefits of Mental Effort A. Expected
Value of Control Theory 1. The Theory 2. The Model 3. Simulations & Predictions B. Motivation and Cognitive Control in Depression C. Decomposing Individual Differences in Cognitive Control D. Estimating the Cost of Cognitive Control from Behavior -
Why Estimate the Cost of Cognitive Control? ¡ Exerting cognitive
control is costly (Botvinick & Braver, 2015; Shenhav et al., 2017) ¡ The cost of cognitive control imposes limitations on task performance (Kool et al., 2010) ¡ Individual differences in the cost of control explain behavior more generally in the real world and are linked to clinical symptoms (Westbrook, Kester & Braver, 2013; Gold et al., 2016) Low predictive validity for individual differences in control cost estimates (conversations with L. Bustamanete, W. Kool, C. Sayali & A. Westbrook, 2017-now) ! -
I. Compute EVC for every control signal !"# $, &
= ( #)**+,- $, & " #)**+,- − #)/-($) $ … ,)3-*)4 /56374 53-+3/5-8 S … ,$**+3 /-7-+ -
I. Compute EVC for every control signal probability of correct
outcome ! "# $, & '()*+(, &-.)/, $ 01' $, & = ! '(++34* $, & 1 '(++34* − '(6*($) $ … 4()*+(, 6-.)/, -)*3)6-*: S … 4$++3) 6*/*3 -
I. Compute EVC for every control signal probability of correct
outcome ! "# $, & '()*+(, &-.)/, $ subjective value "001+12 314/+2 5 "# 65' $, & = ! '(++18* $, & 5 '(++18* − '(:*($) $ … 8()*+(, :-.)/, -)*1):-*> S … 8$++1) :*/*1 -
I. Compute EVC for every control signal probability of correct
outcome ! "# $, & '()*+(, &-.)/, $ subjective value "001+12 314/+2 5 "# Cost of Cognitive Control '(6*($) '()*+(, &-.)/, $ 95' $, & = ! '(++1;* $, & 5 '(++1;* − '(6*($) $ … ;()*+(, 6-.)/, -)*1)6-*> S … ;$++1) 6*/*1 -
I. Compute EVC for every control signal II. Select control
signal that maximizes EVC probability of correct outcome ! "# $, & '()*+(, &-.)/, $ subjective value "001+12 314/+2 5 "# Cost of Cognitive Control '(6*($) '()*+(, &-.)/, $ $∗ = max > ?5' $, & ?5' $, & = ! '(++1@* $, & 5 '(++1@* − '(6*($) -
!"# $, & = ( #)**+,- $, & " #)**+,-
− #)/-($) Estimating the Cost of Cognitive Control from Performance 2!"# $, & 2$ = 2( #)**+,- $, & 2$ 2" #)**+,- 2$ − 2#)/-($) 2$ 0 = 2( #)**+,- $∗, & 2$∗ 2" #)**+,- 2$∗ − 2#)/-($∗) 2$∗ 2#)/-($∗) 2$∗ = 2( #)**+,- $∗, & 2$∗ " #)**+,- Differentiate Consider cost for optimal control signal Solve for derivative of control cost -
Cost Estimation !"#$%"& '()#*& + , -. +, ' Estimating
the Cost of Cognitive Control from Performance: Theoretical Validation Simulated Agent accuracy , -. +, ' !"#$%"& '()#*& + subjective value -001%12 314*%2 5 -. cost !"6$(+) !"#$%"& '()#*& + +∗ = max > ?5! +, ' Task Environment RED RED RED $ $$ $$$ Agents perform a control-demanding task (e.g. Stroop) with a fixed task difficulty under varying reward for correct response $ $$ $$$ 2, !"%%1@$ +∗, ' 2+∗ 5 !"%%1@$ = 2!"6$(+∗) 2+∗ cost !"6$(+) !"#$%"& '()#*& + -
Estimating the Cost of Cognitive Control from Performance: Theoretical Validation
cost !"#$(&) !"($)"* +,-(.* & cost !"#$(&) !"($)"* +,-(.* & True Control Cost Function Estimated Control Cost Function ? -
Estimating the Cost of Cognitive Control from Performance Under Perfect
Knowledge 0 0.5 1 Control Signal Intensity u 0 1 2 3 4 5 6 Exponential (True) Exponential (Estimate) Quadratic (True) Quadratic (Estimate) Linear (True) Linear (Estimate) -
Estimating the Cost of Cognitive Control from Performance Under Perfect
Knowledge 0 0.2 0.4 0.6 0.8 1 Control Signal Intensity 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Probability Of Rewarded Outcome True Measured -
Cost Estimation !"#$%"& '()#*& + , -. +, ' Estimating
the Cost of Cognitive Control from Performance Under Imperfect Knowledge Simulated Agent accuracy , -. +, ' !"#$%"& '()#*& + subjective value -001%12 314*%2 5 -. cost !"6$(+) !"#$%"& '()#*& + +∗ = max > ?5! +, ' Task Environment RED RED RED $ $$ $$$ Agents perform a control-demanding task (e.g. Stroop) with a fixed task difficulty under varying reward for correct response $ $$ $$$ 2, !"%%1@$ +∗, ' 2+∗ 5 !"%%1@$ = 2!"6$(+∗) 2+∗ cost !"6$(+) !"#$%"& '()#*& + -
Estimating the Cost of Cognitive Control from Performance Under Imperfect
Knowledge ! "#$$%&' (, * = ⁄ 1 (1 + %0102) task automaticity 2 Bob Alice 0 0.2 0.4 0.6 0.8 1 Control Signal Intensity 0.2 0.4 0.6 0.8 Accuracy Alice Bob Assumed 0 0.2 0.4 0.6 0.8 1 Control Signal Intensity u 0 2 4 6 8 10 Control Costs Alice (True) Alice (Estimate) Bob (True) Bob (Estimate) 0 0.2 0.4 0.6 0.8 1 Control Signal Intensity u 0 2 4 6 8 10 Control Costs Alice (True) Alice (Estimate) Bob (True) Bob (Estimate) -
Estimating the Cost of Cognitive Control from Performance Under Imperfect
Knowledge ! "#$$%&' = )* "#$$%&' + , reward sensitivity ) Bob Alice 0 0.2 0.4 0.6 0.8 1 Control Signal Intensity u 0 2 4 6 8 10 Control Costs Alice (True) Alice (Estimate) Bob (True) Bob (Estimate) 0 20 40 60 80 100 Reward ($) 0 0.5 1 1.5 Subjective Value Alice Bob Assumed 0 0.2 0.4 0.6 0.8 1 Control Signal Intensity u 0 2 4 6 8 10 Control Costs Alice (True) Alice (Estimate) Bob (True) Bob (Estimate) -
Estimating the Cost of Cognitive Control from Performance Under Imperfect
Knowledge ! "#$$%&' = )* "#$$%&' + , accuracy bias , Bob Alice 0 0.2 0.4 0.6 0.8 1 Control Signal Intensity u 0 2 4 6 8 10 Control Costs Alice (True) Alice (Estimate) Bob (True) Bob (Estimate) 0 20 40 60 80 100 Reward ($) 0 1 2 3 4 5 6 7 Subjective Value Alice Bob Assumed 0 0.2 0.4 0.6 0.8 1 Control Signal Intensity u 0 2 4 6 8 10 Control Costs Alice (True) Alice (Estimate) Bob (True) Bob (Estimate) -
Estimating the Cost of Cognitive Control from Performance Under Imperfect
Knowledge Validity of estimated costs: Correlation between true cost of control and estimated cost of control unaccounted variability with respect to task automaticity ~ 0 5 10 Standard Deviation of Task Automaticity a 0 0.5 1 Correlation Between True and Estimated Control Costs -
Estimating the Cost of Cognitive Control from Performance Under Imperfect
Knowledge Validity of estimated costs: Correlation between true cost of control and estimated cost of control 0 5 10 Standard Deviation of Task Automaticity a 0 0.5 1 Correlation Between True and Estimated Control Costs 0 5 10 Standard Deviation of Reward Sensitivity v 0 0.5 1 Correlation Between True and Estimated Control Costs 0 5 10 Standard Deviation of Accuracy Bias b 0 0.5 1 Correlation Between True and Estimated Control Costs task automaticity reward sensitivity accuracy bias -
Estimating the Cost of Cognitive Control from Performance Under Imperfect
Knowledge Paradigm A Paradigm B Proxy A for Cost of Cognitive Control Proxy B for Cost of Cognitive Control ? 0 0.5 1 Correlation Between Reward Sensitivity v Across Experiments -1 -0.5 0 0.5 1 Correlation Between Control Costs Across Experiments True Correlation reward sensitivity -
Estimating the Cost of Cognitive Control from Performance Under Imperfect
Knowledge Paradigm A Paradigm B Proxy A for Cost of Cognitive Control Proxy B for Cost of Cognitive Control ? 0 0.5 1 Correlation Between Reward Sensitivity v Across Experiments -1 -0.5 0 0.5 1 Correlation Between Control Costs Across Experiments True Correlation 0 0.5 1 Correlation Between Task Automaticity a Across Experiments -1 -0.5 0 0.5 1 Correlation Between Control Costs Across Experiments True Correlation 0 0.5 1 Correlation Between Accuracy Bias b Across Experiments -1 -0.5 0 0.5 1 Correlation Between Control Costs Across Experiments True Correlation task automaticity reward sensitivity accuracy bias -
A Theory of Control Allocation: Expected Value of Control GREEN
EVC(signal,state) = Pr(outcome i i ∑ | signal,state)⋅Value(outcome i ) # $ % & ' (−Cost(signal) Shenhav, Botvinick, & Cohen (2013)