A Mechanistic Account of Constraints on Control-Dependent Processing: Shared Representation, Conflict and Persistence

One of the most fundamental and striking limitations of human cognitive function is the constraint on the number of control-dependent processes that can be executed simultaneously. However, the sources of this capacity constraint remain largely unexplored. Previous work has attributed the constraints on control-dependent processing to the sharing of representations between tasks in neural systems. Here, we examine how shared representations interact with two other factors in producing constraints on control-dependent processing. We first demonstrate that the detrimental effects of shared representations on multitasking performance are contingent on the amount of conflict that is induced by the tasks that share representations. We then examine how the persistence of shared representations between tasks affects processing interference during serial task execution. Finally, we discuss how this set of mechanisms can account for various phenomena in neural architectures, including the psychological refractory period, task switch costs, as well as constraints on cognitive control.

Transcript

1. Sebastian Musslick & Jonathan D. Cohen Princeton Neuroscience Institute Annual Meeting of the Cognitive Science Society 2019 Slides available at: https://speakerdeck.com/musslick
2. Cognitive control – reconfigure information processing away from default (automatic) settings (Cohen et al., 1990; Botvinick & Cohen, 2015)
3. Parse Poster Cognitive control – reconfigure information processing away from default (automatic) settings (Cohen et al., 1990; Botvinick & Cohen, 2015)
4. Cognitive control is limited (Posner & Snyder, 1975; Shiffrin & Schneider, 1977)
5. Cognitive control is limited (Posner & Snyder, 1975; Shiffrin & Schneider, 1977) Capacity Constraints on Control Allocation to Multiple Tasks (Allport, 1980; Meyer & Kieras, 1997; Navon & Gopher, 1979; Salvucci & Taatgen, 2008) Parse Poster 1 Listen to Poster 2
6. Task 1 Task 2
7. Single Channel Theory (Welford, 1952), Broadbent’s Filter Model (Broadbent, 1957, 1958), Central Response Selection Bottleneck (Pashler, 1984, 1994) Task 1 Task 2 Task 1 Central (Response Selection) Bottleneck (1) Single Processing Bottleneck
8. Task 1 Task 2 (Allport, 1972; Allport, 1980; Kinsbourne & Hicks, 1978; Meyer & Kieras, 1997; Navon & Gopher, 1979; Wickens, 1984; Salvucci & Taatgen, 2008; Walley & Weiden, 1973) (2) Multiple Resource Theory Visual Auditory Manual Vocal
9. Task 1 Task 2 (Allport, 1972; Allport, 1980; Kinsbourne & Hicks, 1978; Meyer & Kieras, 1997; Navon & Gopher, 1979; Wickens, 1984; Salvucci & Taatgen, 2008; Walley & Weiden, 1973) (2) Multiple Resource Theory Visual Auditory Manual Vocal
10. (1) Provide a connectionist account of multiple resource theory (2) Investigate the mechanisms in neural architectures that give rise to multitasking limitations
11. (Cohen et al., 1990 ; Feng et al., 2014; Musslick et al., 2016) verbal manual response color word location stimulus internal (hidden) representation
12. verbal manual response color word location stimulus internal (hidden) representation (Cohen et al., 1990 ; Feng et al., 2014; Musslick et al., 2016)
13. color word location verbal manual stimulus internal (hidden) representation response (Cohen et al., 1990 ; Feng et al., 2014; Musslick et al., 2016) control
14. stimulus internal (hidden) representation response color word location verbal manual (Cohen et al., 1990 ; Feng et al., 2014; Musslick et al., 2016) control
15. stimulus internal (hidden) representation response color word location verbal manual multitasking is possible (Cohen et al., 1990 ; Feng et al., 2014; Musslick et al., 2016) control
16. stimulus internal (hidden) representation response color word location verbal manual multitasking is not possible (Cohen et al., 1990 ; Feng et al., 2014; Musslick et al., 2016) control
17. stimulus internal (hidden) representation response color word location verbal manual multitasking is not possible (Cohen et al., 1990 ; Feng et al., 2014; Musslick et al., 2016)
18. stimulus internal (hidden) representation response color word location verbal manual (Cohen et al., 1990 ; Feng et al., 2014; Musslick et al., 2016) control purpose of cognitive control is to limit interference
19. stimulus internal (hidden) representation response color word location verbal manual multitasking is not possible (Cohen et al., 1990 ; Feng et al., 2014; Musslick et al., 2016) control
20. stimulus internal (hidden) representation response color word location verbal manual multitasking is possible (Cohen et al., 1990 ; Feng et al., 2014; Musslick et al., 2016) control
21. stimulus internal (hidden) representation response color word location verbal manual multitasking is possible (Cohen et al., 1990 ; Feng et al., 2014; Musslick et al., 2016)
22. stimulus internal (hidden) representation response color word location verbal manual § Multitasking capability drops drastically with amount of shared representations and is virtually invariant to network size (Feng et al., 2014; Musslick et al., 2016) § Tradeoff between multitasking capability and learning efficiency (Musslick et al., 2017)
23. color word verbal manual
24. color word verbal manual
25. Architecture and Training Environment hidden … a b c i …  x t task stimulus grouped into input dimensions output grouped into response dimensions verbal joystick keyboard color word location
26. Architecture and Training Environment hidden … a b c i …  x t task stimulus grouped into input dimensions output grouped into response dimensions A task defines a one-to-one mapping from a stimulus input dimension to a response dimension verbal joystick keyboard color word location
27. Assessing Multitasking Performance a b c i … …  x t Stimulus a Task Response Associative Layer trained neural network
28. Stimulus Response Associative Layer Assessing Multitasking Performance a b c i … …  x t trained neural network a i Leaky Competitive Accumulator (LCA; Usher & McClelland, 2001) !"#$%&%$' !$ = %)*+$ − !-#"' − %)ℎ%/%$%0) + 2-34-5#%$"$%0) + )0%2- 6-7"6! 6"$- = 8##+6"#' 9:9 + ;: § Response: LCA unit that first reaches threshold* § Reaction Time (RT): Time steps taken to reach threshold* * threshold that maximizes
29. Architecture and Training Environment hidden … a b c i …  x t task stimulus grouped into input dimensions output grouped into response dimensions verbal joystick keyboard color word location A E B C D
30. Architecture and Training Environment hidden … a b c i …  x t task stimulus grouped into input dimensions output grouped into response dimensions verbal joystick keyboard color word location A E B C D
31. A E B C D Task Environment Effects on Dual Task Accuracy
32. A E B C D dual-tasking possible Task Environment Effects on Dual Task Accuracy
33. A E B C D dual-tasking not possible Task Environment Effects on Dual Task Accuracy
34. A E B C D dual-tasking easier? Task Environment Effects on Dual Task Accuracy
35. A E B C D dual-tasking harder? Task Environment Effects on Parallel Processing Accuracy
36. Task Environment 0 0.5 1 % Training on Tasks D and E Compared to Tasks A, B and C 0 50 100 Accuracy (%) A E B C D Effects on Parallel Processing Accuracy
37. Task Environment A E B C D 0 50 100 150 % Training on Tasks D and E Compared to Tasks A, B and C 0 50 100 Accuracy (%) Effects on Parallel Processing Accuracy 0 50 100 150 % Training on Tasks D and E Compared to Tasks A, B and C 0 50 100 Accuracy (%) Peforming Task D alone Performing Task E alone Multitasking Tasks A and C Multitasking Tasks A and B
38. Task Environment A E B C D 0 50 100 150 % Training on Tasks D and E Compared to Tasks A, B and C 0 50 100 Accuracy (%) Effects on Parallel Processing Accuracy 0 50 100 150 % Training on Tasks D and E Compared to Tasks A, B and C 0 50 100 Accuracy (%) Peforming Task D alone Performing Task E alone Multitasking Tasks A and C Multitasking Tasks A and B
39. Task Environment A E B C D 0 50 100 150 % Training on Tasks D and E Compared to Tasks A, B and C 0 50 100 Accuracy (%) Effects on Parallel Processing Accuracy 0 50 100 150 % Training on Tasks D and E Compared to Tasks A, B and C 0 50 100 Accuracy (%) Peforming Task D alone Performing Task E alone Multitasking Tasks A and C Multitasking Tasks A and B
40. (Townsend & Wenger, 2004; Townsend & Altieri, 2012) Predicting Response Time Series Independent Channel Model threshold signal 1 + + noise feedback AND + + noise feedback signal 2
41. Predicting Response Time Series threshold Task A signal + + AND + + Task B signal <=>(:= ≤ $ 8AB :> ≤ $) D%)(<= := ≤ $ , <>(:> ≤ $)) ≤ (upper bound) ≤ <= := ≤ $ + <> := ≤ $ − 1 (lower bound) Inequality by Colonius and Vorberg (1994) (Townsend & Wenger, 2004) := :>
42. Predicting Response Time Series A E B C D A E B C D 0.3 0.4 0.5 0.6 0.7 0.8 Time t in Seconds -1 -0.5 0 0.5 1 Probability of Response before t A + B - 1 min(A, B) shared representations + high conflict shared representations + low conflict 0.3 0.4 0.5 0.6 0.7 0.8 Time t in Seconds -1 -0.5 0 0.5 1 Probability of Response before t A AND B A + B - 1 min(A, B) 0.3 0.4 0.5 0.6 0.7 0.8 Time t in Seconds -1 -0.5 0 0.5 1 Probability of Response before t A AND B A + B - 1 min(A, B)
43. Predicting Response Time Series A E B C D A E B C D separate representations separate representations + dual tasking training 0.3 0.4 0.5 0.6 0.7 0.8 Time t in Seconds -1 -0.5 0 0.5 1 Probability of Response before t A AND C A + C - 1 min(A, C) 0.25 0.3 0.35 0.4 0.45 Time t in Seconds -1 -0.5 0 0.5 1 Probability of Response before t Tasks A and C A AND C A + C - 1 min(A, C)
44. color word verbal manual
45. color word verbal manual (Musslick & Cohen, 2019)
46. … a b c i …  x t stimulus layer response layer associative layer task layer p …persistence t …time integration of net input over time )-$G = 1 − * H )-$G + * H )-$GIJ
47. Psychological Refractory Period (Telford, 1931; Welford, 1952) Task 1 Stimulus 1 Response 1 Processing Task 1 Stimulus 2 Response 2 Processing Task 2 Reaction Time for Task 2 (long SOA) Task 2 Reaction Time for Task 1 SOA (stimulus onset asynchrony) time
48. Psychological Refractory Period (Telford, 1931; Welford, 1952) Task 1 Stimulus 1 Response 1 Processing Task 1 Stimulus 2 Response 2 Processing Task 2 Reaction Time for Task 2 (long SOA) Task 2 Reaction Time for Task 1 SOA (stimulus onset asynchrony) time
49. Psychological Refractory Period (Telford, 1931; Welford, 1952) Task 1 Stimulus 1 Response 1 Processing Task 1 Stimulus 2 Response 2 Processing Task 2 PRP Task 2 SOA Reaction Time for Task 2 (long SOA) Reaction Time for Task 2 (short SOA) time
50. Psychological Refractory Period (Telford, 1931; Welford, 1952) Task 1 Stimulus 1 Response 1 Processing Task 1 Stimulus 2 Response 2 Processing Task 2 PRP Task 2 SOA Reaction Time for Task 2 (long SOA) Reaction Time for Task 2 (short SOA) PRP time
51. Psychological Refractory Period (Telford, 1931; Welford, 1952) Task 1 Stimulus 1 Response 1 Processing Task 1 Stimulus 2 Response 2 Processing Task 2 PRP Task 2 SOA Reaction Time for Task 2 Pashler (1994) time
52. PRP Simulation Procedure a b c i … …  x t Stimulus a Task Response Associative Layer trained neural network 1. Network receives stimulus for first task
53. PRP Simulation Procedure a b c i … …  x t Stimulus a Task Response Associative Layer trained neural network 1. Network receives stimulus for first task 2. Network receives stimulus for second task after SOA
54. Stimulus Response Associative Layer PRP Simulation Procedure a b c i … …  x t trained neural network a i 1. Network receives stimulus for first task 2. Network receives stimulus for second task after SOA 3. Network determines when to respond to second task by maximizing reward rate
55. Psychological Refractory Period (Telford, 1931; Welford, 1952) A (second) E B (first) C D Second Task RT First Task RT 0 2 4 6 8 SOA (s) 0 0.2 0.4 0.6 Reaction Time of Task A (s) 0 2 4 6 8 SOA (s) 0 0.2 0.4 0.6 Reaction Time of Task A (s) 0 2 4 6 8 SOA (s) 0 0.2 0.4 0.6 Reaction Time of Task A (s) 0 2 4 6 8 SOA (s) 0 0.2 0.4 0.6 Reaction Time of Task A (s) 0 5 SOA (s) 0 0.2 0.4 0.6 Reaction Time of Task A (s) Persistence p = 0.9 Persistence p = 0.8 Persistence p = 0.5 Persistence p = 0 0 2 4 6 8 SOA (s) 0 0.2 0.4 0.6 Reaction Time of Task B (s)
56. Psychological Refractory Period (Telford, 1931; Welford, 1952) A (second) E B (first) C D Second Task RT There is no structural bottleneck PRP = strategic delay 0 2 4 6 8 SOA (s) 0 0.2 0.4 0.6 Reaction Time of Task A (s) I.
57. Psychological Refractory Period (Telford, 1931; Welford, 1952) Second Task RT PRP present even if SOA > First Task RT 0 2 4 6 8 SOA (s) 0 0.2 0.4 0.6 Reaction Time of Task A (s) II. 0 2 4 6 8 SOA (s) 0 0.2 0.4 0.6 Reaction Time of Task B (s) First Task RT S1 R1 Task 1 S2 R2 Task 2 SOA time
58. Psychological Refractory Period (Telford, 1931; Welford, 1952) Second Task RT 0 2 4 6 8 SOA (s) 0 0.2 0.4 0.6 Reaction Time of Task A (s) 0 2 4 6 8 SOA (s) 0 0.2 0.4 0.6 Reaction Time of Task B (s) First Task RT (e.g. Marill, 1957) PRP present even if SOA > First Task RT II. Task 1 Task 2 SOA time
59. Psychological Refractory Period (Telford, 1931; Welford, 1952) A (second) E B C (first) D Separate Representation A (second) E B (first) C D Shared Representation 0 2 4 6 8 SOA (s) 0 0.2 0.4 0.6 0.8 Reaction Time of Task A (s) 0 2 4 6 8 SOA (s) 0 0.2 0.4 0.6 0.8 Reaction Time of Task A (s) 0 5 SOA (s) 0 0.2 0.4 0.6 0.8 Reaction Time of Task A (s) Task C First, p = 0.9 Task C First, p = 0.8 Task C First, p = 0.5 Task C First, p = 0 0 5 SOA (s) 0 2 4 6 8 Task B First, p = 0.9 Task B First, p = 0.8 Task B First, p = 0.5 Task B First, p = 0
60. Psychological Refractory Period (Telford, 1931; Welford, 1952) Dual Tasking Training A (second) E B (first) C D 0 2 4 6 8 SOA (s) 0 0.2 0.4 0.6 Reaction Time of Task A (s) 0 5 SOA (s) 0 2 4 6 8 Task B First, p = 0.9 Task B First, p = 0.8 Task B First, p = 0.5 Task B First, p = 0 “virtually perfect time sharing” (Schuhmacher et al., 2001; see also Hazeltine, Teague, and Ivry, 2002; Liepelt, Fischer, French & Schubert, 2009)
61. E Psychological Refractory Period (Telford, 1931; Welford, 1952) B C 0 2 4 6 8 SOA (s) 0 0.2 0.4 0.6 Reaction Time of Task A (s) 0 5 SOA (s) 0 2 4 6 8 Task B First, p = 0.9 Task B First, p = 0.8 Task B First, p = 0.5 Task B First, p = 0 After Single Task Training: High Representational Overlap After Dual Tasking Training: Low Representational Overlap High Sharing Low Sharing A (see Garner & Dux, 2015) D
63. § (a) Sharing of representations, (b) strength of processing and the (c) persistence characteristics of network representations defining a continuum between serial and parallel processing § PRP and task switch costs may reflect a strategic delay, to prevent persisting interference induced by shared representations between tasks § Multitasking constraints may be a rational solution to a fundamental tradeoff between learning efficiency and multitasking capability (see Musslick et al., 2017; Sagiv et al., 2018; Ravi et al., under revision; see also Symposium on Understanding Interactions amongst Cognitive Control, Learning and Representation, Saturday, 14.30-16.10pm) Sharing Conflict Persistence
64. Jonathan Cohen and the Neuroscience of Cognitive Control Laboratory Thank you!