When Mind Meets Models: Reimagining Critical Thinking in The GenAI Era

Transcript

1. IT SLOAN SCHOO F MANAGEMENT When Mind Meets Models: Reimagining Critical Thinking in The GenAI Era Deeksha Gupta Fall 2025 UROP under Professor Melissa Webster
2. Introducing the Speaker Deeksha Gupta Wellesley College’ 26 Psychology and Economics
3. 3 Acknowledgements Special thank you to Melissa Webster, my research team- Coco, Emery, and Taylor- and my Fall 2025 interviewees! MIT Professor Eric So and Thank you to Nataliya Kosmyna et al. @MIT Media Lab for their insights from their paper: Your Brain on ChatGPT (2025)
4. 4 Introduction
6. 6 Defining critical thinking “Critical thinking is the process of analyzing available facts, evidence, observations, and arguments to make sound conclusions or informed choices.” - Wikipedia “The act or practice of thinking critically (as by applying reason and questioning assumptions) in order to solve problems, evaluate information, discern biases, etc.” - Merriam Webster
7. AGENDA 1. Research questions & methodology 2. Findings 3. Implications 4. Conclusion and Future Directions for Research 5. Q&A
8. 8 1 Research Question and Methodology
9. Research Question How do users of generative AI perceive its effects on their critical thinking, and what strategies do they use to maintain it?
10. Research Methodology Primary Sources: 8 interviews conducted with current undergraduate students, post-grad students, and people in the workforce Secondary Sources: 1. Research Papers 2. News Articles 3. Opinion Pieces GenAI Use: Finding research papers and assisting with analysing qualitative data from interviews
11. 11 2 Findings
12. Initial Findings ➔ Shifts Thinking ➔ Over‑Reliance ➔ Adaptation ➔ Role of Errors
13. Across interviews: ➔ Participants consistently described using GenAI to accelerate or automate early‑stage, low‑level work. ➔ They now spend more of their time on evaluation, synthesis, or decision-making. ➔ “So while I may say that I'm not using my brain enough, I think I'm using my mind or things that really matter, where I act, where I add value.” Research study by Lee et al. showed that: ➔ Knowledge workers report that GenAI often reduces the perceived effort of critical thinking, especially when they’re highly confident in the AI. However, those who are confident in their own abilities are more likely to keep doing genuine critical work on top of AI output. Research study by Nasr et al. showed that: ➔ Collaborative human‑AI supported use where students still do the heavy lifting of interpretation is associated with deeper cognitive presence and critical thinking phases, compared to letting the AI do all the work. 13 GenAI boosts efficiency and shifts focus to higher ‑order thinking Source: - The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers - Exploring the Impact of Generative AI ChatGPT on Critical Thinking in Higher Education: Passive AI-Directed Use or Human–AI Supported Collaboration?
14. Across interviews: ➔ Many participants felt that AI encouraged “mental laziness.” ➔ A large subset explicitly described AI making them less inclined to engage deeply with tasks they once did manually. Research by Kosmyna et al. at the MIT Media Lab showed that: ➔ Repeated use of ChatGPT for essay‑writing creates “cognitive debt.” This gives the feeling that AI is dulling their mental edge. - 83.3% of LLM users were unable to quote even one sentence from the essay they had just written. - In contrast, 88.9% of Search and Brain-only users could quote accurately. 14 GenAI can reduce deep thinking and foster over‑reliance Source: Overview ‹ Your Brain on ChatGPT — MIT Media Lab
15. 15 “It's made me lazier by a lot, like it's made me not want to do the basic mundane tasks, and I feel like I'm losing the sight of [all I'm losing] the importance of those mundane tasks because they're there for a reason.” - Industrial Design Student “I can't deny that it's made me a little lazy.” - Founder of an AI Company
16. Across interviews: ➔ To preserve or enhance critical thinking, users deliberately define boundaries for how AI should (and should not) be used. ➔ “...you would still want to be in touch with yourself a little bit more. You would not want to lose yourself.” Research study by Zhou et al. showed that: ➔ Perceived usefulness of GenAI predicts better critical thinking only when students exercise strong self‑regulation (planning, monitoring, evaluating their use). Research study by Drosos et al. showed that: ➔ Injecting short “provocations” (critiques / alternative viewpoints) into AI workflows help knowledge workers do more metacognitive and critical thinking. 16 GenAI can lead users to set boundaries and protective workflows Source: - The Mediating Role of Generative AI Self-Regulation on Students’ Critical Thinking and Problem-Solving - [2501.17247] "It makes you think": Provocations Help Restore Critical Thinking to AI-Assisted Knowledge Work
17. 17 “...that was my first experience of actually having believed what it said, and then being tricked by it, and then that made me very, very aware immediately after that.” - Founder of an AI Company
18. Across interviews: ➔ Almost every interview contained an instance where an AI error made them change the way they interact with the tool. ➔ Errors increased their metacognitive awareness and improved their verification habits. Research study by Guo and Lee showed that: ➔ In a chemistry course activity with 29 students, a three‑stage ChatGPT task (generate → verify & correct → revise) led to a 29% gain in confidence to analyze information and make logical conclusions and explicitly used hallucinated or incorrect references as triggers for fact‑checking. 18 GenAI errors encourage verification and sharper critical awareness Source: Leveraging ChatGPT for Enhancing Critical Thinking Skills
19. New Findings ➔ Limited Prior Reflection ➔ “Fully Engaged” ➔ Emergent Awareness ➔ Concerned Realization
20. Across interviews: ➔ Most participants reported that they had not previously engaged in explicit conversations about what critical thinking is or how AI might influence it. ➔ “Honestly, the mental side of things, I haven’t really given it a lot of thought at all because it hasn’t particularly bothered me. But that’s not a good enough excuse to not think about.” Research study by Cookke et al. showed that: ➔ Faculty see potential for AI to support critical thinking but lack training, indicating that explicit conversations about “what critical thinking is in an AI world” are still emerging. 20 Users reported limited prior reflection on critical thinking Source: Empowering Minds: Cultivating Critical Thinking in the Age of AI
21. Across interviews: ➔ Participants commonly asserted that they remain “100% engaged” or mentally present when using generative AI. ➔ “I would not say I am one of those who get too deeply involved in it and like using it for like, every single thought or thing that I have.” Similar research finding by Lee et al. showed that: ➔ Higher confidence in GenAI is associated with less self‑reported critical‑thinking effort, even as knowledge workers say they are “engaging critically” with the tool. 21 Users reported strong self ‑perception of being “fully engaged” Source: The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers
22. Across interviews: ➔ During the Interview itself several participants became newly aware- sometimes mid‑conversation- that their AI dependence was higher than they had realized. ➔ “And I don't really give it a second thought, but now that you ask that question, I have now started to give it a second thought, and my times will take a lot longer with searches. So thank you for that.” Research study by Xiao et al. observed that: This area of AI metacognitive awareness is under‑researched and it uses reflective journals + interviews to help students recognise how and why they rely on AI. 22 Users reported an emergent awareness got triggered during the interview itself Source: Students’ development of AI metacognitive awareness in an EAP course: A qualitative inspection through the Experiential Learning Theory - ScienceDirect
23. Across interviews: ➔ Many participants expressed worry upon recognizing how much cognitive work they routinely delegate to AI, whether that was during the interview or in their own time recently. ➔ “I started, like, in the two years of using AI, relying on AI for doing the critical thinking work for me. And I kind of, like had that reflection moment of like, oh, I might be using it way too much.” ➔ “I haven't given the mental side of things a lot of thought, because that's not been one of my top priorities, but it it really should be… It's kind of scary.” ➔ We did not find any research done on this which supports our finding. 23 Users’ realization often carried a negative or concerned tone
24. 24 3 Implications
25. Implications Individually 25 ➔ Time saved helps only if users practice active self-regulation. ➔ Learn when offloading helps vs. harms thinking. ➔ New behaviours (e.g., critique-prompting) show evolving cognition. Organizationally ➔ Keep key judgment tasks human for growth. ➔ Set clear norms for acceptable AI use. ➔ Build AI tools with prompts that nudge reflection. ➔ Set boundaries (assistant, not replacement). Doing some “grunt work” themselves to maintain their own skills and voice. Educationally ➔ Design tasks that require interpretation and justification. ➔ Teach AI literacy as cognitive strategy, not tool use. ➔ Redefine critical thinking for AI-mediated work.
26. 26 4 Conclusion and Future Directions for Research
27. Conclusion Initial: ➔ Shifts Thinking ➔ Over‑Reliance ➔ Adaptation ➔ Role of Errors New: ➔ Limited Prior Reflection ➔ “Fully Engaged” ➔ Emergent Awareness ➔ Concerned Realization 27
28. Future Directions for Research 28 ➔ What makes ‘Aha!’ moments? Do some conversations have a more sustained behavioural change than others? ➔ How does the question tone/wording about critical thinking influence interview response? ➔ How can we quantify the gap between perceived engagement and actual cognitive effort? ➔ What happens if we embed short reflective prompts into AI tools? ➔ What organizational policies can be developed? ➔ What are different intervention designs depending on: ◆ Which tasks you use AI for ◆ Things do once you’re using it
29. 29 5 Q&A