1 Integrating LLMs into Database Systems Education Kishore Prakash, Shashwat Rao, Rayan Hamza, Jack Lukich, Vatsal Chaudhari, Arnab Nandi 

LLM-based services are taking over everything 4 

LLM-based services are taking over education 5 

LLMs taking over education 6 • Initial Reaction: ban immediately! • “New Calculator”… “Plagiarism” • Detect and penalize • Understandable: Assignments and Exams • Synthesis and Essay Questions • Multiple Choice Questions: B+ • Unsupervised / Take-homes? 

“Banning ChatGPT” is not an option 7 • Too late: Pervasive use, variants • Readying students for an AI-enabled future • Onus is on educators to discover how to integrate LLMs into educational infrastructure 

Where does an LLM fit into the education landscape? 8 

Class Roles: Where does an LLM fit in? 9 • Instructor • Teaching Assistant • Textbook • Teaching Tools / Software / Autograder • Tutor 

Intuition behind “Tutor” 10 with infinite resources, what would we give every student? a personal tutor who assists the student in their learning journey 

Our Vision: DB Tutor 11 • Provide the students with an LLM-powered chat-based interface that prioritizes personalized learning • Leverage opportunities that are unique to database systems • Building such a system will take some thought and iteration 

Why LLMs are not the best fit 12 • LLMs are designed and trained to get to the right answer as quickly and efficiently as possible • Getting to the right answer without explanations can impede learning` 

DB Tutor: Challenges 13 • Bias in Responses • Students’ over-reliance, critical thinking • Cheating and Misuse • Data Privacy and Security • Sensitivity to prompting 

Challenge: Bias in Responses 14 • LLMs have an inherent bias issue • Training data bias • Recency bias • Demographics bias • Use in learning: amplified effects • Fix training data, or model output 

Challenge: Over-Reliance, Critical Thinking 15 • High convenience = pervasive use • Long-term dependency • Loss of independent skills • Impedes deeper understanding • Loss of critical thinking (especially ability to notice LLM errors) 

Challenge: Cheating and Misuse 16 • “Super Tool” for Misuse • Easy to generate human-sounding content • Essay questions, multiple choice • Are take-home assignments still an option? • Detection is an arms-race • Previous Disruptions • Web search, Wikipedia, Calculators 

Envisioned System Architecture 17 LLM INFRASTRUCTURE (So0ware and data we will set up) ! Course Materials Syllabus, Slides, Tests " LLM Llama v2 or GPT4 via API Virtual Tutor Portal (What the student interacts with) # Learning Outcomes Report $ Chatbot % Database SQLite DBMS Virtual Tutor Engine (So9ware we will build in this research ac;vity) & Data Analysis Engine ' Prompt Engineering 

Elements of a DB Tutor 18 • Can we go beyond “ChatGPT for Database Education?” • What are some gaps we can fill? 

Elements of a DB Tutor 19 • Implicit Query Execution • Data Personalization • Learning Outcomes Report • Visual Step Throughs • Pop Quizzes 

Implicit Query Execution: NL 2 SQL 20 • LLMs hallucinate; let’s pipe all generated code against a runtime (Google Bard) • DBTutor: Before queries are shared with student, execute it against a sandboxed DB • Generate Synthetic Data and Schema • Use results (or errors) to improve query and explanations • Prompt: “What are some possible errors to anticipate with this query?” SQLite Prompt ⚡ SQL Annotated SQL Result Student 

Data Personalization 21 • Students are more engaged when examples are personalized • Use LLMs to generate sample data that they can relate to Travis Kelsey (American Football) Queries Taylor Swift (Music) Queries 

Learning Outcomes Report 22 Case Studies and Applications Entity-Relationship (ER) Model ER-to-Relational Model Relational Algebra Relational Calculus Functional Dependencies and Normalization SQL Object Relational Databases Embedded SQL Graphical User Interfaces Indexing and Query Optimization XML Active Databases Concurrency and Transaction Management ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ Keep track and share what the student is learning; rewrite prompts to highlight gaps or assume knowledge 

Takeaways • Standard LLMs are not designed for education and pose several challenges • Many unique integration opportunities in database systems education • LLM-powered “DB Tutor” that prioritizes student learning 23 

24 Thank you