AI Workshop for University Academics

A 2023-2024 Workshop series for University of South Australia academics on creating Assessments in the Age of AI.

Transcript

1. Designing Assessments in the Age of AI
2. Acknowledgement of Country .
3. Learning Objectives Experience and evaluate the abilities of generative AIs Examine assessment design frameworks and theory in the AI era Apply what you’ve learned to your context Level: Ambitious Beginner
4. Workshop prep checklist .
5. Summary of generative AI overview .
6. Most important take-away
7. Questions on AI Basics?
8. Putting assessments to the test
9. Activity 1: Cold Prompting: Finish this sentence with one word… Use the first, least surprising word that comes to mind Would you like a cup of .
10. Cold prompt: short reflective piece Draft a 300-word teaching philosophy statement that addresses the questions to follow. Ensure you use personal anecdotes from your own teaching and learning background. Questions: In your opinion, how do people learn when studying online? What are your goals for your online students? When working with online students, why do you teach the way you do? How do you teach in the online environment? * 4 3.5
11. Getting results .
12. Activity 2: Contextual prompting .
13. Our original “cold prompt” Draft a 300-word teaching philosophy statement that addresses the questions to follow. Ensure you use personal anecdotes from your own teaching and learning background. Questions: In your opinion, how do people learn when studying online? What are your goals for your online students? When working with online students, why do you teach the way you do? How do you teach in the online environment? * 4 3.5 https://bit.ly/35cold https://bit.ly/40cold
14. Our new contextualised prompt You are a 28-year-old male university academic in the engineering faculty of an Australian University, still new to online teaching. You have been a tutor - which means you didn't write the course, but you taught elements of it - such as the face-to-face tutorials - and you marked papers. You are about to teach your first online course, so are taking a four- week short course on teaching online. Draft a 300-word teaching philosophy statement that addresses the questions to follow. The learning objectives you need to demonstrate are: CO3. Reflect on personal experiences, personal values and beliefs toward improving online student engagement. CO4. Propose strategies to create sustainable online courses. Ensure you use personal anecdotes from your own teaching and learning background. Questions: In your opinion, how do people learn when studying online? What are your goals for your online students? When working with online students, why do you teach the way you do? How do you teach in the online environment? 4 3.5 https://bit.ly/350ext https://bit.ly/400ext
15. Your turn…
16. Activity 3A: Assessment Responses 10 minutes
17. Getting results Who What Why Whom How .
18. Activity 3B: Assessment Response Discussion 10 minutes
19. AI Assessment Design Considerations
20. The “OTHER AI” – What work is cheat-worthy? .
21. AI and AI Most Effective* Least Effective* What does S mean? * According to Dawson, P. https://www.aare.edu.au/blog/?p=13627 .
22. TEQSA Two Principles, Three Propositions (Draft document) Assessment in the Age of AI should emphasise 1. Appropriate, authentic engagement with AI 2. A programmatic/systemic approach aligned with discipline and qualification values 3. The process of learning Equip students to ethically use AI and understand its limitations and impacts Multiple, inclusive, contextualised assessments for trustworthy judgements about student learning Principle 1 Principle 2 https://bit.ly/TEQSAdraft
23. AI and Learning Design Advice .
24. 1. Design focused on mitigating AI misuse From “How ChatGPT killed my discussion boards and prompted new prompts” .
25. 2. Designing to Assess humans: Seven Elements of Human Intelligence
26. Assessing Humans: Our intelligences Luckin, Bearman 2018, Korteling et al. 2018 .
27. Exemplar assessment for human intelligences: Critique, reflect, compare, evaluate YEAR 1: PERSONAL EPISTEMOLOGY YEAR 2: SOCIAL INTELLIGENCE, EVALUATIVE JUDGEMENT YEAR 3: SOCIAL INTELLIGENCE, ADVANCED EVALUATIVE JUDGEMENT Luckin, Bearman 2018
28. Three Pedagogical Features that focus on human intelligences Provide rationales for the assessment Ask students to make and later defend evaluative judgements about their work and that of others Ask students how knowledge is constituted in the task at hand .
30. 3. Cognitive offloading 1. Be clear on learning outcomes, acceptable AI use 2. Programmatically offload lower-order tasks 3. Treat capability with cognitive offloading as an outcome 4. Extend development of evaluative judgement to include products of cognitive offloading 5. Use cognitive offloading authentically. Dawson, P. (2020) Translation, Spellchecking, Search .
31. 4. AI Skills/Integrating Generative AI
32. Five Key Skills and a Framework for AI use: Acar Acar, O. (2023) Harvard Business Publishing Education.
33. AI and Learning Design Advice .
34. Redesigning Assessments
35. Blooms and balancing AI assessment design considerations Action AI capability Human intelligences
36. Discussion: Redesigning our assessment Draft a 300-word teaching philosophy statement that addresses the questions to follow. Ensure you use personal anecdotes from your own teaching and learning background. Questions: In your opinion, how do people learn when studying online? What are your goals for your online students? When working with online students, why do you teach the way you do? How do you teach in the online environment? Mitigate misuse of AI Design for human intelligences Cognitive offloading Teach AI skills/AI Affordances 10 minutes
37. Activity 4a: Assessment Re-design 10 minutes
38. Activity 4b: Discussion . 10 minutes
39. . Summary
40. For more information on AI and AI AAIN https://academicintegrity.edu.au/ HE discussions of AI writing https://www.facebook.com/groups/6329308355 01841 Annotated bibliography of AI in HE Undergrad Text-based Assessment design https://bit.ly/abib823 Ethan and Lilach Mollick AI Series https://bit.ly/molAI .