Jess Kadar

PS Live 2020 Breakout Session: What do you know? And how do you know you know it? A bit of an Assessments 101 for those working in L&D and Technology.

Transcript

3. Ways of thinking about assessments Considerations when designing assessments Case Study Reading list for further learning and exploration
8. Psychology Statistics Education Has expertise in objective measurement of skills and knowledge, abilities, attitudes, personality traits, and educational achievement.
10. Source: Stiggins, Arter, Chappuis, Chappuis (2006) & Earl (2006) learning learning learning
11. learning WHEN After learning has occurred WHY To make a statement about student’s learning outside of or at end of learning WHAT Summative, with a score and consequences EXAMPLES Test after a unit or course completed for a grade, SAT, MCAS, A-Levels, etc. Source: Stiggins, Arter, Chappuis, Chappuis (2006) & Earl (2006)
12. Source: Pluralsight
13. Source: Pluralsight
14. learning learning WHEN After learning has occurred While learning is still occurring WHY To make a statement about student’s learning outside of or at end of learning To solidify new learning, diagnose needs, next steps, help students feel in control & improve. WHAT Summative, with a score and consequences Formative, data informs next steps EXAMPLES Test after a unit or course completed for a grade, SAT, MCAS, A-Levels, etc. Q&A in class, games, low-stakes quizzes with right/wrong feedback Source: Stiggins, Arter, Chappuis, Chappuis (2006) & Earl (2006)
15. Source: Pluralsight
16. Source: Pearson, Masteringphysics.com
17. Source: Edpuzzle.com
18. Source: Edpuzzle.com
19. learning learning learning WHEN After learning has occurred While learning is still occurring While students are monitoring their own learning WHY To make a statement about student’s learning outside of or at end of learning To solidify new learning, diagnose needs, next steps, help students feel in control & improve. To promote self-directed learning, deepen understanding WHAT Summative, with a score and consequences Formative, data informs next steps Formative, Self-assessment, peer-assessment EXAMPLES Test after a unit or course completed for a grade, SAT, MCAS, A-Levels, etc. Q&A in class,, games, low-stakes quizzes with right/wrong feedback Rubrics, checklists, self-assessment surveys, goal setting tools Source: Stiggins, Arter, Chappuis, Chappuis (2006) & Earl (2006)
20. — Robert Stake, PhD Professor Emeritus of Education University of Illinois, Urbana-Champaign When the cook tastes the soup, that’s formative... “ ” Source: Soul Pancake https://giphy.com/gifs/soulpancake-food-youtube-6bBH8I1 5EbMPu Assessment learning
21. — Robert Stake, PhD Professor Emeritus of Education University of Illinois, Urbana-Champaign When the cook tastes the soup, that’s formative... When the guests taste the soup, that’s summative. “ ” learning Assessment learning
23. What are we measuring? Why are we measuring it? How do we want to measure it? What will we do with the output? job-task analysis (JTA); learning objectives, blueprints, content Timing, format, scoring methodology: length and structure, Frequency, costs Stakes: dictate security, (Proctoring, id validation); consequences for the learner
24. depends on what you want to measure & why Criterion-referenced I want to know if a learner has met a predetermined standard regardless of others Norm-referenced: I want to know where a learner ranks relative to others
25. Criterion-referenced I want to know if a learner has met a predetermined standard regardless of others Norm-referenced: I want to know where a learner ranks relative to others depends on what you want to measure & why
26. Valid Reliable Fair Defensible
27. Does it measure what it claims to measure?
28. Does it measure consistently?
29. Does it measure equitably?
30. Did you apply shared standards?
31. PLAN ANALYZE DESIGN/ REDESIGN SHIP BUILD
32. Test maker accuses Boston schools of misusing exam. (2020, February 5). The Boston Globe Organization slams Boston use of its test for exam school admission.(2020, February 5) Commonwealth Magazine
33. Boston Exam Schools Aligning assessments with the curriculum Boston Exam Schools Private Schools Boston Public Schools K-6 MCAS (State) ISEE (Private) ISEE Standards Exam
34. Cognitive Ability Learning Achievement https://www.erblearn.org/sites/default/files/images/parents/Understanding%20the%20ISEE%20Report_r3.pdf Quantitative Reasoning Math Achievement Verbal Reasoning Reading Comprehension
35. https://www.erblearn.org/sites/default/files/images/parents/Understanding%20the%20ISEE%20Report_r3.pdf Cognitive Ability Achievement
36. https://www.erblearn.org/sites/default/files/images/parents/Understanding%20the%20ISEE%20Report_r3.pdf Flattening a multi-dimensional analysis into a unidimensional score
37. Jaggedness is the idea that "we cannot apply one-dimensional thinking to understand something that is complex and has many weakly-related dimensions.”
38. • Assessment products require guidance for interpretation and application • Fight the human tendency to simplify complex results: remember the jaggedness principle • Assessments are only part of a bigger landscape of dependencies required to overcome bias and disparities: make sure your organization is on the right side of this work. More in the Boston Globe: The Great Divide
39. • Pearson Assessment Training Institute’s Classroom Assessment for Student Learning: Doing It Right Using It Well, 2nd edition. • The Perfect Assessment System, Rick Stiggins • The End of Average, Todd Rose • Standards for Educational and Psychological Testing, AERA • Handbook of Test Development, Lane, Raymond & Haladyna • The MisMeasure of Man, Stephen Jay Gould • Weapons of Math Destruction, Cathy O’Neill
40. jess-kadar@pluralsight.com