Pro Yearly is on sale from $80 to $50! »

Jess Kadar

Jess Kadar

PS Live 2020 Breakout Session: What do you know? And how do you know you know it? A bit of an Assessments 101 for those working in L&D and Technology.

Ab791babd5242c86c3a21da89b627c2b?s=128

Jess Kadar

October 13, 2020
Tweet

Transcript

  1. None
  2. None
  3. Ways of thinking about assessments Considerations when designing assessments Case

    Study Reading list for further learning and exploration
  4. None
  5. None
  6. None
  7. None
  8. Psychology Statistics Education Has expertise in objective measurement of skills

    and knowledge, abilities, attitudes, personality traits, and educational achievement.
  9. None
  10. Source: Stiggins, Arter, Chappuis, Chappuis (2006) & Earl (2006) learning

    learning learning
  11. learning WHEN After learning has occurred WHY To make a

    statement about student’s learning outside of or at end of learning WHAT Summative, with a score and consequences EXAMPLES Test after a unit or course completed for a grade, SAT, MCAS, A-Levels, etc. Source: Stiggins, Arter, Chappuis, Chappuis (2006) & Earl (2006)
  12. Source: Pluralsight

  13. Source: Pluralsight

  14. learning learning WHEN After learning has occurred While learning is

    still occurring WHY To make a statement about student’s learning outside of or at end of learning To solidify new learning, diagnose needs, next steps, help students feel in control & improve. WHAT Summative, with a score and consequences Formative, data informs next steps EXAMPLES Test after a unit or course completed for a grade, SAT, MCAS, A-Levels, etc. Q&A in class, games, low-stakes quizzes with right/wrong feedback Source: Stiggins, Arter, Chappuis, Chappuis (2006) & Earl (2006)
  15. Source: Pluralsight

  16. Source: Pearson, Masteringphysics.com

  17. Source: Edpuzzle.com

  18. Source: Edpuzzle.com

  19. learning learning learning WHEN After learning has occurred While learning

    is still occurring While students are monitoring their own learning WHY To make a statement about student’s learning outside of or at end of learning To solidify new learning, diagnose needs, next steps, help students feel in control & improve. To promote self-directed learning, deepen understanding WHAT Summative, with a score and consequences Formative, data informs next steps Formative, Self-assessment, peer-assessment EXAMPLES Test after a unit or course completed for a grade, SAT, MCAS, A-Levels, etc. Q&A in class,, games, low-stakes quizzes with right/wrong feedback Rubrics, checklists, self-assessment surveys, goal setting tools Source: Stiggins, Arter, Chappuis, Chappuis (2006) & Earl (2006)
  20. — Robert Stake, PhD Professor Emeritus of Education University of

    Illinois, Urbana-Champaign When the cook tastes the soup, that’s formative... “ ” Source: Soul Pancake https://giphy.com/gifs/soulpancake-food-youtube-6bBH8I1 5EbMPu Assessment learning
  21. — Robert Stake, PhD Professor Emeritus of Education University of

    Illinois, Urbana-Champaign When the cook tastes the soup, that’s formative... When the guests taste the soup, that’s summative. “ ” learning Assessment learning
  22. None
  23. What are we measuring? Why are we measuring it? How

    do we want to measure it? What will we do with the output? job-task analysis (JTA); learning objectives, blueprints, content Timing, format, scoring methodology: length and structure, Frequency, costs Stakes: dictate security, (Proctoring, id validation); consequences for the learner
  24. depends on what you want to measure & why Criterion-referenced

    I want to know if a learner has met a predetermined standard regardless of others Norm-referenced: I want to know where a learner ranks relative to others
  25. Criterion-referenced I want to know if a learner has met

    a predetermined standard regardless of others Norm-referenced: I want to know where a learner ranks relative to others depends on what you want to measure & why
  26. Valid Reliable Fair Defensible

  27. Does it measure what it claims to measure?

  28. Does it measure consistently?

  29. Does it measure equitably?

  30. Did you apply shared standards?

  31. PLAN ANALYZE DESIGN/ REDESIGN SHIP BUILD

  32. Test maker accuses Boston schools of misusing exam. (2020, February

    5). The Boston Globe Organization slams Boston use of its test for exam school admission.(2020, February 5) Commonwealth Magazine
  33. Boston Exam Schools Aligning assessments with the curriculum Boston Exam

    Schools Private Schools Boston Public Schools K-6 MCAS (State) ISEE (Private) ISEE Standards Exam
  34. Cognitive Ability Learning Achievement https://www.erblearn.org/sites/default/files/images/parents/Understanding%20the%20ISEE%20Report_r3.pdf Quantitative Reasoning Math Achievement Verbal

    Reasoning Reading Comprehension
  35. https://www.erblearn.org/sites/default/files/images/parents/Understanding%20the%20ISEE%20Report_r3.pdf Cognitive Ability Achievement

  36. https://www.erblearn.org/sites/default/files/images/parents/Understanding%20the%20ISEE%20Report_r3.pdf Flattening a multi-dimensional analysis into a unidimensional score

  37. Jaggedness is the idea that "we cannot apply one-dimensional thinking

    to understand something that is complex and has many weakly-related dimensions.”
  38. • Assessment products require guidance for interpretation and application •

    Fight the human tendency to simplify complex results: remember the jaggedness principle • Assessments are only part of a bigger landscape of dependencies required to overcome bias and disparities: make sure your organization is on the right side of this work. More in the Boston Globe: The Great Divide
  39. • Pearson Assessment Training Institute’s Classroom Assessment for Student Learning:

    Doing It Right Using It Well, 2nd edition. • The Perfect Assessment System, Rick Stiggins • The End of Average, Todd Rose • Standards for Educational and Psychological Testing, AERA • Handbook of Test Development, Lane, Raymond & Haladyna • The MisMeasure of Man, Stephen Jay Gould • Weapons of Math Destruction, Cathy O’Neill
  40. jess-kadar@pluralsight.com