Assessment in a Common Core World

This presentation constitutes the confidential information of MetaMetrics, Inc. Copyright
© 2013 MetaMetrics, Inc. All rights reserved. Assessment in a Common Core World May 17, 2013 Malbert Smith III, Ph.D. President and Co-founder, MetaMetrics Research Professor, UNC School of Education

© 2013 MetaMetrics, Inc. All rights reserved. Assessment Landscape Over Time 1970 to 2000 Pre-NCLB Dominated by Standardized Norm- Referenced Assessments (Big Three) 2001 to 2014 NCLB Dominated by State Specific Assessments (TAKS, FCAT, NCEOG) 2014 - Forward Common Core (Next Generation Assessments) Two Testing Consortia SBAC- Computer Adaptive PARCC- Computer Delivered

© 2013 MetaMetrics, Inc. All rights reserved. Pre-NCLB Period “If you would examine these books and the best of the achievement and intelligence tests then available, you might be surprised that so little progress has been made in the past fifty years—in fact, in some areas we are not doing as well. Except for the tremendous advances in electronic scoring, analysis, and reporting of test results, we don’t have a great deal to show for fifty years of work. Essentially, achievement tests are being constructed today in the same way they were fifty years ago—the major changes being the use of more sophisticated statistical procedures for doing what we did then—mistakes and all” [p. 10]. (Buros, 1977).

© 2013 MetaMetrics, Inc. All rights reserved. Widespread Confusion about the Meaning of “Growth” • In the 670-page text of NCLB, “achievement,” “progress,” “learning,” “growth,” “development,” “performance,” and “proficiency,” appeared over 1,600 times interchangeably, with no distinction between status and growth. 6

© 2013 MetaMetrics, Inc. All rights reserved. Common Core World • SBAC and PARCC will embody many of the principles of Next Generation Assessment • Designed to eliminate problems inherent in NCLB Assessments • “Growth” vs. “Status” • Designed to measure and support student learning

© 2013 MetaMetrics, Inc. All rights reserved. • “Really Quetelet…has not adduced a law for the development of human individuals; for he tried to discover in a twinkling that which takes nature 20 years and more.” --J.W.H. Lehmann (1844, p.300, his italics; quoted by Tanner, 2010, p.140) 8

© 2013 MetaMetrics, Inc. All rights reserved. “If we can dramatically increase high school graduation rates, if we can dramatically increase the number of graduates who are college and career ready, that’s what this is about. Everything’s a means to that end. That’s the Holy Grail here. Are our students being prepared to be successful?” -Arne Duncan Education Week, December 9, 2009

© 2013 MetaMetrics, Inc. All rights reserved. Quick Facts • Each year, approximately 1.2 million students fail to graduate from high school, more than half of whom are from minority groups. • Percent of freshmen that enroll in at least one remedial course Community College Four-Year Institution 42% 20% Alliance for Excellent Education, February 2009 edition.

© 2013 MetaMetrics, Inc. All rights reserved. Students who enroll in a remedial reading course are 41 percent more likely to drop out of college (NCES, 2004a) 58% 17% 0% 10% 20% 30% 40% 50% 60% 70% No Remedial Course(s) Remedial Course(s) Students Obtaining Bachelor’s Degree in Eight Years Alliance for Excellent Education, February 2009 edition.

© 2013 MetaMetrics, Inc. All rights reserved. $52,671 $36,645 $26,933 $17,299 $0 $10,000 $20,000 $30,000 $40,000 $50,000 $60,000 High School Dropout High School Diploma Associate's Degree Bachelor's Degree Alliance for Excellent Education, February 2009 edition. Average Income by Educational Attainment

© 2013 MetaMetrics, Inc. All rights reserved. The English Language Arts Standards: Key Changes (Shifts) 1) Text Complexity 2) Balance of fiction/non-fiction 3) Analysis, inference, and evidence 4) Mastery of writing and speaking 5) Academic Vocabulary Goal is College and Career Readiness

© 2013 MetaMetrics, Inc. All rights reserved. Common Core Standards for English Language Arts and Literacy in History/Social Studies & Science Appendix A: Findings • Students who fall short of ACT's college readiness benchmarks have the greatest difficulty with the test items involving the most complex text. • K-12 reading assignments have become much less demanding in the last half-century, with an especially large drop-off in high school expectations. Weston, S. P. (2010). “The giant text complexity challenge inside the new literacy standards.” The Prichard Committee for Academic Excellence

© 2013 MetaMetrics, Inc. All rights reserved. Common Core Standards for English Language Arts and Literacy in History/Social Studies & Science Appendix A: Findings • College reading assignments have moved in the opposite direction, becoming a bit harder over the same fifty years. • High school teachers commonly give students many kinds of support and coaching to help them figure out the material, but college teachers expect students to pull the knowledge from the text on their own, making the gap in practical ability even wider than the gap in the texts themselves.

© 2013 MetaMetrics, Inc. All rights reserved. What is Assessment? Working Definition: All the activities – formal and informal – educators use to guide instruction and to gauge student progress and learning.

© 2013 MetaMetrics, Inc. All rights reserved. Why Do We Need Assessment? • Monitor learning • Gauge student progress • Determine instructional effectiveness • Guide instruction • Differentiate instruction

© 2013 MetaMetrics, Inc. All rights reserved. Assessment Types • Summative • Administered at the end of a course/class • Used to measure learning outcomes • Assessment of learning • Examples: NCLB assessments, EOC/EOG assessments

© 2013 MetaMetrics, Inc. All rights reserved. Assessment Types: Ways to Compare • Criterion-Referenced Assessment • Measured against defined criteria • Used to establish competence, e.g. a driving test • Student is measured against a range of explicit criteria

© 2013 MetaMetrics, Inc. All rights reserved. Assessment Types: Ways to Compare • Norm-Referenced Assessment • Relative to a group taking the assessment, e.g. peers • Allows for comparisons across students. • Standards may vary from year to year depending on performance of the cohort

© 2013 MetaMetrics, Inc. All rights reserved. Assessment: Three Points to Remember • Standard Error of Measurement • Internal/external factors • Growth • Test-specific bias • Equating Item Type with Construct • Don’t confuse the how you measure with what you measure • Assessment item types are proxies for measuring the attribute in question • Lack of Unification • Too many assessments using different non- exchangeable scales to measure same construct, e.g. healthcare, science

© 2013 MetaMetrics, Inc. All rights reserved. Mitigate Summer Learning Loss Fairchild, R. McLaughlin, B. & Brady, J. (2006). Making the Most of Summer: A Handbook on Effective Summer Programming and Thematic Learning. Baltimore, MD: Center for Summer Learning.

© 2013 MetaMetrics, Inc. All rights reserved. Next Generation Assessments Important Features • Blending of assessment and instruction – it’s possible to ‘mine the exhaust’ of the instructional experience for assessment data • Computer adaptive assessments can be applied to instructional content, e.g. test items are targeted to the individual • Assessment engines can connect day-to-day progress with year-to-year summative tests by reporting on common developmental scales “Whatever exists at all, exists in some amount” - E.L. Thorndike

© 2013 MetaMetrics, Inc. All rights reserved. Next Generation Assessments Important Features • Perspectives and monitoring can be longitudinal across the developmental lifespan of the student for each construct and used to track college and career readiness • The focus is ‘student-centric’ by focusing on the critical components of skill acquisition: targeted practice, corrective feedback, intensive practice, distributed practice, and opportunity for self- directed practice

© 2013 MetaMetrics, Inc. All rights reserved. Coming Together to Raise Achievement New Assessments for the Common Core State Standards Updated April 2012 (p.15) Each Consortium committed to build an assessment system for Grades 3-8 and high school that meets the following criteria2: • Builds upon shared standards in mathematics and English language arts (ELA) for college- and career-readiness; • Measures individual growth as well as proficiency; • Measures the extent to which each student is on track, at each grade level tested, toward college or career readiness by the time of high school completion and; • Provides information that is useful in informing: • Teaching, learning, and program improvement; • Determinations of school effectiveness; • Determinations of principal and teacher effectiveness for use in evaluations and the provision of support to teachers and principals; and • Determinations of individual student college- and career-readiness, such as determinations made for high school exit decisions, college course placement to credit-bearing classes, or college entrance. 2 US Department of Education Race to the Top Assessment Program Application for New Grants: Comprehensive Assessment Systems. CFDA Number 84.395B. 2009

© 2013 MetaMetrics, Inc. All rights reserved. "The college instructor blames the high school teacher, the high school teacher complains of the grade teacher, each grade teacher above first grade finds fault with the poor work of the teacher in the grade below, and the first grade teacher in turn is chagrined at the shortcomings of the home training. Must this go on indefinitely? Whose opinion shall prevail? Is it not possible to get away from personal opinion to an agreed-upon consensus of opinion? May we not replace the constantly conflicting subjective standards with definitely defined objective standards?"

© 2013 MetaMetrics, Inc. All rights reserved. Q: Explain one of the processes by which water can be made safe to drink. A: Flirtation makes water safe to drink because it removes large pollutants like grit, sand, dead sheep and canoeists Q&A

© 2013 MetaMetrics, Inc. All rights reserved. Contact Info: Malbert Smith III, Ph.D. President and Co-founder, MetaMetrics Research Professor, UNC [email protected] www.metametricsinc.com www.Lexile.com www.Quantiles.com

© 2013 MetaMetrics, Inc. All rights reserved. For more information about Lexile and Quantile measures, visit www.Lexile.com and www.Quantiles.com. For more information about MetaMetrics, visit www.MetaMetricsInc.com. MetaMetrics 1000 Park Forty Plaza Drive, Suite 120 Durham, North Carolina 27713 Phone: 919.547.3400 Toll Free: 1.888.LEXILES (539.4537) Email: [email protected] 50

Assessment in a Common Core World

Assessment in a Common Core World

More Decks by MetaMetrics

Other Decks in Education

Featured

Transcript