2019/2020 Product Roadmap Unveiling

This presentation constitutes the confidential information of MetaMetrics, Inc. Copyright
© 2019 MetaMetrics, Inc. All rights reserved. MM0185 Alistair Van Moere, Ph.D. Chief Product Officer MetaMetrics Product Roadmap Trilby Berger SVP, Strategic Partnerships MAY 2019

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 Presenters Alistair Van Moere, Ph.D., Chief Product Officer Alistair drives innovation in assessments and helps organizations make sense of measurement and test scores. Before joining MetaMetrics in 2017, Alistair was president of Pearson’s Knowledge Technologies group, where he managed artificial intelligence scoring services for speaking and writing. He also oversaw development and delivery of standardized assessments for millions of learners. Alistair has worked as a teacher, examiner, director of studies, university lecturer and test developer, in the U.S., U.K., Japan and Thailand. He has an MA in English Language Teaching, Ph.D. in Language Testing, and an MBA. He is frequently invited to speak at conferences and has authored over 20 research publications with a focus on educational technology and language assessments. Trilby Berger, Senior Vice President, Strategic Partnerships Trilby oversees all domestic business development for Lexile products and services. Previously, Trilby launched our global effort including expansion into Korea and Japan where learning the English language are national priorities. Her work has included managing professional development, directing marketing and inking many partnerships ranging from product development, licensing, publisher signings and highly visible marketing partnerships with the likes of Barnes & Noble and Amazon. Prior to joining the organization in 2003, Trilby served as marketing and communications manager for All Kinds of Minds from 1998 to 2003.

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 The market requires a literacy construct that is much wider than silent reading alone. • Vocabulary • Oral Reading • Listening • Writing How will the Lexile Framework meet that demand? Lexile Roadmap — 2019 Onwards

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 What proportion of 4th grade students recognized the meaning of these words? o 75% or more o 51% - 75% o Less than 50% brilliant NAEP Vocabulary Assessment just knowingly www.nationsreportcard.gov/reading_2013/vocabulary/#knowledge

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 1. The vocabulary learning task is enormous – 3,000-4,000 words a year. 2. There are more words to learn than we can possibly teach – yes, but we still have to teach them! 3. Children raised in disadvantaged environments enter school with smaller vocabularies. Michael Graves The Vocabulary Book: Learning and Instruction

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 Decoding Listening Comprehension Reading Comprehension Translating text into pronounceable words Understanding words spoken aloud Understanding text WEAK AVERAGE “Classic Dyslexic” WEAK AVERAGE “Poor Comprehender” WEAK “Garden Variety” Poor Reader WEAK Gough & Tunmer, 1986; Adlof, Catts, &Little, 2006 The “Simple Model” provides evidence that reading comprehension consists of primarily of decoding and listening comprehension. Decoding & Listening Comprehension

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 As readers develop, the importance of decoding decreases, and the importance of listening comprehension increases. Authors Students Correlation: oral to silent reading Kim & Wagner, 2015 Grades 1-4 (n=316) grade 1 = 0.76 grade 4 = 0.60 Jenkins et al, 2000 Grade 4 (n=113) 0.65 Fuchs et al, 2000 Grade 4 (n=365) 0.80 Variance associated with reading comprehension Catts, Hogan, and Adlof (2005); Garcia & Cain, 2013 Decoding & Listening Comprehension

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 Poor comprehenders may have intact word decoding capabilities, but their “hidden impairment” may go unnoticed until later grades when they are faced with more complex academic texts.

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 • Listening is tested in 22 state assessments ◦ 8.8m students ◦ Listening is 20% of the ELA exam • Some states are reporting it out separately on their parent reports (CA, CT) • ELL exams (e.g. WIDA, ELPAC, TELPAS, AZELLA) all report listening (4.5m students) The Growing Prominence of Listening Assessments

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 Starting 2020, MetaMetrics’ new scales will address and measure the whole literacy construct through silent reading, oral reading and listening, with a writing measure to follow.

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 As content developers, you need to: • Create grade-level content • Ensure coverage of key vocabulary • Identify academic words across domains • Align vocabulary instruction and assessment What are the Intended Uses?

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 List Author Type Source Academic Word List (AWL) Coxhead (2000) General College textbooks Academic Vocabulary List (AVL) Gardner & Davies (2014) General Corpus of Contemporary American English (COCA) Marzano Marzano (2004) Academic State and national standards documents Middle School Academic Vocabulary Lists (MSAV) Greene & Coxhead (2015) General and Academic Middle School textbooks Most are not derived from texts read by K-12 students! The word lists come from word frequency counts as derived from corpora that adults read. Limitation With Existing Academic Word Lists

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 List Author Type Source Academic Word List (AWL) Coxhead (2000) General College textbooks Academic Vocabulary List (AVL) Gardner & Davies (2014) General Corpus of Contemporary American English (COCA) Marzano Marzano (2004) Academic State and national standards documents Middle School Academic Vocabulary Lists (MSAV) Greene & Coxhead (2015) General and Academic Middle School textbooks Lexile WordBank MetaMetrics General and Academic Landscape Corpus 144 contemporary best-selling K-12 textbooks in math, science, social studies, and reading/ELA. Introducing the Lexile WordBank

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 • A list of words and frequencies • Where the frequencies were derived from texts that adults read • The texts may be 25 years old or more • A list of words and frequencies by grade and domain • Where the frequencies were derived from best-selling K-12 texts • The texts date from 2011 • A database that can be queried and sorted, with comparisons to other academic word lists Which is more helpful to you?

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 What is the solution? THE LEXILE WORDBANK… • Contains approximately: • 50,000 unique words • 3,000 academic words SCIENCE MATH ENGLISH LANGUAGE ARTS SOCIAL STUDIES 40,000,000 RUNNING WORDS 144 BEST SELLING TEXT BOOKS

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 The Lexile WordBank identifies many words that are not in, or are de-prioritized by, other lists! Academic Word List, frequency > 25 (Coxhead, 2000) Lexile Academic WordList Word Overlap With Established Lists culture sphere generation site establish legislature respond evidence communication conflict percent interpret correspond mental function maintain negative require expand abandon forecast original essential solution suppose teamwork combination organize explore system

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 Science Math Social Studies General Grade 1 gas plant liquid equal solve subtract citizen government nation solve detail explain Grade 3 organism photosynthesis cell quotient compute variable military official empire clearly adaptation explanation Grade 5 aquatic cellular inorganic ratio formula calculate conquest rebel rebellion coordinate convey concern Grades 6-8 eukaryote metabolism enzyme integer algebraic midpoint overthrow opposition unrest approximate conjecture calculation Grades 9-12 cyanobacteria metabolic ingest equiangular concurrency tangency politically regime domination effectiveness incorporate counterclaim Example of Academic Words by Domain/Grade

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 A student’s Listening measure is her listening ability level. An audio passage measure is its listening complexity. 770L LISTENING PASSAGE MEASURE 770L LISTENING STUDENT MEASURE Lexile Listening Framework

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 A bank of test questions: • placement testing • progress monitoring • linking “Audio analyzer” software • Content variables, i.e. what is spoken • Acoustic variables, i.e. how it is spoken Lexile Listening Framework LISTENING PASSAGE MEASURE LISTENING STUDENT MEASURE

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 Informational texts Biographies Announcements Current affairs Fiction Academic Domains English Language Arts Science Social studies Math Item Bank — Content TEACHER TALKS RADIO REPORTS NARRATIVES DIALOGUES

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 • Listening for main idea, central purpose, or speaker’s point of view • Identifying specific information or detail • Drawing conclusions or making inferences • Decoding, parsing words and syntax Listening Construct

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 • Evaluate whether students can pay attention and learn from teacher talk • Stimulate interest in audio books • Evaluate comprehension of different genres: social talk, radio reports • Prepare & orient students for listening activities in state exams • Link with state test scores How will listening enhance your program?

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 Condition 1 Condition 2 Passage can be replayed Passage can be heard only once Passages are spoken quickly Passages are spoken slowly Questions are printed onscreen (reading input) Questions are listened to (audio input) Questions are seen while listening Questions are seen only after listening Item presented as reading Item presented as listening Items presented under different conditions. Our Research

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 • Reading aloud activates both reading and speaking skills o Stimulates articulatory organs o Activates greater psycholinguistic range • Repeated reading aloud improves fluency • Assisted (auditory) reading provides a reading model o Assists phonological encoding Relevance to Teaching & Learning

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 Student reads for 1 minute Teacher marks up errors Teacher tallies the errors Here is how some publishers use reading aloud… Read Aloud in Use 1 2 3

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 • Reading accuracy, e.g. 94% of words read correctly • Number of words read correctly per minute, or WCPM WCPM is considered most useful for tracking reading ability vs norms. Struggling On track Advanced 2nd grade 40 WCPM 50 - 70 WCPM 80 WCPM 3rd grade 60 WCPM 70 - 90 WCPM 100 WCPM EXAMPLE: Read Aloud in Use

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 Text Complexity + Reading Performance = Student Lexile Measure MetaMetrics models the complexity and decodability of the reading text, and takes that into account when estimating reading ability.  Almost any text can be submitted by you without the need for prior norming Modeling Text Decodability

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 “Pre-normed” Text Student reading performance + WCPM % accuracy = Decodability of the text Student reading performance + ORF Lexile Measure WCPM % accuracy = Harder text Easier text Slower reading Faster reading Same ORF Lexile Measure Current Approach Lexile Approach Modeling Text Decodability

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 1 MetaMetrics will provide an API service that analyzes audio recordings of students’ oral reading, and then return a Lexile measure, as well as other diagnostic information. How will it work? 2 3

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 NOTE: Not all of these will be available upon launch. The analysis returns both diagnostic and qualitative information about the student’s reading ability. Richer Feedback

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 • Earlier Reader item-types assess below the word level, probably to BR1500L ◦ Letter recognition ◦ Phonemic awareness ◦ Phonics ◦ Vocabulary • Extends the Lexile scale into the Pre-K space • Track growth and monitor progress from the start of the learning journey • Data collection in September 2019 Extending the Lexile Scale Below BR1000L

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 A Lexile Writing Framework is currently in R&D • Have a common metric that other scales and tests can be linked to • Have a yardstick by which students/parents/teachers might understand how they are doing • Provide feedback for instruction • Provide grade-level norms • Predict performance on state end-of year tests • Track against readiness for college & careers Purpose

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 Lexile as a literacy construct, which includes silent reading but encapsulates so much more. • Vocabulary • Oral Reading • Listening • Writing Lexile Roadmap — 2019 Onwards

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 LEARN MORE For more information about Lexile and Quantile measures, visit Lexile.com and Quantiles.com. For more information about MetaMetrics, visit MetaMetricsInc.com. CONNECT WITH US FOLLOW US ON SOCIAL MEDIA Twitter @MetaMetrics_Inc Facebook @LexileFramework and @QuantileFramework YouTube www.YouTube.com/MetaMetrics LinkedIn www.linkedin.com/company/metametrics-inc- 1000 Park Forty Plaza Drive, Suite 120 Durham, North Carolina 27713 919.547.3400 1.888.LEXILES (539-4537) [email protected] METAMETRICS

© 2019 MetaMetrics, Inc. All rights reserved. MM0185 Oral Reading & Listening Adlof, S. M., Catts, H. W., & Little , T. D. (2006) . Should the simple view of reading include a fluency component? Reading and Writing, 19, 933 – 958 . Catts, H. W. , Hogan, T. P., & Adlof, S. M. (2005) . Developmental changes in reading and reading disabilities . In H.W. Catts & A.G. Kamhi (Eds.), The Connections Between Language and Reading Disabilities (pp. 25 – 40) . Mahwah, NJ: Lawrence Erlbaum Associates. Fuchs, L. S., Fuchs, D., Eaton, S., & Hamlett, C. L. (2000). [Relation between reading fluency and reading comprehension as a function of silent versus oral reading mode]. Unpublished data. Cited in: Fuchs, L. S., Fuchs, Hosp, M. K. & Jenkins, J. R. (2001). Oral Reading Fluency as an Indicator of Reading Competence: A Theoretical, Empirical, and Historical Analysis. Scientific Studies of Reading, 5(3), 239–256 García , J. R., & Cain, K . (2013) . Decoding and reading comprehension: A meta-analysis to identify which reader and assessment characteristics influence the strength of the relationship in English . Review of Educational Research. 84, 74 – 111. Gough, P. B., & Tunmer, W. E. (1986). Decoding, reading, and reading disability. Remedial and Special Education, 7, 6 – 10. Hasbrouck, J., & Tindal, G. A. (2006). Oral reading fluency norms: A valuable assessment tool for reading teachers. The Reading Teacher, 59, 636–644. Jenkins, J. R., Fuchs, L. S., van den Broek, P., Espin, C., & Deno, S. L. (2003). Sources of individual differences in reading comprehension and reading fluency. Journal of Educational Psychology, 95, 719–729. Kim, H. J. (2012). Oral reading fluency in second language reading. Reading in a Foreign Language, 24,186–208 Kim, Y.-S. G., & Wagner, R. K. (2015). Text (Oral) reading fluency as a construct in reading development: An investigation of its mediating role for children from Grades 1 to 4. Scientific Studies of Reading, 19, 224–242. Kim, Y.-S. G. (2017). Why the Simple View of Reading Is Not Simplistic: Unpacking Component Skills of Reading Using a Direct and Indirect Effect Model of Reading (DIER). Scientific Studies of Reading, 21(4). Vocabulary Baumann, J. F. & Graves, M. F. (2010). What is academic vocabulary? Journal of Adolescent and Adult Literacy, 54: 4–12 Baker, S., Esther, G., Kieffer, M.J., Lesaux, N., Linan-Thompson, S., Morris, J., Proctor, C.P., & Russell, R. (2014). Teaching academic content and literacy to English learners in elementary and middle school (NCEE 2014- 4012). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Coxhead, A. (2000). A new academic word list. TESOL Quarterly, 34, 213-238. Davies, M. (2008-) The Corpus of Contemporary American English (COCA): 520 million words, 1990-present. Gardner, D., & Davies, M. (2014). A new academic vocabulary list. Applied Linguistics, 35, 305-327. Graves, M. F. (2017) The Vocabulary Book: Learning and Instruction (2nd edition). New York, NY: Teacher's College Press. Graves, M. F., Elmore, .J., & Fitzgerald, J. (in press). The Vocabulary of Core Reading Programs. Elementary School Journal. Greene, .J. Coxhead, A. (2015). Academic vocabulary for middle school students. Baltimore, MD: Brookes Publishing Marzano, R. J. (2004). Building background knowledge for academic achievement. Alexandria, VA: Assoc. for Supervision and Curriculum Development. Nagy, W., & Townsend, D. (2012). Words as tools: learning academic vocabulary as language acquisition. Reading Research Quarterly, 47, 91–108. References

2019/2020 Product Roadmap Unveiling

2019/2020 Product Roadmap Unveiling

More Decks by MetaMetrics

Other Decks in Education

Featured

Transcript