Measuring the Impact of OER Initiatives

Provides a framework for assessing open educational resources (OER) projects at institutions to demonstrate the effectiveness of OER in student learning.

Transcript

1. Measuring the Impact of OER Initiatives: An Assessment Framework Regina Gong, OER Project Manager Dr. Karen Hicks, Director of Assessment Lansing Community College MI-OER Summit, September 21, 2018 This work is licensed under the Creative Commons Attribution 4.0 International License.
2. Here’s what we’re going to talk about: • Context of our OER project at LCC. • Importance of assessment. • Development of our assessment framework. • Elements and design of each assessment framework. • Snapshot of our data. • Questions and discussions.
3. Textbook affordability Allow faculty exploration and innovation in finding new, better, and less costly ways to deliver quality learning materials to students in addition to improved pedagogy. Goals of the OER Initiative at LCC
4. OER and Open Learning are two of the ongoing competitiveness and innovation projects in the 2017-2020 LCC Strategic Plan.
5. Because assessment makes the world go round Photo by John Jackson on Unsplash
6. Why is assessment important? Planned outcomes inform planned work. In turn, planned work drives planned outcomes.
7. How did we develop our assessment framework?
8. https://openedgroup.org/
10. COUP framework Cost Outcomes Use Perceptions Slide from OER Research Brief Presentation (2016) by Rajiv Jhangiani.
11. ◉ Costs of textbooks previously assigned ◉ OER support fee models ◉ Changes in campus bookstore revenue ◉ Changes in tuition revenue due to changes in drop rates ◉ Changes in tuition revenue due to changes in enrollment intensity ◉ Changes in tuition revenue due to changes in persistence Financial Impact Slide from OER Research Brief Presentation (2016) by Rajiv Jhangiani.
12. Educational Outcomes ◉ Changes in the percentage of students receiving a C or better ◉ Changes in rates of completion ◉ Changes in drop rates ◉ Changes in enrollment intensity ◉ Changes in persistence ◉ Changes in attainment of progress milestones (e.g., first 15 credits) ◉ Changes in graduation rates Slide from OER Research Brief Presentation (2016) by Rajiv Jhangiani
13. How are they using it? Adoption? Adaptation? • Deleting material from the OER • Inserting other open material inside the OER • Moving material around within the OER • Editing material in the OER Creation? Are there patterns of use? Why are they using it? Why are they not using it? Use ofOER
14. ◉ How do faculty and studentsperceive OER? ◉ How do they judge its quality? ◦ Does it align with the curriculum? ◦ Is it error free? ◦ Is it up-to-date? ◦ Does it come with ancillary resources? Perceptions of Faculty andStudents Slide from OER Research Brief Presentation (2016) by Rajiv Jhangiani
15. Designing an Assessment Framework • How to go about it (approach) • Collaborative • Reverse engineer/systemically • Aligned • Systematic • Key Elements • Logic model • Assessment plan • Monitoring & tracking plan
16. Elements of an Assessment Plan • Your data collection strategy must include these elements: • Performance indicators (i.e. data) • Sources of data • Methods of data collection • Methods of data analysis • Implementation of assessment plan • Management of data • Monitoring of data • Communication of data
17. Data collection strategies: Student surveys Faculty surveys Mixed methods • quantitative • qualitative CDS supplied data from Banner, Argos Photo by Fancycrave on Unsplash
18. OER Assessment Framework at LCC
19. OER Assessment Framework at LCC
20. OER Assessment Framework at LCC
21. OER Assessment Framework at LCC
23. Let’s look at the student and faculty surveys
24. Let’s look at our OER adoption file http://libguides.lcc.edu/oer/adopt
25. OER Adoptions at LCC 5 10 4 14 16 12 27 26 22 49 11 23 6 101 100 33 150 154 60 298 0 50 100 150 200 250 300 350 Fall 2015 Spring 2016 Summer 2016 Fall 2016 Spring 2017 Summer 2017 Fall 2017 Spring 2018 Summer 2018 Fall 2018 Courses and Sections Using OER Courses Using OER Sections Using OER
26. OER Adoptions at LCC 5 12 4 46 48 26 74 75 41 136 0 20 40 60 80 100 120 140 160 Fall 2015 Spring 2016 Summer 2016 Fall 2016 Spring 2017 Summer 2017 Fall 2017 Spring 2018 Summer 2018 Fall 2018 Faculty Using OER
27. Students Impacted by OER 317 540 129 2,825 2,558 724 3,724 3,711 1,177 6,033 - 1,000 2,000 3,000 4,000 5,000 6,000 7,000 Fall 2015 Spring 2016 Summer 2016 Fall 2016 Spring 2017 Summer 2017 Fall 2017 Spring 2018 Summer 2018 Fall 2018 Students Enrolled
28. Textbook Costs Savings $31,700 $54,000 $12,900 $282,500 $255,800 $72,400 $293,200 $371,100 $117,700 $603,300 $- $100,000 $200,000 $300,000 $400,000 $500,000 $600,000 $700,000 Fall 2015 Spring 2016 Summer 2016 Fall 2016 Spring 2017 Summer 2017 Fall 2017 Spring 2018 Summer 2018 Fall 2018 Total Textbook Savings ($100/student)
29. Forthcoming paper An Analysis of Cost, Outcomes, Use, and Perceptions in a Multi-Section Adoption of Open Textbooks in Introductory Psychology and Economics Courses Regina Gong and Karen Hicks Lansing Community College
31. Image source: https://unsplash.com/collections/167234/questions?photo=i--IN3cvEjg Regina Gong, gongr1@lcc.edu