Sensemaking with Learning Analytics: The need for openness

It's a data-centric world. Analytics are increasingly used to make sense of large (and growing) quantities of data. For educators and researchers, openness and transparency of both data and analytics methods is important. The need for openness is challenged by numerous vendors who make claims about the insight into learning that their platform can provide, but do not make the data used or the process explicit. This presentation will introduce the Open Learning Analytics initiative and detail opportunities for colleges and universities to take control of their own data and begin partnering with advocates for open data, technology, and analytics.

George Siemens, PhD

Transcript

1. Sensemaking with Learning Analytics: The need for openness George Siemens, PhD June 4, 2014 Open Apereo 2014
2. Background Learning Analytics Creativity Sensemaking Open Learning Analytics
3. Background Learning Analytics Creativity Sensemaking Open Learning Analytics
4. Jobs: disappearing & new Automation (Frey & Osborne, 2013) Knowledge work (US Bureau of Economic Analysis, McKinsey & Co, 2012) http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_ Employment.pdf http://www.bea.gov/industry/gdpbyind_data.htm http://www.mckinsey.com/insights/organization/preparing_for_a_new_e ra_of_work
5. McKinsey Quarterly, 2012
6. Student profiles Diversifying (OECD) Less than 50% now full time (US Census Bureau) http://www.oecd.org/edu/skills-beyond-school/EDIF%202013-- N%C2%B015.pdf http://www.census.gov/prod/2013pubs/acsbr11-14.pdf
7. Favours women over men More learners as % (up to 60%) Average entrance age increasing Top three countries for entering students: China, India, USA Traditional science courses waning in popularity Greater international student OECD 2013
8. Complexification of higher education Learning needs are complex, ongoing Simple singular narrative won’t suffice going forward The idea of the university is expanding and diversifying
9. Background Learning Analytics Creativity Sensemaking Open Learning Analytics
10. “… escalating the speed of research on many problems in education.” “Not only can you look at unique learning trajectories of individuals, but the sophistication of the models of learning goes up enormously.” Arthur Graesser, Editor, Journal of Educational Psychology
11. LA approach Example Techniques Modeling Attention metadata Learner modeling Behavior modeling User profile development Relationship Mining Discourse analysis Sentiment analysis A/B Testing Neural networks Knowledge Domain Modeling Natural language processing Ontology development Assessment (matching user knowledge with knowledge domain) Siemens 2013: Adapted from Bienkowski et al, 2012, Baker & Yacef, 2009, Baker & Siemens 2013
12. LA approach Example Applications Trend Analysis and Prediction Early warning, risk identification Measuring impact of interventions Changes in learner behavior, course discussions, identification of error propagation Personalization/Adaptive learning Recommendations: content and social connections Adaptive content provision to learners Attention metadata Structural analysis Social network analysis Latent semantic analysis Information flow analysis Siemens 2013: Adapted from Bienkowski et al, 2012, Baker & Yacef, 2009, Baker & Siemens 2013
14. What will LA do for education Add a new research layer Personalization Optimization (move from negative orientation) Organizational insight Improved decision making New models of learning
15. Personalized learning models Keller Plan (Personalized System of Instruction) Static learner profile (old school) Objective based (adaptivecourseware) Intelligent tutors (CMU OLI) Personalized (Knewton)
16. Parallel developing partners Platform Publisher Knewton Pearson Smart Sparrow McGraw-Hill Desire2Learn adaptcourseware LoudCloud CMU OLI
17. Personalization Content that has been linked/mapped to knowledge & key concepts Learner knowledge graph Personalization through social and technological models
18. Learning has to date been content- oriented
19. It will in the future be learner knowledge-centric
20. Personal Knowledge Graph People – learners, students, everyone – should have a personal knowledge graph (PKG) A network model of what we know Learner-owned
21. Concept architecture http://www.metacademy.org/
23. Knowledge Domains Nursing Computer Science Landscaping Personal Knowledge Graph 82% 65% 38%
24. Background Learning Analytics Creativity Sensemaking Open Learning Analytics
25. Knowledge is a pattern of connections New knowledge builds on (relates to) what is already known Innovation requires openness for new connection forming, new creations, new mashups
27. “a thousand threads that lead from the locomotive to the very beginning of the modern world” Rosen, 2010
28. Metal workers: cylinders Steam Wheels Motion Transportation need Viability Scientific progress Entrepreneurship
29. “The process may be more like stitching together known parts than pioneering a complete route from scratch” W. Bryan Arthur, 2006
30. Maria Popova in order for us to truly create and contribute to the world, we have to be able to connect countless dots, to cross- pollinate ideas from a wealth of disciplines, to combine and recombine these pieces and build new castles.
31. Background Learning Analytics Creativity Sensemaking Open Learning Analytics
32. Sensemaking “Sensemaking is a motivated, continuous effort to understand connections . . . in order to anticipate their trajectories and act effectively” (Klein et al. 2006)
33. or “Sensemaking is about labeling and categorizing to stabilize the streaming of experience” (Weick et al. 2005: 411)
34. We socially sensemake through stories, narratives, knowledge exchange, discourse
35. We turn to technical approaches when the data exceeds our capacity to create social discourse around it But, in fairness, once we technically sensemake, we turn to narrative to share
37. Adaptivity/Personalization addresses these quadrants
38. The future of work is in these quadrants
39. Background Learning Analytics Creativity Sensemaking Open Learning Analytics
42. Why is this important? Enable career mobility & education transition Creating an efficient learning system Analytics as a key value point for education