Modelling Network Dynamics in Social Annotation at #LAK21Network Workshop

Web annotation technology is used in education to facilitate individual learning and social interaction. Departing from a conceptual exploration of social interaction in web annotation as a mediated process, as well as a dissatisfaction with analytical methods applied to web annotation data, we analyzed student interaction in a web annotation environment following the Relational Event Modelling approach. Included in our modelling were various annotation attributes, a contextual factor of student groups, and several social and spatiotemporal factors related to network formation. Results indicated that longer annotations were slightly more likely to attract replies, students in the same project group were not more likely to engage with each other, and several network factors such as student activity, reciprocity, annotation popularity, and annotation location played important roles. This study contributes empirical insights into web annotation and calls for future work to investigate mediated social interaction as a dynamic network phenomenon.

Transcript

1. Modelling Network Dynamics in Social Annotation Bodong Chen, Basel Hussein, Sasha Poquet April 12, 2021 @ #LAK21 Workshop on Using Network Science in Learning Analytics 1
2. Introduction Learning is fundamentally a social process. In digital spaces, social interaction is largely mediated by artifacts. Computer-mediated communication is often used to support social interaction in learning. 2
3. Web Annotation A genre of computer- mediated communication Allows the user to highlight a target in a web document and post an annotation referring to that target Being able to reply creates a layer of interactivity on the web (W3C, 2017; Zhu et al., 2020) 3
4. Hypothesis A leading-edge web annotation tool (hypothes.is) Piloted in college classrooms 4
5. Collaborative Web Annotation The mediated nature of social interaction in web annotation (Dix et al., 2003) 5
6. Collaborative Web Annotation (cont'd) The temporal unfolding of interactional events in a digital space 6
7. Gaps in Prior (Network) Research Mediation not taken seriously The spatial dimension is ignored Limited to relational states instead of events 7
8. Context of the Study A 14-week graduate seminar 14 students and one instructor Hypothesis was used to support knowledge co-construction Students worked in groups (of 2-4) on course projects 8
9. Research Questions RQ1: To what extent did the attributes of web annotations contribute to mediated social interaction? RQ2: To what extent did the course-design factor of student grouping contribute to mediated social interaction? RQ3: To what extent did the endogenous network factors contribute to mediated social interaction? 9
10. Relational Event Modelling (REM) 10
11. REM of Collaborative Annotation Two-mode network of actors (students) and artifacts (annotations) Dependent variable: the next reply event in the sequence Node attributes Location of an annotation: relative location in a document Length of an annotation: word count Question in an annotation: the presence of a question mark Exogenous contextual factors Project groups: a student's membership in project groups 11
12. REM of Collaborative Annotation (cont'd) Endogenous network factors Prior actor activity: a participant's past reply activities Prior annotation popularity: past replies to an annotation Location homophily: the next annotation being close to the previous one Reciprocity (four-cycle): two participants replying to a same annotation in the past increases their propensity to both reply to another annotation in the future 12
13. Findings Descriptive statistics On average, a student wrote about 80 total posts consisting of 45 annotations and 35 replies during the 14-week semester. Temporally, about 64% posting behaviours happened 36 hours before the weekly class meeting. Spatially, 75% of annotations came before the 60% mark of the article. 13
14. REM Results Factors Model 1 Model 2 Model 3 Annotation location Annotation question Annotation length Group homophily Actor activity Location homophily Annotation popularity Reciprocity (four cycle) AIC McFadden pseudo-R −.166 −.150 −.085 .003 .002 .065 .003 .003 .003∗ .001 .004 .121∗∗∗ −.010∗∗∗ 1.121∗∗∗ 1.253∗∗∗ 3677.903 3679.659 3269.597 2 .001 .001 .115 14
15. REM Results (cont'd) Node attributes Only annotation length positively associated with the odds Annotation location and question were non-significant Exogenous contextual factors Group membership was non-significant Endogenous network factors Prior actor activity, prior annotation popularity, and reciprocity (four-cycle) were positively associated; location homophily was negatively associated Active students, popular annotations, prior collaboration --> greater chance of interaction; a recent reply supresses future replies near the same location 15
16. Discussion A step towards modelling mediated peer interaction in digital environments as a dynamic, mediated social process. This study also draws attention to the spatio-temporal properties of mediated social interaction in web annotation environments. For network analysis in learning analytics, this study illuminates the importance of examning relational events, besides relational states. 16
17. Contact Bodong Chen @bod0ng | chenbd@umn.edu OSF project site: https://osf.io/kcfh8/ 17
18. REM Details Calculated statistics of relational event using the rem R package, and then constructed stratified Cox models using the survival R package. fit3 <- clogit( eventDummy ~ # main effects at annotation level location + question + word_count + # exogenous contexual factor group_homophily + # endogenous network statistics activityActor + location_homophily + popularity_annotation + inertia + four_cycle + # specify the strata => stratified cox model strata(eventTime), method = "exact", data = dtrem) 18