Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Interventions in the Online Classroom

Interventions in the Online Classroom

Presented at HCI Lunch in 2012.

8480b47e733a040fba07c32da414b0e0?s=128

Justin Cheng

May 29, 2014
Tweet

Transcript

  1. interventions NLINE CLASSROOM in the JUSTIN CHENG SCOTT KLEMMER CHINMAY

    KULKARNI 1
  2. How do we increase student retention in online classes? (Assignment

    Completion) 2
  3. Retention is important Interested students may need an additional push

    to complete a course Important metric in measuring how engaging a class is 3
  4. Sociality Feedback · Peer Assessment Individual / Group · Grades

    · Action Plans · Contextual Hints · Commitment Contracts · Office Hours · Generic Hints · Personalized Email · 1-on-1 tutors Passive / Active · Co-located group work 4
  5. Social Effects People are consistent in their actions [Cialdini ‘01]

    People are influenced by what their friends do [Cialdini ‘07] – “Scott and 4 other friends completed this assignment” 5
  6. 6

  7. Commitment Sign a commitment contract [Werner, et. al ‘95] A

    small commitment followed by a larger one [Taylor & Booth-Butterfield ‘93] Bet on your success [Giné, et. Al ‘08] 7
  8. We find that The effects of commitment is affected by

    social influence Using checklists encourages task completion … 8
  9. So what did we do? 9

  10. Experimental Setup Conducted on Fall 2012 Stanford HCI Online class

    Waitlist control to minimize participant backlash 10
  11. Social Feedback / not Commitment / not Exp. 1 Social

    Feedback + Commitment 2384 participants 2 x 2 conditions •  How does social feedback and commitment affect student retention? •  How do these interact with each other? 11
  12. Exp. 1 Social Feedback + Commitment Both Social Feedback +

    Commitment Condition “I’ve started button” + List of friends (on Announcements Page & Assignment Page) 12
  13. Exp. 1 Social Feedback + Commitment Both Social Feedback +

    Commitment Condition Email before due date - friends who’ve started Email after due date – see your friends’ work 13
  14. Exp. 1 Social Feedback + Commitment Social Feedback Only Condition

    List of friends (on Announcements Page & Assignment Page) Generic email before due date See friends’ work email after due date 14
  15. Exp. 1 Social Feedback + Commitment Commitment Only Condition “I’ve

    started” button (on Announcements Page & Assignment Page) Generic email before due date Generic email after due date 15
  16. Exp. 1 Social Feedback + Commitment No Social, No Commitment

    Condition 16
  17. Exp. 2 Personalized & Generic Email Personalized Emails Generic Emails

    No Email 1439 participants 3 conditions •  Does sending emails affect retention? •  Are personalized emails more effective than generic emails? 17
  18. Exp. 2 Personalized & Generic Email Generic Email Condition Generic

    encouragement   Examples of good work   18
  19. Exp. 2 Personalized & Generic Email Personalized Email Condition Number

    of videos watched, quiz score reporting Use of student’s name   Generic encouragement   Examples of good work   19
  20. Exp. 2 Personalized & Generic Email No Email Condition 20

  21. Exp. 3 Action Plans Action Plans No Action Plans 1439

    participants 2 conditions •  Do interactive checklists help people complete assignments? 21
  22. Exp. 3 Action Plans Interactive checklist at the top of

    an assignment Inline step completion 22
  23. Facebook Friends have no noticeable impact on retention 23

  24. An “I’ve started” Button reduces assignment completion (and retention) 24

  25. But both together increase completion! (p < 0.05) 25

  26. Control Commitment 0 20 40 60 80 100 Control Social

    Control Social social_feedback_condition count 26
  27. Emails that we sent have no noticeable impact on retention

    27
  28. 0 500 1000 None Generic Personalized intervention_condition count 28

  29. But! Personalized Emails discourage people from clicking on examples! (p

    < 0.05) Assignment Link   Example Link   Example Link   29
  30. 3 4 5 0 50 100 150 Generic Personalized Generic

    Personalized Generic Personalized factor(intervention_condition) count 30
  31. Action Plans increase assignment completion (and retention) (p < 0.05)

    31
  32. 0 100 200 300 400 500 600 Control Action plan

    Experimental condition Number of submissions 32
  33. To conclude, Commitment goes hand-in-hand with social influence Action plans

    or interactive assignments in general are promising 33
  34. What’s next? What are other ways to increase retention? 34