Upgrade to Pro — share decks privately, control downloads, hide ads and more …

SIGCSE 2019: Using GitHub in the Classroom Predicts Positive Learning Outcomes

SIGCSE 2019: Using GitHub in the Classroom Predicts Positive Learning Outcomes

GitHub in the classroom predicts learning outcomes like preparation for the future and feeling part of a developer community. Download the open-access paper in the ACM here: https://dl.acm.org/citation.cfm?id=3287460

@mozzadrella

March 01, 2019
Tweet

More Decks by @mozzadrella

Other Decks in Education

Transcript

  1. Using GitHub in the classroom
    predicts student learning outcomes
    and classroom experiences
    Vanessa Gennarelli
    GitHub Education

    View full-size slide

  2. We wanted to find out:
    • Student experiences
    • Using GitHub for classroom vs. not
    • Which / what kind of implementations
    work best?

    View full-size slide

  3. GitHub in Education
    • GitHub is recognized as an industry standard and developers view knowing
    how to use GitHub as an essential part of securing a job
    • >31M developers building on GitHub
    • >20k teachers have used GitHub in their courses
    • >1.3M students have learned to code with GitHub

    View full-size slide

  4. Past research: what’s the impact of using
    GitHub?
    • Teachers are motivated to adopt GitHub is so
    students get familiar with an industry tool,
    which makes them more competitive on the job
    market (Feliciano)
    • Boosts student engagement in the classroom
    (Gunnarsson et al. 2017)
    • Students develop soft skills that prepare them
    for working in industry (Feliciano, Story, and
    Zagalsky 2016)

    View full-size slide

  5. Survey design overview
    Participants
    • Both users and non-users of GitHub participated
    • We emailed students with currently active student coupons
    • We emailed teachers who applied for the teacher discount either for personal repositories or for free organization
    Survey eligibility
    • Must have taken or taught a class with a programming component this past semester
    Two versions of the survey
    • Student version (n = 7530)
    • Teacher version (n = 300)

    View full-size slide

  6. Respondent Characteristics: Year in school
    Student Survey Teacher Survey
    7.8%
    12.5%
    15.4%
    33.7%
    1.2%
    1.3%
    2.3%
    5.1%
    3.3%
    17.4%
    College Freshman
    College Sophomore
    College Junior
    College Senior
    HS Freshman
    HS Sophomore
    HS Junior
    HS Senior
    Non-traditional student
    Other
    Year in school What year in school was the average
    student in your course?
    8.2%
    20.4%
    17.0%
    17.0%
    2.0%
    1.4%
    6.1%
    5.8%
    2.0%
    20.1%

    View full-size slide

  7. Diary study overview
    Study structure:
    • 4-month long structured diary study
    • Data collection through:
    • Online surveys (open-ended and scale responses)
    • In-depth interviews at the beginning and at the end of the semester
    Participants
    • N = 8 (3 female; 5 male)
    • New to GitHub or slightly familiar with GitHub
    • Undergraduate students (1 freshman, 1 sophomore, 5 juniors, 1 senior)
    • Instructor implemented GitHub in the classroom this semester
    Schedule:
    Initial: Week 1 - 2 Survey 4: Week 5 - 6
    Survey 1: Week 2 - 3 Survey 5: Week 7 - 8
    Survey 2: Week 3 - 4 Survey 6: Week 9 - 10
    Survey 3: Week 4 - 5 Survey 7: Week 14 - 15

    View full-size slide

  8. Survey results: Prepared for the future:
    GitHub versus non GitHub classrooms
    Student Survey Teacher Survey
    Rate how much you have learned about each of the
    following through [course] this past semester:
    Rate how much your students have learned about each
    of the following through [course] this past semester:
    1
    2
    3
    4
    5
    Teamwork and
    collaboration
    Popular industry
    tool(s)
    Project
    management
    Very much
    Very little 1
    2
    3
    4
    5
    Teamwork and
    collaboration
    Popular industry
    tool(s)
    Project
    management
    • Student version (n = 7530)
    • Teacher version (n = 300)

    View full-size slide

  9. The specific GitHub implementation matters
    Student Survey Teacher Survey
    1
    2
    3
    4
    5
    Fewer features (-1SD) More features (+1SD)
    Very much
    Very little
    Teamwork and collaborations Project management
    Preparation for future internship/career Preparation for being part of the developer community
    1
    2
    3
    4
    5
    Fewer features (-1SD) More features (+1SD)
    Very much
    Very little
    Teamwork and collaborations Project management
    Preparation for future internship/career Preparation for being part of the developer community
    Rate how much [course] has prepared
    you for each of the following
    Rate how much [course] has prepared
    your students for each of the following
    • Student version (n = 7530)
    • Teacher version (n = 300)

    View full-size slide

  10. Diary study: GitHub features
    Which of the following GitHub features have you used in your class?
    (n = 8)

    View full-size slide

  11. • Students in the study felt that, about halfway through the semester, they stopped learning new
    things about GitHub and felt comfortable using GitHub for class.
    • However, students who only interacted with a limited set of GitHub features wished they could have
    learned more about GitHub features in their course
    • “The only time we really used GitHub was pulling the starter code for our repositories.
    Unfortunately that was about it, which is a little disappointing because I thought we would
    be using it more…I wanted to get a better understanding of everything it could do…”
    • “I wish we would’ve gone more in depth…I didn’t really want to explore features on my
    own without someone else to show me what it is…definitely think there should have been
    more instruction…forks come to mind. I’m not sure what that is.”
    • “I know I want to use my GitHub as a portfolio, but I’ve never had a teacher show me how
    to do that. I don’t want to mess up.”
    Diary study: Exposure to GitHub features
    N = 8

    View full-size slide

  12. Survey results: Instructor Feedback
    Student Survey
    1
    2
    3
    4
    5
    6
    7
    I
    understand
    the
    instructor's
    feedback
    I pay
    attention to
    the
    instructor's
    feedback
    I use the
    instructor's
    feedback
    effectively
    I find the
    instructor's
    feedback
    helpful
    The
    instructor
    understands
    my needs as
    a student
    Strongly agree
    Strongly disagree
    Used GitHub to provide feedback
    Did not use GitHub to provide
    feedback
    ns ns
    Use your general impression of the instructor’s feedback to rate the items below:
    • Student version (n = 7530)

    View full-size slide

  13. Student Survey
    1
    2
    3
    4
    5
    6
    7
    I
    understand
    feedback
    from peers
    I pay
    attention to
    feedback
    from peers
    I use
    feedback
    from peers
    effectively
    I find
    feedback
    from peers
    helpful
    Strongly agree
    Strongly disagree
    Used GitHub to provide feedback
    Did not use GitHub to provide
    feedback
    ns ns
    Survey results: Peer Feedback
    • Student version (n = 7530)

    View full-size slide

  14. • Students’ sense of belonging can positive predict variables tied to academic success, such as
    intrinsic motivation and academic self-efficacy (Freeman, Anderman, and Jensen 2007)
    Both Belonging in class and Belonging in field variables were composite scores of multiple items:
    Use your general impression of [course] to rate the items below
    1 (Strongly disagree) to 7 (Strongly agree)
    •Belonging in class:
    •My contributions in [course] are valued
    •I feel comfortable in [course]
    •People in [course] accept me
    •Belonging in field:
    •I see myself as part of the developer community
    •I feel that I am a member of the developer community
    •I feel a sense of belonging to the developer community
    Survey results: Feelings of belonging

    View full-size slide

  15. Survey results: Sense of belonging:
    GitHub versus non GitHub classrooms
    1
    2
    3
    4
    5
    6
    7
    Belonging in class Belonging in field
    High
    Low
    Student Survey
    • Student version (n = 7530)

    View full-size slide

  16. • Learning to use GitHub was one variable that directly contributed to students’ feelings of
    belonging:
    • “[Using GitHub] increased my feelings of belonging because learning tools like that always
    does…it is a very common and necessary tool. I feel like it increased my…desire to be a
    programmer.”
    • “…I actually switched out of CS at one point because I had a lot of problems with being a girl in
    CS…sometimes it’s difficult with the amount of boys and it being an old boys’ club…it’s better
    now…” (Before Survey 1)
    • “…over the course of the semester, I feel a lot more that I can identify with the label that I
    am a developer because I am more familiar with how GitHub works…I don’t feel like an
    imposter…”
    • “…now that I know how to use GitHub… I think there’s a lot more that I can do to kind of get
    involved like with other developers…[GitHub] provides a place to do all of that. If I want to do
    this as a career path…it’s important that I’m not just working by myself because that’s not
    how I’m going to work after college…”
    Diary study: Feelings of belonging

    View full-size slide

  17. Survey results: Learning GitHub
    Student Survey Teacher Survey
    49%
    36%
    13%
    3%
    1%
    2%
    14%
    54%
    26%
    4%
    0%
    10%
    20%
    30%
    40%
    50%
    60%
    Not at all
    familiar
    Slightly
    familiar
    Moderately
    familiar
    Very familiar Extremely
    familiar
    How familiar were your
    students with GitHub at
    the beginning of the
    semester?
    How familiar are your
    students with GitHub
    now?
    8%
    21%
    26%
    29%
    15%
    0% 0%
    22%
    49%
    26%
    0%
    10%
    20%
    30%
    40%
    50%
    60%
    Not at all
    familiar
    Slightly
    familiar
    Moderately
    familiar
    Very familiar Extremely
    familiar
    How familiar were you
    with GitHub at the
    beginning of the
    semester?
    How familiar are you with
    GitHub now?
    • Student version (n = 7530)
    • Teacher version (n = 300)

    View full-size slide

  18. • On average, at the beginning of the semester,
    participants expected needing 2 to 2.5 months
    before feeling familiar with GitHub
    • Participants who were not at all to slightly familiar
    with GitHub at the beginning of the semester were
    moderately familiar with GitHub by the end of the
    semester.
    “Creating a repo…using the branches and pull
    requests…I feel like in the beginning of the semester I
    would have been scared to do all that but like now…it’s
    just like easy, comfortable.”
    How familiar are you with GitHub right now?
    1
    2
    3
    4
    5
    Initial Survey 1 Survey 2 Survey 3 Survey 4 Survey 5 Survey 6 Survey 7
    Extremely familiar
    Moderately familiar
    Not at all familiar
    Diary study: Learning GitHub
    (n = 8)

    View full-size slide

  19. Survey results: Learning GitHub
    Student Survey Teacher Survey
    30%
    27% 28%
    14%
    1%
    0%
    10%
    20%
    30%
    40%
    1 2 3 4 5
    Easy Difficult
    How easy or difficult was it to learn how to
    use GitHub?
    8%
    24%
    29%
    34%
    6%
    0%
    10%
    20%
    30%
    40%
    1 2 3 4 5
    Easy Difficult
    How easy or difficult was it for the average
    student in [course] to learn how to use
    GitHub?
    • Student version (n = 7530)

    View full-size slide

  20. Survey: Support while learning GitHub
    Student Survey Teacher Survey
    10%
    15%
    55%
    13%
    8%
    12%
    20%
    47%
    15%
    6%
    0%
    10%
    20%
    30%
    40%
    50%
    60%
    Too little
    support
    Just the right
    amount of
    support
    Too much
    support
    Describe the amount of support you received from each of the
    following as you were learning GitHub
    [course] teachers
    [course] peers
    1%
    11%
    59%
    19%
    6% 5%
    9%
    19%
    45%
    13%
    1%
    13%
    0%
    10%
    20%
    30%
    40%
    50%
    60%
    Too little
    support
    Just the right
    amount of
    support
    Too much
    support
    I don't
    know/not sure
    Describe the amount of support students received on average from
    each of the following as they were learning GitHub
    [course] teachers
    [course] peers
    • Student version (n = 7530)

    View full-size slide

  21. • Students who received too little support from instructors thought instructors were making
    assumptions about their familiarity with GitHub and typically relied on online resources.
    “The instructors assume we know it more…”
    “It’s mostly googling what I need to figure out.”
    “I would like my instructor to show us how to use GitHub in depth, especially in an intro class like the
    one I am in.”
    Support while learning GitHub

    View full-size slide

  22. • Access to resources made learning to use
    GitHub easy
    • “Good resources to learn it from my class.”
    • New terminology made learning to use
    GitHub difficult
    • “There are many commands that GitHub
    does not make clear to learn…”
    1
    2
    3
    4
    5
    Survey 1 Survey 2 Survey 3 Survey 4 Survey 5 Survey 6 Survey 7
    Difficult
    Neither easy nor
    difficult
    Easy
    How easy or difficult is it learning to use GitHub right now?
    Diary study: Ease of learning GitHub
    (n = 8)

    View full-size slide

  23. • Nearly all students relied on peers at the beginning of the semester.
    • However, when there was a perceived gap in familiarity between the participant and their peers, they
    may have been more hesitant to reach out to peers for help.
    “I feel like that [asking peers] will be a very last resort because it’s also like embarrassing to ask
    them”
    • As students started collaborating with peers on larger group projects towards the end of the
    semester, reliance on peers for support increased.
    Diary study: Support while learning
    GitHub
    (n = 8)

    View full-size slide

  24. Diary study: Exploration of GitHub
    Percentage of participants who explored GitHub beyond
    what was necessary for their course
    0%
    20%
    40%
    60%
    80%
    100%
    Survey 1 Survey 2 Survey 3 Survey 4 Survey 5 Survey 6 Survey 7
    (n = 8)

    View full-size slide

  25. • Some participants felt their course did not motivate them to explore GitHub
    beyond what was necessary for the course:
    • “I think they could have added in part of the project…like, you have to make a
    branch or like make a pull request…to like understand what it is. Just like add
    little elements into the project so you can explore additional functionalities.
    You can work that into the project…just a little extra part…then I’d definitely
    walk away with a bigger understanding”
    • “At a certain point, we were learning less in the classroom…Like, everything we
    were doing for the class stayed at one level…the responsibility to learn more
    shifted to me personally…”
    Exploration of GitHub

    View full-size slide

  26. Conclusion
    1.Using GitHub for coursework (versus not using GitHub) predicts outcomes around:
    • Teamwork and collaboration, popular industry tool(s), and project management
    • Sense of belonging in the field
    2.More features used, more outcomes experienced
    3.Feedback is more effective , helpful, and students felt understood by their teachers
    4.Students feel moderately familiar with GitHub 2-2.5 months into the semester
    5. Students may require more support when learning to use GitHub

    View full-size slide

  27. GitHub Education
    education.github.com

    View full-size slide

  28. Survey results: GitHub in the classroom
    predicts greater student preparedness for
    the future
    • Students in GitHub (versus non GitHub) classrooms learned more about:
    • Teamwork and collaboration
    • Popular industry tool(s)
    • Project management
    • Students in GitHub (versus non GitHub) were more prepared for:
    • Being a part of the developer community
    • Developing a portfolio of their work

    View full-size slide