$30 off During Our Annual Pro Sale. View Details »

Contributor Leaderboards to Incentivize Good Community Citizenship

Bitergia
PRO
September 02, 2022

Contributor Leaderboards to Incentivize Good Community Citizenship

It takes a village to build open source software, or really a community of people with many different skills. Leaderboards can help us show the top-performing contributors and celebrate them, but they are double-edge swords in a community's toolbox. Tracking contributions can introduce an element of gamification, especially when contributors gain privileges in a community based on their level of contributions. As a positive outcome, leaderboards can increase participation and contributions which hopefully advances the community.

Bitergia
PRO

September 02, 2022
Tweet

More Decks by Bitergia

Other Decks in Technology

Transcript

  1. OpenJS World
    June 6-10, 2022
    Incentivizing Good
    Community Citizenship
    with Contributor
    Leaderboards
    Georg Link, Director of Sales at Bitergia

    View Slide

  2. Agenda
    ● Leaderboards
    ● Metrics
    ● Best Practices
    @GeorgLink

    View Slide

  3. Leaderboards
    @GeorgLink

    View Slide

  4. What are Leaderboards?
    : a large board for displaying the ranking of the leaders in a competitive
    event (such as a golf tournament)
    Definition of
    leaderboard
    @GeorgLink

    View Slide

  5. Other Examples of Leaderboards
    Countries
    @GeorgLink

    View Slide

  6. Other Examples of Leaderboards
    Countries
    Corporations
    @GeorgLink

    View Slide

  7. Other Examples of Leaderboards
    Countries
    Corporations
    Games
    @GeorgLink

    View Slide

  8. Other Examples of Leaderboards
    Countries
    Corporations
    Games
    People
    @GeorgLink

    View Slide

  9. Other Examples of Leaderboards
    Countries
    Corporations
    Games
    People
    Open Source
    @GeorgLink

    View Slide

  10. Leaderboards for GitHub contributors
    GitHub Insights shows
    contributors with most
    commits
    @GeorgLink

    View Slide

  11. Leaderboards in Drupal Marketplace
    @GeorgLink

    View Slide

  12. Leaderboard Theory
    @GeorgLink

    View Slide

  13. image goes here
    Why do we use Leaderboards?
    ● Give instant feedback for contributors
    ● Create sense of community
    ● Recognize contributors
    ● Reward activity
    ● Create competition
    ● Incentivize more activity
    @GeorgLink

    View Slide

  14. How do Leaderboards work?
    ● The Self-Determination Theory
    ○ The need for competence
    ○ The need for autonomy
    ○ The need for social relatedness
    ● Goal-Setting Theory of Motivation
    ○ Goal specific leaderboards work better
    ● Social Comparison Theory
    ○ Understanding of your own abilities and reactions
    ○ Facilitating social interaction
    @GeorgLink

    View Slide

  15. Motivational issues with Leaderboards
    If competitors were not of equal ability:
    Less skilled competitors became less motivated as the task seemed
    unreachable
    Highly skilled competitors found the activity too easy, eventually making
    them less motivated too
    Landers (2015)
    @GeorgLink

    View Slide

  16. How a Leaderboard Ranks People?
    ● Score
    ○ Progress towards a success measure
    ○ Embedded criteria for what counts
    ○ Bias towards measurable things
    @GeorgLink

    View Slide

  17. Metrics
    @GeorgLink

    View Slide

  18. Metrics
    A method of measuring something, or the results obtained from this:
    project data metric
    method
    @GeorgLink

    View Slide

  19. Metrics
    ● Created accidentally
    ● Contributions and their metadata
    ● Contains personal identifiable information (PII)
    like names and emails
    Project Data
    as Trace Data
    @GeorgLink

    View Slide

  20. Where is the data? Where is the community?
    Metrics
    Git Wiki
    GitHub Discourse
    GitLab Mailing List
    BitBucket IRC
    Jira Slack
    Gerrit Meetup.com
    Confluence StackOverflow
    @GeorgLink

    View Slide

  21. Raw data:
    ● Get data from data sources
    Enriched data:
    ● Unify data
    ● Manage identities
    ● Calculate metrics
    Useful data:
    ● Visualize, report
    How to get the data
    @GeorgLink

    View Slide

  22. Easiest step
    Query logs
    Use APIs
    Challenge: Changing APIs and data
    formats
    Raw data
    @GeorgLink

    View Slide

  23. Unify data
    ● Date formats
    ● Level of detail
    ● Metadata about different contributions
    ● Convert everything into the desired database
    structure
    Manage identities
    Calculate metrics
    Enriched data
    @GeorgLink

    View Slide

  24. Unify data
    Manage identities
    ● Who is who in the community
    ● Who do contributors work for (now and
    before)
    ● Different usernames and email
    ● Assigning contributions to the correct person
    Calculate metrics
    Enriched data
    @GeorgLink

    View Slide

  25. Unify data
    Manage identities
    Calculate metrics
    ● Primary metrics - summarizing original data
    ● Secondary metrics
    ○ Calculation from different data fields
    ○ Combining data from different data sources
    ○ Value judgements on data - e.g., quality models
    Enriched data
    @GeorgLink

    View Slide

  26. Useful data
    @GeorgLink

    View Slide

  27. Useful data
    Who is the data user?
    @GeorgLink

    View Slide

  28. Useful data
    Who is the data user?
    How should the data be presented?
    @GeorgLink

    View Slide

  29. Useful data
    Who is the data user?
    How should the data be presented?
    What visualizations are most meaningful?
    @GeorgLink

    View Slide

  30. Useful data
    Who is the data user?
    How should the data be presented?
    What visualizations are most meaningful?
    What story does the data tell?
    @GeorgLink

    View Slide

  31. ● Cauldron.io
    ● CHAOSS GrimoireLab
    ● CHAOSS Augur
    ● Apache Kibble
    ● CNCF Dev Analytics
    OSS Solutions
    @GeorgLink

    View Slide

  32. Best Practices
    @GeorgLink

    View Slide

  33. image goes here
    About CHAOSS
    Short for: Community Health Analytics Open Source Software
    Linux Foundation project, cofounded by Bitergia
    Started in 2017
    Focused on “creating analytics and metrics to help define
    community health” -- https://chaoss.community
    @GeorgLink

    View Slide

  34. image goes here
    Analytics Best Practices
    Follow Goal-Question-Metric approach
    Use metrics to tell a story
    Evaluate the usefulness of metric strategy
    Minimize gaming of metrics - careful how leaderboards score
    Start small and “get off zero”
    @GeorgLink

    View Slide

  35. image goes here
    Leaderboards Best Practices
    ● Be transparent about scoring algorithm
    ● Revise scoring algorithm with community feedback
    ● Continuously update leaderboard (automate)
    ● Many leaderboards for different categories
    ● Recognize diverse types of contributions
    ● Contextualize by showing immediate neighbors
    ● Reset leaderboard or ignore older contributions
    @GeorgLink

    View Slide

  36. Recap
    ● Leaderboards
    ● Metrics
    ● Best Practices
    @GeorgLink

    View Slide

  37. Email [email protected]
    LinkedIn/GitHub/Twitter @GeorgLink
    Thank You!
    @GeorgLink

    View Slide

  38. References
    ● https://medium.com/practical-motivation-science/how-to-motivate-with-leaderboards-da5a
    461fb2f6
    ● https://www.digits.co.uk/lms/the-psychology-of-leaderboards-why-they-do-in-fact-work/
    ● https://chaoss.community/
    @GeorgLink

    View Slide

  39. View Slide