Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Contributor Leaderboards to Incentivize Good Community Citizenship

September 02, 2022

Contributor Leaderboards to Incentivize Good Community Citizenship

It takes a village to build open source software, or really a community of people with many different skills. Leaderboards can help us show the top-performing contributors and celebrate them, but they are double-edge swords in a community's toolbox. Tracking contributions can introduce an element of gamification, especially when contributors gain privileges in a community based on their level of contributions. As a positive outcome, leaderboards can increase participation and contributions which hopefully advances the community.


September 02, 2022

More Decks by Bitergia

Other Decks in Technology


  1. OpenJS World June 6-10, 2022 Incentivizing Good Community Citizenship with

    Contributor Leaderboards Georg Link, Director of Sales at Bitergia
  2. What are Leaderboards? : a large board for displaying the

    ranking of the leaders in a competitive event (such as a golf tournament) Definition of leaderboard @GeorgLink
  3. image goes here Why do we use Leaderboards? • Give

    instant feedback for contributors • Create sense of community • Recognize contributors • Reward activity • Create competition • Incentivize more activity @GeorgLink
  4. How do Leaderboards work? • The Self-Determination Theory ◦ The

    need for competence ◦ The need for autonomy ◦ The need for social relatedness • Goal-Setting Theory of Motivation ◦ Goal specific leaderboards work better • Social Comparison Theory ◦ Understanding of your own abilities and reactions ◦ Facilitating social interaction @GeorgLink
  5. Motivational issues with Leaderboards If competitors were not of equal

    ability: Less skilled competitors became less motivated as the task seemed unreachable Highly skilled competitors found the activity too easy, eventually making them less motivated too Landers (2015) @GeorgLink
  6. How a Leaderboard Ranks People? • Score ◦ Progress towards

    a success measure ◦ Embedded criteria for what counts ◦ Bias towards measurable things @GeorgLink
  7. Metrics A method of measuring something, or the results obtained

    from this: project data metric method @GeorgLink
  8. Metrics • Created accidentally • Contributions and their metadata •

    Contains personal identifiable information (PII) like names and emails Project Data as Trace Data @GeorgLink
  9. Where is the data? Where is the community? Metrics Git

    Wiki GitHub Discourse GitLab Mailing List BitBucket IRC Jira Slack Gerrit Meetup.com Confluence StackOverflow @GeorgLink
  10. Raw data: • Get data from data sources Enriched data:

    • Unify data • Manage identities • Calculate metrics Useful data: • Visualize, report How to get the data @GeorgLink
  11. Unify data • Date formats • Level of detail •

    Metadata about different contributions • Convert everything into the desired database structure Manage identities Calculate metrics Enriched data @GeorgLink
  12. Unify data Manage identities • Who is who in the

    community • Who do contributors work for (now and before) • Different usernames and email • Assigning contributions to the correct person Calculate metrics Enriched data @GeorgLink
  13. Unify data Manage identities Calculate metrics • Primary metrics -

    summarizing original data • Secondary metrics ◦ Calculation from different data fields ◦ Combining data from different data sources ◦ Value judgements on data - e.g., quality models Enriched data @GeorgLink
  14. Useful data Who is the data user? How should the

    data be presented? @GeorgLink
  15. Useful data Who is the data user? How should the

    data be presented? What visualizations are most meaningful? @GeorgLink
  16. Useful data Who is the data user? How should the

    data be presented? What visualizations are most meaningful? What story does the data tell? @GeorgLink
  17. • Cauldron.io • CHAOSS GrimoireLab • CHAOSS Augur • Apache

    Kibble • CNCF Dev Analytics OSS Solutions @GeorgLink
  18. image goes here About CHAOSS Short for: Community Health Analytics

    Open Source Software Linux Foundation project, cofounded by Bitergia Started in 2017 Focused on “creating analytics and metrics to help define community health” -- https://chaoss.community @GeorgLink
  19. image goes here Analytics Best Practices Follow Goal-Question-Metric approach Use

    metrics to tell a story Evaluate the usefulness of metric strategy Minimize gaming of metrics - careful how leaderboards score Start small and “get off zero” @GeorgLink
  20. image goes here Leaderboards Best Practices • Be transparent about

    scoring algorithm • Revise scoring algorithm with community feedback • Continuously update leaderboard (automate) • Many leaderboards for different categories • Recognize diverse types of contributions • Contextualize by showing immediate neighbors • Reset leaderboard or ignore older contributions @GeorgLink