Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Contributor Leaderboards to Incentivize Good Community Citizenship

Bitergia
PRO
September 02, 2022

Contributor Leaderboards to Incentivize Good Community Citizenship

It takes a village to build open source software, or really a community of people with many different skills. Leaderboards can help us show the top-performing contributors and celebrate them, but they are double-edge swords in a community's toolbox. Tracking contributions can introduce an element of gamification, especially when contributors gain privileges in a community based on their level of contributions. As a positive outcome, leaderboards can increase participation and contributions which hopefully advances the community.

Bitergia
PRO

September 02, 2022
Tweet

More Decks by Bitergia

Other Decks in Technology

Transcript

  1. OpenJS World June 6-10, 2022 Incentivizing Good Community Citizenship with

    Contributor Leaderboards Georg Link, Director of Sales at Bitergia
  2. Agenda • Leaderboards • Metrics • Best Practices @GeorgLink

  3. Leaderboards @GeorgLink

  4. What are Leaderboards? : a large board for displaying the

    ranking of the leaders in a competitive event (such as a golf tournament) Definition of leaderboard @GeorgLink
  5. Other Examples of Leaderboards Countries @GeorgLink

  6. Other Examples of Leaderboards Countries Corporations @GeorgLink

  7. Other Examples of Leaderboards Countries Corporations Games @GeorgLink

  8. Other Examples of Leaderboards Countries Corporations Games People @GeorgLink

  9. Other Examples of Leaderboards Countries Corporations Games People Open Source

    @GeorgLink
  10. Leaderboards for GitHub contributors GitHub Insights shows contributors with most

    commits @GeorgLink
  11. Leaderboards in Drupal Marketplace @GeorgLink

  12. Leaderboard Theory @GeorgLink

  13. image goes here Why do we use Leaderboards? • Give

    instant feedback for contributors • Create sense of community • Recognize contributors • Reward activity • Create competition • Incentivize more activity @GeorgLink
  14. How do Leaderboards work? • The Self-Determination Theory ◦ The

    need for competence ◦ The need for autonomy ◦ The need for social relatedness • Goal-Setting Theory of Motivation ◦ Goal specific leaderboards work better • Social Comparison Theory ◦ Understanding of your own abilities and reactions ◦ Facilitating social interaction @GeorgLink
  15. Motivational issues with Leaderboards If competitors were not of equal

    ability: Less skilled competitors became less motivated as the task seemed unreachable Highly skilled competitors found the activity too easy, eventually making them less motivated too Landers (2015) @GeorgLink
  16. How a Leaderboard Ranks People? • Score ◦ Progress towards

    a success measure ◦ Embedded criteria for what counts ◦ Bias towards measurable things @GeorgLink
  17. Metrics @GeorgLink

  18. Metrics A method of measuring something, or the results obtained

    from this: project data metric method @GeorgLink
  19. Metrics • Created accidentally • Contributions and their metadata •

    Contains personal identifiable information (PII) like names and emails Project Data as Trace Data @GeorgLink
  20. Where is the data? Where is the community? Metrics Git

    Wiki GitHub Discourse GitLab Mailing List BitBucket IRC Jira Slack Gerrit Meetup.com Confluence StackOverflow @GeorgLink
  21. Raw data: • Get data from data sources Enriched data:

    • Unify data • Manage identities • Calculate metrics Useful data: • Visualize, report How to get the data @GeorgLink
  22. Easiest step Query logs Use APIs Challenge: Changing APIs and

    data formats Raw data @GeorgLink
  23. Unify data • Date formats • Level of detail •

    Metadata about different contributions • Convert everything into the desired database structure Manage identities Calculate metrics Enriched data @GeorgLink
  24. Unify data Manage identities • Who is who in the

    community • Who do contributors work for (now and before) • Different usernames and email • Assigning contributions to the correct person Calculate metrics Enriched data @GeorgLink
  25. Unify data Manage identities Calculate metrics • Primary metrics -

    summarizing original data • Secondary metrics ◦ Calculation from different data fields ◦ Combining data from different data sources ◦ Value judgements on data - e.g., quality models Enriched data @GeorgLink
  26. Useful data @GeorgLink

  27. Useful data Who is the data user? @GeorgLink

  28. Useful data Who is the data user? How should the

    data be presented? @GeorgLink
  29. Useful data Who is the data user? How should the

    data be presented? What visualizations are most meaningful? @GeorgLink
  30. Useful data Who is the data user? How should the

    data be presented? What visualizations are most meaningful? What story does the data tell? @GeorgLink
  31. • Cauldron.io • CHAOSS GrimoireLab • CHAOSS Augur • Apache

    Kibble • CNCF Dev Analytics OSS Solutions @GeorgLink
  32. Best Practices @GeorgLink

  33. image goes here About CHAOSS Short for: Community Health Analytics

    Open Source Software Linux Foundation project, cofounded by Bitergia Started in 2017 Focused on “creating analytics and metrics to help define community health” -- https://chaoss.community @GeorgLink
  34. image goes here Analytics Best Practices Follow Goal-Question-Metric approach Use

    metrics to tell a story Evaluate the usefulness of metric strategy Minimize gaming of metrics - careful how leaderboards score Start small and “get off zero” @GeorgLink
  35. image goes here Leaderboards Best Practices • Be transparent about

    scoring algorithm • Revise scoring algorithm with community feedback • Continuously update leaderboard (automate) • Many leaderboards for different categories • Recognize diverse types of contributions • Contextualize by showing immediate neighbors • Reset leaderboard or ignore older contributions @GeorgLink
  36. Recap • Leaderboards • Metrics • Best Practices @GeorgLink

  37. Email [email protected] LinkedIn/GitHub/Twitter @GeorgLink Thank You! @GeorgLink

  38. References • https://medium.com/practical-motivation-science/how-to-motivate-with-leaderboards-da5a 461fb2f6 • https://www.digits.co.uk/lms/the-psychology-of-leaderboards-why-they-do-in-fact-work/ • https://chaoss.community/ @GeorgLink

  39. None