Contributor Leaderboards to Incentivize Good Community Citizenship

Slide 1

Slide 1 text

OpenJS World June 6-10, 2022 Incentivizing Good Community Citizenship with Contributor Leaderboards Georg Link, Director of Sales at Bitergia

Slide 2

Slide 2 text

Agenda ● Leaderboards ● Metrics ● Best Practices @GeorgLink

Slide 3

Slide 3 text

Leaderboards @GeorgLink

Slide 4

Slide 4 text

What are Leaderboards? : a large board for displaying the ranking of the leaders in a competitive event (such as a golf tournament) Deﬁnition of leaderboard @GeorgLink

Slide 5

Slide 5 text

Other Examples of Leaderboards Countries @GeorgLink

Slide 6

Slide 6 text

Other Examples of Leaderboards Countries Corporations @GeorgLink

Slide 7

Slide 7 text

Other Examples of Leaderboards Countries Corporations Games @GeorgLink

Slide 8

Slide 8 text

Other Examples of Leaderboards Countries Corporations Games People @GeorgLink

Slide 9

Slide 9 text

Other Examples of Leaderboards Countries Corporations Games People Open Source @GeorgLink

Slide 10

Slide 10 text

Leaderboards for GitHub contributors GitHub Insights shows contributors with most commits @GeorgLink

Slide 11

Slide 11 text

Leaderboards in Drupal Marketplace @GeorgLink

Slide 12

Slide 12 text

Leaderboard Theory @GeorgLink

Slide 13

Slide 13 text

image goes here Why do we use Leaderboards? ● Give instant feedback for contributors ● Create sense of community ● Recognize contributors ● Reward activity ● Create competition ● Incentivize more activity @GeorgLink

Slide 14

Slide 14 text

How do Leaderboards work? ● The Self-Determination Theory ○ The need for competence ○ The need for autonomy ○ The need for social relatedness ● Goal-Setting Theory of Motivation ○ Goal speciﬁc leaderboards work better ● Social Comparison Theory ○ Understanding of your own abilities and reactions ○ Facilitating social interaction @GeorgLink

Slide 15

Slide 15 text

Motivational issues with Leaderboards If competitors were not of equal ability: Less skilled competitors became less motivated as the task seemed unreachable Highly skilled competitors found the activity too easy, eventually making them less motivated too Landers (2015) @GeorgLink

Slide 16

Slide 16 text

How a Leaderboard Ranks People? ● Score ○ Progress towards a success measure ○ Embedded criteria for what counts ○ Bias towards measurable things @GeorgLink

Slide 17

Slide 17 text

Metrics @GeorgLink

Slide 18

Slide 18 text

Metrics A method of measuring something, or the results obtained from this: project data metric method @GeorgLink

Slide 19

Slide 19 text

Metrics ● Created accidentally ● Contributions and their metadata ● Contains personal identiﬁable information (PII) like names and emails Project Data as Trace Data @GeorgLink

Slide 20

Slide 20 text

Where is the data? Where is the community? Metrics Git Wiki GitHub Discourse GitLab Mailing List BitBucket IRC Jira Slack Gerrit Meetup.com Conﬂuence StackOverﬂow @GeorgLink

Slide 21

Slide 21 text

Raw data: ● Get data from data sources Enriched data: ● Unify data ● Manage identities ● Calculate metrics Useful data: ● Visualize, report How to get the data @GeorgLink

Slide 22

Slide 22 text

Easiest step Query logs Use APIs Challenge: Changing APIs and data formats Raw data @GeorgLink

Slide 23

Slide 23 text

Unify data ● Date formats ● Level of detail ● Metadata about different contributions ● Convert everything into the desired database structure Manage identities Calculate metrics Enriched data @GeorgLink

Slide 24

Slide 24 text

Unify data Manage identities ● Who is who in the community ● Who do contributors work for (now and before) ● Different usernames and email ● Assigning contributions to the correct person Calculate metrics Enriched data @GeorgLink

Slide 25

Slide 25 text

Unify data Manage identities Calculate metrics ● Primary metrics - summarizing original data ● Secondary metrics ○ Calculation from different data ﬁelds ○ Combining data from different data sources ○ Value judgements on data - e.g., quality models Enriched data @GeorgLink

Slide 26

Slide 26 text

Useful data @GeorgLink

Slide 27

Slide 27 text

Useful data Who is the data user? @GeorgLink

Slide 28

Slide 28 text

Useful data Who is the data user? How should the data be presented? @GeorgLink

Slide 29

Slide 29 text

Useful data Who is the data user? How should the data be presented? What visualizations are most meaningful? @GeorgLink

Slide 30

Slide 30 text

Useful data Who is the data user? How should the data be presented? What visualizations are most meaningful? What story does the data tell? @GeorgLink

Slide 31

Slide 31 text

● Cauldron.io ● CHAOSS GrimoireLab ● CHAOSS Augur ● Apache Kibble ● CNCF Dev Analytics OSS Solutions @GeorgLink

Slide 32

Slide 32 text

Best Practices @GeorgLink

Slide 33

Slide 33 text

image goes here About CHAOSS Short for: Community Health Analytics Open Source Software Linux Foundation project, cofounded by Bitergia Started in 2017 Focused on “creating analytics and metrics to help deﬁne community health” -- https://chaoss.community @GeorgLink

Slide 34

Slide 34 text

image goes here Analytics Best Practices Follow Goal-Question-Metric approach Use metrics to tell a story Evaluate the usefulness of metric strategy Minimize gaming of metrics - careful how leaderboards score Start small and “get off zero” @GeorgLink

Slide 35

Slide 35 text

image goes here Leaderboards Best Practices ● Be transparent about scoring algorithm ● Revise scoring algorithm with community feedback ● Continuously update leaderboard (automate) ● Many leaderboards for different categories ● Recognize diverse types of contributions ● Contextualize by showing immediate neighbors ● Reset leaderboard or ignore older contributions @GeorgLink

Slide 36

Slide 36 text

Recap ● Leaderboards ● Metrics ● Best Practices @GeorgLink

Slide 37

Slide 37 text

Email georglink@bitergia.com LinkedIn/GitHub/Twitter @GeorgLink Thank You! @GeorgLink

Slide 38

Slide 38 text

References ● https://medium.com/practical-motivation-science/how-to-motivate-with-leaderboards-da5a 461fb2f6 ● https://www.digits.co.uk/lms/the-psychology-of-leaderboards-why-they-do-in-fact-work/ ● https://chaoss.community/ @GeorgLink

Slide 39

Slide 39 text

No content