Metrics are fun, but which ones really matter?

There is no shortage of algorithms, techniques, and software to produce virtually any metric one may desire. True value is only realized when metrics are chosen that most closely align with your goals.

In this session, learn which goals we chose in the Liferay project (a large, open source portal project), why they were (and continue to be) important, how they are produced, and what we are doing with them.

Transcript

1. metrics are fun, but which ones really matter? james falkner liferay community manager @schtool 2014 FLOSS community metrics meeting portland, oregon #FCM2
2. James Falkner, Liferay Community Manager
3. liferay community metrics 13 years 4.3M LOC 115k registered site members 102 external contributors in 2013 150 SI partners 300 marketplace apps 800 marketplace developers 43 user groups 114 translators for 43 languages 300k forum posts (~100/day) 25k forum participants 2 Top 50 github devs ( , ) #5 #45
4. more metrics
5. MOAR METRICS!!
6. … and more
7. allthingsd.com/20121217/andreessen-and-mixpanel-call-for-an-end-to-bullshit-metrics/!
8. “companies need to start using a new set of metrics that don’t simply make them feel good. They should use actionable metrics that provide insight, provide guidance, and help businesses make better decisions.” - Suhail Doshi, mixpanel founder
9. core contributions
10. downloads
11. step #1 “search your feelings”
12. step #2 identify your goals for metrics
13. goal #1: increase the value of participation
14. www.youtube.com/watch?v=u6XAPnuFjJc
16. goal #2: Measure the effect of business decisions
17. goal #3: understand the relationship between metrics and business performance
19. step #3 know what to do with the results
20. basic 1st-order metrics for the liferay community # of apps/reviews # of contributions # of commits/lines # of authors # of bug reports # of forum posts/answers # of downloads # ignored messages # open tickets / code reviews
21. more interesting non-liferay vs. liferay crowdsourced quality time between contribution->codebase location of community members evolution of contributor age over time evolution of active/inactive users time of bug report -> fixed time from forum q to a # of ignored contributions distribution of commits across functional areas
22. metrics.forEach(function(metric) { How accurate is it? How can it help achieve your goals? What to do when it turns good/bad? What is its relationship to others? });
24. Code
25. Forums
26. Demographics
27. Community Contribution Performance
28. future: cmty career evolution
29. newbie: 0-5 posts, 0 answers, 1 download, 1 fork, … junior: 20 posts, 10% engaged, >1 patch, 5 tweets, 1 app senior: 50 posts, 30% engaged, 10 tweets, 5 apps, moderator, … champion: 100 posts, 50% engaged, bugsquad, …
30. summary identify values and goals avoid focusing on vanity metrics decide actions ahead of time assume inaccuracies experiment
31. thank you! james falkner liferay community manager @schtool 2014 FLOSS community metrics meeting portland, oregon #FCM2