Metrics are fun, but which ones really matter?

Metrics are fun, but which ones really matter?

There is no shortage of algorithms, techniques, and software to produce virtually any metric one may desire. True value is only realized when metrics are chosen that most closely align with your goals.

In this session, learn which goals we chose in the Liferay project (a large, open source portal project), why they were (and continue to be) important, how they are produced, and what we are doing with them.

Transcript

  1. metrics are fun, but which ones really matter? james falkner

    liferay community manager @schtool 2014 FLOSS community metrics meeting portland, oregon #FCM2
  2. James Falkner, Liferay Community Manager

  3. liferay community metrics 13 years 4.3M LOC 115k registered site

    members 102 external contributors in 2013 150 SI partners 300 marketplace apps 800 marketplace developers 43 user groups 114 translators for 43 languages 300k forum posts (~100/day) 25k forum participants 2 Top 50 github devs ( , ) #5 #45
  4. more metrics

  5. MOAR METRICS!!

  6. … and more

  7. allthingsd.com/20121217/andreessen-and-mixpanel-call-for-an-end-to-bullshit-metrics/!

  8. “companies need to start using a new set of metrics

    that don’t simply make them feel good. They should use actionable metrics that provide insight, provide guidance, and help businesses make better decisions.” - Suhail Doshi, mixpanel founder
  9. core contributions

  10. downloads

  11. step #1 “search your feelings”

  12. step #2 identify your goals for metrics

  13. goal #1: increase the value of participation

  14. www.youtube.com/watch?v=u6XAPnuFjJc

  15. None
  16. goal #2: Measure the effect of business decisions

  17. goal #3: understand the relationship between metrics and business performance

  18. None
  19. step #3 know what to do with the results

  20. basic 1st-order metrics for the liferay community # of apps/reviews

    # of contributions # of commits/lines # of authors # of bug reports # of forum posts/answers # of downloads # ignored messages # open tickets / code reviews
  21. more interesting non-liferay vs. liferay crowdsourced quality time between contribution->codebase

    location of community members evolution of contributor age over time evolution of active/inactive users time of bug report -> fixed time from forum q to a # of ignored contributions distribution of commits across functional areas
  22. metrics.forEach(function(metric) { How accurate is it? How can it help

    achieve your goals? What to do when it turns good/bad? What is its relationship to others? });
  23. None
  24. Code

  25. Forums

  26. Demographics

  27. Community Contribution Performance

  28. future: cmty career evolution

  29. newbie: 0-5 posts, 0 answers, 1 download, 1 fork, …

    junior: 20 posts, 10% engaged, >1 patch, 5 tweets, 1 app senior: 50 posts, 30% engaged, 10 tweets, 5 apps, moderator, … champion: 100 posts, 50% engaged, bugsquad, …
  30. summary identify values and goals avoid focusing on vanity metrics

    decide actions ahead of time assume inaccuracies experiment
  31. thank you! james falkner liferay community manager @schtool 2014 FLOSS

    community metrics meeting portland, oregon #FCM2