Metrics are fun, but which ones really matter?

Metrics are fun, but which ones really matter?

There is no shortage of algorithms, techniques, and software to produce virtually any metric one may desire. True value is only realized when metrics are chosen that most closely align with your goals.

In this session, learn which goals we chose in the Liferay project (a large, open source portal project), why they were (and continue to be) important, how they are produced, and what we are doing with them.

Transcript

  1. 1.

    metrics are fun, but which ones really matter? james falkner

    liferay community manager @schtool 2014 FLOSS community metrics meeting portland, oregon #FCM2
  2. 3.

    liferay community metrics 13 years 4.3M LOC 115k registered site

    members 102 external contributors in 2013 150 SI partners 300 marketplace apps 800 marketplace developers 43 user groups 114 translators for 43 languages 300k forum posts (~100/day) 25k forum participants 2 Top 50 github devs ( , ) #5 #45
  3. 8.

    “companies need to start using a new set of metrics

    that don’t simply make them feel good. They should use actionable metrics that provide insight, provide guidance, and help businesses make better decisions.” - Suhail Doshi, mixpanel founder
  4. 10.
  5. 15.
  6. 18.
  7. 20.

    basic 1st-order metrics for the liferay community # of apps/reviews

    # of contributions # of commits/lines # of authors # of bug reports # of forum posts/answers # of downloads # ignored messages # open tickets / code reviews
  8. 21.

    more interesting non-liferay vs. liferay crowdsourced quality time between contribution->codebase

    location of community members evolution of contributor age over time evolution of active/inactive users time of bug report -> fixed time from forum q to a # of ignored contributions distribution of commits across functional areas
  9. 22.

    metrics.forEach(function(metric) { How accurate is it? How can it help

    achieve your goals? What to do when it turns good/bad? What is its relationship to others? });
  10. 23.
  11. 24.
  12. 25.
  13. 29.

    newbie: 0-5 posts, 0 answers, 1 download, 1 fork, …

    junior: 20 posts, 10% engaged, >1 patch, 5 tweets, 1 app senior: 50 posts, 30% engaged, 10 tweets, 5 apps, moderator, … champion: 100 posts, 50% engaged, bugsquad, …
  14. 30.

    summary identify values and goals avoid focusing on vanity metrics

    decide actions ahead of time assume inaccuracies experiment
  15. 31.

    thank you! james falkner liferay community manager @schtool 2014 FLOSS

    community metrics meeting portland, oregon #FCM2