How to use analytics to improve responsiveness in community management

Talk at Community Leadership Summit 2014, Portland, OR, USA.

Large communities, and specially those collaborating to produce some sort of common result, are complex and difficult to understand. It is very difficult to track what's happening in them, to react early to problems, and to help them to grow healthy.

Fortunately, community analytics come to rescue, enabling the community manager to get a knowledge of what's happening that is, in many cases, very difficult to obtain without their help. I will explain how analytics can be used not only to improve this knowledge, but to track the effect of corrective measures, to detect early deviations from intended evolution, and to provide a more person-oriented management of the community.

I will use as examples real cases from free / open source software development communities.

Transcript

1. How to use analytics to improve responsiveness in community management Jesus M. Gonzalez-Barahona jgb@bitergia.com @jgbarah Bitergia / LibreSoft (URJC) http://bit.ly/clsanalytics Community Leadership Summit 2014 Portland, OR (USA), July 18st 2014 Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 1 / 30
2. c 2012-2014 Bitergia Some rights reserved. This presentation is distributed under the “Attribution-ShareAlike 3.0” license, by Creative Commons, available at http://creativecommons.org/licenses/by-sa/3.0/ Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 2 / 30
3. Structure of the presentation 1 About me 2 Dealing with dynamic complexity 3 Example: attraction / retention 4 Example: people entering / leaving 5 Example: performance 6 Example: what are people doing? 7 Final remarks Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 3 / 30
4. About me Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 4 / 30
5. My Uni, my company Uni Rey Juan Carlos: LibreSoft research team Understanding free, open source software development Data analytics approach Bitergia: From research to the real world The software development analytics company Dashboards, reports, consultancy... http://gsyc.es/~jgb http://bitergia.com Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 5 / 30
6. Dealing with dynamic complexity Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 6 / 30
7. Communities may be large and complex [Crowd at FOSDEM 2008, by Jes´ us Corrius, CC Attribution 2.0] http://www.flickr.com/photos/jcorrius/2302302707/ Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 7 / 30
8. Communities may be large and complex... and dynamic It’s difficult to... ...track what’s happening ...understand why it’s happening ...react quickly ...evaluate results of reaction If data is available analytics may come to the rescue Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 8 / 30
9. A continuous process Figure out your interest Find out available data Define key parameters Monitor, understand, detect deviations Act to correct, improve Track results Measure → Monitor → Act Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 9 / 30
10. A continuous process (example) Case: company-led development community Interest: activity Data: changes to code, tickets Parameters: commits, tickets closed Monitoring: charts, numbers Observation: numbers falling down Actions: allocate more developer effort Track results... Measure → Monitor → Act Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 10 / 30
11. Example: tracking activity [Wikimedia activity, July 2014] Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 11 / 30
12. Unfortunately, real cases may be more difficult... Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 12 / 30
13. Example: attraction / retention Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 13 / 30
14. How are you attracting / retaining talent? Interests: people entering / leaving structure of “age” future structure Assumption: people leaving won’t come back Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 14 / 30
15. How are you attracting / retaining talent? [OpenStack authors community: Attraction / retention, July 2014] Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 15 / 30
16. How are you attracting / retaining talent? [OpenStack authors community: Attraction / retention, July 2013] Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 16 / 30
17. Example: people entering / leaving Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 17 / 30
18. Welcoming newcomers Wikimedia: new submitters of code for review http://korma.wmflabs.org/browser/code_contrib_new_gone.html Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 18 / 30
19. Detecting people maybe leaving Wikimedia: frequent contributors not contributing anymore http://korma.wmflabs.org/browser/code_contrib_new_gone.html Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 19 / 30
20. Example: performance Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 20 / 30
21. How long to address issues? Liferay: time-to-close tickets (quantiles) http://blog.bitergia.com/ 2012/10/25/preview-of-the-analysis-of-liferay/ Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 21 / 30
22. How is your community dealing with issues? [MediaWiki community: tickets-related parameters] Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 22 / 30
23. Example: what are people doing? Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 23 / 30
24. In which parts of the system are your developers active? actions authors per actions 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 arch/arc arch/arm arch/i386 arch/mips arch/powerpc arch/s390 arch/tile arch/x86 drivers/acpi drivers/char drivers/dma drivers/gpu drivers/media drivers/net drivers/scsi drivers/spi drivers/staging drivers/tty drivers/usb fs/btrfs fs/ceph fs/cifs fs/ext4 fs/f2fs fs/nfs fs/xfs include/linux net/ipv4 arch Documentation drivers fs include kernel net sound tools virt [Linux kernel: changes per directory during 2013] Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 24 / 30
25. Where are your developers working? (2) actions authors per actions 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 arch/alpha arch/arm arch/i386 arch/ia64 arch/m68k arch/mips arch/powerpc arch/ppc arch/ppc64 arch/sh arch/sparc64 arch/um arch/x86 arch/x86_64 drivers/acpi drivers/block drivers/char drivers/ide drivers/input drivers/isdn drivers/media drivers/net drivers/scsi drivers/usb drivers/video fs/jfs fs/nfsfs/nfsd fs/ntfs fs/xfs include/asm−x86 include/linux net/inet net/ipv4 net/sctp sound/oss sound/pci arch block drivers fs include kernel lib net sound [Linux kernel: changes per directory during 2002] Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 25 / 30
26. Final remarks Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 26 / 30
27. Do you need a community dashboard? [Grimoire Dashboard for the Eclipse community] Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 27 / 30
28. FYI: EU-funded R&D projects Markos: License analyzer New tools for software development analysis Production of linked open data PROSE: Software development analytics to track results of R&D projects Open Source Projects Europe forge: development analytics facilities http://www.markosproject.eu/ http://www.ict-prose.eu/ https://opensourceprojects.eu/ Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 28 / 30
29. Announce: FLOSS Community Metrics Meeting Bringing together people interested in FLOSS (free, libre, open source software) metrics to discuss experiences & FLOSS tools Sunday, 12:00 noon, Puppet Labs, Portland Come and join us!! #fcm2 http://flosscommunitymetrics.org/ Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 29 / 30
30. Summarizing... Measuring is a starting point for understanding You cannot improve what you cannot measure Some dashboards: http://bitergia.com/dashboards This presentation: http://bit.ly/clsanalytics Jesus Gonzalez-Barahona (Bitergia) Analytics to improve responsiveness CLS 2014 30 / 30