TSE 2016: How communication channels challenge a participatory culture in software development

Slides of a guest talk I've given at University of Bari, TU Eindhoven, and TU Delft during June 2016. The paper described here is to be published at the IEEE Transactions on Software Engineering journal (TSE). This talk focuses mainly on the challenges developers face, while the paper further elaborates and extends our findings.

A preprint of the paper can be found here:
http://alexeyza.com/pdf/tse2016.pdf

Transcript

1. How social and communication channels shape and challenge a participatory culture in software development Margaret-Anne Storey, Alexey Zagalsky, Fernando Figueira Filho, Leif Singer, and Daniel M. German
2. PhD candidate under the guidance of Margaret-Anne Storey at the University of Victoria, Canada Main areas of interest: Social media in Software Engineering (Stack Overflow, GitHub, ...) Collaboration and CSCW, Human aspects in SE, HCI CSCL About me 2
3. In the talk today I’ll talk about a single study, however, it is important to note that it is part of a long-term ongoing research The work described here is not solely my own and is based on collaborations with other researchers: Margaret-Anne Storey, Leif Singer, Daniel M. German, Fernando Figueira Filho This study was accepted to the Transactions of Software Engineering (TSE 2016) journal 3
4. Communities of practice arise when “groups of people share a concern or a passion for something they do and learn how to do it better as they interact regularly” [Wenger 2000] 4
5. The Importance of Media in Software Development “We shape our tools and thereafter our tools shape us” - Marshall Mcluhan The collaborative and participatory nature of software development continues to evolve, shape, and be shaped by communication channels Media—i.e., development tools and communication channels—play a critical role in how externalized and tacit knowledge is formed, shared, manipulated, and captured 5
6. The study of Media in SE (just to name a few) Software Engineering at the Speed of Light: How Developers Stay Current Using Twitter [Singer et al. 2013] Mutual Assessment in the Social Programmer Ecosystem: An Empirical Investigation of Developer Profile Aggregators [Singer et al. 2012] How do programmers ask and answer questions on the web? [Treude et al. 2011] The impact of social media on software engineering practices and tools [Storey et al. 2010] Communication in open source software development mailing lists [Guzzi et al. 2013] Exploring the role of instant messaging in a global software development project [Dittrich and Giuffrida 2011] A survey of social media use in software systems development [Black et al. 2010] The (r)evolution of social media in software engineering [Storey et al. 2014] Why Developers Are Slacking Off: Understanding How Software Teams Use Slack [Lin et al. 2016] 6
7. The study of Media in SE (just to name a few) Software Engineering at the Speed of Light: How Developers Stay Current Using Twitter [Singer et al. 2013] Mutual Assessment in the Social Programmer Ecosystem: An Empirical Investigation of Developer Profile Aggregators [Singer et al. 2012] How do programmers ask and answer questions on the web? [Treude et al. 2011] The impact of social media on software engineering practices and tools [Storey et al. 2010] Communication in open source software development mailing lists [Guzzi et al. 2013] Exploring the role of instant messaging in a global software development project [Dittrich and Giuffrida 2011] A survey of social media use in software systems development [Black et al. 2010] The (r)evolution of social media in software engineering [Storey et al. 2014] Why Developers Are Slacking Off: Understanding How Software Teams Use Slack [Lin et al. 2016] The rise of the Social Programmer 7
8. RQ1: Who is the social programmer that participates in these communities? RQ2: Which communication channels do these developers use to support development activities? RQ3: Which communication channels are the most important to developers and why? RQ4: What challenges do developers face using an ecosystem of communication channels to support their activities? 8
9. RQ1: Who is the social programmer that participates in these communities? RQ2: Which communication channels do these developers use to support development activities? RQ3: Which communication channels are the most important to developers and why? RQ4: What challenges do developers face using an ecosystem of communication channels to support their activities? 9
10. RQ1: Who is the social programmer that participates in these communities? RQ2: Which communication channels do these developers use to support development activities? RQ3: Which communication channels are the most important to developers and why? RQ4: What challenges do developers face using an ecosystem of communication channels to support their activities? 10
11. RQ1: Who is the social programmer that participates in these communities? RQ2: Which communication channels do these developers use to support development activities? RQ3: Which communication channels are the most important to developers and why? RQ4: What challenges do developers face using an ecosystem of communication channels to support their activities? 11
12. Methodology We conducted an online survey with 1449 developers who are active on GitHub [deployed at the end of 2013 and 2014] Survey available online at http://leif.me/devsurvey/ 12
13. Methodology We asked about 11 development activities: STAY UP TO DATE about technologies, practices, and tools for software development FIND ANSWERS to technical questions LEARN and improve skills DISCOVER interesting developers CONNECT with interesting developers GET and GIVE FEEDBACK PUBLISH development activities WATCH other developers’ activities DISPLAY my SKILLS/ACCOMPLISHMENTS ASSESS other developers COORDINATE with other developers when participating on projects 13
14. Methodology We asked about the challenges developers face using communication channels Based on our previous work, distractions, privacy, and being overwhelmed were anticipated to be the main challenges, thus the survey specifically inquired about them We also asked about any additional challenges developers face through an open-ended question (432 responses) 14
15. RQ1: Social programmer demographics 15
16. RQ1: Social programmer demographics 16
17. RQ1: Social programmer demographics 17
18. Image created by Kent Nguyen, showing the channels and tools used at Silicon Straits (August 2014) 18
19. RQ2: Which communication channels do these developers use to support development activities? 19
20. RQ2: How many channels developers use? 20
24. RQ3: Which communication channels are the most important to developers? We further elaborate on this in our paper and discuss why these channels are important to developers 24
25. RQ3: Which channels are most important? 25
26. RQ4: What challenges do developers face using an ecosystem of communication channels to support their activities? 26
27. RQ4: What challenges do developers face? 27
28. “Notifications: when used in a moderate way, it is fine, but when overused, it is a distraction for developers. Emails: too many emails from Project Coordination Tools can easily waste 15-30 minutes only to go through them all in the morning, specially when I’m involved in more than a few projects simultaneously.” [P320] Distractions and interruptions from communication channels negatively impact developer productivity 28
29. “Staying cutting edge is a never-ending task.” [P625] “In a big project (WebKit, Mozilla, etc.) it can be hard to filter for only ongoing work that is relevant. Most legacy UIs are terrible (Bugzilla) and new ones (GitHub) lack features for large-scale development.” [P109] Keeping up with new technologies and project activities can be challenging, but social tools help 29
30. “The biggest challenge in soft-dev for me is four-fold: communicating the idea (Hangout), managing the idea (Trello), logging the implemented idea (GitHub), and explaining the implemented idea with the team (Nitrous.io). The first three solutions are pretty solid. It’s the fact you can’t always sit right next to someone and show them the code and explain how everything works that is the most challenging part. Cloud9, Koding, Nitrous, etc. are all trying to solve the last problem.” [P445] There is a lack of adequate tool support for sharing and explaining code 30
31. “Getting quick feedback for internal [proprietary] technology because you cannot ask on Stack Overflow.” [P706] “It’s difficult to share small new projects that aren’t very far along and get feedback.” [P751] Getting feedback on development activities is challenging 31
32. “Live collaborative coding tools. For example, we can currently edit a document collaboratively in Google Docs. Collaborative coding activities need improved tool support If we can have an IDE/tool like that for coding too, that would be useful.” [P1154] 32
33. “The majority of development-related communication I do is primarily written—IRC, chat, email, forums, microblogging, blogging, etc. Considering that the developers I work with come from a variety of nationalities and cultural backgrounds, the intent of communication is often hard to impart or judge, which can lead to misunderstanding.” [P31] Geographic, cultural, and economic factors can pose barriers to participation through social channels 33
34. “I used to think that publishing application code with an open source license would attract collaborators with an interest in using/improving that application, but now I feel like most users of code hosting sites are more interested in collaborating on tools they can incorporate into their own projects, and almost no one is interested in working on application code. I guess the challenge here is convincing developers that your application is interesting even if your code is not.” [P1285] Despite using social channels, finding developers to participate can be difficult 34
35. “When you write, context and expression is lost. There have been so many times when something I wrote did not come across to the other person as intended. This causes problems all the time. You must be careful and spend time on the words and phrases you use when communicating in text. If you can’t pick up the phone, then use IM, and as a last resort, email.” [P126] Miscommunication on text-based channels is common 35
36. “When you write, context and expression is lost. There have been so many times when something I wrote did not come across to the other person as intended. This causes problems all the time. You must be careful and spend time on the words and phrases you use when communicating in text. If you can’t pick up the phone, then use IM, and as a last resort, email.” [P126] Miscommunication on text-based channels is common “[I] have been called an arrogant dickhead once and have also been informed that I was dictating somebody something when I actually tried to SUGGEST something. So, language is also a challenge because I am a Dane.” [P31] 36
37. “Clarity of intent (it can be hard to get a point across through text-based media) — it can be difficult to see the big picture, or even the pieces, without talking face-to-face.” [P1241] “Often collaboration tools are still not as good as face-to- face communication. Many software companies have moved to having all of their staff on-site full-time, because the communication is just better. Especially when two or more developers are collaborating on the same code in real time (pair programming) or ‘whiteboarding’ on a design... being there in person is just different.” [P176] For many developers, face-to-face communication is best 37
38. “Tools facilitate good processes and interactions between individuals who are willing to collaborate and cooperate. They don’t make people willing to cooperate in the first place; in these situations they actually get in the way of identifying the root problems and dealing with them. People can hide behind GitHub better than they can in person.” [P532] People are challenging, no matter what channels are used 38
39. Private vs. public Group size Synchronous vs. asynchronous Ephemeral vs. archival Anonymous vs. identified Text-based vs. verbal Face to face (vs. not) No one tool/channel fits all Communication with users is challenging Developers need to consider channel affordances 39
40. “I still don’t understand how to do simple things in IRC and often don’t bother because of the perceived effort involved— much easier to post on Stack Overflow. With tools like GitHub, there are similar issues (like how to submit patches) although documentation is improving.” [P1385] Developers need to be literate with communication channels 40
41. Technical issues Lack of mobile support and hardware limitations Usability issues Vendor lock-in Poor or scattered tool adoption Communication channel friction can obstruct participatory development activities 41
42. “Making sure everyone else you’re working with also uses the tools. One of the biggest issues with fragmentation of the communication options is that there are so many different ways to communicate that it’s harder to find it all in one place. Important communications get lost; Key people don’t see them; They can’t be retrieved by a single search tool. Companies such as Slack are attempting to solve this problem, but it has a long way to go.” [P1250] Use of many channels leads to information fragmentation 42
43. “Too many channels means that needed or interesting information disappears, and going through all of the channels you mentioned is impossible in limited time.” [P917] The quantity of communicated information is overwhelming 43
44. The quantity of communicated information is overwhelming 44
45. “Judging the reliability and credibility of sources can be a challenge as information changes quickly and isn’t always correct.” [P846] “Technologies are moving so fast, and most of the content on the Internet could be outdated quickly. It’s sometimes hard to filter that outdated information.” [P492] The quality of communicated information is hard to evaluate 45
47. Meta Discussion 47
48. Challenges Qualitative research - forming the survey questions and deciding on the activities - studying channels, activities, roles, and developers - in parallel Conducting a long-term study - channels/tools change, do we update the survey? Reporting qualitative results - finding the “right way” to present the results - conference paper length is a limitation 48
49. Where to next? A bit about our upcoming work 49
50. How Slack is used in Software Development “If Slack is taken out, it’s just like turning our lights off. We’re blind." - an anonymous developer 50
51. Understanding the role of Slack and how it shapes the bug fixing process in collaboration with Bin Lin, Alexander Serebrenik, and Margaret-Anne Storey figure shows an example of specific ticket being resolved with the use of Slack and Trac 51
52. The impact of Bots in software development It is believed that Bots will lead to an improvement in software quality and developer and team productivity What if that disruption has negative consequences? 52
53. Developers Communities of Practice Tools & Channels IDE face-to-face Towards A Theory of Knowledge in Software Engineering 53
54. Developers Communities of Practice Tools & Channels IDE face-to-face Activities Coding Learning 54
55. Actors & Roles Contributors, Stakeholders Assemblages & Communities of Practice Teams, Organization Practices & Activities Tools & Channels Artifacts Code, Documentation, Q&A, History fosters / shaped by Agile Coding IDE face-to-face adapted from Alexey Zagalsky candidacy proposal 55 Tacit knowledge Knowledge as public good Externalized knowledge Knowledge on people & social networks
56. Questions? Alexey Zagalsky http://alexeyza.com 56