TSE 2016: How communication channels challenge a participatory culture in software development

Slides of a guest talk I've given at University of Bari, TU Eindhoven, and TU Delft during June 2016. The paper described here is to be published at the IEEE Transactions on Software Engineering journal (TSE). This talk focuses mainly on the challenges developers face, while the paper further elaborates and extends our findings.

A preprint of the paper can be found here:
http://alexeyza.com/pdf/tse2016.pdf

Transcript

1. How social and communication channels
   shape and challenge a participatory
   culture in software development
   Margaret-Anne Storey, Alexey Zagalsky, Fernando
   Figueira Filho, Leif Singer, and Daniel M. German
   

   View Slide

2. PhD candidate under the guidance of Margaret-Anne Storey
   at the University of Victoria, Canada
   Main areas of interest:
   Social media in Software Engineering
   (Stack Overflow, GitHub, ...)
   Collaboration and CSCW, Human aspects in SE, HCI
   CSCL
   About me
   2
   

   View Slide

3. In the talk today
   I’ll talk about a single study, however, it is important to
   note that it is part of a long-term ongoing research
   The work described here is not solely my own and is based
   on collaborations with other researchers:
   Margaret-Anne Storey, Leif Singer, Daniel M. German, Fernando Figueira Filho
   This study was accepted to the Transactions of Software
   Engineering (TSE 2016) journal
   3
   

   View Slide

4. Communities of practice arise when “groups of people
   share a concern or a passion for something they do and
   learn how to do it better as they interact regularly”
   [Wenger 2000]
   4
   

   View Slide

5. The Importance of Media in Software Development
   “We shape our tools and thereafter our tools shape us”
   - Marshall Mcluhan
   The collaborative and participatory nature of software
   development continues to evolve, shape, and be shaped by
   communication channels
   Media—i.e., development tools and communication
   channels—play a critical role in how externalized and tacit
   knowledge is formed, shared, manipulated, and captured
   5
   

   View Slide

6. The study of Media in SE (just to name a few)
   Software Engineering
   at the Speed of Light:
   How Developers Stay
   Current Using Twitter
   [Singer et al. 2013]
   Mutual Assessment in the
   Social Programmer
   Ecosystem: An Empirical
   Investigation of Developer
   Profile Aggregators
   [Singer et al. 2012]
   How do programmers
   ask and answer
   questions on the web?
   [Treude et al. 2011]
   The impact of social
   media on software
   engineering
   practices and tools
   [Storey et al. 2010]
   Communication in open
   source software
   development mailing lists
   [Guzzi et al. 2013]
   Exploring the role of instant
   messaging in a global software
   development project
   [Dittrich and Giuffrida 2011]
   A survey of social
   media use in
   software systems
   development
   [Black et al. 2010]
   The (r)evolution of social
   media in software engineering
   [Storey et al. 2014]
   Why Developers
   Are Slacking Off:
   Understanding
   How Software
   Teams Use Slack
   [Lin et al. 2016]
   6
   

   View Slide

7. The study of Media in SE (just to name a few)
   Software Engineering
   at the Speed of Light:
   How Developers Stay
   Current Using Twitter
   [Singer et al. 2013]
   Mutual Assessment in the
   Social Programmer
   Ecosystem: An Empirical
   Investigation of Developer
   Profile Aggregators
   [Singer et al. 2012]
   How do programmers
   ask and answer
   questions on the web?
   [Treude et al. 2011]
   The impact of social
   media on software
   engineering
   practices and tools
   [Storey et al. 2010]
   Communication in open
   source software
   development mailing lists
   [Guzzi et al. 2013]
   Exploring the role of instant
   messaging in a global software
   development project
   [Dittrich and Giuffrida 2011]
   A survey of social
   media use in
   software systems
   development
   [Black et al. 2010]
   The (r)evolution of social
   media in software engineering
   [Storey et al. 2014]
   Why Developers
   Are Slacking Off:
   Understanding
   How Software
   Teams Use Slack
   [Lin et al. 2016]
   The rise of the
   Social Programmer
   7
   

   View Slide

8. RQ1: Who is the social programmer that participates in
   these communities?
   RQ2: Which communication channels do these developers
   use to support development activities?
   RQ3: Which communication channels are the most
   important to developers and why?
   RQ4: What challenges do developers face using an
   ecosystem of communication channels to support their
   activities?
   8
   

   View Slide

9. RQ1: Who is the social programmer that participates in
   these communities?
   RQ2: Which communication channels do these developers
   use to support development activities?
   RQ3: Which communication channels are the most
   important to developers and why?
   RQ4: What challenges do developers face using an
   ecosystem of communication channels to support their
   activities?
   9
   

   View Slide

10. RQ1: Who is the social programmer that participates in
    these communities?
    RQ2: Which communication channels do these developers
    use to support development activities?
    RQ3: Which communication channels are the most
    important to developers and why?
    RQ4: What challenges do developers face using an
    ecosystem of communication channels to support their
    activities?
    10
    

    View Slide

11. RQ1: Who is the social programmer that participates in
    these communities?
    RQ2: Which communication channels do these developers
    use to support development activities?
    RQ3: Which communication channels are the most
    important to developers and why?
    RQ4: What challenges do developers face using an
    ecosystem of communication channels to support their
    activities?
    11
    

    View Slide

12. Methodology
    We conducted an online survey with 1449 developers who
    are active on GitHub [deployed at the end of 2013 and 2014]
    Survey available online at http://leif.me/devsurvey/
    12
    

    View Slide

13. Methodology
    We asked about 11 development activities:
    STAY UP TO DATE about technologies,
    practices, and tools for software
    development
    FIND ANSWERS to technical questions
    LEARN and improve skills
    DISCOVER interesting developers
    CONNECT with interesting developers
    GET and GIVE FEEDBACK
    PUBLISH development activities
    WATCH other developers’ activities
    DISPLAY my SKILLS/ACCOMPLISHMENTS
    ASSESS other developers
    COORDINATE with other developers when
    participating on projects
    13
    

    View Slide

14. Methodology
    We asked about the challenges developers face using
    communication channels
    Based on our previous work, distractions, privacy, and
    being overwhelmed were anticipated to be the main
    challenges, thus the survey specifically inquired about them
    We also asked about any additional challenges developers
    face through an open-ended question (432 responses)
    14
    

    View Slide

15. RQ1: Social programmer demographics
    15
    

    View Slide

16. RQ1: Social programmer demographics
    16
    

    View Slide

17. RQ1: Social programmer demographics
    17
    

    View Slide

18. Image created by Kent Nguyen, showing the channels and tools used at Silicon Straits (August 2014)
    18
    

    View Slide

19. RQ2: Which communication channels do these
    developers use to support development
    activities?
    19
    

    View Slide

20. RQ2: How many channels developers use?
    20
    

    View Slide

21. View Slide

22. View Slide

23. View Slide

24. RQ3: Which communication channels are the
    most important to developers?
    We further elaborate on this in our paper and discuss why these
    channels are important to developers
    24
    

    View Slide

25. RQ3: Which channels are most important?
    25
    

    View Slide

26. RQ4: What challenges do developers face using
    an ecosystem of communication channels to
    support their activities?
    26
    

    View Slide

27. RQ4: What challenges do developers face?
    27
    

    View Slide

28. “Notifications: when used in a moderate way, it is fine, but
    when overused, it is a distraction for developers. Emails: too
    many emails from Project Coordination Tools can easily waste
    15-30 minutes only to go through them all in the morning,
    specially when I’m involved in more than a few projects
    simultaneously.” [P320]
    Distractions and interruptions from communication
    channels negatively impact developer productivity
    28
    

    View Slide

29. “Staying cutting edge is a never-ending task.” [P625]
    “In a big project (WebKit, Mozilla, etc.) it can be hard to filter
    for only ongoing work that is relevant. Most legacy UIs are
    terrible (Bugzilla) and new ones (GitHub) lack features for
    large-scale development.” [P109]
    Keeping up with new technologies and project activities
    can be challenging, but social tools help
    29
    

    View Slide

30. “The biggest challenge in soft-dev for me is four-fold:
    communicating the idea (Hangout), managing the idea
    (Trello), logging the implemented idea (GitHub), and
    explaining the implemented idea with the team (Nitrous.io).
    The first three solutions are pretty solid. It’s the fact you can’t
    always sit right next to someone and show them the code
    and explain how everything works that is the most
    challenging part. Cloud9, Koding, Nitrous, etc. are all trying
    to solve the last problem.” [P445]
    There is a lack of adequate tool support for sharing and
    explaining code
    30
    

    View Slide

31. “Getting quick feedback for internal [proprietary] technology
    because you cannot ask on Stack Overflow.” [P706]
    “It’s difficult to share small new projects that aren’t very far
    along and get feedback.” [P751]
    Getting feedback on development activities is challenging
    31
    

    View Slide

32. “Live collaborative coding tools. For example, we can currently
    edit a document collaboratively in Google Docs.
    Collaborative coding activities need improved tool support
    If we can have an
    IDE/tool like that for
    coding too, that
    would be useful.”
    [P1154]
    32
    

    View Slide

33. “The majority of development-related communication I do is
    primarily written—IRC, chat, email, forums, microblogging,
    blogging, etc. Considering that the developers I work with
    come from a variety of nationalities and cultural backgrounds,
    the intent of communication is often hard to impart or judge,
    which can lead to misunderstanding.” [P31]
    Geographic, cultural, and economic factors can pose
    barriers to participation through social channels
    33
    

    View Slide

34. “I used to think that publishing application code with an open
    source license would attract collaborators with an interest in
    using/improving that application, but now I feel like most
    users of code hosting sites are more interested in
    collaborating on tools they can incorporate into their own
    projects, and almost no one is interested in working on
    application code. I guess the challenge here is convincing
    developers that your application is interesting even if your
    code is not.” [P1285]
    Despite using social channels, finding developers to
    participate can be difficult
    34
    

    View Slide

35. “When you write, context and expression is lost. There have
    been so many times when something I wrote did not come
    across to the other person as intended. This causes problems
    all the time. You must be careful and spend time on the words
    and phrases you use when communicating in text. If you can’t
    pick up the phone, then use IM, and as a last resort, email.”
    [P126]
    Miscommunication on text-based channels is common
    35
    

    View Slide

36. “When you write, context and expression is lost. There have
    been so many times when something I wrote did not come
    across to the other person as intended. This causes problems
    all the time. You must be careful and spend time on the words
    and phrases you use when communicating in text. If you can’t
    pick up the phone, then use IM, and as a last resort, email.”
    [P126]
    Miscommunication on text-based channels is common
    “[I] have been called an arrogant dickhead once and have
    also been informed that I was dictating somebody
    something when I actually tried to SUGGEST something.
    So, language is also a challenge because I am a Dane.”
    [P31]
    36
    

    View Slide

37. “Clarity of intent (it can be hard to get a point across through
    text-based media) — it can be difficult to see the big picture,
    or even the pieces, without talking face-to-face.” [P1241]
    “Often collaboration tools are still not as good as face-to-
    face communication. Many software companies have moved
    to having all of their staff on-site full-time, because the
    communication is just better. Especially when two or more
    developers are collaborating on the same code in real time
    (pair programming) or ‘whiteboarding’ on a design... being
    there in person is just different.” [P176]
    For many developers, face-to-face communication is best
    37
    

    View Slide

38. “Tools facilitate good processes and interactions between
    individuals who are willing to collaborate and cooperate. They
    don’t make people willing to cooperate in the first place; in
    these situations they actually get in the way of identifying the
    root problems and dealing with them. People can hide behind
    GitHub better than they can in person.” [P532]
    People are challenging, no matter what channels are used
    38
    

    View Slide

39. Private vs. public
    Group size
    Synchronous vs. asynchronous
    Ephemeral vs. archival
    Anonymous vs. identified
    Text-based vs. verbal
    Face to face (vs. not)
    No one tool/channel fits all
    Communication with users is challenging
    Developers need to consider channel affordances
    39
    

    View Slide

40. “I still don’t understand how to do simple things in IRC and
    often don’t bother because of the perceived effort involved—
    much easier to post on Stack Overflow. With tools like GitHub,
    there are similar issues (like how to submit patches) although
    documentation is improving.” [P1385]
    Developers need to be literate with communication
    channels
    40
    

    View Slide

41. Technical issues
    Lack of mobile support and hardware limitations
    Usability issues
    Vendor lock-in
    Poor or scattered tool adoption
    Communication channel friction can obstruct participatory
    development activities
    41
    

    View Slide

42. “Making sure everyone else you’re working with also uses the
    tools. One of the biggest issues with fragmentation of the
    communication options is that there are so many different
    ways to communicate that it’s harder to find it all in one
    place. Important communications get lost; Key people don’t
    see them; They can’t be retrieved by a single search tool.
    Companies such as Slack are attempting to solve this problem,
    but it has a long way to go.” [P1250]
    Use of many channels leads to information fragmentation
    42
    

    View Slide

43. “Too many channels means that needed or interesting
    information disappears, and going through all of the channels
    you mentioned is impossible in limited time.” [P917]
    The quantity of communicated information is
    overwhelming
    43
    

    View Slide

44. The quantity of communicated information is
    overwhelming
    44
    

    View Slide

45. “Judging the reliability and credibility of sources can be a
    challenge as information changes quickly and isn’t always
    correct.” [P846]
    “Technologies are moving so fast, and most of the content on
    the Internet could be outdated quickly. It’s sometimes hard to
    filter that outdated information.” [P492]
    The quality of communicated information is hard to
    evaluate
    45
    

    View Slide

46. View Slide

47. Meta Discussion
    47
    

    View Slide

48. Challenges
    Qualitative research
    - forming the survey questions and deciding on the activities
    - studying channels, activities, roles, and developers - in parallel
    Conducting a long-term study
    - channels/tools change, do we update the survey?
    Reporting qualitative results
    - finding the “right way” to present the results
    - conference paper length is a limitation
    48
    

    View Slide

49. Where to next?
    A bit about our upcoming work
    49
    

    View Slide

50. How Slack is used in Software Development
    “If Slack is taken out,
    it’s just like turning our
    lights off. We’re blind."
    - an anonymous developer
    50
    

    View Slide

51. Understanding the role of Slack and how it shapes the
    bug fixing process
    in collaboration with
    Bin Lin, Alexander Serebrenik,
    and Margaret-Anne Storey
    figure shows an example of specific ticket being resolved with the use of Slack and Trac 51
    

    View Slide

52. The impact of Bots in software development
    It is believed that Bots will lead to an improvement in
    software quality and developer and team productivity
    What if that disruption has negative consequences?
    52
    

    View Slide

53. Developers
    Communities of
    Practice
    Tools & Channels
    IDE face-to-face
    Towards A Theory of Knowledge in Software Engineering
    53
    

    View Slide

54. Developers
    Communities of
    Practice
    Tools & Channels
    IDE face-to-face
    Activities
    Coding
    Learning
    54
    

    View Slide

55. Actors & Roles
    Contributors, Stakeholders
    Assemblages &
    Communities of
    Practice
    Teams, Organization
    Practices & Activities
    Tools & Channels
    Artifacts
    Code, Documentation,
    Q&A, History
    fosters / shaped by
    Agile Coding
    IDE face-to-face
    adapted from Alexey Zagalsky candidacy proposal
    55
    Tacit knowledge
    Knowledge as public good
    Externalized knowledge
    Knowledge on people &
    social networks
    

    View Slide

56. Questions?
    Alexey Zagalsky
    http://alexeyza.com
    56
    

    View Slide