Affective Trust as a Predictor of Successful Collaboration in Distributed Software Projects

#icse16 #semotionws

Transcript

1. Affective Trust as a Predictor of Successful Collaboration in Distributed Software Projects Fabio Calefato, Filippo Lanubile University of Bari, Italy SEmotion @ ICSE’16 May 17, 2016 – Austin, TX, USA
2. University of Bari • Since 1924 • 2nd largest university of South Italy •  60,000 students,  1,500 professors,  1,400 employees • 4 main campuses: Science and Engineering, Economics, Law and Humanities, Medicine
3. People at Collab • Faculty – Filippo Lanubile – Fabio Calefato – Nicole Novielli • Graduate Students • Final-year undergraduate students Collaborative Development Group Department of Computer Science – http://collab.di.uniba.it
4. Research at COLLAB • Distributed Development – Addressing the problem of distance in distributed workgroups • Collaborative / Global / Social Software Engineering – Special focus on software development as an intense collaborative process
5. A Bit of Theory on Trust 5 INTEGRITY The adherence to intrinsic moral norms which makes a trustee reliable BENEVOLENCE The perceived level of courtesy and positive attitude ABILITY Capability of a trustee (based on knowledge, competence, skills) to perform tasks within a specific domain PREDICTABILITY The degree to which a person is liable and accountable and meets the expectation of another person Cognitive Trust Affective Trust Trustee’s antecedents to trust Trustor’s antecedent to trust PROPENSITY TO TRUST A general, not experience-based inclination to display faith and adopt a trusting attitude toward others
6. Why We Care About Trust • Trust fundamental to build “sense of teamness” • Lack of trust can mean: – Unwillingness to cooperate or share information – Perception of being on separate teams – Decreased goodwill towards others • The detrimental effects of lack of trust indirectly affect project performance (costs, speed, …) – Especially in case of distributed collaboration (no F2F interaction) 6 Al-Ani et al, “Trust in Virtual Teams: Theory and Tools.” CSCW (2013) Bradner, & Mark, “Why distance matters: effects on cooperation, persuasion and deception.” CSCW (2002) Sabherwal, “The role of trust in outsourced IS development projects.” CACM (1999) Wilson et al, “All in due time: The development of trust in computer-mediated and face-to-face groups.” (2006)
7. Building Trust in Distributed Software Teams Previous works • We proposed SocialCDE, a socio-technical system that links Sw artifacts to personal info within the workspace • Others observed socially- oriented communication finding its way in OSS projects’ communication channels: – Wang & Redmiles • Cheap-talk on IRC channels – Guzzi et al. • Social interactions occurring in mailing lists 7 Formal Communication ≈ ≈ Socially-oriented Communication Remote Conferencing Distance Calefato & Lanubile, “Augmenting social awareness in a collaborative development environment” CHASE (2012) Wang & Redmiles, “Cheap talk, cooperation, and trust in global software engineering.” ESE (2015) Guzzi et al. “Communication in open source software development mailing lists.” MSR (2013) ??? Build Affective Trust
8. What We Propose • No previous study has provided evidence connecting affective trust to project performance – Project performance ≈ Successful collaborations – Successful collaboration ≈ Pull Request (PR) • Previous studies on trust have typically relied on self-reported data (questionnaires) to “measure” it – Affective trust ≈ Amount of Affective lexicon in PRs – Trust(Devi ,Devj ,PRx ) ≈ ∑AffectiveLexicon(PRi ) Semotion’16 @ ICSE 2016 8 i
9. Research Model & Hypotheses • H1 – In distributed projects, the amount of affective lexicon in PR comments decreases over time, as affective trust mutually develops between developers • H2 – In distributed projects, the larger the amount of affective lexicon exchanged in prior PR comments between the contributor and the integration manager, the larger the chances for the current PR to be accepted H1 H2 Successful collaborations Affective trust antecedents Affective trust develops over time Social communication between devs 9 Treinen & Miller-Frost (IBM Syst. J, 2006) Jarvenpa & Leidner (J Org. Sci., 1999) Wang & Redmiles (ESE J, 2015)
10. • Annotation study on the Apache Software Foundation issue tracking system – Wide range of both positive and negative emotions Murgia et al., 2014 – Do developers feel emotions? An exploratory analysis of emotions in software artifacts. MSR 2014. Do Developers Express Emotions? Thanks for your input! You're, like, awesome I'm happy with the approach and the code looks good I will come over to your workplace and slap you Sorry for the delay Stephen
11. Sentiment Analysis in Software Engineering MSR‘15 Mining Successful Answers in Stack Overflow (Calefato et al.) Peer J ‘13 Happy software developers solve problems better Peer J (Graziotin et al.) ESEC/FSE ‘13 Towards emotional awareness in software development teams. (Guzman and Bruegge) ICSE‘15 Stuck and Frustrated or In Flow and Happy: Sensing Developers’ Emotions and Progress. (Muller and Fritz) MSR ‘14 Sentiment analysis of commit comments in GitHub: An empirical study (Guzman et al.) RE‘14 How Do Users Like this Feature? A Fine Grained Sentiment Analysis of App Reviews (Guzman and Maalej) Security and emotion: sentiment analysis of security discussions on GitHub (Pletea et al.) CHASE ‘15 Exploring Causes of Frustration for Software Developers. (Ford and Parnin) Towards emotion-based collaborative software engineering (Dewan) Do developers feel emotions? (Murgia et al.) 2013 2014 2015 ‘13 Do moods affect programmers’ debug performance? (Kahn et al) CT&W ‘11 2011 XP‘15 Would you mind fixing this issue? (Ortu et al.)
12. Sentiment Analysis: Tools • Several tools available • Classification of a text according to its positive, negative or neutral semantic orientation – NLTK • Outputs probability for each orientation • Trained on tweets and movie reviews – Stanford Sentiment Analyzer • Issues an overall orientation label + representation of the sentence structure • Trained on movie reviews – SentiStrength • Outputs a score for both positive and negative sentiment orientation • Designed for and validated on general purpose social media NLTK, http://text-processing.com Stanford Sentiment Analyser, http://nlp.stanford.edu/sentiment SentiStrength, http://sentistrength.wlv.ac.uk
13. Domain-dependency in Sentiment Analysis • Sentiment-analysis tools reliable only when used on the same corpora they are trained on – High risk of emotion misclassification • No publicly available analyzer trained on Software Engineering corpora yet • False positives in negative sentiment detection – Domain lexicon ‘What is the best way to kill a critical process’ – Contextual semantics ‘I am missing a parenthesis. But where? – Context of interaction (Q&A) ‘I have a problem, […] please explain what is wrong’ Novielli et al. “The Challenges of Sentiment Detection in the Social Programmer Ecosystem.” SSE (2015)
14. Developers’ Interaction in Pull Requests • The LGTM (“Looks Good To Me”) situation • Building affective trust through interactions – How many PRs before enough trust is established? – How many PRs before a change in affective language is observed? Semotion’16 @ ICSE 2016 14
15. Current & Future Work Gold Standard & Sentiment Analyzer for SE Preliminary Qualitative Analysis • Exploring PRs in GitHub, seeking explicative cases of affective-language shift – Picked from a dataset of “highly discussed” PRs Sentiment Gold standard Train a sentiment analyzer for SE (technical texts) Dump May ‘15 Preprocessing (removed code, urls..) Manual annotation (multiple raters) Tsay et al. “Let's talk about it: evaluating contributions through discussion in GitHub.” FSE (2014) 10+M questions
16. Thanks!