Work Practices and Challenges in Pull-Based Development: The Contributor’s Perspective

Presentation given by Georgios Gousios and Alberto Bacchelli at the 38th International Conference on Software Engineering.

Transcript

1. Work Practices and Challenges in Pull-Based Development: The Contributor’s Perspective Georgios Gousios @gousiosg Margaret-Anne Storey @margeretstorey Alberto Bacchelli @sback_ pull-based contributions
2. Core developers Co-developers Community Users Open-source communities: The onion model
3. GitHub’s pull-based development model core developer contributor (integrator)
4. projects with > 1 integrators 55% shared repository 45% use pull requests GitHub’s pull-based development model: A success story 0k 100k 200k 300k 400k 500k 600k 700k 2010 2011 2012 2013 2014 2015 # of pull requests 2016 135k repos/month 600k pull-requests/month
5. The integrator’s perspective
6. The integrator’s perspective: Plenty of research
7. The integrator’s perspective: Plenty of research
9. Decide change Code/Check quality Submit Discuss intentions Feedback Code review Submit Proposed fixes Discuss fixes Code review Accept Code/Check quality contributors integrators
10. Decide change Code/Check quality Submit Discuss intentions Feedback Code review Submit Proposed fixes Discuss fixes Code review Accept Code/Check quality contributors integrators
11. Decide change Code/Check quality Submit Discuss intentions Feedback Code review Submit Proposed fixes Discuss fixes Code review Accept Code/Check quality The contributor’s perspective contributors integrators
12. Research Questions
13. RQ1: How do contributors prepare contributions? Research Questions
14. RQ1: How do contributors prepare contributions? 1. What do they do before & after coding? Research Questions
15. RQ1: How do contributors prepare contributions? 1. What do they do before & after coding? 2. How do they assess their contribution’s quality? Research Questions
16. RQ1: How do contributors prepare contributions? 1. What do they do before & after coding? 2. How do they assess their contribution’s quality? 3. How do they communicate the intended change? Research Questions
17. RQ1: How do contributors prepare contributions? 1. What do they do before & after coding? 2. How do they assess their contribution’s quality? 3. How do they communicate the intended change? RQ2: What are the key challenges they face? Research Questions
18. GHTorrent Final Survey Performance Reports The contributor’s perspective: Research method
19. GHTorrent Final Survey Performance Reports 4,167 contributors (760 responses) The contributor’s perspective: Research method
20. GHTorrent Final Survey Analysis / filtering (15 closed questions) Performance Reports researchers Card sorting (7 open questions) 4,167 contributors (760 responses) The contributor’s perspective: Research method
21. GHTorrent Final Survey Analysis / filtering (15 closed questions) Performance Reports researchers Card sorting (7 open questions) 4,167 contributors (760 responses) Results The contributor’s perspective: Research method
22. RQ1: How do contributors prepare contributions? 1. What do they do before & after coding? 2. How do they assess their contribution’s quality? 3. How do they communicate the intended change? RQ2: What are the key challenges they face? Research Questions
23. % of responses The contributor’s perspective: Before coding a contribution Look up open issues Check for similar PRs Check rel. discussions Check guidelines Communicate changes Check other forks 0 50 100 150 20 Never or Occasionaly Often or Always 100 0 100
24. Run tests Check against guidelines Check rel. discussions Check for similar PRs Check other branches 0 50 100 150 20 Never or Occasionaly Often or Always % of responses The contributor’s perspective: After coding a contribution 100 0 100
25. RQ1: How do contributors prepare contributions? 1. What do they do before & after coding? 2. How do they assess their contribution’s quality? 3. How do they communicate the intended change? RQ2: What are the key challenges they face? Research Questions
26. Testing Self review Manual testing Compliance Static Analysis CI Self containment Documentation Building 0 15 30 45 60 % of responses The contributor’s perspective: Assessing their contribution’s quality What is evaluated How it is evaluated
27. Testing Self review Manual testing Compliance Static Analysis CI Self containment Documentation Building 0 15 30 45 60 % of responses The contributor’s perspective: Assessing their contribution’s quality What is evaluated How it is evaluated
28. Testing Self review Manual testing Compliance Static Analysis CI Self containment Documentation Building 0 15 30 45 60 % of responses The contributor’s perspective: Assessing their contribution’s quality What is evaluated How it is evaluated
29. seeking information related to change The contributor’s perspective: What happens before submitting not communicating intent verifying quality using testing trying to adhere to project style
30. RQ1: How do contributors prepare contributions? 1. What do they do before & after coding? 2. How do they assess their contribution’s quality? 3. How do they communicate the intended change? RQ2: What are the key challenges they face? Research Questions
31. Issue tracker Pull request Email IRC F2F Sync communication Twitter Other Do not communicate 0 20 40 60 80 % of responses The contributor’s perspective: Communicating the intended change
32. RQ1: How do contributors prepare contributions? 1. What do they do before & after coding? 2. How do they assess their contribution’s quality? 3. How do they communicate the intended change? RQ2: What are the key challenges they face? Research Questions
33. The contributor’s perspective: Key challenges they face Implications for researchers Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges
34. The contributor’s perspective: Key challenges they face Implications for researchers Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Understanding the code Evaluate the impact Ensure project compliance Dealing with git and conflicts “Missing knowledge about inner workings of a project [...] sometimes caused by missing documentation”
35. “Ensuring that my PR doesn’t have unintended side effects due to not being intimately familiar with the entire code base.” The contributor’s perspective: Key challenges they face Implications for researchers Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Understanding the code Evaluate the impact Ensure project compliance Dealing with git and conflicts
36. “Unsure of code style, pull requests format, commit message formats, etc.” The contributor’s perspective: Key challenges they face Implications for researchers Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Understanding the code Evaluate the impact Ensure project compliance Dealing with git and conflicts
37. “Usage of git is not intuitive. Especially for me as [one] who does not contribute regularly, it is every time a challenge to [use it]” The contributor’s perspective: Key challenges they face Implications for researchers Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Understanding the code Evaluate the impact Ensure project compliance Dealing with git and conflicts
38. The contributor’s perspective: Key challenges they face Implications for researchers Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Implications for researchers Dealing with unresponsiveness
 Explaining the rationale
 Fearing rejection
 Social challenges Dealing with unresponsiveness Explaining the rationale Fearing rejection
39. The contributor’s perspective: Key challenges they face Implications for researchers Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Implications for researchers Dealing with unresponsiveness
 Explaining the rationale
 Fearing rejection
 Social challenges Dealing with unresponsiveness Explaining the rationale Fearing rejection “The owner of the repo doesn’t ever respond to the PR and leaves it hanging open forever”
40. Sometimes it’s hard to explain the need for some changes. Some teams will immediately reject them without analyzing them properly The contributor’s perspective: Key challenges they face Implications for researchers Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Implications for researchers Dealing with unresponsiveness
 Explaining the rationale
 Fearing rejection
 Social challenges Dealing with unresponsiveness Explaining the rationale Fearing rejection
41. The contributor’s perspective: Key challenges they face Implications for researchers Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Implications for researchers Dealing with unresponsiveness
 Explaining the rationale
 Fearing rejection
 Social challenges Dealing with unresponsiveness Explaining the rationale Fearing rejection “Fear of looking stupid.” “Fear of rude response.”
42. The contributor’s perspective: Main Themes
43. The contributor’s perspective: Main Themes awareness
44. The contributor’s perspective: Main Themes awareness transparency
45. The contributor’s perspective: Main Themes responsiveness awareness transparency
46. awareness The contributor’s perspective: Main Themes transparency responsiveness asynchrony
47. The contributor’s and integrator’s perspective: It takes two to tango
48. quality The contributor’s and integrator’s perspective: It takes two to tango
49. quality The contributor’s and integrator’s perspective: It takes two to tango lack of process
50. quality The contributor’s and integrator’s perspective: It takes two to tango lack of process workload and responsiveness
51. quality The contributor’s and integrator’s perspective: It takes two to tango lack of process workload and responsiveness communication
52. Actionable points
53. Actionable points Implications for researchers Prioritize work automatically
 Estimate time for merging
 Untangle code changes
 Analyze impact of changes
 Improve awareness & communication Implications for researchers
54. Actionable points Implications for researchers Prioritize work automatically
 Estimate time for merging
 Untangle code changes
 Analyze impact of changes
 Improve awareness & communication Implications for researchers Recommendations for contributors Minimize friction
 Maximize awareness
 Build a solid profile Recommendations for contributors
55. Pull-Based Development: The Contributor’s Perspective
56. Pull-Based Development: The Contributor’s Perspective
57. Pull-Based Development: The Contributor’s Perspective
58. Pull-Based Development: The Contributor’s Perspective
59. Pull-Based Development: The Contributor’s Perspective
60. Pull-Based Development: The Contributor’s Perspective @gousiosg @sback_