Pro Yearly is on sale from $80 to $50! »

Work Practices and Challenges in Pull-Based Development: The Contributor’s Perspective

Work Practices and Challenges in Pull-Based Development: The Contributor’s Perspective

Presentation given by Georgios Gousios and Alberto Bacchelli at the 38th International Conference on Software Engineering.

43df3993acc9af4e9f619e59cd849aee?s=128

Georgios Gousios

May 18, 2016
Tweet

Transcript

  1. Work Practices and Challenges in Pull-Based Development: The Contributor’s Perspective

    Georgios Gousios @gousiosg Margaret-Anne Storey @margeretstorey Alberto Bacchelli @sback_ pull-based contributions
  2. Core developers Co-developers Community Users Open-source communities: The onion model

  3. GitHub’s pull-based development model core developer contributor (integrator)

  4. projects with > 1 integrators 55% shared repository 45% use

    pull requests GitHub’s pull-based development model: A success story 0k 100k 200k 300k 400k 500k 600k 700k 2010 2011 2012 2013 2014 2015 # of pull requests 2016 135k repos/month 600k pull-requests/month
  5. The integrator’s perspective

  6. The integrator’s perspective: Plenty of research

  7. The integrator’s perspective: Plenty of research

  8. None
  9. Decide change Code/Check quality Submit Discuss intentions Feedback Code review

    Submit Proposed fixes Discuss fixes Code review Accept Code/Check quality contributors integrators
  10. Decide change Code/Check quality Submit Discuss intentions Feedback Code review

    Submit Proposed fixes Discuss fixes Code review Accept Code/Check quality contributors integrators
  11. Decide change Code/Check quality Submit Discuss intentions Feedback Code review

    Submit Proposed fixes Discuss fixes Code review Accept Code/Check quality The contributor’s perspective contributors integrators
  12. Research Questions

  13. RQ1: How do contributors prepare contributions? Research Questions

  14. RQ1: How do contributors prepare contributions? 1. What do they

    do before & after coding? Research Questions
  15. RQ1: How do contributors prepare contributions? 1. What do they

    do before & after coding? 2. How do they assess their contribution’s quality? Research Questions
  16. RQ1: How do contributors prepare contributions? 1. What do they

    do before & after coding? 2. How do they assess their contribution’s quality? 3. How do they communicate the intended change? Research Questions
  17. RQ1: How do contributors prepare contributions? 1. What do they

    do before & after coding? 2. How do they assess their contribution’s quality? 3. How do they communicate the intended change? RQ2: What are the key challenges they face? Research Questions
  18. GHTorrent Final Survey Performance Reports The contributor’s perspective: Research method

  19. GHTorrent Final Survey Performance Reports 4,167 contributors (760 responses) The

    contributor’s perspective: Research method
  20. GHTorrent Final Survey Analysis / filtering (15 closed questions) Performance

    Reports researchers Card sorting (7 open questions) 4,167 contributors (760 responses) The contributor’s perspective: Research method
  21. GHTorrent Final Survey Analysis / filtering (15 closed questions) Performance

    Reports researchers Card sorting (7 open questions) 4,167 contributors (760 responses) Results The contributor’s perspective: Research method
  22. RQ1: How do contributors prepare contributions? 1. What do they

    do before & after coding? 2. How do they assess their contribution’s quality? 3. How do they communicate the intended change? RQ2: What are the key challenges they face? Research Questions
  23. % of responses The contributor’s perspective: Before coding a contribution

    Look up open issues Check for similar PRs Check rel. discussions Check guidelines Communicate changes Check other forks 0 50 100 150 20 Never or Occasionaly Often or Always 100 0 100
  24. Run tests Check against guidelines Check rel. discussions Check for

    similar PRs Check other branches 0 50 100 150 20 Never or Occasionaly Often or Always % of responses The contributor’s perspective: After coding a contribution 100 0 100
  25. RQ1: How do contributors prepare contributions? 1. What do they

    do before & after coding? 2. How do they assess their contribution’s quality? 3. How do they communicate the intended change? RQ2: What are the key challenges they face? Research Questions
  26. Testing Self review Manual testing Compliance Static Analysis CI Self

    containment Documentation Building 0 15 30 45 60 % of responses The contributor’s perspective: Assessing their contribution’s quality What is evaluated How it is evaluated
  27. Testing Self review Manual testing Compliance Static Analysis CI Self

    containment Documentation Building 0 15 30 45 60 % of responses The contributor’s perspective: Assessing their contribution’s quality What is evaluated How it is evaluated
  28. Testing Self review Manual testing Compliance Static Analysis CI Self

    containment Documentation Building 0 15 30 45 60 % of responses The contributor’s perspective: Assessing their contribution’s quality What is evaluated How it is evaluated
  29. seeking information related to change The contributor’s perspective: What happens

    before submitting not communicating intent verifying quality using testing trying to adhere to project style
  30. RQ1: How do contributors prepare contributions? 1. What do they

    do before & after coding? 2. How do they assess their contribution’s quality? 3. How do they communicate the intended change? RQ2: What are the key challenges they face? Research Questions
  31. Issue tracker Pull request Email IRC F2F Sync communication Twitter

    Other Do not communicate 0 20 40 60 80 % of responses The contributor’s perspective: Communicating the intended change
  32. RQ1: How do contributors prepare contributions? 1. What do they

    do before & after coding? 2. How do they assess their contribution’s quality? 3. How do they communicate the intended change? RQ2: What are the key challenges they face? Research Questions
  33. The contributor’s perspective: Key challenges they face Implications for researchers

    Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges
  34. The contributor’s perspective: Key challenges they face Implications for researchers

    Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Understanding the code Evaluate the impact Ensure project compliance Dealing with git and conflicts “Missing knowledge about inner workings of a project [...] sometimes caused by missing documentation”
  35. “Ensuring that my PR doesn’t have unintended side effects due

    to not being intimately familiar with the entire code base.” The contributor’s perspective: Key challenges they face Implications for researchers Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Understanding the code Evaluate the impact Ensure project compliance Dealing with git and conflicts
  36. “Unsure of code style, pull requests format, commit message formats,

    etc.” The contributor’s perspective: Key challenges they face Implications for researchers Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Understanding the code Evaluate the impact Ensure project compliance Dealing with git and conflicts
  37. “Usage of git is not intuitive. Especially for me as

    [one] who does not contribute regularly, it is every time a challenge to [use it]” The contributor’s perspective: Key challenges they face Implications for researchers Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Understanding the code Evaluate the impact Ensure project compliance Dealing with git and conflicts
  38. The contributor’s perspective: Key challenges they face Implications for researchers

    Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Implications for researchers Dealing with unresponsiveness
 Explaining the rationale
 Fearing rejection
 Social challenges Dealing with unresponsiveness Explaining the rationale Fearing rejection
  39. The contributor’s perspective: Key challenges they face Implications for researchers

    Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Implications for researchers Dealing with unresponsiveness
 Explaining the rationale
 Fearing rejection
 Social challenges Dealing with unresponsiveness Explaining the rationale Fearing rejection “The owner of the repo doesn’t ever respond to the PR and leaves it hanging open forever”
  40. Sometimes it’s hard to explain the need for some changes.

    Some teams will immediately reject them without analyzing them properly The contributor’s perspective: Key challenges they face Implications for researchers Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Implications for researchers Dealing with unresponsiveness
 Explaining the rationale
 Fearing rejection
 Social challenges Dealing with unresponsiveness Explaining the rationale Fearing rejection
  41. The contributor’s perspective: Key challenges they face Implications for researchers

    Understanding the code
 Evaluate the impact
 Ensure project compliance
 Dealing with git and conflicts Technical challenges Implications for researchers Dealing with unresponsiveness
 Explaining the rationale
 Fearing rejection
 Social challenges Dealing with unresponsiveness Explaining the rationale Fearing rejection “Fear of looking stupid.” “Fear of rude response.”
  42. The contributor’s perspective: Main Themes

  43. The contributor’s perspective: Main Themes awareness

  44. The contributor’s perspective: Main Themes awareness transparency

  45. The contributor’s perspective: Main Themes responsiveness awareness transparency

  46. awareness The contributor’s perspective: Main Themes transparency responsiveness asynchrony

  47. The contributor’s and integrator’s perspective: It takes two to tango

  48. quality The contributor’s and integrator’s perspective: It takes two to

    tango
  49. quality The contributor’s and integrator’s perspective: It takes two to

    tango lack of process
  50. quality The contributor’s and integrator’s perspective: It takes two to

    tango lack of process workload and responsiveness
  51. quality The contributor’s and integrator’s perspective: It takes two to

    tango lack of process workload and responsiveness communication
  52. Actionable points

  53. Actionable points Implications for researchers Prioritize work automatically
 Estimate time

    for merging
 Untangle code changes
 Analyze impact of changes
 Improve awareness & communication Implications for researchers
  54. Actionable points Implications for researchers Prioritize work automatically
 Estimate time

    for merging
 Untangle code changes
 Analyze impact of changes
 Improve awareness & communication Implications for researchers Recommendations for contributors Minimize friction
 Maximize awareness
 Build a solid profile Recommendations for contributors
  55. Pull-Based Development: The Contributor’s Perspective

  56. Pull-Based Development: The Contributor’s Perspective

  57. Pull-Based Development: The Contributor’s Perspective

  58. Pull-Based Development: The Contributor’s Perspective

  59. Pull-Based Development: The Contributor’s Perspective

  60. Pull-Based Development: The Contributor’s Perspective @gousiosg @sback_