Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MIP Talk at SANER2025: Who should review my cod...

MIP Talk at SANER2025: Who should review my code? A File Location-Based Code-Reviewer Recommendation Approach for Modern Code Review

This talk is an invited talk of our paper titled "Who should review my code? A File Location-Based Code-Reviewer Recommendation Approach for Modern Code Review" which received the most influential paper award at the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) 2025. The talk includes an overview of the work and a discussion about the evolution of the subsequent works in this research area.

More Decks by Patanamon (Pick) Thongtanunam

Other Decks in Research

Transcript

  1. Who Should Review My Code? A File Location-Based Code-Reviewer Recommendation

    Approach for Modern Code Review Chakkrit (Kla) Tantithamthavorn Raula G. Kula Norihiro Yoshida Hajimu Iida Ken-ichi Matsumoto — Look Back and Look Ahead — SANER2025 MIP Award Patanamon (Pick) Thongtanunam [email protected] http://patanamon.com Senior Lecturer Senior Lecturer Professor Professor Professor Professor
  2. Who Should Review My Code? A File Location-Based Code-Reviewer Recommendation

    Approach for Modern Code Review Patanamon (Pick) Thongtanunam Chakkrit Tantithamthavorn Raula G. Kula Norihiro Yoshida Hajimu Iida Ken-ichi Matsumoto 22nd IEEE International Conference on Software Analysis,
 Evolution, and Reengineering Montréal, Quebec, Canada March 3 - 6, 2015 PhD candidate PhD candidate Postdoc Fellow Associate Professor Professor Professor
  3. Gerrit Code Review Rietveld Modern Code Review, a tool-based practice

    was increasingly adopted in Open source and industry software projects 4
  4. Reviewers Developer Can you review my code? Code change Gerrit

    Code Review Invite Reviewers 5 Developers may struggle to fi nd reviewers due to lightweight variants of the code review process
  5. Reviewers Developer Code change Gerrit Code Review Invite Reviewers 5

    Who should review my code? Developers may struggle to fi nd reviewers due to lightweight variants of the code review process
  6. Reviewers Developer Code change Gerrit Code Review Invite Reviewers 5

    Who should review my code? Developers may struggle to fi nd reviewers due to lightweight variants of the code review process “Is this a real problem?”
  7. 4% - 30% of reviews discussed about reviewer assignment Studied

    Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% 6
  8. 4% - 30% of reviews discussed about reviewer assignment Studied

    Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% Reviews with code-reviewer assignment problem signi fi cantly take 12 days longer to approve a code change. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Reviewing Time (Days) Reviews with code−reviewer assignment problem Review viewing time (Days) 10 20 30 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignm viewing time (Days) 10 20 30 Review w/ Assignment Problem Review w/o Assignment Problem 18 Days Longer 13 Days Longer 9 Days Longer 6 Days Longer 6
  9. RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java

    Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? Review History 8
  10. RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java

    Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Review History “Files located in similar fi le paths (i.e. similar/related components) would be reviewed by similar experienced reviewers” 8
  11. RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java

    Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Compare Compare(FilePath_R2,FilePath_R3) = 0.3 Review History 8
  12. RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java

    Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Compare(FilePath_R2,FilePath_R3) = 0.3 Compare(FilePath_R1,FilePath_R3) = 0.6 Review History 8 Compare
  13. RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java

    Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Compare(FilePath_R2,FilePath_R3) = 0.3 Compare(FilePath_R1,FilePath_R3) = 0.6 Review History 9
  14. RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java

    Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Compare(FilePath_R1,FilePath_R3) = 0.6 Review History 9 “Pick should review R3”
  15. Empirical Evaluation: Accuracy Top-k Accuracy measures how many reviews does

    an approach can correctly recommend with a Top-k recommendation. Top-10 Accuracy 0% 22.5% 45% 67.5% 90% 10% 28% 41% 29% 74% 69% 87% 86% RevFinder ReviewBot RevFinder can correctly recommended 69%-87% of reviews with a top-10 recommendation. RevFinder is 4 times more accurate than ReviewBot. 10 Top-5 Recommendation Actual Reviewer Correct!
  16. Empirical Evaluation: Ranking Performance Mean Reciprocal Rank measures an average

    rank of correct reviewer that an approach can recommend. RevFinder recommended the correct code-reviewers with a median rank of 2 - 8. The code-reviewers ranking of RevFinder is 3 times better than that of ReviewBot. 11 Recommendation Lists vs Actual Reviewer MRR = 1/3 MRR RevFinder 0.6 0.55 0.31 0.40 ReviewBot 0.25 0.30 0.22 0.07 Rank of 
 Correct Reviewers Android OpenStack Qt LibreOffice • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 1 5 15 50 100 200 RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot Approach Rank RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot MRR = 1/1
  17. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem

    Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Empirical Evidence of Developers’ Challenge Longer Review Time Prevalence of the challenges We uncover the reviewer assignment challenge in modern code review Empirical Evaluation 12 Contributions
  18. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem

    Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Empirical Evidence of Developers’ Challenge Longer Review Time Prevalence of the challenges We uncover the reviewer assignment challenge in modern code review Empirical Evaluation 12 Contributions Reviewer Recommendation Heuristics We can use a similarity of previously reviewed fi le paths to recommend reviewers for new code changes. Accuracy Ranking Performance
  19. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem

    Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Empirical Evidence of Developers’ Challenge Longer Review Time Prevalence of the challenges We uncover the reviewer assignment challenge in modern code review Empirical Evaluation 12 Contributions Reviewer Recommendation Heuristics We can use a similarity of previously reviewed fi le paths to recommend reviewers for new code changes. Accuracy Ranking Performance Replication package: Benchmark data and our approach
  20. 14 Reviewer recommendation leveraged additional information File Path + Text

    information Time, Recency External software libraries
  21. 17 Code Review Automation: Assisting code reviewing tasks Save time

    and effort in the code review process Reviewing Localisation
  22. 17 Code Review Automation: Assisting code reviewing tasks Save time

    and effort in the code review process Reviewing Localisation
  23. 17 Code Review Automation: Assisting code reviewing tasks Save time

    and effort in the code review process Reviewing Localisation
  24. 17 Code Review Automation: Assisting code reviewing tasks Save time

    and effort in the code review process Reviewing Localisation
  25. 18 Code Review Automation: Assisting code reviewing tasks Save time

    and effort in the code review process Code changes revision/ re fi nement Reviewing Localisation
  26. 18 Code Review Automation: Assisting code reviewing tasks Save time

    and effort in the code review process Code changes revision/ re fi nement Reviewing Localisation
  27. 18 Code Review Automation: Assisting code reviewing tasks Save time

    and effort in the code review process Code changes revision/ re fi nement Reviewing Localisation
  28. 18 Code Review Automation: Assisting code reviewing tasks Save time

    and effort in the code review process Code changes revision/ re fi nement Reviewing Localisation
  29. 19 Code Review Automation: Assisting code reviewing tasks Save time

    and effort in the code review process Code changes revision/ re fi nement Review comment generation Reviewing Localisation
  30. 19 Code Review Automation: Assisting code reviewing tasks Save time

    and effort in the code review process Code changes revision/ re fi nement Review comment generation Reviewing Localisation
  31. 19 Code Review Automation: Assisting code reviewing tasks Save time

    and effort in the code review process Code changes revision/ re fi nement Review comment generation Reviewing Localisation
  32. 19 Code Review Automation: Assisting code reviewing tasks Save time

    and effort in the code review process Code changes revision/ re fi nement Review comment generation Reviewing Localisation
  33. 20

  34. 20 Look Back: Code review practices changed ➔ Challenges discovered

    In-person formal inspection ~1976 Online & Tool- based review ~2000s
  35. 20 Look Back: Code review practices changed ➔ Challenges discovered

    In-person formal inspection ~1976 Online & Tool- based review ~2000s
  36. 21

  37. 21 Look Ahead: Code review practices will change again Online

    & Tool- based review ~2000s ~2020s AI-assisted code review
  38. 21 Look Ahead: Code review practices will change again Online

    & Tool- based review ~2000s ~2020s AI-assisted code review Generation Quality Developer & AI Interaction Trustworthy Security Many more..
  39. 21 Look Ahead: Code review practices will change again Online

    & Tool- based review ~2000s ~2020s AI-assisted code review Generation Quality Developer & AI Interaction Trustworthy Security Many more..
  40. 21 Look Ahead: Code review practices will change again Online

    & Tool- based review ~2000s ~2020s AI-assisted code review Generation Quality Developer & AI Interaction Trustworthy Security Many more..
  41. 21 Look Ahead: Code review practices will change again Online

    & Tool- based review ~2000s ~2020s AI-assisted code review Generation Quality Developer & AI Interaction Trustworthy Security Many more..
  42. 22

  43. 23

  44. 23 Patanamon (Pick) Thongtanunam [email protected] http://patanamon.com RevFinder served as an

    important step of my research paving towards automated code reviews and other software engineering tasks RevFinder Software Quality Automated Code Review Agile Planning Code Ownership/ Expertise Continuous Integration Slides available on