MIP Talk at SANER2025: Who should review my code? A File Location-Based Code-Reviewer Recommendation Approach for Modern Code Review

This talk is an invited talk of our paper titled "Who should review my code? A File Location-Based Code-Reviewer Recommendation Approach for Modern Code Review" which received the most influential paper award at the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) 2025. The talk includes an overview of the work and a discussion about the evolution of the subsequent works in this research area.

Transcript

1. Who Should Review My Code? A File Location-Based Code-Reviewer Recommendation Approach for Modern Code Review Chakkrit (Kla) Tantithamthavorn Raula G. Kula Norihiro Yoshida Hajimu Iida Ken-ichi Matsumoto — Look Back and Look Ahead — SANER2025 MIP Award Patanamon (Pick) Thongtanunam patanamon.t@unimelb.edu.au http://patanamon.com Senior Lecturer Senior Lecturer Professor Professor Professor Professor
2. Who Should Review My Code? A File Location-Based Code-Reviewer Recommendation Approach for Modern Code Review Patanamon (Pick) Thongtanunam Chakkrit Tantithamthavorn Raula G. Kula Norihiro Yoshida Hajimu Iida Ken-ichi Matsumoto 22nd IEEE International Conference on Software Analysis,
 Evolution, and Reengineering Montréal, Quebec, Canada March 3 - 6, 2015 PhD candidate PhD candidate Postdoc Fellow Associate Professor Professor Professor
4. Gerrit Code Review Rietveld Modern Code Review, a tool-based practice was increasingly adopted in Open source and industry software projects 4
5. Reviewers Developer Can you review my code? Code change Gerrit Code Review Invite Reviewers 5 Developers may struggle to fi nd reviewers due to lightweight variants of the code review process
6. Reviewers Developer Code change Gerrit Code Review Invite Reviewers 5 Who should review my code? Developers may struggle to fi nd reviewers due to lightweight variants of the code review process
7. Reviewers Developer Code change Gerrit Code Review Invite Reviewers 5 Who should review my code? Developers may struggle to fi nd reviewers due to lightweight variants of the code review process “Is this a real problem?”
8. 4% - 30% of reviews discussed about reviewer assignment Studied Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% 6
9. 4% - 30% of reviews discussed about reviewer assignment Studied Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% Reviews with code-reviewer assignment problem signi fi cantly take 12 days longer to approve a code change. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Reviewing Time (Days) Reviews with code−reviewer assignment problem Review viewing time (Days) 10 20 30 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignm viewing time (Days) 10 20 30 Review w/ Assignment Problem Review w/o Assignment Problem 18 Days Longer 13 Days Longer 9 Days Longer 6 Days Longer 6
10. RevFinder: A File Location-based Code-Reviewer Recommendation Approach 7
11. RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? Review History 8
12. RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Review History “Files located in similar fi le paths (i.e. similar/related components) would be reviewed by similar experienced reviewers” 8
13. RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Compare Compare(FilePath_R2,FilePath_R3) = 0.3 Review History 8
14. RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Compare(FilePath_R2,FilePath_R3) = 0.3 Compare(FilePath_R1,FilePath_R3) = 0.6 Review History 8 Compare
15. RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Compare(FilePath_R2,FilePath_R3) = 0.3 Compare(FilePath_R1,FilePath_R3) = 0.6 Review History 9
16. RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Compare(FilePath_R1,FilePath_R3) = 0.6 Review History 9 “Pick should review R3”
17. Empirical Evaluation: Accuracy Top-k Accuracy measures how many reviews does an approach can correctly recommend with a Top-k recommendation. Top-10 Accuracy 0% 22.5% 45% 67.5% 90% 10% 28% 41% 29% 74% 69% 87% 86% RevFinder ReviewBot RevFinder can correctly recommended 69%-87% of reviews with a top-10 recommendation. RevFinder is 4 times more accurate than ReviewBot. 10 Top-5 Recommendation Actual Reviewer Correct!
18. Empirical Evaluation: Ranking Performance Mean Reciprocal Rank measures an average rank of correct reviewer that an approach can recommend. RevFinder recommended the correct code-reviewers with a median rank of 2 - 8. The code-reviewers ranking of RevFinder is 3 times better than that of ReviewBot. 11 Recommendation Lists vs Actual Reviewer MRR = 1/3 MRR RevFinder 0.6 0.55 0.31 0.40 ReviewBot 0.25 0.30 0.22 0.07 Rank of 
 Correct Reviewers Android OpenStack Qt LibreOffice • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 1 5 15 50 100 200 RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot Approach Rank RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot MRR = 1/1
19. Empirical Evaluation 12 Contributions
20. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Empirical Evidence of Developers’ Challenge Longer Review Time Prevalence of the challenges We uncover the reviewer assignment challenge in modern code review Empirical Evaluation 12 Contributions
21. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Empirical Evidence of Developers’ Challenge Longer Review Time Prevalence of the challenges We uncover the reviewer assignment challenge in modern code review Empirical Evaluation 12 Contributions Reviewer Recommendation Heuristics We can use a similarity of previously reviewed fi le paths to recommend reviewers for new code changes. Accuracy Ranking Performance
22. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Empirical Evidence of Developers’ Challenge Longer Review Time Prevalence of the challenges We uncover the reviewer assignment challenge in modern code review Empirical Evaluation 12 Contributions Reviewer Recommendation Heuristics We can use a similarity of previously reviewed fi le paths to recommend reviewers for new code changes. Accuracy Ranking Performance Replication package: Benchmark data and our approach
23. 13 Research Directions & Relevant Subsequent Works
24. 14 Reviewer recommendation leveraged additional information
25. 14 Reviewer recommendation leveraged additional information File Path + Text information
26. 14 Reviewer recommendation leveraged additional information File Path + Text information Time, Recency
27. 14 Reviewer recommendation leveraged additional information File Path + Text information Time, Recency External software libraries
28. 15 Reviewer recommendation leveraged machine learning
29. 15 Reviewer recommendation leveraged machine learning Deep Neural Network
30. 15 Reviewer recommendation leveraged machine learning Collaborative Filtering Deep Neural Network
31. 15 Reviewer recommendation leveraged machine learning (Hyper-)Graph Modelling Collaborative Filtering Deep Neural Network
32. 16 Reviewer recommendation becomes more human-centric
33. 16 Reviewer recommendation becomes more human-centric
34. 16 Reviewer recommendation becomes more human-centric
35. 16 Reviewer recommendation becomes more human-centric
36. 17 Code Review Automation: Assisting code reviewing tasks Save time and effort in the code review process
37. 17 Code Review Automation: Assisting code reviewing tasks Save time and effort in the code review process Reviewing Localisation
38. 17 Code Review Automation: Assisting code reviewing tasks Save time and effort in the code review process Reviewing Localisation
39. 17 Code Review Automation: Assisting code reviewing tasks Save time and effort in the code review process Reviewing Localisation
40. 17 Code Review Automation: Assisting code reviewing tasks Save time and effort in the code review process Reviewing Localisation
41. 18 Code Review Automation: Assisting code reviewing tasks Save time and effort in the code review process Code changes revision/ re fi nement Reviewing Localisation
42. 18 Code Review Automation: Assisting code reviewing tasks Save time and effort in the code review process Code changes revision/ re fi nement Reviewing Localisation
43. 18 Code Review Automation: Assisting code reviewing tasks Save time and effort in the code review process Code changes revision/ re fi nement Reviewing Localisation
44. 18 Code Review Automation: Assisting code reviewing tasks Save time and effort in the code review process Code changes revision/ re fi nement Reviewing Localisation
45. 19 Code Review Automation: Assisting code reviewing tasks Save time and effort in the code review process Code changes revision/ re fi nement Review comment generation Reviewing Localisation
46. 19 Code Review Automation: Assisting code reviewing tasks Save time and effort in the code review process Code changes revision/ re fi nement Review comment generation Reviewing Localisation
47. 19 Code Review Automation: Assisting code reviewing tasks Save time and effort in the code review process Code changes revision/ re fi nement Review comment generation Reviewing Localisation
48. 19 Code Review Automation: Assisting code reviewing tasks Save time and effort in the code review process Code changes revision/ re fi nement Review comment generation Reviewing Localisation
50. 20 Look Back: Code review practices changed ➔ Challenges discovered
51. 20 Look Back: Code review practices changed ➔ Challenges discovered In-person formal inspection ~1976 Online & Tool- based review ~2000s
52. 20 Look Back: Code review practices changed ➔ Challenges discovered In-person formal inspection ~1976 Online & Tool- based review ~2000s
54. 21 Look Ahead: Code review practices will change again Online & Tool- based review ~2000s ~2020s AI-assisted code review
55. 21 Look Ahead: Code review practices will change again Online & Tool- based review ~2000s ~2020s AI-assisted code review Generation Quality Developer & AI Interaction Trustworthy Security Many more..
56. 21 Look Ahead: Code review practices will change again Online & Tool- based review ~2000s ~2020s AI-assisted code review Generation Quality Developer & AI Interaction Trustworthy Security Many more..
57. 21 Look Ahead: Code review practices will change again Online & Tool- based review ~2000s ~2020s AI-assisted code review Generation Quality Developer & AI Interaction Trustworthy Security Many more..
58. 21 Look Ahead: Code review practices will change again Online & Tool- based review ~2000s ~2020s AI-assisted code review Generation Quality Developer & AI Interaction Trustworthy Security Many more..
60. 22 RevFinder Most In fl uential Paper??
62. 23 RevFinder
63. 23 Patanamon (Pick) Thongtanunam patanamon.t@unimelb.edu.au http://patanamon.com RevFinder served as an important step of my research paving towards automated code reviews and other software engineering tasks RevFinder Software Quality Automated Code Review Agile Planning Code Ownership/ Expertise Continuous Integration Slides available on