Share
Embed
MIP Talk at SANER2025: Who should review my code? A File Location-Based Code-Reviewer Recommendation Approach for Modern Code Review
This talk is an invited talk of our paper titled "Who should review my code? A File Location-Based Code-Reviewer Recommendation Approach for Modern Code Review" which received the most influential paper award at the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) 2025. The talk includes an overview of the work and a discussion about the evolution of the subsequent works in this research area.
Patanamon (Pick) Thongtanunam
March 05, 2025
Transcript
63 slides
-
Who Should Review My Code? A File Location-Based Code-Reviewer Recommendation
Approach for Modern Code Review Chakkrit (Kla) Tantithamthavorn Raula G. Kula Norihiro Yoshida Hajimu Iida Ken-ichi Matsumoto — Look Back and Look Ahead — SANER2025 MIP Award Patanamon (Pick) Thongtanunam [email protected] http://patanamon.com Senior Lecturer Senior Lecturer Professor Professor Professor Professor -
Who Should Review My Code? A File Location-Based Code-Reviewer Recommendation
Approach for Modern Code Review Patanamon (Pick) Thongtanunam Chakkrit Tantithamthavorn Raula G. Kula Norihiro Yoshida Hajimu Iida Ken-ichi Matsumoto 22nd IEEE International Conference on Software Analysis, Evolution, and Reengineering Montréal, Quebec, Canada March 3 - 6, 2015 PhD candidate PhD candidate Postdoc Fellow Associate Professor Professor Professor -
Gerrit Code Review Rietveld Modern Code Review, a tool-based practice
was increasingly adopted in Open source and industry software projects 4 -
Reviewers Developer Can you review my code? Code change Gerrit
Code Review Invite Reviewers 5 Developers may struggle to fi nd reviewers due to lightweight variants of the code review process -
Reviewers Developer Code change Gerrit Code Review Invite Reviewers 5
Who should review my code? Developers may struggle to fi nd reviewers due to lightweight variants of the code review process -
Reviewers Developer Code change Gerrit Code Review Invite Reviewers 5
Who should review my code? Developers may struggle to fi nd reviewers due to lightweight variants of the code review process “Is this a real problem?” -
4% - 30% of reviews discussed about reviewer assignment Studied
Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% 6 -
4% - 30% of reviews discussed about reviewer assignment Studied
Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% Reviews with code-reviewer assignment problem signi fi cantly take 12 days longer to approve a code change. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Reviewing Time (Days) Reviews with code−reviewer assignment problem Review viewing time (Days) 10 20 30 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignm viewing time (Days) 10 20 30 Review w/ Assignment Problem Review w/o Assignment Problem 18 Days Longer 13 Days Longer 9 Days Longer 6 Days Longer 6 -
RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java
Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? Review History 8 -
RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java
Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Review History “Files located in similar fi le paths (i.e. similar/related components) would be reviewed by similar experienced reviewers” 8 -
RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java
Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Compare Compare(FilePath_R2,FilePath_R3) = 0.3 Review History 8 -
RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java
Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Compare(FilePath_R2,FilePath_R3) = 0.3 Compare(FilePath_R1,FilePath_R3) = 0.6 Review History 8 Compare -
RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java
Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Compare(FilePath_R2,FilePath_R3) = 0.3 Compare(FilePath_R1,FilePath_R3) = 0.6 Review History 9 -
RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java
Code-Reviewers Reviewed by Kla Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pick R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Compare(FilePath_R1,FilePath_R3) = 0.6 Review History 9 “Pick should review R3” -
Empirical Evaluation: Accuracy Top-k Accuracy measures how many reviews does
an approach can correctly recommend with a Top-k recommendation. Top-10 Accuracy 0% 22.5% 45% 67.5% 90% 10% 28% 41% 29% 74% 69% 87% 86% RevFinder ReviewBot RevFinder can correctly recommended 69%-87% of reviews with a top-10 recommendation. RevFinder is 4 times more accurate than ReviewBot. 10 Top-5 Recommendation Actual Reviewer Correct! -
Empirical Evaluation: Ranking Performance Mean Reciprocal Rank measures an average
rank of correct reviewer that an approach can recommend. RevFinder recommended the correct code-reviewers with a median rank of 2 - 8. The code-reviewers ranking of RevFinder is 3 times better than that of ReviewBot. 11 Recommendation Lists vs Actual Reviewer MRR = 1/3 MRR RevFinder 0.6 0.55 0.31 0.40 ReviewBot 0.25 0.30 0.22 0.07 Rank of Correct Reviewers Android OpenStack Qt LibreOffice • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 1 5 15 50 100 200 RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot Approach Rank RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot MRR = 1/1 -
Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem
Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Empirical Evidence of Developers’ Challenge Longer Review Time Prevalence of the challenges We uncover the reviewer assignment challenge in modern code review Empirical Evaluation 12 Contributions -
Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem
Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Empirical Evidence of Developers’ Challenge Longer Review Time Prevalence of the challenges We uncover the reviewer assignment challenge in modern code review Empirical Evaluation 12 Contributions Reviewer Recommendation Heuristics We can use a similarity of previously reviewed fi le paths to recommend reviewers for new code changes. Accuracy Ranking Performance -
Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem
Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewing time (Days) Android OpenStack Qt LibreOffice 0 10 20 30 Empirical Evidence of Developers’ Challenge Longer Review Time Prevalence of the challenges We uncover the reviewer assignment challenge in modern code review Empirical Evaluation 12 Contributions Reviewer Recommendation Heuristics We can use a similarity of previously reviewed fi le paths to recommend reviewers for new code changes. Accuracy Ranking Performance Replication package: Benchmark data and our approach -
14 Reviewer recommendation leveraged additional information File Path + Text
information Time, Recency -
14 Reviewer recommendation leveraged additional information File Path + Text
information Time, Recency External software libraries -
17 Code Review Automation: Assisting code reviewing tasks Save time
and effort in the code review process -
17 Code Review Automation: Assisting code reviewing tasks Save time
and effort in the code review process Reviewing Localisation -
17 Code Review Automation: Assisting code reviewing tasks Save time
and effort in the code review process Reviewing Localisation -
17 Code Review Automation: Assisting code reviewing tasks Save time
and effort in the code review process Reviewing Localisation -
17 Code Review Automation: Assisting code reviewing tasks Save time
and effort in the code review process Reviewing Localisation -
18 Code Review Automation: Assisting code reviewing tasks Save time
and effort in the code review process Code changes revision/ re fi nement Reviewing Localisation -
18 Code Review Automation: Assisting code reviewing tasks Save time
and effort in the code review process Code changes revision/ re fi nement Reviewing Localisation -
18 Code Review Automation: Assisting code reviewing tasks Save time
and effort in the code review process Code changes revision/ re fi nement Reviewing Localisation -
18 Code Review Automation: Assisting code reviewing tasks Save time
and effort in the code review process Code changes revision/ re fi nement Reviewing Localisation -
19 Code Review Automation: Assisting code reviewing tasks Save time
and effort in the code review process Code changes revision/ re fi nement Review comment generation Reviewing Localisation -
19 Code Review Automation: Assisting code reviewing tasks Save time
and effort in the code review process Code changes revision/ re fi nement Review comment generation Reviewing Localisation -
19 Code Review Automation: Assisting code reviewing tasks Save time
and effort in the code review process Code changes revision/ re fi nement Review comment generation Reviewing Localisation -
19 Code Review Automation: Assisting code reviewing tasks Save time
and effort in the code review process Code changes revision/ re fi nement Review comment generation Reviewing Localisation -
20 Look Back: Code review practices changed ➔ Challenges discovered
In-person formal inspection ~1976 Online & Tool- based review ~2000s -
20 Look Back: Code review practices changed ➔ Challenges discovered
In-person formal inspection ~1976 Online & Tool- based review ~2000s -
21 Look Ahead: Code review practices will change again Online
& Tool- based review ~2000s ~2020s AI-assisted code review -
21 Look Ahead: Code review practices will change again Online
& Tool- based review ~2000s ~2020s AI-assisted code review Generation Quality Developer & AI Interaction Trustworthy Security Many more.. -
21 Look Ahead: Code review practices will change again Online
& Tool- based review ~2000s ~2020s AI-assisted code review Generation Quality Developer & AI Interaction Trustworthy Security Many more.. -
21 Look Ahead: Code review practices will change again Online
& Tool- based review ~2000s ~2020s AI-assisted code review Generation Quality Developer & AI Interaction Trustworthy Security Many more.. -
21 Look Ahead: Code review practices will change again Online
& Tool- based review ~2000s ~2020s AI-assisted code review Generation Quality Developer & AI Interaction Trustworthy Security Many more.. -
23 Patanamon (Pick) Thongtanunam [email protected] http://patanamon.com RevFinder served as an
important step of my research paving towards automated code reviews and other software engineering tasks RevFinder Software Quality Automated Code Review Agile Planning Code Ownership/ Expertise Continuous Integration Slides available on