Recommending Code Reviews Leveraging Code Changes with Structured Information Retrieval

Ohiduzzaman Shuvo, Parvez Mahbub, and M. Masudur Rahman. Recommending Code Reviews Leveraging Code Changes with Structured Information Retrieval. In Proceeding of The 39th IEEE International Conference on Software Maintenance and Evolution (ICSME 2023), pp. 12, Bogota, Columbia, October 2023

Transcript

1. Recommending Code Reviews Leveraging Code Changes with Structured Information Retrieval 1 Ohiduzzaman Shuvo Dalhousie University ohiduzzaman@dal.ca RAISE Lab Intelligent Automation in Software EngineeRing Masud Rahman Dalhousie University masud.rahman@dal.ca Parvez Mahbub Dalhousie University parvezmrobin@dal.ca
2. 2 A Tale of Modern Code Reviews Review Comments Code Author Reviewer Developed Code Modern Code Reviews ~ 6 hours/week ~ 15-64 hours wait Reduce time Reduce effort Recommended Comments Automated tool
3. Do We Need a New Technique? DL-based techniques require specialize computing resources (e.g., 16 × 40GB GPU [18]) and long training time (e.g., 12 days [18]) IR-based approach outperforms DL-based approach Used only Java projects Ignore code structure Applicable to method- level changes 3
4. We Present - RevCom 4 RevCom- a novel structured IR-based technique Can leverage various structured elements from source code Can recommend reviews for all types of changes
5. 5 How does RevCom Work? 0.13 1.05 0.00 0.11 1.26 0.00 0.09 0.00 0.60 Diff ... {} Source file Corpus dataset Query dataset Structured Information retrieval Ranking Top-N nearest changed code based on the lexical similarity score Compute textual similarity between queries and N retrieved changed code code sim library sim path sim Structured Information File path Changed code Library Top-k relevant review comments Code Review Recommendation TF-IDF representation 1.13 0.00 0.60 0.91 0.75 0.00 0.09 1.05 0.00 Input (a) Vectorization (b) (c) (d) File paths Changed code Libraries TF-IDF representation Diffs ... {} Source files reviews
6. Motivating Examples Listing 1: Example of class-level code change Listing 2: Example of library-level code change 6
7. Recommended Comments by RevCom Listing 3: Example code change 7 Technique Review Comments Ground Truth Please import specific things rather than everything. CommentFinder “ ” CodeReviewer Please only import what you need. RevCom I think one should only import from `ansible.module_utils.basic` what's needed (e.g. `AnsibleModule`)
8. 8 Experiments
9. 9 Experimental Design • Dataset: Evaluated RevCom using 56K pairs (e.g., <code change, comment>) collected from 4 Python and 4 Java subject systems • Metrics: Used three evaluation metrics - BLEU score, perfect prediction and semantic similarity from literature • Comparison: Compare the performance with both DL-based and IR-based baselines
10. 8.3 25.87 12.92 34.19 15.24 37.67 18.02 41.49 0 5 10 15 20 25 30 35 40 45 BLEU% SS% BLEU% SS% BLEU% SS% BLEU% SS% Top-1 Top-3 Top-5 Top-10 Cross-project Fig: Performance of RevCom in cross project settings Performance of RevCom 14.84 2.39 31.03 21.44 3.04 39.61 23.88 3.16 42.8 26.63 3.39 46.24 0 5 10 15 20 25 30 35 40 45 50 BLEU% PP% SS% BLEU% PP% SS% BLEU% PP% SS% BLEU% PP% SS% Top-1 Top-3 Top-5 Top-10 Within Project Fig: Performance of RevCom in within project settings 10
11. Performance Comparison of RevCom with Baselines 0 5 10 15 20 25 30 35 40 45 50 BLEU% PP% SS% BLEU% PP% SS% BLEU% PP% SS% BLEU% PP% SS% Top-1 Top-3 Top-5 Top-10 CommentFInder RevCom Code Reviewer Fig: Performance Comparison of RevCom with baselines 11
12. Take Home Message 12 Recommend for all type of changes ~ 6 hours/week ~ 15-64 hours wait Reduce cognitive effort and time Capture the intent of reviewers Developed Code Modern Code Review Comments Reviewer Author
13. 13 THANK YOU! QUESTIONS? Replication: revcom-icsme23 Contact: ohiduzzamanshuvo@dal.ca RAISE Lab Intelligent Automation in Software EngineeRing