Understanding the Impact of Domain Term Explanation on Duplicate Bug Report Detection

Duplicate bug reports make up 42% of all reports in bug tracking systems (e.g., Bugzilla), causing significant maintenance overhead. Hence, detecting and resolving duplicate bug reports is essential for effective issue management. Traditional techniques often focus on detecting textually similar duplicates. However, existing literature has shown that up to 23% of the duplicate bug reports are textually dissimilar. Moreover, about 78% of bug reports in open-source projects are very short (e.g., less than 100 words) often containing domain-specific terms or jargon, making the detection of their duplicate bug reports difficult. In this paper, we conduct a large-scale empirical study to investigate whether and how enrichment of bug reports with the explanations of their domain terms or jargon can help improve the detection of duplicate bug reports. We use 92,854 bug reports from three open-source systems, replicate seven existing baseline techniques for duplicate bug report detection, and answer two research questions in this work. We found significant performance gains in the existing techniques when explanations of domain-specific terms or jargon were leveraged to enrich the bug reports. Our findings also suggest that enriching bug reports with such explanations can significantly improve the detection of duplicate bug reports that are textually dissimilar.

Transcript

1. preencoded.png Understanding the Impact of Domain Term Explanation on Duplicate Bug Report Detection Usmi Mukherjee usmi.mukherjee@dal.ca Masud Rahman masud.rahman@dal.ca
2. preencoded.png The Duplication Conundrum in Bug Tracking Systems Independent Report Submission Users submit bugs reports asynchronously. High Duplication Rate Up to 42% of reports are duplicates. (Zou et al, TSE 2018) Significant Overhead Duplicates add maintenance burden. (Jalbert et al, DSN 2008)
3. Let’s look at a Bug Report 3 78% bug reports are short, take 121 days longer to resolve (Zhang et al, 2017) Domain-Specific Terms make bug reports are difficult to comprehend, difficult to detect duplicates
4. The probability of two persons using the same text to explain the same issue is very low (e.g., 10%–15%) Furnas et al, Communications ACM 1987
5. Textual Dissimilarity of Bug Reports Different textual descriptions for the same underlying issue Variation in component descriptions and observed behaviors Missing or differently written components across reports Jahan and Rahman, SANER 2023
6. The prevalence of domain-specific terms in bug reports could be crucial for understanding and finding duplicates Motivation 78% of bug reports contain less than 100 words Zhang et al, ICPC 2021 Difficult to understand and find duplicates Zhang et al, ICPC 2021 Designed for textually similar, but 19-23% are textually dissimilar Jahan and Rahman, SANER 2023
7. preencoded.png Workflow Step 1: Construction of Explanation Module Fine-tune LLM for explanations of domain-terms Step 2: Enriching Bug Report Add domain term explanations to reports. Step 3: Duplicate Bug Report Report Detection Apply enriched reports to existing techniques.
8. preencoded.png Step 1 : Construction of Explanation Module
9. preencoded.png Step 2 : Enriching Bug Report
10. preencoded.png Step 3 : Duplicate Bug Report Detection Classification Based 1. Siamese-CNN 2. DC-CNN 3. CTEDB Ranking Based 1. BM25 2. LDA+GloVe 3. SBERT 4. CUPID Ranking Based Recall Rate @ K Classification Based AUC, Precision, Recall, F1Score
11. RQ1 : Does enrichment help improve existing techniques in detecting duplicate bug reports ? RQ2 : Does enrichment help in detecting textually dissimilar duplicate bug reports? Research Questions
12. RQ 1 - Does enrichment help improve existing duplicate bug report detection techniques ? Ranking Based Techniques 66.88% SBERT Recall@1 Highest gain in recall. 41.39% LDA+GloVe Recall@5 Best recall improvement.
13. RQ 1 - Does enrichment help improve existing duplicate bug report detection? Classification Based Techniques 5.29% DC-CNN AUC Significant AUC increase. 5.70% CTEDB Precision Notable performance boost.
14. Impact of the number of domain term explanations
15. preencoded.png RQ1: Key Insights Better representation, bridges vocabulary gap Negligible computational overhead (0.11 sec per report)
16. RQ 2 - Does enrichment help in detecting textually dissimilar duplicate bug reports? Textually Similar Textually Dissimilar Ranking Based Techniques LDA+GloVe Impact 63% recall improvement for similar LDA+GloVe Impact 137% recall improvement for dissimilar
17. RQ 2 - Does enrichment help in detecting textually dissimilar duplicate bug reports? Classification Based Techniques Textually Similar Textually Dissimilar DC-CNN Benefits 4-8% gain for textually similar DC-CNN Benefits 5-9% gain for textually dissimilar
18. preencoded.png RQ2: Key Insights Enhances semantic representation Increases keyword overlap Improved domain-specific context
19. preencoded.png Actionable Insights – Software Practitioners Improved Management Better bug report writing Enhanced Search Better search Bug Report Comprehension Better understandability Other Improved SE Artifacts Improve software artifacts like requirements documentation
20. preencoded.png Actionable Insights – Researchers Extraction Optimization Improve term extraction Project Ecosystem Analysis Variation of terms from open source and industrial projects Impact on other Software Engineering tasks Influence on downstream tasks like bug localization
21. THANK YOU Replication Package QUESTIONS ? Pre-print