Jirayus Jiarpakdee Kla Tantithamthavorn John Grundy Accepted at IEEE Transactions on Software Engineering (TSE) Dilini Rajapaksha Christoph Bergmeir Wray Buntine
but hard to predict and to prevent. With millions LOCs/project, practitioners do not know: Which files are likely to be defective in the future? SQA practices (pre-testing, code review) are adopted, but very time-consuming and expensive.
if a file will be defective in the future Prediction Help developers find defects faster Timeline V2 Testing Data Defect Models Training Data V1 Important Factors Help managers develop quality improvement plans
still don’t know what should they do to decrease the risk of having defects. Second, practitioners still don’t know a risk threshold for each metric (e.g., how small is a file size should be?)
needed to support QA teams make better data-informed decision- and policy-making. R34: “Lessons learnt from projects are documented and common mistakes are included in review checklists to ensure that they are not repeated.”) • They used documentation and review checklists. • Some activities involve human heuristics in decision-making. How do developers conduct SQA Planning?
DEFECT A rule based explanation 8 Defect Prediction Buggy Clean Why a file is predicted as defective? “A file is predicted as defective, since LOC > 100 and Ownership < 0.8” SQAPlanner: Making Defect Prediction Actionable A rule-based model-agnostic approach to generate explanations and actionable guidance Help developers understand the most important aspects that are associated with defects. Help developers understand the risk threshold (how small it should be?). Generate Explanations Magnum Opus Build SQAPlanner Crossover+ Mutation Generate
the highest coverage, confidence, lift, and consistency when compared to LORE and Anchor. Consistency How stable are the rule-based explanations? Effectiveness How accurate are the rule-based explanations?
1 SQAPlanner is one of the importance advancements towards Actionable Software Analytics. 2 • Follow-up after the talk: chakkrit@monash.edu • Website: http://chakkrit.com • Twitter: @klainfo • Materials: http://xai4se.github.io