Explainable Software Engineering

Keynote delivered at the Bits & Chips Event held in Eindhoven, The Netherlands, on October 12, 2023.

In artificial intelligence (AI), it’s increasingly recognized that components that learn from data need to be explainable. In this talk, we take explainability one step further, using it as a lens to rethink the full software engineering life cycle. To that end, we consider explainability of both the software engineering process and the resulting software system. We use this to shed new light on bug fixing, the use of language models during coding, delay prediction during agile planning, and the digitalization of society. Based on this, we envision a future of software engineering in which explainability is a first-class citizen.

The research covered in the talk was partly co-sponsored by ING Bank, through AI for Fintech Research (AFR), an ICAI Lab (2020-2024). The presentation also served as the official announcement of AI4SE, a 5 year collaboration between JetBrains Research and TU Delft.

Relevant links and papers covered:

- Tim Miller. Explanation in artificial intelligence: Insights from the social sciences. https://arxiv.org/abs/1706.07269

- Patrick Altmeyer, Arie van Deursen, Cynthia Liem. Explaining Black-Box Models through Counterfactuals. JuliaCon 2023. https://proceedings.juliacon.org/papers/10.21105/jcon.00130

- Patrick Altmeyer, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, Cynthia Liem. Endogenous Macrodynamics in Algorithmic Recourse. IEEE SaTML, 2023. https://openreview.net/pdf?id=-LFT2YicI9v

- Taija: Trustworthy Artificial Intelligence in Julia. https://github.com/JuliaTrustworthyAI

- Matthías Páll Gissurarson, Leonhard Applis, Annibale Panichella, Arie van Deursen, David Sands. PropR: property-based automatic program repair. ICSE 2022. https://dl.acm.org/doi/abs/10.1145/3510003.3510620

- Code4Me. https://code4me.me/

- The AI for Software Engineering (AI4SE) Lab. JetBrains & TU Delft, 2023. https://lp.jetbrains.com/research/ai-for-se/

- Elvan Kula, Eric Greuter, Arie van Deursen, Georgios Gousios; Dynamic Prediction of Delays in Software Projects using Delay Patterns and Bayesian Modeling. FSE 2023. https://arxiv.org/abs/2309.12449

- Elvan Kula, Eric Greuter, Arie van Deursen, Georgios Gousios; Factors Affecting On-Time Delivery in Large-Scale Agile Software Development. IEEE TSE 2022. https://ieeexplore.ieee.org/abstract/document/9503331

- Adviescollege ICT Toetsing, AcICT / Advisory Council on IT Assessment. 2023. https://www.adviescollegeicttoetsing.nl/

Transcript

1. 1 Explainable Software Engineering Arie van Deursen Delft University of Technology Bits & Chips Event, October 12, 2023 @avandeursen@mastodon.acm.org
2. “Explanation in Artificial Intelligence: Insights from the Social Sciences” (Tim Miller, Artificial Intelligence, 2018) An explanation is an answer to a why-question 2
3. Explanations are Contextual • Contrastive: compared to counterfactual alternative • Selective: focusing on relevant parts of full causal chain • Social: transferring knowledge, assuming prior knowledge (Tim Miller, Artificial Intelligence, 2018) 3
4. Counterfactual Reasoning • Factual: Model denies loan • Counterfactual: Alternative inputs that would accept loan • Algorithmic recourse: Change of behavior to get desired outcome 4 Patrick Altmeyer
5. A Library for Generating Counterfactuals • Possible, faithful, plausible, “close” to the factual, … • Gradient descent in feature space (with extra cost terms) • Rich library of Julia packages • Macro-effects after recourse adoption 5 https://github.com/JuliaTrustworthyAI Patrick Altmeyer IEEE SaTML, 2023 JuliaCon, 2022, 2023
6. 6 Counterfactulals … in bug reports Factual: Buggy behavior Counterfactual: Expected behavior
7. (Automated) Program Repair • Current (factual) code: yields undesired behavior • Imagined (counterfactual) code: should meet desired behavior! • Formulate as (evolutionary) search § Population of candidate fixes § Cross-over & mutation to transform fixes § Fitness function (test failure ratio) to guide selection 7 https://program-repair.org/ Elena @ One day at a time
8. Searching for Fixes … for Haskell? • Strongly typed • Property-based tests besides unit tests • Rich compiler infrastructure Does this help in automated program repair? 8 Matti Gissurarson Leonhard Applis ICSE2022
9. Unit Tests vs Properties 9
10. Typed Holes in Haskell Code • Compiler can suggest “valid hole fits” in relevant context • Compiler can be configured with “fit-finding” plugins 10
11. PropR – Property-Based Repair Evolutionary search for fix candidates: 1. Check tests & properties 2. Find expressions “covered” by violations 3. Find places to punch holes (“perforation”) 4. Find / combine hole-fit candidates (custom compiler plugin) so that the fix meets ever more properties 11
12. Does this Work? Or is it Overfitting? • Experiments on failing student programs • Fix proposed for 43% of programs • 30% of fixes are good (manual inspection): § => Overall fix rate of 13% § Tests pass, concise, natural, … • 70% of fixes are poor: § Overfit on tests; just make them pass § Humans can imagine (counterfactual) test cases that still would fail with proposed fix 12
13. Code4Me: A plugin for AI-Assisted Completion • Amazing (coding) capabilities of large language models • Black box models • Plenty of success stories, but often self-reported (big tech) • What do devs really do with them? • Investigating this with Code4Me completion plugin (JB + VS) 13 Maliheh Izadi
14. Code4Me Telemetry Data • Suggested insertion • Accepted insertion • Final insertion § Line after 30 seconds • Context (opt-in) • Model used InCoder (2022) UniXcoder (2022) CodeGPT (2021) 14
15. After 12 Months and 2 Million Completions 15
16. Trigger Points Matter 16
17. Accepted Suggestions are Still Changed 17
18. Why Were (10,000) Completions Rejected? 55%: Model underperforms: § Variable, function, literal, … § Nr of parameters, types, … 20%: Model not trained for usage: § Mid-token, errors in / no context, … 8%: OK, but user knows better: § Overlap with right context, … 17%: Not Applicable: § No code: comment, latex, settings, … 18
19. [ Intermezzo ] Today: New Lab Announcement! 19
20. JetBrains & TU Delft • AI4SE: AI for Software Engineering • 2023-2028 • Five research tracks: § Validating (AI) generated code § Optimizing code language models § IDE-AI alignment § Run time information in the IDE § Programming education • ICAI Lab with 10 PhD candidates 20 https://jb.gg/ai4se Vladimir Kovalenko Maliheh Izadi
21. [ End of Intermezzo ] 21
22. Beyond Code: Agile at Scale • ING Bank, with 15,000 IT staff • Self-organizing teams (5-9 developers) • Short iterations (1-4 weeks) • User stories, features, epics • Delivered in releases (2-6 months) • Quarterly planning of all releases 22 Elvan Kula
23. Why is My Project Late? • What are factors affecting timely epic delivery? § Let’s ask! • How do these factors impact schedule deviation? § Let’s measure and model! 23 Elvan Kula IEEE TSE 2022
24. Timely Epic Delivery: Perceived Factors • Survey 1: Which factors? • 289 responses • 25 factors; 5 dimensions • Survey 2: Factor importance? • 337 responses • Rated impact level per factor Factor top 10: 1. Requirements refinement 2. Task dependencies 3. Organizational alignment 4. Organizational politics 5. Geographic distribution 6. Technical dependencies 7. Agile maturity 8. Regular delivery 9. Team stability 10. Skills and Knowledge 24
25. Measuring Delay: Balanced Relative Error • If actual delivery date after estimated date (”late”, pos%): • If actual delivery date before estimated date (“early”, neg%): • Collected BRE from 3,771 epics (273 teams), for 3 years 25
26. 13 Predictor Variables • 35 metrics for 20 factors • 13 metrics explain 67% of variation (MARS model) • Match with perception? § Underestimated: size § Agreed: dependencies, seniority, stability § Overestimated: geography, regular delivery § Agreed low/no effect: coverage, code smells, … 26
27. Dynamic Delay Prediction • Delay knowledge increases as epic unfolds (in milestones) • Mobility literature: Delay adheres to patterns, which can be learned from clustering delay time series • Is epic delay subject to patterns? • Can patterns improve delay prediction? 27
28. Epic Delay Patterns 28 Elvan Kula FSE 2023 Dataset: 4,040 epics of at least 10 sprints from 270 teams, 2017--2022 % Epics in category: 36% 44% 14% 6%
29. Delay Patterns Improve Delay Prediction 29
30. Explainable Digitalization? • Why-questions in government IT? • Explainable to whom? § voters, taxpayers, end-users, members of parliament, cabinet, policy makers, civil servants, developers, …? • Will digitalization strengthen or weaken our democracy? 30
31. Assessment Framework in Nine Chapters 1. Business case, benefits, finance 2. Project organization and ownership 3. Risk management and project dependencies 4. Alignment business processes and IT solution 5. Scope management 6. Architecture, functional feasibility, technical realizability 7. Realization and planning 8. Tendering aspects 9. Acceptance, production, and transition to line https://www.adviescollegeicttoetsing.nl/onze- werkwijze/documenten/publicaties/2021/12/01/toetskader-acict 31
32. 100 Reports Since 2015 • Government is complex • Common risks § Cost and time overruns § Wishful thinking in governance § Knowledge deficit w.r.t. (IT) suppliers • Common advice § Keep it small and simple § Deliver measurable results early § Manage IT suppliers rigorously 32
34. The Road Ahead • Recognize explainability as first class citizen in software engineering § In coding and beyond • Build carefully curated (public) software engineering data sets • Focus on domains with constrained counterfactuals • Thoroughly investigate how language models can deliver true value in SE 34
35. 35 Explainable Software Engineering Arie van Deursen Delft University of Technology Bits & Chips Event, October 12, 2023 @avandeursen@mastodon.acm.org Radioactive; Marjane Satrapi
36. Image Credits • Rembrandt van Rijn: De anatomische les (1632), Wikipedia • Malevich: Suprematism, Wikipedia. • One day at a time, Season 2. Elena, 2018. • Simon Greig, Square peg. Flickr, CC-BY-NC-SA • Fractal, Wikipedia • Red Stage Curtain, Sethoscope, Flickr, CC-BY-NC • Visual Inspection, Sanofi, Flickr, CC-BY-NC • “JetBrains huurt Terrace Tower”, www.jll.nl, September 2021 • Edmund Bass, Nederlandse Spoorwegen • Franz Kafka, Metamorphosis, Gyan Publishing House • Tweede Kamer, plenaire zaal • Malevich: Suprematism, Wikipedia. • Radioactive, directed by Marjane Satrapi 36