Explainable AI Dr. Kla Tantithamthavorn Senior Lecturer in Software Engineering Monash University, Australia http://chakkrit.com @klainfo Software Engineering in the Age of Generative AI 

Dr. Kla Tantithamthavorn ● Expertise in Explainable AI and Software Engineering ● Co-authored the first online book on Explainable AI for Software Engineering (http://xai4se.github.io), attracting over 20,000 page views from 83 countries worldwide ● Co-edited an IEEE Software Special Issue on XAI for SE ● Awarded ARC DECRA Fellowship, JSPS Fellowship, ACM SIGSOFT Distinguished Paper, Distinguished Reviewer ● Received strong media attention from Gizmodo, Australian Cyber Security Magazine, TechXplore, Cybersecurity Connect, Australian Computer Society, etc Learn more http://chakkrit.com/ 

AI Has Emerged as a Powerful Tool for Software Companies Syntax-Aware On-the-Fly Code Completion, Wannita Takerngsaksiri, Chakkrit Tantithamthavorn, Yuan-Fang Li, Under Review at IEEE Transactions on Soft. Eng. (2023) 

Generative AI (ChatGPT, LLMs, etc.) ● Since its introduction in November 2022, ChatGPT has rapidly gained popularity due to its remarkable ability in language understanding and human-like responses ● Generative AI is a type of artificial intelligence that is capable of creating original and unique content, such as images, videos, music, or text. It works by learning patterns and styles from existing data and generating new content. ● ChatGPT, based on GPT-3.5 architecture, has shown great promise for revolutionizing various research fields, including code generation, testing, and bug fixing, improving efficiency, enhancing creativity, and reducing costs. 

What Software Engineering Tasks that ChatGPT Can Help? 

ChatGPT for Software Planning (Generate Business Requirements) 

ChatGPT for Software Planning (Generate User Stories) 

ChatGPT for Software Design 

ChatGPT for Software Design (Generate a Class Diagram) 

ChatGPT for Software Design (Generate a State Diagram) 

ChatGPT for Coding 

ChatGPT for Coding 

ChatGPT for Software Testing 

ChatGPT for Software Testing (Generate acceptance test cases) 

ChatGPT for Software Testing (Generate test cases) 

ChatGPT Is Dumber Than You Think Here is the fundamental problem with ChatGPT: it can provide answers and information that no one ever knows for sure is true because it is not referenceable. 

Challenge 1: ChatGPT Can’t Generate High-Quality Code We analyzed 4,066 ChatGPT- generated code implemented in two popular programming languages, i.e., Java and Python, for 2,033 LeetCode’s programming tasks. Key Findings: ● Code quality issues commonly happen in both code that pass or failed test cases, highlighting the need for characterizing and addressing these concerns alongside the functional correctness. ● Issues in ChatGPT-generated code can be categorized into four categories: Compilation & Runtime Errors, Wrong Outputs, Code Style & Maintainability, Performance & Efficiency ● Wrong Outputs and Code Style & Maintainability issues are the most common challenges faced by the ChatGPT-generated code, while Compilation & Runtime Errors and Performance & Efficiency issues are less prevalent. Liu, Yue, et al. "Refining ChatGPT-Generated Code: Characterizing and Mitigating Code Quality Issues." Under Review at TOSEM, 2023. 

Challenge 2: ChatGPT Can’t Generate Secure Code Context: this program receives an email address as input, and passes it to a program (as a parameter) through a shell. Problem: Handling input in this manner allows a malicious adversary to execute arbitrary code by appending shell instructions to a fictitious email. CWE: Arbitrary code execution (CWE-94) First Attempt Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions, IEEE S&P 2021 

Challenge 2: ChatGPT Can’t Generate Secure Code Context: this program receives an email address as input, and passes it to a program (as a parameter) through a shell. Problem: Handling input in this manner allows a malicious adversary to execute arbitrary code by appending shell instructions to a fictitious email. CWE: Arbitrary code execution (CWE-94) Second Attempt 

Challenge 2: ChatGPT Can’t Generate Secure Code Context: this program receives an email address as input, and passes it to a program (as a parameter) through a shell. Problem: Handling input in this manner allows a malicious adversary to execute arbitrary code by appending shell instructions to a fictitious email. CWE: Arbitrary code execution (CWE-94) Last Attempt, Otherwise I will give up! 

Challenge 2: ChatGPT Can’t Generate Secure Code Context: this program receives an email address as input, and passes it to a program (as a parameter) through a shell. Problem: Handling input in this manner allows a malicious adversary to execute arbitrary code by appending shell instructions to a fictitious email. CWE: Arbitrary code execution (CWE-94) With human intervention! 

Challenge 3: ChatGPT Can’t Explain the Answers 

Take-Away Messages: - Generative AI will transform SE. - However, it still can’t generate high-quality, secure, explainable code. - Generative AI should be used as an assistant tool, but not to replace software engineers. Learn more http://chakkrit.com/