Boosting Software Development with Generative AI

Transcript

1. ASOCIO Digital Summit 2024 Yoshitake Kobayashi Toshiba Corporation November 8, 2024 Boosting Software Development with Generative AI
2. 2 © 2024 Toshiba Corporation 2 © 2024 Toshiba Corporation
3. 3 © 2024 Toshiba Corporation Toshiba's Vision for Generative AI Building a Safe, Sustainable Future with Generative AI and Knowledge Integration Achieving Sustainable Society Transforming Data into Knowledge to Tackle Social Challenges Committed to People, Committed to the Future. System IoT Connecting Naturally with Generative AI Carbon Neutrality Global Warming Securing Resources Energy Security Labor Shortage Transfer expertise Aging Infrastructure CPS Security Person ⇔ System Person ⇔ IoT System ⇔ IoT Natural Language Communication Person
4. 4 © 2024 Toshiba Corporation Generative AI-Powered Services (1) Reverse Engineering Service (2) GAI Customization Service (3) GAI-Powered Document Search Service Generative AI-Powered Services : AI-no-teTM © 2024 Toshiba Corporation 4
5. 5 © 2024 Toshiba Corporation Point 1 Cost and Time-efficient service, compared to traditional approaches Point 2 High-quality reverse engineering via skilled IT engineers’ verification Point 3 Adaptable IT consulting and support for systems and client requests (1) Reverse engineering service Code and Documents Reverse engineering service Detailed Documents Verify and Refine Learn and Generate
6. 6 © 2024 Toshiba Corporation GAI Services (2) GAI Customization Service Custom GAI Environment for Internal use User-Specific Databases Confidential documents Internal Resources Point 1 Dedicated cloud to protect customers’ data Point 2 All-in-one service from setup to customization Point 3 Optimization Support by Skilled Engineers
7. 7 © 2024 Toshiba Corporation (3) GAI-Powered Document Search Service Custom GAI Environment for Internal use Customized Front-end Software Development Management Helpdesk Use cases
8. 8 © 2024 Toshiba Corporation Collaborate with GAI in the Software Development Process Identify Understand Requirement Analysis Generate Review Implementation Design Operation & Maintenance Testing Management Trial Collaboration Examples: 1. Automating Design Reviews 2. Leveraging Software Assets 3. Generating Test Codes
9. 9 © 2018 Toshiba Corporation 01 Automating Design Reviews 9 © 2020 Toshiba Corporation
10. 10 © 2024 Toshiba Corporation Overview : Automating Design Review Issues Not Fully Addressed in Design Reviews Rapid Issue Fixes with High-Speed Reviews Using GAI Large amount of Documents Requirements Basic design I/F design … Design Security Accessibility N People x H Hours Create GAI Design Documents Modify Review Comments Check Design Security Accessibility Refer Review Points Goals Issues Oversights Due to Many Check Items High Cost of Frequent Reviews …
11. 11 © 2024 Toshiba Corporation Techniques to improve Design Review Trial 1: Conduct reviews with simple instructions Trial 2: Improve review quality with detailed prompt Design Documents Identify the design issues + Pros: General knowledge issues identified. Cons: Complex or domain-specific issues missed. Review Comments Improved prompt includes: •Review checklists •Domain-specific information •Include comments from similar software •… Improved prompt + • Strengths and weaknesses are highlighted • Useful correction suggestions are provided • Valuable info from checks to corrections is generated • Domain knowledge and checklist items are addressed & Review Comments GAI GAI Checklist Design Documents Improved prompt
12. 12 © 2024 Toshiba Corporation Extracted specs for review points and identified missing information Example of requirements specification review Review comments GAI Specifications for the contact confirmation app (COCOA) *https://cio.go.jp/node/2613 Extract all text data included in the PDF specification Instruct generative AI to review with crafted prompts Specs Missing info. Suggestions
13. 13 © 2018 Toshiba Corporation 02 Leveraging Software Assets 13 © 2020 Toshiba Corporation
14. 14 © 2024 Toshiba Corporation Overview: Leveraging Software Assets Unable to utilize the vast amount of software assets Generative AI teaches how to utilize software assets Refer Software Asset DB Question Answer Find similar products to refer How to use reusable parts Refer to past bug fixes … Docs Codes Product A ... Please tell me how to use the XX module The XX has API X1 and X2, and its usage is... I got an YY error, what should I do? The cause of the error is..., refer to ticket #7... GAI Goals Issues Concerns Difficulty Finding Target Assets Scattered Information Delays Understanding Consolidated Information Delivery Conversational Information Retrieval Docs Codes Product B’ Docs Codes Product B Docs Codes Product C
15. 15 © 2024 Toshiba Corporation Leveraging a large amount of software assets with GAI Retrieval-Augmented Generation (RAG) enhances LLM responses with custom data sources Software Asset DB Document Source Code Product A Document Source Code Product B GAI Chat Application 1. Question 6. Answer Data Preprocessing for GAI • Split original data into appropriate units • Adjust data format • Adjust vocabulary and terminology Techniques to improve response quality • Selection and adjustment of search methods • Adjustment of search result priorities and scope Techniques to improve response quality • Techniques for creating search keywords from questions • Optimization of information related to answers • Utilization of conversation history Techniques to improve response quality …
16. 16 © 2024 Toshiba Corporation Leveraging software assets: Chat examples Explain the API provided by the syntax parsing component ---- <snip> ---- ---- <snip> ---- Provide a code example that shows how to use the component ---- <snip> ---- ---- <snip> ---- API Spec. Details for data flow References Code example Detailed Explanation Suggestions
17. 17 © 2018 Toshiba Corporation 03 Generating Test Codes 17 © 2020 Toshiba Corporation
18. 18 © 2024 Toshiba Corporation Overview: Generating Test Codes Testing is Crucial but Overwhelming Comprehensive Testing with GAI Test Code Generation Unit testing. Integration test Requirement analysis Design. Implementation. Cross-checking GAI Documents Source Code Test cases Test code Reduce Workload with Automated Testing Using Generated Test Code Goals Issues Concerned About Insufficient Test Cases Too Many Test Items for Manual Execution Desire Test Automation but No Time to Write Test Code Generate Test Targets from Specs and Source Code Generate Test Code from Source Code and Test Cases
19. 19 © 2024 Toshiba Corporation Trial 1: Generate test code with simple instructions Trial 2: Improve test code quality Generating Test Codes + Issues • Not executable, lacks expected values, and more. • A lot of effort required before testing. Results • Comprehensive Test Cases for Various Patterns • Executable with Minimal Edits GAI Test cases Step 1: Generate Test Cases Step 2: Generate Test Code Generate test code Document Source code Test code Document Source code Source code GAI GAI Test code
20. 20 © 2024 Toshiba Corporation Generating Test Codes: Example Step1: Generating Test Cases Step2: GeneratingTest Code Results • Generated Test Code Executable with Minimal Edits • 91% Average Branch Coverage of Generated Test Code • 66.7% Reduction in Test Code Creation Time Output All Normal Preconditions Expected Values with Specific Results Abnormal Case Test Generation
21. 21 © 2024 Toshiba Corporation Conclusion • Enhancing software development capabilities with GAI • Highlighted three effective technical use cases: • Automating Design Reviews • Leveraging Software Assets • Generating Test Code • These technologies are also applied in our GAI service, AI-no-te
22. 22 © 2024 Toshiba Corporation Generative AI-Powered Services Agenda (1) Generative AI Seminar • Benefits of Using GAI for reverse engineering (2) Generative AI Hands-On Seminar • Generating detailed design documents (3) Learning Case Studies Upcoming event: • December 5, 2024. 14:00-17:30 AI-no-te : Free Hands-on-seminar © 2024 Toshiba Corporation 22
24. 24 © 2024 Toshiba Corporation