OpenShift Commons: Konveyor AI

Konveyor AI or “Kai”, is an early effort of Generative AI applied to Application Modernization being explored under the Konveyor Ecosystem at konveyor-ecosystem/kai.

Kai implements a Retrieval Augmented Generation (RAG) approach that leverages data from Konveyor to help generate code suggestions to aid migrating legacy code bases to a different technology. The intent of this RAG approach is to shape the code suggestions to be similar to how an organization has solved problems in the past, without additional fine-tuning of the model.

Transcript

1. Generative AI for Application Modernization Konveyor AI 1 Ramón Román Nissen Konveyor Maintainer Principal Product Manager Red Hat Syed M Shaaf Konveyor Contributor Developer Advocate Red Hat
2. 2 Konveyor Overview
3. www.konveyor.io A community of people passionate about helping others modernize and migrate their applications to Kubernetes by building tools and best practices on how to accelerate the journey to Kubernetes Additional contributors welcome CNCF sandbox project Migration Toolkit for Applications Supported Tool Available with OpenShift subscription 3 Konveyor Hub Assets Generation Reporting Assessment Code Transformation Planning Analysis
4. 4 “Surface insights on applications at scale to empower enterprise architects to make better-informed decisions related to modernization activities” github.com/konveyor/community/Charter.md Konveyor Mission and goals
5. 5 Analysis Engine Analysis Report YAML Analysis Rules YAML OpenAPI Spec validation Language Provider Language Server Protocol Implementation Source Code Binaries Translate conditions in a Rule to LSP queries Analysis Rules Community Knowledge + Custom Rules Community contributed rules included with Konveyor konveyor/rulesets ~2400+ Rules and growing Custom Rules for Organization specific frameworks/technologies Report informs Architect of areas of concern
6. 6 Application Analysis Issue identification and guidance for developers Analysis Incident Analysis Info Details Issue location in source code
7. 7 Konveyor AI (Kai) Goal
8. 8 Konveyor AI (Kai) Goal Goal: Improve the economics of re-platforming and refactoring applications to Kubernetes and cloud-native technologies by leveraging Generative AI Approach: ▸ Expand Konveyor/MTA capabilities beyond surfacing information ▸ Generate code suggestions for discovered migration issues ◦ Work with LLMs using structured migration data Konveyor has collected ▸ Avoid fine-tuning each model by using Retrieval Augmented Generation (RAG) ◦ Shape LLM results with examples of how Organization has solved similar problems in the past ▸ Model agnostic ◦ Integration with commercial models ◦ Ability to bring your own model
9. 9 Konveyor AI (Kai) Approach
10. 10 Konveyor AI (Kai) Generative AI to automate source code changes between technologies ▸ Uses data in Konveyor to generate code suggestions ◦ Source code analysis with Rules ▪ Pinpoint issues to adopting a new technology ◦ Changelog history from source repositories ▪ Before/After of previously solved issues ▸ Crafts a tailored LLM prompt based on: ◦ Knowledge of specific problems that need to be addressed ◦ Knowledge of prior successful code changes ▸ Provides suggested code changes via IDE plugin
11. 11 Kai + Konveyor Functional Overview
12. 12 Large Language Model (LLM) Concerns/Limitations ▸ Limited context windows ◦ It’s not feasible to include an entire applications full source code in a prompt ▪ Generally models have a limited context window ▪ Costs increase as more data is included in a request ▸ Desire to augment a LLM’s knowledge without fine-tuning ◦ Corporate internal frameworks are generally not part of an existing models ‘training data’ ◦ We want to leverage a RAG approach to include ‘solved examples’ in the prompt to help shape the results ▸ Integrate with multiple models ◦ Rapid advancements on models increases the desire for flexibility to explore new models as they become available
13. 13 Approach: LLM Limited Context Windows Source Code Analysis Guides LLM interaction ▸ Use static code analysis to help guide interaction with LLM ◦ We assume limited context sizes in LLM requests so we break larger questions into smaller ones ◦ Generally not feasible to upload an entire repository of source code to a LLM ◦ Use Analysis issues to pinpoint smaller focal areas Source Code Repository Konveyor Analysis Analysis Report Issue #1 Issue #2 Issue #N LLM Request LLM Request LLM Request Aggregate Responses
14. 14 Approach: Need to augment a LLM’s knowledge Retrieval Augmented Generation How can we influence a LLM to give us better info on knowledge it wasn’t ‘trained’ on? ▸ We assume that the models we use have NOT been exhaustively trained in the domain we want to leverage it in ◦ Think of custom frameworks private to an organization ◦ Newer libraries that were invented after the model was trained ▸ We will augment knowledge the model can answer by leveraging a ‘Few Shot’ example with passing data into the prompt
15. 15 Retrieval Augmented Generation (RAG) Approach Source Code Solved Issues (code snippets) Analysis Issues Prompt Instructions + Large Language Model (LLM) Updated Source Code Reasoning Issue to fix: "Stateless EJBs can be converted to a cdi bean by importing `jakarta.enterprise.context.ApplicationScoped` and replacing the `@Stateless` annotation with a scope eg `@ApplicationScoped`" Line number: 14 diff --git a/src/main/java/org/jboss/as/quickstarts/bmt/ManagedComponent.java b/src/main/java/org/jboss/as/quickstarts/bmt/ManagedComponent.java … -@Stateless -@TransactionManagement(TransactionManagementType.BEAN) -// tell the container not to manage transactions +@ApplicationScoped …
16. 16 Differentiator What differentiates this approach from a CoPilot clone ▸ Uses structured migration data Konveyor/MTA has collected across the Organization ◦ Application Inventory, Migration Waves, Analysis Reports, Source code repos ▸ Guided workflow uses discovered analysis issues to drive LLM interaction ◦ Kai understands the issues that need to be addressed to adopt a new technology and uses this to focus the LLM ▪ Leverages a shared repository of community knowledge of migration rules ▸ Extracts patterns from previously solved problems in the Organization and leverages to shape LLM results ◦ Beyond this specific application, Konveyor looks at the history of all applications in the configured Organization ▸ Model Agnostic: ◦ Integration with commercial models. ◦ Ability to specify your own models and work with other providers and/or local models ▪ Example: on-premise Hugging Face models hosted by OpenShift AI
17. 17 Demo
18. Technical Debt Lack of agility Security risks Maintenance and Cost Inability to make changes to the stack and frameworks. Hard to develop new features Unable to keep with business demands in a timely manner. Old frameworks and applications pose a threat to business and application and IT operations e.g., data leaks, etc Maintaining legacy applications is expensive. Vendor support, skills are costly and hard to find The pit of legacy applications Nurturing applications can be costly and time consuming..
19. 19 Migration of Legacy code to new Today’s example -> JavaEE to Quarkus
20. 20 Current Application component breakdown Coolstore demo app ➔ Scaling challenges e.g. individual components or app ➔ Lifecycle is prolonged due to dependencies ➔ Everything runs in the same process ➔ Technical debt, hard to innovate ➔ CI/CD
21. Large Language Model 21 Developer IDE AI plugin issues and diffs to apply Konveyor AI Konveyor.io static code analysis Generated Patches Application Facts Tags Source Code Commit Changelog Analysis Report Legacy Code Updated Code Issues description Incidents file line number message Issues and diffs captured by Konveyor for continuous improvement of responses + Konveyor AI - “KAI” Static code analysis + LLM + Inference
22. 22 The world of migration and challenges Mesh between refactor and re-architecture ➔ Scaling is more independent to the component (hypothetically speaking..) ➔ Life cycles are shorter depending on the teams ➔ Everything is no longer in one big process ➔ Distributed, can introduce other challenges like latency, compatibility, single point of failures etc. ➔ Capacity, not all services are used all the time…. wait what???
23. 23 I have migrated. What do I do next?
24. 24 Migration What does “Kai” do and why. And what it doesn’t • Refactor ◦ Migrate JavaEE to JakaratEE and Quarkus ◦ Migrate JMS to Reactive (MicroProfile) ◦ Migrate EJB to REST ◦ Remove unnecessary code, no longer required. • Re-Architecture - NO! ◦ Separation of concern ◦ Domain of Driven Design ◦ Functional, Event driven, serverless… • Process ◦ Automation ◦ DevOps
25. 25 Wanna know more? Come join us! ▸ Get involved with the community: ◦ Community meetings: Learn more about upcoming features and share your insights. ◦ Mailing list: Subscribe for the latest updates on Konveyor. ◦ Kubernetes Slack: Join us at #konveyor
26. Thank you! 26 Ramón Román Nissen Konveyor Maintainer Principal Product Manager Red Hat Syed M Shaaf Konveyor Contributor Developer Advocate Red Hat