Introducing GenC (By: Ahsan Goheer) - DevFest Lahore 2024

Talk by Ahsan Goheer (https://www.linkedin.com/in/ahsangoheer/) at DevFest Lahore 2024 by GDG Lahore.

Transcript

1. Introducing GenC An Open Source Generative AI Orchestration Framework Dec, 2024
2. Who am I? • AI Consultant specialized in Generative AI, NLP, and Computer Vision solutions • Lead the development of Generative AI CoE at Systems Limited • Expert in architecting scalable MLOps solutions across cloud platforms • Currently leading a cross-functional team of 6 data scientists in a healthcare startup by Visionet ( Sister company of Systems Limited)
3. Generative AI Solutions Credits: Ali Arsanjani - https://dr-arsanjani.medium.com/the-genai-reference-architecture-605929ab6b5a
4. • An open-source framework for building GenAI-powered applications • A glue framework that enables developers to compose capabilities across different languages, SDKs, platforms, devices & cloud, and connect them together What is GenC
5. • GenAI ecosystem fragmentation: many GenAI capabilities - LLMs, chains, agent logic, custom functions, RAG, ReAct, model cascades and routers, etc. evolve in siloed domains, and do not compose or interoperate with one-another • GenC connects them together through an abstraction in interfaces, greatly enhance the development efficiency and flexibility Why GenC
6. Ecosystem Fragmentation • The existence of the digital ecosystem in smaller, isolated parts across programming languages, SDKs, deployment backends, environments and platforms. • For example: Prototyping of GenAI logic in a colab notebook and the need to deploy it in a mobile application.
8. • Composability & flexibility ◦ Ability to freely compose building blocks to express your own application logic, and integrate diverse SDKs, platforms, and ecosystems • Portability & platform independence ◦ Across programming languages (e.g., from Python to Java and C++). ◦ Across prototyping and production deployments (e.g., from Colab notebooks, to cloud servers, to mobile apps) ◦ Across on-device and cloud platforms Benefits of GenC
9. • Customizability ◦ Create your own custom runtimes, and incorporate your own libraries & services (models, databases) as building blocks, and mix and match them • Development velocity ◦ Allow logic to be expressed at high level, and easily modifiable ◦ Portability enables prototype code to be deployed with ease • Performance & security ◦ Faster and more secure C++ runtime, asynchronous programming model ◦ Application logic can be statically analyzed, verified, optimized, and executed on scalable distributed platforms Benefits of GenC (cont’d)
10. Simple chain in LangChain powered by a device-to-Cloud model cascade 1) Create a Chain
11. Simple chain in LangChain powered by a device-to-Cloud model cascade 2) Specify a Cloud Model
12. Simple chain in LangChain powered by a device-to-Cloud model cascade 3) Getting the Intermediate Representation
13. Simple chain in LangChain powered by a device-to-Cloud model cascade 4) Create a Runtime
14. Simple chain in LangChain powered by a device-to-Cloud model cascade 5) Define on Device Model
15. Simple chain in LangChain powered by a device-to-Cloud model cascade 6) Create a Model Cascade
16. Simple chain in LangChain powered by a device-to-Cloud model cascade 7) Saving IR
17. Simple chain in LangChain powered by a device-to-Cloud model cascade 7) Porting to Java
18. Tutorials (https://github.com/google/genc/tree/master/gen c/docs/tutorials )
19. Show Some Love
20. Questions?