Rachel Shalom – Devops made easy with GenAI

International LT session by GDSC TMU x WTM 2024

Transcript

1. Internal Use - Confidential DevOps made easy with NativeEdge
2. Internal Use - Confidential Hello! I am Rachel Shalom ❖ A principal data scientist at Dell, now working in the servers and storage group ❖ Worked for healthcare startups, and experienced with Time-Series and NLP ❖ Coffee Fan
3. Internal Use - Confidential Edge Orchestration Install deep north app on 5k cameras needs orchestrator that will manage and perform tasks like: - Choose an OS - Set file system - Install the deep north software on each camera - Set the network between cameras and the application - Turn off all cameras between 10pm to 6am - Perform occasional software upgrade
4. Internal Use - Confidential Configuration file as a blueprint Costumer App OS Data Retrival File System All of these tasks are written into a configuration file that acts as a Blueprint that describe the enviromnment and service Blueprint
5. Internal Use - Confidential How do we overcome the challenge ? Q&A Model Q: what are the steps needed to install OS on 5k edge devices? Blueprints validation Models input: blueprint Output: Security Alert: Password must be kept in vault Export to dev env. Once the blueprint is ready it is exported in 1 click to dev env. Finetuned blueprints generation models User: “create a blueprint that install kvm (os) with 16gb ram and turn the VM on” A: In order to install os on 5k cameras you need to work through the following steps: 1. Provide the os installation file …. Q&A Model 2 models : - code autocompletion - instructions to blueprint
6. Internal Use - Confidential How do these models work? AI based Semantic Search information on dell.com sharepoints Confluence pages Q&A LLM Embeddings saved in a vector DB : Blueprints : Retrieval Augmented Generation Finetuning a model on proprietary data
7. Internal Use - Confidential Finetuning with LoRA An efficient finetuning technique that significantly reduces the number of trainable parameters For in depth understand of Lora read my blog:
8. Internal Use - Confidential Few shots learning
9. Internal Use - Confidential Few shots learning for Blueprints
10. Internal Use - Confidential Why Copilot and GPT is not enough? • Writing configuration files- blueprint is based on a unique proprietary language of nativeEdge • When we tried these tools we got incorrect code or what is known as model hallucination Finetuned model
11. Internal Use - Confidential Let’s See A Demo
12. Internal Use - Confidential demo
13. Internal Use - Confidential Serving tools and Chat-UI https://github.com/huggingface/chat-ui
14. Internal Use - Confidential Questions?