Upgrade to Pro — share decks privately, control downloads, hide ads and more …

.NET Day 2023: Bringing the Power of Deep Learning to .NET with ONNX and ML.NET

dotnetday
September 01, 2023

.NET Day 2023: Bringing the Power of Deep Learning to .NET with ONNX and ML.NET

With the advent of ChatGPT, everyone is talking about AI. And it is destined to become even more embedded into our daily lives. However, while .NET is usually not associated with machine learning, ML.NET is a powerful tool within our ecosystem and allows us to use our existing C#-skills to train and use machine learning models. The talk will start with an introductory look at ML.NET. Getting started with AutoML and Model Builder is easy and training such small models is feasible for everyone. However, training state-of-the-art large models is near impossible and thus, we can integrate existing, pretrained deep learning models. In the talk, we will use ONNX, an open format to exchange deep neural networks between tools and frameworks, to integrate a pretrained BERT model into a .NET application. BERT is a transformer-based natural language model, similar to the famous GPT-models, and will answer questions in natural language for us.

dotnetday

September 01, 2023
Tweet

More Decks by dotnetday

Other Decks in Technology

Transcript

  1. Bringing the Power of Deep Learning to .NET with ONNX

    and ML.NET 29.08.2023 .NET Day Switzerland 2023 Alexander Kayed Image generated with clipdrop by stability.ai
  2. ENGINEER / ARCHITECT / SPEAKER / TRAINER @ NOSER ENGINEERING

    AG • Microservice, CQRS, event-sourced cloud app (.NET+ Angular) • GIS-Project: Topology Data, LoS, Lidar (WPF) • ChatGPT-based “Knowledge-Bot” (.NET, Semantic Kernel) • OSS: Akade.IndexedSet (Performance oriented multi-index data structure) • Focus: C# / .NET, ML / AI Alexander Kayed 01.09.2023 WE KNOW HOW. 3 Alex MSc ETH in Computer Science github.com/akade
  3. Why not just call ChatGPT? 01.09.2023 WE KNOW HOW. 8

    Image: https://www.reddit.com/media?url=https%3A%2F%2Fi.redd.it%2Fdiscovered-this-v0-18nnu31v5tea1.jpg%3Fs%3D23afe7ca4c811106cec84501656b38e8afc06415
  4. Why not just call ChatGPT? 01.09.2023 WE KNOW HOW. 9

    Local training or local inference Approachable & limited time / money Privacy
  5. Llama 2 01.09.2023 WE KNOW HOW. 10 • Most recent,

    fully open-sourced LLM by Meta • OpenAI models are not open-source https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md ~ 4537x A100 for a month ~ 539 tCO2 eq (5t/year/person CH) ~ 13.6M $ on Azure https://www.bafu.admin.ch/bafu/de/home/themen/klima/inkuerze.html https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/ 70B (109) Parameter ~260 GB (FP32) 3.3M h GPU Time 27.5K “Chats” (SFT) 1.4M “Ratings” (RLHF) 2 trillion (1012) tokens
  6. ML.NET 01.09.2023 WE KNOW HOW. 12 Demo Time! Pokemon Image

    Set from https://www.kaggle.com/datasets/kvpratama/pokemon-images-dataset
  7. Transfer Learning 01.09.2023 WE KNOW HOW. 13 • Reusing an

    existing convolutional neural network − Plane: 89% − Boat: 9% − Car: 1% − Apple: 0.5% − … Weights from a pretrained net
  8. Transfer Learning 01.09.2023 WE KNOW HOW. 14 Weights from a

    pretrained net − Plane: 89% − Plant: 9% − Plant: 1% − Apple: 0.5% − …
  9. Transfer Learning 01.09.2023 WE KNOW HOW. 15 − Car: 0.5%

    − House: 1% − Plant: 20% − Apple: 0.5% − … Freeze some weights Weights from a pretrained net
  10. Transfer Learning 01.09.2023 WE KNOW HOW. 16 − Plant: 88%

    − Insects: 5% − Water: 4% − Fire: 2% − … Freeze some weights New output layer Weights from a pretrained net
  11. 32

  12. 01.09.2023 WE KNOW HOW. 34 ML.NET & ONNX 1. Intro

    - Motivation - Llama 2 2. ML.NET - AutoML / Model Builder - Consumption - Training / Pipeline 3. ONNX / BERT - Idea - Model Zoo - BERT - Netron - ONNX Runtime 4. LlamaSharp - Llama2 7B, running locally - Llama.cpp / LlamaSharp Questions github.com/akade Visit us - at our booth - or at www.noser.com Quelle: https://github.com/dotnet/brand [email protected]
  13. 01.09.2023 WE KNOW HOW. 35 ML.NET & ONNX 1. Intro

    - Motivation - Llama 2 2. ML.NET - AutoML / Model Builder - Consumption - Training / Pipeline 3. ONNX / BERT - Idea - Model Zoo - BERT - Netron - ONNX Runtime 4. LlamaSharp - Llama2 7B, running locally - Llama.cpp / LlamaSharp Thank you! github.com/akade Visit us - at our booth - or at www.noser.com Quelle: https://github.com/dotnet/brand [email protected]