FINE TUNING YOUR LLM Jun 5, 2024 D I S C O R D O F F I C E H O U R S Korey Stegared-Pace Nitya Narasimhan, PhD

No content

Welcome What - is fine-tuning? Why, When & How – for usage Topic 1: Data Preparation Topic 2: Training & Evaluation Topic 3: Deployment & Uses Summary

1 0 - M I N O V E R V I E W : F I N E T U N I N G I N T RO Let’s set the stage for today’s discussion

W H AT I S F I N E T U N I N G ? A common practice in machine learning where we retrain an existing model with new data to improve performance on task. This is an advanced technique that requires some expertise to get desired results. Incorrect usage may degrade performance.

W H E N S H O U L D I F I N E - T U N E ? But the approach is valid only if the benefits outweigh costs. • Do you have a good use case? (format, edge cases, new skills) • Have you tried other options? • Did you factor in other costs? (compute, data, maintenance) • Did you confirm the benefits? (evaluation, region availability)

W H Y S H O U L D I F I N E - T U N E ? Fine-Tuning may be appropriate if your desired response quality is not achievable with prompt engineering or RAG approaches. Another reason may be the cost efficiency achieved by reduced token usage or ability to upskill a cheaper model (within reason).

P R E P – T R A I N – E VA L U AT E - D E P LOY https://platform.openai.com/docs/guides/fine-tuning

D I S C U S S I O N : F I N E T U N I N G I N AC T I O N Have you tried fine tuning a model? What models did you use? Where did you get the date? Why did you pick this option? What were the challenges in training – evaluation – deployment?

No content

T U T O R I A L U S E C A S E A factual chatbot that answers questions about periodic table elements using limericks

Prepare & Upload Dataset Tutorial Mode – We have a sample set with 10 records for tutorial walkthrough only. In real world usage, you will need 100+ samples for good results (with cost tradeoffs) Step 1

Create Fine-Tuning Job & Track Status to Completion Step 2

Get Deployed Model ID and Test it using normal Chat Completion API Step 3

Track job status, View evaluated metrics, Compare outcomes No-Code Approach via UI