Using LIT to analyze Gemma Models in Keras (By: Syeda Maria Hussain) - DevFest Lahore 2024

Talk by Syeda Maria Hussain (https://www.linkedin.com/in/syeda-maria-hussain-a69a1aa5/) at DevFest Lahore 2024 by GDG Lahore.

Transcript

1. Using LIT to Analyze Gemma Models Lahore Syeda Maria Hussain
2. Challenge • Large Language Models (LLMs) are powerful, but their behavior is heavily influenced by the input prompts. • Crafting effective prompts can be challenging and time-consuming, often requiring trial-and-error. • Understanding why a prompt works or doesn't work is crucial for improvement.
3. Solution: "Sequence Salience with Language Interpretability Tool (LIT)"
4. Input Salience Methods: • Gradient Based: These methods analyze how small changes to the input affect the model's output. • Attention Scores: Attention scores can be interpreted as a measure of how much the model is "paying attention" to each word or phrase.
5. Tutorial Notebook: https://ai.google.dev/ge mma/docs/lit_gemma
6. Prompt Debugging with Sequence Salience using Language Interpretability Tool (LIT).
8. Model Loading ! pip install -q -U "tensorflow==2.17" "keras >= 3.0, <4.0" "keras-nlp >= 0.14" "lit-nlp >= 1.2" lit_widget = lit_pdbnb.make_notebook_widget( ['sample_prompts'], ["gemma_2b_it:gemma_1.1_instruct _2b_en"], dl_framework="kerasnlp", dl_runtime="tensorflow", batch_size=1, max_examples=5, precision="bfloat16", ) lit_widget.render() Parameters: - datasets_config - models_config - dl_framework - dl_runtime
9. Case Study 1: Debugging Few-Shot Prompts
10. Few-Shot Prompting
11. Demo Video
12. Case Study 2: Assessing Constitutional Principles in Prompts
13. Demo Video
14. Resources: • https://colab.research.google.com/github/google/gener ative-ai-docs/blob/main/site/en/gemma/docs/lit_gemm a.ipynb#scrollTo=yKw8gDsh_nVR • https://pair-code.github.io/lit/tutorials/sequence-sali ence/ • https://arxiv.org/pdf/2404.07498 • https://pair-code.github.io/lit/ documentation/components.html #sequence-salience
15. Q/A