제로부터 시작하는 Flutter with Gemini 생활 Seoul 1

박제창 Flutter Seoul GDG Golang Korea Seoul 2

1. LLM (Large Language Model) 2. Flutter with Gemini (Google GenAI) 3. Google GenAI Gemma 4. Live Demo Seoul 3

LLM Large Language Model Seoul 4

Artificial Intelligence @source: https://blogs.nvidia.com/blog/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/ 5

What’s Large Language Model ● LLM은 Large Language Model의 약자로, 거대 언어 모델이라고도 불림 ● LLM은 인공 신경망 기반의 언어 모델이며, 방대한 양의 텍스트 데이터로 훈련됨 ● 활용 예시 ○ 챗봇: 질문에 답하고 대화를 나눌 수 있음 ○ 텍스트 요약: 긴 문서를 짧게 요약 가능 함 ○ 번역: 한 언어를 다른 언어로 번역할 수 있음 ○ 작문: 창의적인 글쓰기, 코드 작성 등을 수행할 수 있음 6

What’s Large Language Model 7 구분 LLM NLP 정의 방대한 텍스트 데이터를 학습하여 인간 수준의 언어 능력을 갖춘 모델 인공지능이 인간의 언어를 이해하고 처리하는 기술 특징 - 방대한 텍스트 데이터 학습 - 다양한 NLP 작업 수행 - 인간 수준의 언어 능력 - 챗봇, 기계번역, 감정 분석 등 다양한 분야 활용 - 다양한 기술 사용 관계 NLP 기술의 하위 기술 LLM은 NLP 기술 발전에 중요한 역할

Attention Is All You Need @Srouce: Attention Is All You Need, 12 Jun 2017 (v1), last revised 2 Aug 2023 (this version, v7)] 8

Attention Is All You Need 9 https://patents.google.com/patent/US10452978B2/en

The evolutionary tree of modern LLMs source: Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond The evolutionary tree of modern LLMs traces the development of language models in recent years and highlights some of the most well-known models. Models on the same branch have closer relationships. Transformer-based models are shown in non-grey colors: decoder-only models in the blue branch, encoder-only models in the pink branch, and encoder-decoder models in the green branch. The vertical position of the models on the timeline represents their release dates. Open-source models are represented by solid squares, while closed-source models are represented by hollow ones. The stacked bar plot in the bottom right corner shows the number of models from various companies and institutions. 10

11

12 @source: https://a16z.com/100-gen-ai-apps/

13 We use this cluster design for Llama 3 training. Today, we’re sharing details on two versions of our 24,576-GPU data center scale cluster at Meta. we’re aiming to continue to grow our infrastructure build-out that will include 350,000 NVIDIA H100 GPUs as part of a portfolio that will feature compute power equivalent to nearly 600,000 H100s. https://engineering.fb.com/2024/03/12/data-center-engineering/building-metas-genai-infrastructure/

14 Today, we’re sharing details on two versions of our 24,576-GPU data center scale cluster at Meta. 24576 * 60000000 = 1,474,560,000,000 (약 1조 4천억 원) we’re aiming to continue to grow our infrastructure build-out that will include 350,000 NVIDIA H100 GPUs as part of a portfolio that will feature compute power equivalent to nearly 600,000 H100s. 21,000,000,000,000원 (약 21조 원)

HBM(High Bandwidth Memory) 15

Nvidia B200 16 https://www.youtube.com/live/Y2F8yisiS6E?feature=shared

17 https://uxlfoundation.org/

Google GenAI Gemini Seoul 18

19

20 Gemini | Google GenAI 1. LaMDA:(Language Models for Dialog Applications) 137B https://blog.research.google/2022/01/lamda-towards-safe-grounded-and-high.html 2. Bard 137B : LaMDA 기반 3. PaLM(2022), PaLM2 (2023) - 540B https://blog.research.google/2022/04/pathways-language-model-palm-scaling-to.html https://blog.google/technology/ai/google-palm-2-ai-large-language-model/ 4. Gemini 1.0 nano, pro, ultra (2023.12) https://arxiv.org/pdf/2312.11805.pdf 5. Gemini 1.5 (2024.02) https://storage.googleapis.com/deepmind-media/gemini/gemini_v1_5_report.pdf

21 Gemini | Google GenAI

22 Gemini 1.5 | Google GenAI ● Gemini 1.5 Pro is built to handle extremely long contexts; it has the ability to recall and reason over fine-grained information from up to at least 10M (10,000,000) tokens. ● Gemini 1.5 Pro surpasses Gemini 1.0 Pro and performs at a similar level to 1.0 Ultra on a wide array of benchmarks while requiring significantly less compute to train.

23 Gemini 1.5 | Google GenAI

24 Gemini 1.5 | Google GenAI

25 Gemini 1.5 | Google GenAI

26 Gemini 1.5 | Fine-Tuning | Google GenAI

27 Gemini 1.5 | Fine-Tuning | Google GenAI

https://youtu.be/v5tRc_5-8G4?feature=shared Personalized AI for you | Gemini 28

Flutter with Gemini (Google GenAI) Seoul 29

30 Flutter | Multi-Platform ● Flutter is an open source framework by Google for building beautiful, natively compiled, multi-platform applications from a single codebase.

31 Flutter | Multi-Platform

32 Flutter | Gemini ● AI integration ● Gemini Google AI Dart SDK beta release ● The Google AI Dart SDK has been released to beta. This enables you to build generative AI features into your Dart or Flutter app, powered by Gemini, Google’s latest family of AI models. There is now a google_ generative_ai package on pub.dev. ● Learn more about how to build with the Google AI Dart SDK in this blog post or jump straight into the Dart quickstart.

33 Flutter | Gemini @source: https://pub.dev/packages/google_generative_ai

34 Flutter | Gemini @source: https://ai.google.dev/tutorials/dart_quickstart

35 Flutter | Gemini https://aistudio.google.com/app/apikey

Generate text from text-only input 36

37 Flutter | Gemini Generate text from text-only input import 'package:google_generative_ai/google_generative_ai.dart'; void main() async { final apiKey = Platform.environment['API_KEY']; if (apiKey == null) { print('No \$API_KEY environment variable'); exit(1); } final model = GenerativeModel(model: 'gemini-pro', apiKey: apiKey); final content = [Content.text('딥러닝과 머신러닝의 차이를 알려주세요.')]; final response = await model.generateContent(content); print(response.text); }

38 Flutter | Gemini Generate text from text-only input import 'package:google_generative_ai/google_generative_ai.dart'; void main() async { final apiKey = Platform.environment['API_KEY']; if (apiKey == null) { print('No \$API_KEY environment variable'); exit(1); } final model = GenerativeModel(model: 'gemini-pro', apiKey: apiKey); final content = [Content.text('딥러닝과 머신러닝의 차이를 알려주세요.')]; final response = await model.generateContent(content); print(response.text); }

39 Flutter | Gemini Generate text from text-only input import 'package:google_generative_ai/google_generative_ai.dart'; void main() async { final apiKey = Platform.environment['API_KEY']; if (apiKey == null) { print('No \$API_KEY environment variable'); exit(1); } final model = GenerativeModel(model: 'gemini-pro', apiKey: apiKey); final content = [Content.text('딥러닝과 머신러닝의 차이를 알려주세요.')]; final response = await model.generateContent(content); print(response.text); }

40 Flutter | Gemini Generate text from text-only input

41 Flutter | Gemini https://aistudio.google.com/app/apikey Generate text from text-only input Variation Attribute Description Gemini 1.5 Pro (Preview only) Model last updated February 2024 Model code models/gemini-1.5-pro Model capabilities ● Input: text and image ● Output: text ● Optimized for language tasks such as: ● Code generation ● Text generation ● Text editing ● Problem solving ● Recommendations generation ● Information extraction ● Data extraction or generation ● AI agent ● Can handle zero, one, and few-shot tasks. Supported generation methods generateContent Input token limit 1,048,576 Output token limit 8192 Model safety Automatically applied safety settings which are adjustable by developers. See the safety settings topic for details. Rate limit 1 query per minute, 50 queries per day [1]

42 Flutter | Gemini Generate text from text-only input import 'package:google_generative_ai/google_generative_ai.dart'; void main() async { final apiKey = Platform.environment['API_KEY']; if (apiKey == null) { print('No \$API_KEY environment variable'); exit(1); } final model = GenerativeModel(model: 'gemini-pro', apiKey: apiKey); final content = [Content.text('딥러닝과 머신러닝의 차이를 알려주세요.')]; final response = await model.generateContent(content); print(response.text); }

43 Flutter | Gemini final class Content { final String? role; final List parts; Content(this.role, this.parts); static Content text(String text) => Content('user', [TextPart(text)]); static Content data(String mimeType, Uint8List bytes) => Content('user', [DataPart(mimeType, bytes)]); static Content multi(Iterable parts) => Content('user', [...parts]); static Content model(Iterable parts) => Content('model', [...parts]); Map toJson() => { if (role case final role?) 'role': role, 'parts': parts.map((p) => p.toJson()).toList() }; }

44 Flutter | Gemini final class GenerateContentResponse { final List candidates; final PromptFeedback? promptFeedback; GenerateContentResponse(this.candidates, this.promptFeedback);

45 Flutter | Gemini final class GenerateContentResponse { String? get text { return switch (candidates) { [] => switch (promptFeedback) { PromptFeedback(:final blockReason, :final blockReasonMessage, ) => /// throw _ => null, }, [

Generate text from text-and-image input (multi-modal) 46

47 Flutter | Gemini void main() async { // For text-and-image input (multimodal), use the gemini-pro-vision model final model = GenerativeModel(model: 'gemini-pro-vision', apiKey: apiKey); final (firstImage, secondImage) = await ( File('image0.jpg').readAsBytes(), File('image1.jpg').readAsBytes() ).wait; final prompt = TextPart("What's different between these pictures?"); final imageParts = [ DataPart('image/jpeg', firstImage), DataPart('image/jpeg', secondImage), ]; final response = await model.generateContent([Content.multi([prompt, ...imageParts])]); print(response.text); }

48 Flutter | Gemini void main() async { // For text-and-image input (multimodal), use the gemini-pro-vision model final model = GenerativeModel(model: 'gemini-pro-vision', apiKey: apiKey); final (firstImage, secondImage) = await ( File('image0.jpg').readAsBytes(), File('image1.jpg').readAsBytes() ).wait; final prompt = TextPart("What's different between these pictures?"); final imageParts = [ DataPart('image/jpeg', firstImage), DataPart('image/jpeg', secondImage), ]; final response = await model.generateContent([Content.multi([prompt, ...imageParts])]); print(response.text); }

49 Flutter | Gemini void main() async { // For text-and-image input (multimodal), use the gemini-pro-vision model final model = GenerativeModel(model: 'gemini-pro-vision', apiKey: apiKey); final (firstImage, secondImage) = await ( File('image0.jpg').readAsBytes(), File('image1.jpg').readAsBytes() ).wait; final prompt = TextPart("What's different between these pictures?"); final imageParts = [ DataPart('image/jpeg', firstImage), DataPart('image/jpeg', secondImage), ]; final response = await model.generateContent([Content.multi([prompt, ...imageParts])]); print(response.text); }

Count tokens 50

51 Flutter | Gemini // For text-only input final tokenCount = await model.countTokens(Content.text(prompt)); print('Token count: ${tokenCount.totalTokens}');

52 Flutter | Gemini // For text-and-image input (multimodal) final tokenCount = await model.countTokens([ Content.multi([prompt, ...imageParts]) ]); print('Token count: ${tokenCount.totalTokens}');

53 Flutter | Gemini // For multi-turn conversations (like chat) final prompt = Content.text(message); final allContent = [...chat.history, prompt]; final tokenCount = await model.countTokens(allContent); print('Token count: ${tokenCount.totalTokens}');

Google GenAI Gemma Seoul 54

55 Gemma | Open Source | Google GenAI https://ai.google.dev/gemma

56 Gemma | Open Source | Google GenAI ● A family of lightweight, state-of-the art open models built from the same research and technology used to create the Gemini models @source: https://storage.googleapis.com/deepmind-media/gemma/gemma-report.pdf

57 Gemma | Open Source | Google GenAI ● 에뮬레이터 상에서 속도는 생각보다 느리게 처리됨 (cpu-2b-it-4bit) ● Gpu model은 에러가..발생 ● 실 디바이스에서 실행시 다를 수 있음.

Live Demo Seoul 58

Thank you Seoul 59