Slide 1

Slide 1 text

제로부터 시작하는 Flutter with Gemini 생활 Seoul 1

Slide 2

Slide 2 text

박제창 Flutter Seoul GDG Golang Korea Seoul 2

Slide 3

Slide 3 text

1. LLM (Large Language Model) 2. Flutter with Gemini (Google GenAI) 3. Google GenAI Gemma 4. Live Demo Seoul 3

Slide 4

Slide 4 text

LLM Large Language Model Seoul 4

Slide 5

Slide 5 text

Artificial Intelligence @source: https://blogs.nvidia.com/blog/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/ 5

Slide 6

Slide 6 text

What’s Large Language Model ● LLM은 Large Language Model의 약자로, 거대 언어 모델이라고도 불림 ● LLM은 인공 신경망 기반의 언어 모델이며, 방대한 양의 텍스트 데이터로 훈련됨 ● 활용 예시 ○ 챗봇: 질문에 답하고 대화를 나눌 수 있음 ○ 텍스트 요약: 긴 문서를 짧게 요약 가능 함 ○ 번역: 한 언어를 다른 언어로 번역할 수 있음 ○ 작문: 창의적인 글쓰기, 코드 작성 등을 수행할 수 있음 6

Slide 7

Slide 7 text

What’s Large Language Model 7 구분 LLM NLP 정의 방대한 텍스트 데이터를 학습하여 인간 수준의 언어 능력을 갖춘 모델 인공지능이 인간의 언어를 이해하고 처리하는 기술 특징 - 방대한 텍스트 데이터 학습 - 다양한 NLP 작업 수행 - 인간 수준의 언어 능력 - 챗봇, 기계번역, 감정 분석 등 다양한 분야 활용 - 다양한 기술 사용 관계 NLP 기술의 하위 기술 LLM은 NLP 기술 발전에 중요한 역할

Slide 8

Slide 8 text

Attention Is All You Need @Srouce: Attention Is All You Need, 12 Jun 2017 (v1), last revised 2 Aug 2023 (this version, v7)] 8

Slide 9

Slide 9 text

Attention Is All You Need 9 https://patents.google.com/patent/US10452978B2/en

Slide 10

Slide 10 text

The evolutionary tree of modern LLMs source: Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond The evolutionary tree of modern LLMs traces the development of language models in recent years and highlights some of the most well-known models. Models on the same branch have closer relationships. Transformer-based models are shown in non-grey colors: decoder-only models in the blue branch, encoder-only models in the pink branch, and encoder-decoder models in the green branch. The vertical position of the models on the timeline represents their release dates. Open-source models are represented by solid squares, while closed-source models are represented by hollow ones. The stacked bar plot in the bottom right corner shows the number of models from various companies and institutions. 10

Slide 11

Slide 11 text

11

Slide 12

Slide 12 text

12 @source: https://a16z.com/100-gen-ai-apps/

Slide 13

Slide 13 text

13 We use this cluster design for Llama 3 training. Today, we’re sharing details on two versions of our 24,576-GPU data center scale cluster at Meta. we’re aiming to continue to grow our infrastructure build-out that will include 350,000 NVIDIA H100 GPUs as part of a portfolio that will feature compute power equivalent to nearly 600,000 H100s. https://engineering.fb.com/2024/03/12/data-center-engineering/building-metas-genai-infrastructure/

Slide 14

Slide 14 text

14 Today, we’re sharing details on two versions of our 24,576-GPU data center scale cluster at Meta. 24576 * 60000000 = 1,474,560,000,000 (약 1조 4천억 원) we’re aiming to continue to grow our infrastructure build-out that will include 350,000 NVIDIA H100 GPUs as part of a portfolio that will feature compute power equivalent to nearly 600,000 H100s. 21,000,000,000,000원 (약 21조 원)

Slide 15

Slide 15 text

HBM(High Bandwidth Memory) 15

Slide 16

Slide 16 text

Nvidia B200 16 https://www.youtube.com/live/Y2F8yisiS6E?feature=shared

Slide 17

Slide 17 text

17 https://uxlfoundation.org/

Slide 18

Slide 18 text

Google GenAI Gemini Seoul 18

Slide 19

Slide 19 text

19

Slide 20

Slide 20 text

20 Gemini | Google GenAI 1. LaMDA:(Language Models for Dialog Applications) 137B https://blog.research.google/2022/01/lamda-towards-safe-grounded-and-high.html 2. Bard 137B : LaMDA 기반 3. PaLM(2022), PaLM2 (2023) - 540B https://blog.research.google/2022/04/pathways-language-model-palm-scaling-to.html https://blog.google/technology/ai/google-palm-2-ai-large-language-model/ 4. Gemini 1.0 nano, pro, ultra (2023.12) https://arxiv.org/pdf/2312.11805.pdf 5. Gemini 1.5 (2024.02) https://storage.googleapis.com/deepmind-media/gemini/gemini_v1_5_report.pdf

Slide 21

Slide 21 text

21 Gemini | Google GenAI

Slide 22

Slide 22 text

22 Gemini 1.5 | Google GenAI ● Gemini 1.5 Pro is built to handle extremely long contexts; it has the ability to recall and reason over fine-grained information from up to at least 10M (10,000,000) tokens. ● Gemini 1.5 Pro surpasses Gemini 1.0 Pro and performs at a similar level to 1.0 Ultra on a wide array of benchmarks while requiring significantly less compute to train.

Slide 23

Slide 23 text

23 Gemini 1.5 | Google GenAI

Slide 24

Slide 24 text

24 Gemini 1.5 | Google GenAI

Slide 25

Slide 25 text

25 Gemini 1.5 | Google GenAI

Slide 26

Slide 26 text

26 Gemini 1.5 | Fine-Tuning | Google GenAI

Slide 27

Slide 27 text

27 Gemini 1.5 | Fine-Tuning | Google GenAI

Slide 28

Slide 28 text

https://youtu.be/v5tRc_5-8G4?feature=shared Personalized AI for you | Gemini 28

Slide 29

Slide 29 text

Flutter with Gemini (Google GenAI) Seoul 29

Slide 30

Slide 30 text

30 Flutter | Multi-Platform ● Flutter is an open source framework by Google for building beautiful, natively compiled, multi-platform applications from a single codebase.

Slide 31

Slide 31 text

31 Flutter | Multi-Platform

Slide 32

Slide 32 text

32 Flutter | Gemini ● AI integration ● Gemini Google AI Dart SDK beta release ● The Google AI Dart SDK has been released to beta. This enables you to build generative AI features into your Dart or Flutter app, powered by Gemini, Google’s latest family of AI models. There is now a google_ generative_ai package on pub.dev. ● Learn more about how to build with the Google AI Dart SDK in this blog post or jump straight into the Dart quickstart.

Slide 33

Slide 33 text

33 Flutter | Gemini @source: https://pub.dev/packages/google_generative_ai

Slide 34

Slide 34 text

34 Flutter | Gemini @source: https://ai.google.dev/tutorials/dart_quickstart

Slide 35

Slide 35 text

35 Flutter | Gemini https://aistudio.google.com/app/apikey

Slide 36

Slide 36 text

Generate text from text-only input 36

Slide 37

Slide 37 text

37 Flutter | Gemini Generate text from text-only input import 'package:google_generative_ai/google_generative_ai.dart'; void main() async { final apiKey = Platform.environment['API_KEY']; if (apiKey == null) { print('No \$API_KEY environment variable'); exit(1); } final model = GenerativeModel(model: 'gemini-pro', apiKey: apiKey); final content = [Content.text('딥러닝과 머신러닝의 차이를 알려주세요.')]; final response = await model.generateContent(content); print(response.text); }

Slide 38

Slide 38 text

38 Flutter | Gemini Generate text from text-only input import 'package:google_generative_ai/google_generative_ai.dart'; void main() async { final apiKey = Platform.environment['API_KEY']; if (apiKey == null) { print('No \$API_KEY environment variable'); exit(1); } final model = GenerativeModel(model: 'gemini-pro', apiKey: apiKey); final content = [Content.text('딥러닝과 머신러닝의 차이를 알려주세요.')]; final response = await model.generateContent(content); print(response.text); }

Slide 39

Slide 39 text

39 Flutter | Gemini Generate text from text-only input import 'package:google_generative_ai/google_generative_ai.dart'; void main() async { final apiKey = Platform.environment['API_KEY']; if (apiKey == null) { print('No \$API_KEY environment variable'); exit(1); } final model = GenerativeModel(model: 'gemini-pro', apiKey: apiKey); final content = [Content.text('딥러닝과 머신러닝의 차이를 알려주세요.')]; final response = await model.generateContent(content); print(response.text); }

Slide 40

Slide 40 text

40 Flutter | Gemini Generate text from text-only input

Slide 41

Slide 41 text

41 Flutter | Gemini https://aistudio.google.com/app/apikey Generate text from text-only input Variation Attribute Description Gemini 1.5 Pro (Preview only) Model last updated February 2024 Model code models/gemini-1.5-pro Model capabilities ● Input: text and image ● Output: text ● Optimized for language tasks such as: ● Code generation ● Text generation ● Text editing ● Problem solving ● Recommendations generation ● Information extraction ● Data extraction or generation ● AI agent ● Can handle zero, one, and few-shot tasks. Supported generation methods generateContent Input token limit 1,048,576 Output token limit 8192 Model safety Automatically applied safety settings which are adjustable by developers. See the safety settings topic for details. Rate limit 1 query per minute, 50 queries per day [1]

Slide 42

Slide 42 text

42 Flutter | Gemini Generate text from text-only input import 'package:google_generative_ai/google_generative_ai.dart'; void main() async { final apiKey = Platform.environment['API_KEY']; if (apiKey == null) { print('No \$API_KEY environment variable'); exit(1); } final model = GenerativeModel(model: 'gemini-pro', apiKey: apiKey); final content = [Content.text('딥러닝과 머신러닝의 차이를 알려주세요.')]; final response = await model.generateContent(content); print(response.text); }

Slide 43

Slide 43 text

43 Flutter | Gemini final class Content { final String? role; final List parts; Content(this.role, this.parts); static Content text(String text) => Content('user', [TextPart(text)]); static Content data(String mimeType, Uint8List bytes) => Content('user', [DataPart(mimeType, bytes)]); static Content multi(Iterable parts) => Content('user', [...parts]); static Content model(Iterable parts) => Content('model', [...parts]); Map toJson() => { if (role case final role?) 'role': role, 'parts': parts.map((p) => p.toJson()).toList() }; }

Slide 44

Slide 44 text

44 Flutter | Gemini final class GenerateContentResponse { final List candidates; final PromptFeedback? promptFeedback; GenerateContentResponse(this.candidates, this.promptFeedback);

Slide 45

Slide 45 text

45 Flutter | Gemini final class GenerateContentResponse { String? get text { return switch (candidates) { [] => switch (promptFeedback) { PromptFeedback(:final blockReason, :final blockReasonMessage, ) => /// throw _ => null, }, [

Slide 46

Slide 46 text

Generate text from text-and-image input (multi-modal) 46

Slide 47

Slide 47 text

47 Flutter | Gemini void main() async { // For text-and-image input (multimodal), use the gemini-pro-vision model final model = GenerativeModel(model: 'gemini-pro-vision', apiKey: apiKey); final (firstImage, secondImage) = await ( File('image0.jpg').readAsBytes(), File('image1.jpg').readAsBytes() ).wait; final prompt = TextPart("What's different between these pictures?"); final imageParts = [ DataPart('image/jpeg', firstImage), DataPart('image/jpeg', secondImage), ]; final response = await model.generateContent([Content.multi([prompt, ...imageParts])]); print(response.text); }

Slide 48

Slide 48 text

48 Flutter | Gemini void main() async { // For text-and-image input (multimodal), use the gemini-pro-vision model final model = GenerativeModel(model: 'gemini-pro-vision', apiKey: apiKey); final (firstImage, secondImage) = await ( File('image0.jpg').readAsBytes(), File('image1.jpg').readAsBytes() ).wait; final prompt = TextPart("What's different between these pictures?"); final imageParts = [ DataPart('image/jpeg', firstImage), DataPart('image/jpeg', secondImage), ]; final response = await model.generateContent([Content.multi([prompt, ...imageParts])]); print(response.text); }

Slide 49

Slide 49 text

49 Flutter | Gemini void main() async { // For text-and-image input (multimodal), use the gemini-pro-vision model final model = GenerativeModel(model: 'gemini-pro-vision', apiKey: apiKey); final (firstImage, secondImage) = await ( File('image0.jpg').readAsBytes(), File('image1.jpg').readAsBytes() ).wait; final prompt = TextPart("What's different between these pictures?"); final imageParts = [ DataPart('image/jpeg', firstImage), DataPart('image/jpeg', secondImage), ]; final response = await model.generateContent([Content.multi([prompt, ...imageParts])]); print(response.text); }

Slide 50

Slide 50 text

Count tokens 50

Slide 51

Slide 51 text

51 Flutter | Gemini // For text-only input final tokenCount = await model.countTokens(Content.text(prompt)); print('Token count: ${tokenCount.totalTokens}');

Slide 52

Slide 52 text

52 Flutter | Gemini // For text-and-image input (multimodal) final tokenCount = await model.countTokens([ Content.multi([prompt, ...imageParts]) ]); print('Token count: ${tokenCount.totalTokens}');

Slide 53

Slide 53 text

53 Flutter | Gemini // For multi-turn conversations (like chat) final prompt = Content.text(message); final allContent = [...chat.history, prompt]; final tokenCount = await model.countTokens(allContent); print('Token count: ${tokenCount.totalTokens}');

Slide 54

Slide 54 text

Google GenAI Gemma Seoul 54

Slide 55

Slide 55 text

55 Gemma | Open Source | Google GenAI https://ai.google.dev/gemma

Slide 56

Slide 56 text

56 Gemma | Open Source | Google GenAI ● A family of lightweight, state-of-the art open models built from the same research and technology used to create the Gemini models @source: https://storage.googleapis.com/deepmind-media/gemma/gemma-report.pdf

Slide 57

Slide 57 text

57 Gemma | Open Source | Google GenAI ● 에뮬레이터 상에서 속도는 생각보다 느리게 처리됨 (cpu-2b-it-4bit) ● Gpu model은 에러가..발생 ● 실 디바이스에서 실행시 다를 수 있음.

Slide 58

Slide 58 text

Live Demo Seoul 58

Slide 59

Slide 59 text

Thank you Seoul 59