$30 off During Our Annual Pro Sale. View Details »

Build with AI 2024 Seoul - 제로부터 시작하는 ...

Build with AI 2024 Seoul - 제로부터 시작하는 Flutter with Gemini 생활 - 박제창

Build with AI 2024 Seoul - 제로부터 시작하는 Flutter with Gemini 생활 - 박제창

https://festa.io/events/4777

JaiChangPark

March 31, 2024
Tweet

More Decks by JaiChangPark

Other Decks in Programming

Transcript

  1. 1. LLM (Large Language Model) 2. Flutter with Gemini (Google

    GenAI) 3. Google GenAI Gemma 4. Live Demo Seoul 3
  2. What’s Large Language Model • LLM은 Large Language Model의 약자로,

    거대 언어 모델이라고도 불림 • LLM은 인공 신경망 기반의 언어 모델이며, 방대한 양의 텍스트 데이터로 훈련됨 • 활용 예시 ◦ 챗봇: 질문에 답하고 대화를 나눌 수 있음 ◦ 텍스트 요약: 긴 문서를 짧게 요약 가능 함 ◦ 번역: 한 언어를 다른 언어로 번역할 수 있음 ◦ 작문: 창의적인 글쓰기, 코드 작성 등을 수행할 수 있음 6
  3. What’s Large Language Model 7 구분 LLM NLP 정의 방대한

    텍스트 데이터를 학습하여 인간 수준의 언어 능력을 갖춘 모델 인공지능이 인간의 언어를 이해하고 처리하는 기술 특징 - 방대한 텍스트 데이터 학습 - 다양한 NLP 작업 수행 - 인간 수준의 언어 능력 - 챗봇, 기계번역, 감정 분석 등 다양한 분야 활용 - 다양한 기술 사용 관계 NLP 기술의 하위 기술 LLM은 NLP 기술 발전에 중요한 역할
  4. Attention Is All You Need @Srouce: Attention Is All You

    Need, 12 Jun 2017 (v1), last revised 2 Aug 2023 (this version, v7)] 8
  5. The evolutionary tree of modern LLMs source: Harnessing the Power

    of LLMs in Practice: A Survey on ChatGPT and Beyond The evolutionary tree of modern LLMs traces the development of language models in recent years and highlights some of the most well-known models. Models on the same branch have closer relationships. Transformer-based models are shown in non-grey colors: decoder-only models in the blue branch, encoder-only models in the pink branch, and encoder-decoder models in the green branch. The vertical position of the models on the timeline represents their release dates. Open-source models are represented by solid squares, while closed-source models are represented by hollow ones. The stacked bar plot in the bottom right corner shows the number of models from various companies and institutions. 10
  6. 11

  7. 13 We use this cluster design for Llama 3 training.

    Today, we’re sharing details on two versions of our 24,576-GPU data center scale cluster at Meta. we’re aiming to continue to grow our infrastructure build-out that will include 350,000 NVIDIA H100 GPUs as part of a portfolio that will feature compute power equivalent to nearly 600,000 H100s. https://engineering.fb.com/2024/03/12/data-center-engineering/building-metas-genai-infrastructure/
  8. 14 Today, we’re sharing details on two versions of our

    24,576-GPU data center scale cluster at Meta. 24576 * 60000000 = 1,474,560,000,000 (약 1조 4천억 원) we’re aiming to continue to grow our infrastructure build-out that will include 350,000 NVIDIA H100 GPUs as part of a portfolio that will feature compute power equivalent to nearly 600,000 H100s. 21,000,000,000,000원 (약 21조 원)
  9. 19

  10. 20 Gemini | Google GenAI 1. LaMDA:(Language Models for Dialog

    Applications) 137B https://blog.research.google/2022/01/lamda-towards-safe-grounded-and-high.html 2. Bard 137B : LaMDA 기반 3. PaLM(2022), PaLM2 (2023) - 540B https://blog.research.google/2022/04/pathways-language-model-palm-scaling-to.html https://blog.google/technology/ai/google-palm-2-ai-large-language-model/ 4. Gemini 1.0 nano, pro, ultra (2023.12) https://arxiv.org/pdf/2312.11805.pdf 5. Gemini 1.5 (2024.02) https://storage.googleapis.com/deepmind-media/gemini/gemini_v1_5_report.pdf
  11. 22 Gemini 1.5 | Google GenAI • Gemini 1.5 Pro

    is built to handle extremely long contexts; it has the ability to recall and reason over fine-grained information from up to at least 10M (10,000,000) tokens. • Gemini 1.5 Pro surpasses Gemini 1.0 Pro and performs at a similar level to 1.0 Ultra on a wide array of benchmarks while requiring significantly less compute to train.
  12. 30 Flutter | Multi-Platform • Flutter is an open source

    framework by Google for building beautiful, natively compiled, multi-platform applications from a single codebase.
  13. 32 Flutter | Gemini • AI integration • Gemini Google

    AI Dart SDK beta release • The Google AI Dart SDK has been released to beta. This enables you to build generative AI features into your Dart or Flutter app, powered by Gemini, Google’s latest family of AI models. There is now a google_ generative_ai package on pub.dev. • Learn more about how to build with the Google AI Dart SDK in this blog post or jump straight into the Dart quickstart.
  14. 37 Flutter | Gemini Generate text from text-only input import

    'package:google_generative_ai/google_generative_ai.dart'; void main() async { final apiKey = Platform.environment['API_KEY']; if (apiKey == null) { print('No \$API_KEY environment variable'); exit(1); } final model = GenerativeModel(model: 'gemini-pro', apiKey: apiKey); final content = [Content.text('딥러닝과 머신러닝의 차이를 알려주세요.')]; final response = await model.generateContent(content); print(response.text); }
  15. 38 Flutter | Gemini Generate text from text-only input import

    'package:google_generative_ai/google_generative_ai.dart'; void main() async { final apiKey = Platform.environment['API_KEY']; if (apiKey == null) { print('No \$API_KEY environment variable'); exit(1); } final model = GenerativeModel(model: 'gemini-pro', apiKey: apiKey); final content = [Content.text('딥러닝과 머신러닝의 차이를 알려주세요.')]; final response = await model.generateContent(content); print(response.text); }
  16. 39 Flutter | Gemini Generate text from text-only input import

    'package:google_generative_ai/google_generative_ai.dart'; void main() async { final apiKey = Platform.environment['API_KEY']; if (apiKey == null) { print('No \$API_KEY environment variable'); exit(1); } final model = GenerativeModel(model: 'gemini-pro', apiKey: apiKey); final content = [Content.text('딥러닝과 머신러닝의 차이를 알려주세요.')]; final response = await model.generateContent(content); print(response.text); }
  17. 41 Flutter | Gemini https://aistudio.google.com/app/apikey Generate text from text-only input

    Variation Attribute Description Gemini 1.5 Pro (Preview only) Model last updated February 2024 Model code models/gemini-1.5-pro Model capabilities • Input: text and image • Output: text • Optimized for language tasks such as: • Code generation • Text generation • Text editing • Problem solving • Recommendations generation • Information extraction • Data extraction or generation • AI agent • Can handle zero, one, and few-shot tasks. Supported generation methods generateContent Input token limit 1,048,576 Output token limit 8192 Model safety Automatically applied safety settings which are adjustable by developers. See the safety settings topic for details. Rate limit 1 query per minute, 50 queries per day [1]
  18. 42 Flutter | Gemini Generate text from text-only input import

    'package:google_generative_ai/google_generative_ai.dart'; void main() async { final apiKey = Platform.environment['API_KEY']; if (apiKey == null) { print('No \$API_KEY environment variable'); exit(1); } final model = GenerativeModel(model: 'gemini-pro', apiKey: apiKey); final content = [Content.text('딥러닝과 머신러닝의 차이를 알려주세요.')]; final response = await model.generateContent(content); print(response.text); }
  19. 43 Flutter | Gemini final class Content { final String?

    role; final List<Part> parts; Content(this.role, this.parts); static Content text(String text) => Content('user', [TextPart(text)]); static Content data(String mimeType, Uint8List bytes) => Content('user', [DataPart(mimeType, bytes)]); static Content multi(Iterable<Part> parts) => Content('user', [...parts]); static Content model(Iterable<Part> parts) => Content('model', [...parts]); Map<String, Object?> toJson() => { if (role case final role?) 'role': role, 'parts': parts.map((p) => p.toJson()).toList() }; }
  20. 44 Flutter | Gemini final class GenerateContentResponse { final List<Candidate>

    candidates; final PromptFeedback? promptFeedback; GenerateContentResponse(this.candidates, this.promptFeedback);
  21. 45 Flutter | Gemini final class GenerateContentResponse { String? get

    text { return switch (candidates) { [] => switch (promptFeedback) { PromptFeedback(:final blockReason, :final blockReasonMessage, ) => /// throw _ => null, }, [
  22. 47 Flutter | Gemini void main() async { // For

    text-and-image input (multimodal), use the gemini-pro-vision model final model = GenerativeModel(model: 'gemini-pro-vision', apiKey: apiKey); final (firstImage, secondImage) = await ( File('image0.jpg').readAsBytes(), File('image1.jpg').readAsBytes() ).wait; final prompt = TextPart("What's different between these pictures?"); final imageParts = [ DataPart('image/jpeg', firstImage), DataPart('image/jpeg', secondImage), ]; final response = await model.generateContent([Content.multi([prompt, ...imageParts])]); print(response.text); }
  23. 48 Flutter | Gemini void main() async { // For

    text-and-image input (multimodal), use the gemini-pro-vision model final model = GenerativeModel(model: 'gemini-pro-vision', apiKey: apiKey); final (firstImage, secondImage) = await ( File('image0.jpg').readAsBytes(), File('image1.jpg').readAsBytes() ).wait; final prompt = TextPart("What's different between these pictures?"); final imageParts = [ DataPart('image/jpeg', firstImage), DataPart('image/jpeg', secondImage), ]; final response = await model.generateContent([Content.multi([prompt, ...imageParts])]); print(response.text); }
  24. 49 Flutter | Gemini void main() async { // For

    text-and-image input (multimodal), use the gemini-pro-vision model final model = GenerativeModel(model: 'gemini-pro-vision', apiKey: apiKey); final (firstImage, secondImage) = await ( File('image0.jpg').readAsBytes(), File('image1.jpg').readAsBytes() ).wait; final prompt = TextPart("What's different between these pictures?"); final imageParts = [ DataPart('image/jpeg', firstImage), DataPart('image/jpeg', secondImage), ]; final response = await model.generateContent([Content.multi([prompt, ...imageParts])]); print(response.text); }
  25. 51 Flutter | Gemini // For text-only input final tokenCount

    = await model.countTokens(Content.text(prompt)); print('Token count: ${tokenCount.totalTokens}');
  26. 52 Flutter | Gemini // For text-and-image input (multimodal) final

    tokenCount = await model.countTokens([ Content.multi([prompt, ...imageParts]) ]); print('Token count: ${tokenCount.totalTokens}');
  27. 53 Flutter | Gemini // For multi-turn conversations (like chat)

    final prompt = Content.text(message); final allContent = [...chat.history, prompt]; final tokenCount = await model.countTokens(allContent); print('Token count: ${tokenCount.totalTokens}');
  28. 56 Gemma | Open Source | Google GenAI • A

    family of lightweight, state-of-the art open models built from the same research and technology used to create the Gemini models @source: https://storage.googleapis.com/deepmind-media/gemma/gemma-report.pdf
  29. 57 Gemma | Open Source | Google GenAI • 에뮬레이터

    상에서 속도는 생각보다 느리게 처리됨 (cpu-2b-it-4bit) • Gpu model은 에러가..발생 • 실 디바이스에서 실행시 다를 수 있음.