Slide 1

Slide 1 text

Caesar Chi 2024/03 從 Gemini API 到 Line Bot Build with AI 2024 Taichung https://gdg.community.dev/events/details/google-gdg-taichung-presents-build-with-ai-2024-taichung-3/

Slide 2

Slide 2 text

http://exma-square.co/

Slide 3

Slide 3 text

Caesar Chi clonncd ⼯程師成長⼿冊 clonn

Slide 4

Slide 4 text

個⼈經歷 • Career • 2024 - EXMA-Square • 2021 - TransIot — CTO • 2020 - Undercover - Tech infra • 2018 - Awoo Tech Manager • 2017 - EXMA-Square • 2016 - Hiiir Tech Manager • 2014 - Mitac full-stack developer • 2012 - Dlink Front end developer • Community • JSDC core-team Caesar

Slide 5

Slide 5 text

Why Gemini

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

https://cloud.google.com/generative-ai-studio?hl=zh-tw

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

https://aistudio.google.com/app/prompts/new_chat?hl=zh-tw

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

API 版本說明 - 多模態

Slide 14

Slide 14 text

Gemini 在 Node.js const { GoogleGenerativeAI } = require("@google/generative-ai"); // Access your API key as an environment variable (see "Set up your API key" above) const genAI = new GoogleGenerativeAI(process.env.API_KEY);

Slide 15

Slide 15 text

const { GoogleGenerativeAI } = require("@google/generative-ai"); // Access your API key as an environment variable (see "Set up your API key" above) const genAI = new GoogleGenerativeAI(process.env.API_KEY); async function run() { // For text-only input, use the gemini-pro model const model = genAI.getGenerativeModel({ model: "gemini-pro"}); const prompt = "Write a story about a magic backpack." const result = await model.generateContent(prompt); const response = await result.response; const text = response.text(); console.log(text); } run();

Slide 16

Slide 16 text

Gemini 多模態

Slide 17

Slide 17 text

// Converts local file information to a GoogleGenerativeAI.Part object. function fileToGenerativePart(path, mimeType) { return { inlineData: { data: Buffer.from(fs.readFileSync(path)).toString("base64"), mimeType }, }; } async function run() { // For text-and-image input (multimodal), use the gemini-pro-vision model const model = genAI.getGenerativeModel({ model: "gemini-pro-vision" }); const prompt = "What's different between these pictures?"; const imageParts = [ fileToGenerativePart("image1.png", "image/png"), fileToGenerativePart("image2.jpeg", "image/jpeg"), ]; const result = await model.generateContent([prompt, ...imageParts]); const response = await result.response; const text = response.text(); console.log(text);

Slide 18

Slide 18 text

// Converts local file information to a GoogleGenerativeAI.Part object. function fileToGenerativePart(path, mimeType) { return { inlineData: { data: Buffer.from(fs.readFileSync(path)).toString("base64"), mimeType }, }; } async function run() { // For text-and-image input (multimodal), use the gemini-pro-vision model const model = genAI.getGenerativeModel({ model: "gemini-pro-vision" }); const prompt = "What's different between these pictures?"; const imageParts = [ fileToGenerativePart("image1.png", "image/png"), fileToGenerativePart("image2.jpeg", "image/jpeg"), ]; const result = await model.generateContent([prompt, ...imageParts]); const response = await result.response; const text = response.text(); console.log(text);

Slide 19

Slide 19 text

async function run() { // For text-only input, use the gemini-pro model const model = genAI.getGenerativeModel({ model: "gemini-pro"}); const chat = model.startChat({ history: [ { role: "user", parts: [{ text: "Hello, I have 2 dogs in my house." }], }, { role: "model", parts: [{ text: "Great to meet you. What would you like to know?" }], }, ], generationConfig: { maxOutputTokens: 100, }, }); const msg = "How many paws are in my house?"; const result = await chat.sendMessage(msg); const response = await result.response; const text = response.text();

Slide 20

Slide 20 text

//... const result = await model.generateContentStream([prompt, ...imageParts]); let text = ''; for await (const chunk of result.stream) { const chunkText = chunk.text(); console.log(chunkText); text += chunkText; } //...

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

安全性篩選器

Slide 23

Slide 23 text

https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/con fi gure-safety-attributes

Slide 24

Slide 24 text

https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/con fi gure-safety-attributes

Slide 25

Slide 25 text

// Initialize Vertex with your Cloud project and location const vertexAI = new VertexAI({project: projectId, location: location}); // Instantiate the model const generativeModel = vertexAI.getGenerativeModel({ model: model, // The following parameters are optional // They can also be passed to individual content generation requests safety_settings: [ { category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT, threshold: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE, }, ], generation_config: {max_output_tokens: 256}, }); const request = { contents: [{role: 'user', parts: [{text: 'Tell me something dangerous.'}]}], };

Slide 26

Slide 26 text

Why LINE Bot

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

https://today.line.me/tw/v2/article/Qw3PmQe

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

How to build it?

Slide 32

Slide 32 text

https://ai.google.dev/docs?hl=zh-tw

Slide 33

Slide 33 text

LINE BOT 模式設定 • 純⽂字模式 • 多重聊天內容模式 • 圖片 / 影片模式

Slide 34

Slide 34 text

https://aistudio.google.com/app/prompts/new_chat?hl=zh-tw

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

Node.js

Slide 39

Slide 39 text

Node.js

Slide 40

Slide 40 text

純⽂字模式 if (event.message.type === "text") { const msg = await gemini.textOnly(event.message.text); await line.reply(event.replyToken, [{ type: "text", text: msg }]); return res.end(); }

Slide 41

Slide 41 text

https://techblog.lycorp.co.jp/zh-hant/linebot-x-gemini- fi rebase

Slide 42

Slide 42 text

多模態輸入 if (event.message.type === "image") { const imageBinary = await line.getImageBinary(event.message.id); const msg = await gemini.multimodal(imageBinary); await line.reply(event.replyToken, [{ type: "text", text: msg }]); return res.end(); } JavaScript

Slide 43

Slide 43 text

https://techblog.lycorp.co.jp/zh-hant/linebot-x-gemini- fi rebase

Slide 44

Slide 44 text

聊天模式 if (event.message.type === "text") { const msg = await gemini.chat(event.message.text); await line.reply(event.replyToken, [{ type: "text", text: msg }]); return res.end(); }

Slide 45

Slide 45 text

https://techblog.lycorp.co.jp/zh-hant/linebot-x-gemini- fi rebase

Slide 46

Slide 46 text

Reference • https://github.com/kkdai/linebot-smart-namecard • https://github.com/babaohuang/GeminiProChat • https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web

Slide 47

Slide 47 text

Limitation?

Slide 48

Slide 48 text

https://ai.google.dev/models/gemini?hl=zh-tw

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

https://ai.google.dev/docs/migrate_to_cloud?hl=zh-tw

Slide 51

Slide 51 text

https://blog.jason.tools/2016/07/The-most-expensive-free.html

Slide 52

Slide 52 text

No Code Builder

Slide 53

Slide 53 text

技術投資有賺有賠 開發之前請詳細參閱 README

Slide 54

Slide 54 text

botsonic https://writesonic.com/botsonic

Slide 55

Slide 55 text

TAR

Slide 56

Slide 56 text

chatfuel

Slide 57

Slide 57 text

botcore

Slide 58

Slide 58 text

Think about …

Slide 59

Slide 59 text

No content

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

No content

Slide 62

Slide 62 text

No content

Slide 63

Slide 63 text

Node.js

Slide 64

Slide 64 text

No content

Slide 65

Slide 65 text

No content

Slide 66

Slide 66 text

Node.js

Slide 67

Slide 67 text

AI for Developer Taiwan https://www.facebook.com/groups/1362642437933526/ 「AI for Developer」!請點選以下連結加入社群! https://bit.ly/4az4vJt https://line.me/ti/g2/D2m1DA83Qb0KJCjd2mLyHD7y4yB1CA0WUJsOMA? utm_source=invitation&utm_medium=link_copy&utm_campaign=default

Slide 68

Slide 68 text

Caesar Chi clonncd ⼯程師成長⼿冊 clonn

Slide 69

Slide 69 text

http://exma-square.co/