OpenAI APIをNuxt.jsに組み込んでみた

by coconala_engineer

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Copyright coconala Inc. All Rights Reserved. UX Delivery Cost NuxtのServer Middlewareを採用 13 過去にServer Middleware を使ってAPIの実装経験があったイベントストリーム形式でのレスポンスが可能 (AWS Lamdaがレスポンスストリーミング対応していたがAPI Gatewayが対応していなかった) Server Middleware内でバリデーションを行うことでユーザーごとの利用回数を制限することが可能

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Copyright coconala Inc. All Rights Reserved. リクエストを受け取る 17 const middleware: { path: string; handler: ServerMiddleware } = { path: '/api/openai', async handler (req, res) { if (req.method !== 'POST') { res.statusCode = 404 res.end() return } let body = '' for await (const chunk of req) { body += chunk.toString() } const bodyJson = JSON.parse(body) const { prompt } = bodyJson // 〜省略〜 } }

Slide 18

Slide 18 text

Copyright coconala Inc. All Rights Reserved. OpenAI APIにリクエストを送る 18 const chatCompletionCreateParams: OpenAI.ChatCompletionCreateParamsStreaming = { model: 'gpt-3.5-turbo', messages: [ { role: 'user', content: prompt } ], stream: true // レスポンスをストリーミングで受け取る } const stream = await openAi.chat.completions.create(chatCompletionCreateParams)

Slide 19

Slide 19 text

Copyright coconala Inc. All Rights Reserved. イベントストリーム形式でレスポンスする 19 res.setHeader('Content-Type', 'text/event-stream;charset=utf-8') res.setHeader('Cache-Control', 'no-cache, no-transform') for await (const chunk of stream) { const chunkString = chunk.choices.map(v => v.delta.content).filter(v => !!v).join() res.write(chunkString) } res.end()

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Copyright coconala Inc. All Rights Reserved. ブラウザ側の実装 21 import { ref } from 'vue' import axios from 'axios' const prompt = ref('') const response = ref('') const send = () => { axios.post( '/api/openai', { prompt: prompt.value }, { onDownloadProgress: (v) => { response.value = v.target.responseText } } ) }

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Copyright coconala Inc. All Rights Reserved. 愚直にそのままsendStreamへ渡してみるが… 引用：https://nuxt.com/docs/guide/directory-structure/server#sending-streams 30 const stream = await openAi.chat.completions.create(chatCompletionCreateParams) return sendStream(event, stream)

Slide 31

Slide 31 text

Copyright coconala Inc. All Rights Reserved. エラー…（Streamにpipeがない） 31 [nuxt] [request error] [unhandled] [500] data.pipe is not a function at ./node_modules/h3/dist/index.mjs:556:10 at new Promise () at sendStream (./node_modules/h3/dist/index.mjs:555:10) at ./.nuxt/dev/index.mjs:645:10 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async Object.handler (./node_modules/h3/dist/index.mjs:990:19) at async Server.toNodeHandle (./node_modules/h3/dist/index.mjs:1065:7)

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Copyright coconala Inc. All Rights Reserved. AsyncIterableを作ってReadableのインスタンスを生成する必要がある 34 const stream = await openAi.chat.completions.create(chatCompletionCreateParams) const chatCompletionAsyncIterator = async function* (): AsyncIterable { for await (const chunk of stream) yield chunk.choices.map(v => v.delta.content).filter(v => !!v).join() } const readable = Stream.Readable.from(chatCompletionAsyncIterator()) res.setHeader('Content-Type', 'text/event-stream;charset=utf-8') res.setHeader('Cache-Control', 'no-cache, no-transform') return sendStream(event, readable)