🦜️🔗LangChain入門 / LangChain 101

Slide 1

Slide 1 text

© 2023, Amazon Web Services, Inc. or its aﬃliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 🦜🔗 LangChain ⼊⾨ ~ 世はまさに⼤規模⾔語モデル時代 ~ Kazuki Matsuda 俺たちの本当にやりたかった D E V D A Y Sr. Startup Solutions Architect Amazon Web Services Japan G.K.

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

© 2023, Amazon Web Services, Inc. or its aﬃliates. All rights reserved. LLM の抽象化 Amazon SageMaker （推論エンドポイント）サンプル実装 https://github.com/aws-samples/aws-ml-jp/blob/main/tasks/generative-ai/text-to-text/langchain/langchain-sagemaker-intro.ipynb

Slide 14

Slide 14 text

© 2023, Amazon Web Services, Inc. or its aﬃliates. All rights reserved. 参考：SageMaker で LLM を動かす SageMaker JumpStart りんな https://github.com/aws-samples/aws-ml-jp/blob/main/tasks/generative-ai/text-to-text/fine-tuning/instruction-tuning/Transformers/Rinna_Neox_Inference_ja.ipynb

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. プロンプトエンジニアリング • LLM に与えるプロンプトを調整し、求めるレスポンスを得る⼿法 • ⾃然⾔語で応答が返ってくるのはソフトウェア的には都合が悪い • if ⽂や正規表現をあてるにも⼀定の規則は必要 • ソフトウェアのパーツとして LLM を使おうとすると都合が悪い LLM ソフトウェア（コード）ソフトウェア（コード） Input Output

Slide 20

Slide 20 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. ソフトウェアのパーツとしての LLM の都合の悪さ • 各⾔語は組み込み関数や基本的な動作に従ってレスポンスを返す • if ⽂を書けば書いたとおりに動作するランタイム (Python) コード (パラメータや関数) レスポンス (JSON) Input Output • ⼀⽅ LLM は組み込み関数や基本的な動作が無いため、条件分岐や制限事項を書いたとしても意図通り動作しないことがある • Input に条件等を細かく書いても想定外の応答をすることがある • 書き⽅によって効きの強い弱いがある

Slide 21

Slide 21 text

Slide 22

Slide 22 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its aﬃliates. All rights reserved. LangChain はプロンプトエンジニアリングを効率的に⾏うためのツール兼フレームワーク兼ベストプラクティス集 ※ 個⼈の⾒解です

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

© 2023, Amazon Web Services, Inc. or its aﬃliates. All rights reserved. Agent 出典: https://towardsdatascience.com/implementing-a-sales-support-agent-with-langchain-63c4761193e7 出典: https://cobusgreyling.medium.com/agents-f444d165024 概念図内部シーケンス

Slide 26

Slide 26 text

© 2023, Amazon Web Services, Inc. or its aﬃliates. All rights reserved. ConversationalChatAgent Assistant is a large language model trained by OpenAI. Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human- like text based on the input it receives, allowing it to engage in natural- sounding conversations and provide responses that are coherent and relevant to the topic at hand. Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics. Overall, Assistant is a powerful system that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist. チャット形式で応答する A I アシスタントの実装例

Slide 27

Slide 27 text

© 2023, Amazon Web Services, Inc. or its aﬃliates. All rights reserved. ConversationalChatAgent TOOLS ------ Assistant can ask the user to use tools to look up information that may be helpful in answering the users original question. The tools the human can use are: {{tools}} {format_instructions} USER'S INPUT -------------------- Here is the user's input (remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else): {{{{input}}}} チャット形式で応答する A I アシスタントの実装例

Slide 28

Slide 28 text

Slide 29

Slide 29 text

© 2023, Amazon Web Services, Inc. or its aﬃliates. All rights reserved. ConversationalChatAgent TOOLS ------ Assistant can ask the user to use tools to look up information that may be helpful in answering the users original question. The tools the human can use are: duckduckgo_search: A wrapper around DuckDuckGo Search. Useful for when you need to answer questions about current events. Input should be a search query. aws_lambda_email: sends an email with the specified content to [email protected] {format_instructions} USER'S INPUT -------------------- Here is the user's input (remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else): {{{{input}}}} チャット形式で応答する A I アシスタントの実装例

Slide 30

Slide 30 text

© 2023, Amazon Web Services, Inc. or its aﬃliates. All rights reserved. ConversationalChatAgent TOOLS ------ Assistant can ask the user to use tools to look up information that may be helpful in answering the users original question. The tools the human can use are: duckduckgo_search: A wrapper around DuckDuckGo Search. Useful for when you need to answer questions about current events. Input should be a search query. aws_lambda_email: sends an email with the specified content to [email protected] {format_instructions} USER'S INPUT -------------------- Here is the user's input (remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else): {{{{input}}}} チャット形式で応答する A I アシスタントの実装例

Slide 31

Slide 31 text

© 2023, Amazon Web Services, Inc. or its aﬃliates. All rights reserved. ConversationalChatAgent RESPONSE FORMAT INSTRUCTIONS ---------------------------- When responding to me, please output a response in one of two formats: **Option 1:** Use this if you want the human to use a tool. Markdown code snippet formatted in the following schema: ```json {{{{ "action": string, ¥¥ The action to take. Must be one of {tool_names} "action_input": string ¥¥ The input to the action }}}} ``` **Option #2:** Use this if you want to respond directly to the human. Markdown code snippet formatted in the following schema: ```json {{{{ "action": "Final Answer", "action_input": string ¥¥ You should put what you want to return to use here }}}} ``` チャット形式で応答する A I アシスタントの実装例

Slide 32

Slide 32 text

© 2023, Amazon Web Services, Inc. or its aﬃliates. All rights reserved. ConversationalChatAgent チャット形式で応答する A I アシスタントの実装例 LLM Agent 検索エンジン次の⾏動を考える戻り値の JSON をパースし、コードから実⾏ひとくり返す⾃然⾔語で質問プログラムの実⾏ Zero-shot Learning の考え⽅や ReAct フレームワークに基づいて実装されているが、 LangChain を利⽤することで⾼度な知⾒や理解は不要で実装可能

Slide 33

Slide 33 text

Slide 34

Slide 34 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Retriever 出典: https://towardsdatascience.com/implementing-a-sales-support-agent-with-langchain-63c4761193e7 Agent 概念図回答精度を上げるために社内のリソースを適切に参照してほしいけど、全ドキュメントはプロンプトに⼊らないなぁ

Slide 35

Slide 35 text

Slide 36

Slide 36 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. AwsKendraIndexRetriever A M A Z O N K E N D R A を問い合わせ先に利⽤する LLM Agent ⾃然⾔語で問い合わせ • Kendara へのクエリを⽣成する • Kendra の検索結果踏まえ回答を考えるひと⾃然⾔語で質問 Amazon Kendra 社内リソース

Slide 37

Slide 37 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Vector Store 多次元ベクトルをインデックスでき、クエリできるデータストア Vector Store 検索したいドキュメントベクトル化※してインデックスベクトルとしてクエリアプリケーション (LangChain) AWS 上で利⽤可能な Vector Store OpenSearch RDS for PostgreSQL (pgvector) Supabase Vector (Supabase Cloud or Self-hosting) ※ ベクトル化（Embedding）は別サービス‧ツールの利⽤が必要 ※ AWS Marketprace

Slide 38

Slide 38 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Vector Store 多次元ベクトルをインデックスでき、クエリできるデータストア • 主に Agent / Tool / Retriever の参照先として利⽤ • Memory（ Agent が過去のやりとりの履歴を保存する）の保存先としても利⽤できるようになったりしているので、別の機能の内部で利⽤する可能性もある

Slide 39

Slide 39 text

Slide 40

Slide 40 text

© 2023, Amazon Web Services, Inc. or its aﬃliates. All rights reserved. まとめ • LangChain プロンプトエンジニアリングのベストプラクティス集 • プロンプトテンプレート + 出⼒フォーマット規制 + 戻り値のハンドリング • ⼀⽅で⾊々なユースケースの実装例が含まれているのでかなり雑多 • 故に LLM の深い知⾒がなくても利⽤できる • あったほうが better ではあるけど • ビッグウェーブに乗ろう！！

Slide 41

Slide 41 text

© 2023, Amazon Web Services, Inc. or its aﬃliates. All rights reserved. AWS Dev Day の LLM 関連セッション © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. ⼤規模⾔語モデル (LLM) の進化: GPT 以降の最新動向 Yoshitaka Haribara, Ph.D. F - 1 Sr. Startup ML Solutions Architect AWS Japan G.K. ← General Session 2 (インタビュー + ラウンドテーブル) ↓ Breakout Session (F-1)

Slide 42

Slide 42 text

© 2023, Amazon Web Services, Inc. or its aﬃliates. All rights reserved. Thank you! © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Kazuki Matsuda Sr. Startup Solutions Architect Amazon Web Services Japan G.K.