Upgrade to Pro — share decks privately, control downloads, hide ads and more …

🦜️🔗LangChain入門 / LangChain 101

mats
June 21, 2023

🦜️🔗LangChain入門 / LangChain 101

セッション時間が足りずだいぶ端折りました。すいません。

mats

June 21, 2023
Tweet

More Decks by mats

Other Decks in Technology

Transcript

  1. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    🦜🔗 LangChain ⼊⾨
    ~ 世はまさに⼤規模⾔語モデル時代 ~
    Kazuki Matsuda
    俺 た ち の 本 当 に や り た か っ た D E V D A Y
    Sr. Startup Solutions Architect
    Amazon Web Services Japan G.K.

    View full-size slide

  2. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.

    View full-size slide

  3. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    ビッグウェーブ?

    View full-size slide

  4. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.

    View full-size slide

  5. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    About me...
    松⽥ 和樹

    View full-size slide

  6. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    LangChain とは

    View full-size slide

  7. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    LangChain 知っている⽅?

    View full-size slide

  8. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.

    View full-size slide

  9. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    LangChain
    LLM を⽤いたアプリケーションを効率的に作るためのライブラリ
    • LLM の抽象化
    • プロンプトエンジニアリングの効率化(とても多機能)

    View full-size slide

  10. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    LangChain
    LLM を⽤いたアプリケーションを効率的に作るためのライブラリ
    • LLM の抽象化
    • プロンプトエンジニアリングの効率化(とても多機能)

    View full-size slide

  11. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    LLM の抽象化
    ※ ⼀応ふれておきます

    View full-size slide

  12. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    LLM の抽象化
    OpenAI API
    (Model: GPT-3.5)
    Amazon Bedrock
    (Model: Amazon Titan)

    View full-size slide

  13. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    LLM の抽象化
    Amazon SageMaker
    (推論エンドポイント)
    サンプル実装
    https://github.com/aws-samples/aws-ml-jp/blob/main/tasks/generative-ai/text-to-text/langchain/langchain-sagemaker-intro.ipynb

    View full-size slide

  14. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    参考:SageMaker で LLM を動かす
    SageMaker JumpStart
    りんな
    https://github.com/aws-samples/aws-ml-jp/blob/main/tasks/generative-ai/text-to-text/fine-tuning/instruction-tuning/Transformers/Rinna_Neox_Inference_ja.ipynb

    View full-size slide

  15. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    LangChain
    LLM を⽤いたアプリケーションを効率的に作るためのライブラリ
    • LLM の抽象化
    • プロンプトエンジニアリングの効率化(とても多機能)

    View full-size slide

  16. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    プロンプトエンジニアリング

    View full-size slide

  17. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    プロンプトエンジニアリング?
    • Twitter でこういうのよく⾒ますね 👉
    • これも含みますが、ごく⼀部

    View full-size slide

  18. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    プロンプトエンジニアリング
    • LLM に与えるプロンプトを調整し、求めるレスポンスを得る⼿法
    LLM
    ⾃然⾔語 ⾃然⾔語
    Input Output

    View full-size slide

  19. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    プロンプトエンジニアリング
    • LLM に与えるプロンプトを調整し、求めるレスポンスを得る⼿法
    • ⾃然⾔語で応答が返ってくるのはソフトウェア的には都合が悪い
    • if ⽂や正規表現をあてるにも⼀定の規則は必要
    • ソフトウェアのパーツとして LLM を使おうとすると都合が悪い
    LLM
    ソフトウェア
    (コード)
    ソフトウェア
    (コード)
    Input Output

    View full-size slide

  20. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    ソフトウェアのパーツとしての LLM の都合の悪さ
    • 各⾔語は組み込み関数や基本的な動作に従ってレスポンスを返す
    • if ⽂を書けば書いたとおりに動作する
    ランタイム
    (Python)
    コード
    (パラメータや関数)
    レスポンス
    (JSON)
    Input Output
    • ⼀⽅ LLM は組み込み関数や基本的な動作が無いため、条件分岐や
    制限事項を書いたとしても意図通り動作しないことがある
    • Input に条件等を細かく書いても想定外の応答をすることがある
    • 書き⽅によって効きの強い弱いがある

    View full-size slide

  21. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    ソフトウェアのパーツとしての LLM
    LLM を Chain する例
    LLM ①
    ChatBot
    LLM ②
    検索エンジン
    検索クエリを考える
    検索する
    検索結果から適切な回答が
    含まれているか評価する
    ひと

    View full-size slide

  22. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    LangChain は プロンプトエンジニアリング を
    効率的に⾏うための ツール 兼 フレームワーク 兼
    ベストプラクティス集
    ※ 個⼈の⾒解です

    View full-size slide

  23. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    具体的な例をいくつか紹介

    View full-size slide

  24. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Agent

    View full-size slide

  25. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Agent
    出典: https://towardsdatascience.com/implementing-a-sales-support-agent-with-langchain-63c4761193e7
    出典: https://cobusgreyling.medium.com/agents-f444d165024
    概念図
    内部シーケンス

    View full-size slide

  26. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    ConversationalChatAgent
    Assistant is a large language model trained by OpenAI.
    Assistant is designed to be able to assist with a wide range of tasks, from
    answering simple questions to providing in-depth explanations and discussions on a
    wide range of topics. As a language model, Assistant is able to generate human-
    like text based on the input it receives, allowing it to engage in natural-
    sounding conversations and provide responses that are coherent and relevant to the
    topic at hand.
    Assistant is constantly learning and improving, and its capabilities are
    constantly evolving. It is able to process and understand large amounts of text,
    and can use this knowledge to provide accurate and informative responses to a wide
    range of questions. Additionally, Assistant is able to generate its own text based
    on the input it receives, allowing it to engage in discussions and provide
    explanations and descriptions on a wide range of topics.
    Overall, Assistant is a powerful system that can help with a wide range of tasks
    and provide valuable insights and information on a wide range of topics. Whether
    you need help with a specific question or just want to have a conversation about a
    particular topic, Assistant is here to assist.
    チ ャ ッ ト 形 式 で 応 答 す る A I ア シ ス タ ン ト の 実 装 例

    View full-size slide

  27. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    ConversationalChatAgent
    TOOLS
    ------
    Assistant can ask the user to use tools to look up information that may be helpful
    in answering the users original question. The tools the human can use are:
    {{tools}}
    {format_instructions}
    USER'S INPUT
    --------------------
    Here is the user's input (remember to respond with a markdown code snippet of a
    json blob with a single action, and NOTHING else):
    {{{{input}}}}
    チ ャ ッ ト 形 式 で 応 答 す る A I ア シ ス タ ン ト の 実 装 例

    View full-size slide

  28. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    ConversationalChatAgent
    TOOLS
    ------
    Assistant can ask the user to use tools to look up information that may be helpful
    in answering the users original question. The tools the human can use are:
    {{tools}}
    {format_instructions}
    USER'S INPUT
    --------------------
    Here is the user's input (remember to respond with a markdown code snippet of a
    json blob with a single action, and NOTHING else):
    {{{{input}}}}
    チ ャ ッ ト 形 式 で 応 答 す る A I ア シ ス タ ン ト の 実 装 例

    View full-size slide

  29. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    ConversationalChatAgent
    TOOLS
    ------
    Assistant can ask the user to use tools to look up information that may be helpful
    in answering the users original question. The tools the human can use are:
    duckduckgo_search: A wrapper around DuckDuckGo Search.
    Useful for when you need to answer questions about current events.
    Input should be a search query.
    aws_lambda_email: sends an email with the specified content to [email protected]
    {format_instructions}
    USER'S INPUT
    --------------------
    Here is the user's input (remember to respond with a markdown code snippet of a
    json blob with a single action, and NOTHING else):
    {{{{input}}}}
    チ ャ ッ ト 形 式 で 応 答 す る A I ア シ ス タ ン ト の 実 装 例

    View full-size slide

  30. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    ConversationalChatAgent
    TOOLS
    ------
    Assistant can ask the user to use tools to look up information that may be helpful
    in answering the users original question. The tools the human can use are:
    duckduckgo_search: A wrapper around DuckDuckGo Search.
    Useful for when you need to answer questions about current events.
    Input should be a search query.
    aws_lambda_email: sends an email with the specified content to [email protected]
    {format_instructions}
    USER'S INPUT
    --------------------
    Here is the user's input (remember to respond with a markdown code snippet of a
    json blob with a single action, and NOTHING else):
    {{{{input}}}}
    チ ャ ッ ト 形 式 で 応 答 す る A I ア シ ス タ ン ト の 実 装 例

    View full-size slide

  31. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    ConversationalChatAgent
    RESPONSE FORMAT INSTRUCTIONS
    ----------------------------
    When responding to me, please output a response in one of two formats:
    **Option 1:**
    Use this if you want the human to use a tool.
    Markdown code snippet formatted in the following schema:
    ```json
    {{{{
    "action": string, ¥¥ The action to take. Must be one of {tool_names}
    "action_input": string ¥¥ The input to the action
    }}}}
    ```
    **Option #2:**
    Use this if you want to respond directly to the human. Markdown code snippet formatted in the
    following schema:
    ```json
    {{{{
    "action": "Final Answer",
    "action_input": string ¥¥ You should put what you want to return to use here
    }}}}
    ```
    チ ャ ッ ト 形 式 で 応 答 す る A I ア シ ス タ ン ト の 実 装 例

    View full-size slide

  32. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    ConversationalChatAgent
    チ ャ ッ ト 形 式 で 応 答 す る A I ア シ ス タ ン ト の 実 装 例
    LLM
    Agent
    検索エンジン
    次の⾏動を考える
    戻り値の JSON をパースし、
    コードから実⾏
    ひと
    くり返す
    ⾃然⾔語で質問
    プログラムの実⾏
    Zero-shot Learning の考え⽅や ReAct フレームワークに基づいて実装されているが、
    LangChain を利⽤することで⾼度な知⾒や理解は不要で実装可能

    View full-size slide

  33. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Retriever

    View full-size slide

  34. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Retriever
    出典: https://towardsdatascience.com/implementing-a-sales-support-agent-with-langchain-63c4761193e7
    Agent 概念図
    回答精度を上げるために社内の
    リソースを適切に参照してほしいけど、
    全ドキュメントはプロンプトに⼊らない
    なぁ

    View full-size slide

  35. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Retriever
    LLM
    Agent
    社内検索エンジン
    社内リソースの内容踏まえ
    回答を考える
    問い合わせ先毎に
    クエリ⽅法が異なる
    ひと
    ⾃然⾔語で質問
    Vector Store
    問い合わせ先ごとのお作法に合わせたモジュールを Retriever と呼んでいる

    View full-size slide

  36. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    AwsKendraIndexRetriever
    A M A Z O N K E N D R A を 問 い 合 わ せ 先 に 利 ⽤ す る
    LLM
    Agent
    ⾃然⾔語で問い合わせ
    • Kendara へのクエリを⽣成する
    • Kendra の検索結果踏まえ回答を考える
    ひと
    ⾃然⾔語で質問
    Amazon Kendra
    社内リソース

    View full-size slide

  37. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Vector Store
    多次元ベクトルをインデックスでき、クエリできるデータストア
    Vector Store 検索したいドキュメント
    ベクトル化※して
    インデックス
    ベクトルとして
    クエリ
    アプリケーション
    (LangChain)
    AWS 上で利⽤可能な Vector Store
    OpenSearch RDS for PostgreSQL
    (pgvector)
    Supabase Vector
    (Supabase Cloud or Self-hosting)
    ※ ベクトル化(Embedding)は別サービス‧ツールの利⽤が必要
    ※ AWS Marketprace

    View full-size slide

  38. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Vector Store
    多次元ベクトルをインデックスでき、クエリできるデータストア
    • 主に Agent / Tool / Retriever の参照先として利⽤
    • Memory( Agent が過去のやりとりの履歴を保存する )の保存先と
    しても利⽤できるようになったりしているので、別の機能の内部で
    利⽤する可能性もある

    View full-size slide

  39. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    まとめ

    View full-size slide

  40. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    まとめ
    • LangChain プロンプトエンジニアリングのベストプラクティス集
    • プロンプトテンプレート + 出⼒フォーマット規制 + 戻り値のハンドリング
    • ⼀⽅で⾊々なユースケースの実装例が含まれているのでかなり雑多
    • 故に LLM の深い知⾒がなくても利⽤できる
    • あったほうが better ではあるけど
    • ビッグウェーブに乗ろう!!

    View full-size slide

  41. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    AWS Dev Day の LLM 関連セッション
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    ⼤規模⾔語モデル (LLM) の進化:
    GPT 以降の最新動向
    Yoshitaka Haribara, Ph.D.
    F - 1
    Sr. Startup ML Solutions Architect
    AWS Japan G.K.
    ← General Session 2
    (インタビュー + ラウンドテーブル)
    ↓ Breakout Session (F-1)

    View full-size slide

  42. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Thank you!
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Kazuki Matsuda
    Sr. Startup Solutions Architect
    Amazon Web Services Japan G.K.

    View full-size slide