Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Gen AI 時代における「サーバーレス」の価値を理解しよう / Value of Serverless with genAI

kensh
October 25, 2023

Gen AI 時代における「サーバーレス」の価値を理解しよう / Value of Serverless with genAI

サーバーレスを利用することにより、Gen AIにおけるコンシューマとしての価値が向上します。たとえば、以下の様なもの。

* イベント駆動アーキテクチャ により同期の待機時間が解消
* 進化的なアーキテクチャを促進します。
* コンポーネント単位のきめ細かいスケーリング
* 他のビジネス領域へのビジネス拡張も可能
* 障害の影響範囲の軽減

Gen AI 時代における「サーバーレス」の価値を理解し共有しましょう。

kensh

October 25, 2023
Tweet

More Decks by kensh

Other Decks in Technology

Transcript

  1. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Kensuke Shimokawa
    Snr. Serverless Specialist
    Amazon Web Services Japan
    Gen AI 時代における「サーバーレス」
    の価値を理解しよう

    View full-size slide

  2. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 2
    Kensuke Shimokawa
    Amazon Web Services Japan
    Snr. Serverless Specialist
    Slides https://speakerdeck.com/_kensh
    Qiita https://qiita.com/_kensh

    View full-size slide

  3. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 3

    View full-size slide

  4. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 4
    13th Nov 2014

    View full-size slide

  5. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Serverless
    Serverless Application とは、
    “誰も使用していないときは、
    データ ストレージのコストを除いて、
    実行に費用がかからない.”
    5

    View full-size slide

  6. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 6

    View full-size slide

  7. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 7
    静的コンテンツ
    動的データ
    記事/試合状況
    フォロー状況管理
    認可
    (フェデレーション)
    ログの保存
    ETL処理
    ログ収集 データ分析
    クローリング
    記事に対する処理
    (タグ付けなど)
    変更通知
    画像に対する処理
    (顔座標の検出)
    コードで実現 インテグレーションで実現

    View full-size slide

  8. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 8

    View full-size slide

  9. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 9

    View full-size slide

  10. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 10
    Services Services
    Integration
    Event Driven

    View full-size slide

  11. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 11
    Integration
    API
    Destination
    API
    Gateway
    Lambda Batch
    CloudWatch
    ECS
    EventBridge
    Bus
    Kinesis
    Data Firehose
    Inspector
    Kinesis
    Data Streams
    Redshift
    SageMaker
    pipeline
    SNS SQS
    Step
    Functions
    Event Driven

    View full-size slide

  12. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 12
    Integration
    API
    Destination
    API
    Gateway
    Lambda Batch
    CloudWatch
    ECS
    EventBridge
    Bus
    Kinesis
    Data Firehose
    Inspector
    Kinesis
    Data Streams
    Redshift
    SageMaker
    pipeline
    SNS SQS
    Step
    Functions
    EventBridge Pipes
    Event Driven

    View full-size slide

  13. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 13
    サーバーレス のサイズ感と組み立て

    View full-size slide

  14. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 14
    VM
    Container Function

    View full-size slide

  15. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 15
    VM
    Container Function
    たくさんのことを
    一つのインスタンスで
    まぁまぁのことを
    複数のインスタンスで
    ちょっとのことを
    たくさんのインスタンスで

    View full-size slide

  16. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 16
    Function
    ちょっとのことを
    たくさんのインスタンスで
    たくさんの処理を
    ワークフローで

    View full-size slide

  17. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 17
    Function
    ちょっとのことを
    たくさんのインスタンスで
    たくさんの処理を
    ワークフローで
    AWS Step Functions

    View full-size slide

  18. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 18
    6 5 4 3 2 1
    7
    Array
    56
    6 5 4 3 2 1
    7
    MAP
    f (a) = 2 × a
    12 10 8 6 4 2
    14
    Reduce
    g (b, c) = b + c

    View full-size slide

  19. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 19
    Dec 2022

    View full-size slide

  20. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 20
    6 5 4 3 2 1
    7
    Distibuted MAP
    f (a) = 2 × a
    12 10 8 6 4 2
    14
    最大 10,000 の並列同時実行をサポート

    View full-size slide

  21. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 21
    AI と Serverless

    View full-size slide

  22. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.
    Gen AIとサーバーレスの統合
    22
    Prompt
    time>
    Model Consumer 既存 Model
    +
    Model Tuner
    Domain Data +
    新 Model
    Model Builder/Provider
    Internet Scale Dataset
    Or
    Domain Datasets
    +
    新 Model
    Serverless
    Existing Model
    Time To Market

    View full-size slide

  23. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.
    アジリティと Gen AI の組み合わせ
    23
    イノベーション
    をスピーディー
    に!
    Power of Gen AI
    Speed of Serverless
    Amazon
    Bedrock
    Amazon
    SageMaker
    Amazon
    Code Whisperer
    AWS
    Lambda
    Amazon
    ECS
    AWS
    Fargate
    Amazon
    EventBridge
    AWS
    Step
    Functions

    View full-size slide

  24. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.
    同期的なモデル呼び出し
    24
    • Amazon Bedrock
    • Amazon SageMaker
    Jumpstart
    • Models hosted on:
    • EKS
    • Other computes
    FM Endpoint
    • Auth, Rate limiting, Caching
    with Amazon API Gateway
    • AWS Lambda calling an
    endpoint
    API + Compute
    • Simple Q/A
    • Content generation
    • Text Summarization
    Use cases
    Amazon API Gateway AWS Lambda FM Endpoint

    View full-size slide

  25. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.
    非同期的なモデル呼び出し
    25
    • Amazon Bedrock
    • Amazon SageMaker
    Jumpstart
    • Models hosted on:
    • EKS
    • Other computes
    FM Endpoint
    • Auth, Rate limiting, Caching
    with Amazon API Gateway
    • Amazon SQS queue provides
    control over the scaling
    • AWS Lambda calling an
    endpoint
    API + Queue + Compute
    • Simple Q/A
    • Content generation
    • Text Summarization
    Use cases
    Amazon API Gateway AWS Lambda
    Amazon SQS FM Endpoint

    View full-size slide

  26. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.
    非同期の場合のレスポンス
    26
    • Amazon Bedrock
    • Amazon SageMaker
    Jumpstart
    • Models hosted on:
    • EKS
    • Other computes
    FM Endpoint
    • Auth, Rate limiting, Caching
    with Amazon API Gateway
    • Amazon SQS queue provides
    control over the scaling
    • AWS Lambda calling an
    endpoint
    • Use polling, WebSocket, or
    IoT core topic for response
    API + Queue + Compute
    • Simple Q/A
    • Content generation
    • Text Summarization
    Use cases
    Amazon API Gateway AWS Lambda
    Amazon SQS
    Polling/WebSocket
    FM Endpoint

    View full-size slide

  27. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.
    Conversation History をどうするか?
    27
    • Amazon Bedrock
    • Amazon SageMaker
    Jumpstart
    • Models hosted on:
    • EKS
    • Other computes
    FM Endpoint
    • Auth, Rate limiting, Caching
    with Amazon API Gateway
    • Amazon SQS queue provides
    control over the scaling
    • AWS Lambda calling an
    endpoint
    • Use polling, WebSocket, or
    IoT core topic for response
    • Store conversation history
    API + Queue + Compute + DB
    • Simple Q/A
    • Content generation
    • Text Summarization
    • Conversation memory
    Use cases
    Amazon API Gateway AWS Lambda
    Amazon SQS
    Amazon DynamoDB
    FM Endpoint
    Polling/WebSocket

    View full-size slide

  28. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.
    複雑な工程は オーケストレーションする
    28
    • Amazon Bedrock
    • Amazon SageMaker
    Jumpstart
    • Models hosted on:
    • EKS
    • Other computes
    FM Endpoint
    • Auth, Rate limiting, Caching
    with Amazon API Gateway
    • Amazon SQS queue provides
    control over the scaling
    • AWS Lambda calling an
    endpoint
    • Use polling, WebSocket, or
    IoT core topic for response
    • Store conversation history
    • Co-ordinate multiple tasks
    with less code
    API + Queue + Compute + DB
    • Context based Q/A
    • Content generation
    • Text Summarization
    Use cases
    Amazon API Gateway AWS Lambda
    Amazon SQS
    Amazon DynamoDB
    AWS Step Functions workflow
    FM Endpoint
    Polling/WebSocket

    View full-size slide

  29. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.
    最近は、非同期レスポンスに
    IoT Core もよく使われる
    29
    AWS IoT Core
    Amazon SQS
    Amazon API Gateway
    AWS Step Functions workflow
    AWS Lambda

    View full-size slide

  30. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.
    複数モデルをコンテキストで使い分け
    30
    • Amazon Bedrock
    • Amazon SageMaker
    Jumpstart
    • Models hosted on:
    • EKS
    • Other computes
    FM Endpoint
    • Auth, Rate limiting, Caching
    with Amazon API Gateway
    • Amazon EventBridge or
    Amazon SNS to fan-out
    • Amazon SQS as subscribers
    • AWS Lambda or AWS Step
    Functions getting from the
    queue
    API + Fan-out (Pub/Sub)
    • Multi-model evaluation
    • Content generation from
    different models
    Use cases
    Amazon
    API Gateway
    AWS Step Functions
    Amazon
    EventBridge
    AWS Fargate Task
    AWS Lambda Amazon Bedrock
    ANTHROP\C
    Amazon Bedrock
    Amazon
    SageMaker
    Jumpstart
    Rule 1
    Rule 2
    Rule 3
    Amazon SQS
    Amazon SQS
    Amazon SQS

    View full-size slide

  31. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.
    複数モデルをコンテキストで使い分け
    31
    • Amazon Bedrock
    • Amazon SageMaker
    Jumpstart
    • Models hosted on:
    • EKS
    • Other computes
    FM Endpoint
    • Auth, Rate limiting, Caching
    with Amazon API Gateway
    • AWS Step Functions invokes
    models in parallel and
    generates response
    • Introduce human-in-the-
    loop as an when needed
    API + Fan-out (Pub/Sub)
    • Multi-model evaluation
    • Content generation from
    different models
    Use cases
    Amazon
    API Gateway
    AWS Step Functions
    Amazon Bedrock
    ANTHROP\C
    Amazon Bedrock
    Amazon
    SageMaker
    Jumpstart

    View full-size slide

  32. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 32
    RAG のための
    ベクトル埋込 A N S W E R
    Q U E S T I O N
    V E C T O R
    D A T A S T O R E S
    F O U N D A T I O N
    M O D E L
    Vector embeddings

    View full-size slide

  33. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.
    データをベクトル埋め込みに変換する
    33
    Preprocess Chunk Create embeddings
    Store
    embeddings
    Embedding LLM Vector DB
    Documents
    AWS Step Functions workflow

    View full-size slide

  34. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.
    RAG を使用した LLM の呼び出
    34
    • Amazon Bedrock
    • Amazon SageMaker
    Jumpstart
    • Embedding LLMs
    • Amazon Titan
    LLMs
    • Amazon OpenSearch
    • Amazon Kendra
    • Chroma DB
    • Pinecone
    Vector/Knowledge Databases
    • Domain specific Q/A
    • Intelligent Document
    Processing
    • Text Summarization
    Use cases
    • Native integration with
    Amazon S3
    • Orchestration
    Serverless
    Retrieval Augmented Generation (RAG)
    Vector DB
    Documents Amazon S3 Amazon
    EventBridge
    Step Functions
    Amazon
    API Gateway
    AWS Lambda
    Amazon SQS
    AWS IoT Core
    ANTHROP\C
    Claude
    Amazon Bedrock
    Relevant
    Context
    Question?
    Answer
    Q? + C
    A
    Retrieval Augmented Generation
    (RAG)
    Similarity
    Search

    View full-size slide

  35. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 36
    Serverless の今後?

    View full-size slide

  36. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Serverless の価値の今
    • Serverless が担うビジネスロジック、ワークロードは広がる
    • Lambda が担ってきた “糊付け” としての役割は EventBridge や
    Step Functions に多く譲っていくことになりそう
    • genAIでは、TimeTM を意識して Consumer 部分はServerless で
    37
    今後の期待は、、
    • 今は想像もしていないようなワークロードで Lambda が使われて
    いくのではないか?
    • 面白いユースケースがあれば、@_kensh まで DM ください

    View full-size slide

  37. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    How to get started?
    38
    Document chat with AWS Lambda and Amazon Bedrock
    Enhancing AWS Intelligent document processing using Generative AI
    Generative AI with Serverless
    Multi-LLM and Multi-RAG powered chatbot using AWS CDK
    Generative AI for summarization and question answering using your own data

    View full-size slide

  38. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 39
    Thank you !

    View full-size slide