$30 off During Our Annual Pro Sale. View Details »

refactoring-serverless

g-awa
September 21, 2023

 refactoring-serverless

Serverless Days Tokyo 2023

サーバーレスというと、真っ先に思い浮かべるのが AWS Lambda だと思いますが、AWS Lambda 以外にも API Gateway や Step Functions, Event Bridge など様々なサービスと統合されてフルスタックなサーバーレス構成が取れるようになっています。かつては Lamnbda で 全てのビジネスロジックを書いていたところが 周辺の AWS サービスに処理を移譲することができるようになってきました 。このように Lambda のアプリケーションコードを、いかに AWS のサービスにオフロードしていくのか、CDK のコード例も交えながら、リファクタリングの手法・パターンがあるのか、ご説明しています。

g-awa

September 21, 2023
Tweet

More Decks by g-awa

Other Decks in Technology

Transcript

  1. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    淡路 大輔
    アマゾン ウェブ サービス ジャパン合同会社
    ソリューションアーキテクト
    S E R V E R L E S S D A Y S T O K Y O 2 0 2 3
    Refactoring serverless
    ※ 本資料に記載されているプログラムはプレゼンテーション向けに簡略化された表記である場合があります

    View Slide

  2. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Daisuke Awaji
    Amazon Web Services Japan
    Solutions Architect
    3
    @gee0awa

    View Slide

  3. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    History of AWS Lambda
    4
    サーバーレスと出会ってから何年経ちましたか?

    View Slide

  4. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 5
    2014 2015 2016 2017 2018 2019
    2020 2021 2022 2023 2024
    Introducing
    AWS Lambda
    Available in Tokyo
    5 Minute Functions
    Access resources
    within a VPC
    Up to 15 minutes
    Custom runtimes
    Provisioned
    concurrency
    Supports EFS,
    container images
    Lambda extentions
    10GB Memory
    Graviton2
    10GB /tmp
    function URL
    Lambda SnapStart
    Response payload
    streaming
    10th
    Anniversary
    Introducing Amazon API Gateway
    Introducing Amazon EventBridge
    Introducing AWS Step Functions
    2004 Introducing Amazon SQS
    2006 Introducing Amazon S3
    ~ 2006
    Support SQS as Event Source
    Step Functions
    Express Workflows
    Step Functions
    AWS SDK Integration

    View Slide

  5. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    How to design
    Serverless Application
    6

    View Slide

  6. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 7
    AWS Lambda

    View Slide

  7. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 8
    Function
    Node.js
    Python
    Java
    C#
    Go
    Ruby
    Runtime API
    Container images
    Service
    Many more…
    Event
    データの更新
    API リクエスト
    状態の変更
    etc…

    View Slide

  8. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 9
    Event Service
    データの更新
    API リクエスト
    状態の変更
    Many more…
    etc…
    Service integration

    View Slide

  9. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Refactoring to Serverless
    10
    Reduce glue code, extend glue service

    View Slide

  10. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    既存のコード本体を再構築し、
    外部の動作を変更せずに
    内部構造を変更するための規律ある⼿法
    Martin Fowler, Refactoring.com
    リファクタリングとは

    View Slide

  11. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    既存のコード本体を再構築し、
    外部の動作を変更せずに
    内部構造を変更するための規律ある⼿法
    Martin Fowler, Refactoring.com
    リファクタリングとは

    View Slide

  12. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    リファクタリングとテスト駆動開発(TDD)
    RED(失敗するテストを書く)
    GREEN(テストを通す)
    リファクタリング
    TDD

    View Slide

  13. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Serverlessland - Testing Patterns Collection
    14
    https://serverlessland.com/testing/patterns

    View Slide

  14. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Serverless Refactoring Pattern
    15
    Extract send message
    (using Lambda Destination)
    AWS Step Functions
    Service Integration
    Send Message via
    EventBridge Pipes
    02 03
    01

    View Slide

  15. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Serverless Refactoring Pattern
    16
    Extract send message
    (using Lambda Destination)
    AWS Step Functions
    Service Integration
    Send Message via
    EventBridge Pipes
    02 03
    01

    View Slide

  16. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Extract send message (using Lambda Destination)
    17
    AWS Lambda
    Amazon SQS
    (success)
    Amazon SNS
    async
    invocation
    Amazon SQS
    (failure)
    https://aws.amazon.com/jp/blogs/compute/introducing-aws-lambda-destinations/

    View Slide

  17. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Extract send message (using Lambda Destination)
    18
    AWS Lambda
    import { SQS } from "aws-sdk";
    export const handler = async (event) => {
    const result = // Business Logic
    const params = {
    QueueUrl: process.env.QUEUE_URL,
    MessageBody: result,
    };
    await sqs.sendMessage(params).promise();
    };

    View Slide

  18. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Extract send message (using Lambda Destination)
    19
    AWS Lambda
    QueueUrl: process.env.QUEUE_URL,
    await sqs.sendMessage(params).promise();
    ・トポロジーが環境変数に隠れている
    ・アプリケーションコードにロジックとサービスの依存関係が混在

    View Slide

  19. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Extract send message (using Lambda Destination)
    20
    AWS Lambda AWS Lambda
    export const handler = async (event) => {
    const result = // Business Logic
    return { message: result };
    };
    AWS CDK
    new Function(this, ”orderFunction", {
    runtime: Runtime.NODEJS_18_X,
    code: Code.fromAsset("app/functions//index.ts”),
    handler: ”order.handler",
    onSuccess: new SqsDestination(queue),
    });
    Original Refactored
    ・トポロジーは CDK コードに集約される
    ・アプリケーションコードには、ロジックのみ記述される
    import { SQS } from "aws-sdk";
    export const handler = async (event) => {
    const result = // Business Logic
    const params = {
    QueueUrl: process.env.QUEUE_URL,
    MessageBody: result,
    };
    await sqs.sendMessage(params).promise();
    };
    ・トポロジーが環境変数に隠れている
    ・アプリケーションコードにロジックとサービスの依存関係が混在

    View Slide

  20. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Extract send message – Error handling
    21
    AWS Lambda AWS Lambda
    export const handler = async (event) => {
    const result = // Business Logic
    return { message: result };
    };
    AWS CDK
    new Function(this, ”orderFunction", {
    runtime: Runtime.NODEJS_18_X,
    code: Code.fromAsset("app/functions//index.ts”),
    handler: ”order.handler",
    onSuccess: new SqsDestination(queue),
    onFailure: new SqsDestination(failureQueue),
    });
    Original Refactored
    import { SQS } from "aws-sdk";
    export const handler = async (event) => {
    try {
    const result = // Business Logic
    const params = {
    QueueUrl: process.env.QUEUE_URL,
    MessageBody: result,
    };
    await sqs.sendMessage(params).promise();
    } catch (e) {
    const error = // Error handling Logic
    await sqs.sendMessage(error).promise();
    }
    };

    View Slide

  21. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Extract send message – 注意するポイント
    22
    ・Lambda の非同期呼び出しのみサポート
    ・SQS への送信処理は Function の処理の最後に実⾏される
    複数の Queue に送信するシーンでは活⽤できない
    複数の Queue に “同⼀” メッセージを送信したい場合は SNS を Destination の送信先に設定し、Fan Out する
    ・SDK による実装と⽐べて、SQS への送信メッセージの形式が変更される
    SQS Message using AWS SDK SQS Message using Lambda Destination

    View Slide

  22. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    サーバーレスをリファクタリングする目的
    23
    アプリ要件に適⽤する
    AWS の機能追加に適応する
    コストを削減する
    ランタイム特性を改善する
    マネージドサービスを活⽤する

    View Slide

  23. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Serverless Refactoring Pattern
    24
    Extract send message
    (using Lambda Destination)
    AWS Step Functions
    Service Integration
    Send Message via
    EventBridge Pipes
    02 03
    01

    View Slide

  24. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    AWS Step Functions Service Integration
    25
    AWS Step Functions workflow
    Lambda: Invoke
    Detect Labels
    start
    Pass state
    Extract Name
    Choice state
    Is Pizza?
    Fail state
    Failed
    Succeed state
    Passed
    End
    Lambda Function
    Amazon Rekognition
    image
    Amazon S3
    Image Bucket
    Detect Labels API
    AWS Step Functions workflow
    Rekognition:
    Detect Object
    start
    Pass state
    Extract Name
    Choice state
    Is Pizza?
    Fail state
    Failed
    Succeed state
    Passed
    End
    Amazon Rekognition
    image
    Amazon S3
    Image Bucket
    AWS Step Functions から
    直接 AWS サービスの API をコールする

    View Slide

  25. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    AWS Step Functions Service Integration
    26
    ・ランタイム要素(Lambda)を無くすことで、コスト・実⾏時間・複雑さを減らす
    Lambda の同時実⾏数消費の緩和、 15 分の timeout の壁を越える
    ライブラリのアップデートなどの運⽤負荷も軽減する
    ・余分なサービスホップを排除することで、レイテンシーを減らす

    View Slide

  26. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    AWS Lambda
    27
    const handler = async (event) => {
    const result = await rekognition.detectLabels({
    Image: {
    S3Object: {
    Bucket: event.s3Bucket, Name: event.imageName }
    },
    });
    };
    AWS CDK
    const detectObjectLambda = new lambda.Function(this, {…});
    const detectObject = new tasks.LambdaInvoke(this, "Detect Object", {
    lambdaFunction: detectObjectLambda,
    payload: sfn.TaskInput.fromObject({
    s3Bucket: imageBucket.bucketName,
    imageName: this.IMAGE_TO_LABEL,
    }),
    outputPath: "$.Payload",
    });
    Original
    AWS Step Functions Service Integration

    View Slide

  27. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    AWS Lambda
    28
    const result = await rekognition.detectLabels({
    Bucket: event.s3Bucket, Name: event.imageName }
    AWS CDK
    const detectObjectLambda = new lambda.Function(this, {…});
    Glue Code としての Lambda
    Step Functions workflow だけで完結しない複雑さ
    Lambda コードにトポロジーが隠蔽されてしまう
    Original
    AWS Step Functions Service Integration

    View Slide

  28. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    AWS Lambda
    29
    const handler = async (event) => {
    const result = await rekognition.detectLabels({
    Image: {
    S3Object: {
    Bucket: event.s3Bucket, Name: event.imageName }
    },
    });
    };
    AWS CDK
    const detectObjectLambda = new lambda.Function(this, {…});
    const detectObject = new tasks.LambdaInvoke(this, "Detect Object", {
    lambdaFunction: detectObjectLambda,
    payload: sfn.TaskInput.fromObject({
    s3Bucket: imageBucket.bucketName,
    imageName: this.IMAGE_TO_LABEL,
    }),
    outputPath: "$.Payload",
    });
    AWS CDK
    const detectObject = new tasks.CallAwsService(this,””, {
    service: "rekognition",
    action: "detectLabels",
    parameters: {
    Image: {
    S3Object: {
    Bucket: imageBucket.bucketName,
    Name: this.IMAGE_TO_LABEL,
    },
    },
    },

    });
    AWS Lambda
    不要
    Original Refactored
    AWS Step Functions Service Integration

    View Slide

  29. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Serverless Refactoring Pattern
    30
    Extract send message
    (using Lambda Destination)
    AWS Step Functions
    Service Integration
    Send Message via
    EventBridge Pipes
    02 03
    01

    View Slide

  30. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Send Message via EventBridge Pipes
    31
    DynamoDB にドメインイベントを保存してから、 EventBridge にイベントを送信する
    Original
    Lambda
    Application
    EventBridge
    DynamoDB
    Lambda
    Function
    Table
    Event
    Custom
    event bus
    Integration
    ・トポロジーが環境変数に隠れている(DynamoDB テーブル名, Event Bus 名など)
    Rollback
    どうする?
    ・複数サービスへの通信が、同じトランザクションスコープではない

    View Slide

  31. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Send Message via EventBridge Pipes
    32
    Stream Pipes
    Original
    Lambda
    Application
    EventBridge
    Integration
    DynamoDB
    Lambda
    Function
    Table Event Custom
    event bus
    Refactored
    EventBridge
    Lambda
    Application
    EventBridge
    DynamoDB
    Lambda
    Function
    Table
    Event
    Custom
    event bus
    Integration

    View Slide

  32. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Send Message via EventBridge Pipes
    33
    ・ビジネスロジックの明確化
    Event Bridge への送信ロジックがアプリケーションコードから排除される
    ・トポロジーの明確化
    イベントの送信先は Lambda Function の環境変数で指定せず
    ⾃動化コード(AWS CDK, CloudFormation など)に委譲される
    ・トランザクションの整合性保証
    Lambda のエラーハンドリングやロールバック実装を簡略化
    ・DynamoDB Streams による dedupe
    同じ Item を put しても DDB Strems は最初の変更のみ Capture する

    View Slide

  33. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Lambda Function の粒度
    34

    View Slide

  34. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    “Lambda-lith”
    35
    AWS Lambda Amazon DynamoDB
    Amazon API Gateway
    Client
    /*
    /create
    /update
    /delete
    ルーティングロジックを
    内包した Function
    ・セキュリティ(IAM Role)
    ・パフォーマンス設定(メモリ、CPU)
    ・クォータ(同時実⾏数など)
    ・スケーリング
    Function 単位に適⽤される設定

    View Slide

  35. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    “Nano Lambda”
    36
    Amazon DynamoDB
    Amazon API Gateway
    Client
    /create
    /update
    /delete

    View Slide

  36. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 37
    https://speakerdeck.com/_kensh/monolith-first-serverless-development
    AWS Dev Day 2023

    View Slide

  37. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    delete.js
    “Nano Lambda”
    38
    Amazon DynamoDB
    Amazon API Gateway
    Client
    /create
    /update
    /delete
    import { DynamoDB } from "aws-sdk";
    const dynamodb = new DynamoDB();
    export const handler = async (event: any) => {
    const params = {
    TableName: "serverless-days-tokyo-2023",
    Key: {
    id: { S: event.id },
    },
    };
    try {
    const data = await dynamodb.getItem(params).promise();
    return { body: JSON.stringify(data) };
    } catch (err) {
    return { error: err };
    }
    };
    update.js
    import { DynamoDB } from "aws-sdk";
    const dynamodb = new DynamoDB();
    export const handler = async (event: any) => {
    const params = {
    TableName: "serverless-days-tokyo-2023",
    Key: {
    id: { S: event.id },
    },
    };
    try {
    const data = await dynamodb.getItem(params).promise();
    return { body: JSON.stringify(data) };
    } catch (err) {
    return { error: err };
    }
    };
    create.js
    import { DynamoDB } from "aws-sdk";
    const dynamodb = new DynamoDB();
    export const handler = async (event: any) => {
    const params = {
    TableName: "serverless-days-tokyo-2023",
    Key: {
    id: { S: event.id },
    name: { S: event.name },
    },
    };
    try {
    const data = await dynamodb.putItem(params).promise();
    return { body: JSON.stringify(data) };
    } catch (err) {
    return { error: err };
    }
    };

    View Slide

  38. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    API Gateway to Step Functions
    39
    AWS Step Functions Express Workflow を REST API に活⽤する
    start
    What’s method?
    End
    Choice state
    DynamoDB
    GET
    DynamoDB
    PUT
    DynamoDB
    POST
    DynamoDB
    DELETE
    AWS Step Functions Express workflow
    Amazon API Gateway
    Client
    sync
    ・ワークフローとロジックの可視化
    ・Lambda ランタイムの削減
    ・コスト効率の向上
    Express workflow は
    同期的なレスポンスが可能

    View Slide

  39. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Serverlesspresso
    40
    AWS Workshop 🚀
    サーバーレスで構築されたイベント駆動型のコーヒー注⽂アプリ

    View Slide

  40. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Serverlesspresso Order Manager workflow
    41
    Order Manager API
    Mobile Order
    App
    Barista
    App
    AWS Workshop 🚀
    オーダー管理アプリを Step Functions で実装しよう

    View Slide

  41. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    まとめ
    42

    View Slide

  42. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    まとめ
    ・単なる “Gule Code” としての Lambda をやめることで
    AWS プラットフォームの恩恵を最⼤化する
    ・AWS CDK のような⾃動化コードでトポロジーを表現することで
    役割を明確化し、変更に柔軟なアーキテクチャにしていく
    ・Lambda Function の粒度を⾒つめ直す
    “Nano Lambda” が唯⼀解ではない、”Lambda-lith” は必ずしも悪ではない
    43

    View Slide

  43. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Serverlessland – Refactoring to Serverless
    44
    https://serverlessland.com/content/guides/refactoring-serverless/introduction

    View Slide

  44. SERVERLESS DAYS TOKYO 2023 – REFACTORING SERVERLESS
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Thank you!
    © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    淡路 ⼤輔
    アマゾン ウェブ サービス ジャパン合同会社
    ソリューションアーキテクト
    @gee0awa

    View Slide