Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a serverless application with AWS

mingrammer
September 27, 2017

Building a serverless application with AWS

How to build a simple image resizing service based on serverless architecture with AWS

mingrammer

September 27, 2017
Tweet

More Decks by mingrammer

Other Decks in Programming

Transcript

  1. S3 + Lambda + API Gatewayܳ ഝਊೠ ࢲߡܻझ ࢲ࠺झ ѐߊೞӝ
    MinJae Kwon (@mingrammer)
    2017.08.29 AWSKRUG Architecture
    (Make a serverless on-demand image resizing service)

    View Slide

  2. Name
    ӂ޹੤ (MinJae Kwon)
    Nickname
    @mingrammer
    Email
    [email protected]
    Who
    Game Server Engineer @ SundayToz
    Blog
    https://mingrammer.com
    Facebook
    https://facebook.com/mingrammer
    Github
    https://github.com/mingrammer
    Eng Blog
    https://medium.com/@mingrammer

    View Slide

  3. Serverless ୡә੗ܳ ؀࢚ਵ۽ ೤פ׮

    View Slide

  4. 2. On-demand image resizing
    4. Result & Review
    3. Building a service with AWS
    Contents
    1. Serverless?

    View Slide

  5. Serverless?
    By Server Engineers
    Dist
    EC2
    EC2
    Cache
    Dist
    EC2
    EC2
    Cache
    Client
    Client
    API Code
    API Code
    By AWS

    View Slide

  6. Serverless?
    Dist
    EC2
    EC2
    Cache
    Dist
    EC2
    EC2
    Cache
    Client
    Client
    API Code
    API Code
    By Server Engineers
    By AWS
    have to manage entire architectures and logics
    focus on logics

    View Slide

  7. Serverless?
    Amazon API
    Gateway
    AWS
    Lambda
    +
    Amazon

    S3
    +
    Amazon
    Kinesis
    Amazon

    DynamoDB
    Amazon

    SNS


    View Slide

  8. On-demand image resizing
    On-demand image resizing service?

    View Slide

  9. On-demand image resizing
    Resizing
    Server
    image resized
    User Service
    desired parameters
    new
    image

    View Slide

  10. On-demand image resizing
    ੉޷૑ܳ ޷ܻ ࢤࢿ೧ف૑ ঋইب ؽ
    ׮নೠ ࢎ੉ૉ੄ ੉޷૑ܳ ೙ਃೡ ٸ ࢤࢿೡ ࣻ ੓਺
    ੉޷૑ ୊ܻ ো࢑ਸ ݫੋ ۽૒җ ܻ࠙ೡ ࣻ ੓਺
    ׮ࣻ੄ ੉޷૑ ୊ܻ ো࢑ী ٮܲ ஹೊ౴ ࢿמ੄ ਬز੸ੋ झாੌ݂

    View Slide

  11. On-demand image resizing
    Why Serverless?

    View Slide

  12. On-demand image resizing
    ੉޷૑ ਃ୒ী ؀ೠ ౟ې೗ ৘ஏ੉ য۰਑
    ౠࣻ ӝמਸ ਤೠ ৮੹ೠ ױੌ ੋझఢझ ࢎਊ਷ ࠺ਊ ࠺ബਯ੸੉ۄ ౸ױ
    ࢲߡ ҙܻ ࠁױ ࢲ࠺झ ѐߊী݅ ૘઺ೞҊ र਺

    View Slide

  13. Amazon

    S3
    Amazon API
    Gateway
    AWS
    Lambda
    users
    resizing
    Architecture
    On-demand image resizing

    View Slide

  14. Architecture with deploy process
    On-demand image resizing (advanced)
    Amazon

    S3
    Amazon API
    Gateway
    AWS
    Lambda
    users
    resizing
    AWS IAM
    deploy
    Python (boto)

    View Slide

  15. Building a service with AWS
    Amazon

    S3

    View Slide

  16. Building a service with AWS
    Amazon

    S3

    View Slide

  17. Building a service with AWS
    AWS
    Lambda
    Amazon

    S3

    View Slide

  18. Building a service with AWS
    AWS
    Lambda
    handler
    def lambda_handler(event, context)
    Check existence of a resized image
    Get a resized image
    Put a resized image
    Resize an image
    Generate an thumbnail link
    Amazon

    S3
    resizer code

    View Slide

  19. Building a service with AWS
    Amazon API
    Gateway
    AWS
    Lambda
    handler
    def lambda_handler(event, context)
    Check existence of a resized image
    Get a resized image
    Put a resized image
    Resize an image
    Generate an thumbnail link
    Amazon

    S3
    resizer code

    View Slide

  20. Building a service with AWS
    Set up IAM
    Lambda / S3 / API Gatewayܳ ҙܻೡ ࣻ ੓ח ুࣁझ ӂೠਸ ೡ׼

    View Slide

  21. Building a service with AWS
    Set up S3 bucket
    ੐੄੄ ੉޷૑ ੷੢ ҃۽ ࢤࢿ
    ੉ ҃۽ীࢲ੄ ӏ஗਷ mingrammer/{sub}/{category}/{filename}

    View Slide

  22. Building a service with AWS
    Set up S3 bucket
    S3 Bucket Policy ࢸ੿
    {
    "Version": "2008-10-17",
    "Id": "Policy1335892530063",
    "Statement": [
    {
    "Sid": "Stmt1335892150622",
    "Effect": "Allow",
    "Principal": {
    "AWS": "arn:aws:iam::386209384616:root"
    },
    "Action": [
    "s3:GetBucketAcl",
    "s3:GetBucketPolicy"
    ],
    "Resource": "arn:aws:s3:::mingrammer"
    },
    {
    "Sid": "Stmt1335892526596",
    "Effect": "Allow",
    "Principal": {
    "AWS": "arn:aws:iam::386209384616:root"
    },
    "Action": “s3:PutObject",
    "Resource": "arn:aws:s3:::mingrammer/*"
    },
    {
    "Sid": "PublicReadGetObject",
    "Effect": "Allow",
    "Principal": "*",
    "Action": "s3:GetObject",
    "Resource": "arn:aws:s3:::mingrammer/*"
    }
    ]
    }

    View Slide

  23. Building a service with AWS
    Write Lambda handler function
    def lambda_handler(event, context):
    sub = event.get('sub')
    category = event.get('category')
    filename = event.get('filename')
    width = event.get('width')
    # Check if the image url with given parameter exists
    # If the format of image is not supported, redirect to original image
    with Image(file=original_image['Body']) as img:
    # If width is not set, get width of original image
    # Check if the width is valid numeric value
    # If the thumbnail already exists, redirect to that
    # Image resizing
    ]
    event.get('field') ?

    View Slide

  24. Building a service with AWS
    Write Lambda handler function
    ../{sub}/{category}/{filename}?width=400 Amazon API
    Gateway
    AWS
    Lambda
    API Gateway੄ Mapping Templateীࢲ Lambda۽ ౵ۄ޷ఠܳ ֈѹષ

    View Slide

  25. Building a service with AWS
    Write Lambda handler function
    event.get('field')
    Amazon API
    Gateway
    AWS
    Lambda
    Amazon

    S3
    Valid? Format check Param check
    Amazon

    S3
    Exists? Resize Save
    Amazon

    S3
    original thumbnail
    Get object Get object Put object

    View Slide

  26. Building a service with AWS
    Upload Lambda handler function

    View Slide

  27. Building a service with AWS
    Upload Lambda handler function
    ੉޷૑ ୊ܻܳ ਤೠ wand ۄ੉࠳۞ܻ৬ ೩ٜ۞ ೣࣻо ನೣػ ౵ੌਸ ೣԋ ঑୷
    zip -R thumbnailer.zip wand/* generator.py

    View Slide

  28. Building a service with AWS
    Upload Lambda handler function

    View Slide

  29. Building a service with AWS
    Upload Lambda handler function

    View Slide

  30. Building a service with AWS
    Upload Lambda handler function
    Lambda ೣࣻо ࢿҕ੸ਵ۽ ࢤࢿؽ

    View Slide

  31. Building a service with AWS
    (+) thumbnailerۄח IAM Role੉ Lambdaীࢲ
    S3ܳ ઁযೡ ࣻ ੓ب۾ IAMীࢲ Policyܳ ୶о೧઻ঠೣ
    Setup Permissions

    View Slide

  32. Building a service with AWS
    Attach Policyীࢲ AmazonS3FullAccessܳ ୶о
    Setup Permissions

    View Slide

  33. Building a service with AWS
    Set up API Gateway with Lambda

    View Slide

  34. Building a service with AWS
    Set up API Gateway with Lambda
    /mingrammer/{sub}/{category}/{filename}੉ۄח ߡఉ ೞਤ ҃۽ ӏ஗ী ݏѱ ܻࣗझܳ ࢤࢿ

    View Slide

  35. Building a service with AWS
    Set up API Gateway with Lambda
    /mingrammer/{sub}/{category}/{filename}੉ۄח ߡఉ ೞਤ ҃۽ ӏ஗ী ݏѱ ܻࣗझܳ ࢤࢿ
    ҃۽ ੗୓ܳ Object ఃчਵ۽ ࢸ੿೧ S3۽੄ ૒੽ ੽Ӕҗ زੌೠ ੽Ӕ ߑधਸ ஂೞӝ ਤೣ
    ( {variable}਷ ҃۽ ߸ࣻܳ ڷೣ )

    View Slide

  36. Building a service with AWS
    Set up API Gateway with Lambda
    {/filenamae} ҃۽ীࢲ ݫࢲ٘ ࢤࢿਸ ࢶఖೠ റ GET ݫࢲ٘ܳ ࢤࢿ

    View Slide

  37. Building a service with AWS
    Set up API Gateway with Lambda
    {/filenamae} ҃۽ীࢲ ݫࢲ٘ ࢤࢿਸ ࢶఖೠ റ GET ݫࢲ٘ܳ ࢤࢿ
    /mingrammer/{sub}/{category}/{filename} ੹୓ ҃۽ী ؀೧ API ݫࢲ٘ܳ ੿੄ೡ ࣻ ੓਺

    View Slide

  38. Building a service with AWS
    Set up API Gateway with Lambda
    ࢤࢿೠ GET ݫࢲ٘ী ખ ੹ী ݅ٚ thumbnailۄח Lambda ೣࣻܳ োѾ

    View Slide

  39. Building a service with AWS
    Set up API Gateway with Lambda
    /mingrammer/{sub}/{category}/{filename}੉ۄח API ূ٘ನੋ౟ী Lambda ೣࣻо ࢿҕ੸ਵ۽ োѾؽ

    View Slide

  40. Building a service with AWS
    Set up API Gateway with Lambda
    ੉޷૑ ܻࢎ੉૚ਸ ਤ೧ [ݫࢲ٘ ਃ୒]ীࢲ widthۄח ౵ۄ޷ఠܳ ߉ਸ ࣻ ੓ب۾ URL (GET) ௪ܻ ౵ۄ޷ఠ ୶о

    View Slide

  41. Building a service with AWS
    Set up API Gateway with Lambda
    ઁ؀۽ ز੘ೞח૑ ഛੋ ೞӝ ਤ೧ [పझ౟]ী ٜযо ੸੺ೠ ҃۽ ߸ࣻ৬ ௪ܻ ౵ۄ޷ఠܳ ֍Ҋ పझ౟
    None ??

    View Slide

  42. Building a service with AWS
    Set up API Gateway with Lambda
    ҃۽ ߸ࣻܳ ੸੺൤ ࢸ੿೮਺ীب None੉ۄח ч੉ ٜযоҊ ੓਺
    API Gatewayীࢲ Lambda۽ context ੉৻੄ ୶о੸ੋ ౵ۄ޷ఠܳ ੹׳ೞӝ ਤ೧ࢶ
    Mapping Templatesܳ ࢸ੿೧ঠೣ
    Lambda۽ ੘সਸ ਃ୒ೞӝ ੹ API Gatewayী ٜযয়Ѣա ઓ੤ೞח ߸ٜࣻਸ JSON ഋక۽ ݒೝೞৈ ੹׳

    View Slide

  43. Building a service with AWS
    Set up API Gateway with Lambda
    [ా೤ ਃ୒]ী ٜযоݶ ࠄޙ ݒೝ ప੉࠶੉ۄח ೦ݾীࢲ ୶оೡ ࣻ ੓਺

    View Slide

  44. Building a service with AWS
    Set up API Gateway with Lambda
    sub, category, filename, width ౵ۄ޷ఠܳ ߉ই৬ пп JSON ఃчਵ۽ ݒೝೣ

    View Slide

  45. Building a service with AWS
    Set up API Gateway with Lambda
    ౵ۄ޷ఠٜ੉ ઁ؀۽ ੹׳غয ੿࢚੸ਵ۽ ز੘ೞৈ ߸ജػ ੉޷૑੄ URL੉ ߈ജؽ

    View Slide

  46. Building a service with AWS
    Set up API Gateway with Lambda
    S3ীࢲ ߸ജػ ੉޷૑ܳ ഛੋೞৈ ઁ؀۽ ੜ ز੘ೣਸ ഛੋ

    View Slide

  47. Building a service with AWS
    Set up API Gateway with Lambda
    పझ౟ܳ ֈয पઁ۽ ࢎਊೡ ࣻ ੓ب۾ APIܳ ߓನ

    View Slide

  48. Building a service with AWS
    Set up API Gateway with Lambda
    ݒ਋ ࣚऔѱ ߓನо ৮ܐؽ

    View Slide

  49. Building a service with AWS
    Set up API Gateway with Lambda
    ߓನػ APIо ੜ ز੘਷ ೞա ઑӘ ੉࢚ೣ .. ࢚ध੸ਵ۽ח ੉޷૑੄ Location ੿ࠁ ؀न Redirectػ पઁ ੉޷૑о ա৬ঠೣ
    (ਤ੄ Response ഋध਷ Defaultо ইפݴ Lambdaীࢲ ੉޷૑ ߸ജ੉ ՘դ റ, ਤ੄ ഋక۽ Responseܳ ٜ݅য ղ۰ળѢ੐)

    View Slide

  50. Building a service with AWS
    Set up API Gateway with Lambda
    [ݫࢲ٘ ਽׹]ীࢲ Body۽ ڄয૑ח Locationਵ۽ Redirectionਸ ೞӝ ਤ೧ Locationҗ Status ೻؊ܳ ୶о

    View Slide

  51. Building a service with AWS
    Set up API Gateway with Lambda
    ӒܻҊ [ా೤ ਽׹]ীࢲ Locationҗ Status ೻؊ী Body۽ ڄয૑ח чٜਸ ݒೝೣ

    View Slide

  52. Building a service with AWS
    Call the Lambda based API
    APIܳ ੤ߓನ ೠ റ ਗೞח widthчҗ ೣԋ APIܳ ੤ഐ୹ ೞݶ Body ؀न Locationਵ۽ Redirectػ ੉޷૑ܳ ߄۽ ࠅ ࣻ ੓਺

    View Slide

  53. Result & Review
    ੢੼ ױ੼
    ࢲߡ ҙܻী ؀ೠ Ҋ޹੉ ೙ਃহয૗
    ੹୓੸ੋ ೒۽਋ ࣇস੉ ए਑
    ೟ण ࠺ਊ੉ ֫૓ ঋਵա হ૑ب ঋ਺
    ୡࠁ ੑ੢ীࢶ ҳഅ ৻੄ AWSী ؀ೠ
    ӝࠄ੸ੋ ૑ध੉ ਃҳؽ
    ѐߊ/ߓನ/ࢸ੿੉ ݒ਋ рױೞҊ ࡅܴ

    View Slide

  54. хࢎ೤פ׮
    MinJae Kwon (@mingrammer)
    2017.09.27 AWSKRUG Architecture
    (Make a serverless on-demand image resizing service)

    View Slide