Building a serverless application with AWS

How to build a simple image resizing service based on serverless architecture with AWS

Transcript

1. S3 + Lambda + API Gatewayܳ ഝਊೠ ࢲߡܻझ ࢲ࠺झ ѐߊೞӝ
   MinJae Kwon (@mingrammer)
   2017.08.29 AWSKRUG Architecture
   (Make a serverless on-demand image resizing service)
   

   View Slide

2. Name
   ӂ޹੤ (MinJae Kwon)
   Nickname
   @mingrammer
   Email
   k239507@gmail.com
   Who
   Game Server Engineer @ SundayToz
   Blog
   https://mingrammer.com
   Facebook
   https://facebook.com/mingrammer
   Github
   https://github.com/mingrammer
   Eng Blog
   https://medium.com/@mingrammer
   

   View Slide

3. Serverless ୡә੗ܳ ؀࢚ਵ۽ ೤פ׮
   

   View Slide

4. 2. On-demand image resizing
   4. Result & Review
   3. Building a service with AWS
   Contents
   1. Serverless?
   

   View Slide

5. Serverless?
   By Server Engineers
   Dist
   EC2
   EC2
   Cache
   Dist
   EC2
   EC2
   Cache
   Client
   Client
   API Code
   API Code
   By AWS
   

   View Slide

6. Serverless?
   Dist
   EC2
   EC2
   Cache
   Dist
   EC2
   EC2
   Cache
   Client
   Client
   API Code
   API Code
   By Server Engineers
   By AWS
   have to manage entire architectures and logics
   focus on logics
   

   View Slide

7. Serverless?
   Amazon API
   Gateway
   AWS
   Lambda
   +
   Amazon

   S3
   +
   Amazon
   Kinesis
   Amazon

   DynamoDB
   Amazon

   SNS
   …
   …
   

   View Slide

8. On-demand image resizing
   On-demand image resizing service?
   

   View Slide

9. On-demand image resizing
   Resizing
   Server
   image resized
   User Service
   desired parameters
   new
   image
   

   View Slide

10. On-demand image resizing
    ੉޷૑ܳ ޷ܻ ࢤࢿ೧ف૑ ঋইب ؽ
    ׮নೠ ࢎ੉ૉ੄ ੉޷૑ܳ ೙ਃೡ ٸ ࢤࢿೡ ࣻ ੓਺
    ੉޷૑ ୊ܻ ো࢑ਸ ݫੋ ۽૒җ ܻ࠙ೡ ࣻ ੓਺
    ׮ࣻ੄ ੉޷૑ ୊ܻ ো࢑ী ٮܲ ஹೊ౴ ࢿמ੄ ਬز੸ੋ झாੌ݂
    

    View Slide

11. On-demand image resizing
    Why Serverless?
    

    View Slide

12. On-demand image resizing
    ੉޷૑ ਃ୒ী ؀ೠ ౟ې೗ ৘ஏ੉ য۰਑
    ౠࣻ ӝמਸ ਤೠ ৮੹ೠ ױੌ ੋझఢझ ࢎਊ਷ ࠺ਊ ࠺ബਯ੸੉ۄ ౸ױ
    ࢲߡ ҙܻ ࠁױ ࢲ࠺झ ѐߊী݅ ૘઺ೞҊ र਺
    

    View Slide

13. Amazon

    S3
    Amazon API
    Gateway
    AWS
    Lambda
    users
    resizing
    Architecture
    On-demand image resizing
    

    View Slide

14. Architecture with deploy process
    On-demand image resizing (advanced)
    Amazon

    S3
    Amazon API
    Gateway
    AWS
    Lambda
    users
    resizing
    AWS IAM
    deploy
    Python (boto)
    

    View Slide

15. Building a service with AWS
    Amazon

    S3
    

    View Slide

16. Building a service with AWS
    Amazon

    S3
    

    View Slide

17. Building a service with AWS
    AWS
    Lambda
    Amazon

    S3
    

    View Slide

18. Building a service with AWS
    AWS
    Lambda
    handler
    def lambda_handler(event, context)
    Check existence of a resized image
    Get a resized image
    Put a resized image
    Resize an image
    Generate an thumbnail link
    Amazon

    S3
    resizer code
    

    View Slide

19. Building a service with AWS
    Amazon API
    Gateway
    AWS
    Lambda
    handler
    def lambda_handler(event, context)
    Check existence of a resized image
    Get a resized image
    Put a resized image
    Resize an image
    Generate an thumbnail link
    Amazon

    S3
    resizer code
    

    View Slide

20. Building a service with AWS
    Set up IAM
    Lambda / S3 / API Gatewayܳ ҙܻೡ ࣻ ੓ח ুࣁझ ӂೠਸ ೡ׼
    

    View Slide

21. Building a service with AWS
    Set up S3 bucket
    ੐੄੄ ੉޷૑ ੷੢ ҃۽ ࢤࢿ
    ੉ ҃۽ীࢲ੄ ӏ஗਷ mingrammer/{sub}/{category}/{filename}
    

    View Slide

22. Building a service with AWS
    Set up S3 bucket
    S3 Bucket Policy ࢸ੿
    {
    "Version": "2008-10-17",
    "Id": "Policy1335892530063",
    "Statement": [
    {
    "Sid": "Stmt1335892150622",
    "Effect": "Allow",
    "Principal": {
    "AWS": "arn:aws:iam::386209384616:root"
    },
    "Action": [
    "s3:GetBucketAcl",
    "s3:GetBucketPolicy"
    ],
    "Resource": "arn:aws:s3:::mingrammer"
    },
    {
    "Sid": "Stmt1335892526596",
    "Effect": "Allow",
    "Principal": {
    "AWS": "arn:aws:iam::386209384616:root"
    },
    "Action": “s3:PutObject",
    "Resource": "arn:aws:s3:::mingrammer/*"
    },
    {
    "Sid": "PublicReadGetObject",
    "Effect": "Allow",
    "Principal": "*",
    "Action": "s3:GetObject",
    "Resource": "arn:aws:s3:::mingrammer/*"
    }
    ]
    }
    

    View Slide

23. Building a service with AWS
    Write Lambda handler function
    def lambda_handler(event, context):
    sub = event.get('sub')
    category = event.get('category')
    filename = event.get('filename')
    width = event.get('width')
    # Check if the image url with given parameter exists
    # If the format of image is not supported, redirect to original image
    with Image(file=original_image['Body']) as img:
    # If width is not set, get width of original image
    # Check if the width is valid numeric value
    # If the thumbnail already exists, redirect to that
    # Image resizing
    ]
    event.get('field') ?
    

    View Slide

24. Building a service with AWS
    Write Lambda handler function
    ../{sub}/{category}/{filename}?width=400 Amazon API
    Gateway
    AWS
    Lambda
    API Gateway੄ Mapping Templateীࢲ Lambda۽ ౵ۄ޷ఠܳ ֈѹષ
    

    View Slide

25. Building a service with AWS
    Write Lambda handler function
    event.get('field')
    Amazon API
    Gateway
    AWS
    Lambda
    Amazon

    S3
    Valid? Format check Param check
    Amazon

    S3
    Exists? Resize Save
    Amazon

    S3
    original thumbnail
    Get object Get object Put object
    

    View Slide

26. Building a service with AWS
    Upload Lambda handler function
    

    View Slide

27. Building a service with AWS
    Upload Lambda handler function
    ੉޷૑ ୊ܻܳ ਤೠ wand ۄ੉࠳۞ܻ৬ ೩ٜ۞ ೣࣻо ನೣػ ౵ੌਸ ೣԋ ঑୷
    zip -R thumbnailer.zip wand/* generator.py
    

    View Slide

28. Building a service with AWS
    Upload Lambda handler function
    

    View Slide

29. Building a service with AWS
    Upload Lambda handler function
    

    View Slide

30. Building a service with AWS
    Upload Lambda handler function
    Lambda ೣࣻо ࢿҕ੸ਵ۽ ࢤࢿؽ
    

    View Slide

31. Building a service with AWS
    (+) thumbnailerۄח IAM Role੉ Lambdaীࢲ
    S3ܳ ઁযೡ ࣻ ੓ب۾ IAMীࢲ Policyܳ ୶о೧઻ঠೣ
    Setup Permissions
    

    View Slide

32. Building a service with AWS
    Attach Policyীࢲ AmazonS3FullAccessܳ ୶о
    Setup Permissions
    

    View Slide

33. Building a service with AWS
    Set up API Gateway with Lambda
    

    View Slide

34. Building a service with AWS
    Set up API Gateway with Lambda
    /mingrammer/{sub}/{category}/{filename}੉ۄח ߡఉ ೞਤ ҃۽ ӏ஗ী ݏѱ ܻࣗझܳ ࢤࢿ
    

    View Slide

35. Building a service with AWS
    Set up API Gateway with Lambda
    /mingrammer/{sub}/{category}/{filename}੉ۄח ߡఉ ೞਤ ҃۽ ӏ஗ী ݏѱ ܻࣗझܳ ࢤࢿ
    ҃۽ ੗୓ܳ Object ఃчਵ۽ ࢸ੿೧ S3۽੄ ૒੽ ੽Ӕҗ زੌೠ ੽Ӕ ߑधਸ ஂೞӝ ਤೣ
    ( {variable}਷ ҃۽ ߸ࣻܳ ڷೣ )
    

    View Slide

36. Building a service with AWS
    Set up API Gateway with Lambda
    {/filenamae} ҃۽ীࢲ ݫࢲ٘ ࢤࢿਸ ࢶఖೠ റ GET ݫࢲ٘ܳ ࢤࢿ
    

    View Slide

37. Building a service with AWS
    Set up API Gateway with Lambda
    {/filenamae} ҃۽ীࢲ ݫࢲ٘ ࢤࢿਸ ࢶఖೠ റ GET ݫࢲ٘ܳ ࢤࢿ
    /mingrammer/{sub}/{category}/{filename} ੹୓ ҃۽ী ؀೧ API ݫࢲ٘ܳ ੿੄ೡ ࣻ ੓਺
    

    View Slide

38. Building a service with AWS
    Set up API Gateway with Lambda
    ࢤࢿೠ GET ݫࢲ٘ী ખ ੹ী ݅ٚ thumbnailۄח Lambda ೣࣻܳ োѾ
    

    View Slide

39. Building a service with AWS
    Set up API Gateway with Lambda
    /mingrammer/{sub}/{category}/{filename}੉ۄח API ূ٘ನੋ౟ী Lambda ೣࣻо ࢿҕ੸ਵ۽ োѾؽ
    

    View Slide

40. Building a service with AWS
    Set up API Gateway with Lambda
    ੉޷૑ ܻࢎ੉૚ਸ ਤ೧ [ݫࢲ٘ ਃ୒]ীࢲ widthۄח ౵ۄ޷ఠܳ ߉ਸ ࣻ ੓ب۾ URL (GET) ௪ܻ ౵ۄ޷ఠ ୶о
    

    View Slide

41. Building a service with AWS
    Set up API Gateway with Lambda
    ઁ؀۽ ز੘ೞח૑ ഛੋ ೞӝ ਤ೧ [పझ౟]ী ٜযо ੸੺ೠ ҃۽ ߸ࣻ৬ ௪ܻ ౵ۄ޷ఠܳ ֍Ҋ పझ౟
    None ??
    

    View Slide

42. Building a service with AWS
    Set up API Gateway with Lambda
    ҃۽ ߸ࣻܳ ੸੺൤ ࢸ੿೮਺ীب None੉ۄח ч੉ ٜযоҊ ੓਺
    API Gatewayীࢲ Lambda۽ context ੉৻੄ ୶о੸ੋ ౵ۄ޷ఠܳ ੹׳ೞӝ ਤ೧ࢶ
    Mapping Templatesܳ ࢸ੿೧ঠೣ
    Lambda۽ ੘সਸ ਃ୒ೞӝ ੹ API Gatewayী ٜযয়Ѣա ઓ੤ೞח ߸ٜࣻਸ JSON ഋక۽ ݒೝೞৈ ੹׳
    

    View Slide

43. Building a service with AWS
    Set up API Gateway with Lambda
    [ా೤ ਃ୒]ী ٜযоݶ ࠄޙ ݒೝ ప੉࠶੉ۄח ೦ݾীࢲ ୶оೡ ࣻ ੓਺
    

    View Slide

44. Building a service with AWS
    Set up API Gateway with Lambda
    sub, category, filename, width ౵ۄ޷ఠܳ ߉ই৬ пп JSON ఃчਵ۽ ݒೝೣ
    

    View Slide

45. Building a service with AWS
    Set up API Gateway with Lambda
    ౵ۄ޷ఠٜ੉ ઁ؀۽ ੹׳غয ੿࢚੸ਵ۽ ز੘ೞৈ ߸ജػ ੉޷૑੄ URL੉ ߈ജؽ
    

    View Slide

46. Building a service with AWS
    Set up API Gateway with Lambda
    S3ীࢲ ߸ജػ ੉޷૑ܳ ഛੋೞৈ ઁ؀۽ ੜ ز੘ೣਸ ഛੋ
    

    View Slide

47. Building a service with AWS
    Set up API Gateway with Lambda
    పझ౟ܳ ֈয पઁ۽ ࢎਊೡ ࣻ ੓ب۾ APIܳ ߓನ
    

    View Slide

48. Building a service with AWS
    Set up API Gateway with Lambda
    ݒ਋ ࣚऔѱ ߓನо ৮ܐؽ
    

    View Slide

49. Building a service with AWS
    Set up API Gateway with Lambda
    ߓನػ APIо ੜ ز੘਷ ೞա ઑӘ ੉࢚ೣ .. ࢚ध੸ਵ۽ח ੉޷૑੄ Location ੿ࠁ ؀न Redirectػ पઁ ੉޷૑о ա৬ঠೣ
    (ਤ੄ Response ഋध਷ Defaultо ইפݴ Lambdaীࢲ ੉޷૑ ߸ജ੉ ՘դ റ, ਤ੄ ഋక۽ Responseܳ ٜ݅য ղ۰ળѢ੐)
    

    View Slide

50. Building a service with AWS
    Set up API Gateway with Lambda
    [ݫࢲ٘ ਽׹]ীࢲ Body۽ ڄয૑ח Locationਵ۽ Redirectionਸ ೞӝ ਤ೧ Locationҗ Status ೻؊ܳ ୶о
    

    View Slide

51. Building a service with AWS
    Set up API Gateway with Lambda
    ӒܻҊ [ా೤ ਽׹]ীࢲ Locationҗ Status ೻؊ী Body۽ ڄয૑ח чٜਸ ݒೝೣ
    

    View Slide

52. Building a service with AWS
    Call the Lambda based API
    APIܳ ੤ߓನ ೠ റ ਗೞח widthчҗ ೣԋ APIܳ ੤ഐ୹ ೞݶ Body ؀न Locationਵ۽ Redirectػ ੉޷૑ܳ ߄۽ ࠅ ࣻ ੓਺
    

    View Slide

53. Result & Review
    ੢੼ ױ੼
    ࢲߡ ҙܻী ؀ೠ Ҋ޹੉ ೙ਃহয૗
    ੹୓੸ੋ ೒۽਋ ࣇস੉ ए਑
    ೟ण ࠺ਊ੉ ֫૓ ঋਵա হ૑ب ঋ਺
    ୡࠁ ੑ੢ীࢶ ҳഅ ৻੄ AWSী ؀ೠ
    ӝࠄ੸ੋ ૑ध੉ ਃҳؽ
    ѐߊ/ߓನ/ࢸ੿੉ ݒ਋ рױೞҊ ࡅܴ
    

    View Slide

54. хࢎ೤פ׮
    MinJae Kwon (@mingrammer)
    2017.09.27 AWSKRUG Architecture
    (Make a serverless on-demand image resizing service)
    

    View Slide