Slide 1

Slide 1 text

S3 + Lambda + API Gatewayܳ ഝਊೠ ࢲߡܻझ ࢲ࠺झ ѐߊೞӝ MinJae Kwon (@mingrammer) 2017.08.29 AWSKRUG Architecture (Make a serverless on-demand image resizing service)

Slide 2

Slide 2 text

Name ӂ޹੤ (MinJae Kwon) Nickname @mingrammer Email [email protected] Who Game Server Engineer @ SundayToz Blog https://mingrammer.com Facebook https://facebook.com/mingrammer Github https://github.com/mingrammer Eng Blog https://medium.com/@mingrammer

Slide 3

Slide 3 text

Serverless ୡә੗ܳ ؀࢚ਵ۽ ೤פ׮

Slide 4

Slide 4 text

2. On-demand image resizing 4. Result & Review 3. Building a service with AWS Contents 1. Serverless?

Slide 5

Slide 5 text

Serverless? By Server Engineers Dist EC2 EC2 Cache Dist EC2 EC2 Cache Client Client API Code API Code By AWS

Slide 6

Slide 6 text

Serverless? Dist EC2 EC2 Cache Dist EC2 EC2 Cache Client Client API Code API Code By Server Engineers By AWS have to manage entire architectures and logics focus on logics

Slide 7

Slide 7 text

Serverless? Amazon API Gateway AWS Lambda + Amazon
 S3 + Amazon Kinesis Amazon
 DynamoDB Amazon
 SNS … …

Slide 8

Slide 8 text

On-demand image resizing On-demand image resizing service?

Slide 9

Slide 9 text

On-demand image resizing Resizing Server image resized User Service desired parameters new image

Slide 10

Slide 10 text

On-demand image resizing ੉޷૑ܳ ޷ܻ ࢤࢿ೧ف૑ ঋইب ؽ ׮নೠ ࢎ੉ૉ੄ ੉޷૑ܳ ೙ਃೡ ٸ ࢤࢿೡ ࣻ ੓਺ ੉޷૑ ୊ܻ ো࢑ਸ ݫੋ ۽૒җ ܻ࠙ೡ ࣻ ੓਺ ׮ࣻ੄ ੉޷૑ ୊ܻ ো࢑ী ٮܲ ஹೊ౴ ࢿמ੄ ਬز੸ੋ झாੌ݂

Slide 11

Slide 11 text

On-demand image resizing Why Serverless?

Slide 12

Slide 12 text

On-demand image resizing ੉޷૑ ਃ୒ী ؀ೠ ౟ې೗ ৘ஏ੉ য۰਑ ౠࣻ ӝמਸ ਤೠ ৮੹ೠ ױੌ ੋझఢझ ࢎਊ਷ ࠺ਊ ࠺ബਯ੸੉ۄ ౸ױ ࢲߡ ҙܻ ࠁױ ࢲ࠺झ ѐߊী݅ ૘઺ೞҊ र਺

Slide 13

Slide 13 text

Amazon
 S3 Amazon API Gateway AWS Lambda users resizing Architecture On-demand image resizing

Slide 14

Slide 14 text

Architecture with deploy process On-demand image resizing (advanced) Amazon
 S3 Amazon API Gateway AWS Lambda users resizing AWS IAM deploy Python (boto)

Slide 15

Slide 15 text

Building a service with AWS Amazon
 S3

Slide 16

Slide 16 text

Building a service with AWS Amazon
 S3

Slide 17

Slide 17 text

Building a service with AWS AWS Lambda Amazon
 S3

Slide 18

Slide 18 text

Building a service with AWS AWS Lambda handler def lambda_handler(event, context) Check existence of a resized image Get a resized image Put a resized image Resize an image Generate an thumbnail link Amazon
 S3 resizer code

Slide 19

Slide 19 text

Building a service with AWS Amazon API Gateway AWS Lambda handler def lambda_handler(event, context) Check existence of a resized image Get a resized image Put a resized image Resize an image Generate an thumbnail link Amazon
 S3 resizer code

Slide 20

Slide 20 text

Building a service with AWS Set up IAM Lambda / S3 / API Gatewayܳ ҙܻೡ ࣻ ੓ח ুࣁझ ӂೠਸ ೡ׼

Slide 21

Slide 21 text

Building a service with AWS Set up S3 bucket ੐੄੄ ੉޷૑ ੷੢ ҃۽ ࢤࢿ ੉ ҃۽ীࢲ੄ ӏ஗਷ mingrammer/{sub}/{category}/{filename}

Slide 22

Slide 22 text

Building a service with AWS Set up S3 bucket S3 Bucket Policy ࢸ੿ { "Version": "2008-10-17", "Id": "Policy1335892530063", "Statement": [ { "Sid": "Stmt1335892150622", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::386209384616:root" }, "Action": [ "s3:GetBucketAcl", "s3:GetBucketPolicy" ], "Resource": "arn:aws:s3:::mingrammer" }, { "Sid": "Stmt1335892526596", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::386209384616:root" }, "Action": “s3:PutObject", "Resource": "arn:aws:s3:::mingrammer/*" }, { "Sid": "PublicReadGetObject", "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::mingrammer/*" } ] }

Slide 23

Slide 23 text

Building a service with AWS Write Lambda handler function def lambda_handler(event, context): sub = event.get('sub') category = event.get('category') filename = event.get('filename') width = event.get('width') # Check if the image url with given parameter exists # If the format of image is not supported, redirect to original image with Image(file=original_image['Body']) as img: # If width is not set, get width of original image # Check if the width is valid numeric value # If the thumbnail already exists, redirect to that # Image resizing ] event.get('field') ?

Slide 24

Slide 24 text

Building a service with AWS Write Lambda handler function ../{sub}/{category}/{filename}?width=400 Amazon API Gateway AWS Lambda API Gateway੄ Mapping Templateীࢲ Lambda۽ ౵ۄ޷ఠܳ ֈѹષ

Slide 25

Slide 25 text

Building a service with AWS Write Lambda handler function event.get('field') Amazon API Gateway AWS Lambda Amazon
 S3 Valid? Format check Param check Amazon
 S3 Exists? Resize Save Amazon
 S3 original thumbnail Get object Get object Put object

Slide 26

Slide 26 text

Building a service with AWS Upload Lambda handler function

Slide 27

Slide 27 text

Building a service with AWS Upload Lambda handler function ੉޷૑ ୊ܻܳ ਤೠ wand ۄ੉࠳۞ܻ৬ ೩ٜ۞ ೣࣻо ನೣػ ౵ੌਸ ೣԋ ঑୷ zip -R thumbnailer.zip wand/* generator.py

Slide 28

Slide 28 text

Building a service with AWS Upload Lambda handler function

Slide 29

Slide 29 text

Building a service with AWS Upload Lambda handler function

Slide 30

Slide 30 text

Building a service with AWS Upload Lambda handler function Lambda ೣࣻо ࢿҕ੸ਵ۽ ࢤࢿؽ

Slide 31

Slide 31 text

Building a service with AWS (+) thumbnailerۄח IAM Role੉ Lambdaীࢲ S3ܳ ઁযೡ ࣻ ੓ب۾ IAMীࢲ Policyܳ ୶о೧઻ঠೣ Setup Permissions

Slide 32

Slide 32 text

Building a service with AWS Attach Policyীࢲ AmazonS3FullAccessܳ ୶о Setup Permissions

Slide 33

Slide 33 text

Building a service with AWS Set up API Gateway with Lambda

Slide 34

Slide 34 text

Building a service with AWS Set up API Gateway with Lambda /mingrammer/{sub}/{category}/{filename}੉ۄח ߡఉ ೞਤ ҃۽ ӏ஗ী ݏѱ ܻࣗझܳ ࢤࢿ

Slide 35

Slide 35 text

Building a service with AWS Set up API Gateway with Lambda /mingrammer/{sub}/{category}/{filename}੉ۄח ߡఉ ೞਤ ҃۽ ӏ஗ী ݏѱ ܻࣗझܳ ࢤࢿ ҃۽ ੗୓ܳ Object ఃчਵ۽ ࢸ੿೧ S3۽੄ ૒੽ ੽Ӕҗ زੌೠ ੽Ӕ ߑधਸ ஂೞӝ ਤೣ ( {variable}਷ ҃۽ ߸ࣻܳ ڷೣ )

Slide 36

Slide 36 text

Building a service with AWS Set up API Gateway with Lambda {/filenamae} ҃۽ীࢲ ݫࢲ٘ ࢤࢿਸ ࢶఖೠ റ GET ݫࢲ٘ܳ ࢤࢿ

Slide 37

Slide 37 text

Building a service with AWS Set up API Gateway with Lambda {/filenamae} ҃۽ীࢲ ݫࢲ٘ ࢤࢿਸ ࢶఖೠ റ GET ݫࢲ٘ܳ ࢤࢿ /mingrammer/{sub}/{category}/{filename} ੹୓ ҃۽ী ؀೧ API ݫࢲ٘ܳ ੿੄ೡ ࣻ ੓਺

Slide 38

Slide 38 text

Building a service with AWS Set up API Gateway with Lambda ࢤࢿೠ GET ݫࢲ٘ী ખ ੹ী ݅ٚ thumbnailۄח Lambda ೣࣻܳ োѾ

Slide 39

Slide 39 text

Building a service with AWS Set up API Gateway with Lambda /mingrammer/{sub}/{category}/{filename}੉ۄח API ূ٘ನੋ౟ী Lambda ೣࣻо ࢿҕ੸ਵ۽ োѾؽ

Slide 40

Slide 40 text

Building a service with AWS Set up API Gateway with Lambda ੉޷૑ ܻࢎ੉૚ਸ ਤ೧ [ݫࢲ٘ ਃ୒]ীࢲ widthۄח ౵ۄ޷ఠܳ ߉ਸ ࣻ ੓ب۾ URL (GET) ௪ܻ ౵ۄ޷ఠ ୶о

Slide 41

Slide 41 text

Building a service with AWS Set up API Gateway with Lambda ઁ؀۽ ز੘ೞח૑ ഛੋ ೞӝ ਤ೧ [పझ౟]ী ٜযо ੸੺ೠ ҃۽ ߸ࣻ৬ ௪ܻ ౵ۄ޷ఠܳ ֍Ҋ పझ౟ None ??

Slide 42

Slide 42 text

Building a service with AWS Set up API Gateway with Lambda ҃۽ ߸ࣻܳ ੸੺൤ ࢸ੿೮਺ীب None੉ۄח ч੉ ٜযоҊ ੓਺ API Gatewayীࢲ Lambda۽ context ੉৻੄ ୶о੸ੋ ౵ۄ޷ఠܳ ੹׳ೞӝ ਤ೧ࢶ Mapping Templatesܳ ࢸ੿೧ঠೣ Lambda۽ ੘সਸ ਃ୒ೞӝ ੹ API Gatewayী ٜযয়Ѣա ઓ੤ೞח ߸ٜࣻਸ JSON ഋక۽ ݒೝೞৈ ੹׳

Slide 43

Slide 43 text

Building a service with AWS Set up API Gateway with Lambda [ా೤ ਃ୒]ী ٜযоݶ ࠄޙ ݒೝ ప੉࠶੉ۄח ೦ݾীࢲ ୶оೡ ࣻ ੓਺

Slide 44

Slide 44 text

Building a service with AWS Set up API Gateway with Lambda sub, category, filename, width ౵ۄ޷ఠܳ ߉ই৬ пп JSON ఃчਵ۽ ݒೝೣ

Slide 45

Slide 45 text

Building a service with AWS Set up API Gateway with Lambda ౵ۄ޷ఠٜ੉ ઁ؀۽ ੹׳غয ੿࢚੸ਵ۽ ز੘ೞৈ ߸ജػ ੉޷૑੄ URL੉ ߈ജؽ

Slide 46

Slide 46 text

Building a service with AWS Set up API Gateway with Lambda S3ীࢲ ߸ജػ ੉޷૑ܳ ഛੋೞৈ ઁ؀۽ ੜ ز੘ೣਸ ഛੋ

Slide 47

Slide 47 text

Building a service with AWS Set up API Gateway with Lambda పझ౟ܳ ֈয पઁ۽ ࢎਊೡ ࣻ ੓ب۾ APIܳ ߓನ

Slide 48

Slide 48 text

Building a service with AWS Set up API Gateway with Lambda ݒ਋ ࣚऔѱ ߓನо ৮ܐؽ

Slide 49

Slide 49 text

Building a service with AWS Set up API Gateway with Lambda ߓನػ APIо ੜ ز੘਷ ೞա ઑӘ ੉࢚ೣ .. ࢚ध੸ਵ۽ח ੉޷૑੄ Location ੿ࠁ ؀न Redirectػ पઁ ੉޷૑о ա৬ঠೣ (ਤ੄ Response ഋध਷ Defaultо ইפݴ Lambdaীࢲ ੉޷૑ ߸ജ੉ ՘դ റ, ਤ੄ ഋక۽ Responseܳ ٜ݅য ղ۰ળѢ੐)

Slide 50

Slide 50 text

Building a service with AWS Set up API Gateway with Lambda [ݫࢲ٘ ਽׹]ীࢲ Body۽ ڄয૑ח Locationਵ۽ Redirectionਸ ೞӝ ਤ೧ Locationҗ Status ೻؊ܳ ୶о

Slide 51

Slide 51 text

Building a service with AWS Set up API Gateway with Lambda ӒܻҊ [ా೤ ਽׹]ীࢲ Locationҗ Status ೻؊ী Body۽ ڄয૑ח чٜਸ ݒೝೣ

Slide 52

Slide 52 text

Building a service with AWS Call the Lambda based API APIܳ ੤ߓನ ೠ റ ਗೞח widthчҗ ೣԋ APIܳ ੤ഐ୹ ೞݶ Body ؀न Locationਵ۽ Redirectػ ੉޷૑ܳ ߄۽ ࠅ ࣻ ੓਺

Slide 53

Slide 53 text

Result & Review ੢੼ ױ੼ ࢲߡ ҙܻী ؀ೠ Ҋ޹੉ ೙ਃহয૗ ੹୓੸ੋ ೒۽਋ ࣇস੉ ए਑ ೟ण ࠺ਊ੉ ֫૓ ঋਵա হ૑ب ঋ਺ ୡࠁ ੑ੢ীࢶ ҳഅ ৻੄ AWSী ؀ೠ ӝࠄ੸ੋ ૑ध੉ ਃҳؽ ѐߊ/ߓನ/ࢸ੿੉ ݒ਋ рױೞҊ ࡅܴ

Slide 54

Slide 54 text

хࢎ೤פ׮ MinJae Kwon (@mingrammer) 2017.09.27 AWSKRUG Architecture (Make a serverless on-demand image resizing service)