Python × AWS × Serverless ॳֶऀ͕࣍ͷҰาΛ౿Έग़ͨ͢ΊͷςΫχοΫ Python & AWS & Serverless - Step to the next stage from a beginner PyCon JP 2020 #pyconjp_2 @hassaku

͸͡Ίʹ

Speaker hassaku (Twitter: @hassaku_63) Serverworks Co., Ltd.

Speaker ࣮ફ AWS CDK - TypeScript Ͱ
 Πϯϑϥ΋ΞϓϦ΋ʂ ڞஶ͠·ͨ͠ʢٕज़ॻయ8ʣ BOOTH: https://booth.pm/ja/items/1881928

Serverworks • AWS ಛԽͷ SIer • ʮΫϥ΢υͰɺੈքΛɺ΋ͬͱɺ͸ͨΒ͖΍͘͢ʯ • AWSύʔτφʔωοτϫʔΫ
 ʮϓϨϛΞίϯαϧςΟϯάύʔτφʔʯೝఆ

ൃදͷલʹ - ͓͜ͱΘΓ - • ࠂ஌ࡁΈͷΞδΣϯμ͔Βएׯͷมߋ͕͋Γ·͢ʢݟղͷมԽʹ൐͏΋ͷʣ • ૝ఆಡऀ • AWS ʹ͍ͭͯɺଟগͷར༻ܦݧ͕͋Δ • ʮαʔόʔϨεʯͷ֓೦΍ɺྑ͞͸ͳΜͱͳ͘஌͍ͬͯΔ • Lambda Function ΛσϓϩΠͨ͜͠ͱ͕͋Δʗ֓೦Λཧղ͍ͯ͠Δ • ʮαʔόʔϨεʯೖ໳هࣄͷ࣍ʹԿΛͨ͠Βྑ͍ͷ͔Θ͔Βͳ͍ • Զͨͪ͸งғؾͰαʔόʔϨεΛ΍͍ͬͯΔ

Agenda 1. AWSʹ͓͚ΔαʔόʔϨεΞϓϦέʔγϣϯߏஙͷجຊ 2. Serverless Framework جຊͷ͓͞Β͍ 3. ϓϩδΣΫτߏ੒ͷྫ 4. ʮεςʔδʯͷ؅ཧ 5. ϩΪϯά 6. ·ͱΊ

AWSʹ͓͚ΔαʔόʔϨε
 ΞϓϦέʔγϣϯߏஙͷجຊ

αʔόʔϨεͷԿ͕೉͍͠ʁ • ֶशίετͷߴ͞ • ֓೦ʹೃછΈ͕ͳ͍ • ʮೖ໳ॻͷ࣍ʯ໰୊

αʔόʔϨεͳʮࢥߟʯ • ඇಉظ • ΠϕϯτυϦϒϯ • εςʔτϨε

αʔόʔϨεͳʮࢥߟʯ • ඇಉظ • ΠϕϯτυϦϒϯ • εςʔτϨε

αʔόʔϨεͳొ৔ਓ෺ • ϒϩʔΧʔ • Πϕϯτϋϯυϥ • σʔλετΞ CSPLFS FWFOU IBOEMFS %BUBTUPSF CSPLFS e.g SQS, SNS, Kinesis e.g DynamoDB Lambda

See also • [PyCon JP 2017] The theory of Serverless development by Python (ཧ࿦ ͔ΒֶͿPythonʹΑΔαʔόϨε։ൃ) @Masashi Terui

Serverless Framework جຊͷ͓͞Β͍

Serverless Framework • OSSͷσϓϩΠπʔϧ • Infrastructure as Code (IaC) • ϚϧνΫϥ΢υରԠ • Serverless Frameworkͷ࢖͍ํ·ͱΊ https://www.serverless.com/

Serverless Framework 1. ϓϩδΣΫτ࡞੒ 2. ίʔυΛॻ͘ (yaml, python) 3. σϓϩΠύοέʔδΛ࡞੒ 4. σϓϩΠ … sls create … sls deploy (or sls package) … sls deploy

σϓϩΠͷಈ͖ > sls deploy —stage dev

σϓϩΠͷಈ͖ > sls deploy —stage dev $MPVE'PSNBUJPOςϯϓϨ -BNCEBσϓϩΠύοέʔδ Λ࡞੒

σϓϩΠͷಈ͖ S3 όέοτΛੜ੒͠ɺ
 ҎԼΛΞοϓϩʔυ ɾLambda ͷσϓϩΠύοέʔδ
 ɾCloudFormation tempalte "84 4 ΁Ξοϓϩʔυ

σϓϩΠͷಈ͖ $MPVE'PSNBUJPO σϓϩΠ CloudFormation deploy Λ࣮ߦ

Serverless Framework ͷಠࣗ֓೦
 ˞σϓϩΠઌ؀ڥΛ۠ผ͢ΔͨΊͷ༻్ > sls deploy —stage dev > sls deploy —stage prod → “dev” ༻ͷ CloudFormation ελοΫ͕࡞੒͞ΕΔ → “prod” ༻ͷ CloudFormation ελοΫ͕࡞੒͞ΕΔ Note: εςʔδ

See also • [Qiita] Serverless Framework ͷ࢖͍ํ·ͱΊ @horike37

ϓϩδΣΫτߏ੒ͷྫ

ྫʣఆظδϣϒͷ࣮ߦ

ྫʣఆظδϣϒͷ࣮ߦ δϣϒύϥϝʔλੜ੒

ྫʣఆظδϣϒͷ࣮ߦ δϣϒ࣮ߦج൫ δϣϒύϥϝʔλΛ%ZOBNP%#͔Βऔಘ

ྫʣఆظδϣϒͷ࣮ߦ service dir lambda functions scheduled task step functions task utility libs jinja2 tempalte files tests ࣮૷ྫ: Qiita هࣄࣗಈߋ৽ https://github.com/hassaku63/qiita-autogenaration

σΟϨΫτϦߏ੒ͷҙਤ ʮԿʹؔ͢Δίʔυͳͷ͔ʯͷݟ௨͠Λྑ͔ͨͬͨ͘͠ • αʔόʔϨε͸ߏ੒ਤ͕ϐλΰϥεΠονʹͳΓ͕ͪ • ී௨ʹϞδϡʔϧ෼ׂ͍ͯ͘͠ײ֮ͱ΄ͱΜͲಉ͡ͰOK • ϚΠΫϩαʔϏε͝ͱʹτοϓϨϕϧͷ֊૚Λ෼͚Δ • ϚΠΫϩαʔϏε಺͸ Lambda ͱͦΕҎ֎ͷϞδϡʔϧͰ෼཭ • lambda functions ʢτϦΨʔ͝ͱʹαϒϞδϡʔϧʹ෼͚Δʣ • ൚༻Ϟδϡʔϧʗઃఆ༻ͷม਺؅ཧ

ʮεςʔδʯͷ؅ཧ

΍Γ͍ͨ͜ͱ • ʮεςʔδʯʹґଘ͠ͳ͍ίʔυϕʔεʹ͍ͨ͠ • ϩʔΧϧͰ΋ؾָʹಈ͔͍ͨ͠

΍Γ͍ͨ͜ͱ • ʮεςʔδʯʹґଘ͠ͳ͍ίʔυϕʔεʹ͍ͨ͠ • ϩʔΧϧͰ΋ؾָʹಈ͔͍ͨ͠ → ʮεςʔδʯݻ༗ͳ஋Λ͍͍ײ͡ʹ؅ཧ͢Δ → ϩʔΧϧͰ΋ lambda ͷϩδοΫίʔυΛ࣮ߦ͍ͨ͠
 ˞ ίϯςφϕʔεͷϩʔΧϧϥϯλΠϜͷ࿩͸ࠓճׂѪ

1. εςʔδʹґଘ͠ͳ͍ίʔυϕʔεʹ͍ͨ͠

εςʔδ͝ͱʹ஋͕มΘΔ΋ͷ • Ϧιʔε໊ͷࢀরɹ • AWS Ϧιʔεͷ Resource Name, ARN • Endpoint (API Gateway, SQS Queue URL) • ΞϓϦέʔγϣϯݻ༗ͷઃఆ৘ใɹ • ϩάϨϕϧ • ར༻͢ΔAWSϦιʔεͷAPIࢀর༻ͷ໊લʢARN ΍ Ϧιʔε໊ɺURL ͳͲʣ • ઀ଓ͢Δ֎෦αʔϏεͷઃఆ (e.g Slack ͷ௨஌ઌνϟϯωϧʣ

εςʔδ͝ͱʹ஋͕มΘΔ΋ͷ • Ϧιʔε໊ͷࢀর → ελοΫͷఆٛͰରॲ • AWS Ϧιʔεͷ Resource Name, ARN • Endpoint (API Gateway, SQS Queue URL) • ΞϓϦέʔγϣϯݻ༗ͷઃఆ৘ใ → ʢLambda ͷʣίʔυϕʔεͰରॲ • ϩάϨϕϧ • ར༻͢ΔAWSϦιʔεͷAPIࢀর༻ͷ໊લʢARN ΍ Ϧιʔε໊ɺURL ͳͲʣ • ઀ଓ͢Δ֎෦αʔϏεͷઃఆ (e.g Slack ͷ௨஌ઌνϟϯωϧʣ

؀ڥม਺ʹΑΔεςʔδݻ༗ม਺ͷ؅ཧ େࡶ೺ͳ؅ཧํ਑ɿ • ੩తͳ஋͔ͭػີ৘ใͰͳ͍ͳΒ؀ڥม਺ • ੩తͳ஋͔ͭػີ৘ใͳΒ Secret Manager/ Parameter Store • ಈతͳ஋ͳΒσʔλετΞ (e.g. DynamoDB)

؀ڥม਺ʹΑΔεςʔδݻ༗ม਺ͷ؅ཧ • python-dotenv • .env ϑΝΠϧΛѻ͏ύοέʔδ • serverless-dotenv-plugin • Serverless Framework ϓϥάΠϯ • .env. ϑΝΠϧͷએݴΛελοΫఆٛ಺ͰࢀরͰ͖Δ • ϦϦʔε໊ʢͷҰ෦͋Δ͍͸શ෦ʣͷએݴ • Lambda ؀ڥม਺ʹ౉͢ → ${provider.environment.XXX}

2. ϩʔΧϧͰ΋ؾָʹಈ͔͍ͨ͠

جຊ • લड़ͷ͜ͱ͕Ͱ͖͍ͯΕ͹ɺ͋ͱ͸΄΅ʮςετ༰қੑʯͷ࿩ • Lambda handler ͷؔ਺ʹϕλॻ͖͗͢͠ͳ͍ • ϏδωεϩδοΫ΍֎෦αʔϏεݺͼग़͠͸ผϞδϡʔϧʹ෼཭ • ϋϯυϥ಺ͷ࢓ࣄ͸ event ͷύʔεͳͲɺ࠷௿ݶʹཹΊΔ • ϩʔΧϧίϯςφϥϯλΠϜͷ࢖༻΋ࣗવͱ΍Γ΍͘͢ͳΔ • e.g) SAM Local, DynamoDB Local, localstack

αʔόʔϨεͬΆ͍෦෼ • εςʔδґଘͷ஋͸؀ڥม਺ͱͯ͠ѻ͏ • ؅ཧ͢ΔϞδϡʔϧΛ࡞ͬͯू໿ • e.g) settings.py, constants.py • example: hassaku63/sls-env-example settings.py

Example • https://github.com/hassaku63/sls-env-example ૝ఆߏ੒ਤ

Example • https://github.com/hassaku63/sls-env-example ૝ఆߏ੒ਤ 4242VFVF63- Λ஌͍ͬͯΔඞཁ͕͋Δ

templates.yml

templates.yml MBNCEBͷ؀ڥม਺ʹ Ϧιʔεಛఆʹ ඞཁͳ৘ใΛ౉͢

settings.py

settings.py εςʔδ༻ͷ EPUFOWΛϩʔυ 2VFVF63-ͷಛఆʹ ඞཁͳม਺Λϩʔυ

ϩΪϯά

͠Ό΂Δ͜ͱʢ༧ఆʣ • JSON ϩΨʔΛ࢖ͬͯߏ଄Λ࣋ͨͤΔ • τϨʔε༻ͷIDΛ෇͚Δ • σίϨʔλͰڞ௨Խɹ

͠Ό΂Δ͜ͱʢ༧ఆมߋʣ • JSON ϩΨʔΛ࢖ͬͯߏ଄Λ࣋ͨͤΔ • τϨʔε༻ͷIDΛ෇͚Δɹ→ɹߏ੒ਤ͕೐Θ͖ͬͯͨΒ΍Δ • σίϨʔλͰڞ௨Խɹ→ɹ࣮૷ऀͷ޷Έ࣍ୈ

JSONͰߏ଄Խ CloudWatch Logs Insights ͷݕࡧ͕ḿΔ
 (υοτ۠੾ΓͰωετͨ͠ϑΟʔϧυʹΞΫηεՄೳ) Amazon CloudWatch Logs - JSON ϩάͷϑΟʔϧυ https://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/logs/CWL_AnalyzeLogData-discoverable-fields.html#CWL_AnalyzeLogData-discoverable-JSON-logs

JSONͰߏ଄Խ ࣮૷ͷબ୒ࢶ 1. ϩΪϯά࣌ͷ msg Ҿ਺ʹ͸ඞͣ json.dump() ͔ͯ͠Β஋Λ౉͢ 2. JSON ϩΨʔΛαϙʔτ͢ΔύοέʔδΛ࢖͏ 3. ࣗલͰॻ͘ 1. Logger, Formatter Λܧঝ 2. Logging Adapter ͷར༻ʢݕ౼தʣ

JSONͰߏ଄Խ Logger ʹٻΊ͍ͨཁ݅ʢ໛ࡧதʣ • JSONܗࣜͰग़ྗͰ͖Δ͜ͱ • Ұఆͷߏ଄ԽΛڧ੍Ͱ͖Δ͜ͱ • ʢগͳ͘ͱ΋ʣτοϓϨϕϧͷJSONϓϩύςΟ͕Ұ؏͍ͯ͠Δ • Lambda ͷ࣮ߦ؀ڥ৘ใΛউखʹ஫ೖͯ͘͠ΕΔ

JSONͰߏ଄Խ ࢀߟʣLambda Context ͷ৘ใ͸༗༻ͳͷͰϩά౳Ͱ׆༻Ͱ͖Δ (doc) • Lambda ؔ਺໊ʗARNʗόʔδϣϯ • LogGroup / LogStream • Invocation timeout ·Ͱͷ࢒Γ࣌ؒ • etc ..

τϨʔε༻IDͷ෇༩ • αʔόʔϨεߏ੒Ͱ͸ɺ1ͭͷϢʔεέʔε͕ෳ਺ͷϚΠΫϩ αʔϏεΛލ͍Ͱॲཧ͞ΕΔ • ӡ༻্ͷτϥϒϧγϡʔςΟϯάͰ͸Ϣʔεέʔε୯ҐͰϩά ΛߜΓࠐΈ͍ͨ • αʔϏεԣஅͰ࣮ߦ͝ͱʹҰҙͱͳΔID͕औΕΕ͹Good

τϨʔε༻IDͷ෇༩ • جຊ͸ AWS X-Ray Λ࢖͓͚ͬͯ͹OK • τϨʔε༻IDʹ x-amzn-trace-id Λར༻Մೳ • Serverless Framework ͳΒ༗ޮԽ͸؆୯ • Step Functions Λ࢖͏ߏ੒ͩͱҰ೧Γඞཁ • Step Functions ͸ X-Ray ʹඇରԠʢ2020೥8݄ݱࡏʣ

τϨʔε༻IDͷ෇༩ - X-Ray ͷ༗ޮԽ - template.yml ʹ2-3ߦ௥Ճ
 ʢ͓ΘΓʣ

σίϨʔλͰڞ௨Խ Lambda ϋϯυϥ༻ͷσίϨʔλΛ࡞ͬͯΞλον͢Δ • e.g) ϋϯυϥͷ࣮ߦલޙͰඞͣϩΪϯάΛڬΉΑ͏ʹ͢Δ • e.g) ෳ਺ϨίʔυΛόϥͯ͠σίʔυ͔ͯ͠Β౉͢ʢSQS/Kinesis τϦΨʔʣ

https://github.com/hassaku63/qiita-autogenaration/blob/master/qp/logs.py ࢖༻ྫ (Decorator)

࢖༻ྫ (lambda handler) https://github.com/hassaku63/qiita-autogenaration/blob/master/qp/functions/sfn/publish.py#L17-L28

σίϨʔλͰڞ௨Խ Pros • ϋϯυϥ಺ͷهड़ྔ͕ݮΓɺຊ࣭తͳϩδοΫ෦෼͕ڧௐ͞ΕΔ • ൚༻తͳڞ௨ॲཧͷ࠶ར༻ • ։ൃϙϦγʔͷڧ੍ • ࣮ࡍʹ͜ͷख๏Λ࢖࣮ͬͨ૷͕͋Δ (e.g: awslabs/aws-lambda-powertools-python)

σίϨʔλͰڞ௨Խ Cons • ΍Γ͗͢͸٫ͬͯίʔυ͕ಡΈͮΒ͘ͳΔʢओ؍ʣ • ଟগ௕͘ͳͬͯ΋ʮԿͯ͠Δ͔ʯ͕௚ײతʹಡΈԼͤΔͳΒ໰୊ͳ͍ɺͱ͍͏ߟ͑΋͋Δ • logging ࣌ͷελοΫϑϨʔϜ͕σίϨʔλʹͳͬͯ͠·͏ • ※ͨͩ͠ɺ΍ΓΑ͏͸͋Δʢݺͼग़͠ݩͷελοΫϑϨʔϜΛḷΔʗσίϨʔλͷҾ਺ͰରԠʣ • ϋϯυϥͷ్தͰ΋ϩΪϯά͍ͨ͠έʔε͸ग़ͯ͘Δ

·ͱΊ

·ͱΊ • αʔόʔϨεʹͳͬͯ΋ɺΞϓϦέʔγϣϯͷ෦෼͸͘͝ී௨ͷ Python • ߏ੒೺Ѳ͕΍Γ΍͘͢ͳΔΑ͏ʹιʔεΛߏ੒͢Δ • εςʔδݻ༗ͷύϥϝʔλ͸؀ڥม਺ͳͲ࢖ͬͯ؅ཧ • ϩΪϯά • JSONϩΨʔΛ࢖͍ͭͭɺσʔλߏ଄ΛҰ؏ͤ͞Δ • ՄೳͳΒτϨʔεIDΛৼΔ • σίϨʔλʹΑΔڞ௨Խ͸͓޷ΈͰ

ʢࢀߟʣ৘ใݯ • AWS • AWS Ϋϥ΢υαʔϏε׆༻ࢿྉू: https://aws.amazon.com/jp/aws-jp-introduction/ • Webinar: https://aws.amazon.com/jp/about-aws/events/webinars/ • Connpass: https://aws-serverless.connpass.com/ • Serverless community • Connpass: https://serverless.connpass.com/ • Serverless Operations • Blog archives: https://serverless.co.jp/blog-archives • Qiita: https://qiita.com/organizations/serverless-operations • Twitter • Yan Cui (@theburningmonk / AWS Serverless Hero): https://twitter.com/theburningmonk • Kensuke Shimokawa (@kensh /AWS Japan Serverless Specialist SA): https://twitter.com/_kensh • Keisuke Nishitani (@Keisuke69 / AWS Japan): https://twitter.com/Keisuke69 • Takahiro Horike (@horike37 / Serverless Operations): https://twitter.com/horike37

We here hiring!