PyCon JP 2020 ( https://pycon.jp/2020/en/ ) の登壇資料
https://pycon.jp/2020/en/timetable/?id=203588
(2020/08/30 追記) Qiita 補足ブログ書きました https://qiita.com/hassaku_63/items/875ba4694538c6484526
Python × AWS × Serverlessॳֶऀ͕࣍ͷҰาΛ౿Έग़ͨ͢ΊͷςΫχοΫPython & AWS & Serverless - Step to the next stage from a beginnerPyCon JP 2020#pyconjp_2@hassaku
View Slide
͡Ίʹ
Speakerhassaku (Twitter: @hassaku_63)Serverworks Co., Ltd.
Speaker࣮ફ AWS CDK - TypeScript Ͱ ΠϯϑϥΞϓϦʂڞஶ͠·ͨ͠ʢٕज़ॻయ8ʣBOOTH: https://booth.pm/ja/items/1881928
Serverworks• AWS ಛԽͷ SIer• ʮΫϥυͰɺੈքΛɺͬͱɺͨΒ͖͘͢ʯ• AWSύʔτφʔωοτϫʔΫ ʮϓϨϛΞίϯαϧςΟϯάύʔτφʔʯೝఆ
ൃදͷલʹ - ͓͜ͱΘΓ -• ࠂࡁΈͷΞδΣϯμ͔Βएׯͷมߋ͕͋Γ·͢ʢݟղͷมԽʹ͏ͷʣ• ఆಡऀ• AWS ʹ͍ͭͯɺଟগͷར༻ܦݧ͕͋Δ• ʮαʔόʔϨεʯͷ֓೦ɺྑ͞ͳΜͱͳ͍ͬͯ͘Δ• Lambda Function ΛσϓϩΠͨ͜͠ͱ͕͋Δʗ֓೦Λཧղ͍ͯ͠Δ• ʮαʔόʔϨεʯೖهࣄͷ࣍ʹԿΛͨ͠Βྑ͍ͷ͔Θ͔Βͳ͍• ԶͨͪงғؾͰαʔόʔϨεΛ͍ͬͯΔ
Agenda1. AWSʹ͓͚ΔαʔόʔϨεΞϓϦέʔγϣϯߏஙͷجຊ2. Serverless Framework جຊͷ͓͞Β͍3. ϓϩδΣΫτߏͷྫ4. ʮεςʔδʯͷཧ5. ϩΪϯά6. ·ͱΊ
AWSʹ͓͚ΔαʔόʔϨε ΞϓϦέʔγϣϯߏஙͷجຊ
αʔόʔϨεͷԿ͕͍͠ʁ• ֶशίετͷߴ͞• ֓೦ʹೃછΈ͕ͳ͍• ʮೖॻͷ࣍ʯ
αʔόʔϨεͳʮࢥߟʯ• ඇಉظ• ΠϕϯτυϦϒϯ• εςʔτϨε
αʔόʔϨεͳొਓ• ϒϩʔΧʔ• Πϕϯτϋϯυϥ• σʔλετΞCSPLFSFWFOUIBOEMFS%BUBTUPSFCSPLFSe.g SQS, SNS, Kinesise.g DynamoDBLambda
See also• [PyCon JP 2017] The theory of Serverless development by Python (ཧ͔ΒֶͿPythonʹΑΔαʔόϨε։ൃ) @Masashi Terui
Serverless Framework جຊͷ͓͞Β͍
Serverless Framework• OSSͷσϓϩΠπʔϧ• Infrastructure as Code (IaC)• ϚϧνΫϥυରԠ• Serverless Frameworkͷ͍ํ·ͱΊhttps://www.serverless.com/
Serverless Framework1. ϓϩδΣΫτ࡞2. ίʔυΛॻ͘ (yaml, python)3. σϓϩΠύοέʔδΛ࡞4. σϓϩΠ… sls create… sls deploy (or sls package)… sls deploy
σϓϩΠͷಈ͖> sls deploy —stage dev
σϓϩΠͷಈ͖> sls deploy —stage dev$MPVE'PSNBUJPOςϯϓϨ-BNCEBσϓϩΠύοέʔδΛ࡞
σϓϩΠͷಈ͖S3 όέοτΛੜ͠ɺ ҎԼΛΞοϓϩʔυɾLambda ͷσϓϩΠύοέʔδ ɾCloudFormation tempalte"84 4Ξοϓϩʔυ
σϓϩΠͷಈ͖ $MPVE'PSNBUJPOσϓϩΠCloudFormation deploy Λ࣮ߦ
Serverless Framework ͷಠࣗ֓೦ ˞σϓϩΠઌڥΛ۠ผ͢ΔͨΊͷ༻్> sls deploy —stage dev> sls deploy —stage prod→ “dev” ༻ͷ CloudFormation ελοΫ͕࡞͞ΕΔ→ “prod” ༻ͷ CloudFormation ελοΫ͕࡞͞ΕΔNote: εςʔδ
See also• [Qiita] Serverless Framework ͷ͍ํ·ͱΊ @horike37
ϓϩδΣΫτߏͷྫ
ྫʣఆظδϣϒͷ࣮ߦ
ྫʣఆظδϣϒͷ࣮ߦδϣϒύϥϝʔλੜ
ྫʣఆظδϣϒͷ࣮ߦδϣϒ࣮ߦج൫δϣϒύϥϝʔλΛ%ZOBNP%#͔Βऔಘ
ྫʣఆظδϣϒͷ࣮ߦservice dirlambda functionsscheduled taskstep functions taskutility libsjinja2 tempalte filestests࣮ྫ: Qiita هࣄࣗಈߋ৽ https://github.com/hassaku63/qiita-autogenaration
σΟϨΫτϦߏͷҙਤʮԿʹؔ͢Δίʔυͳͷ͔ʯͷݟ௨͠Λྑ͔ͨͬͨ͘͠• αʔόʔϨεߏਤ͕ϐλΰϥεΠονʹͳΓ͕ͪ• ී௨ʹϞδϡʔϧׂ͍ͯ͘͠ײ֮ͱ΄ͱΜͲಉ͡ͰOK• ϚΠΫϩαʔϏε͝ͱʹτοϓϨϕϧͷ֊Λ͚Δ• ϚΠΫϩαʔϏε Lambda ͱͦΕҎ֎ͷϞδϡʔϧͰ• lambda functions ʢτϦΨʔ͝ͱʹαϒϞδϡʔϧʹ͚Δʣ• ൚༻Ϟδϡʔϧʗઃఆ༻ͷมཧ
ʮεςʔδʯͷཧ
Γ͍ͨ͜ͱ• ʮεςʔδʯʹґଘ͠ͳ͍ίʔυϕʔεʹ͍ͨ͠• ϩʔΧϧͰؾָʹಈ͔͍ͨ͠
Γ͍ͨ͜ͱ• ʮεςʔδʯʹґଘ͠ͳ͍ίʔυϕʔεʹ͍ͨ͠• ϩʔΧϧͰؾָʹಈ͔͍ͨ͠→ ʮεςʔδʯݻ༗ͳΛ͍͍ײ͡ʹཧ͢Δ→ ϩʔΧϧͰ lambda ͷϩδοΫίʔυΛ࣮ߦ͍ͨ͠ ˞ ίϯςφϕʔεͷϩʔΧϧϥϯλΠϜͷࠓճׂѪ
1. εςʔδʹґଘ͠ͳ͍ίʔυϕʔεʹ͍ͨ͠
εςʔδ͝ͱʹ͕มΘΔͷ• Ϧιʔε໊ͷࢀরɹ• AWS Ϧιʔεͷ Resource Name, ARN• Endpoint (API Gateway, SQS Queue URL)• ΞϓϦέʔγϣϯݻ༗ͷઃఆใɹ• ϩάϨϕϧ• ར༻͢ΔAWSϦιʔεͷAPIࢀর༻ͷ໊લʢARN Ϧιʔε໊ɺURL ͳͲʣ• ଓ͢Δ֎෦αʔϏεͷઃఆ (e.g Slack ͷ௨ઌνϟϯωϧʣ
εςʔδ͝ͱʹ͕มΘΔͷ• Ϧιʔε໊ͷࢀর → ελοΫͷఆٛͰରॲ• AWS Ϧιʔεͷ Resource Name, ARN• Endpoint (API Gateway, SQS Queue URL)• ΞϓϦέʔγϣϯݻ༗ͷઃఆใ → ʢLambda ͷʣίʔυϕʔεͰରॲ• ϩάϨϕϧ• ར༻͢ΔAWSϦιʔεͷAPIࢀর༻ͷ໊લʢARN Ϧιʔε໊ɺURL ͳͲʣ• ଓ͢Δ֎෦αʔϏεͷઃఆ (e.g Slack ͷ௨ઌνϟϯωϧʣ
ڥมʹΑΔεςʔδݻ༗มͷཧେࡶͳཧํɿ• ੩తͳ͔ͭػີใͰͳ͍ͳΒڥม• ੩తͳ͔ͭػີใͳΒ Secret Manager/ Parameter Store• ಈతͳͳΒσʔλετΞ (e.g. DynamoDB)
ڥมʹΑΔεςʔδݻ༗มͷཧ• python-dotenv• .env ϑΝΠϧΛѻ͏ύοέʔδ• serverless-dotenv-plugin• Serverless Framework ϓϥάΠϯ• .env. ϑΝΠϧͷએݴΛελοΫఆٛͰࢀরͰ͖Δ• ϦϦʔε໊ʢͷҰ෦͋Δ͍શ෦ʣͷએݴ• Lambda ڥมʹ͢ → ${provider.environment.XXX}
2. ϩʔΧϧͰؾָʹಈ͔͍ͨ͠
جຊ• લड़ͷ͜ͱ͕Ͱ͖͍ͯΕɺ͋ͱ΄΅ʮςετ༰қੑʯͷ• Lambda handler ͷؔʹϕλॻ͖͗͢͠ͳ͍• ϏδωεϩδοΫ֎෦αʔϏεݺͼग़͠ผϞδϡʔϧʹ• ϋϯυϥͷࣄ event ͷύʔεͳͲɺ࠷ݶʹཹΊΔ• ϩʔΧϧίϯςφϥϯλΠϜͷ༻ࣗવͱΓ͘͢ͳΔ• e.g) SAM Local, DynamoDB Local, localstack
αʔόʔϨεͬΆ͍෦• εςʔδґଘͷڥมͱͯ͠ѻ͏• ཧ͢ΔϞδϡʔϧΛ࡞ͬͯू• e.g) settings.py, constants.py• example: hassaku63/sls-env-example settings.py
Example• https://github.com/hassaku63/sls-env-exampleఆߏਤ
Example• https://github.com/hassaku63/sls-env-exampleఆߏਤ4242VFVF63-Λ͍ͬͯΔඞཁ͕͋Δ
templates.yml
templates.ymlMBNCEBͷڥมʹϦιʔεಛఆʹඞཁͳใΛ͢
settings.py
settings.pyεςʔδ༻ͷEPUFOWΛϩʔυ2VFVF63-ͷಛఆʹඞཁͳมΛϩʔυ
ϩΪϯά
͠ΌΔ͜ͱʢ༧ఆʣ• JSON ϩΨʔΛͬͯߏΛ࣋ͨͤΔ• τϨʔε༻ͷIDΛ͚Δ• σίϨʔλͰڞ௨Խɹ
͠ΌΔ͜ͱʢ༧ఆมߋʣ• JSON ϩΨʔΛͬͯߏΛ࣋ͨͤΔ• τϨʔε༻ͷIDΛ͚Δɹ→ɹߏਤ͕Θ͖ͬͯͨΒΔ• σίϨʔλͰڞ௨Խɹ→ɹ࣮ऀͷΈ࣍ୈ
JSONͰߏԽCloudWatch Logs Insights ͷݕࡧ͕ḿΔ (υοτ۠ΓͰωετͨ͠ϑΟʔϧυʹΞΫηεՄೳ)Amazon CloudWatch Logs - JSON ϩάͷϑΟʔϧυ https://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/logs/CWL_AnalyzeLogData-discoverable-fields.html#CWL_AnalyzeLogData-discoverable-JSON-logs
JSONͰߏԽ࣮ͷબࢶ1. ϩΪϯά࣌ͷ msg Ҿʹඞͣ json.dump() ͔ͯ͠ΒΛ͢2. JSON ϩΨʔΛαϙʔτ͢ΔύοέʔδΛ͏3. ࣗલͰॻ͘1. Logger, Formatter Λܧঝ2. Logging Adapter ͷར༻ʢݕ౼தʣ
JSONͰߏԽLogger ʹٻΊ͍ͨཁ݅ʢࡧதʣ• JSONܗࣜͰग़ྗͰ͖Δ͜ͱ• ҰఆͷߏԽΛڧ੍Ͱ͖Δ͜ͱ• ʢগͳ͘ͱʣτοϓϨϕϧͷJSONϓϩύςΟ͕Ұ؏͍ͯ͠Δ• Lambda ͷ࣮ߦڥใΛউखʹೖͯ͘͠ΕΔ
JSONͰߏԽࢀߟʣLambda Context ͷใ༗༻ͳͷͰϩάͰ׆༻Ͱ͖Δ (doc)• Lambda ໊ؔʗARNʗόʔδϣϯ• LogGroup / LogStream• Invocation timeout ·ͰͷΓ࣌ؒ• etc ..
τϨʔε༻IDͷ༩• αʔόʔϨεߏͰɺ1ͭͷϢʔεέʔε͕ෳͷϚΠΫϩαʔϏεΛލ͍Ͱॲཧ͞ΕΔ• ӡ༻্ͷτϥϒϧγϡʔςΟϯάͰϢʔεέʔε୯ҐͰϩάΛߜΓࠐΈ͍ͨ• αʔϏεԣஅͰ࣮ߦ͝ͱʹҰҙͱͳΔID͕औΕΕGood
τϨʔε༻IDͷ༩• جຊ AWS X-Ray Λ͓͚ͬͯOK• τϨʔε༻IDʹ x-amzn-trace-id Λར༻Մೳ• Serverless Framework ͳΒ༗ޮԽ؆୯• Step Functions Λ͏ߏͩͱҰ೧Γඞཁ• Step Functions X-Ray ʹඇରԠʢ20208݄ݱࡏʣ
τϨʔε༻IDͷ༩ - X-Ray ͷ༗ޮԽ -template.yml ʹ2-3ߦՃ ʢ͓ΘΓʣ
σίϨʔλͰڞ௨ԽLambda ϋϯυϥ༻ͷσίϨʔλΛ࡞ͬͯΞλον͢Δ• e.g) ϋϯυϥͷ࣮ߦલޙͰඞͣϩΪϯάΛڬΉΑ͏ʹ͢Δ• e.g) ෳϨίʔυΛόϥͯ͠σίʔυ͔ͯ͠Β͢ʢSQS/Kinesis τϦΨʔʣ
https://github.com/hassaku63/qiita-autogenaration/blob/master/qp/logs.py༻ྫ (Decorator)
༻ྫ (lambda handler)https://github.com/hassaku63/qiita-autogenaration/blob/master/qp/functions/sfn/publish.py#L17-L28
σίϨʔλͰڞ௨ԽPros• ϋϯυϥͷهड़ྔ͕ݮΓɺຊ࣭తͳϩδοΫ෦͕ڧௐ͞ΕΔ• ൚༻తͳڞ௨ॲཧͷ࠶ར༻• ։ൃϙϦγʔͷڧ੍• ࣮ࡍʹ͜ͷख๏Λ࣮͕ͬͨ͋Δ (e.g: awslabs/aws-lambda-powertools-python)
σίϨʔλͰڞ௨ԽCons• Γ͗͢٫ͬͯίʔυ͕ಡΈͮΒ͘ͳΔʢओ؍ʣ• ଟগ͘ͳͬͯʮԿͯ͠Δ͔ʯ͕ײతʹಡΈԼͤΔͳΒͳ͍ɺͱ͍͏ߟ͑͋Δ• logging ࣌ͷελοΫϑϨʔϜ͕σίϨʔλʹͳͬͯ͠·͏• ※ͨͩ͠ɺΓΑ͏͋Δʢݺͼग़͠ݩͷελοΫϑϨʔϜΛḷΔʗσίϨʔλͷҾͰରԠʣ• ϋϯυϥͷ్தͰϩΪϯά͍ͨ͠έʔεग़ͯ͘Δ
·ͱΊ
·ͱΊ• αʔόʔϨεʹͳͬͯɺΞϓϦέʔγϣϯͷ෦͘͝ී௨ͷ Python• ߏѲ͕Γ͘͢ͳΔΑ͏ʹιʔεΛߏ͢Δ• εςʔδݻ༗ͷύϥϝʔλڥมͳͲͬͯཧ• ϩΪϯά• JSONϩΨʔΛ͍ͭͭɺσʔλߏΛҰ؏ͤ͞Δ• ՄೳͳΒτϨʔεIDΛৼΔ• σίϨʔλʹΑΔڞ௨Խ͓ΈͰ
ʢࢀߟʣใݯ• AWS• AWS ΫϥυαʔϏε׆༻ࢿྉू: https://aws.amazon.com/jp/aws-jp-introduction/• Webinar: https://aws.amazon.com/jp/about-aws/events/webinars/• Connpass: https://aws-serverless.connpass.com/• Serverless community• Connpass: https://serverless.connpass.com/• Serverless Operations• Blog archives: https://serverless.co.jp/blog-archives• Qiita: https://qiita.com/organizations/serverless-operations• Twitter• Yan Cui (@theburningmonk / AWS Serverless Hero): https://twitter.com/theburningmonk• Kensuke Shimokawa (@kensh /AWS Japan Serverless Specialist SA): https://twitter.com/_kensh• Keisuke Nishitani (@Keisuke69 / AWS Japan): https://twitter.com/Keisuke69• Takahiro Horike (@horike37 / Serverless Operations): https://twitter.com/horike37
We here hiring!
hassaku63
codehex
texmeijin
pylapp
afilina
tanakahisateru
faithandbrave
mot_techtalk
palkan
niftycorp
doggy
commojun
nrslib
hannesfritz
garrettdimon
orderedlist
keavy
sferik
eileencodes
addyosmani
holman
kneath
tanoku
62gerente
hatefulcrawdad