Python × AWS × Serverless 初学者が次の一歩を踏み出すためのテクニック/Python & AWS & Serverless - Step to the next stage from a beginner

Slide 1

Slide 1 text

Python × AWS × Serverless ॳֶऀ͕࣍ͷҰาΛ౿Έग़ͨ͢ΊͷςΫχοΫ Python & AWS & Serverless - Step to the next stage from a beginner PyCon JP 2020 #pyconjp_2 @hassaku

Slide 2

Slide 2 text

͸͡Ίʹ

Slide 3

Slide 3 text

Speaker hassaku (Twitter: @hassaku_63) Serverworks Co., Ltd.

Slide 4

Slide 4 text

Speaker ࣮ફ AWS CDK - TypeScript Ͱ  Πϯϑϥ΋ΞϓϦ΋ʂ ڞஶ͠·ͨ͠ʢٕज़ॻయ8ʣ BOOTH: https://booth.pm/ja/items/1881928

Slide 5

Slide 5 text

Serverworks • AWS ಛԽͷ SIer • ʮΫϥ΢υͰɺੈքΛɺ΋ͬͱɺ͸ͨΒ͖΍͘͢ʯ • AWSύʔτφʔωοτϫʔΫ  ʮϓϨϛΞίϯαϧςΟϯάύʔτφʔʯೝఆ

Slide 6

Slide 6 text

ൃදͷલʹ - ͓͜ͱΘΓ - • ࠂ஌ࡁΈͷΞδΣϯμ͔Βएׯͷมߋ͕͋Γ·͢ʢݟղͷมԽʹ൐͏΋ͷʣ • ૝ఆಡऀ • AWS ʹ͍ͭͯɺଟগͷར༻ܦݧ͕͋Δ • ʮαʔόʔϨεʯͷ֓೦΍ɺྑ͞͸ͳΜͱͳ͘஌͍ͬͯΔ • Lambda Function ΛσϓϩΠͨ͜͠ͱ͕͋Δʗ֓೦Λཧղ͍ͯ͠Δ • ʮαʔόʔϨεʯೖ໳هࣄͷ࣍ʹԿΛͨ͠Βྑ͍ͷ͔Θ͔Βͳ͍ • Զͨͪ͸งғؾͰαʔόʔϨεΛ΍͍ͬͯΔ

Slide 7

Slide 7 text

Agenda 1. AWSʹ͓͚ΔαʔόʔϨεΞϓϦέʔγϣϯߏஙͷجຊ 2. Serverless Framework جຊͷ͓͞Β͍ 3. ϓϩδΣΫτߏ੒ͷྫ 4. ʮεςʔδʯͷ؅ཧ 5. ϩΪϯά 6. ·ͱΊ

Slide 8

Slide 8 text

AWSʹ͓͚ΔαʔόʔϨε  ΞϓϦέʔγϣϯߏஙͷجຊ

Slide 9

Slide 9 text

αʔόʔϨεͷԿ͕೉͍͠ʁ • ֶशίετͷߴ͞ • ֓೦ʹೃછΈ͕ͳ͍ • ʮೖ໳ॻͷ࣍ʯ໰୊

Slide 10

Slide 10 text

αʔόʔϨεͳʮࢥߟʯ • ඇಉظ • ΠϕϯτυϦϒϯ • εςʔτϨε

Slide 11

Slide 11 text

αʔόʔϨεͳʮࢥߟʯ • ඇಉظ • ΠϕϯτυϦϒϯ • εςʔτϨε

Slide 12

Slide 12 text

αʔόʔϨεͳొ৔ਓ෺ • ϒϩʔΧʔ • Πϕϯτϋϯυϥ • σʔλετΞ CSPLFS FWFOU IBOEMFS %BUBTUPSF CSPLFS e.g SQS, SNS, Kinesis e.g DynamoDB Lambda

Slide 13

Slide 13 text

See also • [PyCon JP 2017] The theory of Serverless development by Python (ཧ࿦ ͔ΒֶͿPythonʹΑΔαʔόϨε։ൃ) @Masashi Terui

Slide 14

Slide 14 text

Serverless Framework جຊͷ͓͞Β͍

Slide 15

Slide 15 text

Serverless Framework • OSSͷσϓϩΠπʔϧ • Infrastructure as Code (IaC) • ϚϧνΫϥ΢υରԠ • Serverless Frameworkͷ࢖͍ํ·ͱΊ https://www.serverless.com/

Slide 16

Slide 16 text

Serverless Framework 1. ϓϩδΣΫτ࡞੒ 2. ίʔυΛॻ͘ (yaml, python) 3. σϓϩΠύοέʔδΛ࡞੒ 4. σϓϩΠ … sls create … sls deploy (or sls package) … sls deploy

Slide 17

Slide 17 text

σϓϩΠͷಈ͖ > sls deploy —stage dev

Slide 18

Slide 18 text

σϓϩΠͷಈ͖ > sls deploy —stage dev $MPVE'PSNBUJPOςϯϓϨ -BNCEBσϓϩΠύοέʔδ Λ࡞੒

Slide 19

Slide 19 text

σϓϩΠͷಈ͖ S3 όέοτΛੜ੒͠ɺ  ҎԼΛΞοϓϩʔυ ɾLambda ͷσϓϩΠύοέʔδ  ɾCloudFormation tempalte "84 4 ΁Ξοϓϩʔυ

Slide 20

Slide 20 text

σϓϩΠͷಈ͖ $MPVE'PSNBUJPO σϓϩΠ CloudFormation deploy Λ࣮ߦ

Slide 21

Slide 21 text

Serverless Framework ͷಠࣗ֓೦  ˞σϓϩΠઌ؀ڥΛ۠ผ͢ΔͨΊͷ༻్ > sls deploy —stage dev > sls deploy —stage prod → “dev” ༻ͷ CloudFormation ελοΫ͕࡞੒͞ΕΔ → “prod” ༻ͷ CloudFormation ελοΫ͕࡞੒͞ΕΔ Note: εςʔδ

Slide 22

Slide 22 text

See also • [Qiita] Serverless Framework ͷ࢖͍ํ·ͱΊ @horike37

Slide 23

Slide 23 text

ϓϩδΣΫτߏ੒ͷྫ

Slide 24

Slide 24 text

ྫʣఆظδϣϒͷ࣮ߦ

Slide 25

Slide 25 text

ྫʣఆظδϣϒͷ࣮ߦ δϣϒύϥϝʔλੜ੒

Slide 26

Slide 26 text

ྫʣఆظδϣϒͷ࣮ߦ δϣϒ࣮ߦج൫ δϣϒύϥϝʔλΛ%ZOBNP%#͔Βऔಘ

Slide 27

Slide 27 text

ྫʣఆظδϣϒͷ࣮ߦ service dir lambda functions scheduled task step functions task utility libs jinja2 tempalte ﬁles tests ࣮૷ྫ: Qiita هࣄࣗಈߋ৽ https://github.com/hassaku63/qiita-autogenaration

Slide 28

Slide 28 text

σΟϨΫτϦߏ੒ͷҙਤ ʮԿʹؔ͢Δίʔυͳͷ͔ʯͷݟ௨͠Λྑ͔ͨͬͨ͘͠ • αʔόʔϨε͸ߏ੒ਤ͕ϐλΰϥεΠονʹͳΓ͕ͪ • ී௨ʹϞδϡʔϧ෼ׂ͍ͯ͘͠ײ֮ͱ΄ͱΜͲಉ͡ͰOK • ϚΠΫϩαʔϏε͝ͱʹτοϓϨϕϧͷ֊૚Λ෼͚Δ • ϚΠΫϩαʔϏε಺͸ Lambda ͱͦΕҎ֎ͷϞδϡʔϧͰ෼཭ • lambda functions ʢτϦΨʔ͝ͱʹαϒϞδϡʔϧʹ෼͚Δʣ • ൚༻Ϟδϡʔϧʗઃఆ༻ͷม਺؅ཧ

Slide 29

Slide 29 text

ʮεςʔδʯͷ؅ཧ

Slide 30

Slide 30 text

΍Γ͍ͨ͜ͱ • ʮεςʔδʯʹґଘ͠ͳ͍ίʔυϕʔεʹ͍ͨ͠ • ϩʔΧϧͰ΋ؾָʹಈ͔͍ͨ͠

Slide 31

Slide 31 text

΍Γ͍ͨ͜ͱ • ʮεςʔδʯʹґଘ͠ͳ͍ίʔυϕʔεʹ͍ͨ͠ • ϩʔΧϧͰ΋ؾָʹಈ͔͍ͨ͠ → ʮεςʔδʯݻ༗ͳ஋Λ͍͍ײ͡ʹ؅ཧ͢Δ → ϩʔΧϧͰ΋ lambda ͷϩδοΫίʔυΛ࣮ߦ͍ͨ͠  ˞ ίϯςφϕʔεͷϩʔΧϧϥϯλΠϜͷ࿩͸ࠓճׂѪ

Slide 32

Slide 32 text

1. εςʔδʹґଘ͠ͳ͍ίʔυϕʔεʹ͍ͨ͠

Slide 33

Slide 33 text

εςʔδ͝ͱʹ஋͕มΘΔ΋ͷ • Ϧιʔε໊ͷࢀরɹ • AWS Ϧιʔεͷ Resource Name, ARN • Endpoint (API Gateway, SQS Queue URL) • ΞϓϦέʔγϣϯݻ༗ͷઃఆ৘ใɹ • ϩάϨϕϧ • ར༻͢ΔAWSϦιʔεͷAPIࢀর༻ͷ໊લʢARN ΍ Ϧιʔε໊ɺURL ͳͲʣ • ઀ଓ͢Δ֎෦αʔϏεͷઃఆ (e.g Slack ͷ௨஌ઌνϟϯωϧʣ

Slide 34

Slide 34 text

εςʔδ͝ͱʹ஋͕มΘΔ΋ͷ • Ϧιʔε໊ͷࢀর → ελοΫͷఆٛͰରॲ • AWS Ϧιʔεͷ Resource Name, ARN • Endpoint (API Gateway, SQS Queue URL) • ΞϓϦέʔγϣϯݻ༗ͷઃఆ৘ใ → ʢLambda ͷʣίʔυϕʔεͰରॲ • ϩάϨϕϧ • ར༻͢ΔAWSϦιʔεͷAPIࢀর༻ͷ໊લʢARN ΍ Ϧιʔε໊ɺURL ͳͲʣ • ઀ଓ͢Δ֎෦αʔϏεͷઃఆ (e.g Slack ͷ௨஌ઌνϟϯωϧʣ

Slide 35

Slide 35 text

؀ڥม਺ʹΑΔεςʔδݻ༗ม਺ͷ؅ཧ େࡶ೺ͳ؅ཧํ਑ɿ • ੩తͳ஋͔ͭػີ৘ใͰͳ͍ͳΒ؀ڥม਺ • ੩తͳ஋͔ͭػີ৘ใͳΒ Secret Manager/ Parameter Store • ಈతͳ஋ͳΒσʔλετΞ (e.g. DynamoDB)

Slide 36

Slide 36 text

؀ڥม਺ʹΑΔεςʔδݻ༗ม਺ͷ؅ཧ • python-dotenv • .env ϑΝΠϧΛѻ͏ύοέʔδ • serverless-dotenv-plugin • Serverless Framework ϓϥάΠϯ • .env. ϑΝΠϧͷએݴΛελοΫఆٛ಺ͰࢀরͰ͖Δ • ϦϦʔε໊ʢͷҰ෦͋Δ͍͸શ෦ʣͷએݴ • Lambda ؀ڥม਺ʹ౉͢ → ${provider.environment.XXX}

Slide 37

Slide 37 text

2. ϩʔΧϧͰ΋ؾָʹಈ͔͍ͨ͠

Slide 38

Slide 38 text

جຊ • લड़ͷ͜ͱ͕Ͱ͖͍ͯΕ͹ɺ͋ͱ͸΄΅ʮςετ༰қੑʯͷ࿩ • Lambda handler ͷؔ਺ʹϕλॻ͖͗͢͠ͳ͍ • ϏδωεϩδοΫ΍֎෦αʔϏεݺͼग़͠͸ผϞδϡʔϧʹ෼཭ • ϋϯυϥ಺ͷ࢓ࣄ͸ event ͷύʔεͳͲɺ࠷௿ݶʹཹΊΔ • ϩʔΧϧίϯςφϥϯλΠϜͷ࢖༻΋ࣗવͱ΍Γ΍͘͢ͳΔ • e.g) SAM Local, DynamoDB Local, localstack

Slide 39

Slide 39 text

αʔόʔϨεͬΆ͍෦෼ • εςʔδґଘͷ஋͸؀ڥม਺ͱͯ͠ѻ͏ • ؅ཧ͢ΔϞδϡʔϧΛ࡞ͬͯू໿ • e.g) settings.py, constants.py • example: hassaku63/sls-env-example settings.py

Slide 40

Slide 40 text

Example • https://github.com/hassaku63/sls-env-example ૝ఆߏ੒ਤ

Slide 41

Slide 41 text

Example • https://github.com/hassaku63/sls-env-example ૝ఆߏ੒ਤ 4242VFVF63- Λ஌͍ͬͯΔඞཁ͕͋Δ

Slide 42

Slide 42 text

templates.yml

Slide 43

Slide 43 text

templates.yml MBNCEBͷ؀ڥม਺ʹ Ϧιʔεಛఆʹ ඞཁͳ৘ใΛ౉͢

Slide 44

Slide 44 text

settings.py

Slide 45

Slide 45 text

settings.py εςʔδ༻ͷ EPUFOWΛϩʔυ 2VFVF63-ͷಛఆʹ ඞཁͳม਺Λϩʔυ

Slide 46

Slide 46 text

ϩΪϯά

Slide 47

Slide 47 text

͠Ό΂Δ͜ͱʢ༧ఆʣ • JSON ϩΨʔΛ࢖ͬͯߏ଄Λ࣋ͨͤΔ • τϨʔε༻ͷIDΛ෇͚Δ • σίϨʔλͰڞ௨Խɹ

Slide 48

Slide 48 text

͠Ό΂Δ͜ͱʢ༧ఆมߋʣ • JSON ϩΨʔΛ࢖ͬͯߏ଄Λ࣋ͨͤΔ • τϨʔε༻ͷIDΛ෇͚Δɹ→ɹߏ੒ਤ͕೐Θ͖ͬͯͨΒ΍Δ • σίϨʔλͰڞ௨Խɹ→ɹ࣮૷ऀͷ޷Έ࣍ୈ

Slide 49

Slide 49 text

JSONͰߏ଄Խ CloudWatch Logs Insights ͷݕࡧ͕ḿΔ  (υοτ۠੾ΓͰωετͨ͠ϑΟʔϧυʹΞΫηεՄೳ) Amazon CloudWatch Logs - JSON ϩάͷϑΟʔϧυ https://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/logs/CWL_AnalyzeLogData-discoverable-ﬁelds.html#CWL_AnalyzeLogData-discoverable-JSON-logs

Slide 50

Slide 50 text

JSONͰߏ଄Խ ࣮૷ͷબ୒ࢶ 1. ϩΪϯά࣌ͷ msg Ҿ਺ʹ͸ඞͣ json.dump() ͔ͯ͠Β஋Λ౉͢ 2. JSON ϩΨʔΛαϙʔτ͢ΔύοέʔδΛ࢖͏ 3. ࣗલͰॻ͘ 1. Logger, Formatter Λܧঝ 2. Logging Adapter ͷར༻ʢݕ౼தʣ

Slide 51

Slide 51 text

JSONͰߏ଄Խ Logger ʹٻΊ͍ͨཁ݅ʢ໛ࡧதʣ • JSONܗࣜͰग़ྗͰ͖Δ͜ͱ • Ұఆͷߏ଄ԽΛڧ੍Ͱ͖Δ͜ͱ • ʢগͳ͘ͱ΋ʣτοϓϨϕϧͷJSONϓϩύςΟ͕Ұ؏͍ͯ͠Δ • Lambda ͷ࣮ߦ؀ڥ৘ใΛউखʹ஫ೖͯ͘͠ΕΔ

Slide 52

Slide 52 text

JSONͰߏ଄Խ ࢀߟʣLambda Context ͷ৘ใ͸༗༻ͳͷͰϩά౳Ͱ׆༻Ͱ͖Δ (doc) • Lambda ؔ਺໊ʗARNʗόʔδϣϯ • LogGroup / LogStream • Invocation timeout ·Ͱͷ࢒Γ࣌ؒ • etc ..

Slide 53

Slide 53 text

τϨʔε༻IDͷ෇༩ • αʔόʔϨεߏ੒Ͱ͸ɺ1ͭͷϢʔεέʔε͕ෳ਺ͷϚΠΫϩ αʔϏεΛލ͍Ͱॲཧ͞ΕΔ • ӡ༻্ͷτϥϒϧγϡʔςΟϯάͰ͸Ϣʔεέʔε୯ҐͰϩά ΛߜΓࠐΈ͍ͨ • αʔϏεԣஅͰ࣮ߦ͝ͱʹҰҙͱͳΔID͕औΕΕ͹Good

Slide 54

Slide 54 text

τϨʔε༻IDͷ෇༩ • جຊ͸ AWS X-Ray Λ࢖͓͚ͬͯ͹OK • τϨʔε༻IDʹ x-amzn-trace-id Λར༻Մೳ • Serverless Framework ͳΒ༗ޮԽ͸؆୯ • Step Functions Λ࢖͏ߏ੒ͩͱҰ೧Γඞཁ • Step Functions ͸ X-Ray ʹඇରԠʢ2020೥8݄ݱࡏʣ

Slide 55

Slide 55 text

τϨʔε༻IDͷ෇༩ - X-Ray ͷ༗ޮԽ - template.yml ʹ2-3ߦ௥Ճ  ʢ͓ΘΓʣ

Slide 56

Slide 56 text

σίϨʔλͰڞ௨Խ Lambda ϋϯυϥ༻ͷσίϨʔλΛ࡞ͬͯΞλον͢Δ • e.g) ϋϯυϥͷ࣮ߦલޙͰඞͣϩΪϯάΛڬΉΑ͏ʹ͢Δ • e.g) ෳ਺ϨίʔυΛόϥͯ͠σίʔυ͔ͯ͠Β౉͢ʢSQS/Kinesis τϦΨʔʣ

Slide 57

Slide 57 text

https://github.com/hassaku63/qiita-autogenaration/blob/master/qp/logs.py ࢖༻ྫ (Decorator)

Slide 58

Slide 58 text

࢖༻ྫ (lambda handler) https://github.com/hassaku63/qiita-autogenaration/blob/master/qp/functions/sfn/publish.py#L17-L28

Slide 59

Slide 59 text

σίϨʔλͰڞ௨Խ Pros • ϋϯυϥ಺ͷهड़ྔ͕ݮΓɺຊ࣭తͳϩδοΫ෦෼͕ڧௐ͞ΕΔ • ൚༻తͳڞ௨ॲཧͷ࠶ར༻ • ։ൃϙϦγʔͷڧ੍ • ࣮ࡍʹ͜ͷख๏Λ࢖࣮ͬͨ૷͕͋Δ (e.g: awslabs/aws-lambda-powertools-python)

Slide 60

Slide 60 text

σίϨʔλͰڞ௨Խ Cons • ΍Γ͗͢͸٫ͬͯίʔυ͕ಡΈͮΒ͘ͳΔʢओ؍ʣ • ଟগ௕͘ͳͬͯ΋ʮԿͯ͠Δ͔ʯ͕௚ײతʹಡΈԼͤΔͳΒ໰୊ͳ͍ɺͱ͍͏ߟ͑΋͋Δ • logging ࣌ͷελοΫϑϨʔϜ͕σίϨʔλʹͳͬͯ͠·͏ • ※ͨͩ͠ɺ΍ΓΑ͏͸͋Δʢݺͼग़͠ݩͷελοΫϑϨʔϜΛḷΔʗσίϨʔλͷҾ਺ͰରԠʣ • ϋϯυϥͷ్தͰ΋ϩΪϯά͍ͨ͠έʔε͸ग़ͯ͘Δ

Slide 61

Slide 61 text

·ͱΊ

Slide 62

Slide 62 text

·ͱΊ • αʔόʔϨεʹͳͬͯ΋ɺΞϓϦέʔγϣϯͷ෦෼͸͘͝ී௨ͷ Python • ߏ੒೺Ѳ͕΍Γ΍͘͢ͳΔΑ͏ʹιʔεΛߏ੒͢Δ • εςʔδݻ༗ͷύϥϝʔλ͸؀ڥม਺ͳͲ࢖ͬͯ؅ཧ • ϩΪϯά • JSONϩΨʔΛ࢖͍ͭͭɺσʔλߏ଄ΛҰ؏ͤ͞Δ • ՄೳͳΒτϨʔεIDΛৼΔ • σίϨʔλʹΑΔڞ௨Խ͸͓޷ΈͰ

Slide 63

Slide 63 text

ʢࢀߟʣ৘ใݯ • AWS • AWS Ϋϥ΢υαʔϏε׆༻ࢿྉू: https://aws.amazon.com/jp/aws-jp-introduction/ • Webinar: https://aws.amazon.com/jp/about-aws/events/webinars/ • Connpass: https://aws-serverless.connpass.com/ • Serverless community • Connpass: https://serverless.connpass.com/ • Serverless Operations • Blog archives: https://serverless.co.jp/blog-archives • Qiita: https://qiita.com/organizations/serverless-operations • Twitter • Yan Cui (@theburningmonk / AWS Serverless Hero): https://twitter.com/theburningmonk • Kensuke Shimokawa (@kensh /AWS Japan Serverless Specialist SA): https://twitter.com/_kensh • Keisuke Nishitani (@Keisuke69 / AWS Japan): https://twitter.com/Keisuke69 • Takahiro Horike (@horike37 / Serverless Operations): https://twitter.com/horike37

Slide 64

Slide 64 text

We here hiring!