タップル誕生: オンプレからAWS Fargateまで @ JAWS DAYS 2020

by Tamirlan 893 Torgayev

Slide 1

Slide 1 text

λοϓϧ஀ੜ: ΦϯϓϨ͔ΒAWS Fargate·Ͱ גࣜձࣾαΠόʔΤʔδΣϯτ ٕज़ຊ෦ αʔϏεϦϥΠΞϏϦςΟάϧʔϓ ϓϩάϥϛϯάϠΫβ (@prog893)

Slide 2

Slide 2 text

ࣗݾ঺հ • Torgayev Tamirlan / ϓϩάϥϛϯάϠΫβ / @prog893 τ ϧ Ψ Τ ϑ λ ϝ ϧ ϥ ϯ • αΠόʔΤʔδΣϯτ ٕज़ຊ෦   αʔϏεϦϥΠΞϏϦςΟάϧʔϓ (௨শ SRG) • झຯ: Χϝϥ • ޷͖ͳ΋ͷ: TerraformɺPythonɺElasticsearchɺServerless ❤ • ޷͖ͳAWSͷαʔϏε: ECS • ΞϧόΠτ 14ϲ݄ 2018೥ ৽ଔೖࣾ (ࣗশ4೥໨) AWA վળʢͦͷ1ɺͦͷ2ʣɺτϧςɺCARTEɺREQU ্ཱͪ͛αϙʔτ ݱ AWAɺλοϓϧ஀ੜɺCROSS MEɺͳͲ୲౰ 2

Slide 3

Slide 3 text

αʔϏεϦϥΠΞϏϦςΟάϧʔϓ is Կ • ฐࣾʹ͓͚ΔϝσΟΞ؅׋ͱݺ͹ΕΔ࿮૊Έʹଐ͢ΔαʔϏεͷ  ʮΠϯϑϥʯͳͲΛԣஅతʹݟ͍ͯΔ૊৫ (AbemaTVͱͦͷؔ࿈Λআ) యܕతͳΠϯϑϥ԰Ͱ͸ͳ͍ SREͱDevOpsͱCREͬΆ͍໾ׂ΋Ռ͍ͨͯ͠Δ (ৄࡉ͸ͪ͜Β) 3

Slide 4

Slide 4 text

αʔϏεϦϥΠΞϏϦςΟάϧʔϓ is Կॲ 4

Slide 5

Slide 5 text

ࠓ೔࿩͢͜ͱ • λοϓϧ஀ੜͷҠઃ෺ޠ 2೥ؒҎ্ʹΘͨΔετʔϦʔ • લ൒: ྺ࢙తܦҢɺҠઃ͋Δ͋Δ • ޙ൒: ECS EC2/FargateͷҠઃͱ͔ΑΓσΟʔϓͳ࿩ ऴ൫ʹ࠷ۙग़ͨ৽ػೳΛࢼͯ͠Έͨ࿩΋͋ΔΑ 5

Slide 6

Slide 6 text

໨࣍ 1. αʔϏε֓ཁ 2. ྺ࢙తܦҢ ΦϯϓϨߏ੒ Ҡઃܭը Ҡઃ࣮ߦ 3. ʁʁʁ 4. ʁʁʁ ʁʁʁ ʁʁʁ 5. ॴײ 6

Slide 7

Slide 7 text

λοϓϧ஀ੜ αʔϏε֓ཁ

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

λοϓϧ஀ੜ is Կ • झຯͰͭͳ͕Δ࿀׆αʔϏε • ձһ਺400ສਓҎ্ɺϚονϯά਺͸Ԇ΂1ԯ2,000ສ૊ • ࠃ಺࠷େن໛ 9

Slide 10

Slide 10 text

https://tapple.me 10

Slide 11

Slide 11 text

ྺ࢙తܦҢ

Slide 12

Slide 12 text

໨࣍ 1. αʔϏε֓ཁ 2. ྺ࢙తܦҢ ΦϯϓϨߏ੒ Ҡઃܭը Ҡઃ࣮ߦ 3. ʁʁʁ 4. ʁʁʁ ʁʁʁ ʁʁʁ 5. ॴײ 12

Slide 13

Slide 13 text

ΦϯϓϨ࣌୅ͷߏ੒֓ཁ • Φϯϓr…ϓϥΠϕʔτΫϥ΢υ • GitHubͰίʔυ؅ཧɺJenkinsͰϏϧυˍσϓϩΠ • αʔόͷϓϩϏδϣχϯά͸Ansible • Node.js + MongoDB + Elasticsearch • ϩάपΓ͸ɺFluentd tail -> aggregator -> in-house analyticsج൫ 13

Slide 14

Slide 14 text

Slide 15

Slide 15 text

ΦϯϓϨߏ੒ʹ͓͚Δ՝୊఺ • ύϒϦοΫΫϥ΢υͰ͍͏ͱ͜ΖͷManagedͳαʔϏε͕গͳ͍ ΄ͱΜͲͷίϯϙʔωϯτ͸ࣗ෼ͨͪͰ؅ཧ͠ͳ͚Ε͹ͳΒͳ͍ͨΊɺ؅ཧίετߴ • ΦʔτεέʔϦϯά͸ɺ૬౰࡞Γࠐ·ͳ͍ͱ࣮ݱ͕೉͍͠ ίετ࠷దԽ͕Ͱ͖ͣɺৗʹ࠷େΩϟύγςΟ͕ىಈ͍ͯͯ͠ແବ ͦͯ͠࠷େΩϟύγςΟ͕มΘΕ͹…खಈ૿ઃ… • ࣗಈԽ͕೉͘͠ɺϗετো֐ͳͲ͕ى͖ΔͨͼʹखಈΦϖൃੜ • ϓϥΠϕʔτΫϥ΢υͷӡ༻ϧʔϧ্ɺϦʔδϣϯʹ͸EOL͕͋ΓҠઃൃੜ ౰࣌࢖͍ͬͯͨϦʔδϣϯͷEOL͕͍͖͍ۙͮͯͯͨ  Disclaimer: ผʹϓϥΠϕʔτΫϥ΢υΛdisͬͯΔΘ͚͡Όͳ͍͔Βצҧ͍͠ͳ͍ͰΑͶͬ 15

Slide 16

Slide 16 text

AWSʹҠઃ͢Δ͜ͱʹͳͬͨཧ༝ • ͳͥύϒϦοΫΫϥ΢υ ΦʔτεέʔϦϯά͍ͨ͠ ϚωʔδυαʔϏεͰ؅ཧɾӡ༻ίετԼ͛ͯ։ൃʹूத͍ͨ͠ • ͳͥAWS ϚονϯάαʔϏεͷதͰ͸͢ͰʹAWSͷར༻࣮੷͕͋ͬͨ ϚονϯάαʔϏεΛAWSʹἧ͍͍ͨͱ͍͏૝͍΋ ϝσΟΞ؅׋ͷதͰAWSͷ஌ݟ͕ଟ͔ͬͨ ౰࣌͸།Ұ౦ژϦʔδϣϯΛఏڙ͍ͯͨ͠ 16

Slide 17

Slide 17 text

ҠઃܭըΛ ཱͯΑ͏

Slide 18

Slide 18 text

໨࣍ 1. αʔϏε֓ཁ 2. ྺ࢙తܦҢ ΦϯϓϨߏ੒ Ҡઃܭը Ҡઃ࣮ߦ 3. ʁʁʁ 4. ʁʁʁ ʁʁʁ ʁʁʁ 5. ॴײ 18

Slide 19

Slide 19 text

Ҡઃޙͷߏ੒ v.1: EC2 • ΞϓϦ૚͸ͦͷ·· Ծ૝αʔό EC2 Πϯελϯεͱ͍͚ͯ࣋ͬͯͩ͘͠ • MongoDBͱElasticsearch΋ͦͷ··͍࣋ͬͯ͘ MongoDB΁ͷΞΫηε͸ɺ֤ΞϓϦαʔόʹ৐͍ͬͯΔmongosΛϓϩΩγʹ࢖͏ Elasticsearch͸ɺALBͰΞΫηεΛ෼ࢄ • RedisɺMySQL͸ElastiCacheɺAuroraʹ • nginxΛALBʹϦϓϨΠε • ੩తίϯςϯπͷ഑৴͸ɺS3+CloudFront 19

Slide 20

Slide 20 text

Slide 21

Slide 21 text

ҠઃϩʔυϚοϓ 1. σʔλ૚͸ϝϯςΛೖΕͨঢ়ଶͰҠઃ͠ɺͦΕҎ֎͸ΦϯϥΠϯͰҠઃ σʔλ૚ͷҠઃͷͨΊͷϝϯς͸௒ਖ਼͍͠൑அɺঝ୚ͨؔ͠܎ऀ֤Ґʹѹ౗తײँ ϓϥΠϕʔτΫϥ΢υ͔Βσʔλ૚΁ͷΞΫηε͸ɺઐ༻ճઢܦ༝ 2. ΞϓϦ૚ɺݕূ͠ͳ͕ΒঃʑʹҠઃ͍ͯ͘͠ 21

Slide 22

Slide 22 text

࣮ߦ ͢Δͧ

Slide 23

Slide 23 text

໨࣍ 1. αʔϏε֓ཁ 2. ྺ࢙తܦҢ ΦϯϓϨߏ੒ Ҡઃܭը Ҡઃ࣮ߦ 3. ʁʁʁ 4. ʁʁʁ ʁʁʁ ʁʁʁ 5. ॴײ 23

Slide 24

Slide 24 text

εςοϓ2: ΞϓϦ૚ͷҠઃ…ʁ • EC2 AutoscalingͷͨΊͷɺAMIϏϧυ༻Packer࡞ΓࠐΈɺݕূΛ…  ͢Δ…ͭ΋Γ…͕ͩͬͨ… Ω < ͔ͤͬ͘ͷҠઃͳΒɺΞʔΩςΫνϟมߋ΋͍ͨ͠Ͷͬ Ω < ϚΠΫϩαʔϏεԽͰ͖ͨΒϦϦʔεͷ୯ҐΛখ͘͞Ͱ͖Δ͠੾Γ໭͠ͱָ͔ͩ͠ Ω < Dockerﬁle࡞ͬͨͧ Ω < KubernetesͰͱΓ͋͑ͣಈ͍ͨͧ Ω < KubernetesͰߦ͜͏ͥ Ω < ͔ͤͬͩ͘͠ଞ΋վળ͍ͯ͜͠͏ Μɺɺ͋Δ͐ʁɹҰ୴EC2Ͱߦ͘Μ͡Όͳ͔͚ͬͨͬ 24

Slide 25

Slide 25 text

Ҡઃޙͷߏ੒ v.2: Kubernetes • σʔλ૚͸มΘΒͣ • NLBΛIngestͱͯ͠࢖༻ • kops + helm • ap-northeast-1 3 AZ (a/c/d) • ASGͰworkerϊʔυΛ؅ཧɺΦʔτεέʔϦϯά 25

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Ҡઃޙͷߏ੒ v.2: ϩάपΓ • APIαʔϏεͳͲͰ૬৐Γﬂuentdίϯςφఆٛ • ڞ༗volume͔Βtail Kinesis Lambda in-houseͳanalytics KinesisͰόοϑΝϦϯάɺin-houseͳج൫͕ࢮΜͰ͍ͯ΋ɺ  ʮKinesis Lambdaʯͷ෦෼Ͱ͍͍ײ͡ʹϦτϥΠ͞ΕΔ ৄࡉ: KinesisͱLambdaͰͭ͘ΔServerlessͳϩάج൫ @ AWS DevDay Tokyo 2019 27

Slide 28

Slide 28 text

Slide 29

Slide 29 text

… Ω < ΫϥελͱVPCͱαϒωοτपΓ࡞ͬͨͧ Ω < manifestΧΩΧΩͯ͠Δͧ Ω < Α͘ݟͨΒͪΌΜͱಈ͍ͯͳ͍ͳ Ω < Kubernetesͷόά͔ͳ Ω < Dockerͷόά͔ͳ Ω < ͋ΕΕ 29

Slide 30

Slide 30 text

Ҡઃ͕׬ྃ͠ͳ͍ ··1೥͕ܦͭ

Slide 31

Slide 31 text

ͦͯ࣌͠ݶര஄͕ େരൃ

Slide 32

Slide 32 text

࣌ݶര஄ #ͱ͸ • ࣌ݶര஄ = ઐ༻ճઢ • ϙΠϯτ: 1. σʔλ૚΁ͷΞΫηε͕Ϋϥ΢υΛލ͍Ͱ͍ͨ 2. ઐ༻ճઢ͕อূͳ͠Ͱఏڙ͞Ε͍ͯͨ   (౰࣌͸ SLA ͕ఏڙ͞Ε͍ͯͳ͔ͬͨ) • Ҡઃ͕׬ྃ͠ͳ͍ঢ়ଶͰ1೥͕ܦͬͨ • ͦͯ͠ࣄ͕݅ى͖ͨ 32

Slide 33

Slide 33 text

࣌ݶര஄େരൃࣄ݅ • ઐ༻ճઢͷো֐ͰશAPI஗Ԇര૿ɺDBෛՙߴಅɺ౤ߘ؂ࢹࣦഊɺ  ΞϓϦͷΤϥʔදࣔʹΑΔେྔ࠶ϩάΠϯ ϦʔνҰൃ໳લਗ਼ࣗ໚࿨ฏ࿨υϥ2ɺ௓ຬɺ௓Μͩ… • શମϝϯςφϯεΛೖΕͯɺઐ༻ճઢఏڙݩʹ໰͍߹Θͤͭͭ͠  ༷ࢠΛݟΔ͜ͱʹ • ࡾ࣌ؒޙɺճઢ෮چɺϝϯς໌͚ 33

Slide 34

Slide 34 text

࣌ݶര஄େരൃࣄ݅ɺೋճ໨ • ೋ೔࿈ଓͰ·ͨಉ͡ো֐͕ى͖Δ • ϝϯςΠϯ͔Β3࣌ؒޙɺ෮چͷݟࠐΈ͕ͳ͘ͳΓɺ  ΋͏Ҡઃͯ͠͠·͑͹͍͍͡Όͳ͍͔આΛࢲ͕ޠΓʹ͍͘ • ආ೉ઌͷ४උ։࢝ • ϝϯςΠϯ͔Β5࣌ؒޙɺճઢ͕෮چɺϝϯς໌͚ • ࠓ౓͸ɺ5࣌ؒҎ্ͷϝϯςφϯε • ·͍ͨͭى͖Δ͔Θ͔Βͳ͍͠ා͍͔Βɺආ೉ઌͷ४උΛଓߦ • ͦͷ৔Ͱ࠷௿ݶಈ͘؀ڥͷߏஙͱσϓϩΠϑϩʔͷཱ֬·Ͱ׬ྃͯ͠͠·͏ ͜ͷ͋ͱ໓஡ۤ஡͓ञΛҿΈ·ͨ͠ 34

Slide 35

Slide 35 text

໨࣍ 1. αʔϏε֓ཁ 2. ྺ࢙తܦҢ ΦϯϓϨߏ੒ Ҡઃܭը Ҡઃ࣮ߦ 3. ʁʁʁ 4. ʁʁʁ ʁʁʁ ʁʁʁ 5. ॴײ 35

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

໨࣍ 1. αʔϏε֓ཁ 2. ྺ࢙తܦҢ ΦϯϓϨߏ੒ Ҡઃܭը Ҡઃ࣮ߦ 3. Re: ো֐͔Β࢝ΊΔۓٸҠઃੜ׆ 4. ͋Δ΂͖࢟΁ EC2 -> ECS (EC2 Type) ECS (EC2 Type) -> ECS (Fargate Type) 5. ॴײ 37

Slide 38

Slide 38 text

ҠઃϩʔυϚοϓ • 3ϑΣʔζͰ࣮ࢪ 1. ۓٸEC2Ҡઃ: ͱΓ͋͑ͣ ΦϯϓϨ EC2 (࣌ؒՔ͗) • ϢʔβʹͰ͖Δ͚ͩૣ͘ΑΓ҆ఆͨ͠؀ڥͷఏڙ͢ΔͨΊ 2. EC2 ECS (EC2 Type) 3. ECS (EC2 Type) ECS (Fargate Type) • ͙͢ʹ͸Fargateʹ͠ͳ͔ͬͨཧ༝: • Fargateͷ஌ݟ͕গͳ͔ͬͨ • ౰࣌଍Γͳ͍ػೳ͕͋ͬͨΓɺߴ͔ͬͨΓʢ࠷ۙ൒ֹʹͳͬͨΓػೳ͕૿͑ͨΓʣ • λοϓϧϝϯόʔͷ஌ࣝΛਂΊΔͨΊʹ΋ɺֶ͋͑ͯशͷͨΊEC2ʹ • Fargateͱ͸ҟͳΓɺEC2Ͱ͸docker exec -it ͳͲͰίϯςφʹϩάΠϯͯ͠ௐࠪɺσόοά͕Ͱ͖Δ 38

Slide 39

Slide 39 text

KubernetesΛఘΊͯECSΛ࠾༻ͨ͠ཧ༝ • KubernetesͰؤுͬͯݕূ͍͕ͯͨ͠ɺ  1೥͔͔ͬͯ݁ہຊ൪ʹग़ͤΔϨϕϧʹ͸ࢸΒͳ͔ͬͨʢׂѪʣ • ECSͷํ͕AWSͷଞͷαʔϏεͱͷ਌࿨ੑ͕ߴ͍ • Kubernetes͡Όͳ͍ͱ͍͚ͳ͍ϢʔεέʔεͰ͸ͳ͔ͬͨ Service MeshͷΑ͏ͳෳࡶͳߏ੒Ͱ͸ͳ͘ɺಠཱͨ͠API͍͔ͭ͘ͱόον͍͔͚ͭͩ͘ • SRGͱͯ͠΋ฐࣾϝσΟΞࢠձࣾͱͯ͠΋ɺECSͷϊ΢ϋ΢͕ଟ͍ ࣗݾ঺հʹ͋ΔϦϯΫࢀর ߏஙͷΈͰ4೔ؒ + ΦʔτεέʔϦϯάͷઃܭ 1िؒʢׂ߹มߋͱͱ΋ʹ࣮ࢪʣ 39

Slide 40

Slide 40 text

ۓٸEC2Ҡઃ: ϩʔυϚοϓ • ߏ੒Λେ͖͘ม͑ͣɺαʔόΛͦͷ··EC2Ͱ͍࣋ͬͯ͘ QCɺෛՙࢼݧͳͲΛ௨ͯ͡Ϣʔβʹग़ͯ͠΋͍͍͜ͱ͕ূ໌Ͱ͖Δ·Ͱ͸੾Γସ͑ͳ͍ ͨͩ͠ɺ΋͠·ͨେো֐͕ى͖ͨ৔߹ɺ৽؀ڥΛආ೉ઌͱͯ͠ར༻ • ίϯςφԽߏ੒Λߏஙɺςετ • EC2΁ͷ੾Γସ͑ޙɺίϯςφԽߏ੒ʹ੾Γସ͍͑ͯ͘ KubernetesΛఘΊͯɺECSΛ࠾༻ʢޙʹղઆʣ KubernetesݕূΛͨ͠ͱ͖ͷDockerﬁle΍VPCपΓͳͲྲྀ༻ 40

Slide 41

Slide 41 text

ۓٸEC2Ҡઃ: ߏங • Packerͱ͔Φʔτεέʔϧͱ͔Ұ୴ؾʹͤͣߏங • ര଎ͰTerraformͰEC2ܥɺηΩϡϦςΟάϧʔϓɺALBपΓΛ੔උʢԶʣ • AnsibleΛमਖ਼͠ͳ͕Βྲྀ͢ʢλοϓϧϝϯόʔʣ • ΞϓϦσϓϩΠɺಈ࡞֬ೝɺमਖ਼ͷ܁Γฦ͠ • ߹ܭ12࣌ؒͰͱΓ͋͑ͣಈ͘΋ͷ͕׬੒ • ·ͨো֐͕ى͖ͨΒආ೉ઌͱͯ͠࢖͏ 41

Slide 42

Slide 42 text

ۓٸEC2Ҡઃ: ࠷ऴ֬ೝͱ੾Γସ͑ • ߏஙͷཌ೔: QCɺϦάϨογϣϯςετɺෛՙࢼݧʹ߹֨ • ҰिؒΛ͔͚ͯঃʑʹϦΫΤετΛ৽͍͠؀ڥʹدͤΔ • tapple.me AϨίʔυΛALIASͰRoute 53 Trafﬁc Flowʹ޲͚Δ • ϓϥΠϕʔτΫϥ΢υLBͱAWS ALBͷׂ߹ΛweightͰ੍ޚ • Θ͔Γ΍͘͢͢ΔͨΊʹɺweightͷ߹ܭ͕100ʹͳΔΑ͏ʹઃఆ • ࣮ࡍͷΞΫηεϩάͰׂ߹Λ֬ೝ 42

Slide 43

Slide 43 text

Slide 44

Slide 44 text

Slide 45

Slide 45 text

Route 53 Trafﬁc Flowॴײ • weightͷมߋΛద༻͔ͯ͠Βɺ࣮ࡍͷϦΫΤετͷׂ߹͕มΘΔ·Ͱ ~ 5෼ • DNSΩϟογϡ͕มʹӨڹ͢Δ͜ͱͳ͘ɺ  ࢦఆͨ͠weightͱ࣮ࡍͷΞΫηεͷׂ߹͸2%΄Ͳͷޡࠩ • Ϋϥ΢υҠઃϕετϓϥΫςΟε͗͢Δ… • KibanaͰϦΞϧλΠϜͰׂ߹͕มΘΔͷΛݟΔͷָ͍͠… 45

Slide 46

Slide 46 text

2िؒͰEC2Ҡઃ׬ྃ ര଎

Slide 47

Slide 47 text

͋Δ΂͖࢟΁

Slide 48

Slide 48 text

໨࣍ 1. αʔϏε֓ཁ 2. ྺ࢙తܦҢ ΦϯϓϨߏ੒ Ҡઃܭը Ҡઃ࣮ߦ 3. Re: ো֐͔Β࢝ΊΔۓٸҠઃੜ׆ 4. ͋Δ΂͖࢟΁ EC2 -> ECS (EC2 Type) ECS (EC2 Type) -> ECS (Fargate Type) 5. ॴײ 48

Slide 49

Slide 49 text

Ҡઃޙͷߏ੒ v.3: ECS EC2 ԦಓͳECS EC2: • ALB • ECS Service (EC2 Type) ͨͩ͠ɺBatchܥ͸શ෦Fargate (ޙʹղઆ) • EC2 Auto-Scaling Group (ΫϥελͷόοΫΤϯυ) 49

Slide 50

Slide 50 text

Slide 51

Slide 51 text

ECS EC2Ҡઃ: ECSΫϥελ • ECSόοΫΤϯυ͸ɺEC2 + Autoscaling Group • VanillaͳAmazon Linux (ECS-optimized) AMIͰىಈɺ  user_dataͰͪΐͬͱͨ͠ॳظԽ͚࣮ͩߦ • εέʔϦϯάϝτϦοΫε͸ɺECS ClusterͷCPUReservationͷMaximum 51

Slide 52

Slide 52 text

ECS EC2Ҡઃ: APIܥ • API༻αʔϏε͸EC2 Type • εέʔϦϯάϝτϦοΫε͸ɺECS ServiceͷCPUUtilizationͷMaximum • Dockerﬁle౳ɺΞϓϦΛίϯςφͰಈ͔ͨ͢Ίͷ΋ͷҰࣜ͸ɺ  Kubernetesݕূͷͱ͖ͷ΋ͷΛྲྀ༻ • ϚΠΫϩαʔϏεʹ෼͚Δͱ͜Ζ͸Ұ୴΍Βͳ͍ ཧ༝ɿALBΛར༻͍ͯ͠ΔͨΊɺޙ͔Βಛఆͷύε͚ͩผͷαʔϏεʹϧʔςΟϯάͤ͞Δ  ͜ͱ͕Մೳɺ͙͢΍Δඞཁ͕ͳ͍ 52

Slide 53

Slide 53 text

ECS EC2Ҡઃ: όονܥ • όον༻αʔϏε͸Fargate Type • ݁ߏසൟʹ࣮ߦ͞ΕΔόον͕͋ͬͯɺ  EC2Ͱಈ͔͢ͱASGͷΦʔτεέʔϦϯάͷઃܭ͕໘౗ͳ͜ͱʹͳΔ… όον༻ͷΫϥελΛ࡞ΔͱϦιʔε͕ແବʹͳΔ͠ɺ  ࡞Βͳ͍ͱεέʔϦϯάΞΫγϣϯͷढ़හੑ… • CloudWatch Eventsͷcron͕ࣜͪΐͬͱಛघͳͨΊɺ  طଘͷcronࣜΛͪΐͬͱ͍͡Δඞཁ͕͋ͬͨ • Baikonur OSS fargate-scheduled-task ϞδϡʔϧΛར༻ 53

Slide 54

Slide 54 text

ECS EC2Ҡઃ: DB • MongoDBɺElasticsearch͸ʢ΄΅ʣͦͷ·· • EC2ɺϓϥΠϕʔτΫϥ΢υͰΠϯελϯε͝ͱʹىಈ͍ͤͯͨ͞  mongos͸ɺઐ༻EC2 x 3୆ʹͨ͠ • mongos͕ݱΕͨΓফ͑ͨΓ͢Δͱɺconﬁg DBʹ΋͏͍ͳ͍mongosͷ৘ใ͕ཷ·Δ • ͢Ͱʹ͍ͳ͍mongosʹmongoc͕ping͠Α͏ͱ͍ͯ͠Δʁ • ޙʹ૿ઃ͞Ε…ͦͯ͠·ͨ૿ઃ͞Εͨ… 54

Slide 55

Slide 55 text

ECS EC2Ҡઃ: ϩάपΓ • Kubernetes༻ͷ΋ͷͱҰॹɺ૬৐Γﬂuentdίϯςφར༻ • ڞ༗volume͔Βtail aggregator Kinesis Lambda   in-houseͳanalytics KinesisసૹϓϥάΠϯͱϩάʹର͢Δཁ݅ͷ౎߹Ͱ࠷ऴతʹﬂuentd aggregatorḉര஀ 55

Slide 56

Slide 56 text

ECS EC2Ҡઃ: Cluster ASGͷεέʔϦϯάઃܭ • ΦʔτεέʔϦϯάͱ͍͏֓೦͕ECSҠઃ·Ͱ͸ଘࡏ͠ͳ͔ͬͨ • Ұ୴ɺෛՙͷ͔͔Γํ౳͕ࣅ͍ͯΔαʔϏεͷᮢ஋ΛύΫͬͨ • 1िׂ͔͚ؒͯ߹Λมߋ͢ΔΞϨͱಉ࣌ʹɺ  εέʔϦϯάΛݟकΓͭͭࡉ͔͍ௐ੔Λߦͳͬͨ • εέʔϦϯάͷௐ੔Λߦͳͬͨࡍͷߟ͑ํΛ͝঺հ 56

Slide 57

Slide 57 text

ECS EC2Ҡઃ: Cluster ASGͷεέʔϦϯάઃܭ • CPUReservationͷAverage͕75%-85%ͷؒΛӭ͙Α͏ʹɺ  Step ScalingΛखͰௐ੔ εύΠΫɺϐʔΫΛܴ͑Δͱ͖ʹɺΫϥελͷΩϟύ͕  Taskͷ഑ஔͷ଍ΛҾͬுΒͳ͍Α͏ʹஸೡʹௐ੔ 57

Slide 58

Slide 58 text

Step ScalingɺͩΔ͍Ͱ͠ΐʁ • ઃܭ͕ͩΔ͍͠೉͍͠ • ͦͯ͠ରࡦΛ͠ͳ͍ͱASG scale-in࣌ʹੜ͖͍ͯΔίϯςφ͕ࡴ͞ΕΔ • ରࡦ: EC2 Auto-ScalingͰECS૊ΜͰ·͔͢ʁΤϥʔϨʔτߴ͍Ͱ͔͢ʁٹͬͯ͋͛ͯ΋͍͍Ͱ͔͢ʁ • ECS Cluster Auto Scalingͱ͍͏ਆػೳ͕Ͱͨʂ • ͍͍ײ͡ʹεέʔϦϯάͯ͘͠ΕΔɺStep Scaling͢Δඞཁͳ͠ • CPUɺϝϞϦɺENIͳͲɺ͍ͣΕ͔ҰͭͷΩϟύγςΟͷछྨͰ͸ͳ͘ɺ  ্ཱ͍ͪ͛ͨTask͕࣮ࡍ্ཱͪ͛ΒΕΔͷ͔ͱ͍͏γϛϡϨʔγϣϯͰΩϟύෆ଍Λಛఆ • ىಈ͍ͯ͠Δίϯςφͷ͋ΔEC2Λࡴ͞ͳ͍ઃఆ΋Ͱ͖Δɺൿ఻ͷλϨͳ͠Ͱಈ͘ʂ • ౰࣌ͳ͔ͬͨͷͰͱͯ΋ͭΒ͔ͬͨɺECSͷதͷਓ·ͫ͋Γ͕ͱ❤ 58

Slide 59

Slide 59 text

ECS EC2Ҡઃ: ECS ServiceͷεέʔϦϯάઃܭ • CPUUtilizationͷMaximum͕40%-75%ͷؒΛӭ͙Α͏ʹɺ  Step ScalingΛखͰௐ੔ɺ֤ίϯςφͷෛՙΛ֬ೝ ͜Ε͙Β͍ͷઃఆͰɺ֤ίϯςφ͕݈શతͳෛՙ (~50%) Λड͚Δ ίεύͱϨΠςϯγͷόϥϯγϯά 59

Slide 60

Slide 60 text

Step ScalingɺͩΔ͍Ͱ͠ΐʁ • ઃܭ͕ͩΔ͍͠೉͍͠ • Target Tracking Scaling (for ECS)ͱ͍͏΋ͷ͕ग़ͨ • ʮCPU࢖༻཰͕ฏۉͰ60%͙Β͍ʹͳΔΑ͏ʹͯ͠ɺϤϩʯͱ͍͏ઃఆ͚ͩͰ  ͋ͱ͸ΑΖ͘͠΍ͬͯ͘ΕΔ • ౰੍࣌໿͕͋ͬͯಋೖ͕೉͔ͬͨ͠ • ࢖͑ΔϝτϦΫε͕ݶΒΕ͍ͯͨ…هԱ • ޙʹෛՙࢼݧ؀ڥͰࢼ͢΋ɺզʑͷϢʔεέʔεʹ߹Θͣ • Step Scaling͔Β͍·ͩʹಀ͛ΒΕ͍ͯͳ͍ 60

Slide 61

Slide 61 text

ECS EC2Ҡઃ: ECS ServiceͷεέʔϦϯάઃܭ 61 ίϯςφͷCPU࢖༻཰͕40-60%  ͙Β͍ʹͳ݈ͬͯͯ߁తͳෛՙ

Slide 62

Slide 62 text

ECS EC2Ҡઃ: ࠷ऴ֬ೝͱ੾Γସ͑ • ۓٸEC2ҠઃͱҰॹͩΑ • QCɺϦάϨογϣϯςετɺෛՙࢼݧʹ߹֨͢Δ·Ͱमਖ਼࣮ࢪ • ҰिؒΛ͔͚ͯঃʑʹϦΫΤετΛدͤΔ • tapple.me AϨίʔυΛALIASͰRoute 53 Trafﬁc Flowʹ޲͚Δ • EC2؀ڥͷALBͱECS؀ڥͷALBͷׂ߹ΛweightͰ੍ޚ • Θ͔Γ΍͘͢͢ΔͨΊʹɺweightͷ߹ܭ͕100ʹͳΔΑ͏ʹઃఆ • ࣮ࡍͷΞΫηεϩάͰׂ߹Λ֬ೝ 62

Slide 63

Slide 63 text

ECS EC2Ҡઃ: Ҡઃޙͷ͋Ε͜Ε • ઐ༻ճઢ͕ͳ͘ͳͬͨ͜ͱͰɺAWSྉ͕͔ۚͳΓ҆͘ͳͬͨ • ϦΫΤετΛฦͨ͢Ίͷ௨৴͸AWS಺௨৴ͷΈʹͳͬͯɺ  API͕ര଎ʹͳͬͨ ࣮૷͸΋ͱ΋ͱͱͯ΋༏लͩͬͨ ϨΠςϯγ͕ฏۉͰ120ms଎͘ͳΓɺmsೋܻ୆΁ 63

Slide 64

Slide 64 text

໨࣍ 1. αʔϏε֓ཁ 2. ྺ࢙తܦҢ ΦϯϓϨߏ੒ Ҡઃܭը Ҡઃ࣮ߦ 3. Re: ো֐͔Β࢝ΊΔۓٸҠઃੜ׆ 4. ۓٸҠઃ͔Β͋Δ΂͖࢟΁ EC2 -> ECS (EC2 Type) ECS (EC2 Type) -> ECS (Fargate Type) 5. ॴײ 64

Slide 65

Slide 65 text

No content

Slide 66

Slide 66 text

Fargate! • ECS EC2Ҡઃ͔Β΄΅1೥͕ܦͬͨ • EC2ʹϩάΠϯͯ͠docker execͱ͔͢Δ͜ͱ͕΄΅ͳ͘ͳͬͯ ΞϓϦέʔγϣϯ΋ίϯςφʹదͨ͠ܗʹ͍͖ۙͮͯͨ • ϝϯόʔ΋ECSɺίϯςφΛ࢖ͬͨΞϓϦέʔγϣϯͷӡ༻ʹ׳Ε͖ͯͨ • ͦͯ͠ECS EC2ʹର͢Δෆຬ͕ཷ·͖ͬͯͨ • EC2 ASGͷεέʔϧΛ଴ͨͳ͚Ε͹ͳΒͳ͍ɺͦͷͨΊεέʔϦϯάɺσϓϩΠ͕஗͍ • Ωϟϯϖʔϯ౳Λ࣮ࢪ͢Δࡍʹscheduled scaling͕ඞཁɺͦͷϙϦγʔΛઃܭɾܭࢉ͢Δͷ͸େม • AMIߋ৽͢ΔͨͼʹASGͷEC2ΠϯελϯεΛస͕͢ͷ͕େม • Α͠ɺFargateԽ͠Α͏ʂ 66 ࢲ͸࠷ॳ͔Βɺ࠷ऴతʹFargateʹ͢Δͭ΋Γͩͬͨ

Slide 67

Slide 67 text

Slide 68

Slide 68 text

Fargateͷίετʹؔͯ͠ • Ϧιʔείετ͸On-demand EC2 (RIͳͲͷׂҾͳ͠) ͱൺֱͯ͠  Ή͠Ζ҆͘ͳΔ • EC2ͷόοΫΤϯυΠϯελϯεΛASGͱ͔Ͱ༻ҙ͢ΔࡍʹɺCPU/ϝϞϦͷۭ͖͕ৗʹ  15-20%ʹͳΔΑ͏ʹ͍ͯ͠Δ͔Βʢεέʔϧͷढ़හੑʣ • ͦͯ͠൒ֹʹ஋Լ͛͞ΕͨΓͨ͠ • ͞ΒʹɺEC2ϨΠϠͷΦϖϨʔγϣϯɾ؅ཧίετ͕࡟ݮ • ηΩϡϦςΟύονɺDockerͷύον࡞ۀ͞Α͏ͳΒ • ͍҆͠ɺ͍͍͡ΌΜ 68

Slide 69

Slide 69 text

FargateͷΠΠͱ͜Ζ • Firecrackerͷొ৔Ͱىಈ͕଎͘ͳͬͨ • ENIͷallocation͸ಛʹੲͱൺ΂ͯ଎͘ͳ͍ͬͯΔ • EC2ͷΩϟύ଴ͭඞཁͳ͍͠ɺىಈ΋଎͍ͷͰσϓϩΠɺ੾Γ໭͠ɺεέʔϦϯάͷߴ଎Խ • EC2Ͱͷ՝୊: bridge network͕஗͍͕ɺhostΛ࢖͏ͱϙʔτ͕େมͳ͜ͱʹ ͳΔ͔ΒawsvpcΛݕ౼͕ͨ͠ɺΠϯελϯελΠϓͱαΠζ͋ͨΓͷENI  ෇༩্ݶ਺͕ͱͯ΋গͳ͍ • ࠷ۙENI TrunkingͰ؇࿨͞Ε͍ͯΔ͕ͦΕͰ΋଍Γͳ͔ͬͨ • FargateΛ࢖͏ͱ͖awsvpc͔͠࢖͑ͳ͍͕ɺͦΕͰ͍͍ • Docker Links࢖ͬͯΔϚφʔͷΑΖ͘͠ͳ͍Task͸ɺFargateԽ (+awsvpcԽ) Ͱͳͤ͘Δ 69

Slide 70

Slide 70 text

Fargateͷͪΐͬͱ໘౗ͳͱ͜Ζ • TaskͷCPUͱϝϞϦΛܾΊΒΕͨ૊Έ߹Θͤͷத͔Βબ͹ͳ͍ͱμϝ • ૊Έ߹Θͤ਺΋ଟ͍͠෯޿͘ઃఆͰ͖Δ͕ࣗ෼ͷχʔζʹtailoredͳͪΐ͏Ͳ͍͍Ωϟύʹ͸ ͳΒͳ͍ • AWS͞Μ͕ΫϥελΛ؅ཧͯ͘͠ΕͯΔ͠ɺڐͤΔ • Docker VolumeͰͷϑΝΠϧڞ༗͸ɺ1 volumeͰhard limit 4GB 70

Slide 71

Slide 71 text

͍ͭͰʹ΍Δ͜ͱʹͳͬͨ͜ͱ • ͔ͤͬ͘ServiceɺALBΛ࡞Γ௚͔͢Β΍Γ͍ͨͳͱࢥͬͨ͜ͱ • ALB Least Outstanding RequestsΞϧΰϦζϜͷݕূ • Internal LB༻ͷtarget groupΛServiceʹඥ෇͚Δ • Internal/Global༻ͷTGΛҰͭͷServiceʹඥ෇͚ • લ͸ෳ਺TGͷඥ෇͚͕Ͱ͖ͳ͔ͬͨͨΊInternalઐ༻Service… 71

Slide 72

Slide 72 text

ALB Least Outstanding Requests Ͳ͕ͬͪRound-RobinͰͲ͕ͬͪLORͩͱࢥ͍·͔͢ʁ 72

Slide 73

Slide 73 text

ALB Least Outstanding Requests ਖ਼ղɿ Round-Robin LOR ͙͢໭͢͜ͱʹͳͬͨ 73

Slide 74

Slide 74 text

εέʔϦϯάपΓͷௐ੔ • Datadog APMͷϨΠςϯγɺFargateͷCPU࢖༻཰͕ಉఔ౓ʹͳΔΑ͏ௐ੔ • APMͰͷEC2/Fargateͷ൑ผ͸DD_ENV • DatadogϝτϦΫε͸໊લۭؒ(docker.* vs ecs.*)ͱTask Family໊ • ϩά͸ɺplatform = {“ec2”, “fargate”} ͱ͍͏ϑΟʔϧυ௥Ճ ո͍͠ ո͍͠ چ؀ڥಉ༷ Ϥγʂ چ؀ڥϨΠςϯγ ৽؀ڥϨΠςϯγ 74

Slide 75

Slide 75 text

ׂΓ߹͍มߋͷ֬ೝ on Kibana • platform = {“ec2”, “fargate”} ͳϑΟʔϧυͰ  ΞΫηεϩά͔ΒϦΞϧλΠϜͰՄࢹԽ 75

Slide 76

Slide 76 text

ׂ߹Λ֬ೝɺϤγʂ Τϥʔ֬ೝɺϤγʂ 76

Slide 77

Slide 77 text

LOR੾Γ໭͠ Ϥγʂ ϨΠςϯγ Ϥγʂ ΤϥʔϨʔτ Ϥγʂ εέʔϦϯά Ϥγʂ 77

Slide 78

Slide 78 text

Ҡઃ׬ྃ

Slide 79

Slide 79 text

λοϓϧͷΠϯϑϥपΓͷ͜Ε͔Β • Fargate SpotΛ࢖ͬͨover-provisioningͰٸͳεύΠΫʹରԠ • FireLens + KinesisͳϩάपΓ • FireLens΄Μͱ͍͢͝ΑɺϩάपΓͷ՝୊͸(container nativeͳܗͰ)େ఍ͳΜͱ͔ͳΔ • ͨͩ͠ɺloggingDriverࢦఆʹͳΔͷͰCloudWatch Logsʹ΋FireLensʹ΋ྲྀ͢ͷ͸೉͍͠ • ࠓޙ͜Ε͕վળ͞ΕΔ͜ͱΛفΓͳ͕ΒFeature Request౤͛ ʢͬ'-')‸ = ͟ ͟͞͞ŲƄőƃɹ • ΧελϜεέʔϦϯά • CPUͰ͸ͳ͘ΞΫηε਺ɺͳͲͰΠΠײ͡ͰŧŔŕŰƅźͳεέʔϥΛ࡞͍ͬͯΔ • ϝτϦΫεΛ଴ͨͣʹ1෼ະຬͷϦΫΤετมԽʹରԠ͠ϨΠςϯγΛ͞Βʹ҆ఆԽ • @pahudnet ͞ΜͷŧŔŕŰƅźεέʔϥΛࢀߟʹͭͭ͠ • ΧελϜσϓϩΠ Deployment Controller: EXTERNALͰTask Set APIΛ࢖ͬͨߴ౓ͳαϜγϯάΛ࡞͍ͬͯΔ 79

Slide 80

Slide 80 text

໨࣍ 1. αʔϏε֓ཁ 2. ྺ࢙తܦҢ ΦϯϓϨߏ੒ Ҡઃܭը Ҡઃ࣮ߦ 3. Re: ো֐͔Β࢝ΊΔۓٸҠઃੜ׆ 4. ۓٸҠઃ͔Β͋Δ΂͖࢟΁ EC2 -> ECS (EC2 Type) ECS (EC2 Type) -> ECS (Fargate Type) 5. ॴײ 80

Slide 81

Slide 81 text

ॴײ • ॊೈͰ͋Ε • ౰ॳͷ໨తΛ๨ΕΔͳ • ϩʔυϚοϓΛҾ͘͜ͱ͕େࣄ • ເΛݟΔ͜ͱ͸ѱ͍͜ͱ͡Όͳ͍͕ɺ΍Δ΂͖͜ͱ͸͔ͬ͠Γɺ  ॱ൪ʹ΍͍ͬͯ͘΂͠ • Ϣʔβͷ͜ͱΛߟ͑ͳ͕ΒɺϢʔβମݧΛ࠷༏ઌʹͯ͠࢓ࣄʹऔΓ૊Ή΂͠ • ECS͸͍͍ͧʢAWS Managedͱͷ਌࿨ੑͱֶशۂઢʣ • Fargate͸͍͍ͧʢόοΫΤϯυͷ͜ͱΛ๨ΕͯαʔόϨεʹಀ͍͛ͨਓੜʣ 81

Slide 82

Slide 82 text

એ఻ 82 ← https://baikonur.dev   ʹԿ͔Λग़͠·ͨ͠

Slide 83

Slide 83 text

बۀܕΠϯλʔϯγοϓɺ௨೥ืूதʂ • ΤϯτϦʔక੾: Πϯλʔϯγοϓ։݄࢝ͷ3ϲ݄લͷ݄຤ • ৄࡉ: https://www.cyberagent.co.jp/careers/students/event/detail/id=24422 83