Lancersをコンテナへ本番移行する取り組み

by adachin0817

Slide 1

Slide 1 text

ϥϯαʔζגࣜձࣾ ϓϩμΫτ։ൃ෦/ٕज़ج൫ SRE ҆ୡ ྋ ʲLancers x dipʳEngineer Meetup #1 LancersΛίϯςφ΁ຊ൪Ҡߦ͢ΔऔΓ૊Έ

Slide 2

Slide 2 text

ࣗݾ঺հ • ҆ୡ ྋ(adachin) • Lancers/SRE   2018/5~ • @adachin0817 • RVIRUS0817 • blog.adachin.me • wiki.adachin.me • ݸਓαʔϏε։ൃத(ਐḿѱ) ʲLancers x dipʳEngineer Meetup #1

Slide 3

Slide 3 text

ࠓճ͓࿩͍ͨ͜͠ͱ • 2018೥ͷγεςϜ • TerraformԽਪਐ • Amazon Linux2Խ • ϩάαʔόʔͷഇࢭ • ։ൃ؀ڥͷվળ • ϑϩϯτΤϯυͷվળ • چ։ൃ؀ڥɺEC2Ͱͷӡ༻՝୊ • Batch/AppΛίϯςφ΁ • ࣗલσϓϩΠγεςϜΛഇࢭ • ෛՙରࡦͱϨεϙϯε • ·ͱΊ ʲLancers x dipʳEngineer Meetup #1

Slide 4

Slide 4 text

ฐࣾʹ͍ͭͯ ʲLancers x dipʳEngineer Meetup #1

Slide 5

Slide 5 text

άϧʔϓձࣾ͸ECS/FargateʹҠߦࡁΈ ʲLancers x dipʳEngineer Meetup #1

Slide 6

Slide 6 text

Ҏલొஃ͍ͯͨ͠εϥΠυ͸ͪ͜Β ʲLancers x dipʳEngineer Meetup #1

Slide 7

Slide 7 text

ʲLancers x dipʳEngineer Meetup #1

Slide 8

Slide 8 text

2018೥ϥϯαʔζͷαʔόʔߏ੒ ʲLancers x dipʳEngineer Meetup #1

Slide 9

Slide 9 text

2021೥ϥϯαʔζͷαʔόʔߏ੒ ʲLancers x dipʳEngineer Meetup #1

Slide 10

Slide 10 text

2021/10~(࣮ࢪ) ʲLancers x dipʳEngineer Meetup #1

Slide 11

Slide 11 text

TerraformԽ ʲLancers x dipʳEngineer Meetup #1

Slide 12

Slide 12 text

Ұ෦TerraformԽΛ͍ͯ͠Δ͕׬શͰ͸ͳ͍ • άϧʔϓձࣾ͸શͯTerraformԽ • ຊՈ͸Ұ෦͔͠TerraformԽ͞Ε͍ͯͳ͍   ɾηΩϡϦςΟʔάϧʔϓIDͳͲΛͦͷ··ࢦఆ   ɾEC2Ͱͷߏங͸ಠࣗͷAWS CLIͰࣗಈԽ   ɾͦΕҎ֎͸खಈͰઃఆ • શͯΛterraform importΛ͢Δʹ͸ݫ͍͠   ɾϑΣʔζ͝ͱʹऔΓ૊Ή͜ͱʹ ʲLancers x dipʳEngineer Meetup #1

Slide 13

Slide 13 text

Terraform Phase 1 • VPC   ɾαϒωοτ   ɾϧʔτςʔϒϧ   ɾΠϯλʔωοτήʔτ΢ΣΠ   ɾNatήʔτ΢ΣΠ   ɾΤϯυϙΠϯτ   ɾηΩϡϦςΟάϧʔϓ • ELB   ɾλʔήοτάϧʔϓ   ɾϦεφʔϧʔϧ ʲLancers x dipʳEngineer Meetup #1 • EC2 • RDS • ACM • εςʔδϯάͱຊ൪؀ڥ߹Θͤͯ   ɾ໿1ϲ݄൒   ɾͦΕҎ֎͸ࠓظରԠ

Slide 14

Slide 14 text

Terraform ։ൃ؀ڥͱCI/CD ʲLancers x dipʳEngineer Meetup #1

Slide 15

Slide 15 text

Amazon Linux 2Խ ʲLancers x dipʳEngineer Meetup #1

Slide 16

Slide 16 text

EC2ΛAmazon Linux 2Խ • Amazon Linux1Ͱӡ༻   ɾAppɺ؅ཧը໘(Admin)ɺBatchαʔόʔ   ɾ2020೥6݄3೔ʹηΩϡϦςΟΞοϓσʔτͷఏڙ͕ऴྃ • άϧʔϓձࣾͷҠߦ͕ॏͳΓରԠͰ͖ͣ • ͞ΒʹArm͕ରԠ͞ΕͯύϑΥʔϚϯε্͕ঢͱίετ࡟ݮ • άϧʔϓձࣾͷίϯςφͷϊ΢ϋ΢͕ߴ·ΓҠߦΛܾҙ   ɾಉ࣌ʹMySQL8Խ΋औΓ૊Ή͜ͱʹ   ʲLancers x dipʳEngineer Meetup #1

Slide 17

Slide 17 text

ϩάαʔόʔͷഇࢭ ʲLancers x dipʳEngineer Meetup #1

Slide 18

Slide 18 text

ϩάαʔόʔ͕୯Ұো֐఺ • ෼ੳج൫(Digdag/Embulk/BigQuery/Redash)Λӡ༻   ɾϩάج൫(Fluentd)   ɾApp(ΞΫηεϩά/ΞϓϦέʔγϣϯϩά)   ɹɾS3ʹू໿ • ਖ਼ৗʹಈ࡞͍ͯ͠ͳ͍ঢ়ଶ͕౓ʑݟड͚ΒΕ   ɾϒϥοΫϘοΫεԽͱ௕೥ͷ՝୊ʹ… • td-agentͷόʔδϣϯΞοϓΛࢼΈ͕ͨ   ɾ͏·͍͔͓ͣ͘ख্͛ঢ়ଶ ʲLancers x dipʳEngineer Meetup #1

Slide 19

Slide 19 text

Amazon KinesisʹҠߦ • CloudWatch Logs(awslogs)Λ࠾༻   ɾ։ൃϝϯόʔͰ΋ௐࠪͰ͖ΔΑ͏ʹ   ɾൺֱతֶशίετ͕௿͍awslogsΛબ୒ • ϩάϙϦγʔΛݟ௚͢   ɾΞΫηεϩά͸S3ʹू໿ͷ··   ɾΞϓϦϩά͸෼ੳ༻్ͱͯ͠ར༻͠ͳ͍ • Datadog LogsͰKinesisͷ࿈ܞ   ɾΞϓϦνʔϜ΋ॠ࣌ʹରԠՄೳ   ɾՄ؍ଌੑΛߴΊΒΕΔΑ͏ʹ • ίϯςφʹҠߦ͢Δ४උ͕੔ͬͨ ʲLancers x dipʳEngineer Meetup #1

Slide 20

Slide 20 text

։ൃ؀ڥͷमਖ਼ ʲLancers x dipʳEngineer Meetup #1

Slide 21

Slide 21 text

։ൃ؀ڥͷमਖ਼ • ։ൃ؀ڥ͸ຊ൪ಉ౳Λ࠶ݱ   ɾAnsibleίϯςφ   ɹɾύοέʔδΠϯετʔϧ • มߋ఺   ɾDocker Image͸amazonlinux2   ɾsupervisorͰϓϩηε؅ཧ   ɾෆཁͳύοέʔδΛચ͍ग़͠࡟আ   ɾamazon-linux-extrasʹΑΔPHPΠϯετʔϧ   ɾtd-agentͷ࡟আͱawslogsͷΠϯετʔϧ • ؅ཧը໘͸ECS/FargateʹҠߦ͢ΔͨΊ   ɾDocker fi leΛҰ͔Βbuild͢ΔΑ͏ʹ ʲLancers x dipʳEngineer Meetup #1

Slide 22

Slide 22 text

2022/03~(Ҡߦ) ʲLancers x dipʳEngineer Meetup #1

Slide 23

Slide 23 text

Ҡߦ͢Δʹ͋ͨͬͯ • ؅ཧը໘(Admin)͸ECS/FargateʹҠߦ   ɾCakePHP2͸EC2   ɾCakePHP4͸ECS/FargateͰಈ࡞ • Batch͸ArmʹҠߦ • App͸ArmʹҠߦͰ͖ͣintelͰӡ༻   ɾDatadog   ɹɾdd-trace-php͕ArmΛαϙʔτ❌ ʲLancers x dipʳEngineer Meetup #1

Slide 24

Slide 24 text

ʲLancers x dipʳEngineer Meetup #1

Slide 25

Slide 25 text

͜͜Ͱٙ໰ʹࢥ͏ํ ΋͍ΔͷͰ͸ ʲLancers x dipʳEngineer Meetup #1

Slide 26

Slide 26 text

App͸ίϯςφʹ   Ҡߦ͠ͳ͍ͷ͔ʁ ʲLancers x dipʳEngineer Meetup #1

Slide 27

Slide 27 text

ϑϩϯτΤϯυͷ   ՝୊ ʲLancers x dipʳEngineer Meetup #1

Slide 28

Slide 28 text

ϑϩϯτΤϯυͷωοΫͱվળ • ιʔείʔυ͕5GBҎ্   ɾyarn installͰ30෼Ҏ্͔͔Δ   ɾίϯςφσϓϩΠͰ͕࣌ؒ๲େʹ͔͔Δ • ϑϩϯτΤϯυΛ෼ׂ͔ͯ͠ΒҠߦ   ɾιʔεߏ଄্ɺύεʹΑΔॲཧ෼ׂ͕ࠔ೉Ͱ޻਺͕͔͔Δ • ༧ΊϩʔΧϧͰϏϧυͯ͠ੜ੒͞ΕͨϑΝΠϧΛGit؅ཧ͢Δӡ༻   ɾyarn install͸ෆཁʹͳΓɺCI͕࣌ؒେ෯ʹ୹ॖ   ɾAppαʔόʔͷίϯςφ΁ҠߦͰ͖Δ৚͕݅੔ͬͨ ʲLancers x dipʳEngineer Meetup #1

Slide 29

Slide 29 text

2022/04(࣮ࢪ) ʲLancers x dipʳEngineer Meetup #1

Slide 30

Slide 30 text

چ։ൃ؀ڥɺEC2Ͱͷ ӡ༻՝୊ ʲLancers x dipʳEngineer Meetup #1

Slide 31

Slide 31 text

SREͷӡ༻՝୊ • AnsibleίϯςφʹΑΔ։ൃ؀ڥͷ౷ҰʹΑΓ޻਺͕͔͔Δ   ɾAppίϯςφΛECRͰڞ༻͢Δඞཁ͕͋Δ   ɾύοέʔδͷߋ৽ͳͲίϯςφΛ࡞Γ௚ͯ͠ECRʹ౎౓push • Φʔτεέʔϧͷ࠶ઃఆͰAppαʔόʔΛ࡞Γ௚͞ͳ͚Ε͹ͳΒͳ͍   ɾAMIΛखಈͰ࡞Γ௚͢ͱ͍͏τΠϧ͕࢒ͬͨ·· • ࣗલͷσϓϩΠγεςϜ͕ଐਓԽ   ɾෆ۩߹ͷରԠʹ޻਺͕͔͔Δ ʲLancers x dipʳEngineer Meetup #1

Slide 32

Slide 32 text

։ൃ؀ڥͷվળ ʲLancers x dipʳEngineer Meetup #1

Slide 33

Slide 33 text

৽։ൃ؀ڥ • Docker fi leʹΑΔҰൠతͳӡ༻ • ผϦϙδτϦ(playbook)   ɾLancersຊମͷϦϙδτϦʹ౷Ұ • Ansibleίϯςφ͸ഇࢭ • ϩάج൫Ͱར༻   ɾEC2ͱಉ༷awslogsΛΠϯετʔϧ ʲLancers x dipʳEngineer Meetup #1

Slide 34

Slide 34 text

BatchΛίϯςφ΁ ʲLancers x dipʳEngineer Meetup #1

Slide 35

Slide 35 text

Batchαʔόʔ͸ECS Scheduled TaskʹҠߦ • όον͸100ݸҎ্   ɾ৑௕ԽͤͣEC2 1୆ͷΈͰӡ༻   ɾαʔόʔμ΢ϯ΍AZো֐͕ى͖Δͱ   ɹɾαʔϏεʹӨڹ͕ग़ͯ͠·͏ঢ়گ • CakePHP4   ɾ80ݸͷΈҠߦ   ɾ։ൃϝϯόʔ͕TerraformͰ؅ཧͰ͖ΔΑ͏ʹ • CloudWatch LogsɺSNSɺLambdaͰΤϥʔ௨஌ • CakePHP2ͷόον͸QAνʔϜ͕Ҡߦத ʲLancers x dipʳEngineer Meetup #1

Slide 36

Slide 36 text

ࣗલσϓϩΠγεςϜ ʹ͍ͭͯ ʲLancers x dipʳEngineer Meetup #1

Slide 37

Slide 37 text

ʮDeploy-Sanʯ • 2015~   ɾRuby on RailsͱJenkins   ɾಠࣗγΣϧͰrsync • Φʔτεέʔϧ࣌   ɾDeploy-san͔Β࠷৽ͷίʔυΛऔಘ   ɾ୯Ұো֐఺ • ϦϦʔε࣌ؒ   ɾࠩ෼ߋ৽Ͱ1෼Ҏ಺ ʲLancers x dipʳEngineer Meetup #1

Slide 38

Slide 38 text

CircleCIʹҠߦ ʲLancers x dipʳEngineer Meetup #1

Slide 39

Slide 39 text

CI/CDΛCircleCIʹҠߦ • MasterϚʔδ:ຊ൪   APIܦ༝:εςʔδϯά/ΧφϦΞ • ίϯςφσϓϩΠߴ଎Խ   ɾ.dockerignoreʹΑΔෆཁͳϑΝΠϧ͸ল͘   ɾDocker fi leͰෆཁͳύοέʔδͷ୨Է͠   ɾCircleCIͷcheckout_shallowΛར༻   ɹɾ࠷ޙͷίϛοτͷΈcheckout   ɾremote-docker-layer-cachingΛ༗ޮԽ • 5෼Ҏ಺·ͰॖΊΔ͜ͱ͕Ͱ͖ͨ ʲLancers x dipʳEngineer Meetup #1

Slide 40

Slide 40 text

ΧφϦΞ؀ڥͷσϓϩΠΛ੍ݶ • ຊ൪DBΛࢀর   ɾΧφϦΞ؀ڥʹ୭Ͱ΋ϦϦʔεͰ͖ͯ͠·͏ • CircleCI   ɾcontext   ɹɾSecurity Group   ɹɾGitHub Teamsͷݖݶ͋ΔਓͷΈঝೝ • https://wiki.adachin.me/archives/1313/ ʲLancers x dipʳEngineer Meetup #1

Slide 41

Slide 41 text

ෛՙରࡦͱϨεϙϯε ʲLancers x dipʳEngineer Meetup #1

Slide 42

Slide 42 text

ෛՙରࡦ • EC2࣌୅   ɾc5ܥΠϯελϯε   ɾECSͩͱCPU͸બ୒Ͱ͖ͳ͍ • Fargate Spot   ɾطଘͷ3୆͸Fargate   ɾΦʔτεέʔϧ͸Fargate Spot(ίετ࡟ݮ) • Φʔτεέʔϧ   ɾ୆਺͸࠷খͰ3୆ɺ࠷େͰ20୆   ɾCPU 20%Ͱ૿Ճ 10%Ͱݮগ • php-fpm΋νϡʔχϯά͠௚ͨ͠ ʲLancers x dipʳEngineer Meetup #1

Slide 43

Slide 43 text

2022/06(Ҡߦ) ʲLancers x dipʳEngineer Meetup #1

Slide 44

Slide 44 text

ʲLancers x dipʳEngineer Meetup #1

Slide 45

Slide 45 text

ҠߦޙͷϨεϙϯεʹ͍ͭͯ • ϥϯαʔζ͸೔தʹෛՙ͕ϐʔΫ • αʔόʔϨεϙϯε͸਺ेmsఔ౓ • ݱࡏ   ɾCPU 60%Ͱ૿Ճ 40%Ͱݮগ   ɾΦʔτεέʔϧ7୆ ʲLancers x dipʳEngineer Meetup #1

Slide 46

Slide 46 text

Ҡߦޙͷίετ ʲLancers x dipʳEngineer Meetup #1

Slide 47

Slide 47 text

·ͱΊ ʲLancers x dipʳEngineer Meetup #1

Slide 48

Slide 48 text

·ͱΊ • ໿1೥Ͱ΍Γ͖Δ͜ͱ͕Ͱ͖ͨ   ɾάϧʔϓձࣾͰͷϊ΢ϋ΢Λ׆͔ͤͨ • ੜ࢈ੑ޲্͕ظ଴   ɾSREͷτΠϧ࡟ݮ   ɾ։ൃ෦શମͷϦϦʔεϑϩʔվળ • ࠓޙ   ɾϑϩϯτΤϯυͷϞμϯԽ   ɾPHP8.1/CakePHPͷόʔδϣϯΞοϓ ʲLancers x dipʳEngineer Meetup #1