コンテナをSpot Fleetで起動するという選択肢

JAWS-UG コンテナ支部 #7

Transcript

1. ίϯςφΛSpot FleetͰ ্ཱͪ͛Δͱ͍͏બ୒ࢶ JAWS-UG ίϯςφࢧ෦ #7 @wata727
2. @wata727 • Kazuma Watanabe • Engineer in Actcat, Inc. • AWS Solution Architect • https://twitter.com/wata727_
3. Containers in SideCI
4. Sandbox Containers • ࣮ߦ؀ڥΛίϯςφԽ͙ͯ͢͠ഁغ͢Δ • 5,000 containers / day EPDLFS Ubuntu Host GitHub Clone Comment
5. Sandbox Containers • ࣮ߦ؀ڥΛίϯςφԽ͙ͯ͢͠ഁغ͢Δ • 5,000 containers / day EPDLFS Ubuntu Host GitHub Clone Comment
6. Spot Fleet ?
7. εϙοτΠϯελϯεΛ ຊ൪Ͱ࢖ͬͯΔํʁ ʢ༧૝: 10%ʣ ※௥ه: 5%͘Β͍Ͱͨ͠
8. Amazon Spot Instance • ༧උͷEC2ίϯϐϡʔςΟϯάੑೳʹೖࡳ • ͓͍҆ʂʂʂ • ͱ͋Δc4.2xlargeͷྫ: $0.39 → $0.06 ڻ͖ͷ
   
9. ͨͩ͠… • ೖࡳՁ֨ΛԼճΔͱམͪΔ • ߴ͘ͳͬͨΒผͷϓʔϧʢAZͱΠϯελϯε λΠϓͷ૊Έ߹ΘͤʣͷεϙοτΠϯελϯ εΛىಈ͢Δ is ೉͍͠ • ͦΜͳʹ؁͍࿩͸ͳ͍͔…
10. Spot Fleet • εϙοτΠϯελϯεͷλʔήοτ༰ྔΛҡ ࣋͢ΔͨΊͷ࢓૊Έ • ௥Ճྉۚແ͠ • ஋ஈ͕ߴಅͨ͠ΒɺผͷϓʔϧͰҰ൪͍҆ε ϙοτΠϯελϯε͕ࣗಈͰ্ཱ͕ͪΔ
11. Spot Fleetͷઓུ • lowest: બ୒ͨ͠தͰৗʹҰ൪͍҆ϓʔϧ͔Β ىಈ͢Δɺ͍͕҆ߴಅ͢Δͱશ෦མͪΔ • diversified: ՄೳͳݶΓϓʔϧΛ෼ࢄͭͭ͠ɺ ͍҆ॱʹىಈ͢Δɺߴ͍ͱ͖΋͋Δ͕ɺ৑௕ Խͱߴಅ࣌ͷϦεΫܰݮ͕ظ଴Ͱ͖Δ
12. ͸͡Ί͔ͨ • ެࣜͷνϡʔτϦΞϧ͕͔ͳΓஸೡͳͷͰɺ ͦΕʹैͬͯ࡞ͬͯΈΔͱྑͦ͞͏ • ͦΕ͢Β໘౗ͳਓ΁ • https://github.com/wata727/ tf_aws_ecs_on_spotfleet
13. Demo
14. Spot Fleet in SideCI
15. EC2ΛSpot Fleetʹ͠·ͨ͠
16. Spot Fleetʹ͢ΔͨΊʹ • ಥવΠϯελϯε͕མͪͯ΋ࠔΒͳ͍Α͏ʹ ઃܭ͢ΔʢϦτϥΠͱ͔ʣ • ىಈͨ͠Βɺ͙͢ʹαʔϏεΠϯͰ͖ΔΑ͏ ʹϢʔβσʔλͱ͔ઃఆ͓ͯ͘͠ • େମAutoScalingʹඞཁͳ΋ͷͱಉ͡ɺඞવత ʹো֐ʹڧ͍ઃܭʹͳΔ
17. ҆ఆͯ͠Քಇͤ͞ΔͨΊͷTips • diversifiedͱlowestͰ෼͚Δ
18. ҆ఆͯ͠Քಇͤ͞ΔͨΊͷTips • Spot Fleet ؒͷೖࡳՁ֨ʹࠩΛ͚ͭΔ • ೖࡳՁ֨Λ௒͑ͨ࣌఺ͰผͷҰ൪͍҆ϓʔϧ ͔ΒΠϯελϯε͕ىಈ͢Δ͜ͱΛར༻͢Δ • ೖࡳՁ͕͚֨҆Ε͹ɺఆظతʹҰ൪͍҆Πϯ ελϯεϓʔϧ΁ͷ࠶ׂΓ౰͕ͯߦΘΕΔ
19. Ձ֨҆ͨ͘͠Β͙͢མͪͳ͍ʁ • EC2ͷ՝ۚ୯Ґ͸1࣌ؒ͝ͱ͔ͩΒɺ୹ִ͍ؒ Ͱ࠶ىಈͨ͠Β༨ܭʹߴ͘ͳΔΜ͡Ό… • ࣮͸ىಈ1࣌ؒҎ಺ʹεϙοτՁ֨ͷߴಅͰڧ ੍λʔϛωʔτ͞Εͨ৔߹͸ର৅֎ʹͳΔ • http://aws.typepad.com/sajp/2016/11/aws- black-belt-online-seminar-amazon-ec2-spot- instances.html
20. Ͱɺ҆ఆͯ͠·͔͢ʁ • ҙ֎ͱ҆ఆͯ͠·͢ • 2िؒ͘Β͍ฏؾͰੜ͖࢒ͬͯΔΠϯελϯε ͞Μ΋͍·ͨ͠
21. ͱ͋Δm4.2xlargeͷ1೔
22. Spot Fleet ӡ༻Ͱ ϋϚͬͨ͜ͱ
23. ಥવͷىಈΤϥʔ • Spot FleetͰΠϯελϯε͕ىಈ͠ͳ͘ͳͬͨ • VPC಺ͰIPΞυϨε͕֬อͰ͖ͯͳ͍…ʁ • ͦΜͳഅࣛͳͱࢥ͍ͭͭ΋VPCΛ֬ೝ
24. ʊਓਓਓਓਓਓਓਓਓਓʊ ʼɹར༻ՄೳͳIP਺: 1ɹʻ ʉY^Y^Y^Y^Y^Y^Y^Yʉ
25. VPC಺ͷIP͕ރׇ • /25Ͱ128͋Δ͸ͣͷIP͕ރׇ • ݪҼ͸େྔʹੜ੒͞Ε͍ͯͨENI (!) • TerraformͰىಈͨ͠Spot Fleet͸ಛఆͷ৚݅ ԼͰ DeleteOnTermination ͕falseͷΠϯελ ϯεΛىಈ͢Δ
26. PR౤͛·ͨ͠ • Ͱ΋Ϛʔδ͞ΕΔ༧ײແ͠
27. Ձ֨ͷҟৗͳཚߴԼ • Ұ࣌ظɺ࢖ͬͯΔϓʔϧͷ஋ஈ͕ݢฒΈߴಅ ͯ͠શવ҆ఆͯ͠ΠϯελϯεΛ֬อͰ͖ͳ ͍࣌ظ͕͋ͬͨ • ΦϯσϚϯυฒྻىಈͯ྇͗͠·ͨ͠ • ถେ౷ྖબͷλΠϛϯά͕ؔ܎͔ͯͨ͠΋…
28. diversifiedͳͷʹภΔ • ͳ͔ͥdiversifiedͰىಈ͢ΔΠϯελϯε͕ಉ ͡ϓʔϧ͔Βىಈ͍ͯ͠Δ • AWSͷํʹฉ͍ͨͱ͜Ζɺ෼ࢄ͢ΔͳΒ͹3 ୆Ҏ্͸ඞཁΒ͍͠ • ఆظతʹΠϯελϯεΛ࡟আͰ͖Ε͹ɺ࠷҆ ͷϓʔϧ͔Βىಈ͢ΔͷͰ෼ࢄ͢Δ͔΋…ʁ
29. Chaos Monkeyͷػӡ
30. ͪͳΈʹ • Chaos Monkey in Lambda΋͋ΔΑ
31. ·ͱΊ
32. ·ͱΊ • ͓҆͘ͳΔͷ͸΋ͪΖΜɺෆྀͷো֐Λ૝ఆ ͨ͠ઃܭΛ༨ّͳ͘͞ΕΔͷͰɺΞʔΩςΫ νϟ͕ڧਟʹͳΔ • ੲ͸εϙοτΠϯελϯε͸Ұ࣌తͳόον ॲཧ޲͖͕ͩͬͨɺSpot Fleet͕ঢ়گΛม͑ͨ • ࠷ߴͳͷͰΈΜͳ࢖͓͏ʂʂ