仮想化基盤のリソース最適化 / Resource optimization on virtualization platform

ペパボ・はてな技術大会@福岡の発表資料です。

Transcript

1. Ծ૝Խج൫ͷϦιʔε࠷దԽ id:taketo957
 ͸ͯͳɾϖύϘٕज़େձ@෱Ԭ
2. ࣗݾ঺հ • id:taketo957 • ͸ͯͳͰΠϯϑϥ • 2016೥4݄ೖࣾ • ࢼ༻ظ͕ؒऴΘͬͨ • ޿ࠂपΓ • ֶੜ࣌୅ • ਪનͱ͔ • ISUCON
3. ର৅ௌऺ • ओʹֶੜ • ΠϯϑϥΤϯδχΞͬͯͲΜͳ͜ͱͯ͠Δͷʁ • େֶͰษڧͯ͠Δ͜ͱ͕ͲΜͳ෦෼Ͱ໾ཱͭͷʁ • ͦͷଞ • ෳ਺ͷେن໛ͳαʔϏεӡ༻ͷཪଆ
4. ໨࣍ • ͸ͯͳʹ͓͚ΔԾ૝Խج൫ • Ծ૝Խج൫ͷӡ༻্ͷ՝୊ • Ծ૝Խج൫্ͰͷϦιʔε഑ஔͷ࠷దԽ • ·ͱΊ
5. ͸ͯͳʹ͓͚ΔԾ૝Խج൫
6. WebγεςϜͷయܕߏ੒ -# 1SPYZ "QQMJDBUJPO %BUBCBTF
7. WebγεςϜͷయܕߏ੒ -# 1SPYZ "QQMJDBUJPO %BUBCBTF $16 .FN %JTL /FU $16 .FN %JTL /FU $16 .FN %JTL /FU $16 .FN %JTL /FU
8. ͸ͯͳʹ͓͚ΔWebαʔϏε
9. αʔϏεͱϩʔϧ͕
 ͱʹ͔͘ଟ͍
10. ͸ͯͳʹ͓͚ΔԾ૝Խ • ෺ཧαʔόΛԾ૝Խͯ͠Δ • ෺ཧϗετɿ໿xxx୆ • Ծ૝Խϗετɿ໿xxx୆ • αʔϏεɿxxxݸ • ϩʔϧɿxxxݸ • Ծ૝Խج൫ʹ͸XenΛ࢖ͬͯΔ
11. Xenͱ͸ • Ծ૝ϚγϯϞχλ • ෳ਺ͷOSΛฒྻʹಈ࡞ͤ͞Δ͜ͱ͕Ͱ͖Δ • Dom0ͱDomU • Dom0͕Xenͷ؅ཧ΍ϋʔυ΢ΣΞΞΫηεΛ୲͏ • DomU͸ϋʔυ΢ΣΞʹ௚઀ΞΫηε͠ͳ͍ • AWSͷཪଆ΋Xen
12. XenͷΠϝʔδ • ϋΠύʔόΠβ͕ϋʔυ΢ΣΞ্Ͱ࣮ߦ • Xenىಈ࣌ʹDom0͕ىಈ • γεςϜىಈޙɼDomUͳͲ͕ىಈ ϋʔυ΢ΣΞ 9FOϋΠύʔόΠβ %PN %PN6 %PN6 ήετ্ͷϓϩηε ήετ্ͷϓϩηε ήετ্ͷϓϩηε
13. Ծ૝Խαʔόͷߏங@͸ͯͳ • ϋʔυ΢ΣΞϦιʔεͷར༻཰ͷ޲্ • ϩʔϧ͝ͱʹඞཁͳϦιʔε͸
 ͍͍ܾͩͨ·ͬͯΔ • ྫ͑͹ɼ • CPU͕ۭ͍͍ͯΔˠWebαʔό • IO͕ۭ͍͍ͯΔˠDBαʔό • ಉډΛආ͚Δ૊Έ߹Θͤ • ෛՙͷߴ͍΋ͷಉ࢜͸ආ͚Δ • ಉ͡༻్ͷ΋ͷ͸ආ͚Δ • ॏཁͳϗετ͸͚͞Δ %PN ϋʔυ΢ΣΞ %PN6 8FCαʔό %PN6 %#αʔό %PN6 Ωϟογϡαʔό
14. Ծ૝Խج൫ͷӡ༻্ͷ՝୊
15. Ծ૝ԽपΓͷΦϖϨʔγϣϯ • Ծ૝ԽपΓͷΦϖϨʔγϣϯ • ৽ن࡞੒ • ෛՙରࡦ • Ҿͬӽ͠ • ϋʔυ΢ΣΞނো • ࡟আ • αʔϏεఫୀ • ৽ن࡞੒ɺҾͬӽ͠ͷࡍʹ͸ • DomU࡞੒ઌͷDom0Λબఆ͠ͳ͍ͱ͍͚ͳ͍
16. ήετ࡞੒࣌ʹؾʹ͔͚Δ͜ͱʢϦιʔεฤʣ • CPU࢖༻཰ɺDiskIOɺফඅిྗΛؾʹ͔͚Δ • ϩʔϧ͸Կͳͷ͔ • ͦͷϩʔϧ͸Ͳͷ͘Β͍ͷϦιʔεΛফඅ͢Δͷ͔ • ϥοΫͷফඅిྗ͸େৎ෉͔ʁ • ৑௕ੑΛ࣋ͨͤΔ • ిݯ͕མͪͨ৔߹ • Dom0͝ͱࢮΜͩ৔߹ • εΠον͕ࢮΜͩ৔߹ • ਌ͷϝϞϦͱσΟεΫ༰ྔʹ༨༟͸͋Δ͔
17. ήετ࡞੒࣌ʹؾʹ͔͚Δ͜ͱʢࣾ಺ࣄ৘ฤʣ • ݹ͍؀ڥ͸ආ͚Δ • ੲ͔Β͋Δࣺ͍ͯͨ؀ڥʹ͸࡞Βͳ͍ • ࣺͯΔ༧ఆͷ΋ͷʹ͸࡞Βͳ͍ • ϥοΫ΍αϒωοτͷੑ࣭ • ஔ͖৔ॴ͕ܾ·͍ͬͯΔϩʔϧ • ωοτϫʔΫػث • ॏཁͳϗετͱͷಉډ͸ආ͚Δ • etc…
19. Dom0৬ਓͷ஀ੜ
20. ؅ཧπʔϧ͕͋Δͱ͸͍͑
 ๲େͳϗετ͔ΒબͿͷݫ͍͠
21. ୆਺͕૿Ճ͍ͯ͘͠ͱ
 ਓؒͷצͰ͸ରԠͰ͖ͳ͍
22. ࠷దͳ഑ஔΛ͠ͳ͍ͱͲ͏ͳΔ͔ • ಉډϗετʹҾ͖ͣΒΕͯ਌͝ͱࢮ͵ • ϒϨʔΧʔ͕མͪͨΒಉ͡ిݯܥ౷ͷ΋ͷ͕શ໓ • ωοτϫʔΫଳҬ͕٧·ͬͨΓ
23. Dom0બͼ͸
 αʔϏεΛ҆ఆՔಇͤ͞ΔͨΊʹ
 ۃΊͯॏཁͳ࡞ۀ
24. ߟྀ͢Δ߲໨͕ଟ͍͔Β
 બఆʹ͔͔Δίετ΋ߴ͍
25. ίετ͕ߴ͍બఆ࡞ۀΛ
 ࣗಈԽ͍ͨ͠
26. Ծ૝Խج൫ͷϦιʔε࠷దԽ
27. ໰୊ͷ֓ཁ • εϖοΫ͕ܾ·ͬͨDomUΛͲ͜ʹೖΕΔͷ͕࠷ద͔ ˠ૊Έ߹Θͤ࠷దԽ໰୊ʢࠞ߹੔਺ܭը໰୊ʣ %PN ϋʔυ΢ΣΞ $16 .FN /FU %JTL %PN ϋʔυ΢ΣΞ $16 .FN /FU %JTL %PN ϋʔυ΢ΣΞ $16 .FN /FU %JTL %PN6 8FCαʔό $16 .FN /FU %JTL %PN6 %#αʔό $16 .FN /FU %JTL %PN6 1SPYZαʔό $16 .FN /FU %JTL … … ʁ ʁ ʁ
28. ϏϯύοΩϯά໰୊ • ༩͑ΒΕͨՙ෺Λ٧ΊΔശͷ࠷খ਺Λݟ͚ͭΔ໰୊ • ۩ମྫ • 8୆ͷ৽ंΛτϥοΫͰҠಈ͢Δ • ৽ंͷॏྔ
 [33, 61, 58, 41, 50, 21, 60, 64] • τϥοΫͷ༰ྔ
 120 • τϥοΫͷ࠷খ਺ΛٻΊΔ
29. ϏϯύοΩϯά໰୊ͷΠϝʔδ ॏ͞ɿ33 ॏ͞ɿ61 ॏ͞ɿ58 ॏ͞ɿ41 ॏ͞ɿ50 ॏ͞ɿ21 ॏ͞ɿ60 ॏ͞ɿ64 ༰ྔɿ120 ༰ྔɿ120 1 2 3 4 5 6 7 8 1 2 τϥοΫ͸Կ୆ඞཁͳͷ͔ʁ
30. ϏϯύοΩϯά໰୊ͷΠϝʔδ ॏ͞ɿ33 ॏ͞ɿ61 ॏ͞ɿ58 ॏ͞ɿ41 ॏ͞ɿ50 ॏ͞ɿ21 ॏ͞ɿ60 ॏ͞ɿ64 ༰ྔɿ120 ༰ྔɿ120 1 2 3 4 5 6 7 8 1 2 xi,j 2 {0 , 1} 8 i, j yj 2 {0, 1} 8j X j xij = 1 8 i ☓ X i wixij  c 8 j c=120
31. ϏϯύοΩϯά໰୊ͷఆࣜԽ X i yi xi,j 2 {0 , 1} 8 i, j yj 2 {0, 1} 8j X j xij = 1 8 i X i wixij  c 8 j xi,j  yj 8 i, j ໨తؔ਺ ม਺ ੍໿ →࠷খԽ͢Δ
32. DomUׂ౰໰୊ • ϏϯύοΩϯά໰୊Λn࣍ݩʹ֦ுͨ͠΋ͷΛ࢖͏ • ϏϯύοΩϯάͰߟྀͨ͠ͷ͸ॏ͞ͷΈ • ߟྀ͢Δ͜ͱ • CPU਺ʢΦʔόʔίϛοτ཰Λߟྀʣ • ϝϞϦ༰ྔ • σΟεΫ༰ྔ • ಉډͷ૊Έ߹Θͤ • ࣾ಺ͷӡ༻ࣄ৘
33. DomUׂ౰໰୊ͷΠϝʔδ %PN6 $16 .FN %JTL 1 %PN6 $16 .FN %JTL 2 %PN6 $16 .FN %JTL 3 %PN6 $16 .FN %JTL 4 %PN6 $16 .FN %JTL 5 %PN6 $16 .FN %JTL 6 %PN $16 .FN %JTL %PN $16 .FN %JTL 1 2 DomU͸Կ୆ඞཁͳͷ͔ʁ
34. DomUׂ౰໰୊ͷΠϝʔδ %PN6 $16 .FN %JTL 1 %PN6 $16 .FN %JTL 2 %PN6 $16 .FN %JTL 3 %PN6 $16 .FN %JTL 4 %PN6 $16 .FN %JTL 5 %PN6 $16 .FN %JTL 6 %PN $16 .FN %JTL %PN $16 .FN %JTL 1 2 Xi,j 2 {0, 1} 8i, j Yimemi X j mem0 j Xi,j 8i Yicpui X j cpu0 j Xi,j 8i Yidiski X j disk0 j Xi,j 8i Yi 2 {0, 1}8i ☓ X i Xi,j = 1 8j
35. ಉډͷ૊Έ߹ΘͤΛߟྀ %PN6 $16 .FN %JTL 1 %PN6 $16 .FN %JTL 2 %PN6 $16 .FN %JTL 3 %PN6 $16 .FN %JTL 4 %PN6 $16 .FN %JTL 5 %PN6 $16 .FN %JTL 6 %PN $16 .FN %JTL %PN $16 .FN %JTL 1 2 Xi,j 2 {0, 1} 8i, j Yimemi X j mem0 j Xi,j 8i Yicpui X j cpu0 j Xi,j 8i Yidiski X j disk0 j Xi,j 8i Yi 2 {0, 1}8i X j2W ebservers Xi,j  1 8i 8FC 8FC ☓ αʔϏεʹ͍ͭͯ΋ಉ༷ͷ੍໿ͰදݱͰ͖Δ
36. ࣾ಺ͷӡ༻ࣄ৘Λߟྀ %PN6 $16 .FN %JTL 1 %PN6 $16 .FN %JTL 2 %PN6 $16 .FN %JTL 3 %PN6 $16 .FN %JTL 4 %PN6 $16 .FN %JTL 5 %PN6 $16 .FN %JTL 6 %PN $16 .FN %JTL %PN $16 .FN %JTL 1 2 Xi,j 2 {0, 1} 8i, j Yimemi X j mem0 j Xi,j 8i Yicpui X j cpu0 j Xi,j 8i Yidiski X j disk0 j Xi,j 8i Yi 2 {0, 1}8i Xi,j = 0 9i, j 1SPYZ ☓ 1SPYZ༻
37. DomUׂ౰໰୊ͷఆࣜԽ Xi,j 2 {0, 1} 8i, j Yimemi X j mem0 j Xi,j 8i Yicpui X j cpu0 j Xi,j 8i Yidiski X j disk0 j Xi,j 8i Yi 2 {0, 1}8i X i Xi,j = 1 8j ໨తؔ਺ ม਺ ੍໿ X i Yi →࠷খԽ͢Δ Yi Xi,j 8i, j X j2W ebservers Xi,j  1 8i Xi,j = 0 9i, j X j2DBservers Xi,j  1 8i ಉډͷ૊Έ߹Θͤ ӡ༻ࣄ৘
38. ͜ΕΛղ͘
39. ࣮૷ • Python 2.7 • PuLPΛ࢖ͬͨ • શମͷྲྀΕ • DomUͱDom0ͷσʔλΛఆٛ • ม਺ͷఆٛʢࠓճ͸2͚ͭͩʣ
 • ໨తؔ਺ͱ੍໿৚݅Λॻ͖Լ͍ͯ͘͠ X = LpVariable.dicts('X', (range(M), range(N)), 0, 1, LpBinary) Y = LpVariable.dicts('Y', range(M), 0, 1, LpBinary) for i in range(M): m += Y[i] * dom0s[i]["mem"] * 0.8 >= lpSum([domus[j]["mem"] * X[i][j] for j in range(N)])
40. খ͍͞σʔληοτͰݕূ • ن໛ײ • Dom0 73୆ • DomU 284୆ • ࣮ߦ࣌ؒ
 21.537sec • ݁Ռ
 ௚ײతʹ͸ਖ਼͍݁͠Ռɺվળͷ༨஍͸͋Δ
41. ࣮ݱͰ͖Δ͜ͱ • શମͷ഑ஔͷ࠷దԽ • ࠓ·Ͱͷ͸ͬͪ͜ • DCҠసͱ͔͠ͳ͍͔͗Γ͸ग़൪͕ͳ͍ • ཧ૝తͳ഑ஔʹ͠Α͏ͱ͢Δͱॱ൪ʹਓ͕ࢮ͵ • ৽ن௥Ճ࣌ͷDom0બఆͷࣗಈԽ • ੍໿͸ͦͷ··࢖͍·ΘͤΔ • DomU͕1୆ • Dom0͕ঢ়ଶΛ΋ͬͨঢ়ଶͰ։࢝
42. ࠓޙ΍Γ͍ͨ͜ͱ • ߴ଎Խ͍ͨ͠ • Ϧιʔεͷ࣌ܥྻͷมԽΛߟྀ͍ͨ͠ • ωοτϫʔΫతͳτϙϩδʔ΋ߟྀ͍ͨ͠ • ະ஌ͷϩʔϧʹର͢ΔରԠ
43. ʢ৽ଔ͕ʣֶੜʹ఻͍͑ͨ͜ͱ • ݚڀྖҬͰ͸༷ʑͳݚڀ͕ͳ͞Ε͍ͯΔ • େ఍ͷ৔߹Ԡ༻ݚڀͷࣄྫͱ͔΋͋Δ • ࠓճѻͬͨ໰୊΋݁ߏݚڀ͞Εͯͨ • ࣮຿ͷݱ৔ͩͱ • ۩ମతͳ໰୊ɾधཁͱσʔλ͸͋Δ • ద੾ͳख๏Λબ୒ͯ͠ɺࣗ෼ͷ؀ڥʹద༻Ͱ͖Δྗ • جૅతͳ஌ࣝ͸େࣄ • ঢ়گΛద੾ʹϞσϦϯάͰ͖Δྗ • υϝΠϯಛ༗ͷ஌ࣝΛ׆͔ͯ͠໰୊ղܾ
44. ·ͱΊ • ͸ͯͳʹ͓͍ͯ͸DCͷ෺ཧαʔόΛԾ૝Խͯ͠Δ • ϗεςΟϯάܥͱҧͬͯ؀ڥ͕ۉ࣭Ͱͳ͍ • Xenͷӡ༻पΓͷ࿩ͱ՝୊ • Ϧιʔε࠷దԽ • ϏϯύοΩϯά໰୊ͱͦͷ֦ுʹ͍ͭͯ • ӡ༻Λվળ͢ΔͨΊͷπʔϧΛ࡞ͬͨͯΔ
45. ਓ͕ؒ΍ΔՁ஋ͷͳ͍
 ࡞ۀΛ໓΅͍ͨ͠
46. ׬