仮想化基盤のリソース最適化 / Resource optimization on virtualization platform

仮想化基盤のリソース最適化 / Resource optimization on virtualization platform

ペパボ・はてな技術大会@福岡の発表資料です。

F404d9992c5b98dd5b98ba3a10639646?s=128

Taketo Sasaki

July 09, 2016
Tweet

Transcript

  1. Ծ૝Խج൫ͷϦιʔε࠷దԽ id:taketo957
 ͸ͯͳɾϖύϘٕज़େձ@෱Ԭ

  2. ࣗݾ঺հ • id:taketo957 • ͸ͯͳͰΠϯϑϥ • 2016೥4݄ೖࣾ • ࢼ༻ظ͕ؒऴΘͬͨ •

    ޿ࠂपΓ • ֶੜ࣌୅ • ਪનͱ͔ • ISUCON
  3. ର৅ௌऺ • ओʹֶੜ • ΠϯϑϥΤϯδχΞͬͯͲΜͳ͜ͱͯ͠Δͷʁ • େֶͰษڧͯ͠Δ͜ͱ͕ͲΜͳ෦෼Ͱ໾ཱͭͷʁ • ͦͷଞ •

    ෳ਺ͷେن໛ͳαʔϏεӡ༻ͷཪଆ
  4. ໨࣍ • ͸ͯͳʹ͓͚ΔԾ૝Խج൫ • Ծ૝Խج൫ͷӡ༻্ͷ՝୊ • Ծ૝Խج൫্ͰͷϦιʔε഑ஔͷ࠷దԽ • ·ͱΊ

  5. ͸ͯͳʹ͓͚ΔԾ૝Խج൫

  6. WebγεςϜͷయܕߏ੒ -# 1SPYZ "QQMJDBUJPO %BUBCBTF

  7. WebγεςϜͷయܕߏ੒ -# 1SPYZ "QQMJDBUJPO %BUBCBTF $16 .FN %JTL /FU $16

    .FN %JTL /FU $16 .FN %JTL /FU $16 .FN %JTL /FU
  8. ͸ͯͳʹ͓͚ΔWebαʔϏε

  9. αʔϏεͱϩʔϧ͕
 ͱʹ͔͘ଟ͍

  10. ͸ͯͳʹ͓͚ΔԾ૝Խ • ෺ཧαʔόΛԾ૝Խͯ͠Δ • ෺ཧϗετɿ໿xxx୆ • Ծ૝Խϗετɿ໿xxx୆ • αʔϏεɿxxxݸ •

    ϩʔϧɿxxxݸ • Ծ૝Խج൫ʹ͸XenΛ࢖ͬͯΔ
  11. Xenͱ͸ • Ծ૝ϚγϯϞχλ • ෳ਺ͷOSΛฒྻʹಈ࡞ͤ͞Δ͜ͱ͕Ͱ͖Δ • Dom0ͱDomU • Dom0͕Xenͷ؅ཧ΍ϋʔυ΢ΣΞΞΫηεΛ୲͏ •

    DomU͸ϋʔυ΢ΣΞʹ௚઀ΞΫηε͠ͳ͍ • AWSͷཪଆ΋Xen
  12. XenͷΠϝʔδ • ϋΠύʔόΠβ͕ϋʔυ΢ΣΞ্Ͱ࣮ߦ • Xenىಈ࣌ʹDom0͕ىಈ • γεςϜىಈޙɼDomUͳͲ͕ىಈ ϋʔυ΢ΣΞ 9FOϋΠύʔόΠβ %PN

    %PN6 %PN6 ήετ্ͷϓϩηε ήετ্ͷϓϩηε ήετ্ͷϓϩηε
  13. Ծ૝Խαʔόͷߏங@͸ͯͳ • ϋʔυ΢ΣΞϦιʔεͷར༻཰ͷ޲্ • ϩʔϧ͝ͱʹඞཁͳϦιʔε͸
 ͍͍ܾͩͨ·ͬͯΔ • ྫ͑͹ɼ • CPU͕ۭ͍͍ͯΔˠWebαʔό

    • IO͕ۭ͍͍ͯΔˠDBαʔό • ಉډΛආ͚Δ૊Έ߹Θͤ • ෛՙͷߴ͍΋ͷಉ࢜͸ආ͚Δ • ಉ͡༻్ͷ΋ͷ͸ආ͚Δ • ॏཁͳϗετ͸͚͞Δ %PN ϋʔυ΢ΣΞ %PN6 8FCαʔό %PN6 %#αʔό %PN6 Ωϟογϡαʔό
  14. Ծ૝Խج൫ͷӡ༻্ͷ՝୊

  15. Ծ૝ԽपΓͷΦϖϨʔγϣϯ • Ծ૝ԽपΓͷΦϖϨʔγϣϯ • ৽ن࡞੒ • ෛՙରࡦ • Ҿͬӽ͠ •

    ϋʔυ΢ΣΞނো • ࡟আ • αʔϏεఫୀ • ৽ن࡞੒ɺҾͬӽ͠ͷࡍʹ͸ • DomU࡞੒ઌͷDom0Λબఆ͠ͳ͍ͱ͍͚ͳ͍
  16. ήετ࡞੒࣌ʹؾʹ͔͚Δ͜ͱʢϦιʔεฤʣ • CPU࢖༻཰ɺDiskIOɺফඅిྗΛؾʹ͔͚Δ • ϩʔϧ͸Կͳͷ͔ • ͦͷϩʔϧ͸Ͳͷ͘Β͍ͷϦιʔεΛফඅ͢Δͷ͔ • ϥοΫͷফඅిྗ͸େৎ෉͔ʁ •

    ৑௕ੑΛ࣋ͨͤΔ • ిݯ͕མͪͨ৔߹ • Dom0͝ͱࢮΜͩ৔߹ • εΠον͕ࢮΜͩ৔߹ • ਌ͷϝϞϦͱσΟεΫ༰ྔʹ༨༟͸͋Δ͔
  17. ήετ࡞੒࣌ʹؾʹ͔͚Δ͜ͱʢࣾ಺ࣄ৘ฤʣ • ݹ͍؀ڥ͸ආ͚Δ • ੲ͔Β͋Δࣺ͍ͯͨ؀ڥʹ͸࡞Βͳ͍ • ࣺͯΔ༧ఆͷ΋ͷʹ͸࡞Βͳ͍ • ϥοΫ΍αϒωοτͷੑ࣭ •

    ஔ͖৔ॴ͕ܾ·͍ͬͯΔϩʔϧ • ωοτϫʔΫػث • ॏཁͳϗετͱͷಉډ͸ආ͚Δ • etc…
  18. None
  19. Dom0৬ਓͷ஀ੜ

  20. ؅ཧπʔϧ͕͋Δͱ͸͍͑
 ๲େͳϗετ͔ΒબͿͷݫ͍͠

  21. ୆਺͕૿Ճ͍ͯ͘͠ͱ
 ਓؒͷצͰ͸ରԠͰ͖ͳ͍

  22. ࠷దͳ഑ஔΛ͠ͳ͍ͱͲ͏ͳΔ͔ • ಉډϗετʹҾ͖ͣΒΕͯ਌͝ͱࢮ͵ • ϒϨʔΧʔ͕མͪͨΒಉ͡ిݯܥ౷ͷ΋ͷ͕શ໓ • ωοτϫʔΫଳҬ͕٧·ͬͨΓ

  23. Dom0બͼ͸
 αʔϏεΛ҆ఆՔಇͤ͞ΔͨΊʹ
 ۃΊͯॏཁͳ࡞ۀ

  24. ߟྀ͢Δ߲໨͕ଟ͍͔Β
 બఆʹ͔͔Δίετ΋ߴ͍

  25. ίετ͕ߴ͍બఆ࡞ۀΛ
 ࣗಈԽ͍ͨ͠

  26. Ծ૝Խج൫ͷϦιʔε࠷దԽ

  27. ໰୊ͷ֓ཁ • εϖοΫ͕ܾ·ͬͨDomUΛͲ͜ʹೖΕΔͷ͕࠷ద͔ ˠ૊Έ߹Θͤ࠷దԽ໰୊ʢࠞ߹੔਺ܭը໰୊ʣ %PN ϋʔυ΢ΣΞ $16 .FN /FU %JTL

    %PN ϋʔυ΢ΣΞ $16 .FN /FU %JTL %PN ϋʔυ΢ΣΞ $16 .FN /FU %JTL %PN6 8FCαʔό $16 .FN /FU %JTL %PN6 %#αʔό $16 .FN /FU %JTL %PN6 1SPYZαʔό $16 .FN /FU %JTL … … ʁ ʁ ʁ
  28. ϏϯύοΩϯά໰୊ • ༩͑ΒΕͨՙ෺Λ٧ΊΔശͷ࠷খ਺Λݟ͚ͭΔ໰୊ • ۩ମྫ • 8୆ͷ৽ंΛτϥοΫͰҠಈ͢Δ • ৽ंͷॏྔ
 [33,

    61, 58, 41, 50, 21, 60, 64] • τϥοΫͷ༰ྔ
 120 • τϥοΫͷ࠷খ਺ΛٻΊΔ
  29. ϏϯύοΩϯά໰୊ͷΠϝʔδ ॏ͞ɿ33 ॏ͞ɿ61 ॏ͞ɿ58 ॏ͞ɿ41 ॏ͞ɿ50 ॏ͞ɿ21 ॏ͞ɿ60 ॏ͞ɿ64 ༰ྔɿ120

    ༰ྔɿ120 1 2 3 4 5 6 7 8 1 2 τϥοΫ͸Կ୆ඞཁͳͷ͔ʁ
  30. ϏϯύοΩϯά໰୊ͷΠϝʔδ ॏ͞ɿ33 ॏ͞ɿ61 ॏ͞ɿ58 ॏ͞ɿ41 ॏ͞ɿ50 ॏ͞ɿ21 ॏ͞ɿ60 ॏ͞ɿ64 ༰ྔɿ120

    ༰ྔɿ120 1 2 3 4 5 6 7 8 1 2 xi,j 2 {0 , 1} 8 i, j yj 2 {0, 1} 8j X j xij = 1 8 i ☓ X i wixij  c 8 j c=120
  31. ϏϯύοΩϯά໰୊ͷఆࣜԽ X i yi xi,j 2 {0 , 1} 8

    i, j yj 2 {0, 1} 8j X j xij = 1 8 i X i wixij  c 8 j xi,j  yj 8 i, j ໨తؔ਺ ม਺ ੍໿ →࠷খԽ͢Δ
  32. DomUׂ౰໰୊ • ϏϯύοΩϯά໰୊Λn࣍ݩʹ֦ுͨ͠΋ͷΛ࢖͏ • ϏϯύοΩϯάͰߟྀͨ͠ͷ͸ॏ͞ͷΈ • ߟྀ͢Δ͜ͱ • CPU਺ʢΦʔόʔίϛοτ཰Λߟྀʣ •

    ϝϞϦ༰ྔ • σΟεΫ༰ྔ • ಉډͷ૊Έ߹Θͤ • ࣾ಺ͷӡ༻ࣄ৘
  33. DomUׂ౰໰୊ͷΠϝʔδ %PN6 $16 .FN %JTL 1 %PN6 $16 .FN %JTL

    2 %PN6 $16 .FN %JTL 3 %PN6 $16 .FN %JTL 4 %PN6 $16 .FN %JTL 5 %PN6 $16 .FN %JTL 6 %PN $16 .FN %JTL %PN $16 .FN %JTL 1 2 DomU͸Կ୆ඞཁͳͷ͔ʁ
  34. DomUׂ౰໰୊ͷΠϝʔδ %PN6 $16 .FN %JTL 1 %PN6 $16 .FN %JTL

    2 %PN6 $16 .FN %JTL 3 %PN6 $16 .FN %JTL 4 %PN6 $16 .FN %JTL 5 %PN6 $16 .FN %JTL 6 %PN $16 .FN %JTL %PN $16 .FN %JTL 1 2 Xi,j 2 {0, 1} 8i, j Yimemi X j mem0 j Xi,j 8i Yicpui X j cpu0 j Xi,j 8i Yidiski X j disk0 j Xi,j 8i Yi 2 {0, 1}8i ☓ X i Xi,j = 1 8j
  35. ಉډͷ૊Έ߹ΘͤΛߟྀ %PN6 $16 .FN %JTL 1 %PN6 $16 .FN %JTL

    2 %PN6 $16 .FN %JTL 3 %PN6 $16 .FN %JTL 4 %PN6 $16 .FN %JTL 5 %PN6 $16 .FN %JTL 6 %PN $16 .FN %JTL %PN $16 .FN %JTL 1 2 Xi,j 2 {0, 1} 8i, j Yimemi X j mem0 j Xi,j 8i Yicpui X j cpu0 j Xi,j 8i Yidiski X j disk0 j Xi,j 8i Yi 2 {0, 1}8i X j2W ebservers Xi,j  1 8i 8FC 8FC ☓ αʔϏεʹ͍ͭͯ΋ಉ༷ͷ੍໿ͰදݱͰ͖Δ
  36. ࣾ಺ͷӡ༻ࣄ৘Λߟྀ %PN6 $16 .FN %JTL 1 %PN6 $16 .FN %JTL

    2 %PN6 $16 .FN %JTL 3 %PN6 $16 .FN %JTL 4 %PN6 $16 .FN %JTL 5 %PN6 $16 .FN %JTL 6 %PN $16 .FN %JTL %PN $16 .FN %JTL 1 2 Xi,j 2 {0, 1} 8i, j Yimemi X j mem0 j Xi,j 8i Yicpui X j cpu0 j Xi,j 8i Yidiski X j disk0 j Xi,j 8i Yi 2 {0, 1}8i Xi,j = 0 9i, j 1SPYZ ☓ 1SPYZ༻
  37. DomUׂ౰໰୊ͷఆࣜԽ Xi,j 2 {0, 1} 8i, j Yimemi X j

    mem0 j Xi,j 8i Yicpui X j cpu0 j Xi,j 8i Yidiski X j disk0 j Xi,j 8i Yi 2 {0, 1}8i X i Xi,j = 1 8j ໨తؔ਺ ม਺ ੍໿ X i Yi →࠷খԽ͢Δ Yi Xi,j 8i, j X j2W ebservers Xi,j  1 8i Xi,j = 0 9i, j X j2DBservers Xi,j  1 8i ಉډͷ૊Έ߹Θͤ ӡ༻ࣄ৘
  38. ͜ΕΛղ͘

  39. ࣮૷ • Python 2.7 • PuLPΛ࢖ͬͨ • શମͷྲྀΕ • DomUͱDom0ͷσʔλΛఆٛ

    • ม਺ͷఆٛʢࠓճ͸2͚ͭͩʣ
 • ໨తؔ਺ͱ੍໿৚݅Λॻ͖Լ͍ͯ͘͠ X = LpVariable.dicts('X', (range(M), range(N)), 0, 1, LpBinary) Y = LpVariable.dicts('Y', range(M), 0, 1, LpBinary) for i in range(M): m += Y[i] * dom0s[i]["mem"] * 0.8 >= lpSum([domus[j]["mem"] * X[i][j] for j in range(N)])
  40. খ͍͞σʔληοτͰݕূ • ن໛ײ • Dom0 73୆ • DomU 284୆ •

    ࣮ߦ࣌ؒ
 21.537sec • ݁Ռ
 ௚ײతʹ͸ਖ਼͍݁͠Ռɺվળͷ༨஍͸͋Δ
  41. ࣮ݱͰ͖Δ͜ͱ • શମͷ഑ஔͷ࠷దԽ • ࠓ·Ͱͷ͸ͬͪ͜ • DCҠసͱ͔͠ͳ͍͔͗Γ͸ग़൪͕ͳ͍ • ཧ૝తͳ഑ஔʹ͠Α͏ͱ͢Δͱॱ൪ʹਓ͕ࢮ͵ •

    ৽ن௥Ճ࣌ͷDom0બఆͷࣗಈԽ • ੍໿͸ͦͷ··࢖͍·ΘͤΔ • DomU͕1୆ • Dom0͕ঢ়ଶΛ΋ͬͨঢ়ଶͰ։࢝
  42. ࠓޙ΍Γ͍ͨ͜ͱ • ߴ଎Խ͍ͨ͠ • Ϧιʔεͷ࣌ܥྻͷมԽΛߟྀ͍ͨ͠ • ωοτϫʔΫతͳτϙϩδʔ΋ߟྀ͍ͨ͠ • ະ஌ͷϩʔϧʹର͢ΔରԠ

  43. ʢ৽ଔ͕ʣֶੜʹ఻͍͑ͨ͜ͱ • ݚڀྖҬͰ͸༷ʑͳݚڀ͕ͳ͞Ε͍ͯΔ • େ఍ͷ৔߹Ԡ༻ݚڀͷࣄྫͱ͔΋͋Δ • ࠓճѻͬͨ໰୊΋݁ߏݚڀ͞Εͯͨ • ࣮຿ͷݱ৔ͩͱ •

    ۩ମతͳ໰୊ɾधཁͱσʔλ͸͋Δ • ద੾ͳख๏Λબ୒ͯ͠ɺࣗ෼ͷ؀ڥʹద༻Ͱ͖Δྗ • جૅతͳ஌ࣝ͸େࣄ • ঢ়گΛద੾ʹϞσϦϯάͰ͖Δྗ • υϝΠϯಛ༗ͷ஌ࣝΛ׆͔ͯ͠໰୊ղܾ
  44. ·ͱΊ • ͸ͯͳʹ͓͍ͯ͸DCͷ෺ཧαʔόΛԾ૝Խͯ͠Δ • ϗεςΟϯάܥͱҧͬͯ؀ڥ͕ۉ࣭Ͱͳ͍ • Xenͷӡ༻पΓͷ࿩ͱ՝୊ • Ϧιʔε࠷దԽ •

    ϏϯύοΩϯά໰୊ͱͦͷ֦ுʹ͍ͭͯ • ӡ༻Λվળ͢ΔͨΊͷπʔϧΛ࡞ͬͨͯΔ
  45. ਓ͕ؒ΍ΔՁ஋ͷͳ͍
 ࡞ۀΛ໓΅͍ͨ͠

  46. ׬