Tweet
Tweet

Slide 1

Slide 1 text

ࡾ୐༔հ / Pepabo R&D Institute, GMO Pepabo, Inc. 2018.11.17 Web System Architecture ݚڀձ (WSAݚ) #3 Kaburaya: CPUෛՙʹԠͯ͡ܧଓతʹ ্ݶ஋Λ࠷దԽ͢ΔಈతηϚϑΥ

Slide 2

Slide 2 text

ϓϦϯγύϧΤϯδχΞ ࡾ୐༔հ!NPOPDISPNFHBOF (.0ϖύϘגࣜձࣾϖύϘݚڀॴ IUUQTCMPHNPOPDISPNFHBOFDPN

Slide 3

Slide 3 text

1. The Q. 2. ฒߦॲཧͱνϡʔχϯάʹ͓͚Δ՝୊ 3. CPUෛՙʹԠͯ͡ܧଓతʹ্ݶ஋Λ࠷దԽ͢ΔಈతηϚ ϑΥ 4. ධՁ 5. ·ͱΊ 3 ໨࣍

Slide 4

Slide 4 text

1. The Q.

Slide 5

Slide 5 text

• ॲཧ΍ϥϯλΠϜͷಛੑʹґଘͤͣʹɼϚγϯͷෛՙʹԠͯ͡
 ൓Ԡత͔ͭܧଓతʹɼฒߦ਺Λ࠷దԽ͢Δ͜ͱ͸Մೳ͔ʁ 5 ຊݚڀͷʮ໰͍ʯ

Slide 6

Slide 6 text

2. ฒߦॲཧͱνϡʔχϯάʹ͓͚Δ՝୊

Slide 7

Slide 7 text

• ϚϧνίΞ࣌୅ʹ͓͍ͯɼ୯ҰͷϚγϯͰॲཧੑೳΛ࠷େԽ͢ΔͨΊॲཧͷฒ ߦԽ͕ߦΘΕΔɽ • ࠷దͳฒߦ਺ΛٻΊ͍ͨɽ • ϊϯϒϩοΩϯάͳॲཧํࣜͰ͸࠷దͳฒߦ਺͸CPUίΞ਺ͱ౳͍͠ • ϒϩοΩϯά͕Ұ෦෼Ͱ΋ൃੜ͢Δ৔߹͸CPUίΞ਺Ҏ্͕࠷దʹͳΔ৔ ߹͕ଟ͍ 7 ฒߦॲཧͱ࠷దͳฒߦ਺

Slide 8

Slide 8 text

• ϒϩοΩϯά͕ൃੜ͢Δঢ়گʹ͓͍ͯ͸ɼ࠷దͳฒߦ਺ͷܾఆ͸ґવͱͯ͠։ ൃऀͷܦݧͱ஍ಓͳνϡʔχϯάʹґଘ • nginx, unicornͷworker_processesɼSidekiqͷconcurrencyɼGoroutineͷ ىಈ਺ͳͲ • ࣮ߦ͞ΕΔॲཧͷಛੑͱϥϯλΠϜ΍εέδϡʔϦϯάͷಛੑΛߟྀͨ͠ νϡʔχϯά͕ඞཁ • ͜ΕΒΛશͯߟྀͨ͠ීวతͳνϡʔχϯάख๏ͷൃݟ͸ࠔ೉ 8 ࠷దͳฒߦ਺ΛٻΊΔ

Slide 9

Slide 9 text

• ద੾ͳࢦඪ͕બఆͰ͖Ε͹୯७ͳࢦඪͰ΋ޮՌ͕͋Δ • Ծ૝αʔόͷ༧ଌతΦʔτεέʔϦϯά (*1)Ͱ͸ɼΞϓϦέʔγϣϯॲཧ಺ ༰Ͱ͸ͳ͘ɼӡ༻ܦݧ͔Βಘͨαʔό୯ҐͷεϧʔϓοτΛࢦඪͱͨ͠ɽ • ʮશࣗಈύϥϝʔλνϡʔχϯά͞Μʯ(*2)Ͱ͸ϝτϦΫεΛҰ఺ʹߜΓε ϧʔϓοτ͕࠷େԽ͢Δ஋Λ୳ࡧతʹٻΊͨɽ 9 ैདྷख๏ *1. ࡾ୐ ༔հ, দຊ ྄հ, ྗ෢ ݈࣍, ܀ྛ ݈ଠ࿠, ΞΫηεස౓༧ଌʹجͮ͘Ծ૝αʔόͷܭըతΦʔτεέʔϦϯά, FIT 2018 ୈ17ճ৘ใ Պֶٕज़ϑΥʔϥϜ, CL-002, Sep 2018. *2. https://github.com/mirakui/tuningsan

Slide 10

Slide 10 text

• ༧ଌతɼ୳ࡧతͳํࣜ͸WebαʔόʔϓϩηεͷΑ͏ʹ௕ظՔಈ͢Δ৔߹ʹ ޮՌత͕ͩɼCLIͷΑ͏ͳ୹ظՔಈͰଈ࣌݁Ռ͕ٻΊΒΕΔ৔߹ʹద༻͕೉͠ ͍ 10 ैདྷख๏ • ࢦඪͷݶఆʹՃ͑ɼ൓Ԡత͔ͭޡ͕ࠩগͳ͘ࢦඪ΁௥ै͢Δख๏͕ٻΊΒΕΔ

Slide 11

Slide 11 text

• ॲཧ΍ϥϯλΠϜͷಛੑʹґଘͤͣʹɼϚγϯͷෛՙʹԠͯ͡
 ൓Ԡత͔ͭܧଓతʹɼฒߦ਺Λ࠷దԽ͢Δ͜ͱ͸Մೳ͔ʁ 11 ຊݚڀͷʮ໰͍ʯ

Slide 12

Slide 12 text

3. CPUෛՙʹԠͯ͡ܧଓతʹ্ݶ஋Λ ࠷దԽ͢ΔಈతηϚϑΥ

Slide 13

Slide 13 text

• ॲཧ΍ϥϯλΠϜͷಛੑʹґଘͤͣʹɼϚγϯͷෛՙʹԠͯ͡൓Ԡత͔ͭܧଓ తʹɼฒߦ਺Λ࠷దԽ͢Δ • 1. ॲཧ΍ϥϯλΠϜΛϒϥοΫϘοΫεͱͯ͠ѻ͍ɼܧଓతʹಈ࡞Λ੍ޚ͢ Δ࢓૊ΈͰ͋ΔϑΟʔυόοΫ੍ޚΛ༻͍ͯ࠷దͳฒߦ਺ΛٻΊΔ • 2. ฒߦ਺ͷ੍ޚʹηϚϑΥΛ༻͍Δ • Kaburaya is “ధ໼”. It is japanese arrow with a whistle. • Goݴޠ࣮૷ΛOSSͰެ։: https://github.com/monochromegane/kaburaya 13 Kaburaya ΞʔΩςΫνϟ

Slide 14

Slide 14 text

• ໨ඪ஋ͱग़ྗͷϑΟʔυόοΫͷޡࠩΛ༻͍ͯγεςϜͷೖྗΛܾఆ͠ɼग़ྗ Λ໨ඪ஋ʹ͚ۙͮΔ • ภࠩΛॲཧ͢Δ੍ޚثɼͦͯ͠ೖྗͱग़ྗʹԿΛ༻͍Δ͔ͷઃܭ͕ॏཁ 14 ϑΟʔυόοΫ੍ޚ ੍ޚث γεςϜ ೖྗ ग़ྗ ภࠩ ϑΟʔυόοΫ ໨ඪ஋

Slide 15

Slide 15 text

• ର৅γεςϜ͸ϊϯϒϩοΩϯάॲཧํࣜͷ࠾༻Λલఏͱ͢Δɽ • ϒϩοΩϯάॲཧͷεέδϡʔϦϯάʹΑΓͰ͖Δ͚ͩCPUΛ࢖͍੾Δ • ར༻ՄೳͳCPUϦιʔε͸Ϛγϯ·ͨ͸ෛՙঢ়گʹΑͬͯมಈ͢Δ 15 ੍ޚثͱೖग़ྗͷઃܭ • ϫʔΧʔ਺ΛೖྗɼCPU࢖༻཰Λग़ྗͱ͢Δ • ໨ඪ஋ͱͳΔCPU࢖༻཰͸ঢ়گʹԠͯ͡มԽ͢Δ • ॲཧͷ։࢝ऴྃɼ࣮ߦதʹ࠷খͷϫʔΧʔ਺ͱͳΔΑ͏ௐ੔͢Δ

Slide 16

Slide 16 text

1. ॳظ஋͔ΒϫʔΧʔ਺Λม͑ͳ͍FixController (ൺֱ༻) 2. CPU࢖༻཰100%Λ໨ඪ஋ͱ͠ɼෆ଍ͨ͠৔߹͸ workerΛ1ͣͭ૿Ճͤ͞Δ SimpleController 3. CPU࢖༻཰100%Λ໨ඪͱ͠ɼ໨ඪͱͷࠩ෼ͷKഒΛՃ͑Δ PController(ൺྫ੍ޚ) 4. 3.ͷCPU࢖༻཰ͷ໨ඪ஋Λ90%ͱͨ͠΋ͷ 5. ௚ۙ3؍ଌ఺ͷฏۉΛ໨ඪ஋ͱͯ͠3؍ଌ͝ͱʹPControllerͷ໨ඪ஋ΛมԽͤ͞ΔDynamicController 6. ҰఆظؒͷCPU࢖༻཰ͷඪ४ภࠩΛͱͬͯͦΕ͕Ұఆͷ஋ҎԼͩͬͨΒ҆ఆͨ͠ͱΈͳͯ͠ɼworkerΛݮΒ͍ͯ͠ ͘StabilityController 7. CPU࢖༻཰Ͱ͸ͳͯ͘CPU࢖༻཰ͷมԽ཰Λݩʹ੍ޚ͢ΔRateController 8. 5.ͷDynamicControllerΛݩʹఆظͰ͸ͳ͘େ͖ͳมಈ͝ͱʹ໨ඪ஋Λݟ௚͢ํࣜ 9. 8.ͷDynamicControllerΛݩʹఆৗ࣌ͷෆཁͳworkerΛ࡟ݮ͢Δํࣜ 10. 9.ͷDynamicControllerΛݩʹมಈͷਫ਼౓ͱ଎౓Λ޲্ͤ͞ΔͨΊʹੵ෼ඍ෼੍ޚΛಋೖ͢Δํࣜ 16 ࠷దͳ੍ޚثʹ޲͚ͯ

Slide 17

Slide 17 text

• ηϚϑΥͷ্ݶ஋͕ಈతʹมߋՄೳͰ͋Δ͜ͱ • ௨ৗͷηϚϑΥͱಉ༷ʹPૢ࡞ʹ͓͍ͯ஋͕ෛʹͳΔ৔߹ʹ࣮ߦ͕ϒϩοΫ͞ ΕΔ͜ͱ • ͜ΕΒͷ஋ͷมߋ͕ΞτϛοΫʹߦΘΕΔ͜ͱ 17 ಈతηϚϑΥ • Goݴޠ࣮૷ʹ͓͍ͯɼೋͭͷνϟϯωϧΛ૊Έ߹Θ࣮ͤͯݱ

Slide 18

Slide 18 text

4. ධՁ

Slide 19

Slide 19 text

• ઃܭ੍ͨ͠ޚثͷग़ྗ͢Δىಈ਺͕࠷దͰ͋Δ͜ͱΛݕূ͢Δ • ͜͜Ͱ͍͏࠷దͱ͸ॲཧର৅ͷλεΫશମΛɼ࠷௿ݶͷɼͷ΂ىಈ਺Ͱɼ࠷୹ ͷ࣌ؒͰॲཧͰ͖Δىಈ਺Λࢦ͢ɽ • ධՁʹ͸ɼ൚༻ੑͱ࠶ݱՄೳੑΛߴΊΔͨΊɼγϛϡϨʔλʔΛ༻͍ͨ 19 ੍ޚثͷධՁ

Slide 20

Slide 20 text

• γϛϡϨʔλ͸࣮࣌ؒͰ͸ͳ͘ɼ୯Ґ࣌ؒΛεςοϓͱΈͳ͢ • Job͸WorkloadΛ࣋ͪɼεςοϓ͝ͱͷCPUར༻཰Λఆٛ͢Δ • Workload͕0ͷεςοϓ͸ϒϩοΩϯάॲཧΛදݱ͢Δ • γϛϡϨʔλ͸εςοϓ͝ͱʹ೚ҙͷ਺ͷδϣϒΛ౤ೖ͢Δ • γϛϡϨʔλ͸εςοϓ͝ͱʹىಈՄೳͳϫʔΧʔ਺Λ੍ޚث͔Βऔಘ͢Δ • ϫʔΧʔ͸γϛϡϨʔλͰར༻ՄೳͳCPUར༻཰·Ͱδϣϒͷ֘౰εςοϓͷWorkloadΛফඅ ͢Δ • ফඅͰ͖ͳ͔ͬͨWorkload͸࣍ճͷεςοϓʹճ͞ΕΔ • Workload͕0ͷεςοϓ͸CPUࢿݯΛফඅ͠ͳ͍ͨΊແ৚݅ʹεςοϓΛਐΊΔ 20 γϛϡϨʔλʔཁ݅

Slide 21

Slide 21 text

• ॳظ஋͔ΒϫʔΧʔ਺Λม͑ͳ͍ FixControllerɽ • worker:7 ͙Β͍͕࠷খ͔ͭ࠷୹ͷ ༷ࢠ γϛϡϨʔγϣϯ1 21

Slide 22

Slide 22 text

• CPU࢖༻཰100%Λ໨ඪ஋ͱ͠ɼෆ ଍ͨ͠৔߹͸ workerΛ1ͣͭ૿Ճ͞ ͤΔ SimpleController • ૿Ճ͕௥͍͍͍ͭͯͳ͍ͷͰࡹ͖͖ Δͷʹ͕͔͔࣌ؒͬͯ͠·ͬͨɽ γϛϡϨʔγϣϯ2 22

Slide 23

Slide 23 text

• CPU࢖༻཰90%Λ໨ඪͱ͠ɼ໨ඪͱ ͷࠩ෼ͷKഒΛՃ͑Δ PController(ൺྫ੍ޚ) • K͸0.1ͱͨ͠ɽ • CPU࢖༻཰ͷ໨ඪ஋͕ݻఆͷͨΊɼ ࡹ͖͖ͬͨޙʹ૿Ճ͠ଓ͚ͯ͠·͏ γϛϡϨʔγϣϯ4 23

Slide 24

Slide 24 text

• CPU࢖༻཰ͷ໨ඪ஋͕ݻఆͷͨΊɼ ࡹ͖͖ͬͨޙʹ૿Ճ͠ଓ͚ͯ͠·͏ ໰୊ʹରԠ͢ΔͨΊCPU࢖༻཰ͷ໨ ඪ஋ࣗମΛมಈͤ͞Δ • ௚ۙ3؍ଌ఺ͷฏۉΛ໨ඪ஋ͱͯ͠3 ؍ଌ͝ͱʹPControllerͷ໨ඪ஋Λ มԽͤ͞Δ࢓૊Έ (DynamicController) γϛϡϨʔγϣϯ5 24

Slide 25

Slide 25 text

• ఆظతͰ͸ͳ͘CPUͷେ͖ͳมಈ͝ ͱʹ໨ඪ஋Λݟ௚͢ɽ • ໨ඪ஋Λ੺Ͱϓϩοτ • େ͖ͳ૿ݮΛݩʹΞδϟετ͍ͯ͠ ͘࢓૊Έʹͳͬͨ͜ͱͰɼCPU҆ఆ ࣌ʹੵۃతʹϫʔΧʔΛ࡟ݮ͢Δ͜ ͱ͕Ͱ͖ΔΑ͏ʹͳͬͨ γϛϡϨʔγϣϯ9 25

Slide 26

Slide 26 text

• සൟʹมߋ͞ΕΔ໨ඪ஋ʹߴ଎͔ͭ ޡ͕ࠩগͳ͘దԠ͍ͯͨ͘͠Ίݹయ ੍ޚͷख๏Ͱ͋ΔPID੍ޚΛ༻͍ ͨɽ • ݱࡏͷௐ੔ͷ݁ՌɼPID੍ޚͷύϥ ϝλͱͯ͠Kp: 0.1, Ki: 0.5, Kd: 0.5 ͕σϑΥϧτ஋ͱͯ͠ར༻Ͱ͖ͦ͏ γϛϡϨʔγϣϯ11 26

Slide 27

Slide 27 text

5. ·ͱΊ

Slide 28

Slide 28 text

• ॲཧ΍ϥϯλΠϜͷಛੑʹґଘͤͣʹɼϚγϯͷෛՙʹԠͯ͡൓Ԡత͔ͭܧଓ తʹɼฒߦ਺Λ࠷దԽ͢Δख๏ͱͯ͠ɼCPUෛՙʹԠͯ͡ܧଓతʹ্ݶ஋Λ࠷ దԽ͢ΔಈతηϚϑΥΛఏҊ • γϛϡϨʔγϣϯ؀ڥʹ͓͍ͯɼCPU࢖༻཰ͷ໨ඪ஋Λෛՙ৘ใʹԠͯ͡มಈ ͤ͞Δ੍ޚثͷ༗ޮੑΛ֬ೝ • ຊํࣜ͸ฒߦ਺ΛٻΊͳ͚Ε͹ͳΒͳ͍༷ʑͳ৔໘ʹద༻Մೳ • ࠓޙ͸࣮؀ڥͰͷධՁΛਐΊΔ͜ͱͱPID੍ޚͷύϥϝλઃܭ͕ඞཁ͕ͩ ϑΟʔυόοΫ੍ޚ෼໺ͷϊ΢ϋ΢Λ༗ޮ׆༻ͯ͠ղܾ͍ͨ͠ɽ • దਖ਼ͳϫʔΧʔ਺ΛධՁ͢ΔείΞϦϯάͷઃܭ΋ඞཁ 28 ·ͱΊ

Slide 29

Slide 29 text

No content