$30 off During Our Annual Pro Sale. View Details »

分散アプリケーションの高信頼化のための
運用技術に関する研究 / A Study on Operation Technology for High Reliability of Distributed Applications

分散アプリケーションの高信頼化のための
運用技術に関する研究 / A Study on Operation Technology for High Reliability of Distributed Applications

京都大学大学院 情報学研究科 知能情報学専攻 
中間発表(15min)。

Yuuki Tsubouchi (yuuk1)

August 05, 2021
Tweet

More Decks by Yuuki Tsubouchi (yuuk1)

Other Decks in Research

Transcript

  1. ෼ࢄΞϓϦέʔγϣϯͷߴ৴པԽͷͨΊͷ 
 ӡ༻ٕज़ʹؔ͢Δݚڀ ɾΫϥ΢υΛ࢖༻ͨ͠ΞϓϦέʔγϣϯͷ৴པੑΛ޲্ͤ͞Δ ΞʔΩςΫνϟΛݚڀ͍ͯ͠Δɻ ɾಛʹɺγεςϜͷӡ༻ঢ়گΛࣔ͢σʔλΛޮ཰తʹऩू͢Δख ๏ʹߩݙͨ͠ɻ ɾݱࡏ͸ɺऩू͞Εͨσʔλ͔ΒɺγεςϜʹൃੜ͢Δҟৗͷݪ ҼΛਝ଎ʹಛఆ͢ΔͨΊͷσʔλ෼ੳख๏Λݚڀ͍ͯ͠Δɻ ௶಺

    ༎थ 
 ژ౎େֶେֶӃ ৘ใֶݚڀՊ ஌ೳ৘ใֶઐ߈ ത࢜ޙظ՝ఔ 2೥ 
 தؒൃද 2021೥8݄5೔
  2. 2 1. Ϋϥ΢υ্ͷ෼ࢄΞϓϦέʔγϣϯ Ϋϥ΢υ্ͷ෼ࢄΞϓϦέʔγϣϯͷෳࡶԽ ৴པੑͷ՝୊: ӡ༻σʔλྔͷ૿Ճ 
 2. ݚڀख๏ ӡ༻σʔλͷऩूͷߴޮ཰Խ

    • [ݚڀ੒Ռ1] ଟ਺ͷ࣌ܥྻσʔλΛૠೖɾอଘ͢ΔͨΊͷσʔλϕʔε • [ݚڀ੒Ռ2] ωοτϫʔΫ௨৴ͷґଘؔ܎ͷ௿Φʔόϔουऩू ӡ༻σʔλͷղੳ • [ݚڀ੒Ռ3] ࣌ܥྻσʔλͷத͔ΒݪҼ਍அʹ༗༻ͳܥྻΛநग़ 3. ·ͱΊ/ݚڀ࣮੷ ൃද಺༰ͷߏ੒
  3. 3 Ϋϥ΢υ্ͷ෼ࢄΞϓϦέʔγϣϯͷෳࡶԽ ɾ ɾ ɾ ΞΫηε૿Ճ εέʔϧΞ΢τʹΑΔ ϗετ਺ͷ૿Ճ ػೳͷ૿Ճ ΑΓେن໛ͳ

    
 ෼ࢄΞʔΩςΫνϟ ϛυϧ΢ΣΞͷ૿Ճ RDBαʔό Ωϟογϡ 
 αʔό ݕࡧαʔό Webαʔό ωοτϫʔΫ 
 αʔϏε TCP/UDP ɾ ɾ ɾ ϝσΟΞɺECαΠτɺSNSɺIoTͳͲΛߏ੒͢Δ෼ࢄΞϓϦέʔγϣϯ
  4. 4 ɾίϯϙʔωϯτ਺ͷ૿େʹΑΓɺӡ༻σʔλྔ͕૿Ճ ɾσʔλऩूͷͨΊͷܭࢉػͷෛՙ͕૿Ճ ɾγεςϜ؅ཧऀͷೝ஌ෛՙ͕૿Ճ ෳࡶԽͨ݁͠Ռͷ৴པੑͷ௿Լ ӡ༻σʔλͷ 
 ऩू ɾෛՙঢ়گΛࣔ࣌͢ܥྻσʔλ ɾωοτϫʔΫ௨৴ͷґଘؔ܎

    ɾςΩετϩά ৴པੑͷ௿Լ΁ ো֐෮چ࣌ؒͷ૿େ
  5. 5 ɾӡ༻σʔλͷऩूΛߴޮ཰Խ͢Δ ܭࢉػͷෛՙΛ௿ݮ [ݚڀ੒Ռ1] ࣌ܥྻσʔλΛߴޮ཰ʹૠೖɾอଘ͢Δσʔλϕʔε [ݚڀ੒Ռ2] ωοτϫʔΫ௨৴ͷґଘؔ܎ͷ௿Φʔόϔουऩू ໨త: ӡ༻σʔλͷऩूͱղੳͷߴ౓ԽʹΑΓ৴པੑΛ޲্ ͍ͣΕ΋࣮؀ڥ΁ͷద༻༰қੑΛߟྀ͢Δ

    ɾӡ༻σʔλΛղੳ͠ɺো֐ରԠʹ༗༻ͳಎ࡯ΛಘΔ ೝ஌ෛՙΛ௿ݮ [ݚڀ੒Ռ3] ࣌ܥྻσʔλͷத͔ΒݪҼ਍அʹ༗༻ͳܥྻΛநग़ ʢ↪࿦จࢽ౤ߘதʣ ʢ↪࿦จࢽ࠾࿥ࡁʣ ʢ↪ࠃࡍձٞ౤ߘ४උதʣ
  6. 6 ɾҰൠʹɺσΟεΫϕʔεDB͸ɺࡧҾߏ଄ʹฏߧ໦Λ࢖༻͢Δ ɾ՝୊ɿܥྻ਺(n)ͷ૿Ճʹରͯ͠ɺܭࢉྔ͕O(log n)Ͱ૿Ճ͢Δ ɾ؅ཧͷෛ୲Λܰݮ͢ΔͨΊʹɺطଘͷDB্ʹߏ੒͍ͨ͠ [ݚڀ੒Ռ1] ࣌ܥྻσʔλͷऩूͷߴޮ཰Խ ௶಺༎थ΄͔, HeteroTSDB: ҟछ෼ࢄKVSؒͷࣗಈ֊૚ԽʹΑΔߴੑೳͳ࣌ܥྻσʔλϕʔε,

    ৘ใॲཧֶձ࿦จࢽ, Vol.62, No.3, pp.818-828, 2021೥3݄. ϝϞϦ 
 ϕʔεDB σΟεΫ 
 ϕʔεDB ϋογϡද O(k) ฏߧ໦ O(log n) σʔλ఺ૠೖ dݸͷσʔλ఺Λ஝ੵޙʹ 
 ϑϥογϡॻ͖ࠐΈ ղܾɿϝϞϦϕʔεDBͱσΟεΫϕʔεDBͷ֊૚ԽΛఏҊ M (insertions/s) M / d (insertions/s)
  7. 7 [ݚڀ੒Ռ1] ܥྻ਺ͷ૿Ճʹର͢ΔૠೖεϧʔϓοτมԽ ɾσΟεΫϕʔεDBͷΈͱൺֱ͠ɺ ࠷େͰ3.96ഒͷεϧʔϓοτ ɾεϧʔϓοτͷ௿Լ཰΋վળͨ͠ 0 20 40 60

    80 100 100 1K 10K 100k 1M 0 20 40 60 Insertion throughput (kilo datapoints / sec) Throughput decrease rate (%) The number of series HeteroTSDB (Proposed) KairosDB HeteroTSDB (Proposed) KairosDB ϝϞϦDB → σΟεΫDBͷҠಈεϧʔ ϓοτͱϝϞϦDB΁ͷεϧʔϓοτͱ ಉఔ౓ 0 20 40 60 80 100 0 300 600 900 1200 1500 1800 0 500 1000 1500 2000 Insertion throughput (kilo datapoints / sec) Memory used size (MB) Elapsed time (sec) Flushed datapoints (/sec) Memory used size (MB) ௶಺༎थ΄͔, HeteroTSDB: ҟछ෼ࢄKVSؒͷࣗಈ֊૚ԽʹΑΔߴੑೳͳ࣌ܥྻσʔλϕʔε, ৘ใॲཧֶձ࿦จࢽ, Vol.62, No.3, pp.818-828, 2021೥3݄.
  8. (౤ߘத) Y. Tsubouchi, et al., Low Overhead TCP/UDP Socket-based Tracing

    for Discovering Network Services Dependencies, Journal of Information Processing 2022. [ݚڀ੒Ռ2] ωοτϫʔΫ௨৴ͷґଘؔ܎औಘͷޮ཰Խ . . . Kernel User Service Socket Tracing 
 Process … Event Event Event ετϦʔϛϯά๏(Weave Scope) ϑϩʔू໿๏ ([Datadog], [SAC 20]) ϑϩʔूଋ๏ʢఏҊʣ . . . Kernel Service Socket Tracing 
 Process . . . Event Flow Event Event Event … … . . . . . . User Service Socket Tracing 
 Process . . . ✗ ΧʔωϧˠϢʔβۭؒؒ ͷΠϕϯτͷίϐʔίετ ✗ TCP઀ଓϨʔτ͕૿Ճ͢Δ ͱɺίϐʔίετ͕૿Ճ ෳ਺ͷϑϩʔΛूଋ ϑϩʔ= ྆୺ͷΞυϨεͱϙʔτͷ ૊ʢλϓϧʣ͕ಉҰͷ௨৴୯Ґ Event Event … … Event Event . . . Event Event … Event Event . . . Ұൠతʹ௨৴͸OSΧʔωϧͷTCP/UDPΛ࢖༻͢Δ͜ͱ ʹண໨ 8
  9. 9 ɾఏҊख๏͸ɺCPUར༻཰͸2.2%ҎԼɻ ɾϑϩʔ਺ͷ૿େʹରͯ͠ɺϑϩʔूଋʹΑΓɺ 
 CPUར༻཰Λ௿͘ҡ͍࣋ͯ͠Δ [ݚڀ੒Ռ2] ϑϩʔ਺ͷ૿େʹର͢ΔCPU࢖༻ྔͷมԽ ఏҊख๏ (౤ߘத) Y.

    Tsubouchi, et al., Low Overhead TCP/UDP Socket-based Tracing for Discovering Network Services Dependencies, Journal of Information Processing 2022. 0 5 10 15 20 25 5 10 15 20 25 30 35 CPU usage / core (%) TCP round trips / sec (x103) Streaming(client) Streaming(server) In-Kernel-Aggr(client) In-Kernel-Aggr(server) In-Kernel-Bundling(client) In-Kernel-Bundling(server)
  10. εςοϓ1: ظؒதͷมಈͷ౓߹͍͕খ͍͞ܥྻΛআڈ εςοϓ2: άϥϑͷܗঢ়͕ྨࣅ͢Δ΋ͷ͔Β୅දܥྻΛநग़ 10 ɾ՝୊: ίϯϙʔωϯτ͝ͱʹɺಛ௃ྔͱͯ͠ܥྻͷछผʢCPU࢖༻ɺϝϞϦ࢖ ༻ɺωοτϫʔΫଳҬɺ…ʣΛࢦఆ͢Δඞཁ͋Γ ɾղܾ: ؔ࿈͢ΔશܥྻΛೖྗͱͯ͠༩͑ɺݪҼ਍அʹ༗༻ͳܥྻΛߴ଎ʹநग़

    [ݚڀ੒Ռ3] ࣌ܥྻσʔλ͔ΒݪҼ਍அʹ༗༻ͳܥྻΛநग़ ҟৗ 
 ݕ஌ YES ࣌ܥྻσʔλͷऩू ݪҼ਍அ 
 ҼՌάϥϑΛ 
 ߏங [AutoMAP, Microscope,…] ఏҊ ࣌ܥྻσʔλͷ 
 ࣍ݩ࡟ݮ ௶಺༎थ΄͔, TSifter: ϚΠΫϩαʔϏεʹ͓͚Δੑೳҟৗͷਝ଎ͳ਍அʹ޲͍ͨ࣌ܥྻσʔλͷ࣍ݩ࡟ݮख๏, Πϯλʔωοτͱӡ༻ٕज़γϯϙδ΢Ϝ࿦จू, 2020೥.
  11. 11 [ݚڀ੒Ռ3] ܥྻ࡟ݮੑೳͱߴ଎ੑ ɾCPUίΞ਺4ɺܥྻ਺100k εςοϓ1 
 ܥྻ୯Ґͷআڈ εςοϓ2 ྨࣅܥྻͷू໿ ߹ܭ

    
 ࣮ߦ࣌ؒ 54.41 (sec) 8.68 (sec) 63.09 (sec) ߴ଎ੑ ܥྻ࡟ݮੑೳ ҟৗ஫ೖ ίϯϙʔωϯτ 
 A ίϯϙʔωϯτ 
 B CPUաෛՙ 1545/201/122 
 92.1% 1541/156/89 
 94.2% ωοτϫʔΫ 
 ஗Ԇ 1596/248/128 91.9% 1543/262/128 91.7% ɾܥྻ࡟ݮੑೳ͕ɺߴ͍ਫ਼౓ͷݪҼ ਍அͷͨΊʹे෼͔Ͳ͏͔͸ɺࠓ ޙݕূ༧ఆ ɾݱ৔ͷো֐ରԠʹద༻͢Δ্Ͱɺ े෼ͳੑೳΛಘΒΕͨ ௶಺༎थ΄͔, TSifter: ϚΠΫϩαʔϏεʹ͓͚Δੑೳҟৗͷਝ଎ͳ਍அʹ޲͍ͨ࣌ܥྻσʔλͷ࣍ݩ࡟ݮख๏, Πϯλʔωοτͱӡ༻ٕज़γϯϙδ΢Ϝ࿦จू, 2020೥.
  12. 12 ɾΫϥ΢υ্ͷ෼ࢄΞϓϦέʔγϣϯͷ৴པੑΛ޲্ͤ͞ΔͨΊͷΞʔΩς ΫνϟΛݚڀ͍ͯ͠Δɻ ɾಛʹɺγεςϜͷӡ༻σʔλΛߴޮ཰ʹऩू͢Δख๏ʹߩݙͨ͠ɻ ɾ࣌ܥྻσʔλ ɾωοτϫʔΫ௨৴ͷґଘσʔλ ɾݱࡏ͸ɺऩू͞Εͨσʔλ͔ΒɺγεςϜʹൃੜ͢ΔҟৗͷݪҼΛಛఆ͢ Δख๏Λݚڀ͓ͯ͠Γɺ੒Ռ͕Ͱͭͭ͋Δɻ ɾ͜ΕΒͷߩݙ͸ɺ৴པੑͷ޲্ʹد༩͢Δ΋ͷͰ͋Δɻ ·ͱΊ

  13. 13 ࣮຿্Ͱ͸ɺӡ༻σʔλͷղੳͷࡍʹɺݱ࣮తʹऩूՄೳͳӡ༻σʔλͷ छྨ΍ཻ౓Λ૝ఆ͢Δɻ ֶज़తߩݙ σʔλͷऩूͱղੳͷ2ஈ֊Λ၆ᛌ্ͨ͠Ͱɺ࣮؀ڥ΁ͷద༻ੑͷߴ͍ ΞʔΩςΫνϟΛఏࣔ͢Δɻ ࣮຿্ͷ՝୊ ֤։ൃɾӡ༻૊৫ͷݸผͷࣄ৘ʹґଘ͢Δ σʔλͷऩूͱղੳ͸ݸผʹٞ࿦͞Ε͍ͯΔ طଘ࿦จͷ՝୊

  14. 14 ɾ Y. Tsubouchi, M. Furukawa, R. Matsumoto, Low Overhead

    TCP/UDP Socket-based Tracing for Discovering Network Services Dependencies, Journal of Information Processing (JIP), 2022. ͜Ε·Ͱͷݚڀ੒Ռ: ओͳࠪಡ෇͖࿦จ δϟʔφϧ࿦จ ࠃࡍձٞ ɾ Y. Tsubouchi, M. Furukawa, R. Matsumoto, Transtracer: Socket-Based Tracing of Network Dependencies among Processes in Distributed Applications, The 1st IEEE International COMPSAC Workshop on Advanced IoT Computing (AIOT 2020), July 2020. ࠃ಺γϯϙδ΢Ϝ ɾ ௶಺༎थ, ࿬ࡔேਓ, ᖛా݈, দ໦խ޾, খྛོߒ, Ѩ෦ത, দຊ྄հ, HeteroTSDB: ҟछ෼ࢄKVSؒͷࣗಈ֊૚Խ ʹΑΔߴੑೳͳ࣌ܥྻσʔλϕʔε, ৘ใॲཧֶձ࿦จࢽ, Vol.62, No.3, pp.818-828, 2021೥3݄. ʢ౤ߘதʣ ɾ Y. Tsubouchi, A. Wakisaka, K. Hamada, M. Matsuki, H. Abe, R. Matsumoto, HeteroTSDB: An Extensible Time Series Database for Automatically Tiering on Heterogeneous Key-Value Stores, The 43rd Annual IEEE International Computers, Software & Applications Conference (COMPSAC), pp. 264-269, July 2019. ɾ ௶಺༎थ, ௽ాതจ, ݹ઒խେ, TSifter: ϚΠΫϩαʔϏεʹ͓͚Δੑೳҟৗͷਝ଎ͳ਍அʹ޲͍ͨ࣌ܥྻσʔλ ͷ࣍ݩ࡟ݮख๏, Πϯλʔωοτͱӡ༻ٕज़γϯϙδ΢Ϝ࿦จू, 2020, 9-16 (2020-11-26), 2020೥12݄. ɾ ௶಺༎थ, ҏ໺จ඙, ஔాਅੜ, ࢁ઒૱, ദ໦ַ඙, ഡݪ݉Ұ, ॏෳഉআετϨʔδͷͨΊͷSHA-1ܭ ࢉγεςϜͷSSE໋ྩʹΑΔߴεϧʔϓοτԽ, ిࢠ৘ใ௨৴ֶձ࿦จࢽ D, 96(10), pp.2101-2109 2013೥10݄. ʢຊൃදʹؚ·ͣʣ