分散アプリケーションの高信頼化のための 運用技術に関する研究 / A Study on Operation Technology for High Reliability of Distributed Applications

京都大学大学院 情報学研究科 知能情報学専攻 
中間発表（15min）。

Transcript

1. ෼ࢄΞϓϦέʔγϣϯͷߴ৴པԽͷͨΊͷ
   

   ӡ༻ٕज़ʹؔ͢Δݚڀ
   ɾΫϥ΢υΛ࢖༻ͨ͠ΞϓϦέʔγϣϯͷ৴པੑΛ޲্ͤ͞Δ
   ΞʔΩςΫνϟΛݚڀ͍ͯ͠Δɻ
   
   
   ɾಛʹɺγεςϜͷӡ༻ঢ়گΛࣔ͢σʔλΛޮ཰తʹऩू͢Δख
   ๏ʹߩݙͨ͠ɻ
   
   
   ɾݱࡏ͸ɺऩू͞Εͨσʔλ͔ΒɺγεςϜʹൃੜ͢Δҟৗͷݪ
   ҼΛਝ଎ʹಛఆ͢ΔͨΊͷσʔλ෼ੳख๏Λݚڀ͍ͯ͠Δɻ
   ௶಺ ༎थ
   

   ژ౎େֶେֶӃ ৘ใֶݚڀՊ ஌ೳ৘ใֶઐ߈ ത࢜ޙظ՝ఔ 2೥
   

   தؒൃද 2021೥8݄5೔
   

   View Slide

2. 2
   1. Ϋϥ΢υ্ͷ෼ࢄΞϓϦέʔγϣϯ
   
   
   Ϋϥ΢υ্ͷ෼ࢄΞϓϦέʔγϣϯͷෳࡶԽ
   
   
   ৴པੑͷ՝୊: ӡ༻σʔλྔͷ૿Ճ
   

   2. ݚڀख๏
   
   
   ӡ༻σʔλͷऩूͷߴޮ཰Խ
   
   
   • [ݚڀ੒Ռ1] ଟ਺ͷ࣌ܥྻσʔλΛૠೖɾอଘ͢ΔͨΊͷσʔλϕʔε
   
   
   • [ݚڀ੒Ռ2] ωοτϫʔΫ௨৴ͷґଘؔ܎ͷ௿Φʔόϔουऩू
   
   
   ӡ༻σʔλͷղੳ
   
   
   • [ݚڀ੒Ռ3] ࣌ܥྻσʔλͷத͔ΒݪҼ਍அʹ༗༻ͳܥྻΛநग़
   
   
   3. ·ͱΊ/ݚڀ࣮੷
   ൃද಺༰ͷߏ੒
   

   View Slide

3. 3
   Ϋϥ΢υ্ͷ෼ࢄΞϓϦέʔγϣϯͷෳࡶԽ
   ɾ
   
   
   ɾ
   
   
   ɾ
   ΞΫηε૿Ճ
   εέʔϧΞ΢τʹΑΔ
   
   
   ϗετ਺ͷ૿Ճ
   ػೳͷ૿Ճ
   ΑΓେن໛ͳ
   

   ෼ࢄΞʔΩςΫνϟ
   ϛυϧ΢ΣΞͷ૿Ճ
   RDBαʔό
   Ωϟογϡ
   

   αʔό
   ݕࡧαʔό
   Webαʔό
   ωοτϫʔΫ
   

   αʔϏε
   TCP/UDP
   ɾ
   
   
   ɾ
   
   
   ɾ
   ϝσΟΞɺECαΠτɺSNSɺIoTͳͲΛߏ੒͢Δ෼ࢄΞϓϦέʔγϣϯ
   

   View Slide

4. 4
   ɾίϯϙʔωϯτ਺ͷ૿େʹΑΓɺӡ༻σʔλྔ͕૿Ճ
   
   
   ɾσʔλऩूͷͨΊͷܭࢉػͷෛՙ͕૿Ճ
   
   
   ɾγεςϜ؅ཧऀͷೝ஌ෛՙ͕૿Ճ
   ෳࡶԽͨ݁͠Ռͷ৴པੑͷ௿Լ
   ӡ༻σʔλͷ
   

   ऩू
   ɾෛՙঢ়گΛࣔ࣌͢ܥྻσʔλ
   
   
   ɾωοτϫʔΫ௨৴ͷґଘؔ܎
   
   
   ɾςΩετϩά
   ৴པੑͷ௿Լ΁
   ো֐෮چ࣌ؒͷ૿େ
   

   View Slide

5. 5
   ɾӡ༻σʔλͷऩूΛߴޮ཰Խ͢Δ ܭࢉػͷෛՙΛ௿ݮ
   
   
   [ݚڀ੒Ռ1] ࣌ܥྻσʔλΛߴޮ཰ʹૠೖɾอଘ͢Δσʔλϕʔε
   
   
   [ݚڀ੒Ռ2] ωοτϫʔΫ௨৴ͷґଘؔ܎ͷ௿Φʔόϔουऩू
   ໨త: ӡ༻σʔλͷऩूͱղੳͷߴ౓ԽʹΑΓ৴པੑΛ޲্
   ͍ͣΕ΋࣮؀ڥ΁ͷద༻༰қੑΛߟྀ͢Δ
   ɾӡ༻σʔλΛղੳ͠ɺো֐ରԠʹ༗༻ͳಎ࡯ΛಘΔ ೝ஌ෛՙΛ௿ݮ
   
   
   [ݚڀ੒Ռ3] ࣌ܥྻσʔλͷத͔ΒݪҼ਍அʹ༗༻ͳܥྻΛநग़
   ʢ↪࿦จࢽ౤ߘதʣ
   ʢ↪࿦จࢽ࠾࿥ࡁʣ
   ʢ↪ࠃࡍձٞ౤ߘ४උதʣ
   

   View Slide

6. 6
   ɾҰൠʹɺσΟεΫϕʔεDB͸ɺࡧҾߏ଄ʹฏߧ໦Λ࢖༻͢Δ
   
   
   ɾ՝୊ɿܥྻ਺(n)ͷ૿Ճʹରͯ͠ɺܭࢉྔ͕O(log n)Ͱ૿Ճ͢Δ
   
   
   ɾ؅ཧͷෛ୲Λܰݮ͢ΔͨΊʹɺطଘͷDB্ʹߏ੒͍ͨ͠
   [ݚڀ੒Ռ1] ࣌ܥྻσʔλͷऩूͷߴޮ཰Խ
   ௶಺༎थ΄͔, HeteroTSDB: ҟछ෼ࢄKVSؒͷࣗಈ֊૚ԽʹΑΔߴੑೳͳ࣌ܥྻσʔλϕʔε, ৘ใॲཧֶձ࿦จࢽ, Vol.62, No.3, pp.818-828, 2021೥3݄.
   ϝϞϦ
   

   ϕʔεDB
   σΟεΫ
   

   ϕʔεDB
   ϋογϡද O(k) ฏߧ໦ O(log n)
   σʔλ఺ૠೖ
   dݸͷσʔλ఺Λ஝ੵޙʹ
   

   ϑϥογϡॻ͖ࠐΈ
   ղܾɿϝϞϦϕʔεDBͱσΟεΫϕʔεDBͷ֊૚ԽΛఏҊ
   M (insertions/s) M / d (insertions/s)
   

   View Slide

7. 7
   [ݚڀ੒Ռ1] ܥྻ਺ͷ૿Ճʹର͢ΔૠೖεϧʔϓοτมԽ
   ɾσΟεΫϕʔεDBͷΈͱൺֱ͠ɺ
   ࠷େͰ3.96ഒͷεϧʔϓοτ
   
   
   ɾεϧʔϓοτͷ௿Լ཰΋վળͨ͠
   0
   20
   40
   60
   80
   100
   100 1K 10K 100k 1M
   0
   20
   40
   60
   Insertion throughput (kilo datapoints / sec)
   Throughput decrease rate (%)
   The number of series
   HeteroTSDB (Proposed)
   KairosDB
   HeteroTSDB (Proposed)
   KairosDB
   ϝϞϦDB → σΟεΫDBͷҠಈεϧʔ
   ϓοτͱϝϞϦDB΁ͷεϧʔϓοτͱ
   ಉఔ౓
   0
   20
   40
   60
   80
   100
   0 300 600 900 1200 1500 1800
   0
   500
   1000
   1500
   2000
   Insertion throughput (kilo datapoints / sec)
   Memory used size (MB)
   Elapsed time (sec)
   Flushed datapoints (/sec)
   Memory used size (MB)
   ௶಺༎थ΄͔, HeteroTSDB: ҟछ෼ࢄKVSؒͷࣗಈ֊૚ԽʹΑΔߴੑೳͳ࣌ܥྻσʔλϕʔε, ৘ใॲཧֶձ࿦จࢽ, Vol.62, No.3, pp.818-828, 2021೥3݄.
   

   View Slide

8. (౤ߘத) Y. Tsubouchi, et al., Low Overhead TCP/UDP Socket-based Tracing for Discovering Network Services Dependencies, Journal of Information Processing 2022.
   [ݚڀ੒Ռ2] ωοτϫʔΫ௨৴ͷґଘؔ܎औಘͷޮ཰Խ
   .
   
   
   .
   
   
   .
   Kernel
   User
   Service
   Socket
   Tracing
   

   Process
   … Event
   Event
   Event
   ετϦʔϛϯά๏(Weave Scope)
   ϑϩʔू໿๏ ([Datadog], [SAC 20])
   ϑϩʔूଋ๏ʢఏҊʣ
   .
   
   
   .
   
   
   .
   Kernel
   Service
   Socket
   Tracing
   

   Process
   .
   
   
   .
   
   
   .
   Event
   Flow
   Event
   Event
   Event
   …
   …
   .
   
   
   .
   
   
   . .
   
   
   .
   
   
   .
   User
   Service
   Socket
   Tracing
   

   Process
   .
   
   
   .
   
   
   .
   ✗ ΧʔωϧˠϢʔβۭؒؒ
   ͷΠϕϯτͷίϐʔίετ
   ✗ TCP઀ଓϨʔτ͕૿Ճ͢Δ
   ͱɺίϐʔίετ͕૿Ճ ෳ਺ͷϑϩʔΛूଋ
   ϑϩʔ= ྆୺ͷΞυϨεͱϙʔτͷ
   ૊ʢλϓϧʣ͕ಉҰͷ௨৴୯Ґ
   Event
   Event …
   …
   Event
   Event
   .
   
   
   .
   
   
   .
   Event
   Event …
   Event
   Event
   .
   
   
   .
   
   
   .
   Ұൠతʹ௨৴͸OSΧʔωϧͷTCP/UDPΛ࢖༻͢Δ͜ͱ
   ʹண໨
   8
   

   View Slide

9. 9
   ɾఏҊख๏͸ɺCPUར༻཰͸2.2%ҎԼɻ
   
   
   ɾϑϩʔ਺ͷ૿େʹରͯ͠ɺϑϩʔूଋʹΑΓɺ
   

   CPUར༻཰Λ௿͘ҡ͍࣋ͯ͠Δ
   [ݚڀ੒Ռ2] ϑϩʔ਺ͷ૿େʹର͢ΔCPU࢖༻ྔͷมԽ
   ఏҊख๏
   (౤ߘத) Y. Tsubouchi, et al., Low Overhead TCP/UDP Socket-based Tracing for Discovering Network Services Dependencies, Journal of Information Processing 2022.
   0
   5
   10
   15
   20
   25
   5 10 15 20 25 30 35
   CPU usage / core (%)
   TCP round trips / sec (x103)
   Streaming(client)
   Streaming(server)
   In-Kernel-Aggr(client)
   In-Kernel-Aggr(server)
   In-Kernel-Bundling(client)
   In-Kernel-Bundling(server)
   

   View Slide

10. εςοϓ1: ظؒதͷมಈͷ౓߹͍͕খ͍͞ܥྻΛআڈ
    
    
    εςοϓ2: άϥϑͷܗঢ়͕ྨࣅ͢Δ΋ͷ͔Β୅දܥྻΛநग़
    10
    ɾ՝୊: ίϯϙʔωϯτ͝ͱʹɺಛ௃ྔͱͯ͠ܥྻͷछผʢCPU࢖༻ɺϝϞϦ࢖
    ༻ɺωοτϫʔΫଳҬɺ…ʣΛࢦఆ͢Δඞཁ͋Γ
    
    
    ɾղܾ: ؔ࿈͢ΔશܥྻΛೖྗͱͯ͠༩͑ɺݪҼ਍அʹ༗༻ͳܥྻΛߴ଎ʹநग़
    [ݚڀ੒Ռ3] ࣌ܥྻσʔλ͔ΒݪҼ਍அʹ༗༻ͳܥྻΛநग़
    ҟৗ
    

    ݕ஌
    YES
    ࣌ܥྻσʔλͷऩू
    ݪҼ਍அ
    

    ҼՌάϥϑΛ
    

    ߏங
    [AutoMAP, Microscope,…]
    ఏҊ
    ࣌ܥྻσʔλͷ
    

    ࣍ݩ࡟ݮ
    ௶಺༎थ΄͔, TSifter: ϚΠΫϩαʔϏεʹ͓͚Δੑೳҟৗͷਝ଎ͳ਍அʹ޲͍ͨ࣌ܥྻσʔλͷ࣍ݩ࡟ݮख๏, Πϯλʔωοτͱӡ༻ٕज़γϯϙδ΢Ϝ࿦จू, 2020೥.
    

    View Slide

11. 11
    [ݚڀ੒Ռ3] ܥྻ࡟ݮੑೳͱߴ଎ੑ
    ɾCPUίΞ਺4ɺܥྻ਺100k
    εςοϓ1
    

    ܥྻ୯Ґͷআڈ
    εςοϓ2
    
    
    ྨࣅܥྻͷू໿
    ߹ܭ
    

    ࣮ߦ࣌ؒ
    54.41 (sec) 8.68 (sec) 63.09 (sec)
    ߴ଎ੑ
    ܥྻ࡟ݮੑೳ
    ҟৗ஫ೖ ίϯϙʔωϯτ
    

    A
    ίϯϙʔωϯτ
    

    B
    CPUաෛՙ 1545/201/122
    

    92.1%
    1541/156/89
    

    94.2%
    ωοτϫʔΫ
    

    ஗Ԇ
    1596/248/128
    91.9%
    1543/262/128
    
    
    91.7%
    ɾܥྻ࡟ݮੑೳ͕ɺߴ͍ਫ਼౓ͷݪҼ
    ਍அͷͨΊʹे෼͔Ͳ͏͔͸ɺࠓ
    ޙݕূ༧ఆ
    ɾݱ৔ͷো֐ରԠʹద༻͢Δ্Ͱɺ
    े෼ͳੑೳΛಘΒΕͨ
    ௶಺༎थ΄͔, TSifter: ϚΠΫϩαʔϏεʹ͓͚Δੑೳҟৗͷਝ଎ͳ਍அʹ޲͍ͨ࣌ܥྻσʔλͷ࣍ݩ࡟ݮख๏, Πϯλʔωοτͱӡ༻ٕज़γϯϙδ΢Ϝ࿦จू, 2020೥.
    

    View Slide

12. 12
    ɾΫϥ΢υ্ͷ෼ࢄΞϓϦέʔγϣϯͷ৴པੑΛ޲্ͤ͞ΔͨΊͷΞʔΩς
    ΫνϟΛݚڀ͍ͯ͠Δɻ
    
    
    ɾಛʹɺγεςϜͷӡ༻σʔλΛߴޮ཰ʹऩू͢Δख๏ʹߩݙͨ͠ɻ
    
    
    ɾ࣌ܥྻσʔλ
    
    
    ɾωοτϫʔΫ௨৴ͷґଘσʔλ
    
    
    ɾݱࡏ͸ɺऩू͞Εͨσʔλ͔ΒɺγεςϜʹൃੜ͢ΔҟৗͷݪҼΛಛఆ͢
    Δख๏Λݚڀ͓ͯ͠Γɺ੒Ռ͕Ͱͭͭ͋Δɻ
    
    
    ɾ͜ΕΒͷߩݙ͸ɺ৴པੑͷ޲্ʹد༩͢Δ΋ͷͰ͋Δɻ
    ·ͱΊ
    

    View Slide

13. 13
    ࣮຿্Ͱ͸ɺӡ༻σʔλͷղੳͷࡍʹɺݱ࣮తʹऩूՄೳͳӡ༻σʔλͷ
    छྨ΍ཻ౓Λ૝ఆ͢Δɻ
    ֶज़తߩݙ
    σʔλͷऩूͱղੳͷ2ஈ֊Λ၆ᛌ্ͨ͠Ͱɺ࣮؀ڥ΁ͷద༻ੑͷߴ͍
    ΞʔΩςΫνϟΛఏࣔ͢Δɻ
    ࣮຿্ͷ՝୊
    ֤։ൃɾӡ༻૊৫ͷݸผͷࣄ৘ʹґଘ͢Δ
    σʔλͷऩूͱղੳ͸ݸผʹٞ࿦͞Ε͍ͯΔ
    طଘ࿦จͷ՝୊
    

    View Slide

14. 14
    ɾ Y. Tsubouchi, M. Furukawa, R. Matsumoto, Low Overhead TCP/UDP Socket-based Tracing for
    Discovering Network Services Dependencies, Journal of Information Processing (JIP), 2022.
    ͜Ε·Ͱͷݚڀ੒Ռ: ओͳࠪಡ෇͖࿦จ
    δϟʔφϧ࿦จ
    ࠃࡍձٞ
    ɾ Y. Tsubouchi, M. Furukawa, R. Matsumoto, Transtracer: Socket-Based Tracing of Network Dependencies
    among Processes in Distributed Applications, The 1st IEEE International COMPSAC Workshop on
    Advanced IoT Computing (AIOT 2020), July 2020.
    ࠃ಺γϯϙδ΢Ϝ
    ɾ ௶಺༎थ, ࿬ࡔேਓ, ᖛా݈, দ໦խ޾, খྛོߒ, Ѩ෦ത, দຊ྄հ, HeteroTSDB: ҟछ෼ࢄKVSؒͷࣗಈ֊૚Խ
    ʹΑΔߴੑೳͳ࣌ܥྻσʔλϕʔε, ৘ใॲཧֶձ࿦จࢽ, Vol.62, No.3, pp.818-828, 2021೥3݄.
    ʢ౤ߘதʣ
    ɾ Y. Tsubouchi, A. Wakisaka, K. Hamada, M. Matsuki, H. Abe, R. Matsumoto, HeteroTSDB: An Extensible Time
    Series Database for Automatically Tiering on Heterogeneous Key-Value Stores, The 43rd Annual IEEE
    International Computers, Software & Applications Conference (COMPSAC), pp. 264-269, July 2019.
    ɾ ௶಺༎थ, ௽ాതจ, ݹ઒խେ, TSifter: ϚΠΫϩαʔϏεʹ͓͚Δੑೳҟৗͷਝ଎ͳ਍அʹ޲͍ͨ࣌ܥྻσʔλ
    ͷ࣍ݩ࡟ݮख๏, Πϯλʔωοτͱӡ༻ٕज़γϯϙδ΢Ϝ࿦จू, 2020, 9-16 (2020-11-26), 2020೥12݄.
    ɾ ௶಺༎थ, ҏ໺จ඙, ஔాਅੜ, ࢁ઒૱, ദ໦ַ඙, ഡݪ݉Ұ, ॏෳഉআετϨʔδͷͨΊͷSHA-1ܭ
    ࢉγεςϜͷSSE໋ྩʹΑΔߴεϧʔϓοτԽ, ిࢠ৘ใ௨৴ֶձ࿦จࢽ D, 96(10), pp.2101-2109 2013೥10݄.
    ʢຊൃදʹؚ·ͣʣ
    

    View Slide