超個体型データセンターを目指した分散協調クエリキャッシングの構想 / QuorumCache Architecture

https://www.iot.ipsj.or.jp/meeting/45-program/
第45回情報処理学会インターネットと運用技術研究会 (IOT)
(14) ◯坪内 佑樹，松本 亮介（さくらインターネット株式会社）：超個体型データセンターを目指した分散協調クエリキャッシュ構想

Transcript

1. ͘͞ΒΠϯλʔωοτ
   גࣜձࣾ
   (C) Copyright 1996-2019 SAKURA Internet Inc
   ͘͞ΒΠϯλʔωοτ
   ݚڀॴ
   ௒ݸମܕσʔληϯλʔΛ໨ࢦͨ͠
   ෼ࢄڠௐΫΤϦΩϟογϯάͷߏ૝
   2019/05/24 ݚڀһ ௶಺ ༎थ
   ௶಺༎थ, দຊ྄հ
   ୈ45ճΠϯλʔωοτͱӡ༻ٕज़ݚڀձ(IOT45)
   

   View Slide

2. 2
   1. എܠͱ໨త
   2. ෼ࢄσʔληϯλʔʹ͓͚Δσʔλͷ഑ஔͷ՝୊
   3. QuorumCacheΞʔΩςΫνϟ
   4. ·ͱΊ
   ໨࣍
   

   View Slide

3. 1.
   എܠͱ໨త
   

   View Slide

4. 4
   ɾWebαʔϏεͷԠ౴଎౓͕஗͍ͱར༻ऀ͔Βར༻͞Εͳ͘ͳΔͱ͍
   ͏ใࠂ[1]͕͋ΓɼWebαʔϏεͷԠ౴ੑೳͷ޲্͕ॏཁͰ͋Δ
   ɾWebαʔϏεͷԠ౴஗ԆΛ࣍ͷΑ͏ʹ෼ղͰ͖Δ
   ɾ(1) ར༻ऀͷ୺຤্Ͱͷॲཧ
   ɾ(2) ୺຤͔Βσʔληϯλʔ಺αʔό·ͰͷωοτϫʔΫసૹ
   ɾ(3) σʔληϯλʔ಺Ͱͷॲཧ
   ɾ(2)͕ࢧ഑తͰ͋Δͱ͍͏ݚڀ[2]͕͋Δ(100msͷ͏ͪ80msΛ઎ΊΔ)
   Ԡ౴ੑೳʹ઎ΊΔωοτϫʔΫసૹ஗Ԇͷ໰୊
   [1] Forrester Consulting: eCommerce Web Site Performance Today: An Updated Look At Consumer Reaction To A Poor Online Shopping Experience 2009.
   [2] S Choy, B Wong, G Simon and C Rosenberg: The Brewing Storm in Cloud Gaming: A Measurement Study on Cloud to End-User Latency, 11th Annual
   Workshop on Network and Systems Support for Games, p. 2 2012.
   

   View Slide

5. 5
   ௒ݸମܕσʔληϯλʔ
   ɾதԝूݖͷΫϥ΢υίϯϐϡʔςΟϯάͷΈͰ͸ղܾͰ͖ͳ͍໰୊
   ɾར༻ऀͱσʔληϯλʔؒͷωοτϫʔΫ஗Ԇ
   ɾதԝͷσʔληϯλʔͷ্ΓͱԼΓͷωοτϫʔΫτϥώοΫྔ
   ɾখɾதن໛ͷσʔληϯλʔ΍খܕͷϥοΫ܈Λ஍ཧతʹ෼ࢄͤ͞Δ
   ɾ෼ࢄ֤ͨ͠σʔληϯλʔ͸ಠཱͯ͠ػೳ͠ͳ͕Β΋ɼ૯ମͱͯ͠͸
   ౷཰͞Ε͍ͯΔΑ͏ʹݟ͑Δ௒ݸମతͳσʔληϯλʔ[12]Λ໨ࢦ͢
   ɾ֤σʔληϯλʔͷ৔ॴ΍ن໛Λҙࣝͤͣʹɼಁաత͔ͭߴ଎ʹσʔ
   λΛಡΈॻ͖Ͱ͖Δඞཁ͕͋Δ
   [12] দຊ྄հ, ௶಺༎थ, ٶԼ߶ี: ෼ࢄܕσʔληϯλʔOSΛ໨ࢦͨ͠ϦΞΫςΟϒੑΛ࣋ͭίϯςφ࣮ߦج൫ٕज़ɼ৘ใॲཧֶձݚڀใࠂΠϯλʔωο
   τͱӡ༻ٕज़(IOT) ɼVol. 2019-IOT-44, No. 27, pp. 1–8 2019.
   

   View Slide

6. 6
   ɾཁ݅1: ෼ࢄͨ͠σʔληϯλʔ΁ಁաతʹΞΫηεՄೳ
   ɾ1.1: ΞΫηεͨ͠σʔληϯλʔʹΑΒͣσʔλΛߴ଎ࢀরՄೳ
   ɾશͯͷ෼ࢄσʔληϯλʔʹΩϟογϡͷෳ੡Λ഑ஔ
   ɾ1.2: σʔληϯλʔؒͰҰ؏ͨ͠σʔλΛࢀরՄೳ
   ɾॻ͖ࠐΈʹରͯ͠ߋ৽಺༰Λ֤Ωϟογϡͷෳ੡ʹ఻ൖ
   ɾཁ݅2: طଘͷΞϓϦέʔγϣϯΛมߋͤͣʹར༻Մೳ
   ɾڧ͍Ұ؏ੑͷͨΊʹॻ͖ࠐΈʹରͯ͠ಉظతʹΩϟογϡΛߋ৽
   ɾΞϓϦέʔγϣϯͱσʔλϕʔεͷؒͰΫΤϦͷ݁ՌΛΩϟογϡ
   ௒ݸମܕσʔληϯλʔDBͷཁ݅
   είʔϓ: ಈతίϯςϯπͷ഑৴ͷߴ଎ԽͷͨΊʹಡΈࠐΈੑೳΛ޲্
   

   View Slide

7. 7
   ෼ࢄσʔληϯλʔ্Ͱͷෳ੡ͷ՝୊
   ɾಉظߋ৽࣌ʹωοτϫʔΫ஗Ԇͷେ͖͍σʔληϯλʔʹ཯଎͞Εͯ
   Ԡ౴ੑೳ͕௿Լ͢Δ͜ͱ
   Replica
   Origin
   (1) ॻ͖ࠐΈཁٻ
   Replica
   Replica
   (2) ಉظߋ৽
   ཯଎
   ຊൃදͰ͸ɼ͜ͷ՝୊Λղܾ͢Δ
   ΞʔΩςΫνϟΛఏҊ͢Δ
   Edge
   Datacenter
   Central
   Datacenter
   

   View Slide

8. 8
   ݚڀͷ໨త
   1. ಉظߋ৽ͷൣғΛωοτϫʔΫ஗ԆʹԠͯ͡ௐઅՄೳ
   2. ΞϓϦέʔγϣϯͷಡΈॻ͖ճ਺ͷൺ཰ʹԠͯ͡ಉظߋ৽ͷൣғΛద
   ԠతʹܾఆՄೳ
   ωοτϫʔΫ஗ԆͱΞΫηε܏޲ͷมಈʹԠͯ͡
   Ԡ౴ੑೳΛదԠతʹ࠷େԽ͢Δج൫ͷఏҊ
   ಉظߋ৽࣌ʹωοτϫʔΫ஗Ԇͷେ͖͍σʔληϯλʔʹ཯଎͞ΕΔ
   ՝୊Λղܾ
   

   View Slide

9. 9
   ɾΞϓϦέʔγϣϯ୯ҐͰదԠతʹੑೳௐ੔͕ՄೳͳͨΊɼେྔͷΞ
   ϓϦέʔγϣϯΛ๊͑ΔΫϥ΢υɾϗεςΟϯάࣄۀʹద༻Մೳ
   ɾෳ਺ϦʔδϣϯʹWebαʔϏεΛల։͢Δ৔߹ʹɼϦʔδϣϯ಺Ͱ
   ࢀরॲཧΛ׬݁Մೳ
   ɾωοτϫʔΫ஗ԆͷมԽͷݕ஌͢ΔͨΊɼಛఆͷσʔληϯλʔͷ
   ނো࣌ʹɼࣗಈͰαʔϏεΞ΢τՄೳ
   ɾ֤෼ࢄσʔληϯλʔ্Ͱࢀরॲཧ͢ΔͨΊɼෛՙ෼ࢄ͕Մೳ
   ఏҊ͢Δج൫͕΋ͨΒ͢ޮ༻
   

   View Slide

10. 2.
    ෼ࢄσʔληϯλʔʹ͓͚Δ
    σʔλͷ഑ஔͷ՝୊
    

    View Slide

11. 11
    ෼ࢄσʔληϯλʔ΁ͷσʔλ഑ஔͷ՝୊
    Replica
    Origin
    (1) ॻ͖ࠐΈཁٻ
    Replica
    Replica
    (2) ಉظߋ৽
    (a) ֤σʔληϯλʔͰಉҰσʔλΛ
    ෳ੡഑ஔ ← ಡΈࠐΈओମͳΒͪ͜Β
    (b) ಛఆͷσʔληϯλʔ্ʹͷΈ
    σʔλΛ഑ஔ
    Origin
    Shard
    ॻ͖ࠐΈ࣌ʹωοτϫʔΫ
    ஗Ԇͷେ͖͍σʔληϯ
    λʔʹ཯଎͞ΕΔ
    ಡΈࠐΈ࣌ʹωοτϫʔΫ
    ஗Ԇͷେ͖͍σʔληϯ
    λʔʹ཯଎͞ΕΔ
    (1) ಡΈࠐΈཁٻ
    Shard
    Shard
    (2) σʔλసૹ
    ཁٻ͢Δσʔλ͕ଘࡏ
    ཯଎
    ཯଎
    

    View Slide

12. 12
    ෼ࢄΫΤϦΩϟογϡͷઌߦख๏
    ɾFerdinand[12]͸෼ࢄϋογϡςʔϒϧ
    ʹΑΓෳ਺ͷϊʔυؒͰΫΤϦϦβϧτ
    ΩϟογϡΛڞ༗͢Δ
    ɾϊʔυؒͷϨΠςϯγ͕େ͖͘ͳΔͱε
    ϧʔϓοτ͕௿Լ͢Δ࣮ݧใࠂ͕͋Δ
    ɾಛఆͷϊʔυʹΩϟογϡΛ഑ஔ͢Δͨ
    Ίʹύλʔϯ(b)ͱ͍͑Δ
    [12] C Garrod, A Manjhi, A Ailamaki, B Maggs, T Mowry, O Christopher and T Anthony: Scalable Query Result Caching for Web Applications, VLDB Endowment, Vol.
    1, No. 1, pp. 550–561 2008.
    จݙ[12] Figure 5ΑΓҾ༻
    

    View Slide

13. 3.
    QuorumCacheΞʔΩςΫνϟ
    

    View Slide

14. 14
    มԽʹԠͯ͡ੑೳΛ࠷େԽ͢Δج൫
    1. ಉظߋ৽ͷൣғΛωοτϫʔΫ஗ԆʹԠͯ͡ௐઅՄೳ
    ɾதԝσʔληϯλʔͱ෼ࢄσʔληϯλʔؒͷ஗Ԇ࣌ؒΛڑ཭ʹݟ
    ཱͯɼಉظߋ৽ͷൣғΛʮۙ๣ωοτϫʔΫڑ཭ʯม਺Ͱදݱ
    ɾม਺ͷ஋Λௐઅ͢Δ͜ͱʹΑΓɼԠ౴ੑೳΛ࠷େԽ͢Δ
    2. ΞϓϦέʔγϣϯͷಡΈॻ͖ൺ཰ʹԠͯ͡ಉظߋ৽ͷൣғΛదԠతʹ
    ܾఆՄೳ
    ɾۙ๣ωοτϫʔΫڑ཭ͷ࠷ద஋͸ɼಡΈࠐΈճ਺͕ॻ͖ࠐΈճ਺ʹ
    ൺ΂ͯͲͷఔ౓ࢧ഑త͔Ͱܾ·Δ
    

    View Slide

15. 15
    ɾ෼ࢄσʔληϯλʔؒͷσʔλ഑ஔʹ͓͚ΔಡΈॻ͖ͷτϨʔυΦ
    ϑΛղܾ͢Δʹ͸ɼ(a)ͱ(b)ͷؒΛͱΕ͹Α͍
    ɾதԝσʔληϯλʔ্ͷσʔλϕʔεʹॻ͖ࠐΜͩޙʹɼۙ๣ωο
    τϫʔΫڑ཭಺ͷσʔληϯλʔʹಉظతʹΩϟογϡߋ৽
    ɾಡΈग़͠ཁٻΛड͚ͨσʔληϯλʔ͸ɼۙ๣ωοτϫʔΫڑ཭
    ಺ͷσʔληϯλʔ܈͔ΒΩϟογϡΛಡΈग़͠
    ɾۙ๣ωοτϫʔΫڑ཭Λௐઅ͢Δ͜ͱʹΑΓɼಡΈग़࣌ؒ͠Λ૿Ճ
    ͤͯ͞ॻ͖ࠐΈ࣌ͷಉظ଴ͪ࣌ؒΛ௿ݮͰ͖Δ
    ಉظߋ৽ͷൣғͷม਺Խ
    

    View Slide

16. 16
    QuorumCacheͷಈ࡞ϑϩʔ
    Central
    Datacenter
    Origin
    Replica
    Replica
    Replica
    Replica
    d: ۙ๣ωοτϫʔΫڑ཭
    Edge
    Datacenter
    Replica
    Replica
    d
    (1) ॻ͖ࠐΈཁٻ
    (2) ॻ͖ࠐΈཁٻ
    సૹ
    (3) ಉظߋ৽
    (i) ಡΈࠐΈཁٻ
    (ii) dҎ಺ͷσʔλ
    ηϯλʔ͔Β
    ಡΈࠐΈ
    Replica
    (4) ඇಉظߋ৽
    

    View Slide

17. 17
    ɾΞϓϦέʔγϣϯ͝ͱʹಡΈॻ͖ൺ཰͕ҟͳΔͨΊɼ࠷దͳωοτ
    ϫʔΫڑ཭d͸ΞϓϦέʔγϣϯ͝ͱʹҟͳΔ
    ɾॻ͖ࠐΈ͕΄ͱΜͲͳ͍ΞϓϦέʔγϣϯͰ͋Ε͹ɼಉظ଴ͪ࣌
    ͕ؒ௕ͯ͘΋Α͍ͨΊɼωοτϫʔΫڑ཭d͸େ͖ͯ͘Α͍
    ɾಉҰΞϓϦέʔγϣϯͰ͋ͬͯ΋ɼػೳ௥Ճ΍࣌ؒଳʹΑΓಡΈ
    ॻ͖ൺ཰͸ҟͳΔ
    ɾΞϓϦέʔγϣϯ୯ҐͰԠ౴ੑೳΛ؂ࢹ͠ͳ͕ΒɼԠ౴ੑೳ͕࠷େ
    ͱͳΔΑ͏ʹۙ๣ωοτϫʔΫڑ཭dΛ૿ݮͤ͞Δ
    ۙ๣ωοτϫʔΫڑ཭ͷదԠతܾఆ
    

    View Slide

18. 18
    ɾωοτϫʔΫڑ཭
    ɾ࣮ࡍʹσʔληϯλʔؒͷ௨৴Ͱར༻͞ΕΔL7ϓϩτίϧʹΑΓ
    ܭଌ
    ɾதԝσʔληϯλʔ͕ఆظతʹܭଌ༻ͷཁٻΛ֤σʔληϯλʔ
    ʹ޲͚ͯૹ৴͠ɼͦΕͧΕͷԠ౴࣌ؒΛอଘ͢Δ
    ɾಡΈॻ͖ൺ཰
    ɾHTTPཁٻͷGET/POSTͷൺ཰ʹΑΓܭଌ
    ɾσʔλϕʔε΁ͷࢀরܥ/ߋ৽ܥΫΤϦͷൺ཰ʹΑΓܭଌ
    ɾԠ౴ੑೳ
    ɾར༻ऀ͔ΒͷཁٻΛड͚෇͚ΔHTTPϓϩΩγ͕ܭଌ
    ֤छύϥϝʔλͷܭଌํ๏
    

    View Slide

19. 19
    ɾ೚ҙͷσʔληϯλʔ͕ۙ๣ωοτϫʔΫڑ཭dҎ಺ʹ͋Δ͜ͱΛ
    ͲͷΑ͏ʹ൑அ͢Δͷ͔
    ɾதԝσʔληϯλʔ͕ۙ๣ωοτϫʔΫڑ཭dͱ֤σʔληϯλʔ
    ͷωοτϫʔΫڑ཭Λશͯͷσʔληϯλʔʹఆظతʹૹ৴͢Δ
    ɾ֤σʔληϯλʔ͸ड৴ͨࣗ͠਎ͷωοτϫʔΫڑ཭ͱdΛൺֱ͠
    ࣗ਎ͷঢ়ଶΛ஌Δ
    ɾۙ๣ωοτϫʔΫڑ཭dͷมԽྔΛͲͷΑ͏ʹܾఆ͢Δ͔
    ɾಡΈॻ͖ൺ཰ͱԠ౴ੑೳͱ͋ΘͤͯͲͷΑ͏ͳؔ਺ͱͯ͠ϞσϧԽ
    ͢Δ͔Λݕ౼த
    ֤ύϥϝʔλ͔ΒΞΫγϣϯΛ൑அ͢Δํ๏
    

    View Slide

20. 20
    γεςϜߏ੒
    HTTP
    Proxy
    Central Datacenter
    Web
    App
    DB
    Proxy
    Database
    HTTP Request
    Caching
    ಡΈॻ͖ൺ཰ͷ
    ܭଌ(HTTP)
    ಡΈॻ͖ൺ཰ͷ
    ܭଌ(DBΫΤϦ)
    Ԡ౴ੑೳͷܭଌ
    Query
    Pinger
    Ping
    ωοτϫʔΫڑ཭
    ͷܭଌ
    DB
    Proxy
    Web
    App
    HTTP
    Proxy
    ɾ੍ޚϙΠϯτͱͯ͠HTTPͱDBͷ૚ʹϓϩΩγΛ഑ஔ
    Forward
    Edge Datacenter
    

    View Slide

21. 5.
    ·ͱΊ
    

    View Slide

22. 22
    ɾ௒ݸମܕσʔληϯλʔDBͷͨΊͷΫΤϦϦβϧτΩϟογϯάج
    ൫Λߏ૝ͨ͠
    ɾΩϟογϡͷಉظର৅ͷൣғΛωοτϫʔΫ஗ԆʹΑΓม਺Խ
    ɾωοτϫʔΫ஗ԆͱΞΫηε܏޲ͷมಈʹԠͯ͡ɼԠ౴ੑೳΛద
    Ԡతʹ࠷େԽ
    ɾࠓޙ͸ɼαʔϕΠͱఏҊख๏ͷ࣮૷ͱධՁΛਐΊ͍ͯ͘
    ɾL3ϧʔςΟϯάɼP2Pɼ෼ࢄDBͳͲͷؔ࿈ݚڀΛαʔϕΠ
    ɾ֤छܭଌͱܭଌ݁ՌΛ΋ͱʹͲͷΑ͏ʹ൑அ͢Δ͔ͷख๏Λݕ౼
    ·ͱΊͱࠓޙͷ՝୊
    

    View Slide

23. 23
    ɾωοτϫʔΫ஗ԆͷଌఆʹɼL3Ͱݚڀ͞Ε͍ͯΔ಺༰ΛԠ༻Ͱ͖ͳ
    ͍͔
    ɾར༻ऀۙ๣σʔληϯλʔ͔Βཁٻ͞ΕͨΩϟογϡ·Ͱͷ࠷దͳ
    ܦ࿏ΛٻΊΔख๏ͱͯ͠ɼL3ͷܦ࿏બ୒ɾܦ࿏੍ޚͷख๏͕Ԡ༻Ͱ
    ͖ͳ͍͔
    ɾ؀ڥͷมԽʹ͋ΘͤͯదԠతʹ࠷ద஋ΛٻΊΔ৔߹ͷయܕతͳམͱ
    ͕݀͠ͳ͍͔
    σΟεΧογϣϯ಺༰
    

    View Slide