Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless??そんなことよりDBだ!! / We should know Databases before Serverless

Masashi Terui
November 03, 2018

Serverless??そんなことよりDBだ!! / We should know Databases before Serverless

JAWS Festa 2018の資料です
https://jft2018.jaws-ug.jp/session/729/

Masashi Terui

November 03, 2018
Tweet

More Decks by Masashi Terui

Other Decks in Technology

Transcript

  1. SERVERLESSʁʁ
    ͦΜͳ͜ͱΑΓDBͩʂʂ
    MASASHI TERUI @ JAWS FESTA OSAKA 2018

    View Slide

  2. SERVERWORKS CO.,LTD.
    + FREELANCER
    • Serverless Oji-san
    • Serverless Framework Plugin Developer
    • Serverlessconf Tokyo 2016,2017,2018 speaker
    • Remote worker (in Sapporo-shi, Hokkaido)
    • http://marcy.hatenablog.com/entry/2018/08/04/231241
    MASASHI TERUI
    ARCHITECT / DEVELOPER

    View Slide

  3. SERVERLESS΍ͬͯ·͔͢ʁ
    DynamoDB࢖ͬͯ·͔͢ʁ

    View Slide

  4. • ޷͖
    • ී௨
    • ݏ͍
    • LambdaͱRDSͷ૬ੑ͕ѱ͍͔Β࢓ํͳ͘෇͖߹͍ͬͯΔ
    DynamoDB͸޷͖Ͱ͔͢ʁ

    View Slide

  5. ࢓ํ͕ͳ͍͔ΒDynamoDB……
    ຊ౰͸RDBΛ࢖͍͍͚ͨͲ

    View Slide

  6. RDBʹ؁͑ͯΔΜ͡Όͳ͍Ͱ͔͢ʁʁ
    NoSQL͕࢖͑ͳ͍Μ͡Όͳͯ͘

    View Slide

  7. RDBͷ͜ͱͪΌΜͱ෼͔ͬͯ·͔͢ʁʁ
    ͦ΋ͦ΋

    View Slide

  8. • ίωΫγϣϯϞσϧ
    • Ӭଓత
    • ίωΫγϣϯϓʔϦϯά
    • ηΩϡϦςΟ
    • ૊ΈࠐΈͷऑ͍ೝূ
    • VPC಺΁ͷΞΫηείετʢENIੜ੒ॲཧʣ
    LambdaͱRDSͷ૬ੑ͕Կނѱ͍ͷ͔આ໌Ͱ͖·͔͢ʁ
    ͱͯ΋OLTPͰ͸࢖͑ͳ͍(਺ඵʙे਺ඵ)

    View Slide

  9. DynamoDBͷ͜ͱ෼͔ͬͯ·͔͢ʁʁ
    ͡Ό͋

    View Slide

  10. σʔλϕʔεͷ͜ͱ෼͔ͬͯ·͔͢ʁʁ
    ͱ͍͏͔

    View Slide

  11. • ςʔϒϧઃܭͱ͔ΠϯσοΫεઃܭͬͯͲ͏΍Δͷʁ
    • ACIDτϥϯβΫγϣϯແ͍ͷʹେৎ෉ͳͷʁ
    • ݁Ռ੔߹ੑͬͯେৎ෉ͳͷʁ
    • ্ख͘ੑೳΛҾ͖ग़͢ʹ͸Ͳ͏͢Ε͹ྑ͍ʁ
    DynamoDBͷΑ͋͘Δෆ҆

    View Slide

  12. • ςʔϒϧઃܭͱ͔ΠϯσοΫεઃܭͬͯͲ͏΍Δͷʁ
    • ACIDτϥϯβΫγϣϯແ͍ͷʹେৎ෉ͳͷʁ
    • ݁Ռ੔߹ੑͬͯେৎ෉ͳͷʁ
    • ্ख͘ੑೳΛҾ͖ग़͢ʹ͸Ͳ͏͢Ε͹ྑ͍ʁ
    DynamoDBͷΑ͋͘Δෆ҆
    ෼͔Βͳ͍͔Βෆ҆ʹͳΔ

    View Slide

  13. • AWS͕ఏڙ͢ΔNoSQLαʔϏε
    • KeyʹΑΔ׬શҰகͱIndex୳ࡧ͕Մೳ
    • ϑϧϚωʔδυ
    • ແ੍ݶͷࣗಈ֦ுετϨʔδʢ࢖ͬͨ෼͚ͩʣ
    • Πϯελϯεͷ؅ཧෆཁɺWrite/Readͷੑೳ͚ͩΛࢦఆ͢Δ
    • Ͱ΋ɺΩϟύγςΟͷࣄલ༧໿͕ඞཁͳαʔϏε͸Serverless͡Όͳ͍ͬͯCNCF͞Μ͕ݴͬɾɾɾ͋ɺ୭͔དྷͨΈ͍ͨ
    DynamoDBͷجຊ

    View Slide

  14. DynamoDB͸ͲΜͳ ”AWSαʔϏε” ͔
    Ͱ͸ͳ͘

    View Slide

  15. DynamoDB͸ͲΜͳ ”σʔλϕʔε” ͔
    ஌ͬͯ·͔͢ʁ

    View Slide

  16. • Consistent Hashing
    • Vector clocks with reconciliation during reads
    • Sloppy Quorum and hinted handoff
    • Anti-entropy using Merkle trees
    • Gossip-based membership protocol and failure detection
    Amazon’s Dynamo
    ৄ͘͠஌Γ͍ͨͳΒʢBetterͰ͋ͬͯMustͰ͸ͳ͍ʣ
    https://www.allthingsdistributed.com/2007/10/amazons_dynamo.html

    View Slide

  17. • Ωʔͷϋογϡ஋Ͱ෼ࢄ → Partition Key
    • ͦͷ෼ࢄͨ͠தͰIndex͕ߏங͞Ε͍ͯΔ → Sort Key
    • όοΫάϥϯυͰߋ৽ΛޙউͪͰಉظ
    • ݁Ռ੔߹ੑϕʔε
    • ॻ͖ࠐΈɾಡΈࠐΈͷQuorum਺Ͱ੔߹ੑ͕ܾ·Δ
    • ΩʔΛࢦఆͨ͠ಡΈࠐΈ͸ڧ੔߹ੑ͕બ୒Մೳ
    ཈͑Δ΂͖ϙΠϯτ

    View Slide

  18. Partition Keyͷϋογϡ஋Ͱ෼ࢄ
    ͋ɺ͜ΕRDB(Hash Index)Ͱʢ͈́

    View Slide

  19. • Partition Keyʹ࿈൪Λ࢖͏ͷ͸Կͷҙຯ΋ͳ͍͜ͱ͕෼͔Δ
    • ࿈൪Χ΢ϯλʔΛ࡞ΔͱPartition͕ภΔ
    • Max + 1ͳΜͯҎͯͷ֎
    • ࿦ཧతʹҰҙͱͳΔଐੑ͕͋ΔͳΒͦΕΛ࢖͏
    • ͳ͚Ε͹UUIDͷΑ͏ͳিಥՄೳੑ͕ۃখͳϥϯμϜ஋Ͱྑ͍
    • Sort΋Ͱ͖ͳ͍ͷͰSnowflakeͷΑ͏ͳॱংੑ΋ཁΒͳ͍
    • User ID͸CognitoͰೝূ͍ͯ͠ΔͳΒCognito͕ൃߦ͢ΔIDΛ࢖͏
    • ΞΫηε੍ޚʹ΋࢖͑Δ
    ͜ͷલఏΛ౿·͑Δͱ

    View Slide

  20. • 3000RCU, 1000WCU, Index Size 10GB (2018-10-31ݱࡏ)
    • ͲΕ͔ҰͭͰ΋௒ա͢ΔͱPartition͕෼ׂ͞ΕΔ
    • ෼ׂ͞ΕΔͱ֬อͨ͠Ϣχοτ͕ۉ౳ʹׂΓৼΒΕΔ
    • εϩοτϦϯά࣌ͷҰ࣌తͳόʔετ͕͋Δ
    • ݶఆతͳόʔετ
    • ༨৒෼Λճ͢ (Adaptive Capacity)
    Partition͝ͱͷΩϟύγςΟ

    View Slide

  21. Partition KeyͰ෼ࢄɺSort Key͸ʁ

    View Slide

  22. B+TREE INDEX
    B-treeͳͷ͔B+treeͳͷ͔͸ެද͞Ε͍ͯͳ͍͚Ͳ
    Range Keyͱ͔ݺͿ͘Β͍͔ͩΒ…Ͷʁ
    ͋ɺ͜ΕRDBͰʢ͈́

    View Slide

  23. TARGETING QUERY B+treeͷߏ଄Λ஌͍ͬͯΕ͹
    ૊Έ߹ΘͤΩʔͷΠϯσοΫε͕ແͯ͘΋
    ͜͏΍Ε͹ྑ͍͜ͱ͕෼͔Δ

    View Slide

  24. Partition KeyͰ෼ࢄͨ͠தͰSort KeyͰ੔ྻ
    ͜ͷΠϝʔδΛ࣋ͭ͜ͱ͕ͱͯ΋ॏཁ

    View Slide

  25. • Partition KeyͰ෼ࢄͨ͠ΠϯελϯεʹIndex͕ߏங͞ΕΔ
    • ҰͭͷΠϯελϯε಺Ͱ͸ಉظతʹIndex΋ߋ৽͞ΕΔ
    • ڧ੔߹ੑ͕બ୒Մೳ
    • Primary Sort KeyͷϦʔϑϊʔυʹ࣮σʔλͷΞυϨε͕

    ֨ೲ͞Ε͍ͯΔΠϝʔδʢͱࢥΘΕΔʣ
    • LSI͸࣮σʔλΛ࣋ͨͳ͍ͷͰॏෳՄೳ
    Primary Sort Key, LSI

    View Slide

  26. GSI͸࣮ςʔϒϧͷࣹӨ

    View Slide

  27. • ू߹࿦ʹ͓͚ΔࣹӨ
    • GSIͷPartition KeyΛݩʹ෼ࢄ

    ʢߋ৽͸ඇಉظͰద༻ʣ
    • ϦʔϑϊʔυʹଐੑͱPrimary Key

    ʢৄ͘͠͸ޙड़ʣ
    • ιʔτࡁΈͷσʔλͷ෦෼ෳ੡

    ʢҟ࿦͸ೝΊΔʣ
    ࣹӨ
    ࡶͳMySQLύϑΥʔϚϯενϡʔχϯά from yoku0825
    https://www.slideshare.net/yoku0825/mysql-57449062/62

    View Slide

  28. • KEYS_ONLY: Primary Keyͷ஋ͷΈ
    • INCLUDE: Primary Keyͷ஋ + બ୒ͨ͠ଐੑ
    • ALL: શͯͷଐੑ
    • GSIʹؚΊͨଐੑ͸Primary KeyΛҾ͖௚͢ඞཁ͕ͳ͘ͳΔ
    • ಡΈࠐΈͷޮ཰Խ
    • ߋ৽ίετͱͷτϨʔυΦϑ
    • MySQL΍PostgreSQLͰݴ͏Covering Index / Index Only ScanͷΠϝʔδ
    GSI͕ࣹӨ͢ΔଐੑͷΦϓγϣϯ

    View Slide

  29. ࣹӨͱ͍͏͜ͱ͸ 1:Nͷؔ܎͕͜ͷΑ͏ʹղܾͰ͖Δ

    View Slide

  30. • ϝΠϯςʔϒϧͱ͸ผͷPartitionͰ෼ࢄ͞ΕΔ
    • ߋ৽͸ඇಉظͰ൓ө
    • ݁Ռ੔߹ੑͬͯେৎ෉ͳͷʁ
    • RDSͩͬͯRead ReplicaಡΜͩΒ݁Ռ੔߹ੑͰ͢Αʁʁ
    • GSI΁ͷΫΤϦ͸͍͍ͩͨRead Replicaʹ౤͛ΔΑ͏ͳΫΤϦ
    GSIʹର͢ΔΫΤϦ͸݁Ռ੔߹ੑ

    View Slide

  31. ͋ɺ͜ΕRDBͰʢԿճ໨ʁ

    View Slide

  32. ࢓૊ΈΛ஌Ε͹࣮ʹγϯϓϧ
    શ෦RDBͰطʹ͋ͬͨ࢓૊ΈͰ͢Αʁʁ

    View Slide

  33. ☓ DynamoDB͸บ͕͋Δ
    ○ RDBͷบ͕͋Γ͗͢Δ
    →γϯϓϧ͗͢ΔDynamoDBͱͷΪϟοϓ͕ڧͯͦ͘͏ײ͡Δ͚ͩ

    View Slide

  34. RDBʹચ೴͞ΕͯΔΜ͡Όͳ͍Ͱ͔͢ʁʁ
    ͱ͍͏͔ͦͷRDBͷ͜ͱ͢ΒͪΌΜͱ෼͔ͬͯ͋͛ͯ·͔ͨ͠ʁ

    View Slide

  35. RDBͷ͜ͱ෼͔ͬͯ·͔ͨ͠ʁ
    ͱ͍͏͔ɺͦͷେ޷͖ͳ

    View Slide

  36. DynamoDBͷσʔλϞσϦϯά
    Ͱ͸ͦΕΛ౿·͑ͯ

    View Slide

  37. ͦΕ͸σʔλͷ഑ஔΛܾΊΔ͜ͱ
    ҟ࿦͸ೝΊΔʂʂ

    View Slide

  38. σʔλϞσϦϯάͷߟ͑ํͷϙΠϯτ
    ഑ஔΛܾΊΔͨΊͷ

    View Slide

  39. • εΩʔϚϨεͷҙຯΛߟ͑Δ
    • List, Map͕࣋ͯΔ͜ͱʁ
    • ޙ͔ΒଐੑΛγʔϜϨεʹ௥ՃͰ͖Δ͜ͱʁ
    • A. ͳΜͰ΋ಥͬࠐΊΔ͜ͱ
    • Partition KeyͰ෼ࢄ
    • ςʔϒϧΛ෼͚ͳͯ͘΋ෛՙ͸෼ࢄ͢Δ
    • ۃ࿦͢Δͱ෼͚Δҙຯ͕ͳ͍ɺ෼͚ͳ͍ํ͕ΩϟύγςΟ͕؅ཧ͠΍͍͢
    • “ઃܭ͕༏ΕͨΞϓϦέʔγϣϯͰ͸ɺඞཁͳςʔϒϧ͸ 1 ͭͷΈͰ͢”
    • GSIͷݸ਺੍ݶ(5ݸ)ͳͲ΋͋ΔͷͰۀ຿υϝΠϯຖʹҰͭ͘Β͍ͷؾ࣋ͪ
    ςʔϒϧͷ෼͚ํ

    View Slide

  40. • RDB
    • ·ͣεΩʔϚΛߟ͑Δ
    • ਖ਼نԽ
    • ͦΕʹରͯ͠Ͳ͏ΞΫηε͢Δ͔(SQL)ߟ͑Δ
    • DynamoDB
    • εΩʔϚϨε
    • ඇਖ਼نԽ
    • ·ͣΞΫηεΛߟ͑ͯɺͦΕʹ߹ΘͤͨσʔλΛ࡞Δ
    • ΞϓϦέʔγϣϯͱσʔλϞσϧ͸ಉ࣌ʹઃܭ͢Δ
    RDBͱ͸Ξϓϩʔν͕શ͘ҟͳΔ

    View Slide

  41. ACIDτϥϯβΫγϣϯ͕ແ͍ͷ͸Ͳ͏͢Δͷʁ
    ͱ͸͍͑

    View Slide

  42. • ݪࢠੑ (Atomicity)
    • Ұ؏(੔߹)ੑ (Consistency)
    • ಠཱੑ (Isolation)
    • Ӭଓੑ (Durability)
    ͦ΋ͦ΋ACIDͬͯͳΜ͚ͩͬʁ
    ͜ΕΒΛ୲อ͢Δ͜ͱ͕໨తͰ͋ͬͯ
    τϥϯβΫγϣϯγεςϜͰ͋Δඞཁ͸ແ͍

    View Slide

  43. • RDB
    • ਖ਼نԽ͍ͯ͘͠ͱςʔϒϧͷ਺͸جຊతʹ૿͑Δ
    • ࿦ཧతʹ෼͔Εͨςʔϒϧ(Ϩίʔυ)ؒͷ੔߹ੑΛऔΔ࢓૊Έ͕͋Δ
    • ϦϨʔγϣϯγοϓ
    • ֎෦Ωʔ੍໿
    • ACIDτϥϯβΫγϣϯ
    • DynamoDB
    • ඇਖ਼نԽͯ͠1ͭͷ࣮ମʹ͍ͭͯ1ͭͷΞΠςϜʹऩΊΔ
    • 1ΞΠςϜͷߋ৽͸ΞτϛοΫ
    • ৚݅෇͖ॻ͖ࠐΈΛ࢖ָͬͨ؍తϩοΫ
    ACIDͷ୲อ

    View Slide

  44. • PutItem
    • ΞΠςϜΛ·Δ͝ͱߋ৽͢Δɺແ͔ͬͨΒ࡞Δ
    • Get -> ॻ͖׵͑ -> Putͩͱ੔߹ੑ่͕ΕΔʢՄೳੑ͕͋Δʣ
    • UpdateItem
    • ΞΠςϜΛ෦෼ߋ৽ɺແ͔ͬͨΒ࡞Δ
    • List, Mapͷ෦෼௥Ճɺ࡟আͳͲ΋Մೳ
    • ଐੑͷ஋Λར༻ͨ͠ߋ৽΋Մೳʢ͋Δଐੑʹ+1͢Δ౳)
    • ଐੑͷ஋Λར༻ͨ͠৚݅෇͖ॻ͖ࠐΈ
    DynamoDBͷߋ৽ૢ࡞

    View Slide

  45. ඇਖ਼نԽ͢ΔͱಡΈࠐΈ͕ͭΒ͍
    ϑϧεΩϟϯ͢Δ͔͠ͳ͍
    શ෦GSIషΔͷʁ

    View Slide

  46. CQRSͰ͢Α
    ͦ͜Ͱ

    View Slide

  47. • ॻ͖ࠐΉσʔλͱಡΈࠐΉσʔλ͸ಉ͡Ͱ͋Δඞཁ͸ແ͍
    • ݁Ռ੔߹ੑΛड͚ೖΕͯඇಉظͰಡΈࠐΈσʔλΛ࡞ΕΔ
    • Commandͷ׬ྃΛ΋ͬͯQuery༻σʔλΛ࡞ΕΔ
    • DynamoDB Streams͕Ϛον͢Δ
    • CQRSΛૉ௚ʹ࣮૷͢Δͱ·ͣKinesis Streams͚ͩͲ
    • Materialized View
    CQRS (Command Query Responsibility Segregation)

    View Slide

  48. ARCHITECTURE PATTERN
    AppSync͕͜Ε͔ΒॏཁͳҐஔΛ઎ΊΔ͸ͣ

    View Slide

  49. DATA ATOMICITY MicroserviesతͳΠϝʔδͰ͸͜ͷݪ୯Ґ͕ॏཁ
    σʔλετΞɾςʔϒϧͷ෼͚ํͰ͸ͳ͍
    ލ͕͍ͤͨͳΒMaterialized ViewΛ࡞Δ
    User
    Profile
    User
    Status
    Name Email
    Read/Write
    Search
    Read/Write

    View Slide

  50. ͜ΜͳΠϝʔδ 1ͭͷGSIͰ༷ʑͳݕࡧ͕Ͱ͖Δ

    View Slide

  51. • ෳ਺ͷ࣮ମʹލΔτϥϯβΫγϣϯ
    • DynamoDB୯ମͩͱ͜͜͸Ͳ͏ͯ͠΋ͭΒ͍
    • τϥϯβΫγϣϯ෼཭Ϩϕϧͷ࠷ߴ͸Selializable
    • ͭ·Γ௚ྻԽ
    • Kinesis StreamsͰ௚ྻԽ
    • ίέΔ͔΋͠Εͳ͍ͷͰႈ౳ʹ࣮૷͢Δ
    • ৚݅෇ॻ͖ࠐΈͰطʹߦͬͨॲཧΛSkip͢Δͷ΋ႈ౳
    ͱ͸͍͑ݶք͸͋Δ

    View Slide

  52. • Kinesis StreamsͰಉ࣌઀ଓ਺Λίϯτϩʔϧ͢Ε͹

    ผʹRDS࢖ͬͯ΋Կͷ໰୊΋ͳ͍Θ͚Ͱ
    • ಉ࣌઀ଓ਺ → Shard਺ͱಉ਺Ͱݻఆ
    • Lambda͸جຊWarm Start
    • ͭ·ΓɺENIͷ஗Ԇͱ΄΅ແԑ
    • σϓϩΠ΍Shard਺ม͑ͨλΠϛϯάͰ͸ى͖͏Δ͚Ͳͦ΋ͦ΋ඇಉظͳͷͰʢ͈́
    ͱ͍͏͔

    View Slide

  53. ADVANCED TIPS
    ͕࣌ؒ͋Ε͹͍͔ͭ͘͝঺հ

    View Slide

  54. • DynamoDBͷIndex͸Sparse
    • Indexଐੑ͕ଘࡏ͠ͳ͍৔߹ɺIndexʹͦͷσʔλ͸ؚ·ͳ͍
    • ಛఆͷΞΠςϜͷΈʹଘࡏ͢ΔଐੑΛࢦఆ͢Δ͜ͱͰ

    Index༰ྔ͕େ͖͘ѹॖͰ͖Δ
    Sparse Index

    View Slide

  55. Sort΋ཁΒͳ͍ GSIʹ͸҉໧ͰPrimary Keyؚ͕·ΕΔ

    View Slide

  56. • Α͋͘Δ֊૚ߏ଄Λ࣋ͭσʔλ
    • ૊৫ߏ଄
    • ωετͨ͠ΧςΰϥΠζ
    • ͳͲͳͲ
    • ݕࡧΛߟ͑ͳ͚Ε͹֊૚ߏ଄Λ·Δͬͱ1ΞΠςϜʹԡ͠ࠐΉ
    • Sort KeyͰ֊૚Λදݱ͢Δ
    Hierarchical data

    View Slide

  57. Sort KeyͰલํҰகݕࡧ ͋ɺ͜ΕSQLΞϯνύλʔϯͰʢ͈́
    ঎඼ઃܭ͕ࡶͳͷ͸͝Ѫᇷ

    View Slide

  58. • Itemʹ”Version”ͷΑ͏ͳଐੑΛ࣋ͨͤΔ
    • ଐੑͷ஋Λར༻ͨ͠৚݅෇͖ॻ͖ࠐΈ
    • ConditionExpression=“Version = 1”
    • UpdateExpression=“ADD Version 1”
    • Version = 1Ͱͳ͚Ε͹Τϥʔ͕ฦΔ
    • Version = 1ͳΒ2ʹߋ৽͞ΕΔ
    όʔδϣχϯάͱָ؍తϩοΫ

    View Slide

  59. • ୺తʹݴͬͯόΠϒϧ
    • ͜ΕΛಡΜͰશͯཧղͰ͖ͨΒҰਓલʂʂ
    • https://docs.aws.amazon.com/ja_jp/amazondynamodb/latest/
    developerguide/best-practices.html
    DynamoDB Best Practices

    View Slide

  60. “DynamoDB ދͷר” [ݕࡧ]
    http://marcy.hatenablog.com/entry/2018/07/31/213705
    ҰԠະ׬ͳͷͰ·ͩॻ͔͘΋

    View Slide

  61. “LET’S BECOME THE FRIEND OF RDB”
    THANKS!!
    “LET’S BECOME THE FRIEND OF DYNAMODB”
    “LET’S BECOME THE FRIEND OF SERVERLESS”

    View Slide