Serverless??そんなことよりDBだ!! / We should know Databases before Serverless

JAWS Festa 2018の資料です
https://jft2018.jaws-ug.jp/session/729/

Transcript

1. SERVERLESSʁʁ
   ͦΜͳ͜ͱΑΓDBͩʂʂ
   MASASHI TERUI @ JAWS FESTA OSAKA 2018
   

   View Slide

2. SERVERWORKS CO.,LTD.
   + FREELANCER
   • Serverless Oji-san
   • Serverless Framework Plugin Developer
   • Serverlessconf Tokyo 2016,2017,2018 speaker
   • Remote worker (in Sapporo-shi, Hokkaido)
   • http://marcy.hatenablog.com/entry/2018/08/04/231241
   MASASHI TERUI
   ARCHITECT / DEVELOPER
   

   View Slide

3. SERVERLESS΍ͬͯ·͔͢ʁ
   DynamoDB࢖ͬͯ·͔͢ʁ
   

   View Slide

4. • ޷͖
   • ී௨
   • ݏ͍
   • LambdaͱRDSͷ૬ੑ͕ѱ͍͔Β࢓ํͳ͘෇͖߹͍ͬͯΔ
   DynamoDB͸޷͖Ͱ͔͢ʁ
   

   View Slide

5. ࢓ํ͕ͳ͍͔ΒDynamoDB……
   ຊ౰͸RDBΛ࢖͍͍͚ͨͲ
   

   View Slide

6. RDBʹ؁͑ͯΔΜ͡Όͳ͍Ͱ͔͢ʁʁ
   NoSQL͕࢖͑ͳ͍Μ͡Όͳͯ͘
   

   View Slide

7. RDBͷ͜ͱͪΌΜͱ෼͔ͬͯ·͔͢ʁʁ
   ͦ΋ͦ΋
   

   View Slide

8. • ίωΫγϣϯϞσϧ
   • Ӭଓత
   • ίωΫγϣϯϓʔϦϯά
   • ηΩϡϦςΟ
   • ૊ΈࠐΈͷऑ͍ೝূ
   • VPC಺΁ͷΞΫηείετʢENIੜ੒ॲཧʣ
   LambdaͱRDSͷ૬ੑ͕Կނѱ͍ͷ͔આ໌Ͱ͖·͔͢ʁ
   ͱͯ΋OLTPͰ͸࢖͑ͳ͍(਺ඵʙे਺ඵ)
   

   View Slide

9. DynamoDBͷ͜ͱ෼͔ͬͯ·͔͢ʁʁ
   ͡Ό͋
   

   View Slide

10. σʔλϕʔεͷ͜ͱ෼͔ͬͯ·͔͢ʁʁ
    ͱ͍͏͔
    

    View Slide

11. • ςʔϒϧઃܭͱ͔ΠϯσοΫεઃܭͬͯͲ͏΍Δͷʁ
    • ACIDτϥϯβΫγϣϯແ͍ͷʹେৎ෉ͳͷʁ
    • ݁Ռ੔߹ੑͬͯେৎ෉ͳͷʁ
    • ্ख͘ੑೳΛҾ͖ग़͢ʹ͸Ͳ͏͢Ε͹ྑ͍ʁ
    DynamoDBͷΑ͋͘Δෆ҆
    

    View Slide

12. • ςʔϒϧઃܭͱ͔ΠϯσοΫεઃܭͬͯͲ͏΍Δͷʁ
    • ACIDτϥϯβΫγϣϯແ͍ͷʹେৎ෉ͳͷʁ
    • ݁Ռ੔߹ੑͬͯେৎ෉ͳͷʁ
    • ্ख͘ੑೳΛҾ͖ग़͢ʹ͸Ͳ͏͢Ε͹ྑ͍ʁ
    DynamoDBͷΑ͋͘Δෆ҆
    ෼͔Βͳ͍͔Βෆ҆ʹͳΔ
    

    View Slide

13. • AWS͕ఏڙ͢ΔNoSQLαʔϏε
    • KeyʹΑΔ׬શҰகͱIndex୳ࡧ͕Մೳ
    • ϑϧϚωʔδυ
    • ແ੍ݶͷࣗಈ֦ுετϨʔδʢ࢖ͬͨ෼͚ͩʣ
    • Πϯελϯεͷ؅ཧෆཁɺWrite/Readͷੑೳ͚ͩΛࢦఆ͢Δ
    • Ͱ΋ɺΩϟύγςΟͷࣄલ༧໿͕ඞཁͳαʔϏε͸Serverless͡Όͳ͍ͬͯCNCF͞Μ͕ݴͬɾɾɾ͋ɺ୭͔དྷͨΈ͍ͨ
    DynamoDBͷجຊ
    

    View Slide

14. DynamoDB͸ͲΜͳ ”AWSαʔϏε” ͔
    Ͱ͸ͳ͘
    

    View Slide

15. DynamoDB͸ͲΜͳ ”σʔλϕʔε” ͔
    ஌ͬͯ·͔͢ʁ
    

    View Slide

16. • Consistent Hashing
    • Vector clocks with reconciliation during reads
    • Sloppy Quorum and hinted handoff
    • Anti-entropy using Merkle trees
    • Gossip-based membership protocol and failure detection
    Amazon’s Dynamo
    ৄ͘͠஌Γ͍ͨͳΒʢBetterͰ͋ͬͯMustͰ͸ͳ͍ʣ
    https://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
    

    View Slide

17. • Ωʔͷϋογϡ஋Ͱ෼ࢄ → Partition Key
    • ͦͷ෼ࢄͨ͠தͰIndex͕ߏங͞Ε͍ͯΔ → Sort Key
    • όοΫάϥϯυͰߋ৽ΛޙউͪͰಉظ
    • ݁Ռ੔߹ੑϕʔε
    • ॻ͖ࠐΈɾಡΈࠐΈͷQuorum਺Ͱ੔߹ੑ͕ܾ·Δ
    • ΩʔΛࢦఆͨ͠ಡΈࠐΈ͸ڧ੔߹ੑ͕બ୒Մೳ
    ཈͑Δ΂͖ϙΠϯτ
    

    View Slide

18. Partition Keyͷϋογϡ஋Ͱ෼ࢄ
    ͋ɺ͜ΕRDB(Hash Index)Ͱʢ͈́
    

    View Slide

19. • Partition Keyʹ࿈൪Λ࢖͏ͷ͸Կͷҙຯ΋ͳ͍͜ͱ͕෼͔Δ
    • ࿈൪Χ΢ϯλʔΛ࡞ΔͱPartition͕ภΔ
    • Max + 1ͳΜͯҎͯͷ֎
    • ࿦ཧతʹҰҙͱͳΔଐੑ͕͋ΔͳΒͦΕΛ࢖͏
    • ͳ͚Ε͹UUIDͷΑ͏ͳিಥՄೳੑ͕ۃখͳϥϯμϜ஋Ͱྑ͍
    • Sort΋Ͱ͖ͳ͍ͷͰSnowflakeͷΑ͏ͳॱংੑ΋ཁΒͳ͍
    • User ID͸CognitoͰೝূ͍ͯ͠ΔͳΒCognito͕ൃߦ͢ΔIDΛ࢖͏
    • ΞΫηε੍ޚʹ΋࢖͑Δ
    ͜ͷલఏΛ౿·͑Δͱ
    

    View Slide

20. • 3000RCU, 1000WCU, Index Size 10GB (2018-10-31ݱࡏ)
    • ͲΕ͔ҰͭͰ΋௒ա͢ΔͱPartition͕෼ׂ͞ΕΔ
    • ෼ׂ͞ΕΔͱ֬อͨ͠Ϣχοτ͕ۉ౳ʹׂΓৼΒΕΔ
    • εϩοτϦϯά࣌ͷҰ࣌తͳόʔετ͕͋Δ
    • ݶఆతͳόʔετ
    • ༨৒෼Λճ͢ (Adaptive Capacity)
    Partition͝ͱͷΩϟύγςΟ
    

    View Slide

21. Partition KeyͰ෼ࢄɺSort Key͸ʁ
    

    View Slide

22. B+TREE INDEX
    B-treeͳͷ͔B+treeͳͷ͔͸ެද͞Ε͍ͯͳ͍͚Ͳ
    Range Keyͱ͔ݺͿ͘Β͍͔ͩΒ…Ͷʁ
    ͋ɺ͜ΕRDBͰʢ͈́
    

    View Slide

23. TARGETING QUERY B+treeͷߏ଄Λ஌͍ͬͯΕ͹
    ૊Έ߹ΘͤΩʔͷΠϯσοΫε͕ແͯ͘΋
    ͜͏΍Ε͹ྑ͍͜ͱ͕෼͔Δ
    

    View Slide

24. Partition KeyͰ෼ࢄͨ͠தͰSort KeyͰ੔ྻ
    ͜ͷΠϝʔδΛ࣋ͭ͜ͱ͕ͱͯ΋ॏཁ
    

    View Slide

25. • Partition KeyͰ෼ࢄͨ͠ΠϯελϯεʹIndex͕ߏங͞ΕΔ
    • ҰͭͷΠϯελϯε಺Ͱ͸ಉظతʹIndex΋ߋ৽͞ΕΔ
    • ڧ੔߹ੑ͕બ୒Մೳ
    • Primary Sort KeyͷϦʔϑϊʔυʹ࣮σʔλͷΞυϨε͕

    ֨ೲ͞Ε͍ͯΔΠϝʔδʢͱࢥΘΕΔʣ
    • LSI͸࣮σʔλΛ࣋ͨͳ͍ͷͰॏෳՄೳ
    Primary Sort Key, LSI
    

    View Slide

26. GSI͸࣮ςʔϒϧͷࣹӨ
    

    View Slide

27. • ू߹࿦ʹ͓͚ΔࣹӨ
    • GSIͷPartition KeyΛݩʹ෼ࢄ

    ʢߋ৽͸ඇಉظͰద༻ʣ
    • ϦʔϑϊʔυʹଐੑͱPrimary Key

    ʢৄ͘͠͸ޙड़ʣ
    • ιʔτࡁΈͷσʔλͷ෦෼ෳ੡

    ʢҟ࿦͸ೝΊΔʣ
    ࣹӨ
    ࡶͳMySQLύϑΥʔϚϯενϡʔχϯά from yoku0825
    https://www.slideshare.net/yoku0825/mysql-57449062/62
    

    View Slide

28. • KEYS_ONLY: Primary Keyͷ஋ͷΈ
    • INCLUDE: Primary Keyͷ஋ + બ୒ͨ͠ଐੑ
    • ALL: શͯͷଐੑ
    • GSIʹؚΊͨଐੑ͸Primary KeyΛҾ͖௚͢ඞཁ͕ͳ͘ͳΔ
    • ಡΈࠐΈͷޮ཰Խ
    • ߋ৽ίετͱͷτϨʔυΦϑ
    • MySQL΍PostgreSQLͰݴ͏Covering Index / Index Only ScanͷΠϝʔδ
    GSI͕ࣹӨ͢ΔଐੑͷΦϓγϣϯ
    

    View Slide

29. ࣹӨͱ͍͏͜ͱ͸ 1:Nͷؔ܎͕͜ͷΑ͏ʹղܾͰ͖Δ
    

    View Slide

30. • ϝΠϯςʔϒϧͱ͸ผͷPartitionͰ෼ࢄ͞ΕΔ
    • ߋ৽͸ඇಉظͰ൓ө
    • ݁Ռ੔߹ੑͬͯେৎ෉ͳͷʁ
    • RDSͩͬͯRead ReplicaಡΜͩΒ݁Ռ੔߹ੑͰ͢Αʁʁ
    • GSI΁ͷΫΤϦ͸͍͍ͩͨRead Replicaʹ౤͛ΔΑ͏ͳΫΤϦ
    GSIʹର͢ΔΫΤϦ͸݁Ռ੔߹ੑ
    

    View Slide

31. ͋ɺ͜ΕRDBͰʢԿճ໨ʁ
    

    View Slide

32. ࢓૊ΈΛ஌Ε͹࣮ʹγϯϓϧ
    શ෦RDBͰطʹ͋ͬͨ࢓૊ΈͰ͢Αʁʁ
    

    View Slide

33. ☓ DynamoDB͸บ͕͋Δ
    ○ RDBͷบ͕͋Γ͗͢Δ
    →γϯϓϧ͗͢ΔDynamoDBͱͷΪϟοϓ͕ڧͯͦ͘͏ײ͡Δ͚ͩ
    

    View Slide

34. RDBʹચ೴͞ΕͯΔΜ͡Όͳ͍Ͱ͔͢ʁʁ
    ͱ͍͏͔ͦͷRDBͷ͜ͱ͢ΒͪΌΜͱ෼͔ͬͯ͋͛ͯ·͔ͨ͠ʁ
    

    View Slide

35. RDBͷ͜ͱ෼͔ͬͯ·͔ͨ͠ʁ
    ͱ͍͏͔ɺͦͷେ޷͖ͳ
    

    View Slide

36. DynamoDBͷσʔλϞσϦϯά
    Ͱ͸ͦΕΛ౿·͑ͯ
    

    View Slide

37. ͦΕ͸σʔλͷ഑ஔΛܾΊΔ͜ͱ
    ҟ࿦͸ೝΊΔʂʂ
    

    View Slide

38. σʔλϞσϦϯάͷߟ͑ํͷϙΠϯτ
    ഑ஔΛܾΊΔͨΊͷ
    

    View Slide

39. • εΩʔϚϨεͷҙຯΛߟ͑Δ
    • List, Map͕࣋ͯΔ͜ͱʁ
    • ޙ͔ΒଐੑΛγʔϜϨεʹ௥ՃͰ͖Δ͜ͱʁ
    • A. ͳΜͰ΋ಥͬࠐΊΔ͜ͱ
    • Partition KeyͰ෼ࢄ
    • ςʔϒϧΛ෼͚ͳͯ͘΋ෛՙ͸෼ࢄ͢Δ
    • ۃ࿦͢Δͱ෼͚Δҙຯ͕ͳ͍ɺ෼͚ͳ͍ํ͕ΩϟύγςΟ͕؅ཧ͠΍͍͢
    • “ઃܭ͕༏ΕͨΞϓϦέʔγϣϯͰ͸ɺඞཁͳςʔϒϧ͸ 1 ͭͷΈͰ͢”
    • GSIͷݸ਺੍ݶ(5ݸ)ͳͲ΋͋ΔͷͰۀ຿υϝΠϯຖʹҰͭ͘Β͍ͷؾ࣋ͪ
    ςʔϒϧͷ෼͚ํ
    

    View Slide

40. • RDB
    • ·ͣεΩʔϚΛߟ͑Δ
    • ਖ਼نԽ
    • ͦΕʹରͯ͠Ͳ͏ΞΫηε͢Δ͔(SQL)ߟ͑Δ
    • DynamoDB
    • εΩʔϚϨε
    • ඇਖ਼نԽ
    • ·ͣΞΫηεΛߟ͑ͯɺͦΕʹ߹ΘͤͨσʔλΛ࡞Δ
    • ΞϓϦέʔγϣϯͱσʔλϞσϧ͸ಉ࣌ʹઃܭ͢Δ
    RDBͱ͸Ξϓϩʔν͕શ͘ҟͳΔ
    

    View Slide

41. ACIDτϥϯβΫγϣϯ͕ແ͍ͷ͸Ͳ͏͢Δͷʁ
    ͱ͸͍͑
    

    View Slide

42. • ݪࢠੑ (Atomicity)
    • Ұ؏(੔߹)ੑ (Consistency)
    • ಠཱੑ (Isolation)
    • Ӭଓੑ (Durability)
    ͦ΋ͦ΋ACIDͬͯͳΜ͚ͩͬʁ
    ͜ΕΒΛ୲อ͢Δ͜ͱ͕໨తͰ͋ͬͯ
    τϥϯβΫγϣϯγεςϜͰ͋Δඞཁ͸ແ͍
    

    View Slide

43. • RDB
    • ਖ਼نԽ͍ͯ͘͠ͱςʔϒϧͷ਺͸جຊతʹ૿͑Δ
    • ࿦ཧతʹ෼͔Εͨςʔϒϧ(Ϩίʔυ)ؒͷ੔߹ੑΛऔΔ࢓૊Έ͕͋Δ
    • ϦϨʔγϣϯγοϓ
    • ֎෦Ωʔ੍໿
    • ACIDτϥϯβΫγϣϯ
    • DynamoDB
    • ඇਖ਼نԽͯ͠1ͭͷ࣮ମʹ͍ͭͯ1ͭͷΞΠςϜʹऩΊΔ
    • 1ΞΠςϜͷߋ৽͸ΞτϛοΫ
    • ৚݅෇͖ॻ͖ࠐΈΛ࢖ָͬͨ؍తϩοΫ
    ACIDͷ୲อ
    

    View Slide

44. • PutItem
    • ΞΠςϜΛ·Δ͝ͱߋ৽͢Δɺແ͔ͬͨΒ࡞Δ
    • Get -> ॻ͖׵͑ -> Putͩͱ੔߹ੑ่͕ΕΔʢՄೳੑ͕͋Δʣ
    • UpdateItem
    • ΞΠςϜΛ෦෼ߋ৽ɺແ͔ͬͨΒ࡞Δ
    • List, Mapͷ෦෼௥Ճɺ࡟আͳͲ΋Մೳ
    • ଐੑͷ஋Λར༻ͨ͠ߋ৽΋Մೳʢ͋Δଐੑʹ+1͢Δ౳)
    • ଐੑͷ஋Λར༻ͨ͠৚݅෇͖ॻ͖ࠐΈ
    DynamoDBͷߋ৽ૢ࡞
    

    View Slide

45. ඇਖ਼نԽ͢ΔͱಡΈࠐΈ͕ͭΒ͍
    ϑϧεΩϟϯ͢Δ͔͠ͳ͍
    શ෦GSIషΔͷʁ
    

    View Slide

46. CQRSͰ͢Α
    ͦ͜Ͱ
    

    View Slide

47. • ॻ͖ࠐΉσʔλͱಡΈࠐΉσʔλ͸ಉ͡Ͱ͋Δඞཁ͸ແ͍
    • ݁Ռ੔߹ੑΛड͚ೖΕͯඇಉظͰಡΈࠐΈσʔλΛ࡞ΕΔ
    • Commandͷ׬ྃΛ΋ͬͯQuery༻σʔλΛ࡞ΕΔ
    • DynamoDB Streams͕Ϛον͢Δ
    • CQRSΛૉ௚ʹ࣮૷͢Δͱ·ͣKinesis Streams͚ͩͲ
    • Materialized View
    CQRS (Command Query Responsibility Segregation)
    

    View Slide

48. ARCHITECTURE PATTERN
    AppSync͕͜Ε͔ΒॏཁͳҐஔΛ઎ΊΔ͸ͣ
    

    View Slide

49. DATA ATOMICITY MicroserviesతͳΠϝʔδͰ͸͜ͷݪ୯Ґ͕ॏཁ
    σʔλετΞɾςʔϒϧͷ෼͚ํͰ͸ͳ͍
    ލ͕͍ͤͨͳΒMaterialized ViewΛ࡞Δ
    User
    Profile
    User
    Status
    Name Email
    Read/Write
    Search
    Read/Write
    

    View Slide

50. ͜ΜͳΠϝʔδ 1ͭͷGSIͰ༷ʑͳݕࡧ͕Ͱ͖Δ
    

    View Slide

51. • ෳ਺ͷ࣮ମʹލΔτϥϯβΫγϣϯ
    • DynamoDB୯ମͩͱ͜͜͸Ͳ͏ͯ͠΋ͭΒ͍
    • τϥϯβΫγϣϯ෼཭Ϩϕϧͷ࠷ߴ͸Selializable
    • ͭ·Γ௚ྻԽ
    • Kinesis StreamsͰ௚ྻԽ
    • ίέΔ͔΋͠Εͳ͍ͷͰႈ౳ʹ࣮૷͢Δ
    • ৚݅෇ॻ͖ࠐΈͰطʹߦͬͨॲཧΛSkip͢Δͷ΋ႈ౳
    ͱ͸͍͑ݶք͸͋Δ
    

    View Slide

52. • Kinesis StreamsͰಉ࣌઀ଓ਺Λίϯτϩʔϧ͢Ε͹

    ผʹRDS࢖ͬͯ΋Կͷ໰୊΋ͳ͍Θ͚Ͱ
    • ಉ࣌઀ଓ਺ → Shard਺ͱಉ਺Ͱݻఆ
    • Lambda͸جຊWarm Start
    • ͭ·ΓɺENIͷ஗Ԇͱ΄΅ແԑ
    • σϓϩΠ΍Shard਺ม͑ͨλΠϛϯάͰ͸ى͖͏Δ͚Ͳͦ΋ͦ΋ඇಉظͳͷͰʢ͈́
    ͱ͍͏͔
    

    View Slide

53. ADVANCED TIPS
    ͕࣌ؒ͋Ε͹͍͔ͭ͘͝঺հ
    

    View Slide

54. • DynamoDBͷIndex͸Sparse
    • Indexଐੑ͕ଘࡏ͠ͳ͍৔߹ɺIndexʹͦͷσʔλ͸ؚ·ͳ͍
    • ಛఆͷΞΠςϜͷΈʹଘࡏ͢ΔଐੑΛࢦఆ͢Δ͜ͱͰ

    Index༰ྔ͕େ͖͘ѹॖͰ͖Δ
    Sparse Index
    

    View Slide

55. Sort΋ཁΒͳ͍ GSIʹ͸҉໧ͰPrimary Keyؚ͕·ΕΔ
    

    View Slide

56. • Α͋͘Δ֊૚ߏ଄Λ࣋ͭσʔλ
    • ૊৫ߏ଄
    • ωετͨ͠ΧςΰϥΠζ
    • ͳͲͳͲ
    • ݕࡧΛߟ͑ͳ͚Ε͹֊૚ߏ଄Λ·Δͬͱ1ΞΠςϜʹԡ͠ࠐΉ
    • Sort KeyͰ֊૚Λදݱ͢Δ
    Hierarchical data
    

    View Slide

57. Sort KeyͰલํҰகݕࡧ ͋ɺ͜ΕSQLΞϯνύλʔϯͰʢ͈́
    ঎඼ઃܭ͕ࡶͳͷ͸͝Ѫᇷ
    

    View Slide

58. • Itemʹ”Version”ͷΑ͏ͳଐੑΛ࣋ͨͤΔ
    • ଐੑͷ஋Λར༻ͨ͠৚݅෇͖ॻ͖ࠐΈ
    • ConditionExpression=“Version = 1”
    • UpdateExpression=“ADD Version 1”
    • Version = 1Ͱͳ͚Ε͹Τϥʔ͕ฦΔ
    • Version = 1ͳΒ2ʹߋ৽͞ΕΔ
    όʔδϣχϯάͱָ؍తϩοΫ
    

    View Slide

59. • ୺తʹݴͬͯόΠϒϧ
    • ͜ΕΛಡΜͰશͯཧղͰ͖ͨΒҰਓલʂʂ
    • https://docs.aws.amazon.com/ja_jp/amazondynamodb/latest/
    developerguide/best-practices.html
    DynamoDB Best Practices
    

    View Slide

60. “DynamoDB ދͷר” [ݕࡧ]
    http://marcy.hatenablog.com/entry/2018/07/31/213705
    ҰԠະ׬ͳͷͰ·ͩॻ͔͘΋
    

    View Slide

61. “LET’S BECOME THE FRIEND OF RDB”
    THANKS!!
    “LET’S BECOME THE FRIEND OF DYNAMODB”
    “LET’S BECOME THE FRIEND OF SERVERLESS”
    

    View Slide