Serverless??そんなことよりDBだ!! / We should know Databases before Serverless

by Masashi Terui

Slide 1

Slide 1 text

SERVERLESSʁʁ ͦΜͳ͜ͱΑΓDBͩʂʂ MASASHI TERUI @ JAWS FESTA OSAKA 2018

Slide 2

Slide 2 text

SERVERWORKS CO.,LTD. + FREELANCER • Serverless Oji-san • Serverless Framework Plugin Developer • Serverlessconf Tokyo 2016,2017,2018 speaker • Remote worker (in Sapporo-shi, Hokkaido) • http://marcy.hatenablog.com/entry/2018/08/04/231241 MASASHI TERUI ARCHITECT / DEVELOPER

Slide 3

Slide 3 text

SERVERLESS΍ͬͯ·͔͢ʁ DynamoDB࢖ͬͯ·͔͢ʁ

Slide 4

Slide 4 text

• ޷͖ • ී௨ • ݏ͍ • LambdaͱRDSͷ૬ੑ͕ѱ͍͔Β࢓ํͳ͘෇͖߹͍ͬͯΔ DynamoDB͸޷͖Ͱ͔͢ʁ

Slide 5

Slide 5 text

࢓ํ͕ͳ͍͔ΒDynamoDB…… ຊ౰͸RDBΛ࢖͍͍͚ͨͲ

Slide 6

Slide 6 text

RDBʹ؁͑ͯΔΜ͡Όͳ͍Ͱ͔͢ʁʁ NoSQL͕࢖͑ͳ͍Μ͡Όͳͯ͘

Slide 7

Slide 7 text

RDBͷ͜ͱͪΌΜͱ෼͔ͬͯ·͔͢ʁʁ ͦ΋ͦ΋

Slide 8

Slide 8 text

• ίωΫγϣϯϞσϧ • Ӭଓత • ίωΫγϣϯϓʔϦϯά • ηΩϡϦςΟ • ૊ΈࠐΈͷऑ͍ೝূ • VPC಺΁ͷΞΫηείετʢENIੜ੒ॲཧʣ LambdaͱRDSͷ૬ੑ͕Կނѱ͍ͷ͔આ໌Ͱ͖·͔͢ʁ ͱͯ΋OLTPͰ͸࢖͑ͳ͍(਺ඵʙे਺ඵ)

Slide 9

Slide 9 text

DynamoDBͷ͜ͱ෼͔ͬͯ·͔͢ʁʁ ͡Ό͋

Slide 10

Slide 10 text

σʔλϕʔεͷ͜ͱ෼͔ͬͯ·͔͢ʁʁ ͱ͍͏͔

Slide 11

Slide 11 text

• ςʔϒϧઃܭͱ͔ΠϯσοΫεઃܭͬͯͲ͏΍Δͷʁ • ACIDτϥϯβΫγϣϯແ͍ͷʹେৎ෉ͳͷʁ • ݁Ռ੔߹ੑͬͯେৎ෉ͳͷʁ • ্ख͘ੑೳΛҾ͖ग़͢ʹ͸Ͳ͏͢Ε͹ྑ͍ʁ DynamoDBͷΑ͋͘Δෆ҆

Slide 12

Slide 12 text

• ςʔϒϧઃܭͱ͔ΠϯσοΫεઃܭͬͯͲ͏΍Δͷʁ • ACIDτϥϯβΫγϣϯແ͍ͷʹେৎ෉ͳͷʁ • ݁Ռ੔߹ੑͬͯେৎ෉ͳͷʁ • ্ख͘ੑೳΛҾ͖ग़͢ʹ͸Ͳ͏͢Ε͹ྑ͍ʁ DynamoDBͷΑ͋͘Δෆ҆ ෼͔Βͳ͍͔Βෆ҆ʹͳΔ

Slide 13

Slide 13 text

• AWS͕ఏڙ͢ΔNoSQLαʔϏε • KeyʹΑΔ׬શҰகͱIndex୳ࡧ͕Մೳ • ϑϧϚωʔδυ • ແ੍ݶͷࣗಈ֦ுετϨʔδʢ࢖ͬͨ෼͚ͩʣ • Πϯελϯεͷ؅ཧෆཁɺWrite/Readͷੑೳ͚ͩΛࢦఆ͢Δ • Ͱ΋ɺΩϟύγςΟͷࣄલ༧໿͕ඞཁͳαʔϏε͸Serverless͡Όͳ͍ͬͯCNCF͞Μ͕ݴͬɾɾɾ͋ɺ୭͔དྷͨΈ͍ͨ DynamoDBͷجຊ

Slide 14

Slide 14 text

DynamoDB͸ͲΜͳ ”AWSαʔϏε” ͔ Ͱ͸ͳ͘

Slide 15

Slide 15 text

DynamoDB͸ͲΜͳ ”σʔλϕʔε” ͔ ஌ͬͯ·͔͢ʁ

Slide 16

Slide 16 text

• Consistent Hashing • Vector clocks with reconciliation during reads • Sloppy Quorum and hinted handoff • Anti-entropy using Merkle trees • Gossip-based membership protocol and failure detection Amazon’s Dynamo ৄ͘͠஌Γ͍ͨͳΒʢBetterͰ͋ͬͯMustͰ͸ͳ͍ʣ https://www.allthingsdistributed.com/2007/10/amazons_dynamo.html

Slide 17

Slide 17 text

• Ωʔͷϋογϡ஋Ͱ෼ࢄ → Partition Key • ͦͷ෼ࢄͨ͠தͰIndex͕ߏங͞Ε͍ͯΔ → Sort Key • όοΫάϥϯυͰߋ৽ΛޙউͪͰಉظ • ݁Ռ੔߹ੑϕʔε • ॻ͖ࠐΈɾಡΈࠐΈͷQuorum਺Ͱ੔߹ੑ͕ܾ·Δ • ΩʔΛࢦఆͨ͠ಡΈࠐΈ͸ڧ੔߹ੑ͕બ୒Մೳ ཈͑Δ΂͖ϙΠϯτ

Slide 18

Slide 18 text

Partition Keyͷϋογϡ஋Ͱ෼ࢄ ͋ɺ͜ΕRDB(Hash Index)Ͱʢ͈́

Slide 19

Slide 19 text

• Partition Keyʹ࿈൪Λ࢖͏ͷ͸Կͷҙຯ΋ͳ͍͜ͱ͕෼͔Δ • ࿈൪Χ΢ϯλʔΛ࡞ΔͱPartition͕ภΔ • Max + 1ͳΜͯҎͯͷ֎ • ࿦ཧతʹҰҙͱͳΔଐੑ͕͋ΔͳΒͦΕΛ࢖͏ • ͳ͚Ε͹UUIDͷΑ͏ͳিಥՄೳੑ͕ۃখͳϥϯμϜ஋Ͱྑ͍ • Sort΋Ͱ͖ͳ͍ͷͰSnowﬂakeͷΑ͏ͳॱংੑ΋ཁΒͳ͍ • User ID͸CognitoͰೝূ͍ͯ͠ΔͳΒCognito͕ൃߦ͢ΔIDΛ࢖͏ • ΞΫηε੍ޚʹ΋࢖͑Δ ͜ͷલఏΛ౿·͑Δͱ

Slide 20

Slide 20 text

• 3000RCU, 1000WCU, Index Size 10GB (2018-10-31ݱࡏ) • ͲΕ͔ҰͭͰ΋௒ա͢ΔͱPartition͕෼ׂ͞ΕΔ • ෼ׂ͞ΕΔͱ֬อͨ͠Ϣχοτ͕ۉ౳ʹׂΓৼΒΕΔ • εϩοτϦϯά࣌ͷҰ࣌తͳόʔετ͕͋Δ • ݶఆతͳόʔετ • ༨৒෼Λճ͢ (Adaptive Capacity) Partition͝ͱͷΩϟύγςΟ

Slide 21

Slide 21 text

Partition KeyͰ෼ࢄɺSort Key͸ʁ

Slide 22

Slide 22 text

B+TREE INDEX B-treeͳͷ͔B+treeͳͷ͔͸ެද͞Ε͍ͯͳ͍͚Ͳ Range Keyͱ͔ݺͿ͘Β͍͔ͩΒ…Ͷʁ ͋ɺ͜ΕRDBͰʢ͈́

Slide 23

Slide 23 text

TARGETING QUERY B+treeͷߏ଄Λ஌͍ͬͯΕ͹ ૊Έ߹ΘͤΩʔͷΠϯσοΫε͕ແͯ͘΋ ͜͏΍Ε͹ྑ͍͜ͱ͕෼͔Δ

Slide 24

Slide 24 text

Partition KeyͰ෼ࢄͨ͠தͰSort KeyͰ੔ྻ ͜ͷΠϝʔδΛ࣋ͭ͜ͱ͕ͱͯ΋ॏཁ

Slide 25

Slide 25 text

• Partition KeyͰ෼ࢄͨ͠ΠϯελϯεʹIndex͕ߏங͞ΕΔ • ҰͭͷΠϯελϯε಺Ͱ͸ಉظతʹIndex΋ߋ৽͞ΕΔ • ڧ੔߹ੑ͕બ୒Մೳ • Primary Sort KeyͷϦʔϑϊʔυʹ࣮σʔλͷΞυϨε͕  ֨ೲ͞Ε͍ͯΔΠϝʔδʢͱࢥΘΕΔʣ • LSI͸࣮σʔλΛ࣋ͨͳ͍ͷͰॏෳՄೳ Primary Sort Key, LSI

Slide 26

Slide 26 text

GSI͸࣮ςʔϒϧͷࣹӨ

Slide 27

Slide 27 text

• ू߹࿦ʹ͓͚ΔࣹӨ • GSIͷPartition KeyΛݩʹ෼ࢄ  ʢߋ৽͸ඇಉظͰద༻ʣ • ϦʔϑϊʔυʹଐੑͱPrimary Key  ʢৄ͘͠͸ޙड़ʣ • ιʔτࡁΈͷσʔλͷ෦෼ෳ੡  ʢҟ࿦͸ೝΊΔʣ ࣹӨ ࡶͳMySQLύϑΥʔϚϯενϡʔχϯά from yoku0825 https://www.slideshare.net/yoku0825/mysql-57449062/62

Slide 28

Slide 28 text

• KEYS_ONLY: Primary Keyͷ஋ͷΈ • INCLUDE: Primary Keyͷ஋ + બ୒ͨ͠ଐੑ • ALL: શͯͷଐੑ • GSIʹؚΊͨଐੑ͸Primary KeyΛҾ͖௚͢ඞཁ͕ͳ͘ͳΔ • ಡΈࠐΈͷޮ཰Խ • ߋ৽ίετͱͷτϨʔυΦϑ • MySQL΍PostgreSQLͰݴ͏Covering Index / Index Only ScanͷΠϝʔδ GSI͕ࣹӨ͢ΔଐੑͷΦϓγϣϯ

Slide 29

Slide 29 text

ࣹӨͱ͍͏͜ͱ͸ 1:Nͷؔ܎͕͜ͷΑ͏ʹղܾͰ͖Δ

Slide 30

Slide 30 text

• ϝΠϯςʔϒϧͱ͸ผͷPartitionͰ෼ࢄ͞ΕΔ • ߋ৽͸ඇಉظͰ൓ө • ݁Ռ੔߹ੑͬͯେৎ෉ͳͷʁ • RDSͩͬͯRead ReplicaಡΜͩΒ݁Ռ੔߹ੑͰ͢Αʁʁ • GSI΁ͷΫΤϦ͸͍͍ͩͨRead Replicaʹ౤͛ΔΑ͏ͳΫΤϦ GSIʹର͢ΔΫΤϦ͸݁Ռ੔߹ੑ

Slide 31

Slide 31 text

͋ɺ͜ΕRDBͰʢԿճ໨ʁ

Slide 32

Slide 32 text

࢓૊ΈΛ஌Ε͹࣮ʹγϯϓϧ શ෦RDBͰطʹ͋ͬͨ࢓૊ΈͰ͢Αʁʁ

Slide 33

Slide 33 text

☓ DynamoDB͸บ͕͋Δ ○ RDBͷบ͕͋Γ͗͢Δ →γϯϓϧ͗͢ΔDynamoDBͱͷΪϟοϓ͕ڧͯͦ͘͏ײ͡Δ͚ͩ

Slide 34

Slide 34 text

RDBʹચ೴͞ΕͯΔΜ͡Όͳ͍Ͱ͔͢ʁʁ ͱ͍͏͔ͦͷRDBͷ͜ͱ͢ΒͪΌΜͱ෼͔ͬͯ͋͛ͯ·͔ͨ͠ʁ

Slide 35

Slide 35 text

RDBͷ͜ͱ෼͔ͬͯ·͔ͨ͠ʁ ͱ͍͏͔ɺͦͷେ޷͖ͳ

Slide 36

Slide 36 text

DynamoDBͷσʔλϞσϦϯά Ͱ͸ͦΕΛ౿·͑ͯ

Slide 37

Slide 37 text

ͦΕ͸σʔλͷ഑ஔΛܾΊΔ͜ͱ ҟ࿦͸ೝΊΔʂʂ

Slide 38

Slide 38 text

σʔλϞσϦϯάͷߟ͑ํͷϙΠϯτ ഑ஔΛܾΊΔͨΊͷ

Slide 39

Slide 39 text

• εΩʔϚϨεͷҙຯΛߟ͑Δ • List, Map͕࣋ͯΔ͜ͱʁ • ޙ͔ΒଐੑΛγʔϜϨεʹ௥ՃͰ͖Δ͜ͱʁ • A. ͳΜͰ΋ಥͬࠐΊΔ͜ͱ • Partition KeyͰ෼ࢄ • ςʔϒϧΛ෼͚ͳͯ͘΋ෛՙ͸෼ࢄ͢Δ • ۃ࿦͢Δͱ෼͚Δҙຯ͕ͳ͍ɺ෼͚ͳ͍ํ͕ΩϟύγςΟ͕؅ཧ͠΍͍͢ • “ઃܭ͕༏ΕͨΞϓϦέʔγϣϯͰ͸ɺඞཁͳςʔϒϧ͸ 1 ͭͷΈͰ͢” • GSIͷݸ਺੍ݶ(5ݸ)ͳͲ΋͋ΔͷͰۀ຿υϝΠϯຖʹҰͭ͘Β͍ͷؾ࣋ͪ ςʔϒϧͷ෼͚ํ

Slide 40

Slide 40 text

• RDB • ·ͣεΩʔϚΛߟ͑Δ • ਖ਼نԽ • ͦΕʹରͯ͠Ͳ͏ΞΫηε͢Δ͔(SQL)ߟ͑Δ • DynamoDB • εΩʔϚϨε • ඇਖ਼نԽ • ·ͣΞΫηεΛߟ͑ͯɺͦΕʹ߹ΘͤͨσʔλΛ࡞Δ • ΞϓϦέʔγϣϯͱσʔλϞσϧ͸ಉ࣌ʹઃܭ͢Δ RDBͱ͸Ξϓϩʔν͕શ͘ҟͳΔ

Slide 41

Slide 41 text

ACIDτϥϯβΫγϣϯ͕ແ͍ͷ͸Ͳ͏͢Δͷʁ ͱ͸͍͑

Slide 42

Slide 42 text

• ݪࢠੑ (Atomicity) • Ұ؏(੔߹)ੑ (Consistency) • ಠཱੑ (Isolation) • Ӭଓੑ (Durability) ͦ΋ͦ΋ACIDͬͯͳΜ͚ͩͬʁ ͜ΕΒΛ୲อ͢Δ͜ͱ͕໨తͰ͋ͬͯ τϥϯβΫγϣϯγεςϜͰ͋Δඞཁ͸ແ͍

Slide 43

Slide 43 text

• RDB • ਖ਼نԽ͍ͯ͘͠ͱςʔϒϧͷ਺͸جຊతʹ૿͑Δ • ࿦ཧతʹ෼͔Εͨςʔϒϧ(Ϩίʔυ)ؒͷ੔߹ੑΛऔΔ࢓૊Έ͕͋Δ • ϦϨʔγϣϯγοϓ • ֎෦Ωʔ੍໿ • ACIDτϥϯβΫγϣϯ • DynamoDB • ඇਖ਼نԽͯ͠1ͭͷ࣮ମʹ͍ͭͯ1ͭͷΞΠςϜʹऩΊΔ • 1ΞΠςϜͷߋ৽͸ΞτϛοΫ • ৚݅෇͖ॻ͖ࠐΈΛ࢖ָͬͨ؍తϩοΫ ACIDͷ୲อ

Slide 44

Slide 44 text

• PutItem • ΞΠςϜΛ·Δ͝ͱߋ৽͢Δɺແ͔ͬͨΒ࡞Δ • Get -> ॻ͖׵͑ -> Putͩͱ੔߹ੑ่͕ΕΔʢՄೳੑ͕͋Δʣ • UpdateItem • ΞΠςϜΛ෦෼ߋ৽ɺແ͔ͬͨΒ࡞Δ • List, Mapͷ෦෼௥Ճɺ࡟আͳͲ΋Մೳ • ଐੑͷ஋Λར༻ͨ͠ߋ৽΋Մೳʢ͋Δଐੑʹ+1͢Δ౳) • ଐੑͷ஋Λར༻ͨ͠৚݅෇͖ॻ͖ࠐΈ DynamoDBͷߋ৽ૢ࡞

Slide 45

Slide 45 text

ඇਖ਼نԽ͢ΔͱಡΈࠐΈ͕ͭΒ͍ ϑϧεΩϟϯ͢Δ͔͠ͳ͍ શ෦GSIషΔͷʁ

Slide 46

Slide 46 text

CQRSͰ͢Α ͦ͜Ͱ

Slide 47

Slide 47 text

• ॻ͖ࠐΉσʔλͱಡΈࠐΉσʔλ͸ಉ͡Ͱ͋Δඞཁ͸ແ͍ • ݁Ռ੔߹ੑΛड͚ೖΕͯඇಉظͰಡΈࠐΈσʔλΛ࡞ΕΔ • Commandͷ׬ྃΛ΋ͬͯQuery༻σʔλΛ࡞ΕΔ • DynamoDB Streams͕Ϛον͢Δ • CQRSΛૉ௚ʹ࣮૷͢Δͱ·ͣKinesis Streams͚ͩͲ • Materialized View CQRS (Command Query Responsibility Segregation)

Slide 48

Slide 48 text

ARCHITECTURE PATTERN AppSync͕͜Ε͔ΒॏཁͳҐஔΛ઎ΊΔ͸ͣ

Slide 49

Slide 49 text

DATA ATOMICITY MicroserviesతͳΠϝʔδͰ͸͜ͷݪ୯Ґ͕ॏཁ σʔλετΞɾςʔϒϧͷ෼͚ํͰ͸ͳ͍ ލ͕͍ͤͨͳΒMaterialized ViewΛ࡞Δ User Profile User Status Name Email Read/Write Search Read/Write

Slide 50

Slide 50 text

͜ΜͳΠϝʔδ 1ͭͷGSIͰ༷ʑͳݕࡧ͕Ͱ͖Δ

Slide 51

Slide 51 text

• ෳ਺ͷ࣮ମʹލΔτϥϯβΫγϣϯ • DynamoDB୯ମͩͱ͜͜͸Ͳ͏ͯ͠΋ͭΒ͍ • τϥϯβΫγϣϯ෼཭Ϩϕϧͷ࠷ߴ͸Selializable • ͭ·Γ௚ྻԽ • Kinesis StreamsͰ௚ྻԽ • ίέΔ͔΋͠Εͳ͍ͷͰႈ౳ʹ࣮૷͢Δ • ৚݅෇ॻ͖ࠐΈͰطʹߦͬͨॲཧΛSkip͢Δͷ΋ႈ౳ ͱ͸͍͑ݶք͸͋Δ

Slide 52

Slide 52 text

• Kinesis StreamsͰಉ࣌઀ଓ਺Λίϯτϩʔϧ͢Ε͹  ผʹRDS࢖ͬͯ΋Կͷ໰୊΋ͳ͍Θ͚Ͱ • ಉ࣌઀ଓ਺ → Shard਺ͱಉ਺Ͱݻఆ • Lambda͸جຊWarm Start • ͭ·ΓɺENIͷ஗Ԇͱ΄΅ແԑ • σϓϩΠ΍Shard਺ม͑ͨλΠϛϯάͰ͸ى͖͏Δ͚Ͳͦ΋ͦ΋ඇಉظͳͷͰʢ͈́ ͱ͍͏͔

Slide 53

Slide 53 text

ADVANCED TIPS ͕࣌ؒ͋Ε͹͍͔ͭ͘͝঺հ

Slide 54

Slide 54 text

• DynamoDBͷIndex͸Sparse • Indexଐੑ͕ଘࡏ͠ͳ͍৔߹ɺIndexʹͦͷσʔλ͸ؚ·ͳ͍ • ಛఆͷΞΠςϜͷΈʹଘࡏ͢ΔଐੑΛࢦఆ͢Δ͜ͱͰ  Index༰ྔ͕େ͖͘ѹॖͰ͖Δ Sparse Index

Slide 55

Slide 55 text

Sort΋ཁΒͳ͍ GSIʹ͸҉໧ͰPrimary Keyؚ͕·ΕΔ

Slide 56

Slide 56 text

• Α͋͘Δ֊૚ߏ଄Λ࣋ͭσʔλ • ૊৫ߏ଄ • ωετͨ͠ΧςΰϥΠζ • ͳͲͳͲ • ݕࡧΛߟ͑ͳ͚Ε͹֊૚ߏ଄Λ·Δͬͱ1ΞΠςϜʹԡ͠ࠐΉ • Sort KeyͰ֊૚Λදݱ͢Δ Hierarchical data

Slide 57

Slide 57 text

Sort KeyͰલํҰகݕࡧ ͋ɺ͜ΕSQLΞϯνύλʔϯͰʢ͈́ ঎඼ઃܭ͕ࡶͳͷ͸͝Ѫᇷ

Slide 58

Slide 58 text

• Itemʹ”Version”ͷΑ͏ͳଐੑΛ࣋ͨͤΔ • ଐੑͷ஋Λར༻ͨ͠৚݅෇͖ॻ͖ࠐΈ • ConditionExpression=“Version = 1” • UpdateExpression=“ADD Version 1” • Version = 1Ͱͳ͚Ε͹Τϥʔ͕ฦΔ • Version = 1ͳΒ2ʹߋ৽͞ΕΔ όʔδϣχϯάͱָ؍తϩοΫ

Slide 59

Slide 59 text

• ୺తʹݴͬͯόΠϒϧ • ͜ΕΛಡΜͰશͯཧղͰ͖ͨΒҰਓલʂʂ • https://docs.aws.amazon.com/ja_jp/amazondynamodb/latest/ developerguide/best-practices.html DynamoDB Best Practices

Slide 60

Slide 60 text

“DynamoDB ދͷר” [ݕࡧ] http://marcy.hatenablog.com/entry/2018/07/31/213705 ҰԠະ׬ͳͷͰ·ͩॻ͔͘΋

Slide 61

Slide 61 text

“LET’S BECOME THE FRIEND OF RDB” THANKS!! “LET’S BECOME THE FRIEND OF DYNAMODB” “LET’S BECOME THE FRIEND OF SERVERLESS”