우리는 어떻게 샤딩을 했는가?

by RIDI

Slide 1

Slide 1 text

਋ܻח যڌѱ ࢥ٬ਸ ೮חо? 2018֙ 6ਘ 1ੌ ܻ٣झೖ஖ ҅੿౱ ӣ੿Ӕ © RIDI Corp.

Slide 2

Slide 2 text

2 ߊ಴੗ ࣗѐ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? ӣ੿Ӕ ܻ٣ী 16֙ 11ਘ ੑࢎ ࢶܽ ੋఠ֔ Ҋ١೟Ү ࢑সӝמਃਗ (3ѐਘ թ਺ ƃƃ)

Slide 3

Slide 3 text

1. ߓ҃ 2. ࢥ٬ী ؀೧ Ѿ੿ೠ Ѫٜ • ࢥ٘ ః • ࢥ٬ ߑध • ࢥ٘ ഛ੢ ߑध • ੿੸ ؘ੉ఠ 3. Ӓ റ ੉ঠӝ 3 ݾର ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 4

Slide 4 text

4 1. ߓ҃ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 5

Slide 5 text

5 ҳݒݾ۾ ѐࢶ੘সਸ ೞѱ عणפ׮. ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? ӝઓ੄ ࠗ઒೮؍ ӝמ ௒۩࣌ Ѩ࢝द ױӂ֢୹੉ উؽ -> ௒۩࣌ ٜযоࢲ ଼ਸ ࣚࣻ ଺ইঠೣ ҳݒ, ؀ৈ, ੉ਊӂ ೙ఠ হ਺ ୭न ҳݒࣽ ੿۳݅ ૑ਗ

Slide 6

Slide 6 text

6 ҳݒݾ۾ ѐࢶ੘সਸ ೞѱ عणפ׮. ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? “[ױ೯ࠄ, ো੤]۽ աׇࢲࠅ ࣻ ੓ب۾ ஠పҊܻܳ ٜ݅য઱प ࣻ ੓աਃ?” “؀ৈೠ ଼ ޘ਺੉ 200ৈӂ੉ ֈחؘ Ӓ઺ী ਗೞח ଼ ଺ӝо ցޖ ൨٘֎ਃ” “ҳݒݾ۾ ೠӖࣽਵ۽ աৌغѱ ࠗఌ ખ ೧ਃ” “ױ೯ࠄਵ۽ ೠӂ੄ ࠙۝ੋؘ ো੤۽ 20ചө૑੄ ࠙۝੉ۄҊ ஖ݶ ҳݒӂࣻо ցޖ ഛ ৢۄоਗ਼ইਃ.” Ҋёٜ੄ ૑ࣘ੸ੋ ਃ୒

Slide 7

Slide 7 text

7 ҳݒݾ۾ ਃҳࢎ೦ Ѩ࢝ ੿۳ ೙ఠ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 8

Slide 8 text

8 ࣻ ݆਷ ா੉झ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 9

Slide 9 text

9 ҳݒݾ۾ ؘ੉ఠ ҳࢿೞӝ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? ਃҳ ࢎ೦ٜҗ ݆ࣻ਷ ா੉झٜਸ ୊ܻೡ ؘ੉ఠ ೙ਃ

Slide 10

Slide 10 text

10 ҳݒݾ۾ ؘ੉ఠ ҳࢿೞӝ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? ҳݒݾ۾ ؘ੉ఠח ਬ੷ ଼ ప੉࠶ী ઙࣘ੸੉׮.

Slide 11

Slide 11 text

11 ਬ੷ ଼ ప੉࠶੉ ޖट ప੉࠶ੋоਃ? ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? ҳݒݾ۾ ؘ੉ఠ ҳࢿೞӝ

Slide 12

Slide 12 text

12 • ਬ੷о ҳݒೠ ଼੉ ӝ۾غח ప੉࠶ • ਬ੷ ଼ ప੉࠶਷ ࢎղীࢲ о੢ ௼Ҋ ࡅܰѱ טযաח ప੉࠶ • അ੤ ড 3.5র Ѥ • ਬ੷ ଼ ప੉࠶਷ ݆਷ ౱੉ ࢎਊೞח ప੉࠶ • ਬ੷ ଼ ప੉࠶਷ ୶о/ࣻ੿੉ ࠼ߣೞ׮. • ݆਷ ࢎਊ୊ীࢲ ߽ݾਸ ׸׼ ҳݒݾ۾ ؘ੉ఠ ҳࢿೞӝ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 13

Slide 13 text

13 ઙࣘ੸ੋؘ ߽ݾ૑੼੉ݶ… ޙઁо ੓૑..ঋաਃ? ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 14

Slide 14 text

14 ҳݒݾ۾ ؘ੉ఠ ҳࢿೞӝ • ੋझఢझ 1ѐ/ప੉࠶ 1ѐী Write߽ݾ੉ ߊࢤೠ׮. • ؘ੉ఠ Ѥࣻо ૐоೡ ࣻ۾ ప੉࠶ झః݃ ߸҃੉ ࠗ׸झ۞ਕ૓׮. • झః݃ ߸҃ী 1઱ੌ੉࢚ Ѧܽ׮. • ࢜۽਍ ਃҳࢎ೦੉ ࢤ҂ਸٸ ਬোೞѱ ؀਽ೞӝ ൨ٜ׮. • ੉޷ 3রѤਸ ج౵೮Ҋ ૐо୶ࣁܳ ࠌਸٸ ప੉࠶ 1ѐ۽ ҙܻೞӝ ൨ٜ׮. ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 15

Slide 15 text

15 ӝઓ ҳݒݾ۾ਸ ҙܻೞ؍ ౱਷ ੉޷ ѻҊ ੓ח ޙઁ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 16

Slide 16 text

16 ࢎप ҳݒݾ۾ ѐࢶ ݾ಴ ઺ ੺߈਷ ߽ݾ૑੼ ಽযղӝ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 17

Slide 17 text

17 Ӓېࢲ! ੷൞ח ࢥ٬ਸ ೞӝ۽ ೮णפ׮! ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 18

Slide 18 text

18 2. ࢥ٬ী ؀೧ Ѿ੿ೠ Ѫٜ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 19

Slide 19 text

19 ࢥ٘ ః ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 20

Slide 20 text

20 ؘ੉ఠо যڃ ࢥ٘۽ ٜযт૑ ӝળ੉ غח ч ؘ੉ఠܳ Ӑੌೞѱ ࠙ߓೞӝ ਤ೧ࢲח ࢥ٘ ఃо ݒ਋ ݒ਋ ઺ਃೞ׮. ؘ੉ఠо Ӑੌೞѱ ࠙ߓغ૑ ޅೞݶ ౠ੿ ࢥ٘ী ܻࣗझ ࢎਊ۝੉ ށܾ ࣻ ੓׮. ࢥ٘ ః ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 21

Slide 21 text

21 VS ࢥ٘ ః ਬ੷ ID ଼ ID ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? • ਬ੷੄ Ҋਬ ੋؙझ • Integer ఋੑ • ଼੄ Ҋਬ ੋؙझ • String ఋੑ

Slide 22

Slide 22 text

22 ࢲ࠺झ੄ ౠࢿ࢚ ਬ੷ӝ߈੄ ؘ੉ఠ݅ ઓ੤ೠ׮. ਬ੷ ױਤ۽ ޘযࢲ Write೧ঠ ೞח ҃਋о ੓׮. ೠ ਬ੷੄ ؘ੉ఠо ৈ۞ ࢥ٘ী ಌઉ੓ਵݶ ӝמҳഅ ੗୓о ൨ٜ׮. ࢥ٘ ః ਬ੷ ID ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 23

Slide 23 text

23 ࢥ٬ ߑध ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 24

Slide 24 text

24 ࢥ٬ ߑध • ࢥ٘ ః੄ ߧਤ۽ ࠙ߓ • ࢥ٘ ഛ੢੉ औ׮. • ex) 4001ߣ ~ 5000ߣ Shard D • ܻࣗझ ࢎਊ۝੉ ౠ੿ ࢥ٘ী ށܾ оמࢿ ઓ੤ • (ࢥ٘ ః % ࢥ٘ ࣻ)੄ чਵ۽ ࠙ߓ • Range basedী ࠺೧ ܻࣗझ ࢎਊ۝੉ Ӑੌ • ࢥ٘о ୶оغݶ(ࢥ٘ ః % ࢥ٘ ࣻ)੄ ч੉ ߸ ҃غӝ ٸޙী ੹୓ ؘ੉ఠܳ ੤࠙ߓ ೧ঠೠ׮. VS Range based Shard C Shard A Shard B 1~2000 2000 ~ 3000 3000 ~ 4000 Modulus Based 1 C B A C B A C B A C B A B A 2 3 1 2 3 1 2 3 1 2 3 1 2 ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 25

Slide 25 text

25 ܻࣗझ оਊࢿҗ ؘ੉ఠ ࠙ߓ۝੉ ࠺Ү੸ Ӑੌ೮ਵݶ જѷ׮. Modulus based ࢥ٬ ߑध ࢥ٘ ഛ੢द ؘ੉ఠ ੤࠙ߓо ୭ࣗച عਵݶ જѷ׮. Range based ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 26

Slide 26 text

26 ࢥ٬ ߑध ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? ف о૑ܳ زदী ݅઒ೡ ߑߨ਷ হਸө? ܻࣗझ оਊࢿҗ ؘ੉ఠ ࠙ߓ۝੉ ࠺Ү੸ Ӑੌ೮ਵݶ જѷ׮. Modulus based ࢥ٘ ഛ੢द ؘ੉ఠ ੤࠙ߓо ୭ࣗച عਵݶ જѷ׮. Range based

Slide 27

Slide 27 text

27 ؀ࠗ࠙੄ Key-Value झషযীࢲ ࢎਊೞח ࠙࢑ ߑध ex) Couchbase / Cassandra ١ ֢٘ ࣻо ߸҃ؼٸ ୶о ഑਷ ࢏ઁغח ֢٘੄ ؀೧ࢲ݅ ؘ੉ఠ ߸҃੉ ੌযդ׮. ࢥ٬ ߑध Consistent Hashing(ੌҙػ ೧य) ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 28

Slide 28 text

28 Consistent Hashing(ੌҙػ ೧य) 3ѐ੄ ֢٘о ઓ੤ೞח ࢚ട Shard A 2 4 8 9 Shard B 1 6 11 12 Shard C 3 5 7 10 ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 29

Slide 29 text

29 Consistent Hashing(ੌҙػ ೧य) Shard A 2 4 9 Shard B 1 6 Shard C 3 10 Shard D 8 11 7 5 12 ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? 1ѐо ֢٘о ୶оغݶ п ֢٘ীࢲ ੌࠗ ؘ੉ఠо ࢜۽਍ ֢٘۽ ੉ز

Slide 30

Slide 30 text

30 Consistent Hashing(ੌҙػ ೧य) ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? 1ѐ੄ ֢٘о ࢏ઁغݶ ࢏ઁػ ֢٘੄ ؘ੉ఠܳ աׇࢲ п ֢٘۽ ੹׳ Shard A 2 4 8 9 Shard B 1 6 11 12 Shard C 3 5 7 10 Shard D

Slide 31

Slide 31 text

31 ਗܻܳ ೠߣ ঌইࠇद׮. Consistent Hashing(ੌҙػ ೧य) ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 32

Slide 32 text

32 Consistent Hashing(ੌҙػ ೧य) 3ѐ੄ ֢٘ܳ ೧द೧ࢲ ೧द ݂ਸ ݅ٚ׮. 1 7 4 ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 33

Slide 33 text

33 Consistent Hashing(ੌҙػ ೧य) ؘ੉ఠܳ hash ೠ׮. ӏ஗ী ٮۄ пп ֢٘ী ٜযр׮. 1 7 4 2 3 4 6 5 8 9 ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 34

Slide 34 text

34 Consistent Hashing(ੌҙػ ೧य) ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? Ӓ۞ա അ੤ ݽण਷ ֢٘੄ ࠙ನо ୥୥ೞ૑ ঋ׮. ౵ۆ࢝ ֢٘ ࢏ઁद ࠙ഘ࢝ ֢٘ী ࠗ׸੉ ழ૓׮. 1 4 2 3 4 6 5 8 9

Slide 35

Slide 35 text

35 Consistent Hashing(ੌҙػ ೧य) ֢٘ܳ ৈ۞ ѐ੄ о࢚ ֢٘۽ ҳࢿ ֢٘੄ ࠙ನо ୥୥೧૑ݶ ؘ੉ఠب ബҗ੸ਵ۽ ࠙ߓػ׮. ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? 1 4 7 2 3 5 6 8 9 1 1 1

Slide 36

Slide 36 text

36 ܻࣗझ оਊࢿҗ ؘ੉ఠ ࠙ߓ۝੉ ࠺Ү੸ Ӑੌ೮ਵݶ જѷ׮. ࢥ٘ ഛ੢द ؘ੉ఠ ੤࠙ߓо ୭ࣗച عਵݶ જѷ׮. ࢥ٬ ߑध Consistent Hashing ف о૑ ӝળ ݽف ݅઒ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 37

Slide 37 text

37 Consistent Hashing ؘ੉ఠ ࠙࢑ పझ౟ ਬ੷੄ ࠙ನо о੢ Ӑੌೠ Ѫ ঌҊ્ܻ੄ ࣘبо ࡅܲ Ѫ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 38

Slide 38 text

38 Consistent Hashing ؘ੉ఠ ࠙࢑ పझ౟ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? Replica ஠਍౟ ߹ ਬ੷ ࠙ನ ࠺Ү ߸ز ҅ࣻ (಴ળಞର / ಣӐ) md5 - replica 1024 2.4% md5 - replica 512 4.3% blake2b - replica 1024 2.6% blake2b - replica 512 4.6% xxhash - replica 1024 3.2% xxhash - replica 512 5.9%

Slide 39

Slide 39 text

39 Consistent Hashing ؘ੉ఠ ࠙࢑ పझ౟ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? о࢚ ֢٘ ࣻܳ 1024۽ ೞݶ ೧द ݂੄ ௼ӝо ցޖ ழ૓׮. ௾ ର੉о ੓ח Ѫ਷ ইפӝ ٸޙী 512ѐ۽ Ѿ੿೮׮. ࠙ನח md5о ؊ જ૑݅ blake2b৬੄ ର੉о ௼૑ ঋҊ blake2bо md5ࠁ׮ ࡅܲ ঌҊ્ܻ੉ӝ ٸޙী blake2b۽ Ѿ੿೮׮.

Slide 40

Slide 40 text

40 Consistent Hashing ؘ੉ఠ ࠙࢑ పझ౟ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? blake2b, о࢚ ֢٘ࣻ 512ѐ۽ Ѿ੿!

Slide 41

Slide 41 text

41 ࢥ٘ ഛ੢ ߑউ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 42

Slide 42 text

42 ࢥ٘ ഛ੢ ߑউ ࢥ٬ਸ ೞח ੉ਬח ࠗೞ࠙࢑ਸ ਤ೧ࢲ׮. Ӓ۞ա ࢥ٬ਸ ೧ب ౟ې೗੉ טযաҊ ೠ ࢥ٘ ׼ ؘ੉ఠо ழ૑ݶ ڙэ੉ ࠗೞо ߊࢤೞחؘ ੉ٸ ࢥ٘੄ ࣻܳ ט۰઻ঠ ೠ׮. ೞ૑݅, ࢥ٘ܳ טܻח Ѥ ࠺ਊ੉ ௾ ੘স੉ӝ ٸޙী ޷ܻ Ӓ ߑউਸ ࢤп೧ࠅ ೙ਃо ੓׮. ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 43

Slide 43 text

43 ࢥ٘ ഛ੢ ߑউ ఋࢎ ࢎ۹: Kakao 1. ؘ੉ఠܳ ੤࠙ߓ ೞӝ ਤ೧ Table Crawler৬ Replication Protocolਸ ੉ਊೠ ਕழ ҳഅ 2. ف о૑ ਕழܳ ڸਕ ࢜۽ ڸਕك DB Instanceী ؘ੉ఠܳ Syncೣ 3. ؘ੉ఠ Ѩૐ റ Live • ࠶۽Ӓ Ӗ ଵҊ - ղࠗ Ѩૐ ױ҅ۄҊ ݺदغয੓׮. ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 44

Slide 44 text

44 ࢥ٘ ഛ੢ ߑউ ఋࢎ ࢎ۹: Tumblr ੉۠ Databaseҳࢿ੉ ੓णפ׮. master blog 1-1000 slave blog 1-1000 slave blog 1-1000 ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 45

Slide 45 text

45 ࢥ٘ ഛ੢ ߑউ ఋࢎ ࢎ۹: Tumblr ࢲ࠺झী ైੑೞ૑ ঋ਷ ࢜۽਍ ठۨ੉࠳ ୶оೞҊ ؘ੉ఠܳ ੺߈ਵ۽ ա׃ master blog 1-1000 slave blog 1-1000 slave blog 1-1000 slave blog 501- 1000 slave blog 1-500 slave blog 1-500 slave blog 1-500 slave blog 501- 1000 slave blog 501- 1000 ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 46

Slide 46 text

46 ࢥ٘ ഛ੢ ߑউ ఋࢎ ࢎ۹: Tumblr ݃झఠ۽ о؍ Read/Writeܳ ୶оػ Slave۽ ੉ز master blog 1-1000 slave blog 1-1000 slave blog 1-1000 slave blog 501- 1000 slave blog 1-500 slave blog 1-500 slave blog 1-500 slave blog 501- 1000 slave blog 501- 1000 R/W R/W ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 47

Slide 47 text

47 ࢥ٘ ഛ੢ ߑউ ఋࢎ ࢎ۹: Tumblr ӝઓ ݃झఠ৬੄ ܻ೒ܻா੉࣌ਸ ઁѢೞҊ ؘ੉ఠ ࠁ੿ master2 blog 501- 1000 master1 blog 1-500 slave blog 1-500 slave blog 1-500 slave blog 501- 1000 slave blog 501- 1000 Master blog 1-1000 slave blog 1-1000 slave blog 1-1000 X ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 48

Slide 48 text

48 ࢲ࠺झী ైੑೞ૑ ঋ਷ ठۨ੉࠳ܳ ୶оೠ റ ؘ੉ఠܳ աׇࢲ ੷੢ೠ׮. Tumblr ݈۽ח ޖ઺ױ ഛ੢੉ۄҊ ೠ׮. ࢥ٘ ഛ੢ ߑউ ఋࢎ ࢎ۹: Tumblr ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 49

Slide 49 text

49 ࢥ٘ ഛ੢ ߑউ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? ഛ੢द ௏٘ܳ ੉ਊೠ ࠙ߓ(ਕழ) ഑਷ ؘ੉ఠ ࠁ੿হ੉ ࢥ٘੄ ഛ੢ࢿ/ؘ੉ఠ੄ न܉ࢿਸ ࠁ੢ೞҊ र׮.

Slide 50

Slide 50 text

50 ഛ੢द ௏٘ܳ ੉ਊೠ ࠙ߓ(ਕழ) ഑਷ ؘ੉ఠ ࠁ੿হ੉ ࢥ٘੄ ഛ੢ࢿ/ؘ੉ఠ੄ न܉ࢿਸ ࠁ੢ೞҊ र׮. ࢥ٘ ഛ੢ ߑউ ੋ೐ۄܳ ా೧ࢲ ࢥ٘ܳ ҙܻ೧ࠁחѤ যڄө? ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 51

Slide 51 text

51 ੋझఢझী MySQL੄ Databaseܳ ୽࠙൤ ٜ݅যفҊ ܻࣗझо ࠗ઒ೡٸ ੋझఢझܳ ഛ੢ೞח ߑध Databaseܳ Logical DB, ੋझఢझܳ Physical DBۄҊ ੉ܴ ࠢ৓׮. ࢥ٘ ഛ੢ ߑউ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 52

Slide 52 text

52 ࢥ٘ ഛ੢ ߑউ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 53

Slide 53 text

53 ࢥ٘ ഛ੢ ߑউ Instance B Instance A WAS Shard C Shard B Shard D Event Worker Shard A READ Write Replication ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 54

Slide 54 text

54 ࢥ٘ ഛ੢ ߑউ Instance B Instance A WAS Shard C Shard B Shard D Event Worker Instance A` Instance B` Shard A READ Write Replication ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 55

Slide 55 text

55 ࢥ٘ ഛ੢ ߑউ Instance B Instance A WAS Shard C Shard B Shard D Event Worker Instance A` Instance B` Shard A Shard B Shard D READ Write Replication ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 56

Slide 56 text

56 ࢥ٘ ഛ੢ ߑউ Instance B Instance A WAS Shard C Shard B Shard D Event Worker Instance A` Instance B` Shard A Shard B Shard D READ Write Replication ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 57

Slide 57 text

57 ࢥ٘ ഛ੢ ߑউ Instance B Instance A WAS Shard C Shard B Shard D Event Worker Instance A` Instance B` Shard A Shard B Shard D READ Write Replication ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 58

Slide 58 text

58 ࢥ٘ ഛ੢ ߑউ Instance B Instance A WAS Shard C Shard B Shard D Event Worker Instance A` Instance B` Shard A Shard B Shard D READ Write Replication ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 59

Slide 59 text

59 ࢥ٘ ഛ੢ ߑউ Instance B Instance A WAS Shard C Event Worker Instance A` Instance B` Shard A Shard B Shard D READ Write Replication ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 60

Slide 60 text

60 ࢥ٘ ഛ੢ ߑউ ୭؀۽ ഛ੢ೡ ࣻ ੓ח Physical DBࣻо ੿೧ઉ ੓ח ױ੼੉ ੓׮. Logical DB੄ ࣻ ੗୓ܳ ט۰ঠ ೞחؘ ੉ ҃਋ ؘ੉ఠܳ ੤࠙ߓ ೧ঠೠ׮. ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 61

Slide 61 text

61 ࢥ٘ ഛ੢ ߑউ ೞ૑݅ ҡଳ׮. Consistent Hash ߑधਵ۽ ؘ੉ఠܳ ࠙ߓೞӝ ٸޙী ࢚؀੸ਵ۽ ࠙ߓ/Ѩૐ੄ դ੉بо ծ׮. ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 62

Slide 62 text

62 ੿੸ ؘ੉ఠ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 63

Slide 63 text

63 ੿੸ ؘ੉ఠ ҳݒݾ۾ ਃҳࢎ೦ਸ ҳഅೞӝ ਤ೧ࢲח ଼ ؘ੉ఠ৬ JOIN੉ ೙ࣻ But, ׮ܲ DBী ઓ੤ೞח ؘ੉ఠ৬ JOIN੉ ࠛоמ ݽٚ ࢥ٘ী ҕాਵ۽ ઓ੤ೞח ੿੸ ؘ੉ఠ ప੉࠶੉ ೙ਃೞ׮. ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 64

Slide 64 text

64 ੿੸ ؘ੉ఠ Shard A Book Item Source Book Item Shard B Book Worker ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? ਗࠄਸ فҊ ࢥ٘ী ࠂઁೞח ѐ֛ (Ӓր Replication…)

Slide 65

Slide 65 text

65 ੿੸ ؘ੉ఠ ੿੸ ؘ੉ఠܳ ࢥ٘ী ನೣೣਵ۽ॄ ۽૒ ഑਷ ௪ܻܳ ੘ࢿೡٸ ਬ੷ױਤ۽ ࢤпೞݶ ױੌ DB୊ۢ ࢎਊೡ ࣻ ੓਺ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 66

Slide 66 text

66 ୨ ੿ܻ • ࢥ٘ ః • ਬ੷ ID • ࢥ٬ ߑध • Consistent Hashܳ ੉ਊೠ ࠙ߓ • ࢥ٘ ഛ੢ ߑध • ୽࠙ೠ Logical DBࢤࢿ, Physical DB݅ ഛ੢ • ੿੸ ؘ੉ఠ • Sourceܳ فҊ п ࢥ٘ী ࠂઁೞח ߑध ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 67

Slide 67 text

67 3. Ӓ റ ੉ঠӝ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

Slide 68

Slide 68 text

68 Ѿ੿ೠ ੿଼ٜਸ ҳഅೠ ۄ੉࠳۞ܻ ੘ࢿ https://github.com/ridi/django-shard-library ৻Ҵੋ੉ झఋ 4ѐ ନযષ ƃƃƃ ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? ই૒ ݆੉ ࠗ઒ೠ ஘ҳੑפ׮. ৈ۞࠙੄ ݆਷ ҙब ߄ۉפ׮ :)

Slide 69

Slide 69 text

69 ਋ܻח যڌѱ ࢥ٬ਸ ೮חо? Terraformਸ ాೠ RDSੋ೐ۄ ҙܻ ୶റ ҅ദ ࢥ٘ ഛ੢ ੗زച ࢥ٘ ۄ੉࠳۞ܻ ӝמ ؊ ۄ੉࠳۞ܻ׹ѱ ъച

Slide 70

Slide 70 text

70 Q & A

Slide 71

Slide 71 text

71 ੿੸ ؘ੉ఠ ࠙ߓೡٸ ৵ Replicationਵ۽ উ೮աਃ? MySQL਷ Tableױਤ۽ Replication غਗ਼ইਃ? Q & A Question 1

Slide 72

Slide 72 text

72 Q & A Answer 1 ੿੸ ؘ੉ఠܳ п ࢥ٘ী ֍য઱۰ݶ Master-Master Replicationਵ۽ ҳ ࢿ೧ঠ೤פ׮. അ੤ RDS੄ ҃਋ Aurora৻ীח Master-Master Replicationਸ ૑ਗೞ૑ ঋইࢲ ਕழܳ ా೧ ؘ੉ఠܳ ࠙ߓೞҊ ੓णפ׮.

Slide 73

Slide 73 text

73 ৵ Aurora উ 㛣աਃ? Master-Master Replicationب ૑ਗೞҊ  ࢿמب ؊ ੜ աয়ਗ਼ইਃ? Q & A Question 2

Slide 74

Slide 74 text

74 Q & A Answer 2 ѐߊೞݶࢲ ௪ܻౚ׬੉ ખ ݆੉ ٜযщणפ׮. Auroraо MySQLഐജ੉ۄ Ҋ ೞחؘ ੋఠಕ੉झ݅ ഐജੋ૑ Storage engineө૑ ഐജੋ૑ ഛੋ೧ࠅ ೙ਃо ੓যࢲ بੑೞ૑ ޅ೮णפ׮. ҅ദ਷ о૑Ҋ ੓णפ׮ :)

Slide 75

Slide 75 text

75 ੢গद ؘ੉ఠ ࠂҳ যڌѱ ೞաਃ? Q & A Question 3

Slide 76

Slide 76 text

76 Q & A Answer 3 ੢গदীח RDS੄ Multi AZܳ ా೧ࢲ ࠂҳೡ ҅ദੑפ׮.

Slide 77

Slide 77 text

77 ࢥ٘ ഛ੢द ୭؀ 5࠙੿ب ѦܾѢۄҊ ݈ॹೞ࣑חؘ  5࠙زউ ҳݒݾ۾ ߈৔੉ উغח Ѣ ইצоਃ? Q & A Question 4

Slide 78

Slide 78 text

78 Q & A Answer 4 ୭ୡ ਃҳࢎ೦ ੿ܻೡٸ ইޖܻ ןযب ୭؀ 5࠙উীח ҳݒݾ۾ী ߈৔ػ ׮Ҋ ೮חؘ, ࢥ٘ ഛ੢ө૑ Ҋ۰ೠ 5࠙ੑפ׮. Ӓېب 5࠙੉ ૣ਷ दр਷ ই פۄࢲ ഛ੢ਸ ೞѱ غݶ غب۾ ੼Ѩ ഑਷ ࢜߷दр ੉ਊ೧ࢲ ૓೯ೡ ҅ദ ੑפ׮.

Slide 79

Slide 79 text

79 ࢥ٘ ഛ੢ೡٸ Writeܳ ઁѢೞݶ  ҳݒ೧ب ؘ੉ఠ উऺ੉חѢ ইצоਃ? Q & A Question 5

Slide 80

Slide 80 text

80 Q & A Answer 5 ѐߊೡٸ CQRSಁఢਵ۽ ҳഅਸ ೮णפ׮. ੉੹ী Ӓܿীࢲ Writeܳ ઁѢ ೞחѤ ௸੄ ؘ੉ఠܳ ࢥ٘۽ ॳח Ѫਸ ઁѢೞח Ѣۄࢲ ࢥ٘ ഛ੢઺ীب ҳݒೠ ӝ۾਷ ௸ী ӝ۾ؾפ׮. ੉যࢲ ࢥ٘ ഛ੢੉ ৮ܐغݶ ׮द ୊ܻغ ӝ ٸޙী Write־ۅ਷ হणפ׮.

Slide 81

Slide 81 text

81 ଵҊ ޙࢲ • ఋࢎࢎ۹ • http://tech.kakao.com/2016/07/01/adt-mysql-shard-rebalancing/ • https://pegasuskim.wordpress.com/2015/12/17/tumblr-sharding/ • http://assets.en.oreilly.com/1/event/74/ Massively%20Sharded%20MySQL%20at%20Tumblr%20Presentation.pdf • http://highscalability.com/blog/2012/2/13/tumblr-architecture-15-billion-page-views-a-month-and-harder.html • https://engineering.tumblr.com/post/7658008285/efﬁciently-copying-ﬁles-to-multiple-destinations • Karger Consistent Hashing • https://charsyam.wordpress.com/2016/10/02/%EC%9E%85-%EA%B0%9C%EB%B0%9C-consistent- hashing-%EC%97%90-%EB%8C%80%ED%95%9C-%EA%B8%B0%EC%B4%88/ • https://www.akamai.com/es/es/multimedia/documents/technical-publication/consistent-hashing-and-random- trees-distributed-caching-protocols-for-relieving-hot-spots-on-the-world-wide-web-technical-publication.pdf • Jump Consistent Hashing • https://arxiv.org/pdf/1406.2294.pdf

Slide 82

Slide 82 text

82 ੉޷૑ ୹୊ • https://kr.seaicons.com/ؘ੉ఠ߬੉झ-ই੉௑-3/ • https://kr.seaicons.com/ই੉௑-ࢎਊ੗-֢ز੗/ • https://kr.seaicons.com/ਢ-ࢲߡ-ই੉௑/

Slide 83

Slide 83 text

83 THANK YOU хࢎ೤פ׮