Upgrade to Pro — share decks privately, control downloads, hide ads and more …

우리는 어떻게 샤딩을 했는가?

RIDI
June 01, 2018

우리는 어떻게 샤딩을 했는가?

RIDI

June 01, 2018
Tweet

More Decks by RIDI

Other Decks in Technology

Transcript

  1. ਋ܻח যڌѱ
    ࢥ٬ਸ ೮חо?
    2018֙ 6ਘ 1ੌ ܻ٣झೖ஖ ҅੿౱ ӣ੿Ӕ
    © RIDI Corp.

    View full-size slide

  2. 2
    ߊ಴੗ ࣗѐ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    ӣ੿Ӕ
    ܻ٣ী 16֙ 11ਘ ੑࢎ
    ࢶܽ ੋఠ֔ Ҋ١೟Ү
    ࢑সӝמਃਗ (3ѐਘ թ਺ ƃƃ)

    View full-size slide

  3. 1. ߓ҃
    2. ࢥ٬ী ؀೧ Ѿ੿ೠ Ѫٜ
    • ࢥ٘ ః
    • ࢥ٬ ߑध
    • ࢥ٘ ഛ੢ ߑध
    • ੿੸ ؘ੉ఠ
    3. Ӓ റ ੉ঠӝ
    3
    ݾର
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  4. 4
    1. ߓ҃
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  5. 5
    ҳݒݾ۾ ѐࢶ੘সਸ ೞѱ عणפ׮.
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    ӝઓ੄ ࠗ઒೮؍ ӝמ
    ௒۩࣌ Ѩ࢝द ױӂ֢୹੉ উؽ -> ௒۩࣌ ٜযоࢲ ଼ਸ ࣚࣻ ଺ইঠೣ
    ҳݒ, ؀ৈ, ੉ਊӂ ೙ఠ হ਺
    ୭न ҳݒࣽ ੿۳݅ ૑ਗ

    View full-size slide

  6. 6
    ҳݒݾ۾ ѐࢶ੘সਸ ೞѱ عणפ׮.
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    “[ױ೯ࠄ, ো੤]۽ աׇࢲࠅ ࣻ ੓ب۾ ஠పҊܻܳ ٜ݅য઱प ࣻ ੓աਃ?”
    “؀ৈೠ ଼ ޘ਺੉ 200ৈӂ੉ ֈחؘ Ӓ઺ী ਗೞח ଼ ଺ӝо ցޖ ൨٘֎ਃ”
    “ҳݒݾ۾ ೠӖࣽਵ۽ աৌغѱ ࠗఌ ખ ೧ਃ”
    “ױ೯ࠄਵ۽ ೠӂ੄ ࠙۝ੋؘ ো੤۽ 20ചө૑੄ ࠙۝੉ۄҊ ஖ݶ
    ҳݒӂࣻо ցޖ ഛ ৢۄоਗ਼ইਃ.”
    Ҋёٜ੄ ૑ࣘ੸ੋ ਃ୒

    View full-size slide

  7. 7
    ҳݒݾ۾ ਃҳࢎ೦
    Ѩ࢝
    ੿۳
    ೙ఠ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  8. 8
    ࣻ ݆਷ ா੉झ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  9. 9
    ҳݒݾ۾ ؘ੉ఠ ҳࢿೞӝ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    ਃҳ ࢎ೦ٜҗ ݆ࣻ਷ ா੉झٜਸ ୊ܻೡ ؘ੉ఠ ೙ਃ

    View full-size slide

  10. 10
    ҳݒݾ۾ ؘ੉ఠ ҳࢿೞӝ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    ҳݒݾ۾ ؘ੉ఠח ਬ੷ ଼ ప੉࠶ী ઙࣘ੸੉׮.

    View full-size slide

  11. 11
    ਬ੷ ଼ ప੉࠶੉ ޖट ప੉࠶ੋоਃ?
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    ҳݒݾ۾ ؘ੉ఠ ҳࢿೞӝ

    View full-size slide

  12. 12
    • ਬ੷о ҳݒೠ ଼੉ ӝ۾غח ప੉࠶
    • ਬ੷ ଼ ప੉࠶਷ ࢎղীࢲ о੢ ௼Ҋ ࡅܰѱ טযաח ప੉࠶
    • അ੤ ড 3.5র Ѥ
    • ਬ੷ ଼ ప੉࠶਷ ݆਷ ౱੉ ࢎਊೞח ప੉࠶
    • ਬ੷ ଼ ప੉࠶਷ ୶о/ࣻ੿੉ ࠼ߣೞ׮.
    • ݆਷ ࢎਊ୊ীࢲ ߽ݾਸ ׸׼
    ҳݒݾ۾ ؘ੉ఠ ҳࢿೞӝ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  13. 13
    ઙࣘ੸ੋؘ ߽ݾ૑੼੉ݶ… ޙઁо ੓૑..ঋաਃ?
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  14. 14
    ҳݒݾ۾ ؘ੉ఠ ҳࢿೞӝ
    • ੋझఢझ 1ѐ/ప੉࠶ 1ѐী Write߽ݾ੉ ߊࢤೠ׮.
    • ؘ੉ఠ Ѥࣻо ૐоೡ ࣻ۾ ప੉࠶ झః݃ ߸҃੉ ࠗ׸झ۞ਕ૓׮.
    • झః݃ ߸҃ী 1઱ੌ੉࢚ Ѧܽ׮.
    • ࢜۽਍ ਃҳࢎ೦੉ ࢤ҂ਸٸ ਬোೞѱ ؀਽ೞӝ ൨ٜ׮.
    • ੉޷ 3রѤਸ ج౵೮Ҋ ૐо୶ࣁܳ ࠌਸٸ ప੉࠶ 1ѐ۽ ҙܻೞӝ ൨ٜ׮.
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  15. 15
    ӝઓ ҳݒݾ۾ਸ ҙܻೞ؍ ౱਷ ੉޷ ѻҊ ੓ח ޙઁ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  16. 16
    ࢎप ҳݒݾ۾ ѐࢶ ݾ಴ ઺ ੺߈਷ ߽ݾ૑੼ ಽযղӝ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  17. 17
    Ӓېࢲ! ੷൞ח ࢥ٬ਸ ೞӝ۽ ೮णפ׮!
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  18. 18
    2. ࢥ٬ী ؀೧ Ѿ੿ೠ Ѫٜ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  19. 19
    ࢥ٘ ః
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  20. 20
    ؘ੉ఠо যڃ ࢥ٘۽ ٜযт૑ ӝળ੉ غח ч
    ؘ੉ఠܳ Ӑੌೞѱ ࠙ߓೞӝ ਤ೧ࢲח ࢥ٘ ఃо ݒ਋ ݒ਋ ઺ਃೞ׮.
    ؘ੉ఠо Ӑੌೞѱ ࠙ߓغ૑ ޅೞݶ ౠ੿ ࢥ٘ী ܻࣗझ ࢎਊ۝੉ ށܾ ࣻ ੓׮.
    ࢥ٘ ః
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  21. 21
    VS
    ࢥ٘ ః
    ਬ੷ ID ଼ ID
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    • ਬ੷੄ Ҋਬ ੋؙझ
    • Integer ఋੑ
    • ଼੄ Ҋਬ ੋؙझ
    • String ఋੑ

    View full-size slide

  22. 22
    ࢲ࠺झ੄ ౠࢿ࢚ ਬ੷ӝ߈੄ ؘ੉ఠ݅ ઓ੤ೠ׮.
    ਬ੷ ױਤ۽ ޘযࢲ Write೧ঠ ೞח ҃਋о ੓׮.
    ೠ ਬ੷੄ ؘ੉ఠо ৈ۞ ࢥ٘ী ಌઉ੓ਵݶ ӝמҳഅ ੗୓о ൨ٜ׮.
    ࢥ٘ ః
    ਬ੷ ID
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  23. 23
    ࢥ٬ ߑध
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  24. 24
    ࢥ٬ ߑध
    • ࢥ٘ ః੄ ߧਤ۽ ࠙ߓ
    • ࢥ٘ ഛ੢੉ औ׮.
    • ex) 4001ߣ ~ 5000ߣ Shard D
    • ܻࣗझ ࢎਊ۝੉ ౠ੿ ࢥ٘ী ށܾ оמࢿ ઓ੤
    • (ࢥ٘ ః % ࢥ٘ ࣻ)੄ чਵ۽ ࠙ߓ
    • Range basedী ࠺೧ ܻࣗझ ࢎਊ۝੉ Ӑੌ
    • ࢥ٘о ୶оغݶ(ࢥ٘ ః % ࢥ٘ ࣻ)੄ ч੉ ߸
    ҃غӝ ٸޙী ੹୓ ؘ੉ఠܳ ੤࠙ߓ ೧ঠೠ׮.
    VS
    Range based
    Shard C
    Shard A Shard B
    1~2000 2000 ~ 3000 3000 ~ 4000
    Modulus Based
    1
    C
    B
    A C
    B
    A C
    B
    A C
    B
    A B
    A
    2 3 1 2 3 1 2 3 1 2 3 1 2
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  25. 25
    ܻࣗझ оਊࢿҗ ؘ੉ఠ ࠙ߓ۝੉ ࠺Ү੸ Ӑੌ೮ਵݶ જѷ׮.
    Modulus based
    ࢥ٬ ߑध
    ࢥ٘ ഛ੢द ؘ੉ఠ ੤࠙ߓо ୭ࣗച عਵݶ જѷ׮.
    Range based
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  26. 26
    ࢥ٬ ߑध
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    ف о૑ܳ زदী ݅઒ೡ ߑߨ਷ হਸө?
    ܻࣗझ оਊࢿҗ ؘ੉ఠ ࠙ߓ۝੉ ࠺Ү੸ Ӑੌ೮ਵݶ જѷ׮.
    Modulus based
    ࢥ٘ ഛ੢द ؘ੉ఠ ੤࠙ߓо ୭ࣗച عਵݶ જѷ׮.
    Range based

    View full-size slide

  27. 27
    ؀ࠗ࠙੄ Key-Value झషযীࢲ ࢎਊೞח ࠙࢑ ߑध
    ex) Couchbase / Cassandra ١
    ֢٘ ࣻо ߸҃ؼٸ ୶о ഑਷ ࢏ઁغח ֢٘੄ ؀೧ࢲ݅ ؘ੉ఠ ߸҃੉ ੌযդ׮.
    ࢥ٬ ߑध
    Consistent Hashing(ੌҙػ ೧य)
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  28. 28
    Consistent Hashing(ੌҙػ ೧य)
    3ѐ੄ ֢٘о ઓ੤ೞח ࢚ട
    Shard A
    2 4
    8 9
    Shard B
    1 6
    11 12
    Shard C
    3 5
    7 10
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  29. 29
    Consistent Hashing(ੌҙػ ೧य)
    Shard A
    2 4
    9
    Shard B
    1 6
    Shard C
    3
    10
    Shard D
    8
    11
    7
    5
    12
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    1ѐо ֢٘о ୶оغݶ п ֢٘ীࢲ ੌࠗ ؘ੉ఠо ࢜۽਍ ֢٘۽ ੉ز

    View full-size slide

  30. 30
    Consistent Hashing(ੌҙػ ೧य)
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    1ѐ੄ ֢٘о ࢏ઁغݶ ࢏ઁػ ֢٘੄ ؘ੉ఠܳ աׇࢲ п ֢٘۽ ੹׳
    Shard A
    2 4
    8 9
    Shard B
    1 6
    11 12
    Shard C
    3 5
    7 10
    Shard D

    View full-size slide

  31. 31
    ਗܻܳ ೠߣ ঌইࠇद׮.
    Consistent Hashing(ੌҙػ ೧य)
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  32. 32
    Consistent Hashing(ੌҙػ ೧य)
    3ѐ੄ ֢٘ܳ ೧द೧ࢲ
    ೧द ݂ਸ ݅ٚ׮.
    1
    7
    4
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  33. 33
    Consistent Hashing(ੌҙػ ೧य)
    ؘ੉ఠܳ hash ೠ׮.
    ӏ஗ী ٮۄ пп ֢٘ী ٜযр׮.
    1
    7
    4
    2
    3
    4
    6
    5
    8
    9
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  34. 34
    Consistent Hashing(ੌҙػ ೧य)
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    Ӓ۞ա അ੤ ݽण਷ ֢٘੄ ࠙ನо ୥୥ೞ૑ ঋ׮.
    ౵ۆ࢝ ֢٘ ࢏ઁद ࠙ഘ࢝ ֢٘ী ࠗ׸੉ ழ૓׮.
    1
    4
    2
    3
    4
    6
    5
    8
    9

    View full-size slide

  35. 35
    Consistent Hashing(ੌҙػ ೧य)
    ֢٘ܳ ৈ۞ ѐ੄ о࢚ ֢٘۽ ҳࢿ
    ֢٘੄ ࠙ನо ୥୥೧૑ݶ ؘ੉ఠب ബҗ੸ਵ۽ ࠙ߓػ׮.
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    1
    4
    7
    2
    3
    5
    6
    8
    9
    1
    1
    1

    View full-size slide

  36. 36
    ܻࣗझ оਊࢿҗ ؘ੉ఠ ࠙ߓ۝੉ ࠺Ү੸ Ӑੌ೮ਵݶ જѷ׮.
    ࢥ٘ ഛ੢द ؘ੉ఠ ੤࠙ߓо ୭ࣗച عਵݶ જѷ׮.
    ࢥ٬ ߑध
    Consistent Hashing
    ف о૑ ӝળ ݽف ݅઒
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  37. 37
    Consistent Hashing ؘ੉ఠ ࠙࢑ పझ౟
    ਬ੷੄ ࠙ನо о੢ Ӑੌೠ Ѫ
    ঌҊ્ܻ੄ ࣘبо ࡅܲ Ѫ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  38. 38
    Consistent Hashing ؘ੉ఠ ࠙࢑ పझ౟
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    Replica ஠਍౟ ߹ ਬ੷ ࠙ನ ࠺Ү
    ߸ز ҅ࣻ (಴ળಞର / ಣӐ)
    md5 - replica 1024 2.4%
    md5 - replica 512 4.3%
    blake2b - replica 1024 2.6%
    blake2b - replica 512 4.6%
    xxhash - replica 1024 3.2%
    xxhash - replica 512 5.9%

    View full-size slide

  39. 39
    Consistent Hashing ؘ੉ఠ ࠙࢑ పझ౟
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    о࢚ ֢٘ ࣻܳ 1024۽ ೞݶ ೧द ݂੄ ௼ӝо ցޖ ழ૓׮.
    ௾ ର੉о ੓ח Ѫ਷ ইפӝ ٸޙী 512ѐ۽ Ѿ੿೮׮.
    ࠙ನח md5о ؊ જ૑݅ blake2b৬੄ ର੉о ௼૑ ঋҊ
    blake2bо md5ࠁ׮ ࡅܲ ঌҊ્ܻ੉ӝ ٸޙী blake2b۽ Ѿ੿೮׮.

    View full-size slide

  40. 40
    Consistent Hashing ؘ੉ఠ ࠙࢑ పझ౟
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    blake2b, о࢚ ֢٘ࣻ 512ѐ۽ Ѿ੿!

    View full-size slide

  41. 41
    ࢥ٘ ഛ੢ ߑউ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  42. 42
    ࢥ٘ ഛ੢ ߑউ
    ࢥ٬ਸ ೞח ੉ਬח ࠗೞ࠙࢑ਸ ਤ೧ࢲ׮.
    Ӓ۞ա ࢥ٬ਸ ೧ب ౟ې೗੉ טযաҊ ೠ ࢥ٘ ׼ ؘ੉ఠо ழ૑ݶ
    ڙэ੉ ࠗೞо ߊࢤೞחؘ ੉ٸ ࢥ٘੄ ࣻܳ ט۰઻ঠ ೠ׮.
    ೞ૑݅, ࢥ٘ܳ טܻח Ѥ ࠺ਊ੉ ௾ ੘স੉ӝ ٸޙী
    ޷ܻ Ӓ ߑউਸ ࢤп೧ࠅ ೙ਃо ੓׮.
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  43. 43
    ࢥ٘ ഛ੢ ߑউ
    ఋࢎ ࢎ۹: Kakao
    1. ؘ੉ఠܳ ੤࠙ߓ ೞӝ ਤ೧ Table Crawler৬ Replication Protocolਸ ੉ਊೠ ਕழ ҳഅ
    2. ف о૑ ਕழܳ ڸਕ ࢜۽ ڸਕك DB Instanceী ؘ੉ఠܳ Syncೣ
    3. ؘ੉ఠ Ѩૐ റ Live
    • ࠶۽Ӓ Ӗ ଵҊ - ղࠗ Ѩૐ ױ҅ۄҊ ݺदغয੓׮.
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  44. 44
    ࢥ٘ ഛ੢ ߑউ
    ఋࢎ ࢎ۹: Tumblr
    ੉۠ Databaseҳࢿ੉ ੓णפ׮.
    master
    blog 1-1000
    slave
    blog 1-1000
    slave
    blog 1-1000
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  45. 45
    ࢥ٘ ഛ੢ ߑউ
    ఋࢎ ࢎ۹: Tumblr
    ࢲ࠺झী ైੑೞ૑ ঋ਷ ࢜۽਍ ठۨ੉࠳ ୶оೞҊ
    ؘ੉ఠܳ ੺߈ਵ۽ ա׃
    master
    blog 1-1000
    slave
    blog 1-1000
    slave
    blog 1-1000
    slave
    blog 501- 1000
    slave
    blog 1-500
    slave
    blog 1-500
    slave
    blog 1-500
    slave
    blog 501- 1000
    slave
    blog 501- 1000
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  46. 46
    ࢥ٘ ഛ੢ ߑউ
    ఋࢎ ࢎ۹: Tumblr
    ݃झఠ۽ о؍ Read/Writeܳ ୶оػ Slave۽ ੉ز
    master
    blog 1-1000
    slave
    blog 1-1000
    slave
    blog 1-1000
    slave
    blog 501- 1000
    slave
    blog 1-500
    slave
    blog 1-500
    slave
    blog 1-500
    slave
    blog 501- 1000
    slave
    blog 501- 1000
    R/W R/W
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  47. 47
    ࢥ٘ ഛ੢ ߑউ
    ఋࢎ ࢎ۹: Tumblr
    ӝઓ ݃झఠ৬੄ ܻ೒ܻா੉࣌ਸ ઁѢೞҊ ؘ੉ఠ ࠁ੿
    master2
    blog 501- 1000
    master1
    blog 1-500
    slave
    blog 1-500
    slave
    blog 1-500
    slave
    blog 501- 1000
    slave
    blog 501- 1000
    Master
    blog 1-1000
    slave
    blog 1-1000
    slave
    blog 1-1000
    X
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  48. 48
    ࢲ࠺झী ైੑೞ૑ ঋ਷ ठۨ੉࠳ܳ ୶оೠ റ
    ؘ੉ఠܳ աׇࢲ ੷੢ೠ׮.
    Tumblr ݈۽ח ޖ઺ױ ഛ੢੉ۄҊ ೠ׮.
    ࢥ٘ ഛ੢ ߑউ
    ఋࢎ ࢎ۹: Tumblr
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  49. 49
    ࢥ٘ ഛ੢ ߑউ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    ഛ੢द ௏٘ܳ ੉ਊೠ ࠙ߓ(ਕழ) ഑਷ ؘ੉ఠ ࠁ੿হ੉
    ࢥ٘੄ ഛ੢ࢿ/ؘ੉ఠ੄ न܉ࢿਸ ࠁ੢ೞҊ र׮.

    View full-size slide

  50. 50
    ഛ੢द ௏٘ܳ ੉ਊೠ ࠙ߓ(ਕழ) ഑਷ ؘ੉ఠ ࠁ੿হ੉
    ࢥ٘੄ ഛ੢ࢿ/ؘ੉ఠ੄ न܉ࢿਸ ࠁ੢ೞҊ र׮.
    ࢥ٘ ഛ੢ ߑউ
    ੋ೐ۄܳ ా೧ࢲ ࢥ٘ܳ ҙܻ೧ࠁחѤ যڄө?
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  51. 51
    ੋझఢझী MySQL੄ Databaseܳ ୽࠙൤ ٜ݅যفҊ ܻࣗझо ࠗ઒ೡٸ
    ੋझఢझܳ ഛ੢ೞח ߑध
    Databaseܳ Logical DB, ੋझఢझܳ Physical DBۄҊ ੉ܴ ࠢ৓׮.
    ࢥ٘ ഛ੢ ߑউ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  52. 52
    ࢥ٘ ഛ੢ ߑউ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  53. 53
    ࢥ٘ ഛ੢ ߑউ
    Instance B
    Instance A
    WAS
    Shard C
    Shard B
    Shard D
    Event Worker
    Shard A
    READ Write
    Replication
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  54. 54
    ࢥ٘ ഛ੢ ߑউ
    Instance B
    Instance A
    WAS
    Shard C
    Shard B
    Shard D
    Event Worker
    Instance A`
    Instance B`
    Shard A
    READ Write
    Replication
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  55. 55
    ࢥ٘ ഛ੢ ߑউ
    Instance B
    Instance A
    WAS
    Shard C
    Shard B
    Shard D
    Event Worker
    Instance A`
    Instance B`
    Shard A
    Shard B
    Shard D
    READ Write
    Replication
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  56. 56
    ࢥ٘ ഛ੢ ߑউ
    Instance B
    Instance A
    WAS
    Shard C
    Shard B
    Shard D
    Event Worker
    Instance A`
    Instance B`
    Shard A
    Shard B
    Shard D
    READ Write
    Replication
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  57. 57
    ࢥ٘ ഛ੢ ߑউ
    Instance B
    Instance A
    WAS
    Shard C
    Shard B
    Shard D
    Event Worker
    Instance A`
    Instance B`
    Shard A
    Shard B
    Shard D
    READ Write
    Replication
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  58. 58
    ࢥ٘ ഛ੢ ߑউ
    Instance B
    Instance A
    WAS
    Shard C
    Shard B
    Shard D
    Event Worker
    Instance A`
    Instance B`
    Shard A
    Shard B
    Shard D
    READ Write
    Replication
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  59. 59
    ࢥ٘ ഛ੢ ߑউ
    Instance B
    Instance A
    WAS
    Shard C
    Event Worker
    Instance A`
    Instance B`
    Shard A
    Shard B
    Shard D
    READ Write
    Replication
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  60. 60
    ࢥ٘ ഛ੢ ߑউ
    ୭؀۽ ഛ੢ೡ ࣻ ੓ח Physical DBࣻо ੿೧ઉ ੓ח ױ੼੉ ੓׮.
    Logical DB੄ ࣻ ੗୓ܳ ט۰ঠ ೞחؘ ੉ ҃਋ ؘ੉ఠܳ ੤࠙ߓ ೧ঠೠ׮.
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  61. 61
    ࢥ٘ ഛ੢ ߑউ
    ೞ૑݅ ҡଳ׮.
    Consistent Hash ߑधਵ۽ ؘ੉ఠܳ ࠙ߓೞӝ ٸޙী
    ࢚؀੸ਵ۽ ࠙ߓ/Ѩૐ੄ դ੉بо ծ׮.
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  62. 62
    ੿੸ ؘ੉ఠ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  63. 63
    ੿੸ ؘ੉ఠ
    ҳݒݾ۾ ਃҳࢎ೦ਸ ҳഅೞӝ ਤ೧ࢲח ଼ ؘ੉ఠ৬ JOIN੉ ೙ࣻ
    But, ׮ܲ DBী ઓ੤ೞח ؘ੉ఠ৬ JOIN੉ ࠛоמ
    ݽٚ ࢥ٘ী ҕాਵ۽ ઓ੤ೞח ੿੸ ؘ੉ఠ ప੉࠶੉ ೙ਃೞ׮.
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  64. 64
    ੿੸ ؘ੉ఠ
    Shard A
    Book
    Item
    Source
    Book
    Item
    Shard B
    Book Worker
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    ਗࠄਸ فҊ ࢥ٘ী ࠂઁೞח ѐ֛
    (Ӓր Replication…)

    View full-size slide

  65. 65
    ੿੸ ؘ੉ఠ
    ੿੸ ؘ੉ఠܳ ࢥ٘ী ನೣೣਵ۽ॄ
    ۽૒ ഑਷ ௪ܻܳ ੘ࢿೡٸ ਬ੷ױਤ۽ ࢤпೞݶ ױੌ DB୊ۢ ࢎਊೡ ࣻ ੓਺
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  66. 66
    ୨ ੿ܻ
    • ࢥ٘ ః
    • ਬ੷ ID
    • ࢥ٬ ߑध
    • Consistent Hashܳ ੉ਊೠ ࠙ߓ
    • ࢥ٘ ഛ੢ ߑध
    • ୽࠙ೠ Logical DBࢤࢿ, Physical DB݅ ഛ੢
    • ੿੸ ؘ੉ఠ
    • Sourceܳ فҊ п ࢥ٘ী ࠂઁೞח ߑध
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  67. 67
    3. Ӓ റ ੉ঠӝ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?

    View full-size slide

  68. 68
    Ѿ੿ೠ ੿଼ٜਸ ҳഅೠ ۄ੉࠳۞ܻ ੘ࢿ
    https://github.com/ridi/django-shard-library
    ৻Ҵੋ੉ झఋ 4ѐ ନযષ ƃƃƃ
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    ই૒ ݆੉ ࠗ઒ೠ ஘ҳੑפ׮. ৈ۞࠙੄ ݆਷ ҙब ߄ۉפ׮ :)

    View full-size slide

  69. 69
    ਋ܻח যڌѱ ࢥ٬ਸ ೮חо?
    Terraformਸ ాೠ RDSੋ೐ۄ ҙܻ
    ୶റ ҅ദ
    ࢥ٘ ഛ੢ ੗زച
    ࢥ٘ ۄ੉࠳۞ܻ ӝמ
    ؊ ۄ੉࠳۞ܻ׹ѱ ъച

    View full-size slide

  70. 71
    ੿੸ ؘ੉ఠ ࠙ߓೡٸ ৵ Replicationਵ۽ উ೮աਃ?
    MySQL਷ Tableױਤ۽ Replication غਗ਼ইਃ?
    Q & A
    Question 1

    View full-size slide

  71. 72
    Q & A
    Answer 1
    ੿੸ ؘ੉ఠܳ п ࢥ٘ী ֍য઱۰ݶ Master-Master Replicationਵ۽ ҳ
    ࢿ೧ঠ೤פ׮. അ੤ RDS੄ ҃਋ Aurora৻ীח Master-Master
    Replicationਸ ૑ਗೞ૑ ঋইࢲ ਕழܳ ా೧ ؘ੉ఠܳ ࠙ߓೞҊ ੓णפ׮.

    View full-size slide

  72. 73
    ৵ Aurora উ 㛣աਃ? Master-Master Replicationب ૑ਗೞҊ

    ࢿמب ؊ ੜ աয়ਗ਼ইਃ?
    Q & A
    Question 2

    View full-size slide

  73. 74
    Q & A
    Answer 2
    ѐߊೞݶࢲ ௪ܻౚ׬੉ ખ ݆੉ ٜযщणפ׮. Auroraо MySQLഐജ੉ۄ
    Ҋ ೞחؘ ੋఠಕ੉झ݅ ഐജੋ૑ Storage engineө૑ ഐജੋ૑ ഛੋ೧ࠅ
    ೙ਃо ੓যࢲ بੑೞ૑ ޅ೮णפ׮. ҅ദ਷ о૑Ҋ ੓णפ׮ :)

    View full-size slide

  74. 75
    ੢গद ؘ੉ఠ ࠂҳ যڌѱ ೞաਃ?
    Q & A
    Question 3

    View full-size slide

  75. 76
    Q & A
    Answer 3
    ੢গदীח RDS੄ Multi AZܳ ా೧ࢲ ࠂҳೡ ҅ദੑפ׮.

    View full-size slide

  76. 77
    ࢥ٘ ഛ੢द ୭؀ 5࠙੿ب ѦܾѢۄҊ ݈ॹೞ࣑חؘ

    5࠙زউ ҳݒݾ۾ ߈৔੉ উغח Ѣ ইצоਃ?
    Q & A
    Question 4

    View full-size slide

  77. 78
    Q & A
    Answer 4
    ୭ୡ ਃҳࢎ೦ ੿ܻೡٸ ইޖܻ ןযب ୭؀ 5࠙উীח ҳݒݾ۾ী ߈৔ػ
    ׮Ҋ ೮חؘ, ࢥ٘ ഛ੢ө૑ Ҋ۰ೠ 5࠙ੑפ׮. Ӓېب 5࠙੉ ૣ਷ दр਷ ই
    פۄࢲ ഛ੢ਸ ೞѱ غݶ غب۾ ੼Ѩ ഑਷ ࢜߷दр ੉ਊ೧ࢲ ૓೯ೡ ҅ദ
    ੑפ׮.

    View full-size slide

  78. 79
    ࢥ٘ ഛ੢ೡٸ Writeܳ ઁѢೞݶ

    ҳݒ೧ب ؘ੉ఠ উऺ੉חѢ ইצоਃ?
    Q & A
    Question 5

    View full-size slide

  79. 80
    Q & A
    Answer 5
    ѐߊೡٸ CQRSಁఢਵ۽ ҳഅਸ ೮णפ׮. ੉੹ী Ӓܿীࢲ Writeܳ ઁѢ
    ೞחѤ ௸੄ ؘ੉ఠܳ ࢥ٘۽ ॳח Ѫਸ ઁѢೞח Ѣۄࢲ ࢥ٘ ഛ੢઺ীب
    ҳݒೠ ӝ۾਷ ௸ী ӝ۾ؾפ׮. ੉যࢲ ࢥ٘ ഛ੢੉ ৮ܐغݶ ׮द ୊ܻغ
    ӝ ٸޙী Write־ۅ਷ হणפ׮.

    View full-size slide

  80. 81
    ଵҊ ޙࢲ
    • ఋࢎࢎ۹
    • http://tech.kakao.com/2016/07/01/adt-mysql-shard-rebalancing/
    • https://pegasuskim.wordpress.com/2015/12/17/tumblr-sharding/
    • http://assets.en.oreilly.com/1/event/74/
    Massively%20Sharded%20MySQL%20at%20Tumblr%20Presentation.pdf
    • http://highscalability.com/blog/2012/2/13/tumblr-architecture-15-billion-page-views-a-month-and-harder.html
    • https://engineering.tumblr.com/post/7658008285/efficiently-copying-files-to-multiple-destinations
    • Karger Consistent Hashing
    • https://charsyam.wordpress.com/2016/10/02/%EC%9E%85-%EA%B0%9C%EB%B0%9C-consistent-
    hashing-%EC%97%90-%EB%8C%80%ED%95%9C-%EA%B8%B0%EC%B4%88/
    • https://www.akamai.com/es/es/multimedia/documents/technical-publication/consistent-hashing-and-random-
    trees-distributed-caching-protocols-for-relieving-hot-spots-on-the-world-wide-web-technical-publication.pdf
    • Jump Consistent Hashing
    • https://arxiv.org/pdf/1406.2294.pdf

    View full-size slide

  81. 82
    ੉޷૑ ୹୊
    • https://kr.seaicons.com/ؘ੉ఠ߬੉झ-ই੉௑-3/
    • https://kr.seaicons.com/ই੉௑-ࢎਊ੗-֢ز੗/
    • https://kr.seaicons.com/ਢ-ࢲߡ-ই੉௑/

    View full-size slide

  82. 83
    THANK YOU
    хࢎ೤פ׮

    View full-size slide