Upgrade to Pro — share decks privately, control downloads, hide ads and more …

月間10億PVから学んだMongoDBアンチパターン

yujiosaka
November 11, 2014

 月間10億PVから学んだMongoDBアンチパターン

ZenClerkが月間10億PVを支えるまでの
 過程で経験したアンチパターンを紹介

yujiosaka

November 11, 2014
Tweet

More Decks by yujiosaka

Other Decks in Technology

Transcript

  1. ݄ؒ10ԯPV͔ΒֶΜͩ

    MongoDBΞϯνύλʔϯ
    db tech showcase ౦ژ 2014
    speaker: “Yuji Isobe”

    View full-size slide

  2. history: [

    “violinist”

    “engineer”

    “security specialist”

    “socket.io contributor”

    “startup member

    of ZenClerk ”

    ]
    name: “Yuji Isobe”

    View full-size slide

  3. questions:
    -> MongoDBΛ࢖༻ͨ͜͠ͱ͕͋Γ·͔͢ʁ
    -> ຊ൪؀ڥͰMongoDBΛ࢖༻͍ͯ͠·͔͢ʁ

    View full-size slide

  4. goal:
    -> ZenClerk͕݄ؒ10ԯPVΛࢧ͑Δ·Ͱͷ

    աఔͰܦݧͨ͠ΞϯνύλʔϯΛ঺հ
    -> ओʹύϑΥʔϚϯενϡʔχϯάͷ࿩
    ->ʮ݄ؒ10ԯPVΛࢧ͑ΔMongoDBʯͷͦͷઌ
    IUUQXXXTMJEFTIBSFOFUZVKJPTBLBQWNPOHPEC

    View full-size slide

  5. ϦΞϧλΠϜ

    View full-size slide

  6. ϦΞϧλΠϜ෼ੳ
    Ϗοάσʔλ

    View full-size slide

  7. db.ZenClerk .stats()
    -> ݄ؒ10ԯPV
    -> ඵؒ400PV
    -> ಉ࣌઀ଓ਺5ສ
    -> ݄ؒ10TBอଘ

    View full-size slide

  8. architecture:
    front-end: [
    “AngularJS”
    “Ruby on Rails”
    “MySQL”
    “memcached”

    “AWS”
    ]
    back-end: [
    “Node.js”

    “MongoDB”
    “socket.io”
    “Redis”

    “AWS”
    ]

    View full-size slide

  9. architecture:
    front-end: [
    “AngularJS”
    “Ruby on Rails”
    “MySQL”
    “memcached”

    “AWS”
    ]
    back-end: [
    “Node.js”

    “MongoDB”
    “socket.io”
    “Redis”

    “AWS”
    ]

    View full-size slide

  10. i2.2xlarge m1.large m3.large
    db.ZenClerk .aggregate()
    r3.large

    View full-size slide

  11. ʙ1000ສPV
    mogngod

    View full-size slide

  12. 1000ສʙ1ԯPV
    delayed
    replica
    replica set

    View full-size slide

  13. 1ԯPVʙ10ԯPV
    replica set
    shard
    mongos
    replica set
    shard
    replica set
    shard
    delayed
    replica
    delayed
    replica
    delayed
    replica
    mongoc

    View full-size slide

  14. ͜͜·ͰΞϓϦέʔγϣϯͷมߋ

    ΄ͱΜͲͳ͠ʢҰ෦ΫΤϦΛ࠷దԽʣ

    View full-size slide

  15. we.use(“MongoDB”).explain()
    -> εΩʔϚϨε
    -> τϥϯβΫγϣϯෆཁ

    ʢ࠷௿ݶͷΞτϛοΫૢ࡞ʣ
    -> ॊೈͳΫΤϦ
    -> ༰қͳεέʔϧΞ΢τ

    -> Node.jsͱͷ૬ੑͷྑ͞

    View full-size slide

  16. we.use(“MongoDB”).explain(true)
    -> ಋೖͷ༰қ͞ʢϗεςΟϯάαʔϏε༗ແʣ
    -> ֶशͷ༰қ͞ʢΤϯδχΞҎ֎ʹͱͬͯ΋ʣ

    View full-size slide

  17. ࠓ೔࿩͢ͷ͸ɾɾɾ
    ϕετϓϥΫςΟε Ξϯνύλʔϯ

    View full-size slide

  18. ͪ͜Βʂ
    ϕετϓϥΫςΟε Ξϯνύλʔϯ

    View full-size slide

  19. Ͱ΋ͦͷલʹɾɾɾ

    View full-size slide

  20. ϕετϓϥΫςΟε

    View full-size slide

  21. εΩʔϚσβΠϯ
    -> TwitterͷλΠϜϥΠϯΛอଘ͢Δ
    -> RDBMSͳΒਖ਼نԽ
    -> MongoDBͷϕετϓϥΫςΟε͸ʁ

    View full-size slide

  22. -> ಡΈࠐΈॏࢹ
    -> 1ΫΤϦ1ώοτ
    -> ίϝϯτ਺ʹԠͯ͡

    ॻ͖ࠐΈ͕ॏ͘ͳΔ
    υΩϡϝϯτΛຒΊࠐΉ
    session:

    _id: “session_id”

    device: “iPhone”

    page_views: [

    {url: “/items/1”}

    ]
    timeline:

    _id: “yujiosaka”

    tag: “#dbtechshowcase”

    tweets: [

    {text: “MongoDB Rocks!”}

    ]

    View full-size slide

  23. -> ॻ͖ࠐΈॏࢹ
    -> 2ΫΤϦN+1ώοτ
    -> ॻ͖ࠐΈ͸εέʔϧ͢Δ
    υΩϡϝϯτΛຒΊࠐ·ͳ͍
    session:

    _id: “session_id”

    device: “iPhone”

    page_views: [

    {url: “/items/1”}

    ]
    timeline:

    _id: “yujiosaka”

    tag: “#dbtechshowcase”
    !
    tweet:
    _id: 1234

    user_id: “yujiosaka”

    text: “MongoDB Rocks!”

    View full-size slide

  24. timeline:

    _id: “yujiosaka-20141111”

    tag: “#dbtechshowcase”

    tweets: [

    {text: “MongoDB Rocks!”}

    ]
    -> όϥϯεܕ
    -> 1ΫΤϦ೔਺෼ώοτ
    -> ॻ͖ࠐΈ͸1೔෼·Ͱ

    ͔͠ॏ͘ͳΒͳ͍
    ϋΠϒϦου

    View full-size slide

  25. εΩʔϚϨε ≠ εΩʔϚఆ͕ٛෆཁ
    RDBMSΑΓ΋બ୒ࢶ͕ଟ͘೉͍͠
    tips:
    IUUQXXXNPOHPECDPNQSFTFOUBUJPOTTDIFNBEFTJHOTDBMF

    View full-size slide

  26. Ξϯνύλʔϯ

    View full-size slide

  27. ΠϯσοΫεฤ
    ΫΤϦฤ
    Ξοϓσʔτฤ
    ΞʔΩςΫνϟฤ

    View full-size slide

  28. ᶃΠϯσοΫεͷషΓա͗

    View full-size slide

  29. MongoDB͸ϝϞϦͷ࢖͍ํ͕

    ͋·Γݡ͋͘Γ·ͤΜ

    View full-size slide

  30. before:
    user = new mongoose.Schema
    first: {type: String, required: true, index: true}
    last: {type: String, required: true, index: true}
    city: {type: String, required: true, index: true}
    ...
    created_at: {type: Date, default: Date.now, index: true}
    !
    ॻ͖ࠐΈ͕٘ਜ਼ʹͳͬͯ΋ಡΈࠐΈ͕ૣ͚Ε͹͍͍͡ΌΜʂ

    View full-size slide

  31. after:
    user = new mongoose.Schema
    first: {type: String, required: true}
    last: {type: String, required: true}
    city: {type: String, required: true}
    ...
    created_at: {type: Date, default: Date.now}
    !
    ॻ͖ࠐΈ͕٘ਜ਼ʹͳͬͯ΋ಡΈࠐΈ͕ૣ͚Ε͹͍͍͡ΌΜʂ

    View full-size slide

  32. ಡΈࠐΈ΋ɺΠϯσοΫε͕
    ϝϞϦʹ৐Βͳ͚Ε͹஗͘ͳΔ

    View full-size slide

  33. tips:
    όʔδϣϯ2.8ʹPluggable Storage

    Engine͕༧ఆ͞Ε͍ͯΔͷͰɺ
    ݡ͍Τϯδϯ͕બ΂ΔΑ͏ʹͳΔ
    https://blog.compose.io/the-coming-of-the-mongodb-storage-engines/

    View full-size slide

  34. ᶄจࣈྻͷΠϯσοΫε

    View full-size slide

  35. ΠϯσοΫεͷߏ଄
    -> RDBMSͱ΄ͱΜͲಉ͡
    -> ΠϯσοΫεʹB-TreeΛ࠾༻
    -> B-Tree͸ৗʹฏߧΛอ͍ͬͯΔ

    View full-size slide

  36. จࣈྻͷΠϯσοΫε
    -> ϗοτσʔλ͕෼ࢄ
    -> ϝϞϦޮ཰͕ѱ͍

    View full-size slide

  37. ࣌ؒɾObjectIdͷΠϯσοΫε
    -> ϗοτσʔλ͕ूத
    -> ϝϞϦޮ཰͕ྑ͍

    View full-size slide

  38. tips:
    Ͱ͖Δ͚ͩObjectIdΛ࢖͍·͠ΐ͏
    ޙ͔Βͷஔ͖׵͑͸େมͰ͢

    View full-size slide

  39. ΠϯσοΫεฤ
    ΫΤϦฤ
    Ξοϓσʔτฤ
    ΞʔΩςΫνϟฤ

    View full-size slide

  40. ᶅϑΟʔϧυΛࢦఆ͠ͳ͍ΫΤϦ

    View full-size slide

  41. Ϣʔβʔ৘ใͷ͏ͪ
    ౎ࢢ৘ใ͚͕ͩཉ͍͠
    user: {_id: “yujiosaka”, first: “Yuji”, last: “Isobe”, city: “Tokyo”}
    index: {_id:1, city: 1} // compound index

    View full-size slide

  42. ϑΟʔϧυࢦఆͳ͠
    -> SQLͰͷʮselect *ʯ
    -> ෆཁͳωοτϫʔΫίετ
    -> ෆཁͳσΟεΫΞΫηε
    var user = db.users.findOne({_id: “yujiosaka”});

    View full-size slide

  43. ϑΟʔϧυࢦఆ͋Γ
    -> SQLͰͷʮselect cityʯ
    -> ωοτϫʔΫίετͷઅ໿
    -> covered index ͕׆༻Ͱ͖Δ
    var user = db.users.findOne({_id: “yujiosaka”}, {city: 1});

    View full-size slide

  44. tips:
    covered indexΛ༗ޮ׆༻͢Ε͹ɺ

    σΟεΫ΁ͷΞΫηεҰ੾ͳ͠ʹ

    ϝϞϦ͚͔ͩΒ৘ใΛಡΈࠐΊΔ

    View full-size slide

  45. ᶆίʔϧυσʔλ΁ͷΫΤϦ

    View full-size slide

  46. MongoDBͷಛ௃
    -> ϗοτσʔλ΁ͷΞΫηε͸ߴ଎
    -> ίʔϧυσʔλ΁ͷΞΫηε͸஗͍
    -> શσʔλ΁ͷΞΫηε͸ۤख

    View full-size slide

  47. ZenClerkʹ͓͚Δूܭ
    -> 5෼͝ͱʹσʔλΛཁ໿
    -> 1࣌ؒ͝ͱʹ౷ܭσʔλΛूܭ
    -> 1೔͝ͱʹ౷ܭσʔλΛूܭ

    View full-size slide

  48. tips:
    ৗʹϗοτσʔλΛҙࣝͨ͠

    ࢖͍ํΛ͠·͠ΐ͏

    View full-size slide

  49. ᶇϓϥΠϚϦʹภͬͨΫΤϦ

    View full-size slide

  50. ϨϓϦΧηοτ
    -> ΫΤϦΛ෼ࢄͤ͞Δ͜ͱ͕Ͱ͖Δ
    -> ηΧϯμϦʹΫΤϦΛ౤͛Δ͔͸

    ϓϩάϥϚ͕ܾΊΒΕΔ
    -> σϑΥϧτͰ͸ϓϥΠϚϦʹ౤͛ΒΕΔ

    View full-size slide

  51. σϑΥϧτ
    -> ϓϥΠϚϦ͔Βಡ·ΕΔ
    -> ࠷৽ͷσʔλ͕อো͞ΕΔ
    var user = db.users.find({city: “Tokyo”});

    View full-size slide

  52. ηΧϯμϦΛࢦఆ
    -> ηΧϯμϦ͕͋Ε͹ηΧϯμϦ͔Βɺ

    ͳ͚Ε͹ϓϥΠϚϦ͔Βಡ·ΕΔ
    -> ࠷৽ͷσʔλ͸อো͞Εͳ͍
    var user = db.users.find({city: “Tokyo”})

    .readPref("secondaryPreferred")

    View full-size slide

  53. tips:
    σʔλ͕࠷৽Ͱͳͯ͘΋Α͍৔߹͸

    ಡΈࠐΈΛηΧϯμϦʹ޲͚Δ͜ͱͰ

    ΫΤϦ͕෼ࢄ͢Δ
    IUUQTTQFBLFSEFDLDPNNPOHPECQSBDUJDBM
    TDBMJOHBOETIBSEJOHFMJPUIPSPXJU[HFO

    View full-size slide

  54. ΠϯσοΫεฤ
    ΫΤϦฤ
    Ξοϓσʔτฤ
    ΞʔΩςΫνϟฤ

    View full-size slide

  55. ᶈඇޮ཰ͳΞοϓσʔτ

    View full-size slide

  56. Α͘ݟ͔͚Δίʔυᶃ
    -> ඞͣΫΤϦ͕࣮ߦ͞ΕΔ
    -> ແବͳωοτϫʔΫίετ
    -> ඇΞτϛοΫͳॲཧ
    var user = db.users.findOne({_id: “yujiosaka”});
    user.count += 1;
    user.save();

    View full-size slide

  57. Α͘ݟ͔͚Δίʔυᶃ
    -> updateͱ$incΛ࢖ͬͯ·ͱΊΒΕΔ
    -> 1ճͷΞοϓσʔτͰࡁΉ
    -> ΞτϛοΫͳॲཧ
    db.users.update({_id: “yujiosaka”}, {$inc: {count: 1}});

    View full-size slide

  58. Α͘ݟ͔͚Δίʔυᶄ
    -> σʔλ͕ݟ͔ͭͬͨΒߋ৽͠ɺݟ͔ͭΒͳ͔ͬͨΒ࡞੒͢Δ
    -> Ұݟෳࡶͳίʔυ
    var user = db.users.findOne({_id: “yujiosaka”});
    if (user) {
    user.count += 1;

    user.save();
    } else {
    db.users.insert({_id: “yujiosaka”, count: 1});
    }

    View full-size slide

  59. -> {upsert: true} Λ࢖͏ͱ·ͱΊΒΕΔ
    -> ΑΓγϯϓϧͳίʔυ
    Α͘ݟ͔͚Δίʔυᶄ
    db.users.update({_id: “yujiosaka”}, {$inc: {count: 1}}, {upsert: true});

    View full-size slide

  60. ᶉංେԽ͢ΔυΩϡϝϯτ

    View full-size slide

  61. Ϛ΢εͷ࠲ඪσʔλΛ

    อଘ͢Δ
    behavior:
    points: [{x: 151, y: 100}
    {x: 151, y: 179}
    {x: 151, y: 266}

    {x: 151, y: 340}]

    View full-size slide

  62. MongoDBͷۤख෼໺
    -> ϑΟʔϧυ͕ංେԽ͢ΔΞοϓσʔτ͕஗͍
    -> σʔλ͕ҰఆͷαΠζʹऩ·Βͳ͘ͳΔͱ

    σΟεΫ಺ͰσʔλͷҠಈ͕ى͖Δ
    -> RedisͰMongoDBͷۤख෼໺Λิ͏

    View full-size slide

  63. Redis
    -> ΦϯϝϞϦܕͷKVS
    -> ಡΈࠐΈɾॻ͖ࠐΈ͕ૣ͍
    -> جຊతʹσʔλͷӬଓԽ͸͠ͳ͍
    -> σʔλܕ͕ඇৗʹ๛෋

    ʢϦετɾηοτɾϋογϡʣ

    View full-size slide

  64. RedisΛ࢖Θͳ͍৔߹
    db.col.update({_id: 1234}, {$push: {points: {x: 151, y: 100}});

    db.col.update({_id: 1234}, {$push: {points: {x: 151, y: 179}});

    db.col.update({_id: 1234}, {$push: {points: {x: 151, y: 266}});

    …

    db.col.update({_id: 1234}, {$push: {points: {x: 151, y: 340}});
    -> ࠲ඪ͕Ҡಈ͢ΔͨͼʹMongoDBʹॻ͖ࠐ·ΕΔ
    -> αΠζ͕ऩ·Γ੾Βͳ͍ͱɺσʔλͷҠಈ͕ى͖Δ

    View full-size slide

  65. RedisΛ࢖ͬͨ৔߹
    db.col.insert({points: [

    {x: 151, y: 100},

    {x: 151, y: 179},

    {x: 151, y: 266},

    …

    {x: 151, y: 340}

    ]});
    -> Ұ౓͔͠MongoDBʹॻ͖ࠐ·ͳ͍
    -> υΩϡϝϯυͷαΠζ͸͜ΕҎ্େ͖͘ͳΒͳ͍

    View full-size slide

  66. tips:
    ͢΂ͯΛMongoDB͚ͩͰ΍Ζ͏ͱͤͣɺ

    ۤखͳ෦෼͸ଞͷٕज़Ͱิ͏͜ͱ͕େ੾

    View full-size slide

  67. ΠϯσοΫεฤ
    ΫΤϦฤ
    Ξοϓσʔτฤ
    ΞʔΩςΫνϟฤ

    View full-size slide

  68. ᶊ٧ΊࠐΈա͗DB

    View full-size slide

  69. σʔλΛҰͭͷDBʹ٧ΊࠐΜͰ

    ྑ͍͜ͱ͸ʢࠓͷͱ͜Ζʣ͋·Γͳ͍

    View full-size slide

  70. ͳͥDBΛ෼͚Δͷ͔ʁ
    -> Reader-WriterϩοΫ͕࢖ΘΕ͍ͯΔ
    -> 2.2.0Ҏ߱͸DB୯ҐͰϩοΫ͢Δ
    -> ϚγϯΛ෼͚Ε͹ϝϞϦ͕૿͑Δ

    View full-size slide

  71. ZenClerk ʹ͓͚ΔDBͷ༻్
    -> ϝΠϯσʔλʢHotʣ
    -> ϝΠϯσʔλʢColdʣ
    -> ੜσʔλ
    -> ܭࢉ݁Ռσʔλ
    -> ౷ܭσʔλ
    -> αΠτଐੑσʔλ

    View full-size slide

  72. tips:
    όʔδϣϯ2.8ʹdocument-level locking
    3.0Ҏ߱ʹpartitioned joins͕༧ఆ͞Ε͍ͯΔ

    ʢޙऀ͸͋·Γ͋ͯʹ͠ͳ͍ํ͕ແ೉ͦ͏ʣ
    http://www.slideshare.net/tetsutarowatanabe/mongodb-world-2014-37081258

    View full-size slide

  73. ᶃΠϯσοΫεͷషΓա͗
    ᶄจࣈྻͷΠϯσοΫε
    ᶅϑΟʔϧυΛࢦఆ͠ͳ͍ΫΤϦ
    ᶆίʔϧυσʔλ΁ͷΫΤϦ
    ᶇϓϥΠϚϦʹภͬͨΫΤϦ
    ᶈඇޮ཰ͳΞοϓσʔτ
    ᶉංେԽ͢ΔυΩϡϝϯτ
    ᶊ٧ΊࠐΈա͗DB
    Ξϯνύλʔϯ͓͞Β͍

    View full-size slide

  74. Ξϯνύλʔϯʹ

    ؕΒͳ͍ͨΊʹ

    View full-size slide

  75. ̏ͭͷ؂ࢹ
    -> ϦΞϧλΠϜ؂ࢹ
    -> ݟฦͨ͢Ίͷ؂ࢹ
    -> ஗͍ΫΤϦͷ؂ࢹ

    View full-size slide

  76. ϦΞϧλΠϜ؂ࢹ

    View full-size slide

  77. -> ComposeʢMongoHQʣμογϡϘʔυ
    -> ݱࡏͷ݈߁νΣοΫ༻
    -> ࣾ಺ʹઐ༻σΟεϓϨΠΛઃஔ
    -> ҟৗ͕͋Ε͹ʮ୭͔Mongoࡴͨ͠ʁʯ
    ϦΞϧλΠϜ؂ࢹ

    View full-size slide

  78. ݟฦͨ͢Ίͷ؂ࢹ

    View full-size slide

  79. -> MMSμογϡϘʔυ

    -> ఆظతͳ݈߁਍அ
    -> ো֐ൃੜ࣌ʹݟฦͯ͠

    ɹʮͳΜͰ͜ΜͳϩοΫߴ͘ͳͬͯΜͷʁʯ
    ݟฦͨ͢Ίͷ؂ࢹ

    View full-size slide

  80. ஗͍ΫΤϦͷ؂ࢹ

    View full-size slide

  81. -> explain(true)
    -> system.profile.find()
    -> Dex
    -> Compose (MongoHQ) Slow Queries
    ஗͍ΫΤϦͷ؂ࢹ

    View full-size slide

  82. tips:
    ObjectIdΛ࢖ͬͨΫΤϦʹ
    100msҎ্͔͔ͬͨΒةݥ৴߸

    View full-size slide

  83. my.presentation.aggregate()
    -> MongoDB͸બ୒ࢶ͕ଟ͍
    -> Ξϯνύλʔϯ͸͍͔ͭ͘ଘࡏ͢Δ
    -> ΞϯνύλʔϯʹؕΒͳ͍ͨΊʹ؂ࢹ͢Δ
    -> ࠷৽৘ใͷΩϟονΞοϓ΋ॏཁ

    View full-size slide

  84. we.are “hiring! :)”
    https://www.zenclerk.com/recruit.html

    View full-size slide

  85. any.questions?

    View full-size slide