Upgrade to Pro — share decks privately, control downloads, hide ads and more …

HARD THINGS in Ad-Tech Engineering

jsoizo
April 25, 2017

HARD THINGS in Ad-Tech Engineering

Apr 25, 2017 /
Speee Cafe Meetup #06

の資料です。

jsoizo

April 25, 2017
Tweet

More Decks by jsoizo

Other Decks in Programming

Transcript

  1. HARD THINGS
    in Ad-Tech Engineering
    Apr 25, 2017 /
    Speee Cafe Meetup #06
    @jsoizo

    View Slide

  2. whoami
    • ໊લ : @jsoizo / Jun Sakamoto
    • ݴޠ : JavaScriptͱScalaΛগʑ
    • ॴଐ :
    • ໾ׂ : ৽نΞυϓϩμΫτͷProduct Manager
    • “Project” Manager͔Βͷస਎νϟϨϯδத
    • લ৬͔Β਺͑ͯΞυςΫྺ6೥໨

    View Slide

  3. ࣭໰
    ΞυςΫܦݧऀͷํ?
    ೖ໳/ඇܦݧऀͷํ?

    View Slide

  4. ຊ೔ͷ͓୊

    View Slide

  5. View Slide

  6. ΞυςΫΤϯδχΞత
    ࠷΋HARDͳTHINGSͱ͸?

    View Slide

  7. ഑৴ো֐

    View Slide

  8. γεςϜͱͯ͠ɺ૊৫ͱͯ͠
    ͲͷΑ͏ʹཱͪ޲͔͔ͬͨ

    View Slide

  9. ΞυςΫͱো֐
    • ޿ࠂ഑৴γεςϜ͸ɺओʹҎԼͷཁ݅Λຬͨ͢ඞཁ͕͋Δ
    • ޿ࠂओ͔Β༬͔ͬͨ༧ࢉΛ࠷దʹ࢖͏
    • ഑৴ઌͷϝσΟΞʹରͯ͠ϢʔβମݧΛམͱͣ͞ऩӹΛग़͢
    • Ϣʔβ͕ྑ͍ίϯςϯπ(≒޿ࠂओ)ʹग़ձ͑Δ
    • γεςϜ͕ɺ
    • ਖ਼͘͠ಈ͍͍ͯΕ͹શһ͕ϋοϐʔ
    • ো֐͕͋Δͱશһ͕ͭΒ͍

    View Slide

  10. ϋΠύϑΥʔϚϯε͔ͭ
    Կ͕͋ͬͯ΋ࢮͳͳ͍
    (≒Կ͔͋ͬͯ΋ܰ঱)
    γεςϜߏஙͷඞཁੑ

    View Slide

  11. γεςϜશମ૾

    View Slide

  12. γεςϜશମ૾

    View Slide

  13. γεςϜશମ૾

    View Slide

  14. ࣮ࡍʹ͋ͬͨ͜ͱ
    ͋Δౙͷ೔ͷனɺϥϯνʹߦ͜͏ͱͨ͠ॠؒɺ
    λʔήςΟϯά΍޿ࠂ഑৴ͷফԽֹۚ౳Λ
    ؅ཧ͍ͯͨ͠KVSΫϥελ͕
    ※ ϊʔυͰ͸ͳ͘ɺΫϥελͰ͢
    ʊਓਓਓਓਓਓʊ
    ʼɹಥવͷࢮɹʻ
    ʉY^Y^Y^Y^Yʉ

    View Slide

  15. ࣮ࡍʹ͋ͬͨ͜ͱ
    ͦͷ৔ʹ͍ͨશһͰϊʔυͷ࠶ىಈΛ܁Γฦͭͭ͠ɺ
    ཪͰΠϯϑϥΤϯδχΞ͕ௐࠪ
    ىಈͯ͠΋ىಈͯ͠΋ଈࢮ͢ΔΫϥελ
    ݁ہ͸౰࣌τϥϑΟοΫͷຆͲΛ઎Ί͍ͯͨαΠτɹ
    ͔ΒλάΛണ͕ͯ͠΋Β͍࢑ఆରԠ
    → xx೔ͿΜͷച্͕·Δ·Δਧ͖ඈΜͩ

    View Slide

  16. ࠶ൃ๷ࢭͷͨΊʹͨ͜͠ͱ
    1. ୤EBS(Elastic Block Store)
    2. KVSΫϥελ΁ͷґଘΛ(෦෼తʹ)΍ΊΔ
    3.͋ΒΏΔI/OΛۃྗඇಉظʹ͢Δ
    4. ഑৴αʔό͕མͪͨ৔߹ʹnginx, λά͕र͏
    5. ͋ΒΏΔ؂ࢹ, ϩάམͱ͠Λ௥Ճ

    View Slide

  17. 1. ୤EBS(Elastic Block Store)
    2. KVSΫϥελ΁ͷґଘΛ(෦෼తʹ)΍ΊΔ
    3.͋ΒΏΔI/OΛۃྗඇಉظʹ͢Δ
    4. ഑৴αʔό͕མͪͨ৔߹ʹnginx, λά͕र͏
    5. ͋ΒΏΔ؂ࢹ, ϩάམͱ͠Λ௥Ճ
    ࠶ൃ๷ࢭͷͨΊʹͨ͜͠ͱ

    View Slide

  18. 1. ୤EBS

    View Slide

  19. 1. ୤EBS
    SSD΁ͷRead/WriteಛԽͨ͠KVSΛར༻͍ͯ͠Δ͕ɺ࢓༷ͱͯ͠
    • ϑΝΠϧγεςϜܦ༝Ͱ͸ͳ͘௚઀SSDʹॻ͖ࠐΉ
    • ֤ϊʔυͷϝϞϦ্ʹΠϯσοΫε(=KeyʹରԠͨ͠Value͕Ͳͷϊʔ
    υͷͲͷσΟεΫͷͲ͜ʹॻ͔Ε͍ͯΔ͔ͷΞυϨε৘ใ)Λ࣋ͭ
    • ΫϥελϚωδϝϯτͷػೳ΋උ͍͑ͯΔͷͰ؅ཧָ
    • ͨͩ͠ɺ֤ϊʔυʹରͯ͠͸heartbeatࢮ׆؂ࢹ͢Δ͕ɺ֤ϊʔυ͕ࢗ
    ͍ͯ͠ΔσΟεΫʹରͯ͠͸(΄΅)Կ΋͠ͳ͍

    View Slide

  20. 1. ୤EBS
    EBS = Elastic Block Store
    • EC2ʹϚ΢ϯτՄ
    • ߴ͍Մ༻ੑͱ଱ٱੑ
    • ωοτϫʔΫܦ༝ͰI/O
    • ಺෦తʹ͸৑௕Խ͞Ε͍ͯΔ
    • EC2ىಈ࣌ʹEBS࠷దԽΦϓγϣϯΛ෺ཧతʹೖΕͳ͍ͱΠϯελϯεͱEBS͕
    ෺ཧతʹ཭ΕΔ
    ো֐࣌ʹ͸EBSͷσΟεΫނোͰEBS͕ϑΣΠϧΦʔόʔதʹ਺෼ؒ໧ͬͯ͠·ͬͨ

    View Slide

  21. 1. ୤EBS
    ΠϯελϯεετΞ(=ΤϑΣϝϥϧσΟεΫ)ʹม͑ͨɻ
    • Pros
    • Πϯελϯε௚ࢗ͠ͷͨΊͱʹ͔͘଎͍
    • εϧʔϓοτ͕མͪΔཁҼ͕গͳ͍
    • Cons
    • Πϯελϯε࠶ىಈͰشൃ
    • αΠζ্ݶ͋Γ
    • Մ༻ੑ௿͍

    View Slide

  22. 2. KVSΫϥελґଘΛ΍ΊΔ

    View Slide

  23. 2. KVSΫϥελґଘΛ΍ΊΔ

    View Slide

  24. 2. KVSΫϥελґଘΛ΍ΊΔ

    • 1ͭͷKVSΫϥελ
    • SSDʹอଘ(memcached, redis౳ͱͷҧ͍)
    • ͕ɺϝϞϦʹอଘ͢ΔϓϩμΫτ૬Ԡʹ଎͍
    • ࢖్͸ओʹҎԼͷ3ͭ
    • ഑৴αʔόͷ಺෦ΦʔΫγϣϯ༻༧ࢉ؅ཧ
    • ଐੑλʔήςΟϯά x3 छྨ
    • ϦλʔήςΟϯά

    View Slide

  25. 2. KVSΫϥελґଘΛ΍ΊΔ (1)
    ഑৴αʔόͷΦʔΫγϣϯͰͷࢀরΛվળͯ͠KVSͷར༻ྔΛݮΒͨ͠
    • چ
    • ಺෦ೖࡳͷΦʔΫγϣϯຖʹඞͣফԽֹۚΛࢀরͯ͠༧ࢉͱಥ߹
    • ͜ͷ৔߹ɺKVS͕ࢮ͵ͱԿ΋ग़དྷͳ͘ͳΔ
    • ৽
    • ഑৴αʔό͕RDBMSʹ͍࣋ͬͯΔ݁Ռ͔Βܭࢉͨ͠ফԽֹۚΛఆظ
    తʹΩϟογϡ
    • ༧ࢉ͕ਚ͖ͦ͏ʹͳΔ௚લ·ͰKVSΛࢀর͠ͳ͍Α͏ʹ͢Δ
    • KVS͕ࢮΜͰ΋΄ͱΜͲͷ৔߹޿ࠂ͕ฦͤΔ

    View Slide

  26. 2. KVSΫϥελґଘΛ΍ΊΔ (2)
    • αʔϏεͱͯ͠ఏڙ͍ͯ͠ΔλʔήςΟϯάख๏͸ओʹҎԼ2ͭ
    • ଐੑλʔήςΟϯά x3छ
    • ϦλʔήςΟϯά
    • ଐੑλʔήͷ΄͏͕ѹ౗తʹྔ͸ଟ͍(10:1͘Β͍, ച্͸5:1͘Β͍)
    • ݹ͍ߏ੒Ͱ͸6000ສUUʹؔ͢ΔܭࢉΛόονͰߦ͍KVSʹॻ͘
    • ࠷੝ظ͸8࣌ؒ͘Β͍͔͔ͬͯ͠·͍ͬͯͨ
    • ͣͬͱෛՙࢼݧ͠ͳ͕Β഑৴͍ͯ͠ΔΑ͏ͳ΋ͷ

    View Slide

  27. 2. KVSΫϥελґଘΛ΍ΊΔ (2)
    ଐੑλʔήςΟϯάʹؔ͠
    ຊ౰ʹඞཁͩͬͨ΋ͷ͸ɺ୯ʹϢʔβIDΛΩʔͱͯ͠ɺଐ
    ͢ΔηάϝϯτΛฦͯ͘͠ΕΔγϯϓϧͳػೳͷΈɻߴػೳ
    ͳDBΫϥελΛ࢖͏ͷ͸एׯToo Muchɻ
    → ຊ౰ʹඞཁͳػೳ͚ͩΛ࣋ͬͨ
    ϚΠΫϩαʔϏε(Play2)Λ࡞ͬͯϦϓϨΠεͨ͠

    View Slide

  28. 2. KVSΫϥελґଘΛ΍ΊΔ (2)
    • چߏ੒ਤ

    View Slide

  29. 2. KVSΫϥελґଘΛ΍ΊΔ (2)
    • ৽ߏ੒ਤ

    View Slide

  30. 2. KVSΫϥελґଘΛ΍ΊΔ (2)

    چߏ੒ ৽ߏ੒
    λʔήςΟϯά
    ͷ࢓૊Έ
    ֎෦σʔλΛݩʹͨ͠
    όονͷܭࢉ݁ՌΛ
    ,74ʹอଘ
    SFRຖʹϦΞϧλΠϜܭࢉ
    Ұ෦
    ͸چߏ੒
    ҆ఆੑ
    σΟεΫຊ͕શମʹ
    Өڹٴ΅͢Մೳੑ
    ୆ࢮΜͰ΋Өڹͳ͠
    ,74͕ࢮΜͰ΋Өڹ
    ϨΠςϯγ dNTFD
    NTFD
    ηάϝϯςʔγϣϯॲཧࠐΈ

    ίετ
    SYMBSHF୆
    ଞͷ໨తؚΉ

    DYMBSHF୆
    SYMBSHF୆

    View Slide

  31. 2. KVSΫϥελґଘΛ΍ΊΔ (2)

    ϦΞϧλΠϜηάϝϯςʔγϣϯͷͳ͔Έ
    • ࣄલʹόοναʔόͰଐੑσʔλΛRocksDBͱͯ͠
    ݻΊΔ
    • RocksDBΛWebαʔό͕ಡΈࠐΉ
    • httpϦΫΤετຖʹRocksDB͔ΒϢʔβଐੑΛಡΈ
    ࠐΜͰηάϝϯςʔγϣϯͯ͠ฦ٫

    View Slide

  32. 2. KVSΫϥελґଘΛ΍ΊΔ (2)
    ͳͥRocksDB͔?
    • RocksDB : FacebookͷMySQLΤϯδϯ(MyRocks)΍ࠓ͸๢͖Parse
    ͷmongodbͷόοΫΤϯυͰར༻͞Ε͍ͯΔSSD΁ͷWriteʹ࠷ద
    Խ͞Εͨ૊ΈࠐΈ༻DB࣮૷ɻLevelDBͷforkɻ
    • ૊ΈࠐΈͰ, ߴ଎ʹ, େྔͳσʔλ͔ΒҾ͘࢓૊ΈΛٻΊͯ
    • ಉ༷ʹݕ౼ͨ͠h2dbͱͷൺֱͰɺएׯύϑΥʔϚϯε͕ྑ͔ͬͨ͜
    ͱɺσʔλͷswap(ଐੑσʔλͷ೔࣍Ξοϓσʔτ)࣌ͷύϑΥʔϚ
    ϯεྼԽ౓߹͍͕௿͔ͬͨ͜ͱ͔Β൑அ

    View Slide

  33. 2. KVSΫϥελґଘΛ΍ΊΔ (2)

    h2dbͱͷൺֱͰ
    एׯɺRocksDBͷ΄͏͕ૣ͘ɺϦιʔεফඅ͕গͳ͔ͬͨ

    View Slide

  34. ͜͜·Ͱͷ·ͱΊ
    μ΢ϯ͠ͳ͍ɺμ΢ϯλΠϜΛۃྗগͳ͘͢ΔͨΊʹͲ͏͢Δ͔?
    • ΠϯϑϥɺϛυϧɺΞϓϦέʔγϣϯશͯγϯϓϧͳߏ੒ʹ
    • ͨ·ʹϨΠςϯγΛଟগ٘ਜ਼ʹͯ͠Ͱ΋҆ఆੑΛऔΔ
    • I/O͸ग़དྷΔݶΓඇಉظʹͯ͠ෛͷ࿈࠯Λ๷͙
    • EBS͸ϋΠύϑΥʔϚϯε؀ڥͰ͸ආ͚Δ
    • શͯΛτϨʔεग़དྷΔ࢓૊ΈΛ੔͑Δ
    • ͍͟ͱ͍͏࣌ʹϢʔβ͚ͩʹ͸໎࿭Λ͔͚ͳ͍Α͏ʹ͢Δ

    View Slide

  35. ͦΕ͚ͩͰOK???

    View Slide

  36. No.
    όάʹΑΔো֐Λ
    ͍͔ʹͯ͠๷͙ͷ͔?

    View Slide

  37. ཧ૝͸্ྲྀͰ๷͍͗ͨ
    ҰํͰ…

    View Slide

  38. ΤϯδχΞ͸ۀ຿ͷ
    ౰ࣄऀͰͳ͍ͨΊʹ
    ཁ݅Λਖ਼֬ʹཧղ͠೉͍͠

    View Slide

  39. ۀ຿஌ࣝෆ଍໰୊
    ΤϯδχΞͷۀ຿஌ࣝෆ଍ΛͲ͏ิ͏͔?
    • جຊతʹ͸ΤϯδχΞ΋ؤு֮ͬͯ͑Α͏ελϯε
    • DDD(Domain Driven Design)Λ΍Δͱ͍͏લఏ
    • ॳظͷۀ຿஌ࣝͷֶशίετͷߴ͕͞ωοΫʹͳΔ৔߹΋
    • PM(ProductManager)ͷ຋༁ೳྗ, ۀ຿ઃܭྗʹৼΔ
    • αϘΒͣʹ࢓༷ॻΛॻ͘ମ੍

    View Slide

  40. ߴ͍຋༁ೳྗΛ࣋ͭPM
    Fringe81Ͱ͸ݩʑ3PASΞφϦετͩͬͨϝϯόʔ͕PMΛ΍Δ
    • ݩʑઆ໌ೳྗ͕ߴ͍ͱ͜Ζʹ
    • ඇΤϯδχΞ͕ͩߴ͍ٕज़ཧղྗΛ͍࣋ͬͯΔ
    • ࣾ಺޲͚Ϩϙʔτఔ౓ͳΒউखʹSQLୟ͍ͯ࡞ΕΔ
    ݁ՌɺΤϯδχΞ͸
    • ػೳཁ݅ͷೝࣝζϨɺඇػೳཁ݅ͷݕ౼࿙Ε͕ݮΔ
    • ຊ࣭తͳ໰୊(ઃܭ΍ύϑΥʔϚϯεվળ)ʹूதͰ͖Δ

    View Slide

  41. αϘΒͣʹ࢓༷ॻΛॻ͘ମ੍
    ϕϯνϟʔͰ͸௝͘͠?
    ͔ͳΓ࢓༷ॻΛॻ͘จԽ(AdνʔϜ͸ಛʹ)ɻ
    ݹయతͳ΍ΓํΛ࣮֬ʹண࣮ʹ͜ͳ্͠ྲྀͷ
    ໰୊ΛݮΒ͢ɻΞδϟΠϧͰ΋جຊతʹ΍Δ
    ͜ͱ͸Ұॹɻ
    ※ શͯͷ࢓༷ॻ͸textlintΛCIͰ࣮ߦͯ͠ʹΛ
    ͸΍υϝΠϯ༻ޠͷϒϨΛݕࠪ

    → ݁ՌISO 9001(඼࣭ϚωδϝϯτγεςϜ)Λऔಘͨ͠

    View Slide

  42. ৽͍͠νϟϨϯδ
    • QAνʔϜͷઃஔ : ͞ΒͳΔ඼࣭ͷ޲্
    • ઃܭ/ςετͷϨϏϡʔͱ୳ࡧతςετͷ࣮ࢪ
    • ΤϯδχΞ͔ΒPM΁ : ࣄۀܥPM΁ͷґଘ౓ΛԼ͛Δ
    • ओʹ৽نࣄۀํ໘Ͱൈṅ
    • ఆظతͳϙδγϣϯνΣϯδ : εΩϧτϥϯεϑΝʔ
    • ໾ׂݻఆ͢ΔͱಛఆྖҬͷٕज़/ۀ຿஌͔͔ࣝͭ͠ͳ͍ͷΛ๷͙
    • ௚ۙͰ͸؅ཧը໘ϝϯόʔͱ഑৴αʔόϝϯόʔΛεϫοϓ

    View Slide

  43. ·ͱΊ
    HARD THINGS = ো֐ʹର͠ɺٕज़ͱ૊৫ͷ2໘͔Βཱͪ޲͔ͬͨ
    • ٕज़
    • γϯϓϧ͔ͭFail͕఻೻͠ͳ͍ઃܭ
    • λʔήςΟϯάج൫ΛKVSΫϥελ͔ΒWebAppʹ
    • ૊৫
    • ۀ຿஌ࣝ + ٕज़΁ͷཧղྗߴ͍PMͱͷڠௐ
    • ݹయతͳ΍ΓํΛ࣮֬ʹண࣮ʹ΍ΔେਓελΠϧ
    • QAνʔϜ, ΤϯδχΞൃPM౳ͷ৽͍͠૊৫ମ੍Ͱຏ͘

    View Slide

  44. ँࣙ
    ো֐ൃੜ࣌ʹࣾ֎΁ͷରԠΛ͍͍ͯͨͩͨ͠Productνʔ
    Ϝ΍ɺมΘΒͣӦۀΛଓ͚͍͍ͯͨͩͨSalesνʔϜ͕
    ͋ͬͯͦ͜ɺো֐Λ৐Γ੾Δ͜ͱ͕ग़དྷ·ͨ͠ɻ
    ·ͨɺ೔ࠒΑΓ୔ࢁͷϑΟʔυόοΫΛ͍͍ͯͨͩͯ͠
    ͍Δ͓͔͛Ͱɺ૊৫Λม͑ΔΞΫγϣϯ͕औΕ͍ͯΔ͜
    ͱΛײँ͓ͯ͠Γ·͢ɻ͋Γ͕ͱ͏͍͟͝·͢ɻ

    View Slide

  45. THANK YOU—
    Fringe81Ͱ͸ɺ৽͍͠ൃݟΛݩʹ஍ٿͷະདྷΛ
    ૑ΒΜͱ͢ΔΤϯδχΞΛืूதͰ͢ɻ
    ͋ͳͨͷಘҙͳ͜ͱͰҰॹʹউෛ͠·ͤΜ͔ɻ
    http://www.fringe81.com/recruit/

    View Slide