Upgrade to Pro — share decks privately, control downloads, hide ads and more …

SmartNews Ads における機械学習の活用とその運用 #mlops

SmartNews Ads における機械学習の活用とその運用 #mlops

ML Ops Study # 1 https://ml-ops.connpass.com/event/73814/ における発表資料です。SmartNews Ads でどのように機械学習を活用・運用しているのか、そのありのままの姿をお話しました。

SmartNews でのお仕事にもしご興味がありましたら
https://smartnews.workable.com/ を御覧くださいませ 😳

KOMIYA Atsushi

December 20, 2017
Tweet

More Decks by KOMIYA Atsushi

Other Decks in Technology

Transcript

  1. SmartNews Ads ʹ͓͚Δ
    ػցֶशͷ׆༻ͱͦͷӡ༻
    2017-12-19 ML ops study #1
    KOMIYA Atsushi

    View full-size slide

  2. @komiya_atsushi

    View full-size slide

  3. લఏɾഎܠ

    View full-size slide

  4. ୭͕ػցֶशΛѻ͍ͬͯΔͷ͔ʁ
    • SmartNews ʹ͓͍ͯػցֶशΛѻ͏ͷ͸

    ιϑτ΢ΣΞΤϯδχΞͷ͓࢓ࣄ
    • ϓϩμΫγϣϯ඼࣭ͷίʔυͰ࣮૷͠ɺ

    γεςϜΛӡ༻Ͱ͖Δ͜ͱ͕લఏ
    • ʮSmartNews ͷΤϯδχΞ͸શһػցֶश
    Ͱ͖Δʯͱ͍͏ҙຯͰ͸ͳ͍͜ͱʹ஫ҙ

    View full-size slide

  5. ୭͕ػցֶशΛѻ͍ͬͯΔͷ͔ʁ
    • Ϟσϧ։ൃ͚ͩΛ͢ΔʮσʔλαΠΤϯςΟετʯత
    ͳϝϯόʔ͸͍ͳ͍
    • Ϟσϧ։ൃ͔ΒϓϩμΫγϣϯల։ɺ෼ੳ·Ͱ͕Τ
    ϯδχΞͷ͓࢓ࣄ
    • ʮσʔλαΠΤϯςΟετʯ͕͍ͳ͍ɺͱ͍͏Θ͚
    Ͱ͸ͳ͍͜ͱʹ஫ҙ
    • ֤छ෼ੳΛಘҙͱ͢ΔʮσʔλαΠΤϯςΟετʯ
    ͸͍Δ

    View full-size slide

  6. ͲͷΑ͏ͳ؀ڥͰػցֶशͯ͠Δͷ͔ʁ
    • Uber ͷ Michelangelo Έ͍ͨʹચ࿅͞Εͨ؀ڥͰ͸ͳ͍
    • ࢀߟ: https://eng.uber.com/michelangelo/
    • DMP Λߏங͠ɺͦͷ্ͰػցֶशλεΫΛ͜ͳͯ͠Δ
    • σʔλετΞ: S3 (ϩά), RDS (Ϛελσʔλ)
    • ෼ࢄॲཧ: Hive / Spark on EMR, Presto
    • ϫʔΫϑϩʔ؅ཧ: Azkaban

    View full-size slide

  7. ʮ"%%BUB&OHJOFFSJOHJOQSBDUJDF4NBSU/FXT"ETཪͷσʔλγεςϜʯΑΓ
    IUUQCJUMZ4/5FDI/JHIU-BO

    View full-size slide

  8. ͲͷΑ͏ʹ։ൃ͍ͯ͠Δͷ͔ʁ
    • ಛʹ޿ࠂपΓͷγεςϜʹ͓͍ͯ͸ɺߴ͍ଐਓੑͷ΋ͱͰͷ
    ։ൃ͕ߦΘΕ͍ͯΔ
    • ։ൃεϐʔυΛ༏ઌ͠ɺଐਓੑΛڐ༰
    • ͜ͷ͋ͨΓͷҙࢥܾఆʹ͍ͭͯ͸ɺৄ͘͠͸ SmartNews
    Ads େਤղ (SmartNews TechNight vol.5) ͷൃදࢿྉΛࢀর
    • https://www.slideshare.net/smartnews/smartnews-
    technight-vol5-smartnews-ads

    View full-size slide

  9. ͋ΘͤͯಡΈ͍ͨൃදࢿྉ
    • SmartNews ͷσʔλαΠΤϯςΟετͷ

    ߴ଎ΠςϨʔγϣϯΛࢧ͑Δ޿ࠂγεςϜ

    (AWS Summit Tokyo 2015)
    • http://bit.ly/SN-AWS2015
    • AD Data Engineering in practice: SmartNews Ads
    ཪͷσʔλγεςϜ (SmartNews TechNight vol.5)
    • http://bit.ly/SN-TechNight5-Lan

    View full-size slide

  10. SmartNews Ads ʹ͓͚Δ
    ػցֶशͷ׆༻ɾӡ༻ࣄྫ

    View full-size slide

  11. લॲཧ / ֶश / ༧ଌ / ධՁ
    ͦΕͧΕʹ͍͓ͭͯ࿩͠·͢

    View full-size slide

  12. Ͳ͜ͰػցֶशΛར༻͍ͯ͠Δͷ͔ʁ
    • ʢҎԼ͸Ұྫʣ
    • ༧ଌ୯Ґ: Ϣʔβ
    • ੑผਪఆ, ೥ྸਪఆ
    • ༧ଌ୯Ґ: Ϣʔβ x ޿ࠂ
    • ΫϦοΫ཰༧ଌ, ίϯόʔδϣϯ཰༧ଌ
    • ޿ࠂ഑৴ͷϩάͳͲΛར༻ֶͯ͠श͍ͯ͠Δ

    View full-size slide

  13. ͲͷΑ͏ʹલॲཧ͍ͯ͠Δͷ͔ʁ
    "ETFSWFST
    JNQ
    DMJDL
    DW
    -PHCVDLFU %.1CVDLFU
    "EEFMJWFSZ
    MPH
    +PJO
    )JWF2-

    'FBUVSFFYUSBDUJPO
    )JWF2-

    'FBUVSFT
    JOUFSNFEJBUF

    %BUBTFU
    -JC47.
    GPSNBU

    (FOFSBUFEBUBTFU
    GPSFBDI.-UBTL
    )JWF2-

    View full-size slide

  14. ͲͷΑ͏ʹֶशɾ༧ଌ͍ͯ͠Δͷ͔ʁ
    • ֶश͸ʢࠓͷͱ͜Ζʣ͢΂ͯΦϑϥΠϯͰॲཧ͍ͯ͠Δ
    • ༧ଌ͸ର৅ͷλεΫʹΑͬͯΦϑϥΠϯͱΦϯϥΠϯΛ
    ࢖͍෼͚͍ͯΔ
    • ΦϑϥΠϯ༧ଌ
    • ੑผਪఆ, ೥ྸਪఆ
    • ΦϯϥΠϯ༧ଌ
    • ΫϦοΫ཰༧ଌ, ίϯόʔδϣϯ཰༧ଌ

    View full-size slide

  15. ͲͷΑ͏ʹֶश͍ͯ͠Δͷ͔ʁ
    %.1CVDLFU
    %BUBTFU
    -JC47.
    GPSNBU

    &$*OTUBODFGPSUSBJOJOH
    1SFEJDUJPO
    NPEFM
    5SBJO
    1SFEJDUJPO
    NPEFM
    ೔ຖʹ
    ༧ଌϞσϧΛอ࣋

    View full-size slide

  16. ͲͷΑ͏ʹ༧ଌ͍ͯ͠Δͷ͔ʁ
    %.1CVDLFU
    'FBUVSFT
    JOUFSNFEJBUF

    1SFEJDUJPO
    SFTVMU
    1SFEJDUBMMJUFNT
    )JWF2-X6%'

    "VEJFODFUBCMF
    8SJUFSFTVMU
    )JWF2-X6%'

    ※ΦϑϥΠϯ༧ଌͷ৔߹
    1SFEJDUJPO
    NPEFM

    View full-size slide

  17. ͲͷΑ͏ʹΦϑϥΠϯ༧ଌͷ݁ՌΛ
    ར༻͢Δͷ͔ʁ
    • ޿ࠂϦΫΤετͷ౎౓ɺDynamoDB ͔Β
    item Λඇಉظతʹऔಘ͍ͯ͠Δ
    • Ωϟογϡ͸͍ͯ͠ͳ͍ (ͯ͠΋ҙຯ͕ͳ͍)

    View full-size slide

  18. ͔͜͜Βઌ͸ΦϯϥΠϯ༧ଌ
    ʹಛԽͨ͠࿩ʹͳΓ·͢

    View full-size slide

  19. ͲͷΑ͏ʹΦϯϥΠϯ༧ଌ͍ͯ͠Δͷ͔ʁ
    Ϣʔβ"

    ޿ࠂ
    ޿ࠂ
    ޿ࠂ
    ޿ࠂO
    ʴ
    Ϣʔβͷಛ௃ྔ
    ޿ࠂ͝ͱͷಛ௃ྔ
    ༧ଌϞσϧ





    ༧ଌ݁Ռ
    %ZOBNP%#
    QFSVTFS
    QFSUJNFVOJU
    4+7.IFBQ
    QFSUBTL
    4+7.IFBQ QFSBE

    View full-size slide

  20. ͲͷΑ͏ʹಛ௃ϕΫτϧΛѻ͍ͬͯΔͷ͔ʁ
    • ༧ଌʹඞཁͳಛ௃ϕΫτϧΛ෼ղͨ͠ঢ়ଶͰ֤छετϨʔ
    δʹͯอ࣋͢Δ
    • Ϣʔβͷಛ௃ྔ: DynamoDB (όον/ϦΞϧλΠϜߋ৽)
    • ޿ࠂͷಛ௃ྔ: S3 (JVM ώʔϓ্ͰΩϟογϡ)
    • ޿ࠂϦΫΤετͷதͰɺϢʔβ༝དྷͷಛ௃ྔΛ
    DynamoDB ͔Βऔಘ → ޿ࠂ͝ͱʹಛ௃ϕΫτϧΛ࡞ͬͯ
    ͦΕͧΕ༧ଌɺͱ͍ͯ͠Δ

    View full-size slide

  21. ͲͷΑ͏ʹಛ௃ϕΫτϧΛ४උ͍ͯ͠Δͷ͔ʁ
    "ETFSWFST
    JNQ
    DMJDL DW
    ,JOFTJT
    TUSFBNT
    4QBSLTUSFBNJOH
    PO&.3
    6TFS
    GFBUVSF
    8SJUFFWFSZ
    NJOVUF
    ※Ϣʔβಛ௃ྔͷϦΞϧλΠϜߋ৽

    View full-size slide

  22. ͲͷΑ͏ʹ༧ଌϞσϧΛѻ͍ͬͯΔͷ͔ʁ
    • ֶशࡁΈͷ༧ଌϞσϧ͸ S3 όέοτ্ʹଘࡏ͢Δ
    • ޿ࠂαʔό͸ͦͷ༧ଌϞσϧΛμ΢ϯϩʔυ & ϩʔυͯ͠
    ώʔϓ্Ͱอ࣋͢Δ
    • ༧ଌػೳ͸޿ࠂαʔόʹ಺แ͞Ε͍ͯΔ (͔ͳΓີ݁߹)
    • github.com/komiya-atsushi/xgboost-predictor-java
    • ༧ଌϞσϧ͸޿ࠂαʔόΛఀࢭͤͣʹϦϩʔυͰ͖Δ

    View full-size slide

  23. ͲͷΑ͏ʹ༧ଌϞσϧͷ A/B ςετΛ
    ࣮ݱ͍ͯ͠Δͷ͔ʁ
    • ޿ࠂαʔό͸ෳ਺ͷ༧ଌϞσϧΛಉ࣌ʹϩʔυͰ͖ΔػߏΛ
    උ͍͑ͯΔ
    • ͦΕͧΕͷ༧ଌϞσϧʹʮϥϕϧʯ͕෇༩͞Ε͍ͯΔ
    • A/B ςετͷϑϨʔϜϫʔΫͰͦͷʮϥϕϧʯΛࢦఆ͢Δ͜
    ͱͰɺ࣮ݧ܈͝ͱʹར༻͢Δ༧ଌϞσϧΛ੾Γସ͑ΒΕΔ
    • DynamoDB ্ͷಛ௃ྔʹ΋ϥϕϧ͕෇༩͞Ε͍ͯΔͷ
    Ͱɺ༧ଌϞσϧʹ߹Θͤͨಛ௃ྔΛ༻ҙ͢Δ͜ͱ͕Ͱ͖Δ

    View full-size slide

  24. ͲͷΑ͏ʹධՁ͍ͯ͠Δͷ͔ʁ
    • ༧ଌ݁Ռ vs ࣮੷ͷධՁϝτϦΫεΛ daily/hourly
    Ͱࢉग़ͯ͠ Slack νϟϯωϧʹ௨஌
    • ධՁϝτϦΫεͷྫ
    • Normalized entropy
    • Logloss
    • AUC

    View full-size slide

  25. ͳͥΦϯϥΠϯ༧ଌ͢Δͷ͔ʁ
    • ΦϯϥΠϯ༧ଌ͸γεςϜతʹෳࡶʹͳΓ͕ͪ & Πϯϑϥ
    ౳ͷίετ͕͔͔Γ͕ͪͱ͍͏σϝϦοτ͕͋Δ
    • ޿ࠂʹ͓͚ΔΫϦοΫ཰ɺίϯόʔδϣϯ཰༧ଌ͸ɺ୹ظత
    ʹ΋த௕ظతʹ΋ച্ʹ௓Ͷฦͬͯ͘Δࢪࡦͱ͍͏͜ͱͰऔ
    Γ૊ΜͰ͍Δ
    • ୹ظతɿظ଴ऩӹͷߴ͍޿ࠂΛ഑৴͢ΔͨΊ
    • த௕ظతɿ޿ࠂओ (୅ཧళ) ʹྑ͍ύϑΥʔϚϯε (҆ՁͰ
    ଟ͘ͷ֫ಘ) Λఏڙ͠ɺܧଓͯ͠޿ࠂग़ߘ͍ͨͩͨ͘Ί

    View full-size slide

  26. ·ͱΊ
    • AWS ͷ֤छαʔϏε & Φʔϓϯιʔεϓϩμ
    ΫτΛ׆༻ͯ͠ DMP Λߏங͠ɺػցֶशλε
    ΫΛ͜ͳ͍ͯ͠Δ
    • গ਺ਫ਼ӶͰ։ൃΛਐΊ͍ͯΔ͕Ώ͑ɺػցֶ
    शͷϓϥοτϑΥʔϜͱͯ͠͸·ͩվળͷ༨
    ஍͕͋Δ͸ͣ

    View full-size slide

  27. We’re HIRING ❗❗❗
    • bit.ly/SmartNews-Hiring
    • Data scientist
    • Data engineer
    • Software engineer, Ads machine learning
    • Software engineer, Machine learning (news)

    View full-size slide