SmartNews Ads における機械学習の活用とその運用 #mlops

by KOMIYA Atsushi

Slide 1

Slide 1 text

SmartNews Ads ʹ͓͚Δ ػցֶशͷ׆༻ͱͦͷӡ༻ 2017-12-19 ML ops study #1 KOMIYA Atsushi

Slide 2

Slide 2 text

@komiya_atsushi

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

લఏɾഎܠ

Slide 5

Slide 5 text

୭͕ػցֶशΛѻ͍ͬͯΔͷ͔ʁ • SmartNews ʹ͓͍ͯػցֶशΛѻ͏ͷ͸  ιϑτ΢ΣΞΤϯδχΞͷ͓࢓ࣄ • ϓϩμΫγϣϯ඼࣭ͷίʔυͰ࣮૷͠ɺ  γεςϜΛӡ༻Ͱ͖Δ͜ͱ͕લఏ • ʮSmartNews ͷΤϯδχΞ͸શһػցֶश Ͱ͖Δʯͱ͍͏ҙຯͰ͸ͳ͍͜ͱʹ஫ҙ

Slide 6

Slide 6 text

୭͕ػցֶशΛѻ͍ͬͯΔͷ͔ʁ • Ϟσϧ։ൃ͚ͩΛ͢ΔʮσʔλαΠΤϯςΟετʯత ͳϝϯόʔ͸͍ͳ͍ • Ϟσϧ։ൃ͔ΒϓϩμΫγϣϯల։ɺ෼ੳ·Ͱ͕Τ ϯδχΞͷ͓࢓ࣄ • ʮσʔλαΠΤϯςΟετʯ͕͍ͳ͍ɺͱ͍͏Θ͚ Ͱ͸ͳ͍͜ͱʹ஫ҙ • ֤छ෼ੳΛಘҙͱ͢ΔʮσʔλαΠΤϯςΟετʯ ͸͍Δ

Slide 7

Slide 7 text

ͲͷΑ͏ͳ؀ڥͰػցֶशͯ͠Δͷ͔ʁ • Uber ͷ Michelangelo Έ͍ͨʹચ࿅͞Εͨ؀ڥͰ͸ͳ͍ • ࢀߟ: https://eng.uber.com/michelangelo/ • DMP Λߏங͠ɺͦͷ্ͰػցֶशλεΫΛ͜ͳͯ͠Δ • σʔλετΞ: S3 (ϩά), RDS (Ϛελσʔλ) • ෼ࢄॲཧ: Hive / Spark on EMR, Presto • ϫʔΫϑϩʔ؅ཧ: Azkaban

Slide 8

Slide 8 text

ʮ"%%BUB&OHJOFFSJOHJOQSBDUJDF4NBSU/FXT"ETཪͷσʔλγεςϜʯΑΓ IUUQCJUMZ4/5FDI/JHIU-BO

Slide 9

Slide 9 text

ͲͷΑ͏ʹ։ൃ͍ͯ͠Δͷ͔ʁ • ಛʹ޿ࠂपΓͷγεςϜʹ͓͍ͯ͸ɺߴ͍ଐਓੑͷ΋ͱͰͷ ։ൃ͕ߦΘΕ͍ͯΔ • ։ൃεϐʔυΛ༏ઌ͠ɺଐਓੑΛڐ༰ • ͜ͷ͋ͨΓͷҙࢥܾఆʹ͍ͭͯ͸ɺৄ͘͠͸ SmartNews Ads େਤղ (SmartNews TechNight vol.5) ͷൃදࢿྉΛࢀর • https://www.slideshare.net/smartnews/smartnews- technight-vol5-smartnews-ads

Slide 10

Slide 10 text

͋ΘͤͯಡΈ͍ͨൃදࢿྉ • SmartNews ͷσʔλαΠΤϯςΟετͷ  ߴ଎ΠςϨʔγϣϯΛࢧ͑Δ޿ࠂγεςϜ  (AWS Summit Tokyo 2015) • http://bit.ly/SN-AWS2015 • AD Data Engineering in practice: SmartNews Ads ཪͷσʔλγεςϜ (SmartNews TechNight vol.5) • http://bit.ly/SN-TechNight5-Lan

Slide 11

Slide 11 text

SmartNews Ads ʹ͓͚Δ ػցֶशͷ׆༻ɾӡ༻ࣄྫ

Slide 12

Slide 12 text

લॲཧ / ֶश / ༧ଌ / ධՁ ͦΕͧΕʹ͍͓ͭͯ࿩͠·͢

Slide 13

Slide 13 text

Ͳ͜ͰػցֶशΛར༻͍ͯ͠Δͷ͔ʁ • ʢҎԼ͸Ұྫʣ • ༧ଌ୯Ґ: Ϣʔβ • ੑผਪఆ, ೥ྸਪఆ • ༧ଌ୯Ґ: Ϣʔβ x ޿ࠂ • ΫϦοΫ཰༧ଌ, ίϯόʔδϣϯ཰༧ଌ • ޿ࠂ഑৴ͷϩάͳͲΛར༻ֶͯ͠श͍ͯ͠Δ

Slide 14

Slide 14 text

ͲͷΑ͏ʹલॲཧ͍ͯ͠Δͷ͔ʁ "ETFSWFST JNQ DMJDL DW -PHCVDLFU %.1CVDLFU "EEFMJWFSZ MPH +PJO )JWF2- 'FBUVSFFYUSBDUJPO )JWF2- 'FBUVSFT JOUFSNFEJBUF %BUBTFU -JC47. GPSNBU (FOFSBUFEBUBTFU GPSFBDI.-UBTL )JWF2-

Slide 15

Slide 15 text

ͲͷΑ͏ʹֶशɾ༧ଌ͍ͯ͠Δͷ͔ʁ • ֶश͸ʢࠓͷͱ͜Ζʣ͢΂ͯΦϑϥΠϯͰॲཧ͍ͯ͠Δ • ༧ଌ͸ର৅ͷλεΫʹΑͬͯΦϑϥΠϯͱΦϯϥΠϯΛ ࢖͍෼͚͍ͯΔ • ΦϑϥΠϯ༧ଌ • ੑผਪఆ, ೥ྸਪఆ • ΦϯϥΠϯ༧ଌ • ΫϦοΫ཰༧ଌ, ίϯόʔδϣϯ཰༧ଌ

Slide 16

Slide 16 text

ͲͷΑ͏ʹֶश͍ͯ͠Δͷ͔ʁ %.1CVDLFU %BUBTFU -JC47. GPSNBU &$*OTUBODFGPSUSBJOJOH 1SFEJDUJPO NPEFM 5SBJO 1SFEJDUJPO NPEFM ೔ຖʹ ༧ଌϞσϧΛอ࣋

Slide 17

Slide 17 text

ͲͷΑ͏ʹ༧ଌ͍ͯ͠Δͷ͔ʁ %.1CVDLFU 'FBUVSFT JOUFSNFEJBUF 1SFEJDUJPO SFTVMU 1SFEJDUBMMJUFNT )JWF2-X6%' "VEJFODFUBCMF 8SJUFSFTVMU )JWF2-X6%' ※ΦϑϥΠϯ༧ଌͷ৔߹ 1SFEJDUJPO NPEFM

Slide 18

Slide 18 text

ͲͷΑ͏ʹΦϑϥΠϯ༧ଌͷ݁ՌΛ ར༻͢Δͷ͔ʁ • ޿ࠂϦΫΤετͷ౎౓ɺDynamoDB ͔Β item Λඇಉظతʹऔಘ͍ͯ͠Δ • Ωϟογϡ͸͍ͯ͠ͳ͍ (ͯ͠΋ҙຯ͕ͳ͍)

Slide 19

Slide 19 text

͔͜͜Βઌ͸ΦϯϥΠϯ༧ଌ ʹಛԽͨ͠࿩ʹͳΓ·͢

Slide 20

Slide 20 text

ͲͷΑ͏ʹΦϯϥΠϯ༧ଌ͍ͯ͠Δͷ͔ʁ Ϣʔβ" ộ ޿ࠂ ޿ࠂ ޿ࠂ ޿ࠂO ʴ Ϣʔβͷಛ௃ྔ ޿ࠂ͝ͱͷಛ௃ྔ ༧ଌϞσϧ ộ ༧ଌ݁Ռ %ZOBNP%# QFSVTFS QFSUJNFVOJU 4+7.IFBQ QFSUBTL 4+7.IFBQ QFSBE

Slide 21

Slide 21 text

ͲͷΑ͏ʹಛ௃ϕΫτϧΛѻ͍ͬͯΔͷ͔ʁ • ༧ଌʹඞཁͳಛ௃ϕΫτϧΛ෼ղͨ͠ঢ়ଶͰ֤छετϨʔ δʹͯอ࣋͢Δ • Ϣʔβͷಛ௃ྔ: DynamoDB (όον/ϦΞϧλΠϜߋ৽) • ޿ࠂͷಛ௃ྔ: S3 (JVM ώʔϓ্ͰΩϟογϡ) • ޿ࠂϦΫΤετͷதͰɺϢʔβ༝དྷͷಛ௃ྔΛ DynamoDB ͔Βऔಘ → ޿ࠂ͝ͱʹಛ௃ϕΫτϧΛ࡞ͬͯ ͦΕͧΕ༧ଌɺͱ͍ͯ͠Δ

Slide 22

Slide 22 text

ͲͷΑ͏ʹಛ௃ϕΫτϧΛ४උ͍ͯ͠Δͷ͔ʁ "ETFSWFST JNQ DMJDL DW ,JOFTJT TUSFBNT 4QBSLTUSFBNJOH PO&.3 6TFS GFBUVSF 8SJUFFWFSZ NJOVUF ※Ϣʔβಛ௃ྔͷϦΞϧλΠϜߋ৽

Slide 23

Slide 23 text

ͲͷΑ͏ʹ༧ଌϞσϧΛѻ͍ͬͯΔͷ͔ʁ • ֶशࡁΈͷ༧ଌϞσϧ͸ S3 όέοτ্ʹଘࡏ͢Δ • ޿ࠂαʔό͸ͦͷ༧ଌϞσϧΛμ΢ϯϩʔυ & ϩʔυͯ͠ ώʔϓ্Ͱอ࣋͢Δ • ༧ଌػೳ͸޿ࠂαʔόʹ಺แ͞Ε͍ͯΔ (͔ͳΓີ݁߹) • github.com/komiya-atsushi/xgboost-predictor-java • ༧ଌϞσϧ͸޿ࠂαʔόΛఀࢭͤͣʹϦϩʔυͰ͖Δ

Slide 24

Slide 24 text

ͲͷΑ͏ʹ༧ଌϞσϧͷ A/B ςετΛ ࣮ݱ͍ͯ͠Δͷ͔ʁ • ޿ࠂαʔό͸ෳ਺ͷ༧ଌϞσϧΛಉ࣌ʹϩʔυͰ͖ΔػߏΛ උ͍͑ͯΔ • ͦΕͧΕͷ༧ଌϞσϧʹʮϥϕϧʯ͕෇༩͞Ε͍ͯΔ • A/B ςετͷϑϨʔϜϫʔΫͰͦͷʮϥϕϧʯΛࢦఆ͢Δ͜ ͱͰɺ࣮ݧ܈͝ͱʹར༻͢Δ༧ଌϞσϧΛ੾Γସ͑ΒΕΔ • DynamoDB ্ͷಛ௃ྔʹ΋ϥϕϧ͕෇༩͞Ε͍ͯΔͷ Ͱɺ༧ଌϞσϧʹ߹Θͤͨಛ௃ྔΛ༻ҙ͢Δ͜ͱ͕Ͱ͖Δ

Slide 25

Slide 25 text

ͲͷΑ͏ʹධՁ͍ͯ͠Δͷ͔ʁ • ༧ଌ݁Ռ vs ࣮੷ͷධՁϝτϦΫεΛ daily/hourly Ͱࢉग़ͯ͠ Slack νϟϯωϧʹ௨஌ • ධՁϝτϦΫεͷྫ • Normalized entropy • Logloss • AUC

Slide 26

Slide 26 text

ͳͥΦϯϥΠϯ༧ଌ͢Δͷ͔ʁ • ΦϯϥΠϯ༧ଌ͸γεςϜతʹෳࡶʹͳΓ͕ͪ & Πϯϑϥ ౳ͷίετ͕͔͔Γ͕ͪͱ͍͏σϝϦοτ͕͋Δ • ޿ࠂʹ͓͚ΔΫϦοΫ཰ɺίϯόʔδϣϯ཰༧ଌ͸ɺ୹ظత ʹ΋த௕ظతʹ΋ച্ʹ௓Ͷฦͬͯ͘Δࢪࡦͱ͍͏͜ͱͰऔ Γ૊ΜͰ͍Δ • ୹ظతɿظ଴ऩӹͷߴ͍޿ࠂΛ഑৴͢ΔͨΊ • த௕ظతɿ޿ࠂओ (୅ཧళ) ʹྑ͍ύϑΥʔϚϯε (҆ՁͰ ଟ͘ͷ֫ಘ) Λఏڙ͠ɺܧଓͯ͠޿ࠂग़ߘ͍ͨͩͨ͘Ί

Slide 27

Slide 27 text

·ͱΊ

Slide 28

Slide 28 text

·ͱΊ • AWS ͷ֤छαʔϏε & Φʔϓϯιʔεϓϩμ ΫτΛ׆༻ͯ͠ DMP Λߏங͠ɺػցֶशλε ΫΛ͜ͳ͍ͯ͠Δ • গ਺ਫ਼ӶͰ։ൃΛਐΊ͍ͯΔ͕Ώ͑ɺػցֶ शͷϓϥοτϑΥʔϜͱͯ͠͸·ͩվળͷ༨ ஍͕͋Δ͸ͣ

Slide 29

Slide 29 text

We’re HIRING ❗❗❗ • bit.ly/SmartNews-Hiring • Data scientist • Data engineer • Software engineer, Ads machine learning • Software engineer, Machine learning (news)

Slide 30

Slide 30 text

Thank you!