ネット広告配信サービスにおけるMLシステムの実装と運用

Machine Learning System on Online Advertising Platform.

Transcript

1. ωοτ޿ࠂ഑৴αʔϏεʹ͓͚Δ
   .-γεςϜͷ࣮૷ͱӡ༻
   5BLBTIJ/JTIJCBZBTIJ
   
   .BDIJOF-FBSOJOH$BTVBM5BMLT
   

   View Slide

2. ͓લͩΕΑ
   Name: Takashi Nishibayashi
   twitter.com/@hagino3000
   Job: Software Engineer
   VOYAGE GROUPͰωοτ޿ࠂ഑৴αʔϏε࡞ͬͯ
   ·͢ɻओʹ഑৴ϩδοΫ͔Βσʔλ෼ੳج൫·Ͱɻ

   ࠷ۙͷڵຯ͸ϛΫϩܦࡁֶͱϝΧχζϜσβΠϯɻ
   
   
   

   View Slide

3. ࠷ۙͷ׆ಈ
   ਓ޻஌ೳֶձࢽ Vol. 32 No. 4 (2017/07) ͷʮ޿ࠂͱ
   AI ಛूʯʹʮΞυωοτϫʔΫʹ͓͚Δ޿ࠂ഑৴ܭ
   ըͷ࠷దԽʯ͕ܝࡌ͞Ε·ͨ͠ɻ
   
   
   ΦϥΠϦʔ͔Βʮ࢓ࣄͰ͸͡ΊΔػցֶशʯ͕ग़·
   ͨ͠ɻ @chezou, @tokorotenͱڞஶ
   ࢴ൛ɾిࢠॻ੶྆ํ͋Γ·͢
   

   View Slide

4. ࠓ೔ͷ࿩
   w ༧ଌϞσϧͷ܇࿅ɾ഑෍Λߦͳ͏όονॲཧ
   w 8FCαʔόʔͰͷਪ࿦
   w ӡ༻ɾ༧ଌϞσϧͷੑೳ؂ࢹ
   w ༧ଌϞσϧͷΦϑϥΠϯධՁɾΦϯϥΠϯධՁ
   Քಈதͷ޿ࠂΠϯϓϨογϣϯͷՁ஋ධՁλεΫΛ୊ࡐʹͯ͠ɺ࣍
   ͷτϐοΫΛऔΓ͋͛·͢
   

   View Slide

5. ωοτ޿ࠂ഑৴γεςϜͱ.-
   λεΫͷྫ Ϟνϕʔγϣϯ Ξϓϩʔν
   ΫϦοΫ༧ଌ ޿ࠂ͕ΫϦοΫ͞ΕΔ֬཰͕஌Γ͍ͨ Ϋϥε෼ྨ
   ίϯόʔδϣϯ༧ଌ ΫϦοΫޙʹίϯόʔδϣϯ͢Δ֬཰͕஌Γ͍ͨ Ϋϥε෼ྨ
   ഑৴͢Δ޿ࠂͷܾఆ ͋ΔظؒͷऩӹΛ࠷େʹ͢ΔͨΊʹɺͲͷ޿ࠂΛ
   දࣔ͢΂͖͔ܾΊ͍ͨ
   ଟ࿹όϯσΟοτ
   ૊߹ͤ࠷దԽ
   ޿ࠂ࿮ͷΫϥελϦϯά ࣅ͍ͯΔ޿ࠂ࿮Λ·ͱΊͯѻ͍͍ͨ ΫϥελϦϯά
   ΫϩεσόΠεਪఆ ୺຤Λԣஅͨ͠ΦʔσΟΤϯεͷτϥοΩϯά͕
   ͍ͨ͠
   ଟΫϥε෼ྨ
   

   View Slide

6. ޿ࠂΠϯϓϨογϣϯͷՁ஋ධՁ
   ޿ࠂഔମ
   ɾ
   441
   ഑৴ࣄۀऀ
   ޿ࠂϦΫΤετ9
   ޿ࠂ
   ޿ࠂ
   ޿ࠂO
   
   ޿ࠂબ୒ϩδοΫ
   

   View Slide

7. ޿ࠂΠϯϓϨογϣϯͷՁ஋ධՁ
   ޿ࠂഔମ
   ɾ
   441
   ഑৴ࣄۀऀ
   ޿ࠂϦΫΤετ9
   ޿ࠂ
   ޿ࠂ
   ޿ࠂO
   
   ऩӹ༧ଌϩδοΫ
   ޿ࠂબ୒ϩδοΫ
   ԁ
   ԁ
   ԁ
   ޿ࠂϨεϙϯε
   ޿ࠂZ
   

   View Slide

8. ޿ࠂϦΫΤετ9 ࿮ɾΦʔσΟΤϯεɾίϯςΩετʜ
   ʹରͯ͠ɺԠ౴
   Մೳͳ޿ࠂ͕Oݸ͋Δɻ͜ͷ࣌ʹͦΕͧΕͷ޿ࠂB㱨\ ʜO^Λදࣔ͠
   ͨ࣌ͷऩӹ༧ଌ஋͕ཉ͍͠ɻ
   ऩӹ༧ଌ஋͸ΫϦοΫ཰ $53
   ΍ίϯόʔδϣϯ཰ $73
   ΛͦΕͧΕͷ޿ࠂ
   ʹ͍ͭͯ༧ଌͯ͠ར༻͢Δɻ
   CTR = P(click=1 | X, a)

   CVR = P(conversion=1 | X, a, click=1)
   ͨͩ͠શͯͷॲཧ͕NTҎ಺ʹऴΔࣄ
   ޿ࠂΠϯϓϨογϣϯͷՁ஋ධՁ ଓ͖
   
   

   View Slide

9. "84&$4$MVTUFS
   &$*OTUBODF
   γεςϜߏ੒
   4
   $53༧ଌث
   ܇࿅λεΫ
   $*%FQMPZ
   $73༧ଌث
   ܇࿅λεΫ
   -VJHJ
   4DIFEVMFS
   $53༧ଌث
   +40/
   $73༧ଌث
   +40/
   ऩӹ༧ଌϩδοΫ
   $53༧ଌ
   ॲཧ
   $73༧ଌ
   ॲཧ
   #JH2VFSZ
   ഑৴ϩά ϩάͷ&5-
   'FUDI
   %BJMZ
   6QEBUF
   2VFSZ
   ܇࿅σʔλ
   )551
   ޿ࠂબ୒ϩδοΫ
   

   View Slide

10. શମͷઃܭ
    w 8FCαʔόʔͰϦΫΤετΛॲཧ͢Δϓϩηε಺Ͱਪ࿦ 4DBMBͰ࣮૷
    
    w ϨΠςϯγͱ҆ఆੑΛ༏ઌ
    w ଞͷબ୒ࢶͩͱ"1*αʔόʔԽ
    
    w ϩδεςΟοΫճؼ-ਖ਼ଇԽ
    w ഑෍͞Εͨ༧ଌϞσϧΛΦϯϝϞϦͰอ࣋͢Δ
    w ܇࿅͸σΠϦʔͷόονॲཧ
    w ೔ʑ௥Ճ͞ΕΔ৽ن޿ࠂΩϟϯϖʔϯɾ৽ن޿ࠂ࿮ʹରԠ͢ΔͨΊຖ೔
    ֶशΛ΍Γͳ͓͢ɻݹ͍աڈ͸๨Εͤ͞Δɻ
    w 1ZUIPOͰ࣮૷
    

    View Slide

11. όονॲཧ܈
    

    View Slide

12. όον࣮ߦ؀ڥ
    w "84&$4্ͷ%PDLFSίϯςφͰಈ࡞
    w 1ZUIPO؀ڥͷ࠶ݱੑ୲อͷͨΊʹ1JQpMFΛ࢖༻
    w ϫʔΫϑϩʔΤϯδϯ͸-VJHJΛ࠾༻
    w λεΫؒͷґଘղܾ
    w &5-ɾ༧ଌϞσϧͷ܇࿅ɾϞσϧͷ഑෍ͱෳࡶʹͳΓ͕ͪͳॴ
    w FHར༻͢Δσʔλछʹ͍ͭͯલ೔࣌୆ͷσʔλऔࠐΈ͕ऴͬͨ
    ΒલॲཧΛ։࢝ɻલॲཧ͕ऴͬͨΒ"ͱ#ͷ܇࿅ΛฒྻͰ։࢝
    w ֎෦ཁҼͰίέͨ࣌ͷϦτϥΠɾฒྻ࣮ߦ੍ޚ
    

    View Slide

13. %FQFOEFODZ(SBQI
    ίέͨ৔ॴ΍ҟৗʹ͕͔͍࣌ؒͬͯΔλεΫ͕͙͢ʹΘ͔ͬͯศར
    

    View Slide

14. ܇࿅
    w TDJLJUMFBSOΛ࢖ͬͯϩδεςΟοΫճؼ
    w ΧςΰϦΧϧม਺ͷPOFIPUFODPEJOHͰ਺ສ࣍ݩʹͳΔ͕ɺ-ਖ਼ଇԽͰ

    ສҎԼ·Ͱམ͍ͪͯΔ
    w ޿ࠂΩϟϯϖʔϯຖʹ༧ଌثΛ࡞Δ
    w ঎ࡐ ϦλʔήςΟϯάPSʜ
    ʹΑͬͯ࢖͑Δૉੑ͕ҟͳΔ
    w ਖ਼ྫෛྫͷόϥϯε͕ͩͬͨΓͩͬͨΓͯ͠Ұॹʹѻ͏
    ͷ͕೉͍͠
    w $MBTT*NCBMBODFରࡦͱͯ͠/FHBUJWF%PXOTBNQMJOH͍ͯ͠Δ
    

    View Slide

15. /FHBUJWF%PXOTBNQMJOH
    w ਖ਼ྫ͸શͯར༻ɺෛྫ͸ؒҾ͘FH͚ͩ࢖༻

    *NCBMBODFىҼͷόΠΞεআڈ
    
    w ॲཧର৅σʔλΛେ෯ʹݮΒ͢ࣄ͕Ͱ͖ΔɻܭࢉίετݮͷϝϦοτ͕େɻ
    w ࣺͯΔ෦෼Λม͑ͯ༧ଌثΛ࡞Γ#BHHJOHͰ൚ԽੑೳΛ্͛ΒΕΔ
    w #JH2VFSZͳΒΫΤϦͰ࣍ͷ௨Γॻ͚Δɻ࠶ݱੑ༗Γɻ

    where 

    mod(farm_fingerprint(imp_id), 50) = 0

    or

    click = 1
    

    View Slide

16. /FHBUJWF%PXOTBNQMJOH
    w ਖ਼ྫ͸શͯར༻ɺෛྫ͸ؒҾ͘FH͚ͩ࢖༻

    *NCBMBODFىҼͷόΠΞεআڈ
    
    w ॲཧର৅σʔλΛେ෯ʹݮΒ͢ࣄ͕Ͱ͖ΔɻܭࢉίετݮͷϝϦοτ͕େɻ
    w ࣺͯΔ෦෼Λม͑ͯ༧ଌثΛ࡞Γ#BHHJOHͰ൚ԽੑೳΛ্͛ΒΕΔ
    w #JH2VFSZͳΒΫΤϦͰ࣍ͷ௨Γॻ͚Δɻ࠶ݱੑ༗Γɻ

    $MJDL $MJDL
    %BUB
    where 

    mod(farm_fingerprint(imp_id), 50) = 0

    or

    click = 1
    

    View Slide

17. /FHBUJWF%PXOTBNQMJOH
    w ਖ਼ྫ͸શͯར༻ɺෛྫ͸ؒҾ͘FH͚ͩ࢖༻

    *NCBMBODFىҼͷόΠΞεআڈ
    
    w ॲཧର৅σʔλΛେ෯ʹݮΒ͢ࣄ͕Ͱ͖ΔɻܭࢉίετݮͷϝϦοτ͕େɻ
    w ࣺͯΔ෦෼Λม͑ͯ༧ଌثΛ࡞Γ#BHHJOHͰ൚ԽੑೳΛ্͛ΒΕΔ
    w #JH2VFSZͳΒΫΤϦͰ࣍ͷ௨Γॻ͚Δɻ࠶ݱੑ༗Γɻ

    $MJDL $MJDL
    %BUB
    ࣺͯΔ
    where 

    mod(farm_fingerprint(imp_id), 50) = 0

    or

    click = 1
    

    View Slide

18. ܇࿅݁Ռͷ֬ೝ
    w ࣮ݧ࣌
    w 5JNF4FSJFT4QMJUͰ$SPTT7BMJEBUJPO "6$
    
    w ੾ย߲ͷ஋͕पลԽͨ࣌͠ͷ֬཰ͱҰக͍ͯ͠ΔࣄΛ֬ೝ
    w ཉ͍͠ͷ͸֬཰
    w શମͷΫϦοΫ཰TJHNPJE ΫϦοΫ༧ଌثͷ੾ย߲ͷ஋
    ͳΒ0,
    w "6$΍MPHMPTT͚ͩݟͯ΋0WFS1SFEJDUJPOʹ͸ؾ͚ͮͳ͍
    

    View Slide

19. ༧ଌϞσϧͷ഑෍
    w ܇࿅݁Ռ΍ύϥϝʔλΛ+40/ʹͯ͠4ʹ഑ஔ 1SPUPDPM#V⒎FSͷํ͕ྑ͔͔ͬͨ΋ʜ
    
    w ϑΝΠϧͷ಺༰
    w ϝλ৘ใ όονॲཧىಈ࣌ࠁɾόʔδϣϯ
    
    w ඇθϩͷ܎਺ XFJHIU
    ͱରԠϥϕϧ ֶश݁Ռ
    
    w લॲཧύϥϝʔλ
    w ࿈ଓม਺ͷฏۉɾ෼ࢄ
    w /FHBUJWF%PXOTBNQMJOH཰
    w όονىಈຖʹϑΝΠϧΛ࢒͍ͯ͠ΔͷͰɺ-BUFTUΛ೚ҙͷ࣌఺ʹ໭ͤΔ
    

    View Slide

20. ਪ࿦
    

    View Slide

21. 8FC4FSWFSଆͷܭࢉ
    4DBMBͰ࣮૷ͨ͠෦෼ɻ
    *.1ΛYɺॏΈ͕Xͷ࣌ɺϩδεςΟοΫճؼͷΫϥεࣄޙ֬཰͸࣍ͷ௨Γɻ
    ࣮૷͕ਖ਼͍͠ࣄͷ֬ೝͱͯ͠ɺςετίʔυͰTDJLJUMFBSOͷ
    -PHJTUJD$MBTTJpFSQSFEJDU@QSPCB
    ΛݺΜͩ݁Ռͱಉ͔͡Ͳ͏͔ΛνΣοΫ
    ͜ΕΛQ /FHBUJWF%PXOTBNQMJOH཰ΛSͱͯ͠ݩͷ֬཰ʹ໭͢ʹ͸ɻ
    ㅟ
    Yͷཁૉ͸΄ͱΜͲθϩͳͷͰ*.1
    ʹରԠ͢Δ8FJHIUͷ଍͋͠Θͤ
    

    View Slide

22. ϩδοΫ΍༧ଌث͸੾Γସ͑Մೳʹ͢Δ
    w γεςϜͷ"#5FTUରԠʹඞਢ
    w ࣮ݧΛதஅ͢Δͷʹιʔείʔυͷ3FWFSUαʔόʔશ୆ೖΕ׵͑ͱ͔
    ΍Γͨ͘ͳ͍
    

    View Slide

23. ϦϦʔεޙͷӡ༻
    

    View Slide

24. ༧ଌϞσϧͷੑೳ؂ࢹ
    w Ϗδωεࢦඪͷ؂ࢹͱϞσϧ༧ଌਫ਼౓ͷ؂ࢹΛಉ࣌ʹಈ͔͓ͯ͘͠
    w ༧ଌਫ਼౓͕མͪͨ PSσʔλ͕଍Γͳͯ͘௿͍
    ࣌ʹϏδωεࢦඪ͕ѱ
    ͍ͷ͸ظ଴௨Γͷಈ࡞
    w ༧ଌਫ਼౓ͷྑ͞ͱϏδωεࢦඪͷྑ͕͞࿈ಈ͍ͯ͠Ε͹0,
    w ແ૬ؔͰ͋Ε͹ͦͷ༧ଌॲཧ͸ୀ໾Մೳ
    

    View Slide

25. ༧ଌϞσϧͷੑೳ؂ࢹ ଓ͖
    
    w Ϗδωεࢦඪͷ؂ࢹ
    w ച্ɾૈརɾ$1" ίϯόʔδϣϯ֫ಘ݅͋ͨΓͷίετ
    
    w ༧ଌث͕ࣄۀʹߩݙͰ͖͍ͯΔ͔Θ͔Δ
    w ٸܹʹѱԽͨ͠৔߹ɺେ఍γεςϜʹԿ͔͕ى͖ͯΔ
    w ༧ଌਫ਼౓
    w ࣮ࡍͷ഑৴࣌ͷ༧ଌ஋ͱ഑৴݁ՌΛ࢖ͬͯ޿ࠂΩϟϯϖʔϯຖʹ"6$
    ͷ࣌ܥྻΛϓϩοτ
    w Ωϟϯϖʔϯຖʹϓϩοτ͢ΔࣄͰ঎ࡐʹΑΔ܏޲ͷ͕ࠩݟ͑Δ
    

    View Slide

26. ༧ଌϞσϧͷੑೳ؂ࢹ ଓ͖
    
    w ༧ଌثຖͷ࢖༻ঢ়گ
    w ݺग़ճ਺ɾൃੜച্ ো֐ൃੜ࣌ͷϦΧόϦ༏ઌ౓
    
    w ޿ࠂӡ༻ऀ͕഑৴ϩδοΫ ༧ଌث
    Λબ΂Δέʔε
    w ྑ͍෺Ͱ΋ʮಈ࡞͕೺ѲͰ͖ͳ͍͔Β࢖ͬͯͳ͍ʯͱ͍͏ࣄ΋͋Δ
    w ίϛϡχέʔγϣϯऔ͍ͬͯ͘
    w ࢖ΘΕͳ͘ͳͬͨ෺͸ୀ໾ͤ͞Δ
    w ࢖͍ͬͯͳͯ͘΋ҡ࣋අ͕͔͔Δ
    

    View Slide

27. ܇࿅σʔλͷ܏޲ͷมԽ
    w σʔλੜ੒ܦ࿏ͷো֐
    w ܇࿅σʔλͷݩʹͳΔ্ྲྀͷϑΟϧλ͕ނো
    w ܭଌܥͷো֐ FHίϯόʔδϣϯܭଌ͕ࢭ·ͬͨ
    
    w ো֐ͳΒ܇࿅σʔλΛ៉ྷʹͯ͠࠶܇࿅͢Δͱੑೳ͕ݩʹ໭Δ
    w Ϗδωεঢ়گͷมԽ
    w ༧ଌର৅ͷ܏޲͕มΘͬͨ
    w ܏޲ͷมԽ͕Θ͔Ε͹࣍ͷҰखΛܾΊ΍͍͢ɻ৳ͼ͠Ζͷ͋Δ෦෼ͷ
    νϡʔχϯά΍৽ن։ൃʹணखɻ
    

    View Slide

28. ৽͍͠༧ଌثͷϦϦʔε
    

    View Slide

29. ΦϑϥΠϯධՁ
    w ৽͍͠༧ଌثͷ࣮ݧɺϏδωεࢦඪ͕Ͳ͏ͳΔ͔Λ֬ೝ͢Δ
    w աڈͷ഑৴ϩάΛ࢖ͬͯධՁ
    w աڈͷ഑৴ϩάߦຖʹ༧ଌ஋Λੜ੒
    w #JH2VFSZ͔ΒಡΜͰ#JH2VFSZ΁ॻ͖ࠐΉͷ͸$MPVE%BUBqPX͕ศར
    w ͔͠͠ϩάΛ࢖ͬͨ൓࣮Ծ૝੓ࡦͷධՁ͸ݫີʹ͸Ͱ͖ͳ͍
    w ܇࿅σʔλ΁ͷϑΟʔυόοΫ͕γϛϡϨʔτͰ͖ͳ͍
    w ඇఆৗੑ
    w ධՁ݁Ռ͸ࢀߟఔ౓ʹ
    

    View Slide

30. ΦϯϥΠϯධՁ
    w ͪ͜Β͕ຊ໋
    w ϩδοΫ ༧ଌث
    ͷ"#ςετ
    w ϓϩμΫγϣϯ؀ڥͰαϯϓϧΛແ࡞ҝʹ܈ʹ෼͚ΔϥϯμϜԽൺֱࢼݧ
    w ࣌ؒมԽͷӨڹɾηϨΫγϣϯόΠΞεͷআڈ
    w ίϯτϩʔϧ܈ͱτϦʔτϝϯτ܈ͷϏδωεࢦඪͷ͕ࠩҼՌޮՌ
    w ϏδωεΠϯύΫτͰධՁ ౷ܭతʹ༗ҙͰ΋͕ࠩখ͍͞ͱඍົ
    
    

    View Slide

31. ΦϯϥΠϯධՁ
    w ͪ͜Β͕ຊ໋
    w ϩδοΫ ༧ଌث
    ͷ"#ςετ
    w ϓϩμΫγϣϯ؀ڥͰαϯϓϧΛແ࡞ҝʹ܈ʹ෼͚ΔϥϯμϜԽൺֱࢼݧ
    w ࣌ؒมԽͷӨڹɾηϨΫγϣϯόΠΞεͷআڈ
    w ίϯτϩʔϧ܈ͱτϦʔτϝϯτ܈ͷϏδωεࢦඪͷ͕ࠩҼՌޮՌ
    w ϏδωεΠϯύΫτͰධՁ ౷ܭతʹ༗ҙͰ΋͕ࠩখ͍͞ͱඍົ
    
    ஌Γ͍ͨͷ͸ίί
    ྫૈར͕ͲΕ͚ͩ૿͑Δ͔
    

    View Slide

32. ϦϦʔε
    w "#5FTU3FBEZͳγεςϜʹͳ͍ͬͯΕ͹ϦϦʔε΋༰қ
    w ద༻ϩδοΫΛ੍ޚ͢Δ࢓૊ΈΛ࡞͓ͬͯ͘
    w ؅ཧը໘ͰઃఆͰ͖Δ
    w "#ςετ
    w શମͷ͚ͩϦϦʔε
    w ಛఆҊ݅ͷΈʹద༻
    w ؾܰʹϦϦʔεͯ͠ݕূͰ͖Δ࢓૊Έେࣄ
    

    View Slide

33. ઃఆྫ
    ઃఆ໊ ಺༰ ద༻࿮਺
    999ϩδοΫ"#ςετ
    999-PHJD7
    999-PHJD7
    
    σϑΥϧτ 999-PHJD7
    όάYYͷ࢑ఆରԠ /BJWF-PHJD
    

    View Slide

34. ·ͱΊ
    w ޿ࠂ഑৴αʔόʔͷதͰ༧ଌॲཧ͕ಈ͍͍ͯΔ
    w ϦϦʔε͔ͯ͠Β΋΍Δࣄ͸ͨ͘͞Μ͋Δ
    w ো֐ରԠɾػೳվળ
    w ؂ࢹ͸ॏཁ
    w ݕূɾϦϦʔεɾୀ໾Λ΍Γ΍͘͢
    w "#5FTU3FBEZʹ͓ͯ͘͠ͱָ
    w πϥΠ࿩͋Μ·Γͳ͔ͬͨʜʜ
    

    View Slide

35. ࢀߟจݙ
    <>8BMMBDF #ZSPO$ FUBM$MBTTJNCBMBODF SFEVY%BUB.JOJOH *$%.
    
    *&&&UI*OUFSOBUJPOBM$POGFSFODFPO*&&&
    <>$IBQFMMF 0MJWJFS &SFO.BOBWPHMV BOE3PNFS3PTBMFT4JNQMFBOE
    TDBMBCMFSFTQPOTFQSFEJDUJPOGPSEJTQMBZBEWFSUJTJOH"$.5SBOTBDUJPOTPO
    *OUFMMJHFOU4ZTUFNTBOE5FDIOPMPHZ 5*45
    
    
    <>)F 9JOSBO FUBM1SBDUJDBMMFTTPOTGSPNQSFEJDUJOHDMJDLTPOBETBU
    GBDFCPPL1SPDFFEJOHTPGUIF&JHIUI*OUFSOBUJPOBM8PSLTIPQPO%BUB
    .JOJOHGPS0OMJOF"EWFSUJTJOH"$.
    <>4DVMMFZ % FUBM)JEEFOUFDIOJDBMEFCUJONBDIJOFMFBSOJOH
    TZTUFNT"EWBODFTJOOFVSBMJOGPSNBUJPOQSPDFTTJOHTZTUFNT
    

    View Slide