Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ネット広告配信サービスにおけるMLシステムの実装と運用

 ネット広告配信サービスにおけるMLシステムの実装と運用

Machine Learning System on Online Advertising Platform.

Takashi Nishibayashi

July 13, 2018
Tweet

More Decks by Takashi Nishibayashi

Other Decks in Technology

Transcript

  1. ωοτ޿ࠂ഑৴αʔϏεʹ͓͚Δ
    .-γεςϜͷ࣮૷ͱӡ༻
    5BLBTIJ/JTIJCBZBTIJ

    .BDIJOF-FBSOJOH$BTVBM5BMLT

    View Slide

  2. ͓લͩΕΑ
    Name: Takashi Nishibayashi
    twitter.com/@hagino3000
    Job: Software Engineer
    VOYAGE GROUPͰωοτ޿ࠂ഑৴αʔϏε࡞ͬͯ
    ·͢ɻओʹ഑৴ϩδοΫ͔Βσʔλ෼ੳج൫·Ͱɻ

    ࠷ۙͷڵຯ͸ϛΫϩܦࡁֶͱϝΧχζϜσβΠϯɻ


    View Slide

  3. ࠷ۙͷ׆ಈ
    ਓ޻஌ೳֶձࢽ Vol. 32 No. 4 (2017/07) ͷʮ޿ࠂͱ
    AI ಛूʯʹʮΞυωοτϫʔΫʹ͓͚Δ޿ࠂ഑৴ܭ
    ըͷ࠷దԽʯ͕ܝࡌ͞Ε·ͨ͠ɻ


    ΦϥΠϦʔ͔Βʮ࢓ࣄͰ͸͡ΊΔػցֶशʯ͕ग़·
    ͨ͠ɻ @chezou, @tokorotenͱڞஶ
    ࢴ൛ɾిࢠॻ੶྆ํ͋Γ·͢

    View Slide

  4. ࠓ೔ͷ࿩
    w ༧ଌϞσϧͷ܇࿅ɾ഑෍Λߦͳ͏όονॲཧ
    w 8FCαʔόʔͰͷਪ࿦
    w ӡ༻ɾ༧ଌϞσϧͷੑೳ؂ࢹ
    w ༧ଌϞσϧͷΦϑϥΠϯධՁɾΦϯϥΠϯධՁ
    Քಈதͷ޿ࠂΠϯϓϨογϣϯͷՁ஋ධՁλεΫΛ୊ࡐʹͯ͠ɺ࣍
    ͷτϐοΫΛऔΓ͋͛·͢

    View Slide

  5. ωοτ޿ࠂ഑৴γεςϜͱ.-
    λεΫͷྫ Ϟνϕʔγϣϯ Ξϓϩʔν
    ΫϦοΫ༧ଌ ޿ࠂ͕ΫϦοΫ͞ΕΔ֬཰͕஌Γ͍ͨ Ϋϥε෼ྨ
    ίϯόʔδϣϯ༧ଌ ΫϦοΫޙʹίϯόʔδϣϯ͢Δ֬཰͕஌Γ͍ͨ Ϋϥε෼ྨ
    ഑৴͢Δ޿ࠂͷܾఆ ͋ΔظؒͷऩӹΛ࠷େʹ͢ΔͨΊʹɺͲͷ޿ࠂΛ
    දࣔ͢΂͖͔ܾΊ͍ͨ
    ଟ࿹όϯσΟοτ
    ૊߹ͤ࠷దԽ
    ޿ࠂ࿮ͷΫϥελϦϯά ࣅ͍ͯΔ޿ࠂ࿮Λ·ͱΊͯѻ͍͍ͨ ΫϥελϦϯά
    ΫϩεσόΠεਪఆ ୺຤Λԣஅͨ͠ΦʔσΟΤϯεͷτϥοΩϯά͕
    ͍ͨ͠
    ଟΫϥε෼ྨ

    View Slide

  6. ޿ࠂΠϯϓϨογϣϯͷՁ஋ධՁ
    ޿ࠂഔମ
    ɾ
    441
    ഑৴ࣄۀऀ
    ޿ࠂϦΫΤετ9
    ޿ࠂ
    ޿ࠂ
    ޿ࠂO

    ޿ࠂબ୒ϩδοΫ

    View Slide

  7. ޿ࠂΠϯϓϨογϣϯͷՁ஋ධՁ
    ޿ࠂഔମ
    ɾ
    441
    ഑৴ࣄۀऀ
    ޿ࠂϦΫΤετ9
    ޿ࠂ
    ޿ࠂ
    ޿ࠂO

    ऩӹ༧ଌϩδοΫ
    ޿ࠂબ୒ϩδοΫ
    ԁ
    ԁ
    ԁ
    ޿ࠂϨεϙϯε
    ޿ࠂZ

    View Slide

  8. ޿ࠂϦΫΤετ9 ࿮ɾΦʔσΟΤϯεɾίϯςΩετʜ
    ʹରͯ͠ɺԠ౴
    Մೳͳ޿ࠂ͕Oݸ͋Δɻ͜ͷ࣌ʹͦΕͧΕͷ޿ࠂB㱨\ ʜO^Λදࣔ͠
    ͨ࣌ͷऩӹ༧ଌ஋͕ཉ͍͠ɻ
    ऩӹ༧ଌ஋͸ΫϦοΫ཰ $53
    ΍ίϯόʔδϣϯ཰ $73
    ΛͦΕͧΕͷ޿ࠂ
    ʹ͍ͭͯ༧ଌͯ͠ར༻͢Δɻ
    CTR = P(click=1 | X, a)

    CVR = P(conversion=1 | X, a, click=1)
    ͨͩ͠શͯͷॲཧ͕NTҎ಺ʹऴΔࣄ
    ޿ࠂΠϯϓϨογϣϯͷՁ஋ධՁ ଓ͖

    View Slide

  9. "84&$4$MVTUFS
    &$*OTUBODF
    γεςϜߏ੒
    4
    $53༧ଌث
    ܇࿅λεΫ
    $*%FQMPZ
    $73༧ଌث
    ܇࿅λεΫ
    -VJHJ
    4DIFEVMFS
    $53༧ଌث
    +40/
    $73༧ଌث
    +40/
    ऩӹ༧ଌϩδοΫ
    $53༧ଌ
    ॲཧ
    $73༧ଌ
    ॲཧ
    #JH2VFSZ
    ഑৴ϩά ϩάͷ&5-
    'FUDI
    %BJMZ
    6QEBUF
    2VFSZ
    ܇࿅σʔλ
    )551
    ޿ࠂબ୒ϩδοΫ

    View Slide

  10. શମͷઃܭ
    w 8FCαʔόʔͰϦΫΤετΛॲཧ͢Δϓϩηε಺Ͱਪ࿦ 4DBMBͰ࣮૷

    w ϨΠςϯγͱ҆ఆੑΛ༏ઌ
    w ଞͷબ୒ࢶͩͱ"1*αʔόʔԽ

    w ϩδεςΟοΫճؼ-ਖ਼ଇԽ
    w ഑෍͞Εͨ༧ଌϞσϧΛΦϯϝϞϦͰอ࣋͢Δ
    w ܇࿅͸σΠϦʔͷόονॲཧ
    w ೔ʑ௥Ճ͞ΕΔ৽ن޿ࠂΩϟϯϖʔϯɾ৽ن޿ࠂ࿮ʹରԠ͢ΔͨΊຖ೔
    ֶशΛ΍Γͳ͓͢ɻݹ͍աڈ͸๨Εͤ͞Δɻ
    w 1ZUIPOͰ࣮૷

    View Slide

  11. όονॲཧ܈

    View Slide

  12. όον࣮ߦ؀ڥ
    w "84&$4্ͷ%PDLFSίϯςφͰಈ࡞
    w 1ZUIPO؀ڥͷ࠶ݱੑ୲อͷͨΊʹ1JQpMFΛ࢖༻
    w ϫʔΫϑϩʔΤϯδϯ͸-VJHJΛ࠾༻
    w λεΫؒͷґଘղܾ
    w &5-ɾ༧ଌϞσϧͷ܇࿅ɾϞσϧͷ഑෍ͱෳࡶʹͳΓ͕ͪͳॴ
    w FHར༻͢Δσʔλछʹ͍ͭͯલ೔࣌୆ͷσʔλऔࠐΈ͕ऴͬͨ
    ΒલॲཧΛ։࢝ɻલॲཧ͕ऴͬͨΒ"ͱ#ͷ܇࿅ΛฒྻͰ։࢝
    w ֎෦ཁҼͰίέͨ࣌ͷϦτϥΠɾฒྻ࣮ߦ੍ޚ

    View Slide

  13. %FQFOEFODZ(SBQI
    ίέͨ৔ॴ΍ҟৗʹ͕͔͍࣌ؒͬͯΔλεΫ͕͙͢ʹΘ͔ͬͯศར

    View Slide

  14. ܇࿅
    w TDJLJUMFBSOΛ࢖ͬͯϩδεςΟοΫճؼ
    w ΧςΰϦΧϧม਺ͷPOFIPUFODPEJOHͰ਺ສ࣍ݩʹͳΔ͕ɺ-ਖ਼ଇԽͰ

    ສҎԼ·Ͱམ͍ͪͯΔ
    w ޿ࠂΩϟϯϖʔϯຖʹ༧ଌثΛ࡞Δ
    w ঎ࡐ ϦλʔήςΟϯάPSʜ
    ʹΑͬͯ࢖͑Δૉੑ͕ҟͳΔ
    w ਖ਼ྫෛྫͷόϥϯε͕ͩͬͨΓͩͬͨΓͯ͠Ұॹʹѻ͏
    ͷ͕೉͍͠
    w $MBTT*NCBMBODFରࡦͱͯ͠/FHBUJWF%PXOTBNQMJOH͍ͯ͠Δ

    View Slide

  15. /FHBUJWF%PXOTBNQMJOH
    w ਖ਼ྫ͸શͯར༻ɺෛྫ͸ؒҾ͘FH͚ͩ࢖༻

    *NCBMBODFىҼͷόΠΞεআڈ

    w ॲཧର৅σʔλΛେ෯ʹݮΒ͢ࣄ͕Ͱ͖ΔɻܭࢉίετݮͷϝϦοτ͕େɻ
    w ࣺͯΔ෦෼Λม͑ͯ༧ଌثΛ࡞Γ#BHHJOHͰ൚ԽੑೳΛ্͛ΒΕΔ
    w #JH2VFSZͳΒΫΤϦͰ࣍ͷ௨Γॻ͚Δɻ࠶ݱੑ༗Γɻ

    where 

    mod(farm_fingerprint(imp_id), 50) = 0

    or

    click = 1

    View Slide

  16. /FHBUJWF%PXOTBNQMJOH
    w ਖ਼ྫ͸શͯར༻ɺෛྫ͸ؒҾ͘FH͚ͩ࢖༻

    *NCBMBODFىҼͷόΠΞεআڈ

    w ॲཧର৅σʔλΛେ෯ʹݮΒ͢ࣄ͕Ͱ͖ΔɻܭࢉίετݮͷϝϦοτ͕େɻ
    w ࣺͯΔ෦෼Λม͑ͯ༧ଌثΛ࡞Γ#BHHJOHͰ൚ԽੑೳΛ্͛ΒΕΔ
    w #JH2VFSZͳΒΫΤϦͰ࣍ͷ௨Γॻ͚Δɻ࠶ݱੑ༗Γɻ

    $MJDL $MJDL
    %BUB
    where 

    mod(farm_fingerprint(imp_id), 50) = 0

    or

    click = 1

    View Slide

  17. /FHBUJWF%PXOTBNQMJOH
    w ਖ਼ྫ͸શͯར༻ɺෛྫ͸ؒҾ͘FH͚ͩ࢖༻

    *NCBMBODFىҼͷόΠΞεআڈ

    w ॲཧର৅σʔλΛେ෯ʹݮΒ͢ࣄ͕Ͱ͖ΔɻܭࢉίετݮͷϝϦοτ͕େɻ
    w ࣺͯΔ෦෼Λม͑ͯ༧ଌثΛ࡞Γ#BHHJOHͰ൚ԽੑೳΛ্͛ΒΕΔ
    w #JH2VFSZͳΒΫΤϦͰ࣍ͷ௨Γॻ͚Δɻ࠶ݱੑ༗Γɻ

    $MJDL $MJDL
    %BUB
    ࣺͯΔ
    where 

    mod(farm_fingerprint(imp_id), 50) = 0

    or

    click = 1

    View Slide

  18. ܇࿅݁Ռͷ֬ೝ
    w ࣮ݧ࣌
    w 5JNF4FSJFT4QMJUͰ$SPTT7BMJEBUJPO "6$

    w ੾ย߲ͷ஋͕पลԽͨ࣌͠ͷ֬཰ͱҰக͍ͯ͠ΔࣄΛ֬ೝ
    w ཉ͍͠ͷ͸֬཰
    w શମͷΫϦοΫ཰TJHNPJE ΫϦοΫ༧ଌثͷ੾ย߲ͷ஋
    ͳΒ0,
    w "6$΍MPHMPTT͚ͩݟͯ΋0WFS1SFEJDUJPOʹ͸ؾ͚ͮͳ͍

    View Slide

  19. ༧ଌϞσϧͷ഑෍
    w ܇࿅݁Ռ΍ύϥϝʔλΛ+40/ʹͯ͠4ʹ഑ஔ 1SPUPDPM#V⒎FSͷํ͕ྑ͔͔ͬͨ΋ʜ

    w ϑΝΠϧͷ಺༰
    w ϝλ৘ใ όονॲཧىಈ࣌ࠁɾόʔδϣϯ

    w ඇθϩͷ܎਺ XFJHIU
    ͱରԠϥϕϧ ֶश݁Ռ

    w લॲཧύϥϝʔλ
    w ࿈ଓม਺ͷฏۉɾ෼ࢄ
    w /FHBUJWF%PXOTBNQMJOH཰
    w όονىಈຖʹϑΝΠϧΛ࢒͍ͯ͠ΔͷͰɺ-BUFTUΛ೚ҙͷ࣌఺ʹ໭ͤΔ

    View Slide

  20. ਪ࿦

    View Slide

  21. 8FC4FSWFSଆͷܭࢉ
    4DBMBͰ࣮૷ͨ͠෦෼ɻ
    *.1ΛYɺॏΈ͕Xͷ࣌ɺϩδεςΟοΫճؼͷΫϥεࣄޙ֬཰͸࣍ͷ௨Γɻ
    ࣮૷͕ਖ਼͍͠ࣄͷ֬ೝͱͯ͠ɺςετίʔυͰTDJLJUMFBSOͷ
    -PHJTUJD$MBTTJpFSQSFEJDU@QSPCB
    ΛݺΜͩ݁Ռͱಉ͔͡Ͳ͏͔ΛνΣοΫ
    ͜ΕΛQ /FHBUJWF%PXOTBNQMJOH཰ΛSͱͯ͠ݩͷ֬཰ʹ໭͢ʹ͸ɻ

    Yͷཁૉ͸΄ͱΜͲθϩͳͷͰ*.1
    ʹରԠ͢Δ8FJHIUͷ଍͋͠Θͤ

    View Slide

  22. ϩδοΫ΍༧ଌث͸੾Γସ͑Մೳʹ͢Δ
    w γεςϜͷ"#5FTUରԠʹඞਢ
    w ࣮ݧΛதஅ͢Δͷʹιʔείʔυͷ3FWFSUαʔόʔશ୆ೖΕ׵͑ͱ͔
    ΍Γͨ͘ͳ͍

    View Slide

  23. ϦϦʔεޙͷӡ༻

    View Slide

  24. ༧ଌϞσϧͷੑೳ؂ࢹ
    w Ϗδωεࢦඪͷ؂ࢹͱϞσϧ༧ଌਫ਼౓ͷ؂ࢹΛಉ࣌ʹಈ͔͓ͯ͘͠
    w ༧ଌਫ਼౓͕མͪͨ PSσʔλ͕଍Γͳͯ͘௿͍
    ࣌ʹϏδωεࢦඪ͕ѱ
    ͍ͷ͸ظ଴௨Γͷಈ࡞
    w ༧ଌਫ਼౓ͷྑ͞ͱϏδωεࢦඪͷྑ͕͞࿈ಈ͍ͯ͠Ε͹0,
    w ແ૬ؔͰ͋Ε͹ͦͷ༧ଌॲཧ͸ୀ໾Մೳ

    View Slide

  25. ༧ଌϞσϧͷੑೳ؂ࢹ ଓ͖

    w Ϗδωεࢦඪͷ؂ࢹ
    w ച্ɾૈརɾ$1" ίϯόʔδϣϯ֫ಘ݅͋ͨΓͷίετ

    w ༧ଌث͕ࣄۀʹߩݙͰ͖͍ͯΔ͔Θ͔Δ
    w ٸܹʹѱԽͨ͠৔߹ɺେ఍γεςϜʹԿ͔͕ى͖ͯΔ
    w ༧ଌਫ਼౓
    w ࣮ࡍͷ഑৴࣌ͷ༧ଌ஋ͱ഑৴݁ՌΛ࢖ͬͯ޿ࠂΩϟϯϖʔϯຖʹ"6$
    ͷ࣌ܥྻΛϓϩοτ
    w Ωϟϯϖʔϯຖʹϓϩοτ͢ΔࣄͰ঎ࡐʹΑΔ܏޲ͷ͕ࠩݟ͑Δ

    View Slide

  26. ༧ଌϞσϧͷੑೳ؂ࢹ ଓ͖

    w ༧ଌثຖͷ࢖༻ঢ়گ
    w ݺग़ճ਺ɾൃੜച্ ো֐ൃੜ࣌ͷϦΧόϦ༏ઌ౓

    w ޿ࠂӡ༻ऀ͕഑৴ϩδοΫ ༧ଌث
    Λબ΂Δέʔε
    w ྑ͍෺Ͱ΋ʮಈ࡞͕೺ѲͰ͖ͳ͍͔Β࢖ͬͯͳ͍ʯͱ͍͏ࣄ΋͋Δ
    w ίϛϡχέʔγϣϯऔ͍ͬͯ͘
    w ࢖ΘΕͳ͘ͳͬͨ෺͸ୀ໾ͤ͞Δ
    w ࢖͍ͬͯͳͯ͘΋ҡ࣋අ͕͔͔Δ

    View Slide

  27. ܇࿅σʔλͷ܏޲ͷมԽ
    w σʔλੜ੒ܦ࿏ͷো֐
    w ܇࿅σʔλͷݩʹͳΔ্ྲྀͷϑΟϧλ͕ނো
    w ܭଌܥͷো֐ FHίϯόʔδϣϯܭଌ͕ࢭ·ͬͨ

    w ো֐ͳΒ܇࿅σʔλΛ៉ྷʹͯ͠࠶܇࿅͢Δͱੑೳ͕ݩʹ໭Δ
    w Ϗδωεঢ়گͷมԽ
    w ༧ଌର৅ͷ܏޲͕มΘͬͨ
    w ܏޲ͷมԽ͕Θ͔Ε͹࣍ͷҰखΛܾΊ΍͍͢ɻ৳ͼ͠Ζͷ͋Δ෦෼ͷ
    νϡʔχϯά΍৽ن։ൃʹணखɻ

    View Slide

  28. ৽͍͠༧ଌثͷϦϦʔε

    View Slide

  29. ΦϑϥΠϯධՁ
    w ৽͍͠༧ଌثͷ࣮ݧɺϏδωεࢦඪ͕Ͳ͏ͳΔ͔Λ֬ೝ͢Δ
    w աڈͷ഑৴ϩάΛ࢖ͬͯධՁ
    w աڈͷ഑৴ϩάߦຖʹ༧ଌ஋Λੜ੒
    w #JH2VFSZ͔ΒಡΜͰ#JH2VFSZ΁ॻ͖ࠐΉͷ͸$MPVE%BUBqPX͕ศར
    w ͔͠͠ϩάΛ࢖ͬͨ൓࣮Ծ૝੓ࡦͷධՁ͸ݫີʹ͸Ͱ͖ͳ͍
    w ܇࿅σʔλ΁ͷϑΟʔυόοΫ͕γϛϡϨʔτͰ͖ͳ͍
    w ඇఆৗੑ
    w ධՁ݁Ռ͸ࢀߟఔ౓ʹ

    View Slide

  30. ΦϯϥΠϯධՁ
    w ͪ͜Β͕ຊ໋
    w ϩδοΫ ༧ଌث
    ͷ"#ςετ
    w ϓϩμΫγϣϯ؀ڥͰαϯϓϧΛແ࡞ҝʹ܈ʹ෼͚ΔϥϯμϜԽൺֱࢼݧ
    w ࣌ؒมԽͷӨڹɾηϨΫγϣϯόΠΞεͷআڈ
    w ίϯτϩʔϧ܈ͱτϦʔτϝϯτ܈ͷϏδωεࢦඪͷ͕ࠩҼՌޮՌ
    w ϏδωεΠϯύΫτͰධՁ ౷ܭతʹ༗ҙͰ΋͕ࠩখ͍͞ͱඍົ

    View Slide

  31. ΦϯϥΠϯධՁ
    w ͪ͜Β͕ຊ໋
    w ϩδοΫ ༧ଌث
    ͷ"#ςετ
    w ϓϩμΫγϣϯ؀ڥͰαϯϓϧΛແ࡞ҝʹ܈ʹ෼͚ΔϥϯμϜԽൺֱࢼݧ
    w ࣌ؒมԽͷӨڹɾηϨΫγϣϯόΠΞεͷআڈ
    w ίϯτϩʔϧ܈ͱτϦʔτϝϯτ܈ͷϏδωεࢦඪͷ͕ࠩҼՌޮՌ
    w ϏδωεΠϯύΫτͰධՁ ౷ܭతʹ༗ҙͰ΋͕ࠩখ͍͞ͱඍົ

    ஌Γ͍ͨͷ͸ίί
    ྫૈར͕ͲΕ͚ͩ૿͑Δ͔

    View Slide

  32. ϦϦʔε
    w "#5FTU3FBEZͳγεςϜʹͳ͍ͬͯΕ͹ϦϦʔε΋༰қ
    w ద༻ϩδοΫΛ੍ޚ͢Δ࢓૊ΈΛ࡞͓ͬͯ͘
    w ؅ཧը໘ͰઃఆͰ͖Δ
    w "#ςετ
    w શମͷ͚ͩϦϦʔε
    w ಛఆҊ݅ͷΈʹద༻
    w ؾܰʹϦϦʔεͯ͠ݕূͰ͖Δ࢓૊Έେࣄ

    View Slide

  33. ઃఆྫ
    ઃఆ໊ ಺༰ ద༻࿮਺
    999ϩδοΫ"#ςετ
    999-PHJD7
    999-PHJD7

    σϑΥϧτ 999-PHJD7
    όάYYͷ࢑ఆରԠ /BJWF-PHJD

    View Slide

  34. ·ͱΊ
    w ޿ࠂ഑৴αʔόʔͷதͰ༧ଌॲཧ͕ಈ͍͍ͯΔ
    w ϦϦʔε͔ͯ͠Β΋΍Δࣄ͸ͨ͘͞Μ͋Δ
    w ো֐ରԠɾػೳվળ
    w ؂ࢹ͸ॏཁ
    w ݕূɾϦϦʔεɾୀ໾Λ΍Γ΍͘͢
    w "#5FTU3FBEZʹ͓ͯ͘͠ͱָ
    w πϥΠ࿩͋Μ·Γͳ͔ͬͨʜʜ

    View Slide

  35. ࢀߟจݙ
    <>8BMMBDF #ZSPO$ FUBM$MBTTJNCBMBODF SFEVY%BUB.JOJOH *$%.

    *&&&UI*OUFSOBUJPOBM$POGFSFODFPO*&&&
    <>$IBQFMMF 0MJWJFS &SFO.BOBWPHMV BOE3PNFS3PTBMFT4JNQMFBOE
    TDBMBCMFSFTQPOTFQSFEJDUJPOGPSEJTQMBZBEWFSUJTJOH"$.5SBOTBDUJPOTPO
    *OUFMMJHFOU4ZTUFNTBOE5FDIOPMPHZ 5*45


    <>)F 9JOSBO FUBM1SBDUJDBMMFTTPOTGSPNQSFEJDUJOHDMJDLTPOBETBU
    GBDFCPPL1SPDFFEJOHTPGUIF&JHIUI*OUFSOBUJPOBM8PSLTIPQPO%BUB
    .JOJOHGPS0OMJOF"EWFSUJTJOH"$.
    <>4DVMMFZ % FUBM)JEEFOUFDIOJDBMEFCUJONBDIJOFMFBSOJOH
    TZTUFNT"EWBODFTJOOFVSBMJOGPSNBUJPOQSPDFTTJOHTZTUFNT

    View Slide