BigQuery as a Recommendation System

9/11(火) Data Driven Developer Meetup #1 (https://d3m.connpass.com/event/98934/)での発表内容です．
BigQuery を活用した Wantedly Visit における Recommendation System の事例についてお話させていただきました．

Transcript

1. ©2018 Wantedly, Inc. BigQuery as a Recommendation System BigQuery Λ׆༻ͨ͠ਪનγεςϜࣄྫ(In Wantedly Visit) Data Driven Developer Meetup #1 11.Sep.2018 - Yuya Matsumura - @yu-ya4
2. ©2018 Wantedly, Inc. •Yuya Matsumuraʢদଜ ༏໵ʣ •Wantedly, Inc. (ࣾձਓ6ϲ݄໨) •Recommendation Engineer •From Kyoto Self Introduction @yu-ya4 @yu__ya4 https://www.wantedly.com/users/2390451
3. ©2018 Wantedly, Inc.  ਪનγεςϜͱ͸
    8IZ
   VTF
   #JH2VFSZ
    )PX
   VTF
   #JH2VFSZ Agenda ※ຊൃද͸ɼ2018೥9݄ݱࡏͷWantedly Visit ʹ͓͚ΔਪનγεςϜͷ։ൃϑϩʔɼӡ༻ํ๏ʹج͓ͮ͘࿩Ͱ͢ɽ ಉ͡࿩͕ଞͷόοΫάϥ΢ϯυͰ΋ඞͣਖ਼͍͠ͱ͸ݶΓ·ͤΜɽ
4. ©2018 Wantedly, Inc.  ਪનγεςϜͱ͸
    8IZ
   VTF
   #JH2VFSZ
    )PX
   VTF
   #JH2VFSZ Agenda
5. ©2018 Wantedly, Inc. It is often necessary to make choices without sufficient personal experience of the alternatives. In everyday life, we rely on recommendations from other people either by word of mouth, recommendation letters, movie and book reviews printed in newspapers, or general surveys such as Zagat’s restaurant guides. Recommender systems assist and augment this natural social process. ਪનγεςϜͱ͸ [Resnick+ 97] ࣗ෼ͷܦݧ͔ΒͷΈͰ͸͋·ΓΑ͘Θ͔Βͳ͍΋ͷͷத͔ΒɼͲ͏ ͯ͠΋ͲΕ͔Λબ͹ͳ͚Ε͹ͳΒͳ͍ͱ͍͏͜ͱ͸Α͋͘Δɽ͜ͷ Α͏ͳࡍΘͨͨͪ͠͸ɼޱίϛɼਪનঢ়ɼ ৽ฉͷॻධ΍өըධɼϨ ετϥϯΨΠυͳͲͷଞਓ͔ΒͷਪનʹཔΔ͜ͱΛ ೔ৗతʹߦͬͯ ͍Δɽ ৘ใਪનγεςϜͱ͸ɼ͜͏ͨࣾ͠ձͰී௨ʹߦΘΕ͍ͯΔ Ұ࿈ͷߦҝΛิॿͨ͠Γɼଅਐͨ͠Γ͢Δ΋ͷͰ͋Δɽ [Resnick 97] P. Resnick and H. R. Varian. Recommender systems. Communications of the ACM, Vol. 40, No. 3, pp. 56–58, 1997.
6. ©2018 Wantedly, Inc. Recommenders: Tools to help identify worthwhile stuff ਪનγεςϜͱ͸ [Konstan+ 03] ਪનγεςϜ: Ձ஋ͷ͋Δ΋ͷ͕ͲΕ͔ಛఆ͢Δͷʹ໾ཱͭಓ۩ [Konstan 03] J. A. Konstan and J. Riedl. Recommender systems: Collaborating in commerce and communities. In Proc. of the SIGCHI Conf. on Human Factors in Computing Systems, Tutorial, 2003.
7. ©2018 Wantedly, Inc. ৘ใਪનͷྫ ಛఆͷϢʔβͷᅂ޷ʹ ߹ͬͨ঎඼ͷਪન
8. ©2018 Wantedly, Inc. ৘ใਪનͷྫ ྨࣅɾؔ࿈঎඼ ͷਪન (amazon.co.jp) ଞͷϢʔβʹΑΔධՁ
9. ©2018 Wantedly, Inc. ৘ใਪનͷྫ ਓؾ঎඼ͷਪન (amazon.co.jp) ৽ண঎඼ͷਪન
10. ©2018 Wantedly, Inc. Everything is a Recommendation
11. ©2018 Wantedly, Inc.  ਪનγεςϜͱ͸
     8IZ
    VTF
    #JH2VFSZ
     )PX
    VTF
    #JH2VFSZ Agenda
12. ©2018 Wantedly, Inc. ͱʹ͔͘ૣ͍ Why use BigQuery?
13. ©2018 Wantedly, Inc. ݱࡏͷWantedly Visit ͷ Recommendation TeamʹٻΊΒΕΔ͜ͱ վળαΠΫϧΛૉૣͨ͘͘͞Μճ͢͜ͱ
14. ©2018 Wantedly, Inc. ݱࡏͷWantedly Visit ͷ Recommendation ࣄ৘  γϯϓϧͳมߋͰվળͰ͖ͦ͏ͳՕॴ͕ෳ਺࢒͍ͬͯΔ
     νʔϜ͕খن໛ ࢲΛؚΊ໊ͯ
     αʔϏε͸খن໛͡Όͳ͍ ͨ͘͞ΜͷϢʔβ ͭͭͷվળʹΊͪΌͪ͘Ό࣌ؒΛ͔͚͍͍ͯΘ͚Ͱ͸ͳ͍
15. ©2018 Wantedly, Inc. ͨͱ͑͹… Ϣʔβͷืू΁ͷΫϦοΫ৘ใΛར༻ͯ͠ɼ ͋ΔϢʔβʹืूΛਪન͍ͨ͠ ⭕ ⭕ ⭕ ⭕ ⭕ ⭕ ⭕ ʁ ⭕ - - - - - - - ืू Ϣʔβ
16. ©2018 Wantedly, Inc. աڈͷΫϦοΫ৘ใΛجʹͨ͠Ϣʔβؒͷྨࣅ౓Λࢉग़ͯ͠ɼ ࣅ͍ͯΔϢʔβ͕ΫϦοΫͨ͠ืूΛਪન͠Α͏ʂ (ϢʔβϕʔεڠௐϑΟϧλϦϯά) ྨࣅ౓௿ ྨࣅ౓ߴ ⭕ ⭕ ⭕ ⭕ ⭕ ⭕ ⭕ ʁ ⭕ - - - - - - - ਪન ืू Ϣʔβ ͨͱ͑͹…
17. ©2018 Wantedly, Inc. ୯७ͦ͏Ͱߟ͑Δ͜ͱ/΍Δ͜ͱ͸ͨ͘͞Μ… վળαΠΫϧ͕ͳ͔ͳ͔ճͤͳ͍ ࣮ߦ଎౓͸໰୊ͳ͍ʁ σʔλྔ͍͢͝ ΦϑϥΠϯධՁ͸Ͳ͏͢Δʁ ΦϯϥΠϯධՁ͸ʁ ࣮૷ݴޠ͸ʁ ෼ࢄॲཧͤ͞Δʁ σʔλͷύΠϓϥΠϯઃܭ͸ʁ ϚγϯϦιʔε଍ΓͯΔʁ σʔλͷݕূ͸Ͳ͏͢Δʁ
18. ©2018 Wantedly, Inc.
19. ©2018 Wantedly, Inc. BigQueryͰશ෦΍ͬͪΌ͑
20. ©2018 Wantedly, Inc.  ਪનγεςϜͱ͸
     8IZ
    VTF
    #JH2VFSZ
     )PX
    VTF
    #JH2VFSZ Agenda
21. ©2018 Wantedly, Inc. Overview of architecture of Recommendation System in Wantedly Visit BigQuery merge Job ͷهड़(΄΅SQL) Scheduler ʹొ࿥ ΫΤϦͷ࣮ߦ ࣮ߦ݁ՌΛRDBʹอଘ Ωϟογϡͱͯ͠ར༻
22. ©2018 Wantedly, Inc. BigQuery by Ruby
23. ©2018 Wantedly, Inc. Jobͷఆٛ
24. ©2018 Wantedly, Inc. pros. w 42-
    ͚ͩॻ͚Ε͹͍͍ͷͰɼ࣮૷ίετ͕௿͍ɽ
    w ແݶͷϦιʔε
    w (PPHMFઌੜ
    ʹ՝ۚ͢Δ͚ͩͰര଎ʂ
    w #JH2VFSZ಺Ͱॲཧ͕׬݁
    w σʔλͷҠಈߟ͑Δඞཁͳ͠
    w %PNP #*πʔϧ ্Ͱεέδϡʔϥ͕࣮ߦՄೳ
    w "#ςετͳͲͷΦϯϥΠϯධՁΛܧଓతʹ௥͍΍͍͢
25. ©2018 Wantedly, Inc. cons. w ΫΤϦ͕ଐਓԽ͠΍͍͢
    w ਓͷॻ͍ͨΫΤϦಡΉͷŨŽŘ
    w σόοά͠ʹ͍͘
    w ຊ౰ʹਖ਼͍͠ग़ྗʁ
    w ෳࡶա͗Δॲཧ͸͠ΜͲ͍
    w ;ͱͨ͠ॠؒʹ1ZUIPO
    ͕࿀͘͠ͳΔ ݸਓతͳҙݟ
26. ©2018 Wantedly, Inc. ·ͱΊ BigQueryųƄƃž