会社訪問アプリ「Wantedly Visit」のデータで見る相互推薦システム / deim2022-rrs-wantedly-visit

Slide 1

Slide 1 text

Slide 2

Slide 2 text

©2022 Wantedly, Inc. 1. ͸͡Ίʹ • ࣗݾ঺հɺձࣾͱϓϩμΫτͷ঺հ • ϓϩμΫτʹ͓͚ΔσʔλαΠΤϯεͷऔΓ૊Έࣄྫ • ΞΧσϛΞʹ͓͚Δ׆ಈ 2. ૬ޓਪનγεςϜͱ͸ • ૬ޓਪનγεςϜͷ֓ཁɾಛ௃ • طଘख๏ͷ঺հ 3. ձࣾ๚໰ΞϓϦʮWantedly VisitʯͷσʔλͰݟΔ૬ޓਪનγεςϜ • Wantedly Visit ͷσʔλʹରͯ͠طଘख๏Λద༻࣮ͨ͠ݧɾߟ࡯ • վྑख๏ͷఏҊɾ࣮ݧɾߟ࡯ ձࣾ๚໰ΞϓϦʮWantedly VisitʯͷσʔλͰݟΔ૬ޓਪનγεςϜ

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

©2022 Wantedly, Inc. User Item ैདྷͷҰൠతͳਪનγεςϜ ૬ޓਪનγεςϜ User(Female) User(Male) User(Job Seeker) User(Recruiter/Company) ex. Amazon, Netflix ex. Tinder, Pairs ex. Wantedly, LinkedIn ૬ޓਪનγεςϜʢReciprocal Recommender Systemsʣͱ͸ʁ ʮαʔϏε಺ͷϢʔβΛޓ͍ʹਪન͢ΔγεςϜʯ ਪન ਪન ਪન

Slide 16

Slide 16 text

Slide 17

Slide 17 text

©2022 Wantedly, Inc. User A to User B Preference Score User B to User A Preference Score Reciproca l Preference Score Aggregatio n 1. γεςϜ಺ͷϢʔβ͸ A ͱ B ͷ̎ͭͷάϧʔϓʹ෼͔Ε͓ͯΓɺҟͳΔάϧʔϓͷϢʔβ͕ޓ͍ਪન͞ΕΔ ΋ͷͱ͢Δɻʢe.g. σʔςΟϯάαʔϏεʹ͓͚ΔஉঁɺٻਓαʔϏεʹ͓͚Δٻ৬ऀͱاۀʣ 2. ୯ํ޲ͷᅂ޷ͷେ͖͞Λද͢ Preference Score ΛɺA ͔Β B ΁ͷϢʔβٴͼ B ͔Β A ͷϢʔβͷͦΕͧΕ ʹ͍ͭͯܭࢉ 3. Aggregation Function Λར༻ͯ͠ɺ̎ͭͷ Preference Score Λ૊Έ߹Θͤͯ૒ํ޲ͷᅂ޷ͷେ͖͞Λද͢ Reciprocal Preference Score Λܭࢉ ૬ޓਪનγεςϜʹ͓͚Δᅂ޷ͷ༧ଌ

Slide 18

Slide 18 text

©2022 Wantedly, Inc. ✓ [RECON] (Pizzato 2010) ‣ ϢʔβͷϓϩϑΟʔϧ৘ใΛར༻ͨ͠ίϯςϯπϕʔεϑΟϧλϦϯάͰ Preference Score Λࢉग़ ‣ Aggregation Function ʹ͸ௐ࿨ฏۉΛར༻ ✓ [RCF] (Xia 2015 ) ‣ ߦಈཤྺʹجͮ͘ϝϞϦϕʔεͷϢʔβϕʔεڠௐϑΟϧλϦϯάʢk-ۙ๣ʣͰ Preference Score Λࢉग़ ✓ [LFRR](Neve 2019) ‣ ߦಈཤྺʹج͖ͮ࡞੒ͨ͠ User-User ߦྻʹ Matrix Factorization Λద༻ͯ͠ Preference Score Λࢉग़ ‣ Aggregation Function ʹ͍ͭͯɼௐ࿨ฏۉҎ֎ͷؔ਺ʹ͍ͭͯ΋ൺֱ࣮ݧ ✓ [ImRec](Neve 2020 ) ‣ ϢʔβͷϓϩϑΟʔϧը૾Λར༻ͨ͠ίϯςϯπϕʔεϑΟϧλϦϯάͰ Preference Score Λࢉग़ ૬ޓਪનγεςϜʹ͓͚Δᅂ޷ͷ༧ଌͷطଘख๏

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

©2022 Wantedly, Inc. Ϟνϕʔγϣϯ δϣϒϚονϯάαʔϏεʹ͓͚ΔਪનγεςϜͷॏཁੑ • ʮಇ͘ʯΛऔΓר͘؀ڥ͸೔ʑมԽɺෳࡶԽ • ಇ͖ํͷଟ༷Խʢe.g. ϦϞʔτϫʔΫɺ෭ۀɾϑϦʔϥϯεɺ৽ଔҰׅ࠾༻ͷഇࢭʣ • ৬छͷଟ༷Խʢe.g. σʔλαΠΤϯςΟετɾPdMʣ • ͦ΋ͦ΋ਓ͸ࣗ෼͕ຊ౰ʹ΍Γ͍ͨ͜ͱΛࣗ෼Ͱ෼͔͍ͬͯͳ͍ →ਪનγεςϜʹΑΔҙࢥܾఆͷิॿͷॏཁੑ͕ߴ·Δ

Slide 22

Slide 22 text

©2022 Wantedly, Inc. Ϟνϕʔγϣϯ ॏཁͳҰํͰݚڀ͕·ͩଟ͍Θ͚Ͱͳ͘ɺ͞ΒͳΔվળͷ༨஍ • ηϯγςΟϒͳྖҬͳͷͰσʔλ΍։ൃɾݚڀ੒ՌΛެ։͢Δ͜ͱ͕ࠔ೉ • ૬ޓਪનγεςϜͷطଘݚڀ΋σʔςΟϯάαʔϏεʹ͍ͭͯͷ΋ͷ͕ଟ͍ • طଘݚڀͷධՁ࣮ݧͰར༻͞ΕΔख๏͸·ͩ·ͩൃల్্ • ҰൠతͳਪનγεςϜʹͯ׆༻͞ΕΔख๏Λͦͷ··ྲྀ༻Ͱ͖Δ෦෼΋ • ૬ޓਪનγεςϜͳΒͰ͸ͷख๏΋͜Ε͔ΒͲΜͲΜग़ͯ͘ΔͰ͋Ζ͏

Slide 23

Slide 23 text

©2022 Wantedly, Inc. Ϟνϕʔγϣϯ Wantedly ʹे෼ͳσʔλ͕஝ੵ͞Ε͖ͯͨ • ొ࿥اۀϢʔβ3.2ສࣾҎ্ɺݸਓϢʔβ 330ສਓҎ্ • ΧδϡΞϧͳձࣾ๚໰ΞϓϦͰ͋Δ͔Βͦ͜ɺҰൠతͳస৬αʔϏεΑΓ΋େ ྔͷଟ༷ͳσʔλ͕ू·Δ • స৬׆ಈ࣌ʹʮબߟʯͷલʹͱΓ͋͑ͣͨ͘͞Μͷاۀͱ࿩͢ • ඇస৬࣌ʹ΋৘ใऩू໨తͳͲͰͱΓ͋͑ͣاۀͱ࿩ͯ͠ΈΔ • اۀͱ࿩͢ɺҎ֎ͷ໨తʢe.g. ϓϩϑΟʔϧɾϒϩάػೳɾϛʔτΞοϓػ ೳʣͰ΋ීஈ͔Βར༻͞ΕΔ

Slide 24

Slide 24 text

©2022 Wantedly, Inc. ࣮ݧ֓ཁ Wantedly Visit ʹ͓͚ΔϢʔβͱاۀͷ Matching Λ༧ଌ Company User Ԡื ϝοηʔδฦ৴ Company User εΧ΢τૹ৴ ϝοηʔδฦ৴ Company User Ԡื ӾཡͷΈ Company User εΧ΢τૹ৴ ӾཡͷΈ • Ϣʔβ͕Ԡื or اۀ͕εΧ΢τૹ৴ͨ͠ࡍʹ૬ख͕ϝοηʔδΛฦ৴͢Ε͹ Match (positive ) • Ԡื or εΧ΢τૹ৴Λ૬ख͕Ӿཡ্ͨ͠ͰϝοηʔδΛฦ৴͠ͳ͚Ε͹ Not Match (negative) Match Not Match How

Slide 25

Slide 25 text

©2022 Wantedly, Inc. ᅂ޷σʔλ/ධՁ஋ • Ϣʔβͷᅂ޷σʔλ • اۀ΁ͷԠื • اۀ͔ΒͷεΧ΢τૹ৴ʹର͢Δϝοηʔδฦ৴ • اۀͷᅂ޷σʔλ • Ϣʔβ΁ͷεΧ΢τૹ৴ • Ϣʔβ͔ΒͷԠืʹର͢Δϝοηʔδฦ৴ • ධՁ஋: Ϣʔβͱاۀͷ૊ʹରͯ͠༩͑ΒΕΔ (boolean ) • Ϣʔβ͕Ԡื or اۀ͕εΧ΢τૹ৴ͨ͠ࡍʹ૬ख͕ϝοηʔδΛฦ৴͢Ε͹ Match (positive ) • Ԡื or εΧ΢τૹ৴Λ૬ख͕Ӿཡ্ͨ͠ͰϝοηʔδΛฦ৴͠ͳ͚Ε͹ Not Match (negative ) • Ԡื or εΧ΢τૹ৴͕ൃੜ͕ͨ͠૬ख͕Ӿཡ͍ͯ͠ͳ͍΋ͷͷධՁ஋͸ෆ໌ʢະධՁʣ

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

©2022 Wantedly, Inc. • Preference Score ༧ଌܭࢉͷͨΊͷΞϧΰϦζϜ • ߦಈཤྺʹجͮ͘ϝϞϦϕʔεͷϢʔβϕʔεڠௐϑΟϧλϦϯά [RCF] (Xia 2015 ) • ߦಈཤྺʹج͖ͮ࡞੒ͨ͠ User-User ߦྻʹ Matrix Factorization Λద༻ [LFRR](Neve 2019 ) • Aggregation Function • (Neve 2019)Ͱ࣮ݧ͞Ε͍ͯͨ4छྨ • Arithmetic Mean (AM ) • Geometric Mean (GM ) • Harmonic Mean (HM ) • Cross-Ratio Uninorm (CRU) طଘख๏ᶃ User A to User B Preference Score User B to User A Preference Score Reciproca l Preference Score Aggregatio n Function CRU:

Slide 29

Slide 29 text

©2022 Wantedly, Inc. ࣮ݧ݁Ռᶃ AUC AM GM HM CRU RCF 0.555 0.605 0.623 0.601 LFRR 0.549 0.559 0.566 0.513 RCF LFRR • طଘݚڀͱಉ༷ɺHM(ௐ࿨ฏۉ)͕΋ͬͱ΋ߴ͍ੑೳʹ • طଘݚڀͱҟͳΓɺLFRR ΑΓ΋ RCF ͷํ͕ߴ͍ੑೳʹ Aggregation Function Algorithm

Slide 30

Slide 30 text

©2022 Wantedly, Inc. طଘख๏ᶄ DEIM2021 ͷٕज़ใࠂ https://speakerdeck.com/yuya4/deim2021-rrs-wantedly-visit • ᅂ޷σʔλͷྔͱ࣭Λ্͛ΔͨΊʹෛͷᅂ޷σʔλΛ׆༻ʢNegʣ • Ԡื/εΧ΢τૹ৴͞ΕͯӾཡ͕ͨ͠ϝοηʔδฦ৴͠ͳ͔ͬͨͱ͍͏ෛͷᅂ޷σʔλ΋ར༻ • ਖ਼ͷᅂ޷σʔλΛར༻ͨ͠ਖ਼ͷ Preference Score ͷ༧ଌʹՃ͑ɺෛͷ Preference Score Λ༧ ଌɺ̎छྨͷ Preference Score Λ଍͠߹Θͤͯ Preference Score ΛಘΔ • ݸਓϢʔβͱاۀϢʔβͷ Preference Score ͷ܏޲ͷࠩΛิਖ਼ʢScalerʣ • ݸਓϢʔβͱاۀϢʔβͷ Preference Score ͦΕͧΕʹ MinMaxScaler Λద༻্ͨ͠Ͱ Aggregation Function Λར༻ͯ͠ Reciprocal Preference Score ΛಘΔ

Slide 31

Slide 31 text

©2022 Wantedly, Inc. ࣮ݧ݁Ռᶄ AUC AM GM HM CRU RCF 0.555 0.605 0.623 0.601 LFRR 0.549 0.559 0.566 0.513 LFRR + Scaler + Neg 0.651 0.692 0.713 0.597 LFRR + Scaler + Neg Aggregation Function Algorithm ఏҊख๏Ͱ͋Δ LFRR + Scaler + Neg ͱ HM ͷ૊Έ߹Θ͕ͤ࠷΋ߴ͍ੑೳʹ طଘख๏ᶃͱͷൺֱ

Slide 32

Slide 32 text

©2022 Wantedly, Inc. • Scaler ʹΑͬͯݸਓϢʔβͱاۀϢʔβͷશମͷࠩ͸ߟྀ͕ͨ͠ɺݸਓϢʔβ ͝ͱ/اۀϢʔβ͝ͱͷධՁ܏޲ͷҧ͍ΛߟྀͰ͖͍ͯͳ͍ • ৻ॏʹߟ͑ͯ਺݅ʹԠื͢Δ/ؾܰʹେྔʹԠื͢ΔݸਓϢʔβ • ͓ۚͱਓతϦιʔεͷ५୔ͳاۀ΄ͲεΧ΢τૹ৴ྔ͕େ͖͘ • ༗໊ɾਓؾͳاۀʹ͸Ԡื͸ूத͢Δ • ࿩୊ੑΛूΊΔૂ͍ͳͲͷઑͬͨืूΛग़ͨ͠اۀʹ΋Ԡื͸ूத • ෛͷᅂ޷σʔλͷ׆༻ʢNegʣ ʹΑ͕ͬͯࠩΑΓݦஶʹ • ݸਓϢʔβͷෛͷᅂ޷σʔλ͸εΧ΢τΛड͚औͬͨࡍʹ͔͠ൃੜ͠ͳ͍ • εΧ΢τΛड͚औΕΔͷ͸Ұ෦ͷݸਓϢʔβʔͰ͋ΓɺͦͷதͰภΓ΋ • ԠืΛड͚ͨΒϚον͍ͯ͠ͳ͘ͱ΋ͱΓ͋͑ͣฦ৴͢Δӡ༻ͷاۀ ՝୊

Slide 33

Slide 33 text

©2022 Wantedly, Inc. 1. Preference Score ͷ༧ଌʹ Biased Matrix Factorization Λద༻ʢBiasʣ 2. Reciprocal Preference Score ܭࢉ࣌ͷ Aggregation Function ʹϩδεςΟ ΫεճؼΛ࠾༻͠ɺϚονʹ࠷దͳॏΈΛֶशͯ֫͠ಘʢLR()ʣ ࣮ݧ͢Δख๏ ՝୊Ͱ͋ΔݸਓϢʔβ͝ͱ/اۀϢʔβ͝ͱͷධՁ܏޲ͷҧ͍Λߟྀ͢ΔͨΊ

Slide 34

Slide 34 text

©2022 Wantedly, Inc. • ैདྷͷਪનγεςϜͷݚڀͰߴ͍ੑೳΛތΔɺݸਓϢʔβʔ͝ͱاۀϢʔβʔ͝ ͱͷόΠΞε΋ֶश͢Δ Biased Matrix Factorization Λར༻ • MF : • Biased MF: ࣮ݧ͢Δख๏ᶃ Preference Score ͷ༧ଌʹ Biased Matrix Factorization Λద༻ʢBiasʣ min p,q ∑ u,i∈ℝ+ (rui − pT u qi )2 + λ ( ||pu ||2 + ||qi ||2 ) min p,q ∑ u,i∈ℝ+ (rui − (pT u qi + bui )2 + λ ( ||pu ||2 + ||qi ||2 + b2 u + b2 i ) bui = μ + bu + bi

Slide 35

Slide 35 text

©2022 Wantedly, Inc. • Reciprocal Preference Score ͷ༧ଌੑೳ͕େ͖͘޲্ • ҰํͰɺͦΕͧΕͷ Preference Score ͷ༧ଌੑೳ͸௿Լ • ͜Ε·Ͱ͸ภΓͷେ͖͍ “౰ͯ΍͍͢” ݸਓϢʔβʔ΍اۀϢʔβʔͷᅂ޷Λਖ਼͘͠ ༧ଌ͢Δ͜ͱͰߴ͍༧ଌੑೳͱͳ͍͔ͬͯͨ ࣮ݧ݁Ռ AUC Reciprocal PS User PS Company PS ఏҊख๏ᶄ 0.713 0.723 0.721 ఏҊख๏ᶄ + Bias 0.780 0.657 0.702 Preference Score ͷ༧ଌʹ Biased Matrix Factorization Λద༻ʢBiasʣ

Slide 36

Slide 36 text

©2022 Wantedly, Inc. ࣮ݧ͢Δख๏ᶄ Reciprocal Preference Score ܭࢉ࣌ͷ Aggregation Function ʹϩδεςΟΫεճؼΛ࠾༻͠ɺ Ϛονʹ࠷దͳॏΈΛֶशͯ֫͠ಘʢLR()ʣ • ݸਓϢʔβʔͱاۀϢʔβʔͷ૊ʹରͯ͠ɺͦΕͧΕͷ༧ଌ Preference Score ೖྗͱͯ͠ɺϚον ͔ͨ͠Ͳ͏͔Λ༧ଌ͢ΔϩδεςΟοΫճؼʹΑΓॏΈΛֶशʢLR(preds)ʣ logit(p) = a + b1 * PSu + b2 * PSc PSu PSc : ݸਓϢʔβʔͷ༧ଌ Preference Score : اۀϢʔβʔͷ༧ଌ Preference Score • ݸਓϢʔβʔɺاۀϢʔβʔ͝ͱʹҟͳΔॏΈΛಘΔͨΊʹɺone-hot ϕΫτϧΛೖྗʹՃֶ͑ͯश ʢLR(preds + one-hot)ʣ logit(p) = a + b1 * PSu + b2 * PSc + ∑ u∈ 𝕌 bu * onehotu + ∑ c∈ℂ bc * onehotc logit(p) = a + b1 * PSu + b2 * PSc + b3 * PSu * PSc + ∑ u∈ 𝕌 bu * onehotu + ∑ c∈ℂ bc * onehotc • ௐ࿨ฏۉͷੑೳ͕ྑ͍͜ͱ͔Βɺަޓ࡞༻Λߟྀ͢ΔͨΊʹݸਓͱاۀͷ Preference Score ͷੵΛ ೖྗʹՃֶ͑ͯशʢLR(preds + one-hot + interaction)ʣ

Slide 37

Slide 37 text

©2022 Wantedly, Inc. ࣮ݧ݁Ռᶅ AUC طଘख๏ᶃʢRCFʣ 0.623 طଘख๏ᶃʢLFRRʣ 0.566 طଘख๏ᶄʢLFRR + Scaler + Negʣ 0.713 طଘख๏ᶄ + Bias 0.780 طଘख๏ᶄ + Bias + LR(preds) 0.772 طଘख๏ᶄ + Bias + LR(preds + one-hot) 0.803 طଘख๏ᶄ + Bias + LR(preds + one-hot + interaction) 0.817 ఏҊख๏ͱطଘख๏ᶃᶄͱͷൺֱ·ͱΊ • Aggregation Function ͷॏΈΛϩδεςΟοΫճؼʹΑΓಘΔ͜ͱͰશମతʹ༧ଌ ੑೳ͸޲্ • ࠷ऴతʹɺࠓճͷఏҊख๏Λ͢΂ͯՃ͑ͨ΋ͷͷੑೳ͕࠷΋ߴ͘

Slide 38

Slide 38 text

©2022 Wantedly, Inc. • ૬ޓਪનγεςϜͷ঺հ • ʮαʔϏε಺ͷϢʔβΛޓ͍ʹਪન͢ΔγεςϜʯ • ͓ޓ͍ͷᅂ޷͕Ұகͯ͠ॳΊͯਪન͕"੒ޭ"ͨ͜͠ͱʹͳΔ • ݚڀ͸·ͩ·ͩ͜Ε͔Βൃలͷ༨஍͋Γ • ձࣾ๚໰ΞϓϦ Wantedly Visit ͷ࣮σʔλΛར༻ͨ͠ݕূ࣮ݧͷ঺հ • طଘݚڀͷख๏ΛϕʔεϥΠϯͱͨ͠վળख๏ͷධՁ࣮ݧ • γεςϜ಺ͷ༷ʑͳϢʔβʔͷੑ࣭Λߟྀͨ͠վળख๏ʹΑΓੑೳͷ޲্Λ֬ೝ • Biased Matrix Preference Score ͷ༧ଌʹ Biased Matrix Factorization Λద༻ʢBiasʣ • Reciprocal Preference Score ܭࢉ࣌ͷ Aggregation Function ʹϩδεςΟΫεճؼΛ࠾༻͠ɺ Ϛονʹ࠷దͳॏΈΛֶशͯ֫͠ಘʢLR()ʣ ·ͱΊ

Slide 39

Slide 39 text

©2022 Wantedly, Inc. 3FGT • (Pizzato 2010) Luiz Pizzato, Tomek Rej, Thomas Chung, Irena Koprinska, and Judy Kay. 2010. RECON: a reciprocal recommender for online dating. Proceedings of the fourth ACM conference on Recommender systems P. 207-214 . • (Pizzato 2012) Luiz Pizzato, Tomasz Rej, Joshua Akehurst, Irena Koprinska, Kalina Yacef, and Judy Kay. 2012. Recommending people to people: the nature of reciprocal recommenders with a case study in online dating. User Model User-Adap Inter (2013) 23: 447 . • (Xia 2015) Peng Xia, Benyuan Liu, Yizhou Sun, and Cindy Chen. 2015. Reciprocal Reciprocal recommendation System for Online Dating. Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining P. 234-241. • (Neve 2019) J Neve, I Palomares.2019. Latent Factor Models and Aggregation Operators for Collaborative Filtering in Reciprocal Recommender Systems Proceedings of the 13th ACM Conference on Recommender Systems, 219-227 . • (Neve 2020) J Neve, R McConville.2020. ImRec: Learning Reciprocal Preferences Using Images. Proceedings of the 14th ACM Conference on Recommender Systems, 170-179.