machine learning graph pitch #1 https://machine-learning-pitch.connpass.com/event/130083/ で話した資料です。 有向グラフ上での Graph Embeddings と、それを活用したつながり推薦について話しました。
©2019 Wantedly, Inc.Improving “People You May Know” on Directed Social GraphMachine Learning Graph Pitch #1May 13, 2019 - Naomichi AgataGraph Embedding Λ༻͍ͨํͭͳ͕Γ༧ଌ
View Slide
©2018 Wantedly, Inc.Naomichi Agata- 2018/04~ Wantedly, Inc.- Wantedly People ͷػցֶशΤϯδχΞ- Data Driven Developer Meetup ӡӦϝϯόʔ- https://d3m.connpass.com@agatan_ࣗݾհ
©2019 Wantedly, Inc.• ͳʹΛղܾ͍ͨ͠ͷ͔• Graph Embeddings• ͭͳ͕ΓͷछྨΛ׆༻͢Δ• ·ͱΊAgenda
©2019 Wantedly, Inc.ͳʹΛղܾ͍ͨ͠ͷ͔ղ͖͍ͨͱͦͷલఏ
©2019 Wantedly, Inc.ʮͭͳ͕ΓʯͷΛ;͍ͨ͠ʂղ͖͍ͨ Wantedly Peopleʮͭͳ͕ΓʯΛཧ͢ΔΞϓϦ ʢ໊ΛࡱΔPSͭͳ͕ΓϦΫΤετʣˠͭͳ͕Δʂ ͭͳ͕ͬͨਓʹؔ࿈͢Δใ͕Θ͔Δʂ ͨͱ͑Wantedlyࣾһͱͭͳ͕ΔͱʮWantedly, Inc.ʯʹؔ࿈͢Δχϡʔε͕ݟΒΕΔ Ϣʔβಉ͕࢜ͭͳ͕Δ΄Ͳྑ͍ମݧ͕ఏڙͰ͖Δͣ
©2019 Wantedly, Inc.ํͷʮͭͳ͕ΓʯͷΛ;͍ͨ͠ʂ Wantedly Peopleͷͭͳ͕ΓϞσϧʮ༗άϥϑʯͰɺҰํͷͭͳ͕Γ͋Δ lͭͳ͕ΓϦΫΤετz ͨͱ͑"͞Μ͕#͞Μͷ໊ΛࡱӨ͢Δͱɺ"͞Μˠ#͞ΜʹϦΫΤετ͕ૹΒΕΔ Β͕;͍ͨ͠ͷʮํͷͭͳ͕Γʯ Ұํతͳͭͳ͕ΓϦΫΤετΛ;͍ͨ͠Θ͚Ͱͳ͍ݫີʹʜ
©2019 Wantedly, Inc.- ʮ͔ͯ͠͠Γ߹͍͔ʁʯΛද͍ࣔͨ͠- “People You May Know”- Link Prediction : ͋ΔϢʔβ A ͱ B ͕ͭͳ͕ΔΛ༧ଌ͍ͨ͠Ͳ͏ͬͯղ͔͘???
©2019 Wantedly, Inc.ॳظ ॳظϧʔϧϕʔε͔Βελʔτ ʮಉ͡ձࣾʯʮڞ௨ͷͭͳ͕Γ͕NਓҎ্ʯʮ͋ͳͨͷ໊ΛಡΈऔͬͨਓʯͳͲ ϧʔϧͷෳࡶԽਫ਼ɺਪન͕՝ʹͳΓɺվળϑΣʔζʹ ෳͷͭͳ͕ΓΛ͍ͬͯΔͷʹϧʔϧʹͯ·Βͳ͍ͨΊʹʮਪન͕ग़ͤͳ͍ʯϢʔβ͕͍ͨ
©2019 Wantedly, Inc.Graph Embeddings
©2019 Wantedly, Inc. ͜͜ͰGraphதͷ֤Nodeʹରͯ͠ʮϕΫτϧදݱʯΛ֫ಘ͢Δ͜ͱ ͭͳ͕͍ͬͯΔNodeͨͪۙ͘ɺͭͳ͕͍ͬͯͳ͍Nodeͨͪԕ͘ͳΔΑ͏ʹֶश͢Δ ʮྨࣅʯCosineྨࣅੵͰఆٛ͢ΔGraph Embeddings?AB Csimilarity(A, B) = ⃗a ⋅ ⃗bsimilarity(A, C) = ⃗a ⋅ ⃗c
©2019 Wantedly, Inc. ෳࡶͳؔΛදݱͰ͖Δʢ͜ͱ͕ظ͞ΕΔʣ άϥϑͱ͍͏ෳࡶͳσʔλߏͷใΛ࣍ݩϕΫτϧͱͯ͠ѻ͑ͨΒ͏Ε͍͠ʂ ʢยํͷΤοδؚΊΔͱʣԯΤοδͷάϥϑ͕γϯϓϧͳදݱʹམͱ͠ࠐΊΔ ਪનީิͷݕࡧ͕ߴʹ࣮ݱͰ͖Δ શ୳ࡧ͢Δͱͯ͠ੵΛऔΔ͚ͩ ʢଞͷλεΫͷԠ༻ʣ ಘΒΕͨembeddingsΛdownstream tasksͷೖྗʹ͔ͭ͑Δ Link Prediction downstream tasksͷͻͱͭͳͥGraph EmbeddingsΛ͔ͭ͏ͷ͔
©2019 Wantedly, Inc.ਪનͷશମ૾Embeddings ۙ୳ࡧ ճؼϞσϧ ਪન
©2019 Wantedly, Inc.ਪનͷશମ૾Embeddings ۙ୳ࡧ ճؼϞσϧ ਪન ͯ͢ͷϖΞΛྻڍͯ͠ܭࢉ͢Δͷܭࢉྔతʹ͖ͼ͍͠ embeddingͷใΛ͔ͭͬͯਪનީิΛߜΔ ճؼϞσϧRe-Rankͷׂ ۙࣅ࠷ۙ୳ࡧ(Approximate Nearest Neighbors) ͕͑ΔͷͰεέʔϧ͢Δ
©2019 Wantedly, Inc.ਪનͷશମ૾Embeddings ۙ୳ࡧ ճؼϞσϧ ਪન ϢʔβͷϖΞ͔Βʮͭͳ͕ΔʯΛ༧ଌ͢ΔϞσϧ ಛྔͱ֤ͯ͠ϢʔβͷϕΫτϧදݱΛ͏ ͦͷ΄͔ͷใʢ࣍ڞ௨ͷͭͳ͕ΓͳͲʣΛಛྔͱ͔ͯͭ͠͏͜ͱͰ͖Δ ͳͯ͘embeddingͷੵͷେ͖͞ॱʹιʔτ্ͯ͠ҐΛਪન͢Δํ๏͕ͱΕΔ ͋ͬͨ΄͏͕ਫ਼͕Α͔ͬͨ ֬ͱͯ͠ղऍͰ͖ΔΑ͏ʹͳΔͷͰѻ͍͍͢
©2019 Wantedly, Inc.ਪનͷશମ૾Embeddings ۙ୳ࡧ ճؼϞσϧ ਪન ͕͜͜ॏཁʂ ͪΌΜͱʮͭͳ͕Γͦ͏ͳϢʔβʯ͕ۙʹҐஔ͍ͯ͠ͳ͍ͱީิ͔Β࿙Εͯ͠·͏ άϥϑͷใΛ͏·͘ຒΊࠐΊ͍ͯͳ͍ͱճؼϞσϧͷਫ਼͕Ͱͳ͍ ͦͦEmbedding͕ఆٛͰ͖ͳ͍ϢʔβʹਪનΛग़ͤͳ͍
©2019 Wantedly, Inc.ͬͯΈͨ ·ͣํͷͭͳ͕ΓͷΈΛ׆༻ֶͯ͠शͨ͠ γϯϓϧͳDeepWalk Graph ্ΛϥϯμϜΥʔΫͯ͠ಘͨ Node ྻ͔Β Word2Vec ͱಉ͡Α͏ͳํ๏Ͱֶश ਪનΛग़ͤΔϢʔβ͕;͑ͨ͜ͱͰɺैདྷͷ2ഒͷͭͳ͕ΓΛͭͬͨ͘ʂ%FFQ8BMLPOMJOFMFBSOJOHPGTPDJBMSFQSFTFOUBUJPOT<#1SP[[J ,%%>
©2019 Wantedly, Inc.Ծઆ ͭͳ͕ΓϦΫΤετ͕ঝೝ͞Εͳ͔ͬͨ͜ͱ ʮ୯ํͷͭͳ͕ΓʯΛܦ༝ͯ͠ऑͭ͘ͳ͕͍ͬͯΔͱ͍͏ใҰํͰɺํͰͳ͍ΤοδΛࣺ͍ͯͯΔͷͰ͕׆͔͍ͤͯͳ͍ʜ͜ΕΒΛ׆༻Ͱ͖ͨΒͬͱྑ͘Ͱ͖ΔͷͰʁϦΫΤετ
©2019 Wantedly, Inc.ͭͳ͕ΓͷछྨΛ׆༻͢Δ
©2019 Wantedly, Inc.ੵίαΠϯྨࣅΛείΞͱͯ͠͏߹ͭͳ͕ΓͷछྨΛͲ͏දݱ͢Δ͔similarity(A, B) = ⃗a ⋅ ⃗b = ⃗b ⋅ ⃗a = similarity(B, A)ͳͷͰɺʮ"ˠ#ͷΤοδ͋Δͷʹ#ˠ"ͷΤοδͳ͍ʯΛදݱͰ͖ͳ͍ɻ
©2019 Wantedly, Inc.ͭͳ͕ΓͷछྨΛͲ͏දݱ͢Δ͔ͦ͜Ͱɺ"͔Β#ʹରͯ͠Sͱ͍͏Τοδ͕͋Δͱ͖ɺͦΕΛ࣍ͷΑ͏ʹදݱ͢Δscore(A, r, B) = ⃗a ⋅ fr( ⃗b )score(B, r, A) = ⃗b ⋅ fr( ⃗a )Τοδͷछྨ͝ͱʹఆٛ͞ΕΔϕΫτϧΛม͢ΔؔfrA Brfrখ͘͞ͳΔΑ͏ʹֶश⃗a ⋅ fr( ⃗b ) େ͖⃗͘b ⋅ fr( ⃗a )
©2019 Wantedly, Inc.ࠓճComplex EmbeddingsΛࢀߟʹ࣍ͷΑ͏ͳมΛߦͬͨ$PNQMFY&NCFEEJOHTGPS4JNQMF-JOL1SFEJDUJPO<55SPVJMMPO *$.->real, imag = embedding[:dim // 2], embedding[dim // 2:]concat(real * W_real - imag * W_imag, real * W_imag + image * W_real)W_real, W_imagֶश͞ΕΔύϥϝʔλͭͳ͕ΓͷछྨΛͲ͏දݱ͢Δ͔
©2019 Wantedly, Inc.࣮ݧ ʮํͷͭͳ͕ΓʯʮҰํ௨ߦͷͭͳ͕ΓʯΛ྆ํͻͱͭͷάϥϑ্Ͱදݱͯ͠embeddingsΛֶश ֶशͨ͠ϕΫτϧදݱΛಛྔʹͨ͠LightGBMͷϞσϧΛֶश͠ɺͦͷਫ਼ΛධՁ͢Δ ʮํͷͭͳ͕ΓʯͷΈΛͬͨ߹ͷੑೳͱൺֱ͢Δ
©2019 Wantedly, Inc.ࠓճͷ࣮ݧͰͷදݱํA͞Μ B͞Μ⃗a ⋅ ⃗b⃗b ⋅ ⃗a- src, dst ͱʹมͳ͠ ͲͪΒͷ͖͔Βݟͯಉ݁͡ՌʹͳΔΑ͏ʹscore(A, bidirected, B) = ⃗a ⋅ ⃗bscore(B, bidirected, A) = ⃗b ⋅ ⃗a
©2019 Wantedly, Inc.ࠓճͷ࣮ݧͰͷදݱҰํ௨ߦA͞Μ B͞Μ⃗a ⋅ f( ⃗b )⃗b ⋅ f( ⃗a )- dstଆΛม A → B positive͔ͭB → A negativeΛ දݱͰ͖ΔΑ͏ʹ͢Δ⃗a ⋅ f( ⃗b ) Λେ͖͘ʢA → BҰํ௨ߦʣ⃗b ⋅ f( ⃗a ) Λখ͘͞ʢ# → Aଘࡏ͠ͳ͍ʣ⃗a ⋅ ⃗b Λখ͘͞ʢA ͱ Bํͭͳ͕ΓͰͳ͍ʣ
©2019 Wantedly, Inc.݁Ռ ʮTop0Ҏʹͭͳ͕ΔϢʔβΛਪનͰ͖ΔʯϢʔβ10.3% ૿Ճ ͱͱਪનΛग़͍ͤͯͨϢʔβʹର͢Δਪનͷ࣭΄΅ಉ ʢAUC: 0.819 → 0.823ʣ ʮ୯ํͷΤοδΛ׆༻͢Δ͜ͱͰ࣭͕͋Δʯͱ͍͏݁ՌಘΒΕͳ͔ͬͨ खΛ͛ͯਫ਼͕མͪͳ͔ͬͨͱ͍͑ͦ͏
©2019 Wantedly, Inc.ͦͷଞͷ࣮ݧ͜Ε͔Β ͍Ζ͍ΖͬͯΈ͚ͨͲྑ͍݁Ռ͕ಘΒΕͳ͔࣮ͬͨݧͨͪʜ ʮ໊ܦ༝ʯʮϦΫΤετܦ༝ʯͱ͍ͬͨใΤοδͷϥϕϧͱͯ͠ѻͬͯΈΔ ϦΫΤετͷRejectΤοδͱͯ͠ѻͬͯΈΔ ৽͘͠ਪન͕ग़ͤΔΑ͏ʹͳͬͨϢʔβʹߜͬͨධՁ͕Ͱ͖͍ͯͳ͍ ୯ํͷΤοδ͔ͭ͏Α͏ʹͳͬͨ͜ͱͰɺ࣍ͷখ͍͞Ϣʔβଟؚ͘Ήάϥϑʹͳͬͨ ΄͔ͷλεΫͷԠ༻
©2019 Wantedly, Inc.·ͱΊ ํͱ୯ํͷͭͳ͕ΓΛ۠ผͯ͠ѻ͏͜ͱͰɺΑΓϦονͳใΛຒΊࠐΊͨʢʁʣ EstଆͷϕΫτϧදݱΛม͢Δ ༗ޮͳਪન͕Ͱ͖ͨϢʔβ͕10.3%૿Ճͨ͠ʂ ࠓޙ Τοδʹͬͱͨ͘͞Μͷใ͕͋ΔͷͰ׆༻͍ͨ͠ Link PredictionҎ֎ͷλεΫͷ׆༻