Graph Convolutional Matrix Completion

書誌情報
タイトル: Graph Convolutional Matrix Completion
著者: Rianne van den Berg, Thomas N. Kipf, Max Welling
所属: AMLab (University of Amsterdam)
採択: KDD(Deep Learning Day) 2018

概要
レコメンドタスクにおけるMatrix CompletionをGraphにおけるラベル付きリンク予測に置き換えて解く手法を提案
ユーザーとアイテムのレーティング行列を2部グラフとして解釈し，Graph Convolutionを使ったGraph Auto-Encoderでユーザーとアイテムの特徴量を獲得する．
その際に，ユーザーやアイテムに関するコンテンツ情報も取り入れることができるモデル構造．
実験で複数のデータセットでSoTAを達成し，MovieLens10M等の大規模データセットにも適用可能であることを確認した．
また，コンテンツ情報をレコメンドに導入することで，Cold-start問題に対しても有効な手法であることを実験的に確認できた．

1. Graph Convolutional Matrix Completion Tomoki Tanimura Nakazawa Lab, Keio University, B3
2. Bibliography • Title: Graph Convolutional Matrix Completion • Accepted: KDD DeepLearning Day(2018) • Author: R. vd Berg, T. N. Kipf, M. Welling
 ʢAMLab, University of Amsterdamʣ • Max Wellingઌੜͷݚڀࣨ(Kingma͞Μ΋͜͜ʣ • ཧ࿦෺ཧग़਎ऀ͕ଟ͍ɽڧͦ͏ɽɽɽ • Graph Convolution͕ྲྀߦΔ͖͔͚ͬΛ࡞ͬͨϥϘ • ಛʹɼKipf͞Μ͸Graph ConvolutionerʢΠέϝϯ͗͢ΔPh.Dʣ 2 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
3. Abstract • ϨʔςΟϯάͷߦྻʢϢʔβʔͷΞΠςϜʹର͢ΔධՁ஋͕ ೖͬͨߦྻʣΛ࠶ߏ੒͢ΔϨίϝϯυλεΫ(CF)Λղ͍ͨ • ϨʔςΟϯάߦྻΛ2෦άϥϑͱͯ͠ղऍ͠ɼGraph ConvolutionΛ࢖༻ͨ͠Graph Auto-encoderΛ࢖༻ͯ͠༧ଌ 3 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
4. Agenda • Preliminary for Recommender System • Preliminary for Graph Convolution • Method Overview • Method Details • Experiment • Conclusion and Future works 4 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
5. Agenda • Preliminary for Recommender System • Preliminary for Graph Convolution • Method Overview • Method Details • Experiment • Conclusion and Future works 5 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
6. What is Recommender System? • ϨίϝϯγεςϜ • Recommenders: Tools to help identify worthwhile stuff
 (ਪનγεςϜ:ͲΕʹՁ஋͕͋Δ͔Λಛఆ͢ΔͷΛॿ͚Δಓ۩) • ྫʣݱࡏ·ͰͷϢʔβʔͷߪങཤྺΛར༻ͯ͠ɼࠓޙͷߪങ׆ ಈΛ༧ଌ͢Δ • ϨίϝϯυγεςϜ͸େ͖͘෼͚ͯ2ͭ • Content-based • Collaborative Filtering 6 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
7. Content-based • ΞΠςϜʢϢʔβʔʣʹؔ͢Δ৘ใ͔ΒɼΞΠςϜʢϢʔβʔʣ ͷಛ௃ྔΛઃܭ͠ɼͦͷྨࣅ౓Λܭࢉ
 →ྨࣅ౓ͷߴ͍ΞΠςϜΛϢʔβʔʹਪન 7 Πϝʔδ* * https://qiita.com/haminiku/items/f5008a57a870e0188f63 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
8. Collaborative Filtering • ΞΠςϜͱϢʔβʔͷߪങ׆ಈʹؔ͢Δߦྻʢ֤ཁૉ͸ධՁ ஋΍ߪೖ਺Λද͢ʣ͔ΒϢʔβʔ΍ΞΠςϜͷڞىੑ౳Λࢉ ग़͢Δ 8 Πϝʔδ* * https://qiita.com/haminiku/items/f5008a57a870e0188f63 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
9. Recent CF Method • ߦྻ෼ղʢMarix Factorizationʣ→༗໊ • ϨʔςΟϯάߦྻΛϢʔβʔͷಛ௃ྔͱΞΠςϜͷಛ௃ྔ ͷߦྻͷੵͰද͢͜ͱͰɼະ؍ଌͷϨʔςΟϯάΛ༧ଌ 9 * https://towardsdatascience.com/paper-summary-matrix-factorization-techniques-for-recommender-systems-82d1a7ace74 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
10. Matrix Factorization 10 R R I I J J X U V ஍Ҭϝ ογϡ ΰϛͷ࢒ᕓ ؍ଌ͞Εͨ਺ × ~ ~ R R I I J J X U V ஍Ҭϝ ογϡ ΰϛͷ࢒ᕓ ؍ଌ͞Εͨ਺ × ~ ~   %
    " 
    X UVT R R I I J J X U V ஍Ҭϝ ογϡ ΰϛͷ࢒ᕓ ؍ଌ͞Εͨ਺ × ~ ~ U R R I I J J X U V ஍Ҭϝ ογϡ ΰϛͷ࢒ᕓ ؍ଌ͞Εͨ਺ × ~ ~ V #!$      user item userͷಛ௃ྔ Λද͢ߦྻ itemͷಛ௃ྔ Λද͢ߦྻ ࠶ߏ੒͞Εͨ ϨʔςΟϯάߦྻ ݩͷϨʔςΟϯάߦྻ ʢͨͩ͠؍ଌ͞Εͨ஋ͷΈͷࠩ෼Λ࠷খԽ͢Δʣ Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
11. Interprete Matrix as Graph • ϨʔςΟϯάߦྻ͸ϢʔβʔͱΞΠςϜΛϊʔυͱ͢Δ2෦ά ϥϑͰදͤΔ 11 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
12. How to use Graph Convolution • άϥϑߏ଄Ͱදͨ͠ϨʔςΟϯάߦྻΛ༻͍ͯɼϢʔβʔͱ ΞΠςϜͷಛ௃ྔΛಘΔ • ϢʔβʔͱΞΠςϜͷಛ௃ྔΛݩʹɼݩͷάϥϑΛ࠶ߏ੒͢ ΔΠϝʔδͰະ؍ଌͷϨʔςΟϯάΛϥϕϧ෇͖ϦϯΫ༧ଌ 12 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
13. Why use Graph Convolution? • ϊʔυʹؔ͢Δิॿ৘ใʢContent-basedͰ࢖ΘΕ͍ͯΔΑ͏ ͳ৘ใʣΛಛ௃ྔʹؚΊΔ͜ͱ͕Ͱ͖Δ • ϨʔςΟϯάͷؔ܎Ҏ֎ͷ৘ใΛར༻͢Δ͜ͱͰɼCold-start ໰୊ͷղܾʹۙͮ͘ 13 ͜͜ʹิॿ৘ใΛಥͬࠐΊΔ Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
14. Summary(1) • Graph ConvΛ࢖ͬͨϨίϝϯυͷఏҊΛ͍ͯ͠Δ࿦จ • ϨίϝϯυγεςϜ͸ɼConten-basedͱCFͷେ͖͘2ͭ
 ʢࠓճ͸ओʹCFʣ • CFͷதͰ΋ߦྻ෼ղ*͸༗໊Ͱࠓճ΋͜Εʹ͍ۙ
 ʢ* user-item matrixΛuserͷಛ௃ͱitemͷಛ௃ʹ෼ղ͢Δʣ • ࠓճ͸͜ͷߦྻΛGraphͱͯ͠ଊ͑Δ
 Graphߏ଄Λೖྗͱͯ͠userͱitemͷಛ௃ྔΛಘΔ
 ิॿ৘ใΛؚΊͯͦΕͧΕͷಛ௃ྔΛಘΒΕΔͷ͕ར఺ 14 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
15. Agenda • Preliminary for Recommender System • Preliminary for Graph Convolution • Method Overview • Method Details • Experiment • Conclusion and Future works 15 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
16. What is Graph Convolution? • Graph Convolution • ͦͷ໊ͷ௨ΓGraphΛ৞ΈࠐΜͰɼपลʹ৘ใΛ఻೻ͤ͞ Δܭࢉํ๏ • Graph Convolutionͷछྨ • Graph Fourierม׵Λ༻͍ͨղੳతͳConvolution • ௚઀తͳํ๏
 ೖྗͱͳΔಛ௃ྔϕΫτϧͷઢܗ݁߹͔Βͷඇઢܗม׵Ͱ NNͰֶश 16 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
17. Various Graph Structure 17 * https://tech-blog.abeja.asia/entry/2017/04/27/105613 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
18. Convolution for Image 18 * https://github.com/vdumoulin/conv_arithmetic Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
19. • Image • ϊʔυ͕άϦουঢ়ʹฒͼɼ઀ଓؔ܎͕Ұఆ • άϥϑߏ଄ͰΑ͘ද͞ΕΔσʔλʢSNS΍ࠓճʣ • ϊʔυͷ઀ଓؔ܎͕ෆఆܗ Difference between Image and Ordinary Graph 19 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
20. • ۙ๣ϊʔυͷ৘ใΛͨͨΈ͜Ή • Relationͷछྨ͝ͱʹߦ͏
 ʢRelation͕RGBͷΠϝʔδʣ
 ʢRelation: ධՁ஋౳ʣ • ৔߹ʹΑͬͯ͸ࣗ਎ͷϊʔυ΋
 ܭࢉ͢Δ
 ʢࠓճͷ࿦จͰ͸͠ͳ͍ʣ Directly Graph Convolution 20 * Modeling Relational Data with Graph Convolutional Networks Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
21. • ࣜͰද͢ͱ͜Μͳײ͡ Directly Graph Convolution 21 * Modeling Relational Data with Graph Convolutional Networks ׆ੑԽؔ਺ ۙ๣ϊʔυ શͯͰ࣮ߦ Weight ࣍ͷӅΕ૚ ͷಛ௃ྔ શͯͷRelation Ͱ࣮ߦ ਖ਼نԽ ۙ๣ϊʔυ ͷಛ௃ྔ ࣗ਎ͷϊʔυ ͷಛ௃ྔ Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
22. Summary(2) • Graph Fourierม׵Λ࢖ͬͨ΋ͷͱ௚઀తͳํ๏͕͋Δɽࠓճ ͸ޙऀ • ը૾΋Graphߏ଄ͱͯ͠ଊ͑Δ͜ͱ͕Ͱ͖Δ
 ࠓճͷΑ͏ͳGraphͱҧ͏ͷ͸ϊʔυͷ੔ྻ۩߹ • Graph Convolution͸઀ଓ͞ΕͯΔϊʔυ৘ใΛ఻೻ͤ͞Δ • ෆఆܗͳϊʔυؔ܎ͷGraphߏ଄΋ֶशՄೳʹͨ͠ͷ͕Graph Convolutionͷ͍͢͝ͱ͜Ζ 22 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
23. Agenda • Preliminary for Recommender System • Preliminary for Graph Convolution • Method Overview • Method Details • Experiment • Conclusion and Future works 23 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
24. • X: ϊʔυͷಛ௃ྔΛؚΉߦྻ
 ʢϊʔυ਺ʣxʢಛ௃࣍ݩ਺ʣ • A: άϥϑߏ଄ʹ͓͚Δྡ઀ߦྻ
 ʢϊʔυ਺ʣx ʢϊʔυ਺ʣ • Z: ϊʔυͷຒΊࠐΈಛ௃ྔͷߦྻ
 ʢϊʔυ਺ʣxʢຒΊࠐΈͷಛ௃࣍ݩ਺ʣ • A^: ࠶ߏ੒͞Εͨྡ઀ߦྻ Graph Auto-Encoder 24 Encoder Decoder Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
25. Graph for Matrix Completion 25 ←ϊʔυू߹ ←Itemͷϊʔυू߹ ←Userͷϊʔυू߹ ←ϊʔυؒͷΤοδ ←Τοδʹඥͮ͘ϦϨʔγϣϯ
 ʢࠓճ͸ධՁ஋ʣ Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
26. Graph Auto-Encoder (Matrix ver) 26 Encoder Decoder ←X͸ϊʔυ৘ใͷߦྻ ←UserͱItemͷಛ௃ྔߦྻ ↖︎ ධՁ஋͝ͱʹྡ઀ߦྻ͕࡞੒͞ΕΔ ↖︎ ࠶ߏ੒͞Εͨྡ઀ߦྻ Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
27. Graph Auto-Encoder (Matrix ver) 27 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
28. • Graph Convolution Layer => Dense Layer ͷྲྀΕ • Conv Layerͷํ͸࠷ॳʹઆ໌ͨ͠Graph Convͱ΄ͱΜͲಉ͡
 ʢҧ͍͸ࣗ਎ͷϊʔυͷܭࢉΛؚΊͳ͍ʣ • Dense Layer͸ઢܗม׵͔Βͷඇઢܗม׵
 ʢͨͩ͠weight͸userͱitemͰڞ༗͢Δʁʣ Graph Convolutional Encoder 28 Graph Convolution Layer Dense Layer where Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
29. • userͷಛ௃ྔ x Q (ֶशՄೳͳߦྻ) x itemͷಛ௃ྔ
 Λܭࢉ͠ɼ֤ධՁ஋͝ͱʹεΧϥʔ஋Λࢉग़ • u, Q, v͕ύϥϝʔλͷιϑτϚοΫεͰ r (ධՁ஋)ʹؔ͢Δ֬ ཰෼෍ΛԾఆ
 r ͷظ଴஋Λܭࢉ͢Δ Bilinear Decoder 29 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
30. Agenda • Preliminary for Recommender System • Preliminary for Graph Convolution • Method Overview • Method Details • Experiment • Conclusion and Future works 30 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
31. • r ʹؔ͢Δ֬཰෼෍ʹରͯ͠ɼෛͷର਺໬౓ΛLossͱ͢Δ • ݩʑ؍ଌ͞Ε͍ͯΔධՁ஋ͷΈΛLossʹՃࢉ Loss Function 31 ϚεΫߦྻ ؍ଌ஋ͷi, jͰ iteration r = Mij (؍ଌ஋) ͷ࣌͸1ɼͦΕҎ֎0 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
32. • ಛఆͷϊʔυͷґଘ͗͢͠ͳ͍ϩόετͳϊʔυಛ௃ྔΛ֫ ಘ͢ΔͨΊ • Ұఆ֬཰ (p dropout)Ͱɼ͋Δϊʔυͷ͔Β఻೻͞ΕΔ৘ใ͕0 ʹͳΔ • Edge Dropoutʢ͋ΔΤοδ͔Β఻೻͞ΕΔ৘ใ͕0ʹͳΔʣΑ Γ΋Node Dropoutͷํ͕ਫ਼౓͕͍͍Β͍͠ Node Dropout 32 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
33. • େن໛σʔληοτʢRating͕10M݅ͱ͔ʣ͸MemoryʹͷΒ ͳ͍ͷͰɼbatchॲཧΛಋೖ͢Δ • ϝϞϦ࡟ݮ͚ͩͰͳ͘ɼਖ਼ଇԽͱͯ͠΋ػೳ͢Δ • Loss function΁ͷContributionΛجʹɼbatchΛαϯϓϦϯά ͢Δʁ • Full-batch trainingͱਫ਼౓͸΄ͱΜͲมΘΒͳ͍͜ͱ͸࣮ݧͰ ֬ೝͰ͖ͨ • FullͰͰ͖ΔDataset͸ऩଋͷ଎౓Λߟྀͯ͠FullͰ΍ͬͨ Mini-batching 33 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
34. • ࣮૷͢Δ࣌͸ޮ཰Λߟ͑ͯɼߦྻͷ··ܭࢉͰ͖ΔΑ͏ௐ੔
 ʢཁૉ͝ͱʹIteration͍ͯ͠Δͱແݶʹ͕࣌ؒ৯͏ͨΊʣ Vectorized Implementation 34 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
35. • ೖྗͷಛ௃ߦྻʹ௚઀ίϯςϯπ৘ใΛೖྗ͢Δͱɼ͏·͘ ఻೻Ͱ͖ͳ͍໰୊͕ൃੜ͢Δ
 →ίϯςϯπ৘ใ͸ิॿ৘ใͱͯ͠ѻ͏ • ೖྗ͸NodeΛࣝผ͢Δ͚ͩͷone-hot vector • ίϯςϯπ৘ใΛར༻ͨ͠ϊʔυಛ௃ྔΛผʹࢉग़͢Δ Input feature representation 35 Dense Layer ίϯςϯπϕʔε ͷಛ௃ྔ ίϯςϯπ ৘ใͷೖྗ Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
36. • User΍Item͕΋ͭධՁͷ਺͸ɼධՁஈ֊͝ͱʹόϥόϥ
 ʢ͋ΔItem͸ධՁ͕ߴ͍ͨΊ5͕ଟ͘ɼ1͸΄ͱΜͲͳ͍౳ʣ
 →࠷దԽ͞ΕΔස౓͕ҟͳͬͯ͠·͍ɼաֶशͷՄೳੑ͋Γ • WeightΛධՁஈ֊ؒͰڞ༗͢Δ͜ͱͰόϥϯεΛͱΔ Weight Sharing 36 Ordinal Weight Sharingɼ r ҎԼͷධՁஈ֊ͷॏΈΛ શͯར༻͢Δ WS using Basis Weight Matrixɼ جఈߦྻPsͱ܎਺aͷઢܗ݁߹Ͱ Weight matrixΛఆٛ͢Δ Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
37. Agenda • Preliminary for Recommender System • Preliminary for Graph Convolution • Method Overview • Method Details • Experiment • Conclusion and Future works 37 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
38. Dataset 38 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
39. Main Baselines 39 • sRGCNN:
 Graph CNNͰ
 DecoderʹRNNΛར༻͍ͯ͠Δ • CF-NADE:
 Auto-EncoderͷҰछͰɼະ؍ଌϦϯΫʹॳظ஋ͱͯ͠3Λ༩͑ͯ ͍Δɽ·ͨͦͷଞ͍ΖΜͳςΫχοΫΛಋೖ
 ʢͪΌΜͱཧղͰ͖ͳ͔ͬͨʣ
 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
40. MovieLens 100K 40 • େࠩͰউར • sRGCNNΑΓਫ਼౓͕͍͍ͱ͍͏͜ͱ͸ɼRNNΛ࢖ͬͯෳࡶʹ ܭࢉ͢ΔΑΓ΋ಛ௃ྔΛ௚઀ߦྻΛ࠶ߏ੒͢Δํ͕͍͍ Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
41. MovieLens 1M and 10M 41 • ߴ͍ਫ਼౓͕ग़͍ͤͯΔͷͰɼେن໛ͳσʔληοτͰͷֶश͕Մೳ Ͱ͋Δ͜ͱ͕ࣔ͞Εͨ • CF-NADE͸͍ΖΜͳςΫχοΫΛ࢖͓ͬͯΓɼGC-MC΋ͦΕΛ࢖ ͏͜ͱ͸Ͱ͖Δʢͭ·Γ࢖͑͹উͯΔͥతͳʁʣ Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
42. Flixster, Douban, & YahooMusic 42 • શͯಉ͡hyper parameterͰ΍͚ͬͨͲউͬͨ Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
43. Cold-start Analysis 43 • ML-100KͰɼNcਓͷϢʔβʔͷධՁ஋ΛɼNr·ͰݮΒ࣮͠ݧ • ίϯςϯπ৘ใΛಋೖͨ͜͠ͱʹΑΔޮՌΛݕূ͍ͯ͠Δ ಛʹ৚͕݅ ݫ͍͠΄Ͳ վળ͕େ͖͍ Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
44. Agenda • Preliminary for Recommender System • Preliminary for Graph Convolution • Method Overview • Method Details • Experiment • Conclusion and Future works 44 Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
45. Conclusion 45 • Recommender SystemͷͨΊͷGraph Auto-EncoderΛ༻͍ͨ Matrix Completionख๏ɼGC-MCΛఏҊ • σʔληοτʹΑͬͯ͸SoTAΛୡ੒͠ɼstochastic mini batch samplingʹΑΔେن໛σʔληοτ΁ͷద༻΋Մೳʹͨ͠ Preliminary 1 Preliminary 2 Method Details Experiment Conclusion
46. Future Work 46 • Content-basedͷಛ௃ྔͷநग़ํ๏ΛCNN΍RNNʹม͑Δ͜ ͱͰɼը૾΍ࣗવݴޠΛϨίϝϯυʹಋೖͰ͖ΔΑ͏ʹ͢Δ • ScalabilityΛߴΊΔͨΊʹɼsub-sampling local neighborhood ͳͲͷޮ཰తͳۙࣅํ๏Λಋೖ͢Δ • ·ͨɼਖ਼نԽ߲౳ʹAttention MechanismΛಋೖͯ͠Ϟσϧ ͷ͞ΒͳΔ֦ுΛ໨ࢦ͢ Preliminary 1 Preliminary 2 Method Details Experiment Conclusion