KDD 2016勉強会/Images Don’t Lie: Transferring Deep Visual Semantic Features to Large-Scale Multimodal Learning to Rank

KDD 2016勉強会, 2016/10/01

Transcript

1. Images Don’t Lie: Transferring Deep Visual Semantic Features to Large-Scale Multimodal Learning to Rank @tn1031 2016/10/01, KDD 2016ษڧձ
2. ঺հ͢Δ࿦จ Images Don’t Lie: Transferring Deep Visual Semantic Features to Large-Scale Multimodal Learning to Rank Corey Lynch, Kamelia Aryafar, Josh Atterberg http://www.kdd.org/kdd2016/papers/files/adp0804-lynchA.pdf • Etsy(ϋϯυϝΠυͷϚʔέοτϓϨΠε)ͷதͷਓ • ECαΠτͷݕࡧ݁ՌΛϥϯΩϯάֶशʹΑͬͯ࠷దԽ͢Δ • ঎඼ৄࡉͷϚϧνϞʔμϧͳಛ௃Λֶशʹ༻͍Δ 2
3. ࣗݾ঺հ • தଜ ୓ຏ / @tn1031 • σʔλαΠΤϯςΟετ • VASILY, Inc. • Machine LearningΛ׆༻ͨ͠αʔϏε։ൃ 3 • ྨࣅը૾ݕࡧ • Convolutional Auto-Encoder, Approximate Nearest Neighbor • Ϩίϝϯυ • Collaborative Filtering, Matrix Factorization • ͪΐͬͱมΘͬͨΞΠςϜݕࡧ • Conditional VAE-GAN Machine LearningΛ׆༻ͨ͠αʔϏεͷྫ
4. Images Don’t Lie: Transferring Deep Visual Semantic Features to Large-Scale Multimodal Learning to Rank
5. എܠ ΞΠςϜͷdescription͸ϊΠζΛؚΉͨΊݕࡧͷ࣭͕௿Լ͢Δ 5 • λΠτϧ΍descriptionʹ ”wedding dress” ΛؚΉɹɹ ΞΠςϜ͕ώοτ͢Δ • ࣮ࡍ͸ wedding dress ͱ͸ؔ܎ͳ͍ΞΠςϜ͕ɹ େྔʹؚ·ΕΔ • ϊΠζͷݪҼ • ग़඼ऀ͕ಠࣗʹ෇༩͢Δ • τϥϑΟοΫΛՔ͙ͨΊ “wedding dress” ͷݕࡧ݁Ռ ը૾ΛݟΕ͹ wedding dress Ͱͳ͍͜ͱ͸໌Β͔
6. ఏҊख๏ ը૾ͱςΩετ྆ํΛར༻ͯ͠ݕࡧਫ਼౓Λվળ͢Δ 6 Multimodal Listing Embedding • ը૾ͱςΩετͷϕΫτϧԽ • ը૾͸CNNͰϕΫτϧԽ • ΞΠςϜʹ෇ਵ͢ΔςΩετ৘ใΛBoWͰϕΫτϧԽ • ը૾༝དྷͷϕΫτϧͱςΩετ༝དྷͷϕΫτϧΛconcat Learning To Rank • 2஋൑ผ໰୊ͱͯ͠ఆࣜԽ • ΫΤϦ͝ͱʹRankingSVMΛద༻ͯ͠ϥϯΩϯάΛٻΊΔ
7. ಛ௃நग़ɿMultimodal Listing Embedding ը૾ͱςΩετΛҟͳΔख๏ͰϕΫτϧԽͯ͠concat͢Δ 7 ը૾ • ImageNetͰֶशͤͨ͞VGG19 • Ϟσϧ͸શΫΤϦڞ௨ • fine-tuning͠ͳ͍ • ࠷ऴखલͷfc૚ͷग़ྗ4096࣍ݩ Λಛ௃ྔͱ͢Δ ςΩετ • ز͔ͭͷཁૉͷBoW • ΞΠςϜID,γϣοϓID,λΠτϧ, λά • unigram,bigram
8. ֶशɿLearning To Rank ΫΤϦʹର͢Δؔ࿈౓Λࢉग़͢Δؔ਺Λranking໰୊ͱֶͯ͠श͢Δ 8 pairwise preference approach ΫΤϦ q ɼؔ܎͋ΔΞΠςϜ d+ ɼؔ܎ͳ͍ΞΠςϜ d- ʹ͍ͭͯɼɹ ҎԼͷΑ͏ͳؔ਺ fq Λֶश͢Δ fq(d+) > fq(d ) min w m X i=1 max(1 yi h xi, w i , 0) + 1 || w ||1 + 2 || w ||2 RankingSVMͱͯ͠ɼҎԼͷ࠷খԽΛղ͘ ( xi, yi) = ( d + d , +1 d d + , 1 a well-ordered pair a poorly ordered pair
9. σʔλऩू ΫΤϦʹର͢Δؔ࿈ੑͷ༗ແ͸Ϣʔβʔͷߦಈ͔Βऔಘ͢Δ 9 pairwiseͷऔಘ • ݕࡧ݁Ռʹ͍ͭͯpositiveͳ ൓Ԡ͕ಘΒΕͨΞΠςϜͱɹ ಘΒΕͳ͔ͬͨΞΠςϜɹɹ ͻͱͭΛબ୒͢Δ ΞΠςϜ͔Βಛ௃நग़ • ֤ΞΠςϜͷಛ௃ྔΛɹɹɹ ͦΕͧΕநग़͢Δ ਖ਼ྫ/ෛྫͷऔಘ • pos - neg ͕ਖ਼ྫɺٯ͕ෛྫ
10. ධՁࢦඪ ݕࡧ݁Ռͷྑ͞ΛnDCGͰධՁ͢Δ 10 Normalized Discounted Cumulative Gain (nDCG) • ϥϯΩϯάʹର͢ΔධՁࢦඪ • [0, 1]Ͱେ͖͍΄Ͳྑ͍ • ্ҐͷΞΠςϜ͕ΑΓධՁ͞ΕΔΑ͏ͳ܏͕͍͍ࣼͭͯΔ 1Ґ͔Β p Ґ·ͰͷϥϯΩϯάΛߟ͑Δ nPCGp = DCGp idealDCGp DCGp = p X i=1 2 reli 1 log2( i + 1)
11. σʔληοτ Etsyͷݕࡧϩά͔Βऔಘ͢Δ 11 σʔλऔಘ • औಘظؒɿ2िؒ • σʔλ਺ɿ8.82 million training preference pairs ɹɹɹɹɹɹ1.9 million validation sessions ɹɹɹɹɹɹ1.9 million test sessions • ΫΤϦ਺ɿ1394 total queries औಘ࣌ͷ޻෉ • FairPairs method • όΠΞεͷӨڹΛܰݮ͢Δख๏
12. ݁Ռ 12 ςΩετͷΈͷ৔߹ͱൺֱͯ͠ฏۉ1.7ˋͷਫ਼౓վળʹ੒ޭ nDCGͷ஋Ͱൺֱ • શ1394ΫΤϦͷ͏ͪɼ 51.4% ʹ͍ͭͯnDCGͷվળ͕ݟΒΕͨ • ฏۉ͢Δͱ • text onlyɿbaseline • image onlyɿ2.2%ѱԽ • multimodalɿ1.7%վળ
13. ݁Ռ 13
14. ݁Ռ 14 ςΩετʹϊΠζؚ͕·ΕΔΞΠςϜʹ͍ͭͯվળ͕ݟΒΕΔ ࠨɿtext model ӈɿmultimodal model
15. ·ͱΊ ը૾ͱςΩετ྆ํΛར༻ͯ͠ݕࡧਫ਼౓Λվળ͢Δ 15 ओு • ը૾ಛ௃͸ݕࡧ݁Ռͷ࠷దԽʹ༗ޮ • textͰදݱ͖͠Ε͍ͯͳ͍৘ใΛѻ͏͜ͱ͕Ͱ͖Δ • ࣮ࡍʹݕࡧ݁Ռ͕վળ͞Εͨ ݸਓతͳٙ໰ͳͲ • textͷಛ௃நग़͕BoW • ෼ࢄදݱʗjoint model͸Ͳ͏ͳΔ͔ؾʹͳΔ • ੒ޭใुܕͷϏδωεϞσϧͷͱ͖͸஫ҙ͕ඞཁ • ݕࡧ݁Ռͷ࠷దԽ͕ച্ͷ࠷దԽʹͳΔͱ͸ݶΒͳ͍
16. EOF.