Share
Embed
検索キーワードをPythonのScikit learnでクラスタリングした話 〜機械学習を使って自然検索に強いサイトを作る〜
自然検索のサジェストキーワードを、PythonのScikit learnでクラスタリングすることで、検索キーワードに強いサイト作りをする
soheiyagi
July 14, 2019
Transcript
38 slides
-
ࣗݾհ גࣜձࣾιදऔక TPVDPKQ ɾίϯςϯπϚʔέςΟϯά ɾ8FCࠂӡ༻ ɾϚʔέςΟϯάπʔϧ࡞ ຊொΦʔϓϯιʔεϥϘӡӦ ɾΤϯδχΞͷίϛϡχςΟεϖʔε େࡕ1ZUIPOͷձɹΦʔΨφΠβʔ
தখاۀிϛϥαϙઐՈݣɹొઐՈ 8FCࠂਓࡐϏδωεͷӦۀ͔ΒࣄΛ͡Ίɺ 8FCӡ༻ࠂӡ༻ΛΓͭͭɺϓϩάϥϜΛॻ͘ਓ ίʔυ͕ॻ͚ΔϚʔέολʔ !TPIFJZBHJ TPIFJZBHJ 4PIFJ:BHJʗീฏ -
ࣗݾհ גࣜձࣾιදऔక TPVDPKQ ɾίϯςϯπϚʔέςΟϯά ɾ8FCࠂӡ༻ ɾϚʔέςΟϯάπʔϧ࡞ ຊொΦʔϓϯιʔεϥϘӡӦ ɾΤϯδχΞͷίϛϡχςΟεϖʔε େࡕ1ZUIPOͷձɹΦʔΨφΠβʔ
தখاۀிϛϥαϙઐՈݣɹొઐՈ 8FCࠂਓࡐϏδωεͷӦۀ͔ΒࣄΛ͡Ίɺ 8FCӡ༻ࠂӡ༻ΛΓͭͭɺϓϩάϥϜΛॻ͘ਓ ίʔυ͕ॻ͚ΔϚʔέολʔ !TPIFJZBHJ TPIFJZBHJ 4PIFJ:BHJʗീฏ ࠷ۙͷझຯ ےτϨͰ͢ɻ -
αδΣετϫʔυͷऔಘ Πϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯࣄྫຊ Πϊϕʔγϣϯࣄྫ࠷ۙ Πϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯࣄྫւ֎ ΠϊϕʔγϣϯࣄྫJQIPOF Πϊϕʔγϣϯࣄྫ࠷৽ Πϊϕʔγϣϯࣄྫۙ ΠϊϕʔγϣϯࣄྫΞϝϦΧ
ΠϊϕʔγϣϯࣄྫΥʔΫϚϯ ΤίγεςϜΠϊϕʔγϣϯࣄྫ ӦۀΠϊϕʔγϣϯࣄྫ ΠϊϕʔγϣϯΦϑΟεࣄྫ ΦʔϓϯΠϊϕʔγϣϯࣄྫ ΦʔϓϯΠϊϕʔγϣϯࣄྫຊ ΦʔϓϯσʔλΠϊϕʔγϣϯࣄྫ େࡕΨεΠϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯձࣾࣄྫ Πϊϕʔγϣϯڥࣄྫ ՁΠϊϕʔγϣϯࣄྫ ֶੜΠϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯࣄྫاۀ Πϊϕʔγϣϯۚ༥ࣄྫ େاۀΠϊϕʔγϣϯࣄྫ ٕज़Πϊϕʔγϣϯࣄྫ ۀքΠϊϕʔγϣϯࣄྫ ΠϊϕʔγϣϯࣄྫΈ߹Θͤ ݚڀ։ൃΠϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯܦࡁࣄྫ ΠϊϕʔγϣϯܦӦࣄྫ খചۀΠϊϕʔγϣϯࣄྫ খചΠϊϕʔγϣϯࣄྫ ࢠڙΠϊϕʔγϣϯࣄྫ খചۀΠϊϕʔγϣϯࣄྫ ΠϊϕʔγϣϯࣄྫαʔϏε ࢈ֶ࿈ܞΠϊϕʔγϣϯࣄྫ αʔϏεۀΠϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯࣄྫू Πϊϕʔγϣϯࣄྫࣦഊ Πϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯ৽݁߹ࣄྫ ΤίγεςϜΠϊϕʔγϣϯࣄྫ ࣾձ՝Πϊϕʔγϣϯࣄྫ ࣾΠϊϕʔγϣϯࣄྫ ΠϊϕʔγϣϯδϨϯϚࣄྫ Πϊϕʔγϣϯ࣋ଓతࣄྫ Πϊϕʔγϣϯࣄྫੈք Πϊϕʔγϣϯࣄྫ ۀΠϊϕʔγϣϯࣄྫ ଟ༷ੑΠϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯ৫ࣄྫ Πϊϕʔγϣϯग़ࣄྫ ιʔγϟϧΠϊϕʔγϣϯࣄྫຊ ൃΠϊϕʔγϣϯࣄྫ ଟ༷ੑΠϊϕʔγϣϯࣄྫ େاۀΠϊϕʔγϣϯࣄྫ μΠόʔγςΟΠϊϕʔγϣϯࣄྫ தখاۀΠϊϕʔγϣϯࣄྫ ΠϊϕʔγϣϯνʔϜࣄྫ Πϊϕʔγϣϯࣝࣄྫ ҬΠϊϕʔγϣϯࣄྫ ΠϊϕʔγϣϯࣄྫσβΠϯࢥߟ σβΠϯΠϊϕʔγϣϯࣄྫ σδλϧΠϊϕʔγϣϯࣄྫ ഁյతΠϊϕʔγϣϯࣄྫ ඇ࿈ଓΠϊϕʔγϣϯࣄྫ ཱΠϊϕʔγϣϯࣄྫ ϏδωεϞσϧΠϊϕʔγϣϯࣄྫ ϏδωεΠϊϕʔγϣϯࣄྫ ϏοάσʔλΠϊϕʔγϣϯࣄྫ ࢜ϑΠϧϜΠϊϕʔγϣϯࣄྫ ࢜௨ϑΟʔϧυΠϊϕʔγϣϯࣄྫ ࢜௨Πϊϕʔγϣϯࣄྫ ϓϩηεΠϊϕʔγϣϯࣄྫ ϓϩμΫτΠϊϕʔγϣϯࣄྫ ϔϧεέΞΠϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯࣄྫຊ ΠϊϕʔγϣϯࣄྫϚΠΫϩιϑτ ϚʔέςΟϯάΠϊϕʔγϣϯࣄྫ ϢʔβʔΠϊϕʔγϣϯࣄྫ ϦόʔεΠϊϕʔγϣϯࣄྫ ΞʔΩςΫνϟϧΠϊϕʔγϣϯࣄྫ ࢈ֶ࿈ܞΠϊϕʔγϣϯࣄྫ QHΠϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯࣄྫ NΠϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯͭͷػձࣄྫ λʔήοτΩʔϫʔυʮΠϊϕʔγϣϯࣄྫʯ݅ -
ҙຯͷ͋Δմʹάϧʔϐϯά ɾཱΠϊϕʔγϣϯࣄྫ ɾ࢜ϑΠϧϜΠϊϕʔγϣϯࣄྫ ɾେࡕΨεΠϊϕʔγϣϯࣄྫ ɾμΠόʔγςΟΠϊϕʔγϣϯࣄྫ ɾଟ༷ੑΠϊϕʔγϣϯࣄྫ ɾଟ༷ੑΠϊϕʔγϣϯࣄྫ ɾΠϊϕʔγϣϯδϨϯϚࣄྫ ɾΠϊϕʔγϣϯ࣋ଓతࣄྫ ɾഁյతΠϊϕʔγϣϯࣄྫ
اۀࣄྫ ଟ༷ੑ δϨϯϚ ࢹͰ͍ͬͯͨͷΛɺ ػցֶशͰάϧʔϓ͚Λߦͬͨ -
࣮ࡍͷਫ਼ ΠϊϕʔγϣϯܦӦࣄྫ Πϊϕʔγϣϯܦࡁࣄྫ Πϊϕʔγϣϯࣄྫ ΠϊϕʔγϣϯࣄྫΥʔΫϚϯ ϓϩηεΠϊϕʔγϣϯࣄྫ ඇ࿈ଓΠϊϕʔγϣϯࣄྫ
Πϊϕʔγϣϯձࣾࣄྫ ΠϊϕʔγϣϯࣄྫΞϝϦΧ ΠϊϕʔγϣϯࣄྫαʔϏε Πϊϕʔγϣϯࣄྫاۀ Πϊϕʔγϣϯࣄྫू Πϊϕʔγϣϯࣄྫۙ Πϊϕʔγϣϯࣄྫੈք Πϊϕʔγϣϯࣄྫ ϓϩμΫτΠϊϕʔγϣϯࣄྫ ٕज़Πϊϕʔγϣϯࣄྫ ࢠڙΠϊϕʔγϣϯࣄྫ μΠόʔγςΟΠϊϕʔγϣϯࣄྫ ଟ༷ੑΠϊϕʔγϣϯࣄྫ ଟ༷ੑΠϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯࣄྫࣦഊ Πϊϕʔγϣϯ৫ࣄྫ ֶੜΠϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯࣄྫւ֎ ΦʔϓϯΠϊϕʔγϣϯࣄྫ ΦʔϓϯΠϊϕʔγϣϯࣄྫຊ ΠϊϕʔγϣϯࣄྫσβΠϯࢥߟ σβΠϯΠϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯࣄྫ࠷ۙ Πϊϕʔγϣϯࣄྫ࠷৽ Πϊϕʔγϣϯࣄྫຊ Πϊϕʔγϣϯࣄྫ αʔϏεۀΠϊϕʔγϣϯࣄྫ ϏδωεΠϊϕʔγϣϯࣄྫ ۀքΠϊϕʔγϣϯࣄྫ ۀΠϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯڥࣄྫ Πϊϕʔγϣϯۚ༥ࣄྫ ιʔγϟϧΠϊϕʔγϣϯࣄྫຊ ϏδωεϞσϧΠϊϕʔγϣϯࣄྫ ΠϊϕʔγϣϯδϨϯϚࣄྫ Πϊϕʔγϣϯ࣋ଓతࣄྫ ഁյతΠϊϕʔγϣϯࣄྫ খചΠϊϕʔγϣϯࣄྫ খചۀΠϊϕʔγϣϯࣄྫ খചۀΠϊϕʔγϣϯࣄྫ ࢜௨Πϊϕʔγϣϯࣄྫ ࢜௨ϑΟʔϧυΠϊϕʔγϣϯࣄྫ NΠϊϕʔγϣϯࣄྫ QHΠϊϕʔγϣϯࣄྫ ΞʔΩςΫνϟϧΠϊϕʔγϣϯࣄྫ ΠϊϕʔγϣϯΦϑΟεࣄྫ ΠϊϕʔγϣϯνʔϜࣄྫ ΠϊϕʔγϣϯࣄྫϚΠΫϩιϑτ Πϊϕʔγϣϯࣄྫຊ Πϊϕʔγϣϯग़ࣄྫ Πϊϕʔγϣϯࣝࣄྫ ΦʔϓϯσʔλΠϊϕʔγϣϯࣄྫ σδλϧΠϊϕʔγϣϯࣄྫ ϏοάσʔλΠϊϕʔγϣϯࣄྫ ϔϧεέΞΠϊϕʔγϣϯࣄྫ ϢʔβʔΠϊϕʔγϣϯࣄྫ ϦόʔεΠϊϕʔγϣϯࣄྫ ӦۀΠϊϕʔγϣϯࣄྫ ݚڀ։ൃΠϊϕʔγϣϯࣄྫ ࢈ֶ࿈ܞΠϊϕʔγϣϯࣄྫ ࢈ֶ࿈ܞΠϊϕʔγϣϯࣄྫ ࣾձ՝Πϊϕʔγϣϯࣄྫ ࣾΠϊϕʔγϣϯࣄྫ ൃΠϊϕʔγϣϯࣄྫ େࡕΨεΠϊϕʔγϣϯࣄྫ ҬΠϊϕʔγϣϯࣄྫ ཱΠϊϕʔγϣϯࣄྫ ࢜ϑΠϧϜΠϊϕʔγϣϯࣄྫ -
࣮ࡍͷਫ਼ ΤίγεςϜΠϊϕʔγϣϯࣄྫ ΤίγεςϜΠϊϕʔγϣϯࣄྫ େاۀΠϊϕʔγϣϯࣄྫ େاۀΠϊϕʔγϣϯࣄྫ தখاۀΠϊϕʔγϣϯࣄྫ ΠϊϕʔγϣϯࣄྫJQIPOF
ΠϊϕʔγϣϯࣄྫΈ߹Θͤ Πϊϕʔγϣϯ৽݁߹ࣄྫ ϚʔέςΟϯάΠϊϕʔγϣϯࣄྫ ՁΠϊϕʔγϣϯࣄྫ Πϊϕʔγϣϯͭͷػձࣄྫ ·ͩ·ͩਫ਼্͛Δඞཁ͋Δ͕ɺ ࢹͰҰ͔ΒΔΑΓஅવ࣌ؒॖ͕࣮ݱ -
ຊொΦʔϓϯιʔεϥϘ8FCαΠτͰެ։࣮ݧ Ωʔϫʔυ ॱҐ ݕࡧϘϦϡʔϜ ϓϩάϥϛϯάษڧձॳ৺ऀ ϓϩάϥϛϯάษڧձॳ৺ऀ
JUษڧձॳ৺ऀ JUॳ৺ऀηϛφʔ VEFNZQZUIPO͓͢͢Ί ίϯύεษڧձ JUษڧձ JUษڧձ DPNQBTTษڧձ ษڧձαΠτ ΤϯδχΞษڧձ ษڧձJU EPUTษڧձ ษڧձɹJU JUษڧձΧϨϯμʔ ΤϯδχΞษڧձ JUษڧॳ৺ऀ ໊ݹQZUIPO QZUIPOॳ৺ऀษڧձ Ωʔϫʔυ ॱҐ ݕࡧϘϦϡʔϜ JUษڧձେࡕ େࡕJUษڧձ ϓϩάϥϛϯάษڧձ େࡕJUษڧձ JUษڧձେࡕ QZUIPOษڧձ౦ژ େࡕษڧձJU ԬJUษڧձ ϓϩάϥϛϯάษڧձ SVCZؔ େࡕJUษڧձ ໊ݹษڧձ QZUIPOηϛφʔ QZUIPOVEFNZ VEFNZQZUIPO JUษڧ ૂͬͨΩʔϫʔυͰͷ্Ґදࣔͱ$7Λ֫ಘ ˞݄࣌ɹ"ISFGTௐ -
˞݄࣌ɹ"ISFGTௐ ຊொΦʔϓϯιʔεϥϘ8FCαΠτͰެ։࣮ݧ Ωʔϫʔυ ॱҐ ݕࡧϘϦϡʔϜ ϓϩάϥϛϯάษڧձॳ৺ऀ ϓϩάϥϛϯάษڧձॳ৺ऀ
JUษڧձॳ৺ऀ JUॳ৺ऀηϛφʔ VEFNZQZUIPO͓͢͢Ί ίϯύεษڧձ JUษڧձ JUษڧձ DPNQBTTษڧձ ษڧձαΠτ ΤϯδχΞษڧձ ษڧձJU EPUTษڧձ ษڧձɹJU JUษڧձΧϨϯμʔ ΤϯδχΞษڧձ JUษڧॳ৺ऀ ໊ݹQZUIPO QZUIPOॳ৺ऀษڧձ Ωʔϫʔυ ॱҐ ݕࡧϘϦϡʔϜ JUษڧձେࡕ େࡕJUษڧձ ϓϩάϥϛϯάษڧձ େࡕJUษڧձ JUษڧձେࡕ QZUIPOษڧձ౦ژ େࡕษڧձJU ԬJUษڧձ ϓϩάϥϛϯάษڧձ SVCZؔ େࡕJUษڧձ ໊ݹษڧձ QZUIPOηϛφʔ QZUIPOVEFNZ VEFNZQZUIPO JUษڧ ࠓճͷςʔϚʹؔͳ͍Ͱ͕͢ɺ ݕࡧΩʔϫʔυͷϘϦϡʔϜͱ $73ͷ૬ؔແ͍ͨΊɺ ࣮ࡍʹ$7ʹ͍ۙΩʔϫʔυͰ ্ҐදࣔͰ͖Δ͜ͱ͕ॏཁ ˞$73ɹίϯόʔδϣϯʢʣ ˞$7ɹίϯόʔδϣϯ -
ίʔυͲΜͳײ͡ # -*- coding: utf-8 -*- import pandas as pd
from sklearn.cluster import KMeans ίʔυൈਮ pred = KMeans( n_clusters=int(query_len/5), init='k-means++' ).fit_predict(cust_array) -
# -*- coding: utf-8 -*- import pandas as pd from
sklearn.cluster import KMeans ίʔυൈਮ pred = KMeans( n_clusters=int(query_len/5), init='k-means++' ).fit_predict(cust_array) ,.FBOTΛར༻ ίʔυͲΜͳײ͡ -
# -*- coding: utf-8 -*- import pandas as pd from
sklearn.cluster import KMeans ίʔυൈਮ pred = KMeans( n_clusters=int(query_len/5), init='k-means++' ).fit_predict(cust_array) Ϋϥελ ίʔυͲΜͳײ͡ -
# -*- coding: utf-8 -*- import pandas as pd from
sklearn.cluster import KMeans ίʔυൈਮ pred = KMeans( n_clusters=int(query_len/5), init='k-means++' ).fit_predict(cust_array) αδΣετΩʔϫʔυΛ ͰׂͬͨࣈΛઃఆ ˞ҙͷࣈ ίʔυͲΜͳײ͡ -
# -*- coding: utf-8 -*- import pandas as pd from
sklearn.cluster import KMeans ίʔυൈਮ pred = KMeans( n_clusters=int(query_len/5), init='k-means++' ).fit_predict(cust_array) ॳظԽͷઃఆ ίʔυͲΜͳײ͡ -
# -*- coding: utf-8 -*- import pandas as pd from
sklearn.cluster import KMeans ίʔυൈਮ pred = KMeans( n_clusters=int(query_len/5), init='k-means++' ).fit_predict(cust_array) ֤σʔλʹର͢Δ Ϋϥελ൪߸Λฦ͢ ίʔυͲΜͳײ͡ -
# -*- coding: utf-8 -*- import pandas as pd from
sklearn.cluster import KMeans ίʔυൈਮ pred = KMeans( n_clusters=int(query_len/5), init='k-means++' ).fit_predict(cust_array) <<> <> <> <>> ֤ΩʔϫʔυΛϕΫτϧԽͯ͠pU@QSFEJDUʹ͢ Ωʔϫʔυ< > Ωʔϫʔυ< > ίʔυͲΜͳײ͡ -
# -*- coding: utf-8 -*- import pandas as pd from
sklearn.cluster import KMeans ίʔυൈਮ pred = KMeans( n_clusters=int(query_len/5), init='k-means++' ).fit_predict(cust_array) < > ֤ΩʔϫʔυͷΫϥελ൪߸͕ฦͬͯ͘Δ Ωʔϫʔυɿ Ωʔϫʔυɿ̍̒ ίʔυͲΜͳײ͡