Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
motemote-data-science-2
kur0cky
August 01, 2020
Technology
2
460
motemote-data-science-2
ネタですが,Rからelasticsearchを使ったり,elasticsearchの便利な機能を一部紹介しています笑
kur0cky
August 01, 2020
Tweet
Share
More Decks by kur0cky
See All by kur0cky
The bootstrapping method for everyone
kur0cky
3
760
音楽理論と方向統計学の初歩/introduction of circular statistics and musicology
kur0cky
5
1.2k
NLP introduction in R 1
kur0cky
0
59
tidyverse tutorial 2
kur0cky
1
26
tidyverse tutorial 1
kur0cky
1
35
rating_introduction
kur0cky
1
760
motemote data science 1
kur0cky
1
450
Other Decks in Technology
See All in Technology
三越伊勢丹の接客DXを支える「DevOps基盤」とは
imdigitallab
0
240
Understanding Python attributes
reuven
0
550
動画配信技術について
yaminoma
0
180
Salesforce女子部-権限についてまとめてみたその1
sfggjp
0
160
Adopting Kafka for the #1 job site in the world
ymyzk
1
230
jaws-ug-asa-datasync-20220510
hiashisan
0
440
Microsoft Power Automate で 始めるRPAと自動化
taikiyoshida
0
1.7k
Deeplearning from almost scratch
hn410
0
570
エンジニアインターンの採用〜実際の開発への関与について for EM meetup#10
dmiyamoto
1
250
ここらでGPSマルチユニットが如何に使いやすいか本気で説明するから聞きなさい
mitsuzono
0
190
組織でPower Virtual Agentsを導入するために知っておきたいこと
miyakemito
0
1.3k
Motto Go Forward スライドトップと Goを支える文化とコミュニティ してご利用ください 〜なぜ我々はコミュニティにコントリ ビュートするのか〜
luccafort
0
130
Featured
See All Featured
What's in a price? How to price your products and services
michaelherold
229
9.3k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
7
1k
Typedesign – Prime Four
hannesfritz
33
1.3k
Fantastic passwords and where to find them - at NoRuKo
philnash
25
1.4k
Designing for humans not robots
tammielis
241
23k
Building Flexible Design Systems
yeseniaperezcruz
310
33k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
314
19k
How To Stay Up To Date on Web Technology
chriscoyier
780
250k
Navigating Team Friction
lara
175
11k
The Most Common Mistakes in Cover Letters
jrick
PRO
4
24k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_i
21
14k
Art, The Web, and Tiny UX
lynnandtonic
280
17k
Transcript
σʔλͰʮϞςʯΛIBDLͤΑ BU5PLZP3 LVSDLZ dσΟφʔฤd
ࣗݾհ w 5XJUUFSɿ!LVSDLZ@Z w ॴଐɿ໊ཧͷIPHFIPHFʢϲ݄ʣ w झຯɿԻָɼөըɼ͓ञɼλΠϐϯάɼFUD w ΈɿϞς͍ͨ 2
લճdөըσʔτฤd w POFUPPOFͳөըσʔτମݧΛఏڙ͢ΔͨΊʹɼࣗવͳձͷத͔ΒϨ ϏϡʔΛਪఆ͢Δ͜ͱΛతͱͨ͠ɽ w ͦͷͨΊʹɼөըυϝΠϯઐ༻ͷۃੑࣙॻΛ࡞͠ɼϨίϝϯυʹ׆༻ 3 https://speakerdeck.com/kur0cky/motemote-data-science-1
өըΛݟͯऴΘΓͰྑ͔ͬͨͷͩΖ͏͔
࣮͕ޠ͍ͬͯΔ
Կ͕͍͚ͳ͔ͬͨͷ͔
ࠗ׆αΠτʹΑΔͱ өըͷ༨ӆΛָ͠ΜͩΓɺөըͱ͍͏ڞ௨ͷͷձ͔Β૬खͷঁੑʹ͍ͭͯ͞Β ʹΕͨΓͱɺөըͷޙઈͷίϛϡχέʔγϣϯͷػձͱͳΓ·͢ɻ͜ͷνϟϯε Λ͍͔ͨ͢Ίʹɺөըͷޙʹ͝൧͓ͳͲͷ༧ఆΛೖΕ·͠ΐ͏ɻ ภݟʹ·ΈΕ͍ͯΔ͕ɼ͍ͬͨΜ͜ΕΛ৴͡Δɽ
POFUPPOFͳөը͚ͩͰμϝ ͦͷޙͷձ͕ॏཁ
ϩʔϧϓϨΠϯά 9 ͓͠Ζ͔ͬͨͶʂʂ దʹೖͬͪΌ͓͏͔ ͜Εതଧɽऑऀʹద͕Θ͔ΒΜ ͓͍͍͠ΠλϦΞϯ༧͓͍ͯͨ͠Μͩʂʂ ༻ҙप౸͗ͯ͢ॏ͍͠ɼ͓ෲ͕ݮͬͯͳ͍͔͠Εͳ͍ ͳʹ͔৯ͳ͕ΒΖ͏ʂ গ͠า͍ͯɼྑ͛͞ͳͱ͜Ζ୳ͦ͞͏͔ ૬खͷੑ֨ॴʹେ͖͘ґଘɽา͖͗͢/(ɽϦεΩʔ
ͦͷͰαΫͬͱݕࡧͰ͖Δ ΞϓϦ͕ॏཁ
3ͱFMBTUJDTFBSDI BU5PLZP3 LVSDLZ
w શจݕࡧγεςϜ w ಛɿ w ߴɽࢄܕͰεέʔϥϒϧɽΦʔϓϯιʔεɽ3&45"1*ɽ +40/ʹΑΔॊೈͳσʔλߏɽείΞϦϯάͷΧελϚΠζɽ 12 ՄࢹԽɾੳ ݕࡧ
σʔλऩू
Πϯετʔϧɾىಈ w .BDͷ߹ brew tap elastic/tap brew install elastic/tap/elasticsearch-full brew
install elastic/tap/kibana-full brew install elastic/tap/logstash-full brew install elastic/tap/metricbeat-full elasticsearch & kibana & w IUUQMPDBMIPTUͰใ͕ฦͬͯ͘Εىಈޭ w ,JCBOB IUUQMPDBMIPTU ͷ$POTPMFΛ͏ͱ৭ʑࢼ͍͢͠ 13 ଞͷ04ɿIUUQTXXXFMBTUJDDPHVJEFFOFMBTUJDTUBDLDVSSFOUJOTUBMMJOHFMBTUJDTUBDLIUNM
جຊ༻ޠ w ΠϯσοΫε w FMBTUJDTFBSDI͕ݕࡧɾղੳͷରͱ͢Δσʔλͷอଘઌ w υΩϡϝϯτλΠϓ w ΠϯσοΫεͷάϧʔϓ w
υΩϡϝϯτ w FMBTUJDTFBSDIʹอଘ͞Εͨσʔλ w ϑΟʔϧυ w υΩϡϝϯτʹؚ·ΕΔଐੑ 14 σʔλϕʔε ςʔϒϧ Ϩίʔυ ΧϥϜ 3%#Ͱ͍͏ͱɹɹ
؆୯ͳ͍ํ w +40/ͱ3&45"1*ͰઃఆɾΠϯσΩγϯάʢೖʣɾݕࡧͯ͢Λߦ͏ 15 • ݕࡧ GET index_name/_search { "query"
: { "match" : { "comment" : "σʔτ" } } } • ΠϯσΩγϯά PUT index_name/ { "name" : "ϥʔϝϯೋ", "genre" : "όʔɾμΠχϯά" } • ઃఆ PUT index_name/ { "settings" : { hogehoge }, "mappings" : { fugafuga } } ܗଶૉղੳશ֯ͷ౷ҰͳͲ ༷ʑͳઃఆΛهड़ ೖ͢Δσʔλ͕ͲͷΑ͏ͳ ϑΟʔϧυΛͪ͏Δͷ͔ɼͦͷܕهड़
3͔ΒFMBTUJDTFBSDIΛୟ͘ w FMBTUJDύοέʔδ͔Βୟ͘ɽ w σʔλϑϨʔϜΛΠϯσΩγϯάͰ͖Δɽ w શͯΛ3Ͱ͍݁ͤͨ͞ʂʂ w ΫΤϦɼϦετͰॻ͍ͯKTPOMJUFͰ+40/ʹ w
جຊͷૢ࡞ 1. conn <- connect(host="127.0.0.1", port=9200) 2. docs_bulk(conn, df, index) 3. Search(conn, index, body = <query>) 16
࣮ફ
త w өըσʔτײઓͰ͢Δ w ϩʔϧϓϨΠϯάͷ݁Ռɼ͍͔ͭ͘ͷཁ݅Λຬͨ͢ඞཁ͕͋Δ w ͦͷͷঢ়گʹ߹ΘͤͯαΫοͱܾΊΔ͜ͱ͕ॏཁ w جຊతʹ͋·Γา͔ͤͳ͍ w
Ͱ͖Δ͚࣭ͩͷྑ͍ళʹೖΔ 18
ͬͨ͜ͱ w ༻σʔλ w ौ୩ۙลͷϨετϥϯ݅ w ళ໊ɼδϟϯϧɼਓؾͷΫνίϛɼͦͷଞళฮใ 19 HFPDPEJOH"1* ݕࡧ6*
MFBqFUʹΑΔਤ %5ʹΑΔ៉ྷͳද ݱࡏͷऔಘ ݕࡧ εΫϨΠϐϯά
ཁ݅ͷୡ w LVSPNPKJQMVHJOʹΑΔલॲཧ w τʔΫϯԽɼશ֯౷ҰɼεςϛϯάɼFUD w σϑΥϧτͷείΞʢ#.ʣΛ͏ w ۙ͞ʹԠͨ͡ݮਰؔ͏ w
৯ϩάείΞ͏ 20 w ຊޠͷݕࡧ w ΫΤϦͱͷϚον w ͋·Γา͔ͤͳ͍ w ͍͍ళʹೖΔ https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-kuromoji-analyzer.html
ڑʹԠͨ͡ݮਰؔ w ࠓճɼҢɾܦͦΕͧΕʹ ΨγΞϯΛ༻ w ଞʹɼࢦؔઢܗ͕͋Δ 21 https://www.elastic.co/guide/en/elasticsearch/reference/current/query- dsl-function-score-query.html
{ "query": { "function_score": { "query": { ී௨ͷΫΤϦ͜͜ʹॻ͘ }, "functions":
[ { "gauss": { "latitude": { "origin": [35.6591], "scale": [0.003] } } }, { "gauss": { "longitude": { "origin": [139.7003], "scale": [0.003] } } }, { "field_value_factor": { "field": ["score"], "factor": [3], "modifier": ["log"], "missing": [1] } } ], "score_mode": ["multiply"] } }, "size": [1000], "_source": ["name", "score", "genre", "tel_number"] } GVODUJPO@TDPSFʹΑΔείΞͷ౷߹ 22 Ңͷݮਰ ܦͷݮਰ ϑΟʔϧυͷ ͦͷͷΛ͏ ͜ΕΒͷֻ͚ࢉͰ ࠷ऴతͳείΞͱ͢Δ KTPOΈʹͯ͘͘ਃ͠༁ͳ͍Ͱ͢ɾɾɾ
σϞ
None
·ͱΊ w POFUPPOFөըσʔτମݧΛఏڙ͢ΔͨΊʹɼ૬खͷΈΛεϜʔζ ʹఆྔԽ͢Δ͚ͩͰμϝͩͬͨɽ w ʮөըσʔτײઓͰ͢Δʯͱ͍͏Ծઆͷͱɼྑ͍ళΛαΫͬͱ ݕࡧͰ͖ΔΞϓϦΛ࡞ͨ͠ɽ w ಛʹɼݱࡏҐஔ͔Βͷڑɾ৯ϩάͷείΞɾΫΤϦͱͷϚονΛ ߟྀͨ͠είΞϦϯάΛߦͬͨɽ
25 FMBTUJDTFBSDI͍͍ͧʂ ʢެࣜυΩϡϝϯτ͕ຊʹʣ
&OKPZ