Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
motemote-data-science-2
Search
kur0cky
August 01, 2020
Technology
2
580
motemote-data-science-2
ネタですが,Rからelasticsearchを使ったり,elasticsearchの便利な機能を一部紹介しています笑
kur0cky
August 01, 2020
Tweet
Share
More Decks by kur0cky
See All by kur0cky
The bootstrapping method for everyone
kur0cky
3
900
音楽理論と方向統計学の初歩/introduction of circular statistics and musicology
kur0cky
4
1.6k
NLP introduction in R 1
kur0cky
0
72
tidyverse tutorial 2
kur0cky
1
47
tidyverse tutorial 1
kur0cky
1
58
rating_introduction
kur0cky
1
820
motemote data science 1
kur0cky
1
510
Other Decks in Technology
See All in Technology
Java EE/Jakarta EEの現状と将来―クラウドネイティブ時代にJava EEは対応できるのか?―
takakiyo
1
130
継続的な改善 x ⾮連続的な進化
sansantech
PRO
3
140
KubeConにproposalを送りたい人へのアドバイス
sat
PRO
3
230
TechFeed Experts Night#27 〜 フロントエンドフレームワーク最前線 (Svelte)
baseballyama
1
190
Google Cloud Next '24でブログを10本書いた方法と勉強会を沸かせた方法
yasumuusan
0
290
SIEMを用いて、セキュリティログ分析の可視化と分析を実現し、PDCAサイクルを回してみた
coconala_engineer
0
280
Tellus の衛星データを見てみよう #mf_fukuoka
kongmingstrap
0
160
長期間TiDBを使ってきた話 @ 私たちはなぜNewSQLを使うのかTiDB選定5社が語る選定理由と活用LT / Experiences with TiDB Over Time
chibiegg
2
870
長期運用プロジェクトでのMySQLからTiDB移行の検証
colopl
2
830
JAWS-UG Bedrock Claude Night
yamahiro
3
540
推しは推せるときに推せ! プロダクトにフィードバックしていこう
nakasho
0
290
Janus
bkuhlmann
1
490
Featured
See All Featured
The Invisible Side of Design
smashingmag
294
49k
Gamification - CAS2011
davidbonilla
76
4.6k
Agile that works and the tools we love
rasmusluckow
325
20k
How to train your dragon (web standard)
notwaldorf
73
5.2k
ParisWeb 2013: Learning to Love: Crash Course in Emotional UX Design
dotmariusz
104
6.6k
[RailsConf 2023] Rails as a piece of cake
palkan
23
3.9k
KATA
mclloyd
15
12k
Fashionably flexible responsive web design (full day workshop)
malarkey
398
65k
Making the Leap to Tech Lead
cromwellryan
124
8.5k
The Mythical Team-Month
searls
216
42k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
155
14k
Reflections from 52 weeks, 52 projects
jeffersonlam
345
19k
Transcript
σʔλͰʮϞςʯΛIBDLͤΑ BU5PLZP3 LVSDLZ dσΟφʔฤd
ࣗݾհ w 5XJUUFSɿ!LVSDLZ@Z w ॴଐɿ໊ཧͷIPHFIPHFʢϲ݄ʣ w झຯɿԻָɼөըɼ͓ञɼλΠϐϯάɼFUD w ΈɿϞς͍ͨ 2
લճdөըσʔτฤd w POFUPPOFͳөըσʔτମݧΛఏڙ͢ΔͨΊʹɼࣗવͳձͷத͔ΒϨ ϏϡʔΛਪఆ͢Δ͜ͱΛతͱͨ͠ɽ w ͦͷͨΊʹɼөըυϝΠϯઐ༻ͷۃੑࣙॻΛ࡞͠ɼϨίϝϯυʹ׆༻ 3 https://speakerdeck.com/kur0cky/motemote-data-science-1
өըΛݟͯऴΘΓͰྑ͔ͬͨͷͩΖ͏͔
࣮͕ޠ͍ͬͯΔ
Կ͕͍͚ͳ͔ͬͨͷ͔
ࠗ׆αΠτʹΑΔͱ өըͷ༨ӆΛָ͠ΜͩΓɺөըͱ͍͏ڞ௨ͷͷձ͔Β૬खͷঁੑʹ͍ͭͯ͞Β ʹΕͨΓͱɺөըͷޙઈͷίϛϡχέʔγϣϯͷػձͱͳΓ·͢ɻ͜ͷνϟϯε Λ͍͔ͨ͢Ίʹɺөըͷޙʹ͝൧͓ͳͲͷ༧ఆΛೖΕ·͠ΐ͏ɻ ภݟʹ·ΈΕ͍ͯΔ͕ɼ͍ͬͨΜ͜ΕΛ৴͡Δɽ
POFUPPOFͳөը͚ͩͰμϝ ͦͷޙͷձ͕ॏཁ
ϩʔϧϓϨΠϯά 9 ͓͠Ζ͔ͬͨͶʂʂ దʹೖͬͪΌ͓͏͔ ͜Εതଧɽऑऀʹద͕Θ͔ΒΜ ͓͍͍͠ΠλϦΞϯ༧͓͍ͯͨ͠Μͩʂʂ ༻ҙप౸͗ͯ͢ॏ͍͠ɼ͓ෲ͕ݮͬͯͳ͍͔͠Εͳ͍ ͳʹ͔৯ͳ͕ΒΖ͏ʂ গ͠า͍ͯɼྑ͛͞ͳͱ͜Ζ୳ͦ͞͏͔ ૬खͷੑ֨ॴʹେ͖͘ґଘɽา͖͗͢/(ɽϦεΩʔ
ͦͷͰαΫͬͱݕࡧͰ͖Δ ΞϓϦ͕ॏཁ
3ͱFMBTUJDTFBSDI BU5PLZP3 LVSDLZ
w શจݕࡧγεςϜ w ಛɿ w ߴɽࢄܕͰεέʔϥϒϧɽΦʔϓϯιʔεɽ3&45"1*ɽ +40/ʹΑΔॊೈͳσʔλߏɽείΞϦϯάͷΧελϚΠζɽ 12 ՄࢹԽɾੳ ݕࡧ
σʔλऩू
Πϯετʔϧɾىಈ w .BDͷ߹ brew tap elastic/tap brew install elastic/tap/elasticsearch-full brew
install elastic/tap/kibana-full brew install elastic/tap/logstash-full brew install elastic/tap/metricbeat-full elasticsearch & kibana & w IUUQMPDBMIPTUͰใ͕ฦͬͯ͘Εىಈޭ w ,JCBOB IUUQMPDBMIPTU ͷ$POTPMFΛ͏ͱ৭ʑࢼ͍͢͠ 13 ଞͷ04ɿIUUQTXXXFMBTUJDDPHVJEFFOFMBTUJDTUBDLDVSSFOUJOTUBMMJOHFMBTUJDTUBDLIUNM
جຊ༻ޠ w ΠϯσοΫε w FMBTUJDTFBSDI͕ݕࡧɾղੳͷରͱ͢Δσʔλͷอଘઌ w υΩϡϝϯτλΠϓ w ΠϯσοΫεͷάϧʔϓ w
υΩϡϝϯτ w FMBTUJDTFBSDIʹอଘ͞Εͨσʔλ w ϑΟʔϧυ w υΩϡϝϯτʹؚ·ΕΔଐੑ 14 σʔλϕʔε ςʔϒϧ Ϩίʔυ ΧϥϜ 3%#Ͱ͍͏ͱɹɹ
؆୯ͳ͍ํ w +40/ͱ3&45"1*ͰઃఆɾΠϯσΩγϯάʢೖʣɾݕࡧͯ͢Λߦ͏ 15 • ݕࡧ GET index_name/_search { "query"
: { "match" : { "comment" : "σʔτ" } } } • ΠϯσΩγϯά PUT index_name/ { "name" : "ϥʔϝϯೋ", "genre" : "όʔɾμΠχϯά" } • ઃఆ PUT index_name/ { "settings" : { hogehoge }, "mappings" : { fugafuga } } ܗଶૉղੳશ֯ͷ౷ҰͳͲ ༷ʑͳઃఆΛهड़ ೖ͢Δσʔλ͕ͲͷΑ͏ͳ ϑΟʔϧυΛͪ͏Δͷ͔ɼͦͷܕهड़
3͔ΒFMBTUJDTFBSDIΛୟ͘ w FMBTUJDύοέʔδ͔Βୟ͘ɽ w σʔλϑϨʔϜΛΠϯσΩγϯάͰ͖Δɽ w શͯΛ3Ͱ͍݁ͤͨ͞ʂʂ w ΫΤϦɼϦετͰॻ͍ͯKTPOMJUFͰ+40/ʹ w
جຊͷૢ࡞ 1. conn <- connect(host="127.0.0.1", port=9200) 2. docs_bulk(conn, df, index) 3. Search(conn, index, body = <query>) 16
࣮ફ
త w өըσʔτײઓͰ͢Δ w ϩʔϧϓϨΠϯάͷ݁Ռɼ͍͔ͭ͘ͷཁ݅Λຬͨ͢ඞཁ͕͋Δ w ͦͷͷঢ়گʹ߹ΘͤͯαΫοͱܾΊΔ͜ͱ͕ॏཁ w جຊతʹ͋·Γา͔ͤͳ͍ w
Ͱ͖Δ͚࣭ͩͷྑ͍ళʹೖΔ 18
ͬͨ͜ͱ w ༻σʔλ w ौ୩ۙลͷϨετϥϯ݅ w ళ໊ɼδϟϯϧɼਓؾͷΫνίϛɼͦͷଞళฮใ 19 HFPDPEJOH"1* ݕࡧ6*
MFBqFUʹΑΔਤ %5ʹΑΔ៉ྷͳද ݱࡏͷऔಘ ݕࡧ εΫϨΠϐϯά
ཁ݅ͷୡ w LVSPNPKJQMVHJOʹΑΔલॲཧ w τʔΫϯԽɼશ֯౷ҰɼεςϛϯάɼFUD w σϑΥϧτͷείΞʢ#.ʣΛ͏ w ۙ͞ʹԠͨ͡ݮਰؔ͏ w
৯ϩάείΞ͏ 20 w ຊޠͷݕࡧ w ΫΤϦͱͷϚον w ͋·Γา͔ͤͳ͍ w ͍͍ళʹೖΔ https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-kuromoji-analyzer.html
ڑʹԠͨ͡ݮਰؔ w ࠓճɼҢɾܦͦΕͧΕʹ ΨγΞϯΛ༻ w ଞʹɼࢦؔઢܗ͕͋Δ 21 https://www.elastic.co/guide/en/elasticsearch/reference/current/query- dsl-function-score-query.html
{ "query": { "function_score": { "query": { ී௨ͷΫΤϦ͜͜ʹॻ͘ }, "functions":
[ { "gauss": { "latitude": { "origin": [35.6591], "scale": [0.003] } } }, { "gauss": { "longitude": { "origin": [139.7003], "scale": [0.003] } } }, { "field_value_factor": { "field": ["score"], "factor": [3], "modifier": ["log"], "missing": [1] } } ], "score_mode": ["multiply"] } }, "size": [1000], "_source": ["name", "score", "genre", "tel_number"] } GVODUJPO@TDPSFʹΑΔείΞͷ౷߹ 22 Ңͷݮਰ ܦͷݮਰ ϑΟʔϧυͷ ͦͷͷΛ͏ ͜ΕΒͷֻ͚ࢉͰ ࠷ऴతͳείΞͱ͢Δ KTPOΈʹͯ͘͘ਃ͠༁ͳ͍Ͱ͢ɾɾɾ
σϞ
None
·ͱΊ w POFUPPOFөըσʔτମݧΛఏڙ͢ΔͨΊʹɼ૬खͷΈΛεϜʔζ ʹఆྔԽ͢Δ͚ͩͰμϝͩͬͨɽ w ʮөըσʔτײઓͰ͢Δʯͱ͍͏Ծઆͷͱɼྑ͍ళΛαΫͬͱ ݕࡧͰ͖ΔΞϓϦΛ࡞ͨ͠ɽ w ಛʹɼݱࡏҐஔ͔Βͷڑɾ৯ϩάͷείΞɾΫΤϦͱͷϚονΛ ߟྀͨ͠είΞϦϯάΛߦͬͨɽ
25 FMBTUJDTFBSDI͍͍ͧʂ ʢެࣜυΩϡϝϯτ͕ຊʹʣ
&OKPZ