Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Faceting analyzed fields with some sprinkles of...

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for Boaz Leskes Boaz Leskes
June 04, 2013
Technology
80
0

Faceting analyzed fields with some sprinkles of probability theory

Talk given at Berlin buzzwords 2013

Avatar for Boaz Leskes

Boaz Leskes

June 04, 2013

More Decks by Boaz Leskes

See All by Boaz Leskes
Every Shard Deserves a Home - Shard Allocation in Elasticsearch
Avatar for Boaz Leskes bleskes
0
330
Life of a Document in Elasticsearch
Avatar for Boaz Leskes bleskes
3
3.3k
Resiliency in Elasticsearch & Lucene
Avatar for Boaz Leskes bleskes
0
550
Resiliency in Elasticsearch & Lucene
Avatar for Boaz Leskes bleskes
0
260
Designing Concurrent Distributed Sequence Numbers for Elasticsearch
Avatar for Boaz Leskes bleskes
2
740
Not all Nodes are Created Equal - Scaling Elasticsearch
Avatar for Boaz Leskes bleskes
1
380
Not all Nodes are Created Equal - Scaling Elasticsearch
Avatar for Boaz Leskes bleskes
6
710
The ELK Stack: For Real-Time Enlightenment
Avatar for Boaz Leskes bleskes
1
1.7k
Staying Ahead of Users And Time - two use cases of scaling data with Elasticsearch
Avatar for Boaz Leskes bleskes
1
350

Other Decks in Technology

See All in Technology
もっとコンテンツをよく構造化して理解したいので、LLM 時代こそ Taxonomy の設計品質に目を向けたい〜！
Avatar for MasatoMasaMasa morinota
0
170
Fabric MCPの紹介と使い分け
Avatar for Ryoma Nagata ryomaru0825
1
120
Percolatorを廃止し、マルチ検索サービスへ刷新した話 / Search Engineering Tech Talk 2026 Spring
Avatar for Visional Engineering ＆ Design visional_engineering_and_design
0
310
ファインディの事業拡大を支える 拡張可能なデータ基盤へのリアーキテクチャ
Avatar for Noriaki Hiraki hiracky16
0
850
国内外の生成AIセキュリティの最新動向 & AIガードレール製品「chakoshi」のご紹介 / Latest Trends in Generative AI Security (Domestic & International) & Introduction to AI Guardrail Product "chakoshi"
Avatar for NTT docomo Business nttcom
4
1.7k
VespaのParent Childを用いたフィードパフォーマンスの改善
Avatar for 小野崇之 taking
0
260
AIと乗り切った1,500ページ超のヘルプサイト基盤刷新とさらにその先の話
Avatar for mugi / Hajime Mugishima mugi_uno
2
300
QAエンジニアはどうやって プロダクト議論の場に入れるのか？
Avatar for Sammy(MoritaMasami) moritamasami
2
380
20260428_Product Management Summit_Loglass_JoeHirose
Avatar for 広瀬丈 loglassjoe
4
6.8k
AWS Transform CustomでIaCコードを自由自在に変換しよう
Avatar for k-kun duelist2020jp
0
240
ハーネスエンジニアリング入門
Avatar for 西岡 賢一郎 (Kenichiro Nishioka) knishioka
0
110
プラットフォームエンジニアリングの実践 - AWS コンテナサービスで構築する社内プラットフォーム / AWS Containers Platform Meetup #1
Avatar for Masatoshi Hayashi literalice
1
240

Featured

See All Featured
The Spectacular Lies of Maps
Avatar for Per Axbom axbom
PRO
1
730
Skip the Path - Find Your Career Trail
Avatar for Mark Kilby mkilby
1
110
Facilitating Awesome Meetings
Avatar for Lara Hogan lara
57
6.8k
[SF Ruby Conf 2025] Rails X
Avatar for Vladimir Dementyev palkan
2
1k
How to audit for AI Accessibility on your Front & Back End
Avatar for davetheseo davetheseo
0
350
For a Future-Friendly Web
Avatar for Brad Frost brad_frost
183
10k
The Illustrated Children's Guide to Kubernetes
Avatar for Chris Short chrisshort
51
52k
Git: the NoSQL Database
Avatar for Brandon Keepers bkeepers
PRO
432
67k
Into the Great Unknown - MozCon
Avatar for Noah Learner thekraken
41
2.4k
Performance Is Good for Brains [We Love Speed 2024]
Avatar for Tammy Everts tammyeverts
12
1.6k
A brief & incomplete history of 
UX Design for the World Wide Web: 1989–2019
Avatar for Jason CranfordTeague jct
1
370
A Soul's Torment
Avatar for Nat Ma seathinner
6
2.8k

Transcript

  1. Faceting analyzed fields with some sprinkles of probability theory conjures

    trending topic analysis and other interesting insights Boaz Leskes Elasticsearch @bleskes work done for Buzzcapture

  2. Trending?

  3. © Buzzcapture

  4. © Buzzcapture

  5. reference reference topic © Buzzcapture

  6. topic reference ≠

  7. topic reference P(w|T) = kDt |w 2 Dt k kDt

    k
  8. None

  9. topic reference P(w|T) = kDt |w 2 Dt k kDt

    k

  10. P(w|T) = kDt |w 2 Dt k kDt k

  11. brown dog fox quick 2 5 10 12 5 6

    12 13 2 5 6 10 12 13 brown dog fox quick

  12. In our index. • Terms = 12GB • “Arrows” =

    41GB

  13. { tweet: { type: "string", analyzer: "whitespace" fielddata: { filter:

    { regex: "^#.*", frequency: { min: 10 } } } } } Drop terms which occur too little

  14. Drop docs with too many terms

  15. reference reference topic © Buzzcapture

  16. iculture 10,122 floor 8,998 cover 6,874 toy 4,402 ground 3,841

    4.0 7,878 4.1 4,292 rtacties 4,078 jelly 2,905 bean 2,857