Talk given at Berlin buzzwords 2013
Faceting analyzed fieldswith some sprinkles ofprobability theoryconjures trending topic analysis and otherinteresting insightsBoaz LeskesElasticsearch@bleskeswork done forBuzzcapture
View Slide
Trending?
© Buzzcapture
reference referencetopic© Buzzcapture
topic reference≠
topic referenceP(w|T) =kDt|w 2 DtkkDtk
P(w|T) =kDt|w 2 DtkkDtk
browndogfoxquick2 5 10 125 6 12 13256101213browndogfoxquick
In our index.• Terms = 12GB• “Arrows” = 41GB
{tweet: {type: "string",analyzer: "whitespace"fielddata: {filter: {regex: "^#.*",frequency: {min: 10}}}}}Drop terms which occur too little
Drop docs with too many terms
iculture 10,122floor 8,998cover 6,874toy 4,402ground 3,8414.0 7,8784.1 4,292rtacties 4,078jelly 2,905bean 2,857