ELASTICSEARCH text to full-text within 1 hour 

Mark van Straten @markvanstraten Q42 https://q42.com 

What is full text searching? 

What is filtering 

What is suggesting 

Small history lesson Lucene Solr Elasticsearch 

How does full text searching work The document’s text is put in an inverted index doc1 = “no limit, no boundaries” doc2 = “no limit made music” $q=boundaries => doc1 $q=limit => doc1, doc2 no 1,1 2 limit 1 2 boundaries 1 made 2 music 2 

Tokenizers Tokenizers are used to break a string down into a stream of terms or tokens. A simple tokenizer might split the string up into terms wherever it encounters whitespace or punctuation. “The|quick|brown|fox…” 

TokenFilters Removing words based on given criteria. Commonly used: StopTokenFilter “We are changing the world with the full text searching capabilities of elasticsearch.” (culture aware, configurable) 

Analyzers Modify tokens to make them better searchable 

Download Elasticsearch + Unzip somewhere 

Download HEAD $elasticsearch-1.3.2\bin\plugin --install mobz/elasticsearch-head 

Create a new (C#) project File -> New -> Console application Install the C# Client library NEST from NuGet $Install-package -Id Nest 

Connecting to Elasticsearch var client = new ElasticClient(new ConnectionSettings(new Uri("http://localhost:9200")).SetDefaultIndex("my-index")); client.CreateIndex("my-index", s => s .AddMapping(mapMyObjectForIndex) ); 

Index mappings Default all is analyzed. Some fields [url] might not need that. private static PutMappingDescriptor mapStaatberichtForIndex(PutMappingDescriptor obj) { return obj .Properties(p => p .String(n => n.Name(nn => nn.Name).Index(FieldIndexOption.NotAnalyzed)) .String(n => n.Name(nn => nn.Url).Index(FieldIndexOption.NotAnalyzed)) ); } 

Indexing //load stuff into memory var data = JsonConvert.DeserializeObject>(File.ReadAllText(@" C:\development\my-object-input-data.json")); //index it client.IndexMany(data, "my-index"); 

Searching Searching will return results with a score, lower is less-relevant. var queryStringQuery = client.Search(s => s.QueryString("Lightning fast ")); Console.WriteLine("Found " + queryStringQuery.Total + " results, first 10:"); foreach (var doc in queryStringQuery.Documents) { Console.WriteLine(" - Found: " + doc.Url); } 

Filtering Filtering is binary (yes/no) but gives no scoring. (FAST! just the inverted index) var filteredItems = client.Search(s => s.Filter(f => f.Term(t => t.Name, "IronMan"))); foreach (var doc in filteredItems.Documents) { Console.WriteLine(" - Found: " + doc.Url); } 

Aggregations (previously: Facets) var facetsSearch = client.Search(s => s .FacetTerm(t => t.OnField(tf => tf.Category)) ); foreach (var facet in facetsSearch.Facets) { Console.WriteLine("Facet: " + facet.Key); foreach (var facetValue in (facet.Value as TermFacet).Items) Console.WriteLine(" - Option: " + facetValue.Term); } 

Suggest var suggestQuery = client.Search(s => s .SuggestTerm("suggestterm", sg => sg.OnField(sgf => sgf.TextBody).Text("fastest man alive")) ); foreach (var suggest in suggestQuery.Suggest["suggestterm"]) { Console.WriteLine(" - Maybe you mean: " + suggest.Text); } 

Questions?