Einführung in Lucene Solr (PHPUG D/DU/KR)

Transcript

1. PHP UserGroup D / DU / KR 14.01.2013
3. Wer? Dominik Siebel Software Engineer @ trivago PHP since 2005 Search since ~2009
4. Lucene • Doug Cutting (1999) • Programmbibliothek zur Volltextsuche (Java) • “Document Store” • Open Source • Apache top-level Projekt seit Feb. 2005
5. Lucene - Features • phrase queries, wildcard queries, proximity queries, range queries • Multi-Index search • “Non blocking” • Facetting, Highlighting, Joins, Grouping • Query suggest • Erweiterbares Scoring (Plugins) & Sortierung • Konfigurierbare Speicherformate (Codecs, z.B. VSM, Okapi BM25) • Hoch skalierbar*
6. Lucene – Features KPIs • 150GB/h auf moderner Hardware* • Geringer Memory Footprint (1MB Heap) • T Inkrementelle Indizierung == T Batch Indizierung • Indexgröße entspricht ca. 20-30% des indizierten Texts*
7. The Lucene Family • Mahout • Nutch • Tika • • Lucene Core (Java) • Lucene.NET • Solr, SolrJ, ... • ...
8. So, why Solr?
9. Solr
10. Solr • Yonik Seeley (CNET Networks, 2004) • Enterprise Search Server auf Apache Lucene • Apache top-level Projekt seit Jan. 2006 (unter Lucene) • Merge mit Lucene im März 2010 • Aktuelle Version: 4.2.0 • Kommerzieller Support, Consulting und Training durch Lucid Imagination
11. Solr – Features • Faceted Search • Hit Highlighting • Replication • Distributed Search • Clustering • Plugins • Embeddable • Admin Interface • Diverse In-/Output formate (JSON, XML, PHP, Ruby, Python, XSLT, Velocity, Java binary) • Caching • Relevanz Boosting • Geo-spatial search • Spellcheck
12. Integrating Solr • REST API • PHP: Apache Solr PHP extension, Solarium • Java: SolrJ • Ruby: rsolr • Python: sunburnt, solrpy, PySolr... • Javascript: AJAX Solr • Perl: SolPerl, Solr.pm
13. Solr in 5 minutes...
14. Solr in 5 minutes ... (1) Download Solr (2) Download Oracle JDK (3) Start: java -jar solr- 4.2.0/examples/start.jar [- Djetty.port=4711] (4) Index: solr- 4.2.0/examples/exampledocs/post.sh *.xml (5) Search: http://localhost:8983/solr/collection1/select?q=*:* (6) Play around!
15. Lucene
16. Lucene - Index
17. Lucene – Search “Lifecycle”
18. Field Token DocIds title benthic 3 title carbon 1,2,3 title composition 2 title isotope 1,2 title oxygen 1,2,3 title ratios 1 ... ... ... latitude 74.1 1,3 latitude 63.9 2 longitude 11.0 1,2 longitude 12.3 3 Lucene – Inverted Index
19. • http://lucene.apache.org/core/old_versioned_docs • http://nlp.stanford.edu/IR-book/html/htmledition/qu • org.apache.lucene.search.similarities.Similari ty & Unterklassen • Lucene – Scoring
20. Solr - Schema • Definiert Felder und Feld-Typen • Deklarative Lucene Analyzer Konfiguration • DynamicFields • CopyFields • Similarity Plugins pro Feld konfigurierbar
21. Solr - Solrconfig
22. Solr in 10 minutes...
23. Literatur Lucene In Action Manning ISBN-10: 1933988177 ISBN-13: 978-1933988177 Apache Solr 4 Cookbook Packt Publishing ISBN-10: 1782161325 ISBN-13: 978-1782161325 Apache Solr 3 Enterprise Search Server Packt Publishing ISBN-10: 1849516065 ISBN-13: 978-1849516068
24. Resourcen • http://searchhub.org/ • http://wiki.apache.org/solr • http://lucene.apache.org/solr/discussion.html • http://search-lucene.com/ • http://www.solrtutorial.com/ • http://solr.pl/en/
25. Quellen • http://en.wikipedia.org/wiki/Apache_Solr • http://en.wikipedia.org/wiki/Lucene • http://wiki.apache.org/solr/ • http://www.solrtutorial.com/ • Apache Solr 3 Enterprise Search Server, Packt Publishing, ISBN- 10: 1849516065 •