Enabling search in your Grails application via Solr

Learn how to leverage the power of Solr for indexing and searching in your grails web application. We'll discuss how to set up a solr development environment, integrate Grails domain classes into a Solr index and show you how to query the Solr index.

Transcript

1. Enabling Search in Grails With Apache Solr Mike Hugo Entagen, LLC
2. ACCELERATING INSIGHT ENTAGEN
10. Java API Indexing and Searching Text
11. indexing storage
12. Document
13. Document field value ID 2 name “Mike Hugo” company “Entagen” bio “lorem ipsum dolor sum etc...”
14. Index field value name “mike hugo” company “Entagen” bio “lorem ipsum dolor sum etc...” field value name “mike hugo” company “Entagen” bio “lorem ipsum dolor sum etc...” field value name “mike hugo” company “Entagen” bio “lorem ipsum dolor sum etc...” field value name “mike hugo” company “Entagen” bio “lorem ipsum dolor sum etc...” field value name “mike hugo” company “Entagen” bio “lorem ipsum dolor sum etc...” field value name “mike hugo” company “Entagen” bio “lorem ipsum dolor sum etc...” field value id 2 name “Mike Hugo” company “Entagen” bio “lorem ipsum dolor sum etc...”
15. Index term docid mike 1,2,3,5 hugo 1,2 entagen 2,5,6 lorem 1,2,3,4,5,6 ipsum 2,3,6
16. name: mike Query:
17. name: mike field value id 2 field value id 2 field value id 2 field value id 2 Query: Matching Documents:
19. server-lization
20. Standalone WAR
21. HTTP REST API
22. Faceting
23. Highlighting
24. Spell Check
25. More Like This
26. Caching
27. Replication Master Slave1 Slave2 Slave3
28. Sharding* Shard1 Shard2 Client Request Aggregated Response *client defined
29. Sharding* Shard1 Shard2 Client Request Aggregated Response *client defined “SolrCloud” coming soon in Solr4
30. And much more Grouping Spatial (location search) Tika Integration Boosting Numeric Field Stats
34. Flat Index field value name “mike hugo” company “Entagen” bio “lorem ipsum dolor sum etc...” field value name “mike hugo” company “Entagen” bio “lorem ipsum dolor sum etc...” field value name “mike hugo” company “Entagen” bio “lorem ipsum dolor sum etc...” field value name “mike hugo” company “Entagen” bio “lorem ipsum dolor sum etc...” field value name “mike hugo” company “Entagen” bio “lorem ipsum dolor sum etc...” field value name “mike hugo” company “Entagen” bio “lorem ipsum dolor sum etc...” field value id 1 title Academy Dinosaur categories documentary broadcast description A Epic Drama of a Feminist And a Mad Scientist who must Battle a Teacher in The Canadian Rockies (Relational) Database
35. field value id 1 title Academy Dinosaur categories documentary, broadcast, informative description A Epic Drama of a Feminist And a Mad Scientist.... “flatten”
36. Data Import Handler http://localhost:8983/solr/db/ dataimport?command=full-import
37. Data Import Handler http://localhost:8983/solr/db/ dataimport?command=delta-import
38. Config> dataSource driver="org.hsqldb.jdbcDriver" url="jdbc:hsqldb:./example-DIH/hsqldb/ex document> <entity name="item" query="select * from item" deltaQuery="select id from item where last_modified > '${dataimporter.l <field column="NAME" name="name"/> <field column="NAME" name="nameSort"/> <field column="NAME" name="alphaNameSort"/> <entity name="feature" query="select DESCRIPTION from FEATURE where ITEM_ID='${item.ID}'" deltaQuery="select ITEM_ID from FEATURE where last_modified > '${da parentDeltaQuery="select ID from item where ID=${feature.ITEM_ID}"> <field name="features" column="DESCRIPTION"/> </entity> <entity name="item_category" query="select CATEGORY_ID from item_category where ITEM_ID='${item. deltaQuery="select ITEM_ID, CATEGORY_ID from item_category where la parentDeltaQuery="select ID from item where ID=${item_category.ITEM <entity name="category" query="select DESCRIPTION from category where ID = '${item_cate deltaQuery="select ID from category where last_modified > '${da parentDeltaQuery="select ITEM_ID, CATEGORY_ID from item_categor <field column="description" name="cat"/> </entity> </entity> Data Import Handler
39. ouch
40. Solrj
41. @Grab(group='org.apache.solr', module='solr-solrj', version='1.4.1') @Grab(group='org.slf4j', module='slf4j-jdk14', version='1.5.5') import org.apache.solr.client.solrj.impl.CommonsHttpSolrServer import org.apache.solr.common.SolrInputDocument String url = "http://localhost:8983/solr" def server = new CommonsHttpSolrServer( url ); def doc = new SolrInputDocument() doc.addField("id", 2) doc.addField("word_s", "Cow") doc.addField("desc_t", "Farm Animal") server.add(doc) server.commit()
43. runtime 'mysql:mysql-connector-java: 5.1.16' } plugins { runtime ":solr:0.2" runtime ":hibernate:$grailsVersion" runtime ":jquery:1.7.1" runtime ":resources:1.1.6" // Uncomment these (or add new ones) to enable additional resources capabilities //runtime ":zipped-resources:1.0" //runtime ":cached-resources:1.0" //runtime ":yui-minify-resources: 0.1.4"
44. runtime 'mysql:mysql-connector-java: 5.1.16' } plugins { runtime ":solr:0.2" runtime ":hibernate:$grailsVersion" runtime ":jquery:1.7.1" runtime ":resources:1.1.6" // Uncomment these (or add new ones) to enable additional resources capabilities //runtime ":zipped-resources:1.0" //runtime ":cached-resources:1.0" //runtime ":yui-minify-resources: 0.1.4"
45. git clone git@github.com:entagen/grails-solr-plugin.git
46. runtime 'mysql:mysql-connector-java: 5.1.16' } grails.plugin.location.solr = "/path/to/git/clone/grails-solr-plugin" plugins { runtime ":solr:0.2" runtime ":hibernate:$grailsVersion" runtime ":jquery:1.7.1" runtime ":resources:1.1.6" // Uncomment these (or add new ones) to enable additional resources capabilities //runtime ":zipped-resources:1.0" //runtime ":cached-resources:1.0"
47. grails start-solr
48. $ grails start-solr | Configuring classpath..... Starting Solr - Solr HOME is /Users/ mjhugo/.grails/2.0.3/projects/sakila/ solr-home Solr logs can be found here: /Users/ mjhugo/.grails/2.0.3/projects/sakila/ solr-home/logs Console access: http://localhost:8983/ solr/
51. package com.piragua.sakila class Film { static enableSolrSearch = true static solrAutoIndex = true String title String description Integer releaseYear Byte rentalDuration BigDecimal rentalRate //...
52. package com.piragua.sakila class Film { static enableSolrSearch = true static solrAutoIndex = true String title String description Integer releaseYear Byte rentalDuration BigDecimal rentalRate //... Add dynamic search methods to domain
53. package com.piragua.sakila class Film { static enableSolrSearch = true static solrAutoIndex = true String title String description Integer releaseYear Byte rentalDuration BigDecimal rentalRate //... (re)index after save
54. build the index
55. Film.list().each{ it.indexSolr() }
56. search!
57. http://localhost:8983/solr/ select/?q=dinosaur&wt=json HTTP REST API
58. import com.piragua.sakila.* def r = Film.searchSolr('dinosaur') println r.resultList.title ['ACADEMY DINOSAUR', 'CENTER DINOSAUR', 'DINOSAUR SECRETARY']
59. * grails-solr:0.3
61. magic! http://www.cs.cmu.edu/~chaki/magic/magic.gif
62. package com.piragua.sakila class Film { String description Date lastUpdated //...
63. def static final typeMapping =[ "int":"_i", "float":"_f", "long":"_l", "boolean":"_b", "double": "_d", "class java.lang.String":"_s", "class java.util.Date": "_tdt"] <dynamicField name="*_i" type="int" indexed="true" stored="true"/> <dynamicField name="*_s" type="string" indexed="true" stored="true"/> <dynamicField name="*_l" type="long" indexed="true" stored="true"/> <dynamicField name="*_t" type="text" indexed="true" stored="true"/> SolrUtil schema.xml
64. <fieldType name="string" class="solr.StrField" sortMissingLast="true" omitNorms="true"/> A Epic Drama of a Feminist And a Mad Scientist who must Battle a Teacher in The Canadian Rockies A Epic Drama of a Feminist And a Mad Scientist who must Battle a Teacher in The Canadian Rockies Input Indexed
65. <fieldType name="text" class="solr.TextField" positionIncrementGap="100"> <analyzer type="index"> <tokenizer class="solr.WhitespaceTokenizerFactory"/> <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" /> <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.SnowballPorterFilterFactory" language="English" protected="protwords.txt"/> </analyzer> A Epic Drama of a Feminist And a Mad Scientist who must Battle a Teacher in The Canadian Rockies battl canadian drama epic feminist mad must rocki scientist teacher who Input Indexed
66. class Film { static enableSolrSearch = true static solrAutoIndex = true @Solr(asText=true) String title @Solr(asText=true) String description Integer releaseYear Byte rentalDuration BigDecimal rentalRate
67. extending indexing
68. select special_features from film where film_id limit 10; special_features -------------------------------- Deleted Scenes,Behind the Scenes Trailers,Deleted Scenes Trailers,Deleted Scenes Commentaries,Behind the Scenes Deleted Scenes Deleted Scenes Trailers,Deleted Scenes Trailers Trailers,Deleted Scenes Trailers,Deleted Scenes
69. @Solr(field="specialFeatures") String specialFeatures def indexSolrSpecialFeatures(doc){ if(specialFeatures) { def fieldName = this.solrFieldName("specialFeatures") doc.addField(fieldName, specialFeatures.split(',')) } } indexSolr propertyName solrInputDocument
70. @Solr(field="specialFeatures") String specialFeatures def indexSolrSpecialFeatures(doc){ if(specialFeatures) { def fieldName = this.solrFieldName("specialFeatures") doc.addField(fieldName, specialFeatures.split(',')) } }
72. faceting
73. facet results facetfield=foo
74. scripts
75. grails start-solr
76. grails stop-solr
77. grails delete-solr-index
78. dynamic methods
79. Film.searchSolr(...)
80. def f = Film.read(1) f.indexSolr()
81. def f = Film.read(1) f.deleteSolr()
82. mapping
83. @Solr(field="overridden_field_name") String specialFeatures
84. static enableSolrSearch = true static solrAutoIndex = true @Solr(ignore="true") String notToBeIndexed
85. static enableSolrSearch = true static solrAutoIndex = true static solrExplicitFieldAnnotation = true @Solr(asText=true) String firstName String middleName // won’t be indexed @Solr(asText=true) String lastName
86. service
87. solrService.search("foo")
88. SolrQuery q = new SolrQuery("foo") solrService.search(q)
89. tags
90. <solr:facet field="genre_s" result="${result}" fq="${fq}" q="${q}" min="2"> <h3>Filter by Genre</h3> </solr:facet>
91. <solr:resultLink result="${item}"> ${item.name} </solr:resultLink>
93. domain class reloading
94. type mapping gaps
95. bulk indexing
96. component fields (1:n and m:n)
97. Thank You happy searching mike@entagen.com @piragua http://www.triplemap.com http://www.entagen.com TripleMap