Hippo GetTogether 2013-06-21 1200 Radical performance increases in 7.8 - Ard Schrijvers Hippo

Ard Schrijves, Hippo R&D developer details the changes made in Hippo CMS 7.8 to dramatically increase site performance in projects.

Transcript

1. This presentation is about .... 21 jun 2013 Ard Schrijvers a.schrijvers@onehippo.com ard@apache.org
2. RSI
4. Long spinning wheels.... Is there on your computer anything more annoying? more useless? or even more violent?
5. Avoid spinning wheels...the 7.8 helps!
6. 7.7 / 7.8 HST performance
7. Where is the improvement?
8. It is more subtle First of all : A blistering fast HST does not guarantee blistering fast websites for the 7.7 ...nor for the 7.8
9. A durable fast delivery requires 1. A blistering fast HST ( + repository) 2. Confident developers questioning and challenging customers requirements 3. Customers willing to KISS during first iterations : Don't start innovating directly 4. An efficient content model 5. No developer mistakes 6. Performance measurements upfront 7. Performance logging
10. So what improved in the 7.8?
11. 7.7 Vulnerabilities / easy mistakes 1. Date range queries 2. Large hst configuration resulting in hiccups during changes 3. Many needless virtual nodes 4. Possible expensive searches for CMS users with read access to a small part of the repository 5. Getting total number of search results is expensive 6. CMS is not lazy 7. CMS template editor encourages incorrect usage of compounds 8. Repository based website menus loading large parts of the repository
12. 7.7 Vulnerabilities / easy mistakes 1. No page caching / thundering herd vulnerability 2. Server side external (service) calls 3. CMS JCR events 4. Free text queries possibly blowing up the repository 5. Easy feedback / page diagnostics missing
13. Why extra performance focus with 7.8?
14. Enough enough enough
15. Fast Date Range Queries 1. Supported in 7.7.9 and higher 2. For 7.7.9, Lucene re-index needed 3. In hst-config.properties set the default with default.query.date.range.resolution = 4. Default in 7.8.x for HST is resolution DAY 5. Default in 7.7.9 for HST is resolution MILLISECOND 6. Resolution based query supported through HstFilter api
16. How much faster?
17. Log scale
18. Bonus Gimme all documents of 2013 or Gimme all documents of 2013/06/21 do not need range queries anymore!! Filter#addEqualTo("my:date", Calendar.getInstance(), DateTools.Resolution. YEAR) Filter#addEqualTo("my:date", Calendar.getInstance(), DateTools.Resolution. DAY)
19. HST stale model support 1. Supported in 7.7.9 and higher 2. What it solves : hiccups in case reloads of large hst configuration
20. Virtual nodes reduced I think star trek is stupid but just want to be one of the geeks and I think managers and marketers like this because they then think I must be really smart and technical.... Ever wondered what 'trek' stands for? To make a sloooooow or arduous journey ... How dull
21. Virtual nodes reduced
22. Virtual nodes reduced
23. Authorized searches Required by accessing canonical content instead of virtual nodes
24. How did we do it? The Authorization Query
25. Does it perform? Oh yes! See my blog
26. Bonus 1. Users with little read access have instant authorized searches! 2. Correct instant faceted navigation authorized counts!
27. Efficient total number of search hits javax.jcr.query.QueryResult#getNodes()#getSize() close to broken in Jackrabbit in practice....
28. Why? QUIZ : Assume there are 1.000.000 newsdocs javax.jcr.QueryManager mngr = ... String q = "//element(*,my:newsdoc)"; long size = mngr.createQuery(q,"xpath").execute() .getNodes().getSize(); System.out.println(size); q = "//element(*,my:newsdoc) order by @jcr:score"; size = mngr.createQuery(q,"xpath").execute().getNodes().getSize(); System.out.println(size); output: -1 1000000 (after an hour)
29. But...this looks really dumb It isn't really... The problem is namely very complex Ask search gurus about it... "Authorization...hmmm, yeah, I don't care"
30. But you have the Authorization Query So you have instant correct authorized counts from Lucene? Correct Is Hippo R&D then smarter than all the search gurus and Jackrabbit developers? No not really... We just chose a security model that could be mapped to Lucene queries But isn't that super smart and bright?
31. Authorized total search hits HstQueryResult#getTotalSize() or HippoNodeIterator#getTotalSize()
32. CMS Laziness
33. CMS Laziness Why wasn't this done before? Well....it's like the north-south line Also realize: It is easy to state the obvious or know the problem but way harder to do something about it
34. HST Page cache In a running environment it can be switched on/off per 1. Virtual host 2. Channel (mount) 3. Sitemap item 4. Component
35. HST Page cache
36. HST Page cache bonus Stampeding herd protection
37. What about partially cacheable pages?
38. And without client side ajax?
39. Caching and relevance / targeting? Page caching works with targeting! HOW? number of uniquely rendered pages << unique number of site visitors
40. And 100% personalized blocks?
41. Server side external (service) calls
42. Server side external (service) calls Suppose requesting a remote football JSON object... but the service cannot make up its mind... Use ASYNC = true Optionally ASYNCMODE = esi Or asynchronous server side calls & caching
43. CMS JCR events Much more efficient now Ask this guy
44. Free text queries
45. Page Diagnostics Configurable in repository hst config
46. Page Diagnostics
47. 7.8 Tackled vulnerabilities wrap up 1. Date range queries 2. Large hst configuration resulting in hiccups during changes 3. Many needless virtual nodes 4. Possible expensive searches for CMS users with read access to a small part of the repository 5. Getting total number of search results is expensive 6. CMS is not lazy 7. CMS template editor encourages incorrect usage of compounds 8. Repository based website menus loading large parts of the repository
48. 7.8 Tackled vulnerabilities wrap up 1. No page caching / thundering herd vulnerability 2. Server side external (service) calls 3. CMS JCR events 4. Free text queries possibly blowing up the repository 5. Easy feedback / page diagnostics missing
49. 7.8 improvements in progress / done 1. Fine grained publishing of hst configuration 2. Reduced memory consumption of hst model
50. 7.8 further improvements / features 1. Lazy loading hst model 2. Reduce usage of facetselect and mirrors with multivalued properties containing uuids 3. Stale page cache support 4. Cluster wide (couchbase memcached bucket) shared hst page cache with stale page support
51. References 1. www.onehippo.com/en/resources/blogs/Author/Ard+Schrijvers 2. www.onehippo.com/en/resources/blogs?query=performance 3. www.onehippo. org/7_8/librarysearch/Version/7_8/Category/performance