Elasticsearch for Jekyll: JekyllConf 2016

How to add powerful search to your Jekyll site with Elasticsearch and completely customizable Javascript, all open-source.

Transcript

1. Elasticsearch for Jekyll Using searchyll
2. Jekyll is awesome and easy. Search with Jekyll is hard. - I <3 Jekyll. So do you. - Most Jekyll sites don’t include search - Search is complicated - There are other solutions, but no me gusta - My blog needs search But I really, really, really want to use powerful search, and I don’t want to have to break up with Jekyll for it.
3. No search makes you like
4. But, we’re going to fix this. Project: build a Jekyll site with awesome search It needs to support: - Full-text search. - Fast queries. - Type-ahead find. - Highlighted results. - The option to filter based on document data (dates, categories, tags). - Hosting on Github pages. - Completely open-source & customizeable.
5. - Demo time. Cmd + shift + } A month later, I’ve got badass search on my Jekyll blog.
6. - I’m Allison. I make my living on search! - I work at One More Cloud - Bonsai - Hosted Elasticsearch - Websolr - Hosted Solr - Find us on Heroku! Okay, but who the hell are you to talk search?
7. - Gem: Searchyll - React components: Searchkit To build this, I used Elasticsearch via Searchyll and React.
8. 1. Why Elasticsearch? 2. What do I need? 3. How to get stuff into Elasticsearch using searchyll 4. How searchyll works 5. How to get stuff out of Elasticsearch (ie, searching it) What we’re going to talk about
9. - It’s fast. - It’s: - An open-source search engine. - Based on Apache Lucene, like Solr. - Distributed, enterprise-grade, scalable. - It does cool stuff, like highlighting, filtering, and a million cool things. The API is extensive and very developer friendly. - It is its own Webserver - It speaks HTTP! 1. Why Elasticsearch?
10. 1. A full-access Elasticsearch cluster URL 2. A public read-only URL to the same cluster 3. A Jekyll site with posts. 4. A client-side javascript framework to search things. Q: What do I need to get it to all work? A: 4 things.
11. 1. Add searchyll to your gemfile 2. Add searchyll to your config.yml, with some options 3. Wrap the content you want to be able to search in an <article/> tag 4. Run the indexing process a. Locally: $ BONSAI_URL=url jekyll build b. In production: $ BONSAI_URL=url bin/deploy Q: How do I get my posts to Elasticsearch? A: POST your posts to an index
12. 1. Add searchyll to your gemfile 2. Add searchyll to your config, and some opts
13. 3. Wrap to-be-searched content with <article/>. _layouts/post.html $ BONSAI_URL="https://user_name:password@trial-jekyll-1468587631.us-east-1.bonsai.io" jekyll build $ BONSAI_URL="https://user_name:password@trial-jekyll-1468587631.us-east-1.bonsai.io" bin/deploy 4. Run the indexing process from the terminal, locally or on deploy.
14. 1. We grab the full html output of posts by hooking to the : post_render phase 2. Run it through Nokogiri to pick up text in the article tag 3. Send it off to an Indexer, which communicates with Elasticsearch via ruby netHttp Q: How does Searchyll work? A: Jekyll Hooks + Indexer.
15. The main Jekyll Hook:
16. The Indexer: - Follows best practices when interacting with Elasticsearch. - Hot reindexes using the _alias endpoint. - Indexes in batches using with _bulk endpoint. - Removes any old indices under the same alias if you’re reindexing.
17. From my logs - the reindexing process:
18. 1. Set up your project to use Searchkit a. So, I also have to use React b. I’m using webpack to build, compile, and test my scripts. 2. I have a handy blog post on how to do this in more detail that I’ll link at the end. Q: How do I get stuff out of Elasticsearch? A: Public read-only cluster URL + Javascript.
19. Searchkit == easy development process and cool components!
20. 1. Set up your project to use Searchkit: set up a searchkit manager with your public readonly url
21. 2. Create a search component using Searchkit’s pre-defined components
22. 3. Give React a place to render your search component webpack/entry.js _includes/header.html
23. 1. Set up your project with the gem and config vars. 2. Run your indexing locally just before publishing to github: 3. Remove searchyll from the gemfile and config 4. Publish to Github Pages Q: How can I use this if I’m on Github Pages? A: index locally + remove searchyll. $ BONSAI_URL="https://user_name:password@trial-jekyll-1468587631.us-east-1.bonsai.io" jekyll build
24. Jekyll + Searchyll + Searchkit ==
25. - Twitter: @allizad - See search work: allizad.com - Break it for me! - Fork, customize, collaborate, submit issues: - https://github.com/omc/searchyll - Set up Search with your Jekyll site: - http://allizad.com/code/2016/05/02/using-webpack-with-jekyll/ - http://allizad.com/code/2016/05/06/elasticserch-for-jekyll/ - Share your cool use cases!: - @bonsaisearch - Get a Free developer Sandbox cluster with the public read-only feature: - allison@bonsai.io subject_line: JekyllConf & Bonsai Thanks!