Lexer + Elasticsearch

How does Lexer leverage Elasticsearch for it's business

Transcript

1. Lexer + Elasticsearch
2. aaron.wallis@lexer.io - @d2kagw - http://lexer.io Hi I’m Aaron, and I’m the founder of Lexer
3. (Pppsssssst…. We’re Hiring) Human Data for Genuine Engagement
4. aaron.wallis@lexer.io - @d2kagw - http://lexer.io Our whole business is about search….
5. aaron.wallis@lexer.io - @d2kagw - http://lexer.io Started using Elasticsearch back in 2008. Stopped in 2009. Started again in 2013 and haven’t looked back.
6. aaron.wallis@lexer.io - @d2kagw - http://lexer.io Fast Searching. Unstructured Data. Aggregations and Facets. “Cloud Scalable”.
7. aaron.wallis@lexer.io - @d2kagw - http://lexer.io Our primary goal is to make Data invisible. Making data-zero people, data-heroes.
8. aaron.wallis@lexer.io - @d2kagw - http://lexer.io DEMO
9. Our Cluster Unstructured Content (all our incoming public data) Identities (highly structured identities with thousands of fields) Metadata (searchable stuff for dashboard interaction) 3 indexes, 4 clusters, 5 TB of Data, 2B Objects, ~100 EC2 Nodes… (Pppsssssst…. Did I mention we’re hiring?)
10. Our Cluster > Heavy use of Spot instances to help keep cost down and performance high. > Currently implementing a hot vs. warm cluster our unstructured content. > All running on self-managed EC2 Nodes in Sydney. aaron.wallis@lexer.io - @d2kagw - http://lexer.io
11. aaron.wallis@lexer.io - @d2kagw - http://lexer.io Love & Hate
12. Hate > As ES matures there have been some breaking changes that have caused us headaches. > The logic behind ES shard allocation causes us pain as the cluster expands > Our topology is constantly changing > So sick of hearing about ELK Stacks! aaron.wallis@lexer.io - @d2kagw - http://lexer.io
13. Love > JSON, JSON, JSON > It’s hella fast > Elastic are maturing the product effectively > We’re able to train analysts to write raw queries > It’s becoming more commonplace aaron.wallis@lexer.io - @d2kagw - http://lexer.io
14. aaron.wallis@lexer.io - @d2kagw - http://lexer.io If you’re on the fence about Elasticsearch, don’t be.
15. aaron.wallis@lexer.io - @d2kagw - http://lexer.io Cheers! (Questions?)