is strictly prohibited Elasticsearch in 10 seconds • Schema-free, REST & JSON based document store • Distributed and horizontally scalable • Open Source: Apache License 2.0 • Zero configuration • Written in Java, extensible
is strictly prohibited Feature summary • Fully-featured search Relevance-ranked text search Scalable search High-performance geo, temporal, numeric range and key lookup Highlighting Support for complex document types (nested structures) * Spelling suggestions Powerful query DSL * “Standing” queries * Real-time results * Extensible via plugins * ! • Powerful faceting/analysis Summarise large sets by any combinations of time, geo, category and more. * “Kibana” visualisation tool * ! • Management Simple and robust deployments * REST APIs for handling all aspects of administration/ monitoring * “Marvel” console for monitoring and administering clusters * Special features to manage the life cycle of content * • Integration Hadoop (MapRed,Hive, Pig, Cascading..)* Client libraries (Python, Java, Ruby, javascript…) Data connectors (Twitter, JMS…) Logstash ETL framework * • Support Development and Production support with tiered levels Support staff are the core developers of the product * * Features we see as differentiators
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Unstructured search
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Aggregation to find languages
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Structured search
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Enrichment
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Pagination
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Suggestions
is strictly prohibited 2 minutes to live $ wget https://download.elasticsearch.org/... $ tar -xf elasticsearch-1.3.2.tar.gz $ ./elasticsearch-1.3.2/bin/elasticsearch ... [2014-04-16 14:53:11,508][INFO ][node] [Scanner] started ... Also puppet modules and RPM/DEB
is strictly prohibited Basic terms • Index Logical collection of data; might be time based Analogous to a database • Sharding Split logical data over several machines Write scalability Control data flows • Replication Read scalability Removing SPOF
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited What is data? • Whatever provides value for your business ! • Domain data Internal: Orders, products External: Social media streams, email • Application data Log files Metrics
is strictly prohibited Ecosystem • Plugins Many third party plugins available • Clients for many languages Ruby, python, php, perl, javascript, (.NET coming) Scala, clojure, go • Kibana • Logstash • Hadoop integration
is strictly prohibited Monitor your cluster with Marvel • Point in time views are a start • Marvel shows historical trends • Visualize cluster behavior, act before problems ! • Free for development, $500/year for up to 5 nodes
is strictly prohibited Logstash in 10 seconds • Managing events and logs • Collect, parse, enrich, store data • Modular: many, many inputs and outputs • Apache License 2.0 • Ruby app (JRuby) • Part of Elasticsearch family
is strictly prohibited What is a log? • Time-based data • This data is everywhere! Server logs Twitter stream Financial transactions Metric / monitoring data ... • Log all things
is strictly prohibited Why collect & centralize logs? • Access log files without system access • Shell scripting: Too limited or slow • Using unique ids for errors, aggregate it across your stack • Reporting (everyone can create his/her own report) • Tip: Unify your data to make it easily searchable
is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Visualize with Kibana
is strictly prohibited Kibana in 10 seconds • Visualize data in Elasticsearch • See real-time updates to the data • Build custom charts and dashboards • Apache License 2.0 • Runs in browser (Chrome, FF, IE, Safari) • Part of Elasticsearch family
is strictly prohibited More info • Github: https://github.com/elasticsearch Code, issues there • Mailing lists Google groups, logstash-users and elasticsearch • IRC channels #logstash and #elasticsearch on freenode • We’re hiring! [email protected]