Tasty, Tasty Dogfood: How We Use the Elastic Stack at Elastic

Learn how Elastic uses the Elastic Stack throughout the company — from how and where its data stores are built to best practices for production clusters, as well as what is built on premise and when it leverages its Cloud service.

Transcript

1. ‹#› Morgan Goeller Solution Architect Tasty, Tasty Dogfood: How We Use the Elastic Stack at Elastic Tyler Langlois Infrastructure Engineer
2. Prelude: Why Dogfood? 2 • We run into bugs before you have to • Discover needs we can work towards before they’re more widely needed • Some bugs surface through use, not tests Test Monkeys • Insights into machine data has been extremely helpful • Simplifies application datastores • Makes infra scaling much easier It’s Useful! • Keep skills sharp • Uncover tool needs • Learn what pieces of software need usability improvements Be a User
3. Overview 3 Application Datastores Analytics and Visualization for a Worldwide Team Lessons Learned Infrastructure Monitoring and Tracking
4. ‹#› Application Datastores 4 Rails + Elasticsearch + Cloud
5. Application Datastore: Licensing Portal Commercial Licensing Tool Built on Elasticsearch 5 Elastic Cloud cluster (basic snapshotting, access control, etc.) 3 Ruby on Rails with Elasticsearch persistence backend Web application
6. Application Datastore: Implementation 6 Access Control Backups Lifecycle • Cloud + Shield • High-level Rails abstractions make this transparent to dev • Elastic Cloud snapshots • Additional snapshot repositories • Extra: Rails json dumps • Standard Rails app practices • Gemfiles, native models, etc.
7. Application Datastore: Licensing Portal Benefits & Caveats 7 Search is a first-class citizen Highly-available backend Dovetails with Kibana, Shield, etc. Schema changes require reindexing Subtle behavior when mappings go awry
8. ‹#› Puppet @ Elastic 8 Decentralize Recentralize
9. Puppet reporting 9
10. ‹#› Facter knows useful things
11. ‹#› But it’s a lot
12. ‹#› From every machine in your fleet
13. Factbeat - Fact shipper made with libbeat 13 https://github.com/jarpy/factbeat
14. ‹#›
15. Custom facts are trivial 15 # echo 'color=green' > $FACT_DIR/color.txt
16. Facts are now time-series data Visualize anything with a one-liner on each host
17. ‹#› Solutions Architecture 17 Showing the world
18. ‹#› We are Prestige worldwide.
20. Our Infrastructure Evolution
21. Want to see Groot? Look at any demo booth at ElasticON.
22. ‹#› Infrastructure Monitoring: Logs & Metrics CI, web serving, and machine logs
23. Lots of Pipelines 23
24. ‹#› 24
25. ‹#› Lessons Learned 25
26. 26 Elasticsearch As a Service • Significantly eases operational complexity • Lower barrier for upgrades/backups • Baked-in usability across teams • Additional access (extra snapshot repositories, etc.) useful for operational customization • Be mindful of latency • Take time with permissions An Infrastructure Perspective
27. 27 From a Devops Perspective • Design with maintenance/failover in mind (aliases, node types, etc.) • Design with fault-tolerance in mind (backpressure, etc.) • Little tools make things easier (use the _cat apis, plugins, etc.) • People will love and use information! Give them dashboards and data
28. ‹#› Questions? We’ll also be at the AMA booth
29. ‹#› Please attribute Elastic with a link to elastic.co Except where otherwise noted, this work is licensed under http://creativecommons.org/licenses/by-nd/4.0/ Creative Commons and the double C in a circle are registered trademarks of Creative Commons in the United States and other countries. Third party marks and brands are the property of their respective holders. 29