Elastic as a Service on C2S and Other Platforms at Decision Lab

An in-depth look into DecisionLab's Elastic as Service offering available on C2S and other secure networks. In this talk, Nathan and Drew highlight the lessons learned and best practices (including a concentration on well oiled automation) in setting up and deploying their managed Elastic service.

Transcript

1. Nathan Necaise and Drew Malone Oct 6, 2015 1   Elastic as a Service on C2S and Other Platforms
2. In the Beginning... 2
3. We Need a Bigger Box § Lots of data § More data coming in every day § We need data analysis! 3
4. The Search § Try out a technology, throw out what doesn't work. § Constantly setting up and tearing down technology stacks. § Very time consuming. § Found not available with FedRAMP 4
5. This is Great!.... But it Takes Work § Problem - Building/Running clusters is *hard* o Learning Curve o Error Prone o Slow o Scaling o Security o ATO o *BORING* (Eventually, it's a solved problem, we're just doing it again) We need to automate this... 5
6. Do the Hard Things Often § Solution- Relentless automation. o Everything under configuration management o Scaling is just repeating the automation Let the machine do the boring work so we can solve the interesting problems. We bore easily. If we're doing something more than once, we automate it so we don’t get bored. We *hate* doing things over and over again. 6
7. Reap the Rewards § Automation: o Consistency (predictable behavior) o Lets us focus on solving problems (not fixing servers) o Assurance of security (automated configs) o Ensures we meet security (ATO for free!) 7
8. It's a Data Party! § Results: o More data processed, faster. o Answers/Insights *now* o Makes it easy to solve multiple problems with the same system o Frees up our time to work on the interesting problems Like productizing.... 8
9. And Everyone is Invited! § Everyone else has this same problem o They also have tons of data o They also need answers o They also... don't want to manage a cluster What if we could productize our work? We can help others gain insight into their own data. We can't analyze everyone's data, but we *can* give them the tools. 9
10. One Ticket Please.... § Problem: People need to know what toolset fits their needs. o Each trial involves work (setup, test, evaluate) o Work means time (weeks, months...) o Most trials also involve money (a lot of it) What if you could try a solution *today*, for pennies? 10
11. I'll Take the Red Pill §  Solution: Elastic as a Service o Fill out a simple form, get an ES cluster o Your cluster is built-to-spec immediately o You only pay for your infrastructure o Shield compatible Why? 11
12. Why ESaaS? §  Get started now §  Answer questions in minutes §  Other "turn-key" solutions don't offer trial periods §  Intensely low cost (only pay for infrastructure) §  Shield Compatible Need more? 12
13. Money Talks... •  What’s better: §  2 people each on 10 projects building and architecting solutions and maintaining infrastructure for data science problems = 20 people, fixed at 10 projects o From scratch each time o Sustainment and maintenance tail o Larger labor requirement §  2-4 people supporting 10 projects with a service that provides the capability to solve their data science problems = 2-4 people people, scales to many problems o Data to insight faster o Focus on problem domain and not technology o Built in expertise, security and best practices 13
14. Coming to You Live from Elastic{ON}! 14
15. The Tech •  Powered by: §  SaltStack §  Elasticsearch, Logstash, Kibana §  Node.js §  Custom code and contributions back to Open Source •  Runs on: §  OpenStack §  AWS §  DigitalOcean §  Bare Metal §  More… 15
16. Where to From Here? §  More cloud providers §  More data sources 16