Building Metrics In-House with Keen IO

Kate Heddleston shares how she built analytics in-house with Keen IO's APIs and analytics infrastructure.

Transcript

1. Building Metrics In- House Using Keen IO
2. Kate Heddleston kateheddleston.com @heddle317
3. Building Metrics In-House Why use Keen IO? Keen IO’s API
4. What does it look like to build metrics in-house?
5. Infrastructure
6. Skylight Coordinator Kafka Storm Spout Bolt Cassandra Java
7. -Tom Dale and Yehuda Katz “One year of research. Six months to rewrite.”
8. Skylight Coordinator Kafka Storm Spout Bolt Cassandra Java
9. Keen Testing Monitoring/ validation Data fail- overs and uptime Separate infrastructure
10. –New Relic Website Marketing Person “New Relic Insights is powered by a highly distributed cloud-hosted event database with an innovative architecture that does not require indexing. The database runs on a super-cluster built to support our thousands of customers — giving you access to a database that can query your big data in seconds.” New Relic
11. New Relic Challenges Database tuning Optimizing for reads vs. writes Determining which metrics can be aggregated.
12. Building metrics infrastructure is really hard…
13. UX
14. Metrics can easily end up like…
15. …or…
16. …or…
17. Metrics don’t mean anything unless they are answering specific questions.
18. Data Visualization Telling meaningful stories with your data.
19. Building Metrics UX is really hard…
20. …how many hard problems do you want to solve at once?
21. Why use Keen IO?
22. 1. Fast prototyping. 2. They deal with infrastructure while you deal with creating customer value. 3. You don’t have to hire as many software engineers.
23. What does Keen’s API look like?
24. Event Collection Timeframe and interval Group By Filter Analyses
26. Event Collection Timeframe and interval Group By Filter Analyses
28. Event Collection Timeframe and interval Group By Filter Analyses
30. Event Collection Timeframe and interval Group By Filter Analyses
32. Event Collection Timeframe and interval Group By Filter Analyses
34. Event Collection Timeframe and interval Group By Filter Analyses
36. Conlusion
37. Solve less hard problems.
38. Use Keen IO
39. Questions Kate Heddleston kateheddleston.com @heddle317
40. Resources ✤ Skylight - Tom Dale and Yehuda Katz ✤ http://www.confreaks.com/videos/3394-railsconf-how-to-build-a-smart-profiler-for-rails ✤ Keen Infrastructure Stack ✤ http://blog.leanstack.io/keen-io-tech-stack/ ✤ Data Visualization Experts ✤ http://www.information-management.com/resource-center/?id=10024000 ✤ New Relic Infrastructure ✤ http://newrelic.com/insights/technology#the-database ✤ http://highscalability.com/blog/2011/7/18/new-relic-architecture-collecting-20-billion-metrics- a-day.html (2011) ✤ Keen API Docs ✤ https://keen.io/docs/data-analysis/