Metrics Matter

An overview of Pipedream and Snowbeast, our internal metrics services at Wistia.

Transcript

1. I’m @brendan.
6. Breadroutes S3 Primes Edgecast CDNetworks CDNs Bakery (Origin) Load Balancer Temporary Storage Permanent Storage DNS Dyn
7. Things go wrong
8. Things go wrong
9. Support Ticket
10. Anything on fire?
11. Try to reproduce. Cannot reproduce.
12. Clear your cookies?
13. Long tail of problems Problems we can see Problems we can reproduce Problems we can measure Problems we can’t see Problems we can’t reproduce Problems we can’t measure
14. Confession
15. Metrics Matter
16. 1. HTTP POST ingest 2. Realtime subscription 3. Store all data forever 4. Scale out
17. Pipedream
18. curl -d "some-message" \ https://pipedream/pub?topic=my-topic
19. Collector curl -d "some-message" \ https://pipedream/pub?topic=my-topic
20. NSQ Collector curl -d "some-message" \ https://pipedream/pub?topic=my-topic
21. NSQ Collector Consumer #topic-1 Consumer #topic-2 curl -d "some-message" \ https://pipedream/pub?topic=my-topic
22. NSQ Collector Archiver S3 Consumer #topic-1 Consumer #topic-2 curl -d "some-message" \ https://pipedream/pub?topic=my-topic
23. Now what?
24. Snowbeast
25. Metrics.count('player/play')
31. 2 billion messages in the last month
32. Redis Sinatra Processor Angular & D3 NSQ #metrics Pipedream Snowbeast
33. Support Ticket
37. Clear your cookies?
38. Clear your cookies?
39. $ wget https://pipedream/last_hour.log.gz $ gunzip last_hour.log.gz $ grep customer last_hour.log > customer.log $ ruby tools/show-loading.rb customer.log
43. Default to Flash
59. Metrics Matter
60. 1. Own your data. 2. Keep it simple. 3. Move that line!
61. Long tail of problems
62. Long tail of problems
63. http:/ /bit.ly/wistia-video-puzzle
64. Thank you!