Broken Dreams and Shattered Promises

We simply cannot trust our guts, we need to make decisions based on empirical evidence. We need to make measurable improvements to our applications.

Whether we're improving performance, hunting a memory leak, or planning a new feature, it all becomes easier and more reliable when we have real data to build on top of.

In order to call ourselves engineers or scientists, we'd better start acting like real scientists & engineers.

Transcript

1. Broken Promises and Shattered Dreams No more gut driven development! Demand decision making based on empirical evidence!
2. Kenneth Kalmer Chief Rocket Scientist @ ValuationUP.com Utopian @ Zero-One.io @kennethkalmer github/kennethkalmer www.opensourcery.co.za kennethkalmer
3. #186 What We Actually Know About Software Development and Why We Believe It's True with Greg Wilson and Andreas Stefik
4. We’re Eternal Optimists We don’t handle errors We don’t instrument our code It works on my machine!
5. We trust our gut Hindsight is the only 20/20 vision With more experience we look better, but do we actually know better?
6. - Morris Kline, Mathematics: The Loss of Certainty “The most fertile source of insight is hindsight.”
7. How do we change that?
8. - Peter Drucker “What gets measured, gets managed.”
10. We measure! New Relic RPM & RUM Skylight Librato Zabbix Many more
11. Microservices We’re all building them, right? We’re all consuming them, right?
12. HTTP!
13. Your challenge Configure the network timeout for your HTTP requests
14. Exhibit A - GitHub API
15. Exhibit B.1 - Twitter API
16. Exhibit B.2 - Twitter API
18. Easy? No? Each service behaves completely different Within a service, each endpoint differs substantially
20. Ruby has us covered! 1 require 'benchmark' 2 3 time = Benchmark.benchmark do 4 benchmark_lovingly 5 end 6 7 puts time 1 require 'benchmark' 2 3 time = Benchmark.benchmark do 4 benchmark_the_service_lovingly 5 end 6 7 puts time #=> 0.010000 0.000000 0.010000 (0.003298)
21. Falls a bit short Works great with p-driven development Gnarly to parse Tricky to scale
22. ActiveSupport::Notification Secret weapon lying dormant in our apps Separates instrumentation from publishing/reporting
23. Instrument 1 AS::Notifications.instrument('foo-service', {}) do 2 fooish_things 3 end
24. Subscribe 1 AS::Notifications.subscribe 'foo-service' do |*args| 2 event = AS::Notifications::Event.new(*args) 3 event.duration #=> 10 (in miliseconds) 4 end
25. Instrumenting Faraday 1 conn = Faraday.new( host: 'foo-service' ) do |c| 2 c.use :instrumentation 3 4 c.adapter Faraday.default_adapter 5 end
26. Instrumenting Faraday 1 AS::Notifications.subscribe('request.faraday') do |*args| 2 event = ActiveSupport::Notifications::Events.new(*args) 3 url = event.payload[:url] 4 http_method = event.payload[:method].to_s.upcase 5 duration = event.duration 6 7 Rails.logger.debug '[%s] %s %s (%.3f s)' % [ 8 url.host, http_method, url.request_uri, duration 9 ] 10 end
28. How do we visualise/report?
29. InfluxDB Open source, distributed, time series databases with no external dependencies Created by Paul Dix (amongst others)
30. Grafana An open source, feature rich metrics dashboard and graph editor for InfluxDB, Graphite & OpenTSDB Client side application
34. Pesky dependencies! Easy to use docker image with InfluxDB & Grafana docker run [lotsa options] tutum/influxdb InfluxDB has binaries for brew, Debians & RedHats Grafana has no dependencies, client-side app
35. Easy to plug in! Drop influxdb-rails into your project Out the box metrics for controller, view & db runtimes Configured client ready use with ActiveSupport::Notifications InfluxDB client supports async write operations
36. Already using statsd? There is an InfluxDB backend available Simple, fast UDP, so the client is resilient to failures Only a transport, needs to ship the stats to…
37. Lo-Fi RPM Track what really matters to you and your team Virtually unlimited data retention Long tail of information to establish solid baselines
38. Hi-Fi ain’t always best More intrusive tools with higher fidelity are most useful for local debugging Huge variety of data can lead to analysis paralysis
39. Share the data Lo-Fi is easier to share Don’t assume the provider of a service measures the same things you do Provide them with your consistent view of their service, watch them improve Inspire them to science too
40. Engineering creeping in Aaron’s Adequate Record work Charles’ JRuby 9000 work Just use double quoted string - viget.com blog Experimentally verified - “Why client-side rendering is wrong” - onebigfluke.com [YOUR WORK HERE]
42. Stop gut driven development
43. Demand evidence
44. Think critically
45. Thanks!