Helping Developers Monitor their own Applications

This is my talk from DevOpsDays Minneapolis 2015. The video can be found on YouTube: https://www.youtube.com/watch?v=Swa31wqw5MI

Transcript

1. Helping Developers Monitor their own Applications Luke Francl
2. I'm a developer
8. Operations?!?
10. (via RightScale)
12. photo: Daniel Catt JOHN ALLSPAW
17. What is DevOps?
20. Luser Attitude Readjustment Tool
21. “Do you think DevOps is just for ops?”
23. Application MongoDB API requests
24. Application MongoDB API requests
25. Application MongoDB API requests
26. Application MongoDB API requests
27. Application MongoDB API requests
28. Application MongoDB Uh-oh... API requests
29. Application MongoDB Router Customer A Customer B API requests
30. Don’t hate the database, hate the game
31. Input validation Create journal Raw JSON Basic input validation only. No MongoDB queries. Enqueue
32. How do we make sure this keeps working?
33. MONITORING
35. module'SwiftypeMonitoring ''class'Check ''''# ''''#'Helpers'for'defining'default'thresholds ''''# ''''def'self.warning_option(desc,'default);'end ''''def'self.critical_option(desc,'default);'end ''''#'Implement'the'`run`'method'in'a'subclass'to'run'the'check ''''def'run ''''end ''''# ''''#'Helpers'from'Sensu::Plugin::CLI'to'indicate'alert'status ''''# ''''def'ok;'end ''''def'warning;'end ''''def'critical;'end ''end end
36. class'FoobarCheck'<'SwiftypeMonitoring::Check ''#'Check'thresholds ''warning_option('Warning'threshold'for'Foobar','3600) ''critical_option('Critical'threshold'for'Foobar','7200) ''def'run ''''foobar'='get_some_metric ''''if'foobar'>'config[:critical] ''''''critical("Foobar'is'too'large'(#{foobar}).'"'+ '''''''''''''''"Limit'is'#{config[:critical]}") ''''end ''''if'foobar'>'config[:warning] ''''''warning("Foobar'is'too'large'(#{foobar}).'"'+ ''''''''''''''"Limit'is'#{config[:critical]}") ''''end ''''ok("Foobar'(#{foobar})'is'within'acceptable'limits") ''end end
37. def'run ''#'Establish'MySQL'Connection ''mysql'='mysql_connection('core') ''#'Get'the'oldest'document'in'the'journal ''sql'='"SELECT'id,'TIMESTAMPDIFF(SECOND,'created_at,'NOW())'AS'lag'"'+ ''''''''"FROM'document_creation_journals'ORDER'BY'created_at'ASC'LIMIT'1" ''results'='mysql.query(sql) ''#'Check'if'the'journal'is'empty ''ok("Document'creation'journal'is'empty")'if'results.count'=='0 ''#'Get'the'record'from'MySQL ''oldest_record'='results.first ''creation_lag'='oldest_record['lag'].to_i ''oldest_journal_id'='oldest_record['id'] ''#'Omitted:'ok/warn/critical'checks end
38. def'fetch_lag_info(args) ''table'='args.fetch(:table) ''id_columns'='args.fetch(:id_columns) ''time_column'='args[:timestamp_column]'||''created_at' ''raise'"id_columns'must'not'be'empty"'if'id_columns.empty? ''sql'='"SELECT'#{id_columns.join(','')},'"'+ ''''''''"TIMESTAMPDIFF(SECOND,'#{time_column},'NOW())'AS'lag'"'+' ''''''''"FROM'#{table}'ORDER'BY'#{time_column}'ASC'LIMIT'1" ''results'='query(sql) ''return'nil'if'results.count'=='0 ''oldest_row'='results.first ''lag_info'='OpenStruct.new(:lag'=>'oldest_row['lag'].to_i) ''id_columns.each'do'|id_column| ''''lag_info.send("#{id_column}=".to_sym,'oldest_row[id_column]) ''end ''lag_info end
39. def'run ''#'Establish'MySQL'Connection ''mysql'='mysql_connection('core') ''lag_info'='mysql.fetch_lag_info(:table'=>''document_creation_journals',' '''''''''''''''''''''''''''''''''':id_columns'=>'['id']) ''ok("document_creation_journals'table'is'empty")'if'lag_info.nil? ''#'Check'the'lag ''if'lag_info.lag'>'config[:critical] ''''critical("Document'creation'lag'is'too'large'(#{lag_info.lag}'sec") ''end ''if'lag_info.lag'>'config[:warning] ''''warning("Document'creation'lag'is'too'large'(#{lag_info.lag}'sec") ''end ''#'Ok,'we're'good ''ok("Document'creation'lag'is'#{lag_info.lag}'sec") end
40. Monitoring is addictive
41. github.com/swiftype/swiftypeimonitoring
42. Application Infrastructure Development Production – after Donnie Berkholz Developers Operations Application Infrastructure
43. Application Infrastructure Developers Operations – after Donnie Berkholz Development Production Application Infrastructure
44. DevOps needs developers