Share
Embed
What's happening in production?
Lightning talk delivered at Agilasverige (Agile Sweden) on metrics and that you as a developer should be able to know what's happening in your production environment in real time
Approx 10 minutes.
Mårten Gustafson
May 23, 2011
Transcript
61 slides
-
Hi! Monday, May 23, 2011 My name is Mårten, I
work as a software developer But I usually also have one foot - sometimes a whole leg - in our infrastructure -
Monday, May 23, 2011 I work here. We provide a
web site, that’s our offering. Our core. -
Monday, May 23, 2011 Our site is, amongst other things,
built from a slew of Java services We have multiple services on each server -
Monday, May 23, 2011 We like to be availbale, thus
we have multiple servers Same service, on multiple servers Multiple services on same server -
https://github.com /codahale/metrics ...for java & scala use: Monday, May 23,
2011 ...or at least look at this to get inspired -
)","counter":0},{"name":"AutocompleteService","counter":0},{"name":"Autocomplet counter":0}],"monitored_services":[{"name":"AutocompleteService","status":"Ok"} tion_status":"Ok","counter_tot":0,"request_counters":[{"name":"PointConverter", r-deprecated","counter":0}],"monitored_services":[{"name":"PointConverter","sta ion_status":"Ok","counter_tot":1,"request_counters":[{"name":"RouteService-iPhon Service","counter":1}],"monitored_services":[{"name":"RouteService","status":"Ok ication_status":"Ok","counter_tot":19,"request_counters":[{"name":"ImageService "name":"ImageService","status":"Ok"}]},{"name":"locality","application_status": calityResolverService-cityName","counter":0},{"name":"LocalityResolverService-c e-countyName","counter":0}],"monitored_services":[{"name":"LocalityResolverServ
ion_status":"Ok","counter_tot":0,"request_counters":[{"name":"SmsService:messag Service","counter":0}],"monitored_services":[{"name":"SmsService","status":"Ok"} n_status":"Ok","counter_tot":0,"request_counters":[{"name":"WeatherService-fore observation","counter":0},{"name":"WeatherService-forecast","counter":0}],"monit tatus":"Ok"}]},{"name":"autocomplete","application_status":"Ok","counter_tot":3, Service:route","counter":0},{"name":"AutocompleteService","counter":3}],"monitor "status":"Ok"}]},{"name":"poi-service","application_status":"Ok","counter_tot": ks","counter":47},{"name":"PoiService:metadata","counter":8},{"name":"PoiService "name":"PoiService","status":"Ok"}]},{"name":"JvmMonitor","application_status":" nters":[{"name":"memory.allocated","counter":4079},{"name":"memory.free","counte er":4079},{"name":"memory.used","counter":1329},{"name":"cpu","counter":8}],"mo atus":"Ok"}]},{"name":"weather.v2","application_status":"Ok","counter_tot":0,"re e-hourbyhour-forecast","counter":0},{"name":"WeatherService-rectangle-forecasts" tails-forecast","counter":0},{"name":"WeatherService-total","counter":0},{"name" nter":0},{"name":"WeatherService-longrange-forecast","counter":0},{"name":"Weath ,"counter":0}],"monitored_services":[{"name":"WeatherService","status":"Ok"}]}]} Monday, May 23, 2011 JSON, not always funny to look at The really interesting things are small tidbits of information “hidden” in this blob of data -
)","counter":0},{"name":"AutocompleteService","counter":0},{"name":"Autocomplet counter":0}],"monitored_services":[{"name":"AutocompleteService","status":"Ok"} tion_status":"Ok","counter_tot":0,"request_counters":[{"name":"PointConverter", r-deprecated","counter":0}],"monitored_services":[{"name":"PointConverter","sta ion_status":"Ok","counter_tot":1,"request_counters":[{"name":"RouteService-iPhon Service","counter":1}],"monitored_services":[{"name":"RouteService","status":"Ok ication_status":"Ok","counter_tot":19,"request_counters":[{"name":"ImageService "name":"ImageService","status":"Ok"}]},{"name":"locality","application_status": calityResolverService-cityName","counter":0},{"name":"LocalityResolverService-c e-countyName","counter":0}],"monitored_services":[{"name":"LocalityResolverServ
ion_status":"Ok","counter_tot":0,"request_counters":[{"name":"SmsService:messag Service","counter":0}],"monitored_services":[{"name":"SmsService","status":"Ok"} n_status":"Ok","counter_tot":0,"request_counters":[{"name":"WeatherService-fore observation","counter":0},{"name":"WeatherService-forecast","counter":0}],"monit s":"Ok"}]},{"name":"autocomplete","application_status":"ok","counter_tot Service:route","counter":0},{"name":"AutocompleteService","counter":3}],"monitor "status":"Ok"}]},{"name":"poi-service","application_status":"Ok","counter_tot": ks","counter":47},{"name":"PoiService:metadata","counter":8},{"name":"PoiService "name":"PoiService","status":"Ok"}]},{"name":"JvmMonitor","application_status":" nters":[{"name":"memory.allocated","counter":4079},{"name":"memory.free","counte er":4079},{"name":"memory.used","counter":1329},{"name":"cpu","counter":8}],"mo atus":"Ok"}]},{"name":"weather.v2","application_status":"Ok","counter_tot":0,"re e-hourbyhour-forecast","counter":0},{"name":"WeatherService-rectangle-forecasts" tails-forecast","counter":0},{"name":"WeatherService-total","counter":0},{"name" nter":0},{"name":"WeatherService-longrange-forecast","counter":0},{"name":"Weath ,"counter":0}],"monitored_services":[{"name":"WeatherService","status":"Ok"}]}]} Monday, May 23, 2011 Is it ok? -
)","counter":0},{"name":"AutocompleteService","counter":0},{"name":"Autocomplet counter":0}],"monitored_services":[{"name":"AutocompleteService","status":"Ok"} tion_status":"Ok","counter_tot":0,"request_counters":[{"name":"PointConverter", r-deprecated","counter":0}],"monitored_services":[{"name":"PointConverter","sta ion_status":"Ok","counter_tot":1,"request_counters":[{"name":"RouteService-iPhon Service","counter":1}],"monitored_services":[{"name":"RouteService","status":"Ok ication_status":"Ok","counter_tot":19,"request_counters":[{"name":"ImageService "name":"ImageService","status":"Ok"}]},{"name":"locality","application_status": calityResolverService-cityName","counter":0},{"name":"LocalityResolverService-c e-countyName","counter":0}],"monitored_services":[{"name":"LocalityResolverServ
ion_status":"Ok","counter_tot":0,"request_counters":[{"name":"SmsService:messag Service","counter":0}],"monitored_services":[{"name":"SmsService","status":"Ok"} n_status":"Ok","counter_tot":0,"request_counters":[{"name":"WeatherService-fore observation","counter":0},{"name":"WeatherService-forecast","counter":0}],"monit tatus":"Ok"}]},{"name":"autocomplete","application_status":"Ok","counter_tot":3, Service:route","counter":0},{"name":"AutocompleteService","counter":3}],"monitor "status":"Ok"}]},{"name":"poi-service","application_status":"Ok","counter_tot": ks","counter":47},{"name":"PoiService:metadata","counter":8},{"name":"PoiService "name":"PoiService","status":"Ok"}]},{"name":"JvmMonitor","application_status":" nters":[{"name":"memory.allocated","counter":4079},{"name":"memory.free","counte 079},{"name":"memory.used","counter": 1329},{"name":"cpu","counter":8} atus":"Ok"}]},{"name":"weather.v2","application_status":"Ok","counter_tot":0,"re e-hourbyhour-forecast","counter":0},{"name":"WeatherService-rectangle-forecasts" tails-forecast","counter":0},{"name":"WeatherService-total","counter":0},{"name" nter":0},{"name":"WeatherService-longrange-forecast","counter":0},{"name":"Weath ,"counter":0}],"monitored_services":[{"name":"WeatherService","status":"Ok"}]}]} Monday, May 23, 2011 How much memory do we consume? -
)","counter":0},{"name":"AutocompleteService","counter":0},{"name":"Autocomplet counter":0}],"monitored_services":[{"name":"AutocompleteService","status":"Ok"} tion_status":"Ok","counter_tot":0,"request_counters":[{"name":"PointConverter", r-deprecated","counter":0}],"monitored_services":[{"name":"PointConverter","sta ion_status":"Ok","counter_tot":1,"request_counters":[{"name":"RouteService-iPhon Service","counter":1}],"monitored_services":[{"name":"RouteService","status":"Ok tion_status":"Ok","counter_tot": 19,"request_counters":[{"name":"ImageServ "name":"ImageService","status":"Ok"}]},{"name":"locality","application_status": calityResolverService-cityName","counter":0},{"name":"LocalityResolverService-c
e-countyName","counter":0}],"monitored_services":[{"name":"LocalityResolverServ ion_status":"Ok","counter_tot":0,"request_counters":[{"name":"SmsService:messag Service","counter":0}],"monitored_services":[{"name":"SmsService","status":"Ok"} n_status":"Ok","counter_tot":0,"request_counters":[{"name":"WeatherService-fore observation","counter":0},{"name":"WeatherService-forecast","counter":0}],"monit tatus":"Ok"}]},{"name":"autocomplete","application_status":"Ok","counter_tot":3, Service:route","counter":0},{"name":"AutocompleteService","counter":3}],"monitor "status":"Ok"}]},{"name":"poi-service","application_status":"Ok","counter_tot": ks","counter":47},{"name":"PoiService:metadata","counter":8},{"name":"PoiService "name":"PoiService","status":"Ok"}]},{"name":"JvmMonitor","application_status":" nters":[{"name":"memory.allocated","counter":4079},{"name":"memory.free","counte er":4079},{"name":"memory.used","counter":1329},{"name":"cpu","counter":8}],"mo atus":"Ok"}]},{"name":"weather.v2","application_status":"Ok","counter_tot":0,"re e-hourbyhour-forecast","counter":0},{"name":"WeatherService-rectangle-forecasts" tails-forecast","counter":0},{"name":"WeatherService-total","counter":0},{"name" nter":0},{"name":"WeatherService-longrange-forecast","counter":0},{"name":"Weath ,"counter":0}],"monitored_services":[{"name":"WeatherService","status":"Ok"}]}]} Monday, May 23, 2011 How many requests have we served? -
Monday, May 23, 2011 Visualization makes it easy to distinguish
worse from better -
Monday, May 23, 2011 Visualization makes it easy to distinguish
worse from better -
Monday, May 23, 2011 Visualization makes it easy to distinguish
worse from better -
think build server... Monday, May 23, 2011 Your project /
team probably have a build server monitor, which is a good thing. But how much “dashboard estate” do you spend on production - where your code actually (hopefully) delivers value? -
Monday, May 23, 2011 The dashboard where I work -
all servers, most services