Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PDX 2017 - Roy Rapoport

PDX 2017 - Roy Rapoport

Monitorama

May 22, 2017
Tweet

More Decks by Monitorama

Other Decks in Technology

Transcript

  1. undergoes the ordeal Damn, this won’t scale. Also, we probably

    shouldn’t have coded a JS interpreter in Erlang.
  2. 6H 3D 18D Publish MR Query S3 Anatomy of an

    Atlas Deployment Group (ADG) m etrics
  3. Server Proxy us-west-2 Server Proxy us-east-1 eu-west-1 Server Proxy ADG

    ADG iep ADG
 iep ADG
 iep ADG
 Critical Critical Critical
  4. “This morning around 0504 US/Pacific, the active primary region hosting

    Atlas metrics data began experiencing issues and a number of alerts were triggered across the board.“ February 9, 2017
  5. What’s a Metric? • name = cpu • nf.region =

    us-east-1 • nf.zone = us-east-1a • nf.app = api • nf.cluster = api-main • nf.asg = api-main-v103 • nf.node = i-103efbdd • device.operationalName = roku • isSupplementalVideo = false • OnComplete = operation • …
  6. Fun Metric Dimensions
 A True Story • IP address (~

    232) • Account ID (~109) • Customer ID (~109) • Execution time, ms (~30K)
  7. ?

  8. Attributions • https://www.flickr.com/photos/kalleboo/2240042641/ • http://imgur.com/gallery/QCrR73q • https://www.flickr.com/photos/35028096@N04/6119259650 • https://www.flickr.com/photos/86639298@N02/9420223184 •

    http://paristext2015.com/2015/06/1-5-degrees-celsius-or-2-degrees-maybe-its-turtles-all- the-way-down/ • https://www.flickr.com/photos/60969081@N00/8292545493 • By Photo courtesy of National Nuclear Security Administration / Nevada Site Office - This image is available from the National Nuclear Security Administration Nevada Site Office Photo Library under number XX-34.This tag does not indicate the copyright status of the attached work. A normal copyright tag is still required., Public Domain, https:// commons.wikimedia.org/w/index.php?curid=190949 • http://www.animal-space.net/2010/12/16-funny-pictures-of-shocked-cats.html • https://www.flickr.com/photos/10459174@N03/34563622796/