Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Do You Know If Your Service Is Working Properly? A Guide To Being Paranoid.

7067ff85573929e5257aa9e9c1069de9?s=47 Eric Sigler
October 27, 2016

Do You Know If Your Service Is Working Properly? A Guide To Being Paranoid.

End-to-end monitoring, testing, and validation of an application or service can be helpful to both development and the business if done properly. It lets you answer the age old question "Is $APP up?" with confidence. Especially when individual service-owning teams are part of a larger whole, having a way to rapidly iterate and verify your changes is vital. This talk went over background & history on end-to-end testing efforts, then discuss best practices for care and feeding of such systems, and pitfalls to avoid, with references.


Eric Sigler

October 27, 2016


  1. @esigler Eric Sigler, Head of DevOps, PagerDuty “Do you know

    if your service is working properly?”
 A Guide To Being Paranoid
  2. Details changed to protect the innocent.

  3. @esigler wat? why? tips pitfalls

  4. @esigler Unit Testing Integration Testing Functional Testing Production Testing “Ad

  5. @esigler Production Environment External Functional Test Suite Specifically:

  6. @esigler Monitoring Testing ??? New Relic Keynote Dynatrace Jenkins TravisCI

  7. Why bother? @esigler

  8. None
  9. @esigler Monoliths SOAs Microservices Serverless services Artisanal hand crafted services

    etc, etc, etc.
  10. @esigler Production functional testing becomes a shared resource.

  11. @esigler Checks & Balances

  12. @esigler One Two Three Zero

  13. @esigler

  14. @esigler

  15. @esigler What to test for?

  16. Flaky tests are sVll a thing. @esigler

  17. @esigler Test failures will involve triage.

  18. @esigler Consider doing functional testing in production It can buffer

    your users from Conway’s Law It can speed up your developers Moderation in all things Beware of the usual testing failure modes Have clear agreement on ownership
  19. “The more they overthink the plumbing, the easier it is

    to stop up the drain.” @esigler
  20. @esigler Thank you! Questions?

  21. @esigler Appendix • https:/ /upload.wikimedia.org/wikipedia/commons/thumb/c/cc/Spectrum.svg/2000px-Spectrum.svg.png • http:/ /haacked.com/images/haacked_com/WindowsLiveWriter/ConwaysLaw_6DF4/organizational_charts_thumb.png • https:/