Puneet Khanduri & Prashant Khanduri

16b6c87229eaf58768d25ed7b2bbbf52?s=47 CodeFest
June 25, 2018
230

Puneet Khanduri & Prashant Khanduri

16b6c87229eaf58768d25ed7b2bbbf52?s=128

CodeFest

June 25, 2018
Tweet

Transcript

  1. 2.

    The Problem “I just refactored a critical part of my

    service. How do I know I didn’t break anything?” - Every Backend Developer diffy.ai
  2. 3.

    The Other Problem “They just refactored a critical part of

    my service. How do I know they didn’t break anything?” - Every Site Reliability Engineer diffy.ai
  3. 4.

    The Tipping Point “Does my new microservice behave the same

    as my monolith?” - Twitter in 2014 - Everyone trying to break a monolith diffy.ai
  4. 5.

    Tier#0 - Unit Tests 5 independent code paths => 5

    tests = 100% coverage => 1 test = 20% coverage diffy.ai
  5. 6.

    Tier#1 - Component Tests 5^6 independent code paths => ~15k

    tests = 100% coverage => 1 test = ~0.06% coverage diffy.ai
  6. 7.

    Tier#2 - Integration Tests A request path has 4 services,

    6 methods/service, 5 paths/method => 5^(4x6) independent code paths => ~60 Quadrillion tests = 100% coverage => 1 test = ~0% coverage diffy.ai
  7. 9.

    Diffy Approach - more coverage for free Free Test Inputs

    • Sample production traffic • Or whatever traffic source you prefer Free Assertions • Use “known good” versions of your code to generate assertions Note: You still provide the environment (mocked or staged or production) Currently, most customers use a combination of staged and production environments. diffy.ai
  8. 10.

    but … what about noise? • server generated timestamps •

    random number generators • downstream non-determinism • race conditions diffy.ai
  9. 11.

    Diffy Topology diffy secondary candidate primary raw differences non-deterministic noise

    filtered differences production cluster staging environment production environment diffy.ai
  10. 13.

    Automation Compare latest in master against last deploy to production

    • Automatically deploy master as candidate • Automatically deploy last prod tag as primary and secondary Reporting • Diffy emails report with highlighted critical endpoints and fields • Sample requests and responses available for further analysis diffy.ai
  11. 14.

    First version for personal use Generic Thrift version for Twitter

    HTTP support for Twitter Open Source! Community traction Diffy - the company - diffy.ai … the story so far diffy.ai