Need for Speed (GTAC 2016)

1f686da361195e15bb4e478397a4fc8f?s=47 emanuil
November 16, 2016

Need for Speed (GTAC 2016)

This is the shorter version of the presentation given at VelocityConf Amsterdam 2016

1f686da361195e15bb4e478397a4fc8f?s=128

emanuil

November 16, 2016
Tweet

Transcript

  1. NEED FOR SPEED accelerate tests from 3 hours to 3

    minutes emo@komfo.com
  2. 3 hours 3 minutes 600 API tests

  3. Before After The 3 Minute Goal

  4. It’s not about the numbers or techniques you’ll see. It’s

    all about continuous improvement.
  5. Dedicated Environment

  6. Execution Time in Minutes 180 123 New Environment

  7. Empty Databases

  8. The time needed to create data for one test: And

    then the test starts Call 12 API endpoints Modify data in 11 tables Takes about 1.2 seconds
  9. 180 123 Execution Time in Minutes 89 Empty Databases

  10. Simulate Dependencies

  11. +Some More STUB STUB STUB STUB STUB STUB STUB Stub

    all external dependencies Core API
  12. Transparent Fake SSL certs Dynamic Responses Local Storage Return Binary

    Data Regex URL match Existing Tools (March 2016) Stubby4J WireMock Wilma soapUI MockServer mounteback Hoverfly Mirage We created project Nagual, open source soon.
  13. 180 123 89 Execution Time in Minutes 65 Stub Dependencies

  14. Move to Containers

  15. 180 123 89 65 Execution Time in Minutes 104 Using

    Containers
  16. Run Databases in Memory

  17. 180 123 89 65 104 Execution Time in Minutes 61

    Run Databases in Memory
  18. Don’t Clean Test Data

  19. 180 123 89 65 104 61 Execution Time in Minutes

    46 Don’t delete test data
  20. Run in Parallel

  21. 4 6 8 10 12 14 16 Time to execute

    12 9 7 5 8 12 17 The Sweet Spot
  22. 180 123 89 65 104 61 46 Execution Time in

    Minutes 5 Run in Parallel
  23. Equalize Workload

  24. None
  25. None
  26. 180 123 89 65 104 61 46 5 Execution Time

    in Minutes 3 Equal Batches Run in Parallel Don’t delete test data Run Databases in Memory Using Containers Stub Dependencies Empty Databases New Environment
  27. After Hardware Upgrade The Outcome 2:15 min. 1:38 min.

  28. The tests are slow The tests are unreliable The tests

    can’t exactly pinpoint the problem High Level Tests Problems 3 Minutes No external dependencies It’s cheap to run all tests after every change
  29. In a couple of years, running all your automated tests,

    after every code change, for less than 3 minutes, will be standard development practice.
  30. Recommended Reading

  31. EmanuilSlavov.com @EmanuilSlavov

  32. Slide #, Photo Credits 1. https://www.flickr.com/photos/thomashawk 5. https://www.flickr.com/photos/100497095@N02 7. https://www.flickr.com/photos/andrewmalone

    10. https://www.flickr.com/photos/astrablog 14. https://www.flickr.com/photos/foilman 16. https://www.flickr.com/photos/missusdoubleyou 18. https://www.flickr.com/photos/canonsnapper 20. https://www.flickr.com/photos/anotherangle 23. https://www.flickr.com/photos/-aismist