Need for Speed (GTAC 2016)

This is the shorter version of the presentation given at VelocityConf Amsterdam 2016

1. NEED FOR SPEED accelerate tests from 3 hours to 3 minutes emo@komfo.com
2. 3 hours 3 minutes 600 API tests
3. Before After The 3 Minute Goal
4. It’s not about the numbers or techniques you’ll see. It’s all about continuous improvement.
5. Dedicated Environment
6. Execution Time in Minutes 180 123 New Environment
7. Empty Databases
8. The time needed to create data for one test: And then the test starts Call 12 API endpoints Modify data in 11 tables Takes about 1.2 seconds
9. 180 123 Execution Time in Minutes 89 Empty Databases
10. Simulate Dependencies
11. +Some More STUB STUB STUB STUB STUB STUB STUB Stub all external dependencies Core API
12. Transparent Fake SSL certs Dynamic Responses Local Storage Return Binary Data Regex URL match Existing Tools (March 2016) Stubby4J WireMock Wilma soapUI MockServer mounteback Hoverfly Mirage We created project Nagual, open source soon.
13. 180 123 89 Execution Time in Minutes 65 Stub Dependencies
14. Move to Containers
15. 180 123 89 65 Execution Time in Minutes 104 Using Containers
16. Run Databases in Memory
17. 180 123 89 65 104 Execution Time in Minutes 61 Run Databases in Memory
18. Don’t Clean Test Data
19. 180 123 89 65 104 61 Execution Time in Minutes 46 Don’t delete test data
20. Run in Parallel
21. 4 6 8 10 12 14 16 Time to execute 12 9 7 5 8 12 17 The Sweet Spot
22. 180 123 89 65 104 61 46 Execution Time in Minutes 5 Run in Parallel
23. Equalize Workload
26. 180 123 89 65 104 61 46 5 Execution Time in Minutes 3 Equal Batches Run in Parallel Don’t delete test data Run Databases in Memory Using Containers Stub Dependencies Empty Databases New Environment
27. After Hardware Upgrade The Outcome 2:15 min. 1:38 min.
28. The tests are slow The tests are unreliable The tests can’t exactly pinpoint the problem High Level Tests Problems 3 Minutes No external dependencies It’s cheap to run all tests after every change
29. In a couple of years, running all your automated tests, after every code change, for less than 3 minutes, will be standard development practice.
30. Recommended Reading
31. EmanuilSlavov.com @EmanuilSlavov
32. Slide #, Photo Credits 1. https://www.flickr.com/photos/thomashawk 5. https://www.flickr.com/photos/100497095@N02 7. https://www.flickr.com/photos/andrewmalone 10. https://www.flickr.com/photos/astrablog 14. https://www.flickr.com/photos/foilman 16. https://www.flickr.com/photos/missusdoubleyou 18. https://www.flickr.com/photos/canonsnapper 20. https://www.flickr.com/photos/anotherangle 23. https://www.flickr.com/photos/-aismist