Need for Speed (GTAC 2016)

This is the shorter version of the presentation given at VelocityConf Amsterdam 2016

Transcript

1. NEED FOR SPEED
   accelerate tests from 3 hours to 3 minutes
   emo@komfo.com
   

   View Slide

2. 3
   hours
   3
   minutes
   600 API tests
   

   View Slide

3. Before After
   The
   3 Minute
   Goal
   

   View Slide

4. It’s not about the numbers or
   techniques you’ll see.
   It’s all about continuous
   improvement.
   

   View Slide

5. Dedicated
   Environment
   

   View Slide

6. Execution Time in Minutes
   180
   123 New Environment
   

   View Slide

7. Empty Databases
   

   View Slide

8. The time needed to create data for one test:
   And then the test starts
   Call 12 API endpoints
   Modify data in 11 tables Takes about 1.2 seconds
   

   View Slide

9. 180
   123
   Execution Time in Minutes
   89 Empty Databases
   

   View Slide

10. Simulate
    Dependencies
    

    View Slide

11. +Some
    More
    STUB
    STUB
    STUB
    STUB
    STUB
    STUB
    STUB
    Stub all external dependencies
    Core
    API
    

    View Slide

12. Transparent
    Fake SSL certs
    Dynamic Responses
    Local Storage
    Return Binary Data
    Regex URL match
    Existing Tools (March 2016)
    Stubby4J
    WireMock
    Wilma
    soapUI
    MockServer
    mounteback
    Hoverfly
    Mirage
    We created project Nagual,
    open source soon.
    

    View Slide

13. 180
    123
    89
    Execution Time in Minutes
    65 Stub Dependencies
    

    View Slide

14. Move to Containers
    

    View Slide

15. 180
    123
    89
    65
    Execution Time in Minutes
    104 Using Containers
    

    View Slide

16. Run Databases
    in Memory
    

    View Slide

17. 180
    123
    89
    65
    104
    Execution Time in Minutes
    61 Run Databases in Memory
    

    View Slide

18. Don’t Clean
    Test Data
    

    View Slide

19. 180
    123
    89
    65
    104
    61
    Execution Time in Minutes
    46 Don’t delete test data
    

    View Slide

20. Run in Parallel
    

    View Slide

21. 4 6 8 10 12
    14 16
    Time to execute 12 9 7
    5 8 12 17
    The Sweet Spot
    

    View Slide

22. 180
    123
    89
    65
    104
    61
    46
    Execution Time in Minutes
    5 Run in Parallel
    

    View Slide

23. Equalize Workload
    

    View Slide

24. View Slide

25. View Slide

26. 180
    123
    89
    65
    104
    61
    46
    5
    Execution Time in Minutes
    3 Equal Batches
    Run in Parallel
    Don’t delete test data
    Run Databases in Memory
    Using Containers
    Stub Dependencies
    Empty Databases
    New Environment
    

    View Slide

27. After Hardware Upgrade
    The Outcome
    2:15 min.
    1:38 min.
    

    View Slide

28. The tests are slow
    The tests are unreliable
    The tests can’t exactly pinpoint the problem
    High Level Tests Problems
    3 Minutes
    No external dependencies
    It’s cheap to run all tests after every change
    

    View Slide

29. In a couple of years, running all your
    automated tests, after every code
    change, for less than 3 minutes, will be
    standard development practice.
    

    View Slide

30. Recommended Reading
    

    View Slide

31. EmanuilSlavov.com
    @EmanuilSlavov
    

    View Slide

32. Slide #, Photo Credits
    1. https://www.flickr.com/photos/thomashawk
    5. https://www.flickr.com/photos/100497095@N02
    7. https://www.flickr.com/photos/andrewmalone
    10. https://www.flickr.com/photos/astrablog
    14. https://www.flickr.com/photos/foilman
    16. https://www.flickr.com/photos/missusdoubleyou
    18. https://www.flickr.com/photos/canonsnapper
    20. https://www.flickr.com/photos/anotherangle
    23. https://www.flickr.com/photos/-aismist
    

    View Slide