Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Need for Speed (GTAC 2016)

emanuil
November 16, 2016

Need for Speed (GTAC 2016)

This is the shorter version of the presentation given at VelocityConf Amsterdam 2016

emanuil

November 16, 2016
Tweet

More Decks by emanuil

Other Decks in Programming

Transcript

  1. NEED FOR SPEED
    accelerate tests from 3 hours to 3 minutes
    [email protected]

    View Slide

  2. 3
    hours
    3
    minutes
    600 API tests

    View Slide

  3. Before After
    The
    3 Minute
    Goal

    View Slide

  4. It’s not about the numbers or
    techniques you’ll see.
    It’s all about continuous
    improvement.

    View Slide

  5. Dedicated
    Environment

    View Slide

  6. Execution Time in Minutes
    180
    123 New Environment

    View Slide

  7. Empty Databases

    View Slide

  8. The time needed to create data for one test:
    And then the test starts
    Call 12 API endpoints
    Modify data in 11 tables Takes about 1.2 seconds

    View Slide

  9. 180
    123
    Execution Time in Minutes
    89 Empty Databases

    View Slide

  10. Simulate
    Dependencies

    View Slide

  11. +Some
    More
    STUB
    STUB
    STUB
    STUB
    STUB
    STUB
    STUB
    Stub all external dependencies
    Core
    API

    View Slide

  12. Transparent
    Fake SSL certs
    Dynamic Responses
    Local Storage
    Return Binary Data
    Regex URL match
    Existing Tools (March 2016)
    Stubby4J
    WireMock
    Wilma
    soapUI
    MockServer
    mounteback
    Hoverfly
    Mirage
    We created project Nagual,
    open source soon.

    View Slide

  13. 180
    123
    89
    Execution Time in Minutes
    65 Stub Dependencies

    View Slide

  14. Move to Containers

    View Slide

  15. 180
    123
    89
    65
    Execution Time in Minutes
    104 Using Containers

    View Slide

  16. Run Databases
    in Memory

    View Slide

  17. 180
    123
    89
    65
    104
    Execution Time in Minutes
    61 Run Databases in Memory

    View Slide

  18. Don’t Clean
    Test Data

    View Slide

  19. 180
    123
    89
    65
    104
    61
    Execution Time in Minutes
    46 Don’t delete test data

    View Slide

  20. Run in Parallel

    View Slide

  21. 4 6 8 10 12
    14 16
    Time to execute 12 9 7
    5 8 12 17
    The Sweet Spot

    View Slide

  22. 180
    123
    89
    65
    104
    61
    46
    Execution Time in Minutes
    5 Run in Parallel

    View Slide

  23. Equalize Workload

    View Slide

  24. View Slide

  25. View Slide

  26. 180
    123
    89
    65
    104
    61
    46
    5
    Execution Time in Minutes
    3 Equal Batches
    Run in Parallel
    Don’t delete test data
    Run Databases in Memory
    Using Containers
    Stub Dependencies
    Empty Databases
    New Environment

    View Slide

  27. After Hardware Upgrade
    The Outcome
    2:15 min.
    1:38 min.

    View Slide

  28. The tests are slow
    The tests are unreliable
    The tests can’t exactly pinpoint the problem
    High Level Tests Problems
    3 Minutes
    No external dependencies
    It’s cheap to run all tests after every change

    View Slide

  29. In a couple of years, running all your
    automated tests, after every code
    change, for less than 3 minutes, will be
    standard development practice.

    View Slide

  30. Recommended Reading

    View Slide

  31. EmanuilSlavov.com
    @EmanuilSlavov

    View Slide

  32. Slide #, Photo Credits
    1. https://www.flickr.com/photos/thomashawk
    5. https://www.flickr.com/photos/[email protected]
    7. https://www.flickr.com/photos/andrewmalone
    10. https://www.flickr.com/photos/astrablog
    14. https://www.flickr.com/photos/foilman
    16. https://www.flickr.com/photos/missusdoubleyou
    18. https://www.flickr.com/photos/canonsnapper
    20. https://www.flickr.com/photos/anotherangle
    23. https://www.flickr.com/photos/-aismist

    View Slide