Let's make this test suite run faster

The more the tests, the longer the build. And when the build gets longer, the bugs take longer to fix, the features take longer to deploy.

A build should be less than five minutes long, all tests included. Now lets say, our test suite takes much longer than that. How to reduce its duration? Where to start?

David will present different strategies: Making some tests become useless, converting functional tests to unit tests, running tests in parallel, building projects in parallel, doing slow tasks only once, writing fast DBMS tests... Let's share dozens of tips to accelerate your test suite A LOT.

Beware! This presentation is targeted at lazy people and cheaters but also brave ones. Three approaches to combine for the best result.

Transcript

1. Let's make this test suite run faster! Paris Java User Group
2. www.parisjug.org Copyright(c) 2010 Paris JUG, Licence CC-Creatives Commons 2.0 France - Paternité - Pas d’utilisation commerciale - Partage des Conditions à l’identique
3. What for ? Let's make this test suite run faster! Why? http://farm1.static.flickr.com/158/349497988_fb751a5e3a_z.jpg
4. You test early and often http://farm1.static.flickr.com/97/246816211_573c2901e1_z.jpg?zz=1 Don’t you?
5. Continuous integration, continuous testing... http://vator.tv/images/attachments/150609195630ctd.gif ...even continuous deployment
6. Testing often http://farm3.static.flickr.com/2505/3772118924_11fe195ef1.jpg Often means waiting a lot
7. What for ? Your coding room doesn’t have to look like this http://upload.wikimedia.org/wikipedia/commons/3/3f/ Nespresso_boutique.JPG
8. Tests can be fast Even if lines of code keep growing
9. Easy! Let’s distribute across multiple hudson servers
10. It doesn’t have to be complicated I’ll share a few simple tricks
11. Photo: http://www.flickr.com/photos/zoutedrop/2317065892/ It can make your product better Simple to test, means simple to write, deploy and USE
12. http://all-the-movies.cowblog.fr/images/arton1122.jpg How can we accelerate the tests? The Cheater The Lazy The Brave
13. The Cheater The Lazy The Brave http://all-the-movies.cowblog.fr/images/arton1122.jpg
14. Buy a faster machine http://farm1.static.flickr.com/28/93569705_1c562b413a_z.jpg?zz=1 Tests are cpu/memory bound
15. Be warned http://farm5.static.flickr.com/4096/4819945812_735744e0fc_z.jpg Single threaded tests get slower over time 0 25 50 75 100 2007 2008 2009 2010
16. Use all the cores mvn -T1 clean install : 5:05s mvn -T4 clean install : 3:10s parallel build with maven3
17. Use all the cores parallel build with maven3
18. Use all the cores Surefire 2.7.1 For JUnit/TestNG tests .
19. The Cheater The Lazy The Brave http://all-the-movies.cowblog.fr/images/arton1122.jpg
20. http://geekandpoke.typepad.com/geekandpoke/images/2008/04/24/ turingtest.jpg Delete redundant tests It’s so simple, we don’t do it...
21. Even better, delete dead code http://farm1.static.flickr.com/58/191393602_3d9b643ab0_z.jpg To delete yet other useless tests
22. Work in a sandbox In-memory database: H2 Behaves more like MySql than Hsqldb The network is too slow
23. It’s Not only SQL If going with NoSQL, take a server that can run in-process eg. Voldemort
24. In-memory SMTP Server SubEtha SMTP
25. Everything local and in-memory Apache VFS (Virtual File System) Spring Resource... Files As a bonus, tests will run smoother .
26. The Cheater The Lazy The Brave http://all-the-movies.cowblog.fr/images/arton1122.jpg
27. Don’t test business rules in integration tests Unit tests is often a better place http://upload.wikimedia.org/wikipedia/commons/b/bd/ Contortionist_Ravi_standing.jpg
28. Don’t test business rules in integration tests Unit tests is often a better place Functional test (on a web page) 10s Unit test 0.01s
29. Take the longer integration test Break it in one faster integration test and a lot of small unit tests http://farm2.static.flickr.com/1168/3166709586_96f9a3fd95.jpg
30. Or mock the slowest layers eg. functional test with Spring and Mockito Mocks are not just for unit tests
31. Don’t test through the browser Selenium is often overkill Photo: http://www.flickr.com/photos/zoutedrop/2317065892/ «But my application is complex!» «My users want complex features, My users want Ajax» «I need to test browser compatibility!»
32. Really ? http://i.zdnet.com/blogs/complexity-and-false-hope.jpg
33. Complexity has a cost That you pay each time a test runs http://thehongkongfixereng.files.wordpress.com/2008/07/couteau-suisse.jpg
34. Test through the browser the strict minimal Use javascript unit tests for the rest http://farm2.static.flickr.com/1168/3166709586_96f9a3fd95.jpg
35. I tend to be old school And write server-side code most of the time http://www.celebridiot.com/wp-content/uploads/2009/06/ rick_astley_death_hoax.jpg
36. One more thing... Simplify and optimize your code Tests will run faster .
37. Thank you Q/A
38. www.parisjug.org Copyright(c) 2010 Paris JUG, Licence CC-Creatives Commons 2.0 France - Paternité - Pas d’utilisation commerciale - Partage des Conditions à l’identique