Developer Tests - Things to Know (Dnipropetrovsk Wix R&D)

Transcript

1. Developer Tests Things to Know Vaidas Pilkauskas 2014 Dnipropetrovsk
2. me My hobbies • developer at Wix.com • main language - Scala • main professional interest - developer communities If there is time left after my hobbies • mountain bicycle rider, snowboarder • consumer of rock music, contemporary art, etc
3. me - how to contact me connect with me on LinkedIn http://lt.linkedin. com/pub/vaidas-pilkauskas/8/77/863/ add me on G+ https://www.google.com/+VaidasPilkauskas follow on Twitter @liucijus
4. “We couldn’t understand why people without technical knowledge had to tell programmers “what” to do and, furthermore, they had to supervise “how” programmers did it.” Cristian Rennella http://qz.com/260846/why-our-startup-has-no-bosses-no-office-and-a-four-day-work-week/
5. What this talk is about • Things we argue about during code reviews • Things that took me time to understand and prove that they are actually good way to go • Small things we have no time to discuss during big talks
7. “Legacy Code is code without Tests” Michael Feathers Working Effectively with Legacy Code
8. So what is test? It is system’s exercise under predefined conditions and then verification of an expected outcome.
9. Test structure http://healthysimulation.com/2743/structure-collapse-simulator-constructed-by-strategic-response-initiatives/
10. Test phases Setup Exercise Verify Teardown
11. Test phases in code Server server = new NotesServer(); // setup Note note = new Note("test note"); // setup Status status = server.add(note); // exercise assertEquals(SUCCESS, status); // verify server.shutdown(); // teardown
12. Start everything in one method @Test public void serverShouldAddNoteSuccessfully() { Server server = new NotesServer(); // setup Note note = new Note("test note"); // setup Status status = server.add(note); // exercise assertEquals(SUCCESS, status); // verify server.shutdown(); // teardown }
13. Refactor to lifecycle methods @Before public void before() { server = new NotesServer(); // setup note = new Note("test note"); // setup } @Test public void serverShouldAddNoteSuccessfully() { Status status = server.add(note); // exercise assertEquals(SUCCESS, status); // verify } @After public void after() { server.shutdown(); // teardown }
14. Rule #1 Do not start from setup/teardown
15. @Test public void shouldServeEmptyUserList() { server.start(); assertThat(server.getUsers(), is(empty())); server.stop(); } @Before public void startServer() { server.start(); } @Test public void shouldServeEmptyUserList() { //todo: implement } @After public void stopServer() { server.stop(); } BAD GOOD
16. How many colors are there in the picture? Source: Josef Albers Interaction of Color
17. Context matters! Source: Josef Albers Interaction of Color
18. Let’s talk about setup in @Before @Before public void before() { server = new NotesServer(); // setup note = new Note("test note"); // setup } @Test public void serverShouldAddNoteSuccessfully() { Status status = server.add(note); // exercise assertEquals(SUCCESS, status); // verify } @After public void after() { server.shutdown(); // teardown }
19. http://www.worldwildlife.org/threats/pollution Setup pollution
20. Setup pollution - #1 • Every line in setup must be relevant to all tests.
21. BAD @Before public void preparePets() { cat = new Cat(); // only needed by dogShouldSayFoo dog = new Dog(); // only needed by catShouldSayBar } @Test public void dogShouldSayFoo() { assertThat(dog.says(), is("foo")) } @Test public void catShouldSayBar() { assertThat(cat.says(), is("bar")) }
22. Setup pollution - #2 • It is tempting to add additional setup tuning just to fix/enhance one test.
23. BAD @Before public void preparePets() { cat = new Cat().withHairstyle(CURLY); dog = new Dog(); } @Test public void curlyCatShouldFeelBeautiful(){ assertThat(cat.isFeelingBeautiful(), is(true)); } @Test public void dogShouldSayFoo() { assertThat(dog.says(), is("foo")); } @Test public void catShouldSayBar() { assertThat(cat.says(), is("bar")); }
24. Setup pollution - #3 • Rich setup makes tests slow!
25. Setup pollution - #4 • Setup is to bootstrap your SUT.
26. Setup pollution - #5 • Setup hides details! Do not hide test preconditions.
27. Setting up note inside test method @Before public void before() { server = new NotesServer(); // setup } @Test public void serverShouldAddNoteSuccessfully() { note = new Note("test note"); // setup Status status = server.add(note); // exercise assertEquals(SUCCESS, status); // verify }
28. But then be more specific @Test public void serverShouldAddSingleLineNoteSuccesfully() { // * set up which is actual for the current method // * use scope specific name Note singleLineNote = new Note("test note"); // setup Status status = server.add(singleLineNote); // exercise assertEquals(SUCCESS, status); // verify }
29. Give good names to setup methods @Before public void createNotesServer() { server = new NotesServer(); }
30. Summary of test code organization • DRY principle. • Readability. BDD vs. DRY • Consistency. Maintain the same style across your codebase. • Complexity. It may dictate the way you go.
31. Refactoring Refactoring is about improving the design of existing code. It is the process of changing a software system in such a way that it does not alter the external behavior of the code, yet improves its internal structure. Martin Fowler Refactoring: Improving the Design of Existing Code
32. What do we test? http://www.dumpaday.com/random-pictures/daily-randomness-40-pics/
33. Test behaviour not methods @Test public void testGetBalance() { assertEquals(0, account.getBalance()) } @Test public void shouldBeEmptyAfterCreation() { assertEquals(0, account.getBalance()); }
34. Test behaviour not methods • Think of a contract
35. Test behaviour not methods • Think of a contract • And responsibilities
36. Test behaviour not methods • Think of a contract • And responsibilities • Specify requirements as tests
37. Test behaviour not methods • Think of a contract • And responsibilities • Specify requirements as tests • Happens naturally when done in test-first approach
38. Matchers http://www.allthatiknow.com/2013/10/comparing-google-analytics-standard-vs-premium/
39. Matchers • Assert on the right level of abstraction • Encapsulate testing logic
40. Matcher libraries • Hamcrest - standard matcher lib for JUnit • AssertJ - fluent assertions (IDE friendly) • Provide common matchers • You can write your own custom matchers
41. Hamcrest assertThat(frodo.getName(), equalTo("Frodo")); assertThat(frodo.getName(), is(equalTo("Frodo"))); assertThat(frodo.getName(), is("Frodo")); //JUnit assertEquals("Frodo", frodo.getName())
42. AssertJ assertThat(frodo.getName()).isEqualTo("Frodo"); assertThat(frodo).isNotEqualTo(sauron).isIn(fellowshipOfTheRing); assertThat(sauron).isNotIn(fellowshipOfTheRing); //JUnit assertEquals("Frodo", frodo.getName())
43. Custom matchers http://mimiandeunice.com/2011/04/19/compare/
44. Custom matchers Are matchers that we write specifically for our projects.
45. Custom matchers • Help communicate test intention • Abstract assertion logic
46. Custom matchers • Help communicate test intention • Abstract assertion logic • Are reusable and save time in large projects • You may have a custom message to be more specific about test failure
47. Custom matchers @Test public void shouldHaveIsbnGenerated() { Book book = new Book(1l, "5555", "A book"); assertThat(book, hasIsbn("1234")); }
48. Failing a test
49. fail() In some cases (e.g. testing exceptions) you may want to force test to fail if some expected situation does not happen
50. fail() try { // do stuff... fail("Exception not thrown"); } catch(Exception e){ assertTrue(e.hasSomeFlag()); }
51. fail() • Fundamentally not bad, but better use matchers for expected failure • Matchers help to clarify test intention • Don’t forget - expected behaviour is an opposite of a failing test
52. Anti-pattern: The Ugly Mirror http://www.iwillnotdiet.com/?attachment_id=2605
53. Anti-pattern: The Ugly Mirror @Test public void personToStringShouldIncludeNameAndSurname() { Person person = new Person("Волк", "Серый"); String expected = "Person[" + person.getName() + " " + person.getSurname() + "]" assertEquals(expected, person.toString()); }
54. Anti-pattern: The Ugly Mirror @Test public void personToStringShouldIncludeNameAndSurname() { Person person = new Person("Волк", "Серый"); String expected = "Person[" + person.getName() + " " + person.getSurname() + "]" assertEquals(expected, person. toString()); }
55. Anti-pattern: The Ugly Mirror @Test public void personToStringShouldIncludeNameAndSurname() { Person person = new Person("Волк", "Серый"); assertEquals("Person[Волк Серый]", person.toString()); }
56. How to turn off the test? http://namo-rekonstrukcija.blogspot.com/2013/05/elektra-izeminimas.html
57. Why would you want to turn off the test?
58. Why would you want to turn off the test? Well, because it fails… :)
59. Ignoring tests • Always use ignore/pending API from your test library (JUnit @Ignore) @Ignore("charge simplicator not ready") @Test public void shouldChargeMonthly() { //implementation which fails }
60. Ignoring tests • Do not comment out or false assert your test //Will fix it later™ //@Test public void shouldBeZeroInitialy() { // assertEquals(0, account.getBalance()); //} @Test public void shouldBeZeroInitialy() { assertEquals(1, account.getBalance()); }
61. Ignoring tests • Do not comment out or false assert your test //Will fix it later™ //@Test public void shouldBeZeroInitialy() { // assertEquals(0, account.getBalance()); //} @Test public void shouldBeZeroInitialy() { assertEquals(1, account.getBalance()); } Who is Will?
62. Ignoring tests • If you do not need a test - delete it
63. What to do with exceptions? http://s273.photobucket.com/user/eliteskater08/media/meateatingrabbit.jpg.html
64. Exceptions • If you can, use matchers instead of ◦ @Test(expected=?) ◦ try-catch approach
65. JUnit expected exception @Test(expected=IndexOutOfBoundsException.class) public void shouldThrowIndexOutOfBoundsException() { ArrayList emptyList = new ArrayList(); Object o = emptyList.get(0); } //matcher in Specs2 (Scala) server.process(None) must throwA[NothingToProccess]
66. try and catch public void shouldThrowIndexOutOfBoundsException() { ArrayList emptyList = new ArrayList(); try { Object o = emptyList.get(0); fail("Should throw IndexOutOfBoundsException"); } catch(IndexOutOfBoundsException e)){ //consider asserting message! } }
67. Exceptions • catch-exception lib
68. catch-exception lib List myList = new ArrayList(); catchException(myList).get(1); assertThat(caughtException(), allOf( is(IndexOutOfBoundsException.class), hasMessage("Index: 1, Size: 0"), hasNoCause() ) );
69. Exceptions • What about ExpectedException Rule? ◦ My personal opinion - not that intuitive ◦ breaks arrange/act/assert flow
70. ExpectedException rule @Rule public ExpectedException exception = ExpectedException.none(); @Test public void testExpectedException() { exception.expect(IllegalArgumentException.class); exception.expectMessage(containsString('Invalid age')); new Person('Vilkas', -1); } //Person constructor public Person(String name, int age) { if (age <= 0) throw new IllegalArgumentException('Invalid age:' + age); // ... }
71. Testing with time http://wisconsinrelocation.net/post/1295775/tickle-your-funny-bone-----time-management-is-everything--
72. Problem public class MyService { ... public void process(LocalDate date) { if (date.isBefore(LocalDate.now()) { ... } else { ... } } }
73. Testing with Time • Design your system where time is a collaborator • Inject test specific time provider in your test ◦ constant time ◦ slow time ◦ boundary cases time
74. Control time with Clock public class MyService { private Clock clock; // dependency inject ... public void process(LocalDate date) { if (date.isBefore(LocalDate.now(clock)) { ... } else { ... } } }
75. Collections http://mycoolguitars.com/cool-guitar-collections/
76. Collections - multiple properties to assert • Is null? • Size • Order • Content
77. Collections • Most of the time you want to assert on collection content assertThat(users, containsInAnyOrder("andrius", "vaidas"))
78. Collections • Prefer exact content matching assertThat(users, contains("andrius", "vaidas"))
79. Collections • Avoid incomplete assertions //users=List.of("andrius", "vaidas", "valdemaras", "jonas") assertThat(users, hasItems("andrius", "vaidas"))
80. Collections • Do not sort just because it is easier to assert! assertThat(sort(users), contains("andrius", "vaidas"))
81. Collections • Multiple assertions are worse than single content assertion assertThat(users, hasSize(2)) assertThat(users, hasItems("andrius", "vaidas"))
82. Collections • Use matchers!
83. Random values
84. Random values in tests • Most of the time you do not want it @Test public void shouldStartWithFirst() { String a = randomString(); String b = randomString(); assertThat(a + b, startsWith(a)); }
85. Random values in tests • Unless you depend on randomness a lot (eg. password generation*) *Thanks to Aleksandar Tomovski for a good example
86. Random values in tests • Use property based testing (which is also hard) //example from ScalaCheck property("startsWith") = forAll { (a: String, b: String) => (a+b).startsWith(a) }
87. Random values in tests • Do not make dummy values random cat = new Cat(randomString()); cat = new Cat("dummyName");
88. What if we still need random cases?
89. Generate Multiple Test Cases • Quality over quantity
90. Generate Multiple Test Cases • Quality over quantity • Think of boundary cases, that you may want to detect with random test
91. Generate Multiple Test Cases • Quality over quantity • Think of boundary cases, that you may want to detect with random test • Use parameterized tests
92. Generate Multiple Test Cases • Quality over quantity • Think of boundary cases, that you may want to detect with random test • Use parameterized tests • Random is hard to repeat
93. Generate Multiple Test Cases • Quality over quantity • Think of boundary cases, that you may want to detect with random test • Use parameterized tests • Random is hard to repeat • Flickering tests
94. How many assertions per test? http://dailypicksandflicks.com/2012/05/11/daily-picdump-455/4-beers-at-the-same-time/
95. How many assertions per test? • Unit test - one assertion per test. Must be clear and readable • Proper tests should fail for exactly one reason • End to end - best case one assertion per test, but more are allowed • Consider custom matchers
96. How many assertions per test? • Unit test - one assertion per test. Must be clear and readable • Proper tests should fail for exactly one reason • End to end - best case one assertion per test, but more are allowed • Consider custom matchers
97. Thing #13 Decoupling in End-to-end tests
98. What can be better in this test? [pseudocode] @Test shouldRetrieveUserByLogin() { String userJson = "{\"username\": \"vaidas\"}"; HttpRequest post = new Post("https://localhost:8080/users", userJson); HttpResponse postResp = HttpClient().execute(post); assertThat(postResp.status, is(200)); HttpRequest get = new Get("https://localhost:8080/users/vaidas"); HttpResponse getResp = HttpClient().execute(get); User user = mapper.readValue(getResp, User.class); assertThat(user.username, is("vaidas")); }
99. What can be better in this test? [pseudocode] @Test shouldRetrieveUserByLogin() { String userJson = "{\"username\": \"vaidas\"}"; HttpRequest post = new Post("https://localhost:8080/users", userJson); HttpResponse postResp = HttpClient().execute(post); assertThat(postResp.status, is( 200)); HttpRequest get = new Get("https://localhost:8080/users/vaidas"); HttpResponse getResp = HttpClient().execute(get); User user = mapper.readValue(getResp, User.class); assertThat(user.username, is("vaidas")); }
100. Decoupling from low level details [pseudocode] @Test shouldRetrieveUserByUsername() { CreateUserResponse createResp = aCreateUserRequest().withUsername("vaidas").execute(); assertThat(createResp, isSuccessful()); GetUserResponse getResp = aGetUserRequest().withUsername("vaidas").execute(); assertThat(getResp, allOf(isSuccessful(), hasUsername("vaidas"))); }
101. Summary • Context matters • Testings is controversial topic • And very opinionated
102. Thanks! Q & A