Supercharge your Automated Testing with Component and Integration Tests

These slides are for a talk I gave at AnDevCon DC 2017:
http://www.andevcon.com/dc2017/sessions#SuperchargeYourAutomatedTestingwithComponentandIntegrationTests

You can hear a video of the presentation here:
https://www.youtube.com/watch?v=vTzAFoQQcCo

------------------------------------

SUMMARY

You've probably heard of unit tests and UI tests, but what about component and integration tests?

Component and integration tests are critical layers in the test automation pyramid. They speed up your development and release cycles, reduce risk in your project, and they even provide hidden benefits like infrastructure monitoring!

This talk covers:
- why we write automated tests
- the differences between unit tests, component tests, integration tests and UI tests
- how to decide which kinds of tests to write
- the secret benefits of component and integration tests.

Themes in this talk are:
- don't be ideological, know when and why to write automated tests
- unit tests often leave gaps and need to be refactored whenever the implementation changes, so higher level tests are important ("the whole is greater than the sum of the parts")
- it's important to automate everything, this is a key skill in multiplying your impact as a developer.

Transcript

1. SUPERCHARGE YOUR AUTOMATED TESTING
2. • Fundamentals • Automated testing playbook • Case study • Golden rules 05 Agenda
3. AUTOMATED TESTING FUNDAMENTALS
4. • Why write tests • Testing pyramid • Types of automated test 05 Fundamentals
6. Why write automated tests? • Deliver faster and higher quality • Your product is always ready to deploy • Capture institutional knowledge in tests • Automation hugely improves productivity
8. 05 The Testing Pyramid Slower Higher level Better QA Harder to debug Faster More specific Less useful for QA Better for debugging
9. The Testing Pyramid - Android • Android UI Tests, Real Servers • Android Tests, Real Servers • JVM/Robolectric Tests, Mock Data • Android Tests, Mock Data
10. Client Server View Presenter Data Access User Types of Automated Test
11. Client Server User Types of Automated Test Unit Test Unit Test Unit Test
12. Component Test Client Server User Types of Automated Test Component Test
13. Integration Test Client Server User Types of Automated Test
14. Black Box Test Client Server User Types of Automated Test
15. AUTOMATED TESTING PLAYBOOK
16. • Tedious • Slow • Necessary 05 Problem 1 – Manual QA Testing
17. Black Box Test Client Server User Black Box Tests
19. @Test public void testLoginSmokeTest() { TestAccountModel testAccount = testAccountManager.get(TestAccountType.blackbox); new LoginAndFtuxRobot() .skipWelcomeScreen() .username(testAccount.username) .password(testAccount.password) .login() .assertTermsAndConditionsShown() .acceptTermsAndConditions(); }
20. @Test public void testLoginSmokeTest() { TestAccountModel testAccount = testAccountManager.get(TestAccountType.blackbox); new LoginAndFtuxRobot() .skipWelcomeScreen() .username(testAccount.username) .password(testAccount.password) .login() .assertTermsAndConditionsShown() .acceptTermsAndConditions(); }
21. @Test public void testLoginSmokeTest() { TestAccountModel testAccount = testAccountManager.get(TestAccountType.blackbox); new LoginAndFtuxRobot() .skipWelcomeScreen() .username(testAccount.username) .password(testAccount.password) .login() .assertTermsAndConditionsShown() .acceptTermsAndConditions(); }
22. @Test public void testLoginSmokeTest() { TestAccountModel testAccount = testAccountManager.get(TestAccountType.blackbox); new LoginAndFtuxRobot() .skipWelcomeScreen() .username(testAccount.username) .password(testAccount.password) .login() .assertTermsAndConditionsShown() .acceptTermsAndConditions(); }
23. @Test public void testLoginSmokeTest() { TestAccountModel testAccount = testAccountManager.get(TestAccountType.blackbox); new LoginAndFtuxRobot() .skipWelcomeScreen() .username(testAccount.username) .password(testAccount.password) .login() .assertTermsAndConditionsShown() .acceptTermsAndConditions(); }
24. Pros • Avoid tedious manual QA testing • Faster releases after code freeze • High confidence that basic features work Cons • Fail if your APIs/environment are flakey • Slow build/test cycle for developers • Hard to debug when something goes wrong 05 Black Box Tests
25. • Need to find bugs early • Tests need to be super fast 05 Problem 2 – Developers Need Tests Too
26. Unit Tests Client Server User Unit Test Unit Test Unit Test
27. • Static utilities 05 Unit Test Playbook
28. @Test public void testSafeCompareEqual() { assertThat(NumberUtils.safeCompare(null, null)).isEqualTo(0); assertThat(NumberUtils.safeCompare(21, 21)).isEqualTo(0); } @Test public void testSafeCompareLessThan() { // Right param is less than left param assertThat(NumberUtils.safeCompare(21, null)).isEqualTo(-1); assertThat(NumberUtils.safeCompare(23, 21)).isEqualTo(-1); } @Test public void testSafeCompareMoreThan() { // Left param is less than right param assertThat(NumberUtils.safeCompare(null, 21)).isEqualTo(1); assertThat(NumberUtils.safeCompare(21, 22)).isEqualTo(1); }
29. @Test public void testSafeCompareEqual() { assertThat(NumberUtils.safeCompare(null, null)).isEqualTo(0); assertThat(NumberUtils.safeCompare(21, 21)).isEqualTo(0); } @Test public void testSafeCompareLessThan() { // Right param is less than left param assertThat(NumberUtils.safeCompare(21, null)).isEqualTo(-1); assertThat(NumberUtils.safeCompare(23, 21)).isEqualTo(-1); } @Test public void testSafeCompareMoreThan() { // Left param is less than right param assertThat(NumberUtils.safeCompare(null, 21)).isEqualTo(1); assertThat(NumberUtils.safeCompare(21, 22)).isEqualTo(1); }
30. @Test public void testFormatAsCurrencyPositive() { assertThat(formatAsCurrency("0.01")).isEqualTo("$0.01"); assertThat(formatAsCurrency("3.45")).isEqualTo("$3.45"); assertThat(formatAsCurrency("3.5")).isEqualTo("$3.50"); assertThat(formatAsCurrency("22.22")).isEqualTo("$22.22"); assertThat(formatAsCurrency("333.33")).isEqualTo("$333.33"); assertThat(formatAsCurrency("4444.44")).isEqualTo("$4,444.44"); assertThat(formatAsCurrency("55555.55")).isEqualTo("$55,555.55"); }
31. @Test public void testFormatAsCurrencyPositive() { assertThat(formatAsCurrency("0.01")).isEqualTo("$0.01"); assertThat(formatAsCurrency("3.45")).isEqualTo("$3.45"); assertThat(formatAsCurrency("3.5")).isEqualTo("$3.50"); assertThat(formatAsCurrency("22.22")).isEqualTo("$22.22"); assertThat(formatAsCurrency("333.33")).isEqualTo("$333.33"); assertThat(formatAsCurrency("4444.44")).isEqualTo("$4,444.44"); assertThat(formatAsCurrency("55555.55")).isEqualTo("$55,555.55"); }
32. @Test public void testFormatAsCurrencyPositive() { assertThat(formatAsCurrency("0.01")).isEqualTo("$0.01"); assertThat(formatAsCurrency("3.45")).isEqualTo("$3.45"); assertThat(formatAsCurrency("3.5")).isEqualTo("$3.50"); assertThat(formatAsCurrency("22.22")).isEqualTo("$22.22"); assertThat(formatAsCurrency("333.33")).isEqualTo("$333.33"); assertThat(formatAsCurrency("4444.44")).isEqualTo("$4,444.44"); assertThat(formatAsCurrency("55555.55")).isEqualTo("$55,555.55"); }
33. @Test public void testDecrypt() { byte[] decryptedData = decrypt(ENCRYPTED_DATA); String decryptedText = new String(decryptedData); assertThat("expectedUnencryptedText").isEqualTo(decryptedText); }
34. • Static utilities • Edge cases 05 Unit Test Playbook
35. @Captor ArgumentCaptor<Exception> exceptionCaptor; @Test public void testTransactionsErrorWithNullReferenceId() { TransactionsResponse event = new TransactionsResponse(new BaseError("error message")); fragment.receiveTransactions(event); assertThat(fragment.loadingMessage).isVisible(); assertThat(fragment.loadingMessage).hasText( "Unable to load transactions"); verify(analytics).trackError( "Unable to load transactions", "error message"); verify(loggingUtil).logHandledException(exceptionCaptor.capture()); assertThat(exceptionCaptor.getValue().getMessage()) .isEqualTo("Unable to load transactions"); }
36. @Captor ArgumentCaptor<Exception> exceptionCaptor; @Test public void testTransactionsErrorWithNullReferenceId() { TransactionsResponse event = new TransactionsResponse(new BaseError("error message")); fragment.receiveTransactions(event); assertThat(fragment.loadingMessage).isVisible(); assertThat(fragment.loadingMessage).hasText( "Unable to load transactions"); verify(analytics).trackError( "Unable to load transactions", "error message"); verify(loggingUtil).logHandledException(exceptionCaptor.capture()); assertThat(exceptionCaptor.getValue().getMessage()) .isEqualTo("Unable to load transactions"); }
37. @Captor ArgumentCaptor<Exception> exceptionCaptor; @Test public void testTransactionsErrorWithNullReferenceId() { TransactionsResponse event = new TransactionsResponse(new BaseError("error message")); fragment.receiveTransactions(event); assertThat(fragment.loadingMessage).isVisible(); assertThat(fragment.loadingMessage).hasText( "Unable to load transactions"); verify(analytics).trackError( "Unable to load transactions", "error message"); verify(loggingUtil).logHandledException(exceptionCaptor.capture()); assertThat(exceptionCaptor.getValue().getMessage()) .isEqualTo("Unable to load transactions"); }
38. @Captor ArgumentCaptor<Exception> exceptionCaptor; @Test public void testTransactionsErrorWithNullReferenceId() { TransactionsResponse event = new TransactionsResponse(new BaseError("error message")); fragment.receiveTransactions(event); assertThat(fragment.loadingMessage).isVisible(); assertThat(fragment.loadingMessage).hasText( "Unable to load transactions"); verify(analytics).trackError( "Unable to load transactions", "error message"); verify(loggingUtil).logHandledException(exceptionCaptor.capture()); assertThat(exceptionCaptor.getValue().getMessage()) .isEqualTo("Unable to load transactions"); }
39. @Captor ArgumentCaptor<Exception> exceptionCaptor; @Test public void testTransactionsErrorWithNullReferenceId() { TransactionsResponse event = new TransactionsResponse(new BaseError("error message")); fragment.receiveTransactions(event); assertThat(fragment.loadingMessage).isVisible(); assertThat(fragment.loadingMessage).hasText( "Unable to load transactions"); verify(analytics).trackError( "Unable to load transactions", "error message"); verify(loggingUtil).logHandledException(exceptionCaptor.capture()); assertThat(exceptionCaptor.getValue().getMessage()) .isEqualTo("Unable to load transactions"); }
40. • Static utilities • Edge cases • Important features 05 Unit Test Playbook
41. @Test public void testSecureWindowFlagSet() { int FLAG_SECURE = WindowManager.LayoutParams.FLAG_SECURE; ShadowWindow shadowWindow = shadowOf(activity.getWindow()); assertThat(shadowWindow.getFlag(FLAG_SECURE)).isTrue(); }
42. • Static utilities • Edge cases • Important features • Collaboration 05 Unit Test Playbook
44. @Mock public CardControlsFeature feature; @Test public void testLockedCardFeatureOn() { when(feature.isCardLockEnabled()).thenReturn(true); assertThat(cardLock).isVisible(); assertThat(cardLock).hasText("Card is Locked"); assertThat(rewards).isGone(); } @Test public void testLockedCardFeatureOff() { when(feature.isCardLockEnabled()).thenReturn(false); assertThat(cardLock).isGone(); assertThat(rewards).isVisible(); }
45. @Mock public CardControlsFeature feature; @Test public void testLockedCardFeatureOn() { when(feature.isCardLockEnabled()).thenReturn(true); assertThat(cardLock).isVisible(); assertThat(cardLock).hasText(”This Card is Locked"); assertThat(rewards).isGone(); } @Test public void testLockedCardFeatureOff() { when(feature.isCardLockEnabled()).thenReturn(false); assertThat(cardLock).isGone(); assertThat(rewards).isVisible(); }
46. @Mock public CardControlsFeature feature; @Test public void testLockedCardFeatureOn() { when(feature.isCardLockEnabled()).thenReturn(true); assertThat(cardLock).isVisible(); assertThat(cardLock).hasText(”This Card is Locked"); assertThat(rewards).isGone(); } @Test public void testLockedCardFeatureOff() { when(feature.isCardLockEnabled()).thenReturn(false); assertThat(cardLock).isGone(); assertThat(rewards).isVisible(); }
47. @Mock public CardControlsFeature feature; @Test public void testLockedCardFeatureOn() { when(feature.isCardLockEnabled()).thenReturn(true); assertThat(cardLock).isVisible(); assertThat(cardLock).hasText(”This Card is Locked"); assertThat(rewards).isGone(); } @Test public void testLockedCardFeatureOff() { when(feature.isCardLockEnabled()).thenReturn(false); assertThat(cardLock).isGone(); assertThat(rewards).isVisible(); }
48. @Mock public CardControlsFeature feature; @Test public void testLockedCardFeatureOn() { when(feature.isCardLockEnabled()).thenReturn(true); assertThat(cardLock).isVisible(); assertThat(cardLock).hasText(”This Card is Locked"); assertThat(rewards).isGone(); } @Test public void testLockedCardFeatureOff() { when(feature.isCardLockEnabled()).thenReturn(false); assertThat(cardLock).isGone(); assertThat(rewards).isVisible(); }
49. Pros • Fast • Great for small components • Forces you to write clean, testable code Cons • Often need to be rewritten when refactoring components • Easy to miss things in the gaps between tests 05 Unit Tests
50. • Often don’t guarantee component behaves correctly as a whole • Tests are tightly coupled 05 Problem 3 – Unit Tests Can’t Do Everything
51. public class MyTwoAPICalls { private class Part1Callback { @Override public void onSuccessResponse(Call<Part1Response> call, Response<Part1Response> part1Response) { bus.post(new Part2Request()); } } @Subscribe public void busListener(Part2Request event) { remote.part2(event).enqueue(new Part2Callback()); } private class Part2Callback { @Override public void onSuccessResponse(Call<Part2Response> call, Response<Part1Response> part1Response) { // Do more stuff } } }
52. public class MyTwoAPICalls { private class Part1Callback { @Override public void onSuccessResponse(Call<Part1Response> call, Response<Part1Response> part1Response) { bus.post(new Part2Request()); } } @Subscribe public void busListener(Part2Request event) { remote.part2(event).enqueue(new Part2Callback()); } private class Part2Callback { @Override public void onSuccessResponse(Call<Part2Response> call, Response<Part1Response> part1Response) { // Do more stuff } } }
53. public class MyTwoAPICalls { private class Part1Callback { @Override public void onSuccessResponse(Call<Part1Response> call, Response<Part1Response> part1Response) { bus.post(new Part2Request()); } } @Subscribe public void busListener(Part2Request event) { remote.part2(event).enqueue(new Part2Callback()); } private class Part2Callback { @Override public void onSuccessResponse(Call<Part2Response> call, Response<Part1Response> part1Response) { // Do more stuff } } }
54. public class MyTwoAPICalls { private class Part1Callback { @Override public void onSuccessResponse(Call<Part1Response> call, Response<Part1Response> part1Response) { bus.post(new Part2Request()); } } @Subscribe public void busListener(Part2Request event) { remote.part2(event).enqueue(new Part2Callback()); } private class Part2Callback { @Override public void onSuccessResponse(Call<Part2Response> call, Response<Part1Response> part1Response) { // Do more stuff } } }
55. public class MyTwoAPICalls { private class Part1Callback { @Override public void onSuccessResponse(Call<Part1Response> call, Response<Part1Response> part1Response) { bus.post(new Part2Request()); } } @Subscribe public void busListener(Part2Request event) { remote.part2(event).enqueue(new Part2Callback()); } private class Part2Callback { @Override public void onSuccessResponse(Call<Part2Response> call, Response<Part1Response> part1Response) { // Do more stuff } } }
56. Component Tests Client Server User Component Test
57. • Edge cases 05 Component Test Playbook
58. Example Component Client Server API Component Event Bus HTTP UI
59. Example Component Client Server API Component Event Bus Request UI
60. Example Component Client Server API Component Event Bus Request UI
61. Example Component Client Server API Component Event Bus HTTP Request UI
62. Example Component Client Server API Component Event Bus HTTP Response UI
63. Client Server API Component Event Bus Response UI Example Component
64. Client Server API Component Event Bus UI Component Test
65. Example Component Test Client API Component Event Bus Component Test UI Mocked Server
66. Example Component Test Client API Component Event Bus Component Test UI Mocked Server
67. • Edge cases 05 Component Test Playbook
68. @Test public void testServiceUnavailable503Html() { stubResponse(503, "service_unavailable_503.html"); bus.post(new Request()); assertBusEventReceived(); assertTrue(response.isError()); }
69. @Test public void testServiceUnavailable503Html() { stubResponse(503, "service_unavailable_503.html"); bus.post(new Request()); assertBusEventReceived(); assertTrue(response.isError()); }
70. @Test public void testServiceUnavailable503Html() { stubResponse(503, "service_unavailable_503.html"); bus.post(new Request()); assertBusEventReceived(); assertTrue(response.isError()); }
71. @Test public void testServiceUnavailable503Html() { stubResponse(503, "service_unavailable_503.html"); bus.post(new Request()); assertBusEventReceived(); assertTrue(response.isError()); }
72. @Test public void testServiceUnavailable503Html() { stubResponse(503, "service_unavailable_503.html"); bus.post(new Request()); assertBusEventReceived(); assertTrue(response.isError()); }
73. @Test public void testServiceUnavailable503Json() { stubResponse(503, "service_unavailable_503.json"); bus.post(new Request()); assertBusEventReceived(); assertEquals("SERVICE_UNAVAILABLE", response.getStatus()); assertEquals("We are undergoing system maintenance", response.getDescription()); }
74. @Test public void testServiceUnavailable503Json() { stubResponse(503, "service_unavailable_503.json"); bus.post(new Request()); assertBusEventReceived(); assertEquals("SERVICE_UNAVAILABLE", response.getStatus()); assertEquals("We are undergoing system maintenance", response.getDescription()); }
75. • Edge cases • Validate contracts 05 Component Test Playbook
76. @Test public void testGetTransactionsSuccess() { stubTransactions(200, "transactions_success.json"); bus.post(new TransactionsRequest()); assertBusEventReceived(); assertSuccessResponse(response); assertTrue(response.getTransactions().size() > 0); }
77. @Test public void testGetTransactionsSuccess() { stubTransactions(200, "transactions_success.json"); bus.post(new TransactionsRequest()); assertBusEventReceived(); assertSuccessResponse(response); assertTrue(response.getTransactions().size() > 0); }
78. Component UI Tests Component Test Client Server User
79. • Difficult to test UI flows 05 Component Test Playbook
81. @Test public void testActivateSingleCard() { stubberator.stubItAll(mockAccount); login(mockAccount); new ThirdPartyPayRobot() .assertActivateCardScreenShown() .activate() .assertActivationSuccessScreen() .gotIt(); }
82. @Test public void testActivateSingleCard() { stubberator.stubItAll(mockAccount); login(mockAccount); new ThirdPartyPayRobot() .assertActivateCardScreenShown() .activate() .assertActivationSuccessScreen() .gotIt(); }
83. @Test public void testActivateSingleCard() { stubberator.stubItAll(mockAccount); login(mockAccount); new ThirdPartyPayRobot() .assertActivateCardScreenShown() .activate() .assertActivationSuccessScreen() .gotIt(); }
84. @Test public void testActivateSingleCard() { stubberator.stubItAll(mockAccount); login(mockAccount); new ThirdPartyPayRobot() .assertActivateCardScreenShown() .activate() .assertActivationSuccessScreen() .gotIt(); }
87. • Difficult to test UI flows • Hermetic tests to avoid flakey tests 05 Component Test Playbook
88. @Test public void testLoginSuccess() { TestAccountModel testAccount = testAccountManager.get(TestAccountType.blackbox); new LoginAndFtuxRobot() .skipWelcomeScreen() .username(testAccount.username) .password(testAccount.password) .login() .assertTermsAndConditionsShown(); }
89. @Test public void testLoginSuccess() { stubberator.stubItAll(mockAccount); activityRule.launchActivity( new Intent(applicationContext, SplashActivity.class)); new LoginAndFtuxRobot() .username("user") .password("pwd") .login() .assertTermsAndConditionsScreenShown(); }
90. @Test public void testLoginSuccess() { stubberator.stubItAll(mockAccount); activityRule.launchActivity( new Intent(applicationContext, SplashActivity.class)); new LoginAndFtuxRobot() .username("user") .password("pwd") .login() .assertTermsAndConditionsScreenShown(); }
91. @Test public void testLoginSuccess() { stubberator.stubItAll(mockAccount); activityRule.launchActivity( new Intent(applicationContext, SplashActivity.class)); new LoginAndFtuxRobot() .username("user") .password("pwd") .login() .assertTermsAndConditionsScreenShown(); }
92. @Test public void testLoginIncorrectUsernameOrPassword() { stubberator.stubItAll(mockAccount); activityRule.launchActivity( new Intent(applicationContext, SplashActivity.class)); new LoginAndFtuxRobot(). .username("user") .password("pwd") .login() .assertTermsAndConditionsScreenShown(); }
93. @Test public void testLoginIncorrectUsernameOrPassword() { stubberator.stubItAll(mockAccount); stubFor(StubMappings.login().willReturn(aResponse() .withStatus(401).withBody(gson.toJson(ERROR_RESPONSE)))); activityRule.launchActivity( new Intent(applicationContext, SplashActivity.class)); new LoginAndFtuxRobot(). .username("user") .password("pwd") .login() .assertTermsAndConditionsScreenShown(); }
94. @Test public void testLoginIncorrectUsernameOrPassword() { stubberator.stubItAll(mockAccount); stubFor(StubMappings.login().willReturn(aResponse() .withStatus(401).withBody(gson.toJson(ERROR_RESPONSE)))); activityRule.launchActivity( new Intent(applicationContext, SplashActivity.class)); new LoginAndFtuxRobot(). .username("user") .password("pwd") .login() .assertInvalidUsernameOrPassword(); }
97. Pros • Validate component correctness at its interfaces • No need to update tests when refactoring component internals • Component UI tests aren’t flakey • Screenshots work as documentation Cons • Slower than unit tests 05 Component & Component UI Tests
98. • Your project slips if you do API integration testing too late • Test environment issues 05 Problem 4 – APIs Are Hard
99. Integration Test Client Server User Integration Tests User
100. • Check the API works as expected 05 Integration Test Playbook
101. Component Test Client API Component Event Bus Component Test UI Mocked Server
102. Component Test -> Integration Test Client API Component Event Bus Integration Test UI Server
103. • Check the API works as expected 05 Integration Test Playbook
104. @Test public void testCardLockStatusUpdateSuccess() { Card card = loginAndGetCard(TestAccountType.card_lock_eligible); boolean wasCardOriginallyLocked = card.isCardLocked(); updateCardLockStatus(!wasCardOriginallyLocked, card); // Now change it back to the original lock status again updateCardLockStatus(wasCardOriginallyLocked, card); }
105. @Test public void testCardLockStatusUpdateSuccess() { Card card = loginAndGetCard(TestAccountType.card_lock_eligible); boolean wasCardOriginallyLocked = card.isCardLocked(); updateCardLockStatus(!wasCardOriginallyLocked, card); // Now change it back to the original lock status again updateCardLockStatus(wasCardOriginallyLocked, card); }
106. @Test public void testCardLockStatusUpdateSuccess() { Card card = loginAndGetCard(TestAccountType.card_lock_eligible); boolean wasCardOriginallyLocked = card.isCardLocked(); updateCardLockStatus(!wasCardOriginallyLocked, card); // Now change it back to the original lock status again updateCardLockStatus(wasCardOriginallyLocked, card); }
107. @Test public void testCardLockStatusUpdateSuccess() { Card card = loginAndGetCard(TestAccountType.card_lock_eligible); boolean wasCardOriginallyLocked = card.isCardLocked(); updateCardLockStatus(!wasCardOriginallyLocked, card); // Now change it back to the original lock status again updateCardLockStatus(wasCardOriginallyLocked, card); }
108. @Test public void testCardLockStatusUpdateSuccess() { Card card = loginAndGetCard(TestAccountType.card_lock_eligible); boolean wasCardOriginallyLocked = card.isCardLocked(); updateCardLockStatus(!wasCardOriginallyLocked, card); // Now change it back to the original lock status again updateCardLockStatus(wasCardOriginallyLocked, card); }
109. • Check the API works as expected • Add useful assertions 05 Integration Test Playbook
110. @Test public void testCardsSuccess() { loginAndGetCards(testAccount); assertSuccessResponse(response); List<Card> cards = response.getCards(); assertFalse("No cards were returned for " + testAccount, cards.isEmpty()); for (Card card : cards) { assertNotNull( "reference_id is missing on cards API, ” + ”transactions and receipt images will not be available", card.getReferenceId()); } }
111. @Test public void testCardsSuccess() { loginAndGetCards(testAccount); assertSuccessResponse(response); List<Card> cards = response.getCards(); assertFalse("No cards were returned for " + testAccount, cards.isEmpty()); for (Card card : cards) { assertNotNull( "reference_id is missing on cards API, ” + ”transactions and receipt images will not be available", card.getReferenceId()); } }
112. @Test public void testCardsSuccess() { loginAndGetCards(testAccount); assertSuccessResponse(response); List<Card> cards = response.getCards(); assertFalse("No cards were returned for " + testAccount, cards.isEmpty()); for (Card card : cards) { assertNotNull( "reference_id is missing on cards API, ” + ”transactions and receipt images will not be available", card.getReferenceId()); } }
114. • Check the API works as expected • Add useful assertions • Use them for environment monitoring! 05 Integration Test Playbook
116. Pros • Trivial to write if you have component tests • Automatically diagnose API issues • Easy to turn into API monitoring tools • Useful for validating older client versions still work Cons • Fail if your APIs/environment are flakey 05 Integration Tests
117. CASE STUDY WALLET
118. Capital One Wallet Git Commits Pull Request PR Merged Assemble APKs and Test APKs Integration Tests Component Tests Component UI Tests Blackbox Tests ✓ ✓ ✓ ✓ ✓ ✓ Component UI Tests Unit Tests Component Tests ✓ ✓ ✓ Unit Tests ✓
120. 05 Our Testing Pyramid QA team Dev team
121. GOLDEN RULES
122. • Every bug needs an automated test (Production, QA, master) • Write tests at the same time as coding • Write a failing test before making it pass • Write tests as low in the testing pyramid as possible • Run tests before merging pull requests 05 Golden Rules
123. JOIN OUR TEAM
124. 05 Capital One Wallet • Building a great app • Launch partner for FourSquare Pilgrim SDK • Top rated finance app by a US bank • First US bank with native Android contactless payments • Passionate about • Automated testing and continuous integration • Learning and sharing
125. THANK YOU