Automated Testing @Square - Droidcon NYC 2014

Scratching the surface on what appears to be a very difficult problem to get right on Android. Goals, techniques and useful tips to get started.

Transcript

1. Automated Testing Dimitris K.
2. Quality Brand Automated Testing
3. Automated Testing Extremely important to us
4. Quality Yesterday Automated Testing
5. Quality Yesterday Manual Testing Automated Testing
6. Unit Testing Quality Yesterday Automated Testing
7. Quality Yesterday “Add ability to compile and run tests” - Eric Burke Automated Testing
8. Quality Today Automated Testing
9. No Flakiness Quality Today Automated Testing
10. Easy to write tests Quality Today Automated Testing
11. Catch edge cases Quality Today Automated Testing
12. Quality Today Automated Testing Graybox testing is preferred
13. Quality Today Automated Testing Tests are red == master is red
14. Robots Automated Testing
15. Robots Thin layer above UI framework Automated Testing Fluent API Single point to update your code
16. Robots public class InvoiceScreenRobot extends ScreenRobot { public InvoiceScreenRobot typeTitle(String title) { onView(withId(R.id.title)).perform(typeText(title)); return this; } // More here... public void clickSend() { onView(withId(R.id.send)).perform(click()); } } Automated Testing
17. Robots public void testSendInvoice() { withRobot(InvoiceScreenRobot.class) // .typeTitle("Sample Title") // .typeNumber("#000001") // .typeName("Square User") // .typeEmail("squareuser@squareup.com") // .clickSend(); } Automated Testing
18. Robots public void testSendInvoice() { withRobot(InvoiceScreenRobot.class) // .checkIsDisabled(R.id.send) // .typeTitle("Sample Title") // .typeNumber("#000001") // .typeName("Square User") // .typeEmail("squareuser@squareup.com") // .checkIsEnabled(R.id.send) // .clickSend(); } Automated Testing
19. Mock Mode Automated Testing
20. Mock Mode Eliminate network flakiness Automated Testing Rapid development Test network edge cases
21. Mock Mode @Provides @Singleton MyService provideMyService(RestAdapter adapter) { return adapter.create(MyService.class); } Automated Testing
22. Mock Mode @Provides @Singleton MyService provideMyService(RestAdapter adapter) { MyService real = restAdapter.create(MyService.class); MyService mock = new MockMyService(); return ServiceProxy.newInstance(MyService.class, mock, real); } Automated Testing
23. Mock Mode public class ServiceProxy implements InvocationHandler { @Override public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { Object service = mockModeOn() ? mock : real; try { return method.invoke(service, args); } catch (InvocationTargetException e) { throw e.getTargetException(); } } Automated Testing
24. Mock Mode public class MockMyService implements MyService { @Override public Response doSomething(Request request) { delay(); boolean success = true; return new Response(success); } } Automated Testing
25. Test Matrix Automated Testing
26. Test Matrix Automated Testing Card Not Present Flow
27. Test Matrix 7 card types Automated Testing 4 languages 4 receipt types
28. Test Matrix Automated Testing 112 tests to write
29. Burst Automated Testing
30. Burst Test Generator Automated Testing
31. Burst Variation Proximity Automated Testing
32. Burst Type Safety Automated Testing
33. Burst Use Everywhere Automated Testing
34. Burst Automated Testing Not @Parameterized.Parameters(name = "{0}") public static List<Object[]> parameters() { return Arrays.asList(new Object[] { new Factory() { @Override public BufferedSink create(Buffer data) { return data; } @Override public String toString() { return "Buffer"; } }, }, new Object[] { new Factory() { @Override public BufferedSink create(Buffer data) { return new RealBufferedSink(data); } @Override public String toString() { return "RealBufferedSink"; } } }); } @Parameterized.Parameter public Factory factory;
35. Burst public enum Soda { COKE, PEPSI, RC_COLA } @RunWith(BurstJUnit4.class) public class SampleTest { @Test public void sodas(Soda soda) { // Test will run 3 times assertNotNull(soda); } } Automated Testing
36. Burst public enum Soda { COKE, PEPSI, RC_COLA } @RunWith(BurstJUnit4.class) public class SampleTest { @Test public void sodasAndSnacks(Soda soda, Snack snack) { // Test will run 9 times assertNotNull(soda); assertNotNull(snack); } } public enum Snack { CHIPS, KITKAT, CANDY } Automated Testing
37. Burst Friendly Test Naming Automated Testing
38. Burst sodasAndSnacks[Soda.COKE, Snack.CHIPS] sodasAndSnacks[Soda.COKE, Snack.KITKAT] sodasAndSnacks[Soda.COKE, Snack.CANDY] Automated Testing
39. Burst @TabletOnly public void testTabletFeature() { // This test will only be executed on tablets. } Automated Testing
40. Burst @TabletOnly public void testTabletFeature() { // This test will only be executed on tablets. } @PhoneOnly public void testPhoneFeature() { // This test will only be executed on phones. } Automated Testing
41. Automated Testing @TabletOnly public void testTipBringsAmountOver25() { loginAsAndStartActivity("bart"); performAuthSteps(2499L); // Tip withRobot(TipScreenRobot.class).clickTipOptionOne(); // Sign withRobot(SignScreenRobot.class) .performSign() .clickContinue(); // Receipt withRobot(ReceiptScreenRobot.class) .performEmailReceipt("bart@squareup.com") .clickNewSale(); }
42. Tips Automated Testing
43. Tips Animations OFF Automated Testing
44. Don’t Sleep() Tips Automated Testing
45. Tips Screenshots Automated Testing
46. Tips Automated Testing @Override protected void setUp() throws Exception { super.setUp(); setFailureHandler(new RegisterFailureHandler(this)); }
47. Tips Automated Testing public class RegisterFailureHandler implements FailureHandler { private final TestCase testCase; public RegisterFailureHandler(TestCase testCase) { this.testCase = testCase; } @Override public void handle(Throwable throwable, Matcher<View> viewMatcher) { // Very useful for finding out what went wrong... takeAndUploadScreenshot(testCase.getName()); // Delegate to real failure handler here. } }
48. Tips Automated Testing public class ScreenShotProxy implements InvocationHandler { @Override public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { try { Object result = method.invoke(proxy, args); takeScreenShot(); // <----- magic. return result; } catch (InvocationTargetException e) { throw e.getTargetException(); } } }
49. Tips Keyboard :( Automated Testing
50. A https://github.com/square/burst https://squareup.com/careers https://github.com/square/burst