Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Automated Testing @Square - Droidcon NYC 2014

Dimitris
September 22, 2014

Automated Testing @Square - Droidcon NYC 2014

Scratching the surface on what appears to be a very difficult problem to get right on Android. Goals, techniques and useful tips to get started.

Dimitris

September 22, 2014
Tweet

Other Decks in Technology

Transcript

  1. Automated Testing
    Dimitris K.

    View Slide

  2. Quality Brand
    Automated Testing

    View Slide

  3. Automated Testing
    Extremely important to us

    View Slide

  4. Quality Yesterday
    Automated Testing

    View Slide

  5. Quality Yesterday
    Manual Testing
    Automated Testing

    View Slide

  6. Unit Testing
    Quality Yesterday
    Automated Testing

    View Slide

  7. Quality Yesterday
    “Add ability to compile and run tests”
    - Eric Burke
    Automated Testing

    View Slide

  8. Quality Today
    Automated Testing

    View Slide

  9. No Flakiness
    Quality Today
    Automated Testing

    View Slide

  10. Easy to write tests
    Quality Today
    Automated Testing

    View Slide

  11. Catch edge cases
    Quality Today
    Automated Testing

    View Slide

  12. Quality Today
    Automated Testing
    Graybox testing is preferred

    View Slide

  13. Quality Today
    Automated Testing
    Tests are red == master is red

    View Slide

  14. Robots
    Automated Testing

    View Slide

  15. Robots
    Thin layer above UI framework
    Automated Testing
    Fluent API
    Single point to update your code

    View Slide

  16. Robots
    public class InvoiceScreenRobot extends ScreenRobot {
    public InvoiceScreenRobot typeTitle(String title) {
    onView(withId(R.id.title)).perform(typeText(title));
    return this;
    }
    // More here...
    public void clickSend() {
    onView(withId(R.id.send)).perform(click());
    }
    }
    Automated Testing

    View Slide

  17. Robots
    public void testSendInvoice() {
    withRobot(InvoiceScreenRobot.class) //
    .typeTitle("Sample Title") //
    .typeNumber("#000001") //
    .typeName("Square User") //
    .typeEmail("[email protected]") //
    .clickSend();
    }
    Automated Testing

    View Slide

  18. Robots
    public void testSendInvoice() {
    withRobot(InvoiceScreenRobot.class) //
    .checkIsDisabled(R.id.send) //
    .typeTitle("Sample Title") //
    .typeNumber("#000001") //
    .typeName("Square User") //
    .typeEmail("[email protected]") //
    .checkIsEnabled(R.id.send) //
    .clickSend();
    }
    Automated Testing

    View Slide

  19. Mock Mode
    Automated Testing

    View Slide

  20. Mock Mode
    Eliminate network flakiness
    Automated Testing
    Rapid development
    Test network edge cases

    View Slide

  21. Mock Mode
    @Provides @Singleton MyService provideMyService(RestAdapter adapter) {
    return adapter.create(MyService.class);
    }
    Automated Testing

    View Slide

  22. Mock Mode
    @Provides @Singleton
    MyService provideMyService(RestAdapter adapter) {
    MyService real = restAdapter.create(MyService.class);
    MyService mock = new MockMyService();
    return ServiceProxy.newInstance(MyService.class, mock, real);
    }
    Automated Testing

    View Slide

  23. Mock Mode
    public class ServiceProxy implements InvocationHandler {
    @Override public Object invoke(Object proxy, Method
    method, Object[] args) throws Throwable {
    Object service = mockModeOn() ? mock : real;
    try {
    return method.invoke(service, args);
    } catch (InvocationTargetException e) {
    throw e.getTargetException();
    }
    }
    Automated Testing

    View Slide

  24. Mock Mode
    public class MockMyService implements MyService {
    @Override public Response doSomething(Request request) {
    delay();
    boolean success = true;
    return new Response(success);
    }
    }
    Automated Testing

    View Slide

  25. Test Matrix
    Automated Testing

    View Slide

  26. Test Matrix
    Automated Testing
    Card Not Present Flow

    View Slide

  27. Test Matrix
    7 card types
    Automated Testing
    4 languages
    4 receipt types

    View Slide

  28. Test Matrix
    Automated Testing
    112 tests to write

    View Slide

  29. Burst
    Automated Testing

    View Slide

  30. Burst
    Test Generator
    Automated Testing

    View Slide

  31. Burst
    Variation Proximity
    Automated Testing

    View Slide

  32. Burst
    Type Safety
    Automated Testing

    View Slide

  33. Burst
    Use Everywhere
    Automated Testing

    View Slide

  34. Burst
    Automated Testing
    Not
    @Parameterized.Parameters(name = "{0}")
    public static List parameters() {
    return Arrays.asList(new Object[] {
    new Factory() {
    @Override public BufferedSink create(Buffer data) {
    return data;
    }
    @Override public String toString() {
    return "Buffer";
    }
    },
    }, new Object[] {
    new Factory() {
    @Override public BufferedSink create(Buffer data) {
    return new RealBufferedSink(data);
    }
    @Override public String toString() {
    return "RealBufferedSink";
    }
    }
    });
    }
    @Parameterized.Parameter
    public Factory factory;

    View Slide

  35. Burst
    public enum Soda {
    COKE,
    PEPSI,
    RC_COLA
    }
    @RunWith(BurstJUnit4.class)
    public class SampleTest {
    @Test public void sodas(Soda soda) {
    // Test will run 3 times
    assertNotNull(soda);
    }
    }
    Automated Testing

    View Slide

  36. Burst
    public enum Soda {
    COKE,
    PEPSI,
    RC_COLA
    }
    @RunWith(BurstJUnit4.class)
    public class SampleTest {
    @Test public void sodasAndSnacks(Soda soda,
    Snack snack) {
    // Test will run 9 times
    assertNotNull(soda);
    assertNotNull(snack);
    }
    }
    public enum Snack {
    CHIPS,
    KITKAT,
    CANDY
    }
    Automated Testing

    View Slide

  37. Burst
    Friendly Test Naming
    Automated Testing

    View Slide

  38. Burst
    sodasAndSnacks[Soda.COKE, Snack.CHIPS]
    sodasAndSnacks[Soda.COKE, Snack.KITKAT]
    sodasAndSnacks[Soda.COKE, Snack.CANDY]
    Automated Testing

    View Slide

  39. Burst
    @TabletOnly
    public void testTabletFeature() {
    // This test will only be executed on tablets.
    }
    Automated Testing

    View Slide

  40. Burst
    @TabletOnly
    public void testTabletFeature() {
    // This test will only be executed on tablets.
    }
    @PhoneOnly
    public void testPhoneFeature() {
    // This test will only be executed on phones.
    }
    Automated Testing

    View Slide

  41. Automated Testing
    @TabletOnly
    public void testTipBringsAmountOver25() {
    loginAsAndStartActivity("bart");
    performAuthSteps(2499L);
    // Tip
    withRobot(TipScreenRobot.class).clickTipOptionOne();
    // Sign
    withRobot(SignScreenRobot.class)
    .performSign()
    .clickContinue();
    // Receipt
    withRobot(ReceiptScreenRobot.class)
    .performEmailReceipt("[email protected]")
    .clickNewSale();
    }

    View Slide

  42. Tips
    Automated Testing

    View Slide

  43. Tips
    Animations OFF
    Automated Testing

    View Slide

  44. Don’t Sleep()
    Tips
    Automated Testing

    View Slide

  45. Tips
    Screenshots
    Automated Testing

    View Slide

  46. Tips
    Automated Testing
    @Override protected void setUp() throws Exception {
    super.setUp();
    setFailureHandler(new RegisterFailureHandler(this));
    }

    View Slide

  47. Tips
    Automated Testing
    public class RegisterFailureHandler implements FailureHandler {
    private final TestCase testCase;
    public RegisterFailureHandler(TestCase testCase) {
    this.testCase = testCase;
    }
    @Override public void handle(Throwable throwable,
    Matcher viewMatcher) {
    // Very useful for finding out what went wrong...
    takeAndUploadScreenshot(testCase.getName());
    // Delegate to real failure handler here.
    }
    }

    View Slide

  48. Tips
    Automated Testing
    public class ScreenShotProxy implements InvocationHandler {
    @Override
    public Object invoke(Object proxy, Method method, Object[]
    args) throws Throwable {
    try {
    Object result = method.invoke(proxy, args);
    takeScreenShot(); // <----- magic.
    return result;
    } catch (InvocationTargetException e) {
    throw e.getTargetException();
    }
    }
    }

    View Slide

  49. Tips
    Keyboard :(
    Automated Testing

    View Slide

  50. A
    https://github.com/square/burst
    https://squareup.com/careers
    https://github.com/square/burst

    View Slide