Testing apps at Facebook

Talk at the Droidcon Berlin Barcamp in May 2014

Transcript

1. View Slide

2. Testing Android applications
   Martin Konicek
   Software Engineer

   fb.me/martin.konicek1

   May 10, 2014
   

   View Slide

3. View Slide

4. View Slide

5. Unit tests
   As a first line of defense
   !
   !
   !
   !
   !
   $ buck test javatests/com/facebook/timeline/
   PASS 10 Passed 0 Failed TimelineDbCacheTest
   PASS 16 Passed 0 Failed TimelineRamCacheTest
 PASS 4 Passed 0 Failed
   FetchTimelineHeaderMethodTest
 ...
 TESTS PASSED

   

   View Slide

6. Unit tests
   As a first line of defense
   !
   !
   !
   Mocked:
   ▪ The OS
   ▪ Backend
   $ buck test javatests/com/facebook/timeline/
   PASS 10 Passed 0 Failed TimelineDbCacheTest
   PASS 16 Passed 0 Failed TimelineRamCacheTest
 PASS 4 Passed 0 Failed
   FetchTimelineHeaderMethodTest
 ...
 TESTS PASSED

   

   View Slide

7. End-to-end testing
   

   View Slide

8. Instrumentation
   !
   ▪ Toolkits:
   ▪ Robotium
   ▪ Espresso
   ▪ Test code runs on the device

   

   View Slide

9. Selendroid
   !
   ▪ Embedded HTTP server running on the device
   ▪ JUnit test runs locally
   

   View Slide

10. !
    !
    

    public class AudiencePickerTest {
    !
    private WebDriver driver;
    private Wait wait;
    private NewsFeed newsFeed;
    !
    @Before
    public void login() {
    driver = DroidDriver.create();
    wait = new DroidWait(mDriver, 20, TimeUnit.SECONDS);
    !
    TestUser user = TestUser.create();
    newsFeed = new Login(driver).as(user);
    }

    !
    !
    !
    

    View Slide

11. @Test
    public void changePrivacy() {
    AudiencePicker audiencePicker = newsFeed
    .goToStatusComposer()
    .openAudiencePicker();
    !
    assertEquals(
    "Default audience should be public",
    Audience.PUBLIC,
    audiencePicker.getSelectedAudience());
    !
    StatusComposer composer = audiencePicker
    .selectAudience(Audience.FRIENDS);
    composer.type("Droidcon Berlin is the best!").post();
    !
    // Fetch the story
    FeedStory story = wait.until(GraphQL.visibilityOfStory(user));
    !
    assertEquals(
    "Story audience should respect selection",
    Audience.FRIENDS,
    story.getAudience());
    }

    

    View Slide

12. View Slide

13. Fast feedback
    Before pushing to master
    

    View Slide

14. Performance testing
    

    View Slide

15. View Slide

16. View Slide

17. Discovering hard-to-find regressions
    

    View Slide

18. Looking for glitches
    !
    ▪ FPS
    ▪ Skipped frames (android.os.Choreographer)
    $ adb logcat
    

    04-09 17:02:15.878 I/Choreographer(10122): Skipped 3 frames!
...

    04-09 17:02:17.109 I/Choreographer(10122): Skipped 2 frames!
    ...
    

    View Slide

19. Measuring memory, network,
    battery usage
    

    View Slide

20. Measuring memory, network, battery usage
    !
    ▪ adb shell dumpsys
    ▪ reading system files using adb

    

    View Slide

21. Measuring time
    

    View Slide

22. View Slide

23. Measuring time
    Challenges

    

    


    The app provides timing information

    

    $ adb logcat | grep CommentsLoad
    !
    D/fb4a:PerformanceLogger: Name: CommentsLoad; Timestamp: 9318686
    D/fb4a:PerformanceLogger: Name: CommentsLoad; Timestamp: 9318722; Elapsed: 36

    ...
    

    View Slide

24. Measuring time
    Challenges
    $ adb logcat | grep CommentsLoad | grep Elapsed | awk -F';' '{print $3}'

    Elapsed: 46
    Elapsed: 66
    Elapsed: 76
    Elapsed: 36
    Elapsed: 40
    Elapsed: 83
    Elapsed: 34
    Elapsed: 36
    Elapsed: 33
    

    View Slide

25. Measuring time
    Challenges
    ▪ Garbage collection
    ▪ Thread scheduling

    

    !
    !
    ▪ Network
    ▪ Overhead of measuring code

    

    

    View Slide

26. Measuring time
    Challenges
    ▪ Garbage collection
    ▪ Thread scheduling

    

    !
    !
    ▪ Network
    ▪ Overhead of measuring code

    

    

    View Slide

27. @RunWith(PerformanceTestRunner.class)
    @TestRepetitions(50)
    public class CommentsLoadTest {
    !
    ...
    !
    @BeforeRepetition
    public void loginUser() {
    ...

    driver.disableNewsFeedAutoRefresh();
    new Login(driver).as(user);
    }
    !
    @Test
    public void openComments() {
    // Click the first "n comments" TextView in the News Feed
    WebElement feedbackTextView = wait.until(
    visibilityOfElement(By.id("feed_feedback_text")));
    driver.gc();
    feedbackTextView.click();
    waitForMarkers("CommentsLoad", 5, TimeUnit.SECONDS);
    }
    !
    @After
    public void goBackToNewsFeed() {
    driver.sendKeys(SelendroidKeys.BACK);
    }
    }
    

    View Slide

28. Time to load comments by device
    

    View Slide

29. Time to load comments by repetition
    Repetition
    

    View Slide

30. Profiling
    Traceview

    

    View Slide

31. Debugging allocations
    $ adb logcat

    

    ...

    05-08 06:13:49: D/dalvikvm(6759): GC_FOR_ALLOC freed 762K, 13% free 4116K/4688K, 

    paused 181ms, total 182ms
    !
    

    View Slide

32. Debugging Allocations
    Android Device Monitor

    

    View Slide

33. Debugging OOM Errors
    Eclipse Memory Analyzer

    

    View Slide

34. View Slide

35. View Slide

36. Questions
    

    View Slide