Testing apps at Facebook

Talk at the Droidcon Berlin Barcamp in May 2014

Transcript

2. Testing Android applications Martin Konicek Software Engineer
 fb.me/martin.konicek1
 May 10, 2014
5. Unit tests As a first line of defense ! ! ! ! ! $ buck test javatests/com/facebook/timeline/ PASS 10 Passed 0 Failed TimelineDbCacheTest PASS 16 Passed 0 Failed TimelineRamCacheTest
 PASS 4 Passed 0 Failed FetchTimelineHeaderMethodTest
 ...
 TESTS PASSED

6. Unit tests As a first line of defense ! ! ! Mocked: ▪ The OS ▪ Backend $ buck test javatests/com/facebook/timeline/ PASS 10 Passed 0 Failed TimelineDbCacheTest PASS 16 Passed 0 Failed TimelineRamCacheTest
 PASS 4 Passed 0 Failed FetchTimelineHeaderMethodTest
 ...
 TESTS PASSED

7. End-to-end testing
8. Instrumentation ! ▪ Toolkits: ▪ Robotium ▪ Espresso ▪ Test code runs on the device

9. Selendroid ! ▪ Embedded HTTP server running on the device ▪ JUnit test runs locally
10. ! ! 
 public class AudiencePickerTest { ! private WebDriver driver; private Wait<WebDriver> wait; private NewsFeed newsFeed; ! @Before public void login() { driver = DroidDriver.create(); wait = new DroidWait(mDriver, 20, TimeUnit.SECONDS); ! TestUser user = TestUser.create(); newsFeed = new Login(driver).as(user); }
 ! ! !
11. @Test public void changePrivacy() { AudiencePicker audiencePicker = newsFeed .goToStatusComposer() .openAudiencePicker(); ! assertEquals( "Default audience should be public", Audience.PUBLIC, audiencePicker.getSelectedAudience()); ! StatusComposer composer = audiencePicker .selectAudience(Audience.FRIENDS); composer.type("Droidcon Berlin is the best!").post(); ! // Fetch the story FeedStory story = wait.until(GraphQL.visibilityOfStory(user)); ! assertEquals( "Story audience should respect selection", Audience.FRIENDS, story.getAudience()); }

13. Fast feedback Before pushing to master
14. Performance testing
17. Discovering hard-to-find regressions
18. Looking for glitches ! ▪ FPS ▪ Skipped frames (android.os.Choreographer) $ adb logcat 
 04-09 17:02:15.878 I/Choreographer(10122): Skipped 3 frames!
...
 04-09 17:02:17.109 I/Choreographer(10122): Skipped 2 frames! ...
19. Measuring memory, network, battery usage
20. Measuring memory, network, battery usage ! ▪ adb shell dumpsys ▪ reading system files using adb

21. Measuring time
23. Measuring time Challenges
 
 

 The app provides timing information
 
 $ adb logcat | grep CommentsLoad ! D/fb4a:PerformanceLogger: Name: CommentsLoad; Timestamp: 9318686 D/fb4a:PerformanceLogger: Name: CommentsLoad; Timestamp: 9318722; Elapsed: 36
 ...
24. Measuring time Challenges $ adb logcat | grep CommentsLoad | grep Elapsed | awk -F';' '{print $3}'
 Elapsed: 46 Elapsed: 66 Elapsed: 76 Elapsed: 36 Elapsed: 40 Elapsed: 83 Elapsed: 34 Elapsed: 36 Elapsed: 33
25. Measuring time Challenges ▪ Garbage collection ▪ Thread scheduling
 
 ! ! ▪ Network ▪ Overhead of measuring code
 

26. Measuring time Challenges ▪ Garbage collection ▪ Thread scheduling
 
 ! ! ▪ Network ▪ Overhead of measuring code
 

27. @RunWith(PerformanceTestRunner.class) @TestRepetitions(50) public class CommentsLoadTest { ! ... ! @BeforeRepetition public void loginUser() { ...
 driver.disableNewsFeedAutoRefresh(); new Login(driver).as(user); } ! @Test public void openComments() { // Click the first "n comments" TextView in the News Feed WebElement feedbackTextView = wait.until( visibilityOfElement(By.id("feed_feedback_text"))); driver.gc(); feedbackTextView.click(); waitForMarkers("CommentsLoad", 5, TimeUnit.SECONDS); } ! @After public void goBackToNewsFeed() { driver.sendKeys(SelendroidKeys.BACK); } }
28. Time to load comments by device
29. Time to load comments by repetition Repetition
30. Profiling Traceview

31. Debugging allocations $ adb logcat
 
 ...
 05-08 06:13:49: D/dalvikvm(6759): GC_FOR_ALLOC freed 762K, 13% free 4116K/4688K, 
 paused 181ms, total 182ms !
32. Debugging Allocations Android Device Monitor

33. Debugging OOM Errors Eclipse Memory Analyzer

36. Questions