Boost the Quality of Your App with Firebase

Every time your app crashes or hangs, it’s an invitation for the user to uninstall it and give it a bad rating. Fortunately, Firebase provides the tools you need to keep the quality of your app high. Join this session to see how Firebase can help you during development and in production to understand where your app is crashing and where it’s performance is poor. We’ll take a look at Test Lab, Crash Reporting, Remote Config, and the newly released Performance Monitoring tools to gauge the quality of your app. With these, and some clever tricks, you can make sure your users have the best possible experience. Down with one-star ratings!

Transcript

1. Boost the Quality of Your App
   with Firebase
   Doug Stevenson
   @CodingDoug
   

   View Slide

2. 50%
   Stability & bugs
   When leaving a 1 star review, 50% of the time the user mentions
   the app stability and bugs
   Google analysis on Play reviews, top 10 English speaking countries, last 365 days of reviews
   

   View Slide

3. Prevention
   After dev & test cycle, it’s too
   late!
   

   View Slide

4. View Slide

5. View Slide

6. Firebase Test
   Lab for Android
   Test Artifacts
   • Logcat (crashes)
   • Videos
   • Screenshots
   • Performance

   CPU

   memory

   network ingress/egress
   

   View Slide

7. Instrumented Tests
   • Specific to Android
   • Focus on the user story / use case
   • Espresso / UI Automator / Robotium
   • Adds extensions to JUnit
   • Android Testing Support Library: goo.gl/xSxzoj
   • Testing Codelab: goo.gl/RHdFBY
   

   View Slide

8. @RunWith(AndroidJUnit4.class)
   public class NotesScreenTest {
   @Rule
   public ActivityTestRule mNotesActivityTestRule =
   new ActivityTestRule<>(NotesActivity.class);
   @Test
   public void clickAddNoteButton_opensAddNoteUi() throws Exception {
   // Click on the add note button
   onView(withId(R.id.fab_add_notes)).perform(click());
   // Check if the add note screen is displayed
   onView(withId(R.id.add_note_title)).check(matches(isDisplayed()));
   }
   }
   

   View Slide

9. (Local) Unit Tests
   • Runs fast on your computer
   • Focus on testing individual classes and methods
   • Dependency injection / object mocking
   

   View Slide

10. Android Instrumented Tests
    Local Unit Tests
    

    View Slide

11. Protip: Run your unit tests along with instrumented tests
    android {

    sourceSets {

    androidTest {

    java.srcDirs += "src/test/java"

    }

    }

    }
    testCompile 'org.mockito:mockito-core:2.8.47'
    androidTestCompile 'org.mockito:mockito-android:2.8.47'
    

    View Slide

12. Protip: Enable StrictMode when running in Test Lab
    Turn StrictMode violations into actionable crash reports in Test Lab.
    

    View Slide

13. private void enableStrictMode() {
    if (shouldEnableStrictMode()) {
    StrictMode.setThreadPolicy(new StrictMode.ThreadPolicy.Builder()
    .detectAll()
    .penaltyLog()
    .penaltyDeath()
    .build());
    StrictMode.setVmPolicy(new StrictMode.VmPolicy.Builder()
    .detectAll()
    .penaltyLog()
    .penaltyDeath()
    .build());
    }
    }
    

    View Slide

14. private boolean shouldEnableStrictMode() {
    return BuildConfig.DEBUG;
    }
    

    View Slide

15. private boolean shouldEnableStrictMode() {
    ContentResolver resolver = context.getContentResolver();
    String isInTestLab =
    Settings.System.getString(resolver, "firebase.test.lab");
    return BuildConfig.DEBUG || "true".equals(isInTestLab);
    }
    

    View Slide

16. // https://issuetracker.google.com/issues/36951662
    private void workaroundEnableStrictMode() {
    if (Build.VERSION.SDK_INT >= 9) {
    enableStrictMode();
    }
    if (Build.VERSION.SDK_INT >= 16) {
    // restore strict mode after onCreate() returns.
    new Handler().postAtFrontOfQueue(new Runnable() {
    @Override
    public void run() {
    enableStrictMode();
    }
    });
    }
    }
    

    View Slide

17. But I don’t want to write tests
    

    View Slide

18. Espresso Test Recorder
    • Android Studio

    Menu → Run → ⚫︎Record Espresso Test
    • Will generate Espresso code to match your actions
    

    View Slide

19. View Slide

20. View Slide

21. But I don’t even want to have tests
    

    View Slide

22. View Slide

23. Robo Test
    • Only an APK with is required
    • Automatically logs in with Google Auth
    • Can pre-fill form fields (e.g. user/pass login, search)
    • Configure maximum crawl depth and timeout
    • Algorithm improves over time
    • Additional test artifact: Activity map
    

    View Slide

24. But I’m building a game
    

    View Slide

25. View Slide

26. Game Loop Test (beta)
    • Sequence of scenarios driven by Android Intents
    • Performance += FPS
    • Game Loop Codelab: goo.gl/sn4RNp
    • “Mechahamster” sample game
    • Project home: goo.gl/8chQ8p
    • Test Lab Guide: goo.gl/16xSPn
    

    View Slide

27. View Slide

28. View Slide

29. View Slide

30. View Slide

31. gcloud firebase test android run \
    --type instrumentation \
    --app app-debug-unaligned.apk \
    --test app-debug-test-unaligned.apk \
    --device model=Nexus6,version=21,orientation=portrait \
    --device model=Nexus7,version=19,orientation=landscape
    

    View Slide

32. Community Support for testing with CI
    • Walmart’s “Flank” for sharding tests

    goo.gl/5ApxVU
    • Circle CI integration

    goo.gl/xoPFqY
    

    View Slide

33. View Slide

34. Billing for Firebase Test Lab
    • Unpaid projects: daily limits
    • Virtual devices: 10 tests/day
    • Physical devices: 5 tests/day
    • Paid projects: per-minute billing
    • Virtual devices: $1/device/hour
    • Physical devices: $5/device/hour
    

    View Slide

35. View Slide

36. View Slide

37. View Slide

38. View Slide

39. View Slide

40. Confidential + Proprietary
    ● Unity support
    ● NDK support
    ● Search
    ● Webhooks
    Crashlytics Firebase Crash Reporting
    ● Tightly integrated with Analytics
    ○ Generates “app_exception”
    events
    ○ Analytics events in crash logs
    

    View Slide

41. public class MyHandler implements Thread.UncaughtExceptionHandler {
    private final Thread.UncaughtExceptionHandler mPriorExceptionHandler;
    public MyHandler(Thread.UncaughtExceptionHandler prior) {
    mPriorExceptionHandler = prior;
    }
    @Override
    public void uncaughtException(Thread thread, Throwable throwable) {
    // Deal with throwable here
    if (mPriorExceptionHandler != null) {
    mPriorExceptionHandler.uncaughtException(thread, throwable);
    }
    }
    }
    

    View Slide

42. Thread.UncaughtExceptionHandler prior =
    Thread.getDefaultUncaughtExceptionHandler();
    Thread.setDefaultUncaughtExceptionHandler(
    new MyUncaughtExceptionHandler(prior));
    

    View Slide

43. View Slide

44. Performance == Quality
    

    View Slide

45. 60%
    Speed, design & usability
    When leaving a 5 star review, 60% of the time the user mentions
    the speed, design or usability
    Google analysis on Play reviews, top 10 English speaking countries, last 365 days of reviews
    

    View Slide

46. https://goo.gl/BnWcPK
    

    View Slide

47. Systrace
    Advantages
    • Tons of detail
    • Custom traces
    Disadvantages
    • App runs slow
    • You typically have to know what you're looking for
    • Can't collect data from the wild, only connected devices
    

    View Slide

48. View Slide

49. Firebase Performance Monitoring
    • Runs in production
    • Negligible overhead
    • Automatic and Custom traces
    • Automatic HTTP/S traffic metrics
    

    View Slide

50. Trace
    • A report for a period of time with a well-defined beginning and end
    • Nominally has a duration
    • Also may contain "counters" for performance-related events during the trace
    

    View Slide

51. Automatic Traces - no coding necessary!
    • App Start (cold)
    • Time in foreground
    • Time in background
    

    View Slide

52. Automatic Traces - App Start
    ContentProvider.onCreate() Activity.onResume()
    But you might want something different!
    

    View Slide

53. Counters
    • Associate a string/count value to a Trace
    • Typically better to measure ratios than absolute values
    • Abs value if you want to compare similar counters between traces
    cache_hit++
    cache_miss++
    dropped_frames += 10
    

    View Slide

54. An idea for pseudo-automatic traces
    

    View Slide

55. public class PerfLifecycleCallbacks
    implements Application.ActivityLifecycleCallbacks {
    private static final PerfLifecycleCallbacks instance =
    new PerfLifecycleCallbacks();
    private PerfLifecycleCallbacks() {}
    public static PerfLifecycleCallbacks getInstance() {
    return instance;
    }
    // Continued...
    

    View Slide

56. private final HashMap traces =
    new HashMap<>();
    @Override
    public void onActivityStarted(Activity activity) {
    String name = activity.getClass().getSimpleName();
    Trace trace = FirebasePerformance.startTrace(name);
    traces.put(activity, trace);
    }
    @Override
    public void onActivityStopped(Activity activity) {
    Trace trace = traces.remove(activity);
    trace.stop();
    }
    

    View Slide

57. public Trace getTrace(Activity activity) {
    return traces.get(activity);
    }
    

    View Slide

58. public class PerfInitContentProvider extends ContentProvider {
    @Override
    public boolean onCreate() {
    Context context = getContext();
    if (context != null) {
    Application app = (Application) context.getApplicationContext();
    app.registerActivityLifecycleCallbacks(
    PerfLifecycleCallbacks.getInstance());
    }
    return false;
    }
    // Overrides elided...
    }
    

    View Slide

59. public class MyActivity extends Activity {
    private void loadStuff() {
    Trace trace = PerfLifecycleCallbacks.getInstance().getTrace(this);
    trace.incrementCounter("foo");
    }
    }
    

    View Slide

60. Android Vitals - get clues for what to measure
    • New in the Google Play Console: goo.gl/rkztG1
    • Performance-related stats
    • Slow rendering (jank)
    • Frozen frames
    • Stuck WakeLocks
    

    View Slide

61. Automatic HTTP/S transaction metrics
    • Response time
    • Payload size
    • Success rate
    • URL pattern globbing

    

    yourcdn.com/*.jpg

    api.yourdomain.com/v1/users/*

    api.yourdomain.com/v1/users/*/history/*
    

    View Slide

62. HTTP/S transaction metrics breakdown
    • App version
    • Device
    • Country
    • OS level
    • Carrier
    • Radio
    

    View Slide

63. How HTTP/S monitoring works
    Bytecode manipulation (with ASM)
    URL url = new URL("https://firebase.google.com");

    HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
    URL url = new URL(“https://firebase.google.com”);

    HttpsURLConnection conn = (HttpsURLConnection)

    FirebasePerfUrlConnection.instrument(url.openConnection());
    Decorator inside!
    

    View Slide

64. How HTTP/S monitoring works
    dependencies {

    compile 'com.google.firebase:firebase-perf:11.0.2'

    }
    apply plugin: 'com.google.firebase.firebase-perf'
    Uses the Transform API: goo.gl/PwBcLE
    

    View Slide

65. View Slide

66. Firebase Remote Config
    • Use it to plan and stage rollouts of new features
    • Quickly back off the new feature if it has problems
    

    View Slide

67. View Slide

68. View Slide

69. View Slide

70. View Slide

71. FirebaseRemoteConfig rc = FirebaseRemoteConfig.getInstance();
    if (rc.getBoolean("using_cool_new_feature")) {
    // do cool stuff
    }
    else {
    // do the boring old stuff
    }
    

    View Slide

72. Firebase Remote Config
    • Find out what your users prefer by experimentation
    • Study the results in Google Analytics for Firebase
    

    View Slide

73. View Slide

74. Firebase Remote Config (for perf)
    • Compare CDN/server performance
    • Compare cache configuration strategies
    

    View Slide

75. final FirebaseRemoteConfig rc = FirebaseRemoteConfig.getInstance();
    int cache_size = (int) rc.getLong("cache_size");
    // init cache with cache_size
    // later...
    Trace trace = FirebasePerformance.startTrace("my_trace");
    Item item = cache.get("some_resource");
    if (item != null) {
    trace.incrementCounter("cache_hit_size_" + cache_size);
    }
    else {
    trace.incrementCounter("cache_miss_size_" + cache_size);
    }
    trace.stop();
    

    View Slide

76. Thank you!
    Confidential + Proprietary
    Doug Stevenson
    @CodingDoug
    youtube.com/firebase
    firebase.googleblog.com
    

    View Slide