Boost the Quality of Your App with Firebase

Every time your app crashes or hangs, it’s an invitation for the user to uninstall it and give it a bad rating. Fortunately, Firebase provides the tools you need to keep the quality of your app high. Join this session to see how Firebase can help you during development and in production to understand where your app is crashing and where it’s performance is poor. We’ll take a look at Test Lab, Crash Reporting, Remote Config, and the newly released Performance Monitoring tools to gauge the quality of your app. With these, and some clever tricks, you can make sure your users have the best possible experience. Down with one-star ratings!

Transcript

1. Boost the Quality of Your App with Firebase Doug Stevenson @CodingDoug
2. 50% Stability & bugs When leaving a 1 star review, 50% of the time the user mentions the app stability and bugs Google analysis on Play reviews, top 10 English speaking countries, last 365 days of reviews
3. Prevention After dev & test cycle, it’s too late!
6. Firebase Test Lab for Android Test Artifacts • Logcat (crashes) • Videos • Screenshots • Performance
 CPU
 memory
 network ingress/egress
7. Instrumented Tests • Specific to Android • Focus on the user story / use case • Espresso / UI Automator / Robotium • Adds extensions to JUnit • Android Testing Support Library: goo.gl/xSxzoj • Testing Codelab: goo.gl/RHdFBY
8. @RunWith(AndroidJUnit4.class) public class NotesScreenTest { @Rule public ActivityTestRule<NotesActivity> mNotesActivityTestRule = new ActivityTestRule<>(NotesActivity.class); @Test public void clickAddNoteButton_opensAddNoteUi() throws Exception { // Click on the add note button onView(withId(R.id.fab_add_notes)).perform(click()); // Check if the add note screen is displayed onView(withId(R.id.add_note_title)).check(matches(isDisplayed())); } }
9. (Local) Unit Tests • Runs fast on your computer • Focus on testing individual classes and methods • Dependency injection / object mocking
10. Android Instrumented Tests Local Unit Tests
11. Protip: Run your unit tests along with instrumented tests android {
 sourceSets {
 androidTest {
 java.srcDirs += "src/test/java"
 }
 }
 } testCompile 'org.mockito:mockito-core:2.8.47' androidTestCompile 'org.mockito:mockito-android:2.8.47'
12. Protip: Enable StrictMode when running in Test Lab Turn StrictMode violations into actionable crash reports in Test Lab.
13. private void enableStrictMode() { if (shouldEnableStrictMode()) { StrictMode.setThreadPolicy(new StrictMode.ThreadPolicy.Builder() .detectAll() .penaltyLog() .penaltyDeath() .build()); StrictMode.setVmPolicy(new StrictMode.VmPolicy.Builder() .detectAll() .penaltyLog() .penaltyDeath() .build()); } }
14. private boolean shouldEnableStrictMode() { return BuildConfig.DEBUG; }
15. private boolean shouldEnableStrictMode() { ContentResolver resolver = context.getContentResolver(); String isInTestLab = Settings.System.getString(resolver, "firebase.test.lab"); return BuildConfig.DEBUG || "true".equals(isInTestLab); }
16. // https://issuetracker.google.com/issues/36951662 private void workaroundEnableStrictMode() { if (Build.VERSION.SDK_INT >= 9) { enableStrictMode(); } if (Build.VERSION.SDK_INT >= 16) { // restore strict mode after onCreate() returns. new Handler().postAtFrontOfQueue(new Runnable() { @Override public void run() { enableStrictMode(); } }); } }
17. But I don’t want to write tests
18. Espresso Test Recorder • Android Studio
 Menu → Run → ⚫︎Record Espresso Test • Will generate Espresso code to match your actions
21. But I don’t even want to have tests
23. Robo Test • Only an APK with is required • Automatically logs in with Google Auth • Can pre-fill form fields (e.g. user/pass login, search) • Configure maximum crawl depth and timeout • Algorithm improves over time • Additional test artifact: Activity map
24. But I’m building a game
26. Game Loop Test (beta) • Sequence of scenarios driven by Android Intents • Performance += FPS • Game Loop Codelab: goo.gl/sn4RNp • “Mechahamster” sample game • Project home: goo.gl/8chQ8p • Test Lab Guide: goo.gl/16xSPn
31. gcloud firebase test android run \ --type instrumentation \ --app app-debug-unaligned.apk \ --test app-debug-test-unaligned.apk \ --device model=Nexus6,version=21,orientation=portrait \ --device model=Nexus7,version=19,orientation=landscape
32. Community Support for testing with CI • Walmart’s “Flank” for sharding tests
 goo.gl/5ApxVU • Circle CI integration
 goo.gl/xoPFqY
34. Billing for Firebase Test Lab • Unpaid projects: daily limits • Virtual devices: 10 tests/day • Physical devices: 5 tests/day • Paid projects: per-minute billing • Virtual devices: $1/device/hour • Physical devices: $5/device/hour
40. Confidential + Proprietary • Unity support • NDK support • Search • Webhooks Crashlytics Firebase Crash Reporting • Tightly integrated with Analytics ◦ Generates “app_exception” events ◦ Analytics events in crash logs
41. public class MyHandler implements Thread.UncaughtExceptionHandler { private final Thread.UncaughtExceptionHandler mPriorExceptionHandler; public MyHandler(Thread.UncaughtExceptionHandler prior) { mPriorExceptionHandler = prior; } @Override public void uncaughtException(Thread thread, Throwable throwable) { // Deal with throwable here if (mPriorExceptionHandler != null) { mPriorExceptionHandler.uncaughtException(thread, throwable); } } }
42. Thread.UncaughtExceptionHandler prior = Thread.getDefaultUncaughtExceptionHandler(); Thread.setDefaultUncaughtExceptionHandler( new MyUncaughtExceptionHandler(prior));
44. Performance == Quality
45. 60% Speed, design & usability When leaving a 5 star review, 60% of the time the user mentions the speed, design or usability Google analysis on Play reviews, top 10 English speaking countries, last 365 days of reviews
46. https://goo.gl/BnWcPK
47. Systrace Advantages • Tons of detail • Custom traces Disadvantages • App runs slow • You typically have to know what you're looking for • Can't collect data from the wild, only connected devices
49. Firebase Performance Monitoring • Runs in production • Negligible overhead • Automatic and Custom traces • Automatic HTTP/S traffic metrics
50. Trace • A report for a period of time with a well-defined beginning and end • Nominally has a duration • Also may contain "counters" for performance-related events during the trace
51. Automatic Traces - no coding necessary! • App Start (cold) • Time in foreground • Time in background
52. Automatic Traces - App Start ContentProvider.onCreate() Activity.onResume() But you might want something different!
53. Counters • Associate a string/count value to a Trace • Typically better to measure ratios than absolute values • Abs value if you want to compare similar counters between traces cache_hit++ cache_miss++ dropped_frames += 10
54. An idea for pseudo-automatic traces
55. public class PerfLifecycleCallbacks implements Application.ActivityLifecycleCallbacks { private static final PerfLifecycleCallbacks instance = new PerfLifecycleCallbacks(); private PerfLifecycleCallbacks() {} public static PerfLifecycleCallbacks getInstance() { return instance; } // Continued...
56. private final HashMap<Activity, Trace> traces = new HashMap<>(); @Override public void onActivityStarted(Activity activity) { String name = activity.getClass().getSimpleName(); Trace trace = FirebasePerformance.startTrace(name); traces.put(activity, trace); } @Override public void onActivityStopped(Activity activity) { Trace trace = traces.remove(activity); trace.stop(); }
57. public Trace getTrace(Activity activity) { return traces.get(activity); }
58. public class PerfInitContentProvider extends ContentProvider { @Override public boolean onCreate() { Context context = getContext(); if (context != null) { Application app = (Application) context.getApplicationContext(); app.registerActivityLifecycleCallbacks( PerfLifecycleCallbacks.getInstance()); } return false; } // Overrides elided... }
59. public class MyActivity extends Activity { private void loadStuff() { Trace trace = PerfLifecycleCallbacks.getInstance().getTrace(this); trace.incrementCounter("foo"); } }
60. Android Vitals - get clues for what to measure • New in the Google Play Console: goo.gl/rkztG1 • Performance-related stats • Slow rendering (jank) • Frozen frames • Stuck WakeLocks
61. Automatic HTTP/S transaction metrics • Response time • Payload size • Success rate • URL pattern globbing
 
 yourcdn.com/*.jpg
 api.yourdomain.com/v1/users/*
 api.yourdomain.com/v1/users/*/history/*
62. HTTP/S transaction metrics breakdown • App version • Device • Country • OS level • Carrier • Radio
63. How HTTP/S monitoring works Bytecode manipulation (with ASM) URL url = new URL("https://firebase.google.com");
 HttpsURLConnection conn = (HttpsURLConnection) url.openConnection(); URL url = new URL(“https://firebase.google.com”);
 HttpsURLConnection conn = (HttpsURLConnection)
 FirebasePerfUrlConnection.instrument(url.openConnection()); Decorator inside!
64. How HTTP/S monitoring works dependencies {
 compile 'com.google.firebase:firebase-perf:11.0.2'
 } apply plugin: 'com.google.firebase.firebase-perf' Uses the Transform API: goo.gl/PwBcLE
66. Firebase Remote Config • Use it to plan and stage rollouts of new features • Quickly back off the new feature if it has problems
71. FirebaseRemoteConfig rc = FirebaseRemoteConfig.getInstance(); if (rc.getBoolean("using_cool_new_feature")) { // do cool stuff } else { // do the boring old stuff }
72. Firebase Remote Config • Find out what your users prefer by experimentation • Study the results in Google Analytics for Firebase
74. Firebase Remote Config (for perf) • Compare CDN/server performance • Compare cache configuration strategies
75. final FirebaseRemoteConfig rc = FirebaseRemoteConfig.getInstance(); int cache_size = (int) rc.getLong("cache_size"); // init cache with cache_size // later... Trace trace = FirebasePerformance.startTrace("my_trace"); Item item = cache.get("some_resource"); if (item != null) { trace.incrementCounter("cache_hit_size_" + cache_size); } else { trace.incrementCounter("cache_miss_size_" + cache_size); } trace.stop();
76. Thank you! Confidential + Proprietary Doug Stevenson @CodingDoug youtube.com/firebase firebase.googleblog.com