Boost the Quality of Your App with Firebase

Boost the Quality of Your App with Firebase

Every time your app crashes or hangs, it’s an invitation for the user to uninstall it and give it a bad rating. Fortunately, Firebase provides the tools you need to keep the quality of your app high. Join this session to see how Firebase can help you during development and in production to understand where your app is crashing and where it’s performance is poor. We’ll take a look at Test Lab, Crash Reporting, Remote Config, and the newly released Performance Monitoring tools to gauge the quality of your app. With these, and some clever tricks, you can make sure your users have the best possible experience. Down with one-star ratings!

3acd4fb373289e71fd7ebfb287a75a3b?s=128

Doug Stevenson

July 14, 2017
Tweet

Transcript

  1. 2.

    50% Stability & bugs When leaving a 1 star review,

    50% of the time the user mentions the app stability and bugs Google analysis on Play reviews, top 10 English speaking countries, last 365 days of reviews
  2. 4.
  3. 5.
  4. 6.

    Firebase Test Lab for Android Test Artifacts • Logcat (crashes)

    • Videos • Screenshots • Performance
 CPU
 memory
 network ingress/egress
  5. 7.

    Instrumented Tests • Specific to Android • Focus on the

    user story / use case • Espresso / UI Automator / Robotium • Adds extensions to JUnit • Android Testing Support Library: goo.gl/xSxzoj • Testing Codelab: goo.gl/RHdFBY
  6. 8.

    @RunWith(AndroidJUnit4.class) public class NotesScreenTest { @Rule public ActivityTestRule<NotesActivity> mNotesActivityTestRule =

    new ActivityTestRule<>(NotesActivity.class); @Test public void clickAddNoteButton_opensAddNoteUi() throws Exception { // Click on the add note button onView(withId(R.id.fab_add_notes)).perform(click()); // Check if the add note screen is displayed onView(withId(R.id.add_note_title)).check(matches(isDisplayed())); } }
  7. 9.

    (Local) Unit Tests • Runs fast on your computer •

    Focus on testing individual classes and methods • Dependency injection / object mocking
  8. 11.

    Protip: Run your unit tests along with instrumented tests android

    {
 sourceSets {
 androidTest {
 java.srcDirs += "src/test/java"
 }
 }
 } testCompile 'org.mockito:mockito-core:2.8.47' androidTestCompile 'org.mockito:mockito-android:2.8.47'
  9. 12.

    Protip: Enable StrictMode when running in Test Lab Turn StrictMode

    violations into actionable crash reports in Test Lab.
  10. 13.

    private void enableStrictMode() { if (shouldEnableStrictMode()) { StrictMode.setThreadPolicy(new StrictMode.ThreadPolicy.Builder() .detectAll()

    .penaltyLog() .penaltyDeath() .build()); StrictMode.setVmPolicy(new StrictMode.VmPolicy.Builder() .detectAll() .penaltyLog() .penaltyDeath() .build()); } }
  11. 15.

    private boolean shouldEnableStrictMode() { ContentResolver resolver = context.getContentResolver(); String isInTestLab

    = Settings.System.getString(resolver, "firebase.test.lab"); return BuildConfig.DEBUG || "true".equals(isInTestLab); }
  12. 16.

    // https://issuetracker.google.com/issues/36951662 private void workaroundEnableStrictMode() { if (Build.VERSION.SDK_INT >= 9)

    { enableStrictMode(); } if (Build.VERSION.SDK_INT >= 16) { // restore strict mode after onCreate() returns. new Handler().postAtFrontOfQueue(new Runnable() { @Override public void run() { enableStrictMode(); } }); } }
  13. 18.

    Espresso Test Recorder • Android Studio
 Menu → Run →

    ⚫︎Record Espresso Test • Will generate Espresso code to match your actions
  14. 19.
  15. 20.
  16. 22.
  17. 23.

    Robo Test • Only an APK with is required •

    Automatically logs in with Google Auth • Can pre-fill form fields (e.g. user/pass login, search) • Configure maximum crawl depth and timeout • Algorithm improves over time • Additional test artifact: Activity map
  18. 25.
  19. 26.

    Game Loop Test (beta) • Sequence of scenarios driven by

    Android Intents • Performance += FPS • Game Loop Codelab: goo.gl/sn4RNp • “Mechahamster” sample game • Project home: goo.gl/8chQ8p • Test Lab Guide: goo.gl/16xSPn
  20. 27.
  21. 28.
  22. 29.
  23. 30.
  24. 31.

    gcloud firebase test android run \ --type instrumentation \ --app

    app-debug-unaligned.apk \ --test app-debug-test-unaligned.apk \ --device model=Nexus6,version=21,orientation=portrait \ --device model=Nexus7,version=19,orientation=landscape
  25. 32.

    Community Support for testing with CI • Walmart’s “Flank” for

    sharding tests
 goo.gl/5ApxVU • Circle CI integration
 goo.gl/xoPFqY
  26. 33.
  27. 34.

    Billing for Firebase Test Lab • Unpaid projects: daily limits

    • Virtual devices: 10 tests/day • Physical devices: 5 tests/day • Paid projects: per-minute billing • Virtual devices: $1/device/hour • Physical devices: $5/device/hour
  28. 35.
  29. 36.
  30. 37.
  31. 38.
  32. 39.
  33. 40.

    Confidential + Proprietary • Unity support • NDK support •

    Search • Webhooks Crashlytics Firebase Crash Reporting • Tightly integrated with Analytics ◦ Generates “app_exception” events ◦ Analytics events in crash logs
  34. 41.

    public class MyHandler implements Thread.UncaughtExceptionHandler { private final Thread.UncaughtExceptionHandler mPriorExceptionHandler;

    public MyHandler(Thread.UncaughtExceptionHandler prior) { mPriorExceptionHandler = prior; } @Override public void uncaughtException(Thread thread, Throwable throwable) { // Deal with throwable here if (mPriorExceptionHandler != null) { mPriorExceptionHandler.uncaughtException(thread, throwable); } } }
  35. 43.
  36. 45.

    60% Speed, design & usability When leaving a 5 star

    review, 60% of the time the user mentions the speed, design or usability Google analysis on Play reviews, top 10 English speaking countries, last 365 days of reviews
  37. 47.

    Systrace Advantages • Tons of detail • Custom traces Disadvantages

    • App runs slow • You typically have to know what you're looking for • Can't collect data from the wild, only connected devices
  38. 48.
  39. 49.

    Firebase Performance Monitoring • Runs in production • Negligible overhead

    • Automatic and Custom traces • Automatic HTTP/S traffic metrics
  40. 50.

    Trace • A report for a period of time with

    a well-defined beginning and end • Nominally has a duration • Also may contain "counters" for performance-related events during the trace
  41. 51.

    Automatic Traces - no coding necessary! • App Start (cold)

    • Time in foreground • Time in background
  42. 53.

    Counters • Associate a string/count value to a Trace •

    Typically better to measure ratios than absolute values • Abs value if you want to compare similar counters between traces cache_hit++ cache_miss++ dropped_frames += 10
  43. 55.

    public class PerfLifecycleCallbacks implements Application.ActivityLifecycleCallbacks { private static final PerfLifecycleCallbacks

    instance = new PerfLifecycleCallbacks(); private PerfLifecycleCallbacks() {} public static PerfLifecycleCallbacks getInstance() { return instance; } // Continued...
  44. 56.

    private final HashMap<Activity, Trace> traces = new HashMap<>(); @Override public

    void onActivityStarted(Activity activity) { String name = activity.getClass().getSimpleName(); Trace trace = FirebasePerformance.startTrace(name); traces.put(activity, trace); } @Override public void onActivityStopped(Activity activity) { Trace trace = traces.remove(activity); trace.stop(); }
  45. 58.

    public class PerfInitContentProvider extends ContentProvider { @Override public boolean onCreate()

    { Context context = getContext(); if (context != null) { Application app = (Application) context.getApplicationContext(); app.registerActivityLifecycleCallbacks( PerfLifecycleCallbacks.getInstance()); } return false; } // Overrides elided... }
  46. 59.

    public class MyActivity extends Activity { private void loadStuff() {

    Trace trace = PerfLifecycleCallbacks.getInstance().getTrace(this); trace.incrementCounter("foo"); } }
  47. 60.

    Android Vitals - get clues for what to measure •

    New in the Google Play Console: goo.gl/rkztG1 • Performance-related stats • Slow rendering (jank) • Frozen frames • Stuck WakeLocks
  48. 61.

    Automatic HTTP/S transaction metrics • Response time • Payload size

    • Success rate • URL pattern globbing
 
 yourcdn.com/*.jpg
 api.yourdomain.com/v1/users/*
 api.yourdomain.com/v1/users/*/history/*
  49. 62.

    HTTP/S transaction metrics breakdown • App version • Device •

    Country • OS level • Carrier • Radio
  50. 63.

    How HTTP/S monitoring works Bytecode manipulation (with ASM) URL url

    = new URL("https://firebase.google.com");
 HttpsURLConnection conn = (HttpsURLConnection) url.openConnection(); URL url = new URL(“https://firebase.google.com”);
 HttpsURLConnection conn = (HttpsURLConnection)
 FirebasePerfUrlConnection.instrument(url.openConnection()); Decorator inside!
  51. 64.

    How HTTP/S monitoring works dependencies {
 compile 'com.google.firebase:firebase-perf:11.0.2'
 } apply

    plugin: 'com.google.firebase.firebase-perf' Uses the Transform API: goo.gl/PwBcLE
  52. 65.
  53. 66.

    Firebase Remote Config • Use it to plan and stage

    rollouts of new features • Quickly back off the new feature if it has problems
  54. 67.
  55. 68.
  56. 69.
  57. 70.
  58. 72.

    Firebase Remote Config • Find out what your users prefer

    by experimentation • Study the results in Google Analytics for Firebase
  59. 73.
  60. 75.

    final FirebaseRemoteConfig rc = FirebaseRemoteConfig.getInstance(); int cache_size = (int) rc.getLong("cache_size");

    // init cache with cache_size // later... Trace trace = FirebasePerformance.startTrace("my_trace"); Item item = cache.get("some_resource"); if (item != null) { trace.incrementCounter("cache_hit_size_" + cache_size); } else { trace.incrementCounter("cache_miss_size_" + cache_size); } trace.stop();