Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Boost the Quality of Your App with Firebase

Boost the Quality of Your App with Firebase

Every time your app crashes or hangs, it’s an invitation for the user to uninstall it and give it a bad rating. Fortunately, Firebase provides the tools you need to keep the quality of your app high. Join this session to see how Firebase can help you during development and in production to understand where your app is crashing and where it’s performance is poor. We’ll take a look at Test Lab, Crash Reporting, Remote Config, and the newly released Performance Monitoring tools to gauge the quality of your app. With these, and some clever tricks, you can make sure your users have the best possible experience. Down with one-star ratings!

3acd4fb373289e71fd7ebfb287a75a3b?s=128

Doug Stevenson

July 14, 2017
Tweet

Transcript

  1. Boost the Quality of Your App with Firebase Doug Stevenson

    @CodingDoug
  2. 50% Stability & bugs When leaving a 1 star review,

    50% of the time the user mentions the app stability and bugs Google analysis on Play reviews, top 10 English speaking countries, last 365 days of reviews
  3. Prevention After dev & test cycle, it’s too late!

  4. None
  5. None
  6. Firebase Test Lab for Android Test Artifacts • Logcat (crashes)

    • Videos • Screenshots • Performance
 CPU
 memory
 network ingress/egress
  7. Instrumented Tests • Specific to Android • Focus on the

    user story / use case • Espresso / UI Automator / Robotium • Adds extensions to JUnit • Android Testing Support Library: goo.gl/xSxzoj • Testing Codelab: goo.gl/RHdFBY
  8. @RunWith(AndroidJUnit4.class) public class NotesScreenTest { @Rule public ActivityTestRule<NotesActivity> mNotesActivityTestRule =

    new ActivityTestRule<>(NotesActivity.class); @Test public void clickAddNoteButton_opensAddNoteUi() throws Exception { // Click on the add note button onView(withId(R.id.fab_add_notes)).perform(click()); // Check if the add note screen is displayed onView(withId(R.id.add_note_title)).check(matches(isDisplayed())); } }
  9. (Local) Unit Tests • Runs fast on your computer •

    Focus on testing individual classes and methods • Dependency injection / object mocking
  10. Android Instrumented Tests Local Unit Tests

  11. Protip: Run your unit tests along with instrumented tests android

    {
 sourceSets {
 androidTest {
 java.srcDirs += "src/test/java"
 }
 }
 } testCompile 'org.mockito:mockito-core:2.8.47' androidTestCompile 'org.mockito:mockito-android:2.8.47'
  12. Protip: Enable StrictMode when running in Test Lab Turn StrictMode

    violations into actionable crash reports in Test Lab.
  13. private void enableStrictMode() { if (shouldEnableStrictMode()) { StrictMode.setThreadPolicy(new StrictMode.ThreadPolicy.Builder() .detectAll()

    .penaltyLog() .penaltyDeath() .build()); StrictMode.setVmPolicy(new StrictMode.VmPolicy.Builder() .detectAll() .penaltyLog() .penaltyDeath() .build()); } }
  14. private boolean shouldEnableStrictMode() { return BuildConfig.DEBUG; }

  15. private boolean shouldEnableStrictMode() { ContentResolver resolver = context.getContentResolver(); String isInTestLab

    = Settings.System.getString(resolver, "firebase.test.lab"); return BuildConfig.DEBUG || "true".equals(isInTestLab); }
  16. // https://issuetracker.google.com/issues/36951662 private void workaroundEnableStrictMode() { if (Build.VERSION.SDK_INT >= 9)

    { enableStrictMode(); } if (Build.VERSION.SDK_INT >= 16) { // restore strict mode after onCreate() returns. new Handler().postAtFrontOfQueue(new Runnable() { @Override public void run() { enableStrictMode(); } }); } }
  17. But I don’t want to write tests

  18. Espresso Test Recorder • Android Studio
 Menu → Run →

    ⚫︎Record Espresso Test • Will generate Espresso code to match your actions
  19. None
  20. None
  21. But I don’t even want to have tests

  22. None
  23. Robo Test • Only an APK with is required •

    Automatically logs in with Google Auth • Can pre-fill form fields (e.g. user/pass login, search) • Configure maximum crawl depth and timeout • Algorithm improves over time • Additional test artifact: Activity map
  24. But I’m building a game

  25. None
  26. Game Loop Test (beta) • Sequence of scenarios driven by

    Android Intents • Performance += FPS • Game Loop Codelab: goo.gl/sn4RNp • “Mechahamster” sample game • Project home: goo.gl/8chQ8p • Test Lab Guide: goo.gl/16xSPn
  27. None
  28. None
  29. None
  30. None
  31. gcloud firebase test android run \ --type instrumentation \ --app

    app-debug-unaligned.apk \ --test app-debug-test-unaligned.apk \ --device model=Nexus6,version=21,orientation=portrait \ --device model=Nexus7,version=19,orientation=landscape
  32. Community Support for testing with CI • Walmart’s “Flank” for

    sharding tests
 goo.gl/5ApxVU • Circle CI integration
 goo.gl/xoPFqY
  33. None
  34. Billing for Firebase Test Lab • Unpaid projects: daily limits

    • Virtual devices: 10 tests/day • Physical devices: 5 tests/day • Paid projects: per-minute billing • Virtual devices: $1/device/hour • Physical devices: $5/device/hour
  35. None
  36. None
  37. None
  38. None
  39. None
  40. Confidential + Proprietary • Unity support • NDK support •

    Search • Webhooks Crashlytics Firebase Crash Reporting • Tightly integrated with Analytics ◦ Generates “app_exception” events ◦ Analytics events in crash logs
  41. public class MyHandler implements Thread.UncaughtExceptionHandler { private final Thread.UncaughtExceptionHandler mPriorExceptionHandler;

    public MyHandler(Thread.UncaughtExceptionHandler prior) { mPriorExceptionHandler = prior; } @Override public void uncaughtException(Thread thread, Throwable throwable) { // Deal with throwable here if (mPriorExceptionHandler != null) { mPriorExceptionHandler.uncaughtException(thread, throwable); } } }
  42. Thread.UncaughtExceptionHandler prior = Thread.getDefaultUncaughtExceptionHandler(); Thread.setDefaultUncaughtExceptionHandler( new MyUncaughtExceptionHandler(prior));

  43. None
  44. Performance == Quality

  45. 60% Speed, design & usability When leaving a 5 star

    review, 60% of the time the user mentions the speed, design or usability Google analysis on Play reviews, top 10 English speaking countries, last 365 days of reviews
  46. https://goo.gl/BnWcPK

  47. Systrace Advantages • Tons of detail • Custom traces Disadvantages

    • App runs slow • You typically have to know what you're looking for • Can't collect data from the wild, only connected devices
  48. None
  49. Firebase Performance Monitoring • Runs in production • Negligible overhead

    • Automatic and Custom traces • Automatic HTTP/S traffic metrics
  50. Trace • A report for a period of time with

    a well-defined beginning and end • Nominally has a duration • Also may contain "counters" for performance-related events during the trace
  51. Automatic Traces - no coding necessary! • App Start (cold)

    • Time in foreground • Time in background
  52. Automatic Traces - App Start ContentProvider.onCreate() Activity.onResume() But you might

    want something different!
  53. Counters • Associate a string/count value to a Trace •

    Typically better to measure ratios than absolute values • Abs value if you want to compare similar counters between traces cache_hit++ cache_miss++ dropped_frames += 10
  54. An idea for pseudo-automatic traces

  55. public class PerfLifecycleCallbacks implements Application.ActivityLifecycleCallbacks { private static final PerfLifecycleCallbacks

    instance = new PerfLifecycleCallbacks(); private PerfLifecycleCallbacks() {} public static PerfLifecycleCallbacks getInstance() { return instance; } // Continued...
  56. private final HashMap<Activity, Trace> traces = new HashMap<>(); @Override public

    void onActivityStarted(Activity activity) { String name = activity.getClass().getSimpleName(); Trace trace = FirebasePerformance.startTrace(name); traces.put(activity, trace); } @Override public void onActivityStopped(Activity activity) { Trace trace = traces.remove(activity); trace.stop(); }
  57. public Trace getTrace(Activity activity) { return traces.get(activity); }

  58. public class PerfInitContentProvider extends ContentProvider { @Override public boolean onCreate()

    { Context context = getContext(); if (context != null) { Application app = (Application) context.getApplicationContext(); app.registerActivityLifecycleCallbacks( PerfLifecycleCallbacks.getInstance()); } return false; } // Overrides elided... }
  59. public class MyActivity extends Activity { private void loadStuff() {

    Trace trace = PerfLifecycleCallbacks.getInstance().getTrace(this); trace.incrementCounter("foo"); } }
  60. Android Vitals - get clues for what to measure •

    New in the Google Play Console: goo.gl/rkztG1 • Performance-related stats • Slow rendering (jank) • Frozen frames • Stuck WakeLocks
  61. Automatic HTTP/S transaction metrics • Response time • Payload size

    • Success rate • URL pattern globbing
 
 yourcdn.com/*.jpg
 api.yourdomain.com/v1/users/*
 api.yourdomain.com/v1/users/*/history/*
  62. HTTP/S transaction metrics breakdown • App version • Device •

    Country • OS level • Carrier • Radio
  63. How HTTP/S monitoring works Bytecode manipulation (with ASM) URL url

    = new URL("https://firebase.google.com");
 HttpsURLConnection conn = (HttpsURLConnection) url.openConnection(); URL url = new URL(“https://firebase.google.com”);
 HttpsURLConnection conn = (HttpsURLConnection)
 FirebasePerfUrlConnection.instrument(url.openConnection()); Decorator inside!
  64. How HTTP/S monitoring works dependencies {
 compile 'com.google.firebase:firebase-perf:11.0.2'
 } apply

    plugin: 'com.google.firebase.firebase-perf' Uses the Transform API: goo.gl/PwBcLE
  65. None
  66. Firebase Remote Config • Use it to plan and stage

    rollouts of new features • Quickly back off the new feature if it has problems
  67. None
  68. None
  69. None
  70. None
  71. FirebaseRemoteConfig rc = FirebaseRemoteConfig.getInstance(); if (rc.getBoolean("using_cool_new_feature")) { // do cool

    stuff } else { // do the boring old stuff }
  72. Firebase Remote Config • Find out what your users prefer

    by experimentation • Study the results in Google Analytics for Firebase
  73. None
  74. Firebase Remote Config (for perf) • Compare CDN/server performance •

    Compare cache configuration strategies
  75. final FirebaseRemoteConfig rc = FirebaseRemoteConfig.getInstance(); int cache_size = (int) rc.getLong("cache_size");

    // init cache with cache_size // later... Trace trace = FirebasePerformance.startTrace("my_trace"); Item item = cache.get("some_resource"); if (item != null) { trace.incrementCounter("cache_hit_size_" + cache_size); } else { trace.incrementCounter("cache_miss_size_" + cache_size); } trace.stop();
  76. Thank you! Confidential + Proprietary Doug Stevenson @CodingDoug youtube.com/firebase firebase.googleblog.com