$30 off During Our Annual Pro Sale. View Details »

Boost the Quality of Your App with Firebase

Boost the Quality of Your App with Firebase

Every time your app crashes or hangs, it’s an invitation for the user to uninstall it and give it a bad rating. Fortunately, Firebase provides the tools you need to keep the quality of your app high. Join this session to see how Firebase can help you during development and in production to understand where your app is crashing and where it’s performance is poor. We’ll take a look at Test Lab, Crash Reporting, Remote Config, and the newly released Performance Monitoring tools to gauge the quality of your app. With these, and some clever tricks, you can make sure your users have the best possible experience. Down with one-star ratings!

Doug Stevenson

July 14, 2017
Tweet

More Decks by Doug Stevenson

Other Decks in Programming

Transcript

  1. Boost the Quality of Your App
    with Firebase
    Doug Stevenson
    @CodingDoug

    View Slide

  2. 50%
    Stability & bugs
    When leaving a 1 star review, 50% of the time the user mentions
    the app stability and bugs
    Google analysis on Play reviews, top 10 English speaking countries, last 365 days of reviews

    View Slide

  3. Prevention
    After dev & test cycle, it’s too
    late!

    View Slide

  4. View Slide

  5. View Slide

  6. Firebase Test
    Lab for Android
    Test Artifacts
    • Logcat (crashes)
    • Videos
    • Screenshots
    • Performance

    CPU

    memory

    network ingress/egress

    View Slide

  7. Instrumented Tests
    • Specific to Android
    • Focus on the user story / use case
    • Espresso / UI Automator / Robotium
    • Adds extensions to JUnit
    • Android Testing Support Library: goo.gl/xSxzoj
    • Testing Codelab: goo.gl/RHdFBY

    View Slide

  8. @RunWith(AndroidJUnit4.class)
    public class NotesScreenTest {
    @Rule
    public ActivityTestRule mNotesActivityTestRule =
    new ActivityTestRule<>(NotesActivity.class);
    @Test
    public void clickAddNoteButton_opensAddNoteUi() throws Exception {
    // Click on the add note button
    onView(withId(R.id.fab_add_notes)).perform(click());
    // Check if the add note screen is displayed
    onView(withId(R.id.add_note_title)).check(matches(isDisplayed()));
    }
    }

    View Slide

  9. (Local) Unit Tests
    • Runs fast on your computer
    • Focus on testing individual classes and methods
    • Dependency injection / object mocking

    View Slide

  10. Android Instrumented Tests
    Local Unit Tests

    View Slide

  11. Protip: Run your unit tests along with instrumented tests
    android {

    sourceSets {

    androidTest {

    java.srcDirs += "src/test/java"

    }

    }

    }
    testCompile 'org.mockito:mockito-core:2.8.47'
    androidTestCompile 'org.mockito:mockito-android:2.8.47'

    View Slide

  12. Protip: Enable StrictMode when running in Test Lab
    Turn StrictMode violations into actionable crash reports in Test Lab.

    View Slide

  13. private void enableStrictMode() {
    if (shouldEnableStrictMode()) {
    StrictMode.setThreadPolicy(new StrictMode.ThreadPolicy.Builder()
    .detectAll()
    .penaltyLog()
    .penaltyDeath()
    .build());
    StrictMode.setVmPolicy(new StrictMode.VmPolicy.Builder()
    .detectAll()
    .penaltyLog()
    .penaltyDeath()
    .build());
    }
    }

    View Slide

  14. private boolean shouldEnableStrictMode() {
    return BuildConfig.DEBUG;
    }

    View Slide

  15. private boolean shouldEnableStrictMode() {
    ContentResolver resolver = context.getContentResolver();
    String isInTestLab =
    Settings.System.getString(resolver, "firebase.test.lab");
    return BuildConfig.DEBUG || "true".equals(isInTestLab);
    }

    View Slide

  16. // https://issuetracker.google.com/issues/36951662
    private void workaroundEnableStrictMode() {
    if (Build.VERSION.SDK_INT >= 9) {
    enableStrictMode();
    }
    if (Build.VERSION.SDK_INT >= 16) {
    // restore strict mode after onCreate() returns.
    new Handler().postAtFrontOfQueue(new Runnable() {
    @Override
    public void run() {
    enableStrictMode();
    }
    });
    }
    }

    View Slide

  17. But I don’t want to write tests

    View Slide

  18. Espresso Test Recorder
    • Android Studio

    Menu → Run → ⚫︎Record Espresso Test
    • Will generate Espresso code to match your actions

    View Slide

  19. View Slide

  20. View Slide

  21. But I don’t even want to have tests

    View Slide

  22. View Slide

  23. Robo Test
    • Only an APK with is required
    • Automatically logs in with Google Auth
    • Can pre-fill form fields (e.g. user/pass login, search)
    • Configure maximum crawl depth and timeout
    • Algorithm improves over time
    • Additional test artifact: Activity map

    View Slide

  24. But I’m building a game

    View Slide

  25. View Slide

  26. Game Loop Test (beta)
    • Sequence of scenarios driven by Android Intents
    • Performance += FPS
    • Game Loop Codelab: goo.gl/sn4RNp
    • “Mechahamster” sample game
    • Project home: goo.gl/8chQ8p
    • Test Lab Guide: goo.gl/16xSPn

    View Slide

  27. View Slide

  28. View Slide

  29. View Slide

  30. View Slide

  31. gcloud firebase test android run \
    --type instrumentation \
    --app app-debug-unaligned.apk \
    --test app-debug-test-unaligned.apk \
    --device model=Nexus6,version=21,orientation=portrait \
    --device model=Nexus7,version=19,orientation=landscape

    View Slide

  32. Community Support for testing with CI
    • Walmart’s “Flank” for sharding tests

    goo.gl/5ApxVU
    • Circle CI integration

    goo.gl/xoPFqY

    View Slide

  33. View Slide

  34. Billing for Firebase Test Lab
    • Unpaid projects: daily limits
    • Virtual devices: 10 tests/day
    • Physical devices: 5 tests/day
    • Paid projects: per-minute billing
    • Virtual devices: $1/device/hour
    • Physical devices: $5/device/hour

    View Slide

  35. View Slide

  36. View Slide

  37. View Slide

  38. View Slide

  39. View Slide

  40. Confidential + Proprietary
    ● Unity support
    ● NDK support
    ● Search
    ● Webhooks
    Crashlytics Firebase Crash Reporting
    ● Tightly integrated with Analytics
    ○ Generates “app_exception”
    events
    ○ Analytics events in crash logs

    View Slide

  41. public class MyHandler implements Thread.UncaughtExceptionHandler {
    private final Thread.UncaughtExceptionHandler mPriorExceptionHandler;
    public MyHandler(Thread.UncaughtExceptionHandler prior) {
    mPriorExceptionHandler = prior;
    }
    @Override
    public void uncaughtException(Thread thread, Throwable throwable) {
    // Deal with throwable here
    if (mPriorExceptionHandler != null) {
    mPriorExceptionHandler.uncaughtException(thread, throwable);
    }
    }
    }

    View Slide

  42. Thread.UncaughtExceptionHandler prior =
    Thread.getDefaultUncaughtExceptionHandler();
    Thread.setDefaultUncaughtExceptionHandler(
    new MyUncaughtExceptionHandler(prior));

    View Slide

  43. View Slide

  44. Performance == Quality

    View Slide

  45. 60%
    Speed, design & usability
    When leaving a 5 star review, 60% of the time the user mentions
    the speed, design or usability
    Google analysis on Play reviews, top 10 English speaking countries, last 365 days of reviews

    View Slide

  46. https://goo.gl/BnWcPK

    View Slide

  47. Systrace
    Advantages
    • Tons of detail
    • Custom traces
    Disadvantages
    • App runs slow
    • You typically have to know what you're looking for
    • Can't collect data from the wild, only connected devices

    View Slide

  48. View Slide

  49. Firebase Performance Monitoring
    • Runs in production
    • Negligible overhead
    • Automatic and Custom traces
    • Automatic HTTP/S traffic metrics

    View Slide

  50. Trace
    • A report for a period of time with a well-defined beginning and end
    • Nominally has a duration
    • Also may contain "counters" for performance-related events during the trace

    View Slide

  51. Automatic Traces - no coding necessary!
    • App Start (cold)
    • Time in foreground
    • Time in background

    View Slide

  52. Automatic Traces - App Start
    ContentProvider.onCreate() Activity.onResume()
    But you might want something different!

    View Slide

  53. Counters
    • Associate a string/count value to a Trace
    • Typically better to measure ratios than absolute values
    • Abs value if you want to compare similar counters between traces
    cache_hit++
    cache_miss++
    dropped_frames += 10

    View Slide

  54. An idea for pseudo-automatic traces

    View Slide

  55. public class PerfLifecycleCallbacks
    implements Application.ActivityLifecycleCallbacks {
    private static final PerfLifecycleCallbacks instance =
    new PerfLifecycleCallbacks();
    private PerfLifecycleCallbacks() {}
    public static PerfLifecycleCallbacks getInstance() {
    return instance;
    }
    // Continued...

    View Slide

  56. private final HashMap traces =
    new HashMap<>();
    @Override
    public void onActivityStarted(Activity activity) {
    String name = activity.getClass().getSimpleName();
    Trace trace = FirebasePerformance.startTrace(name);
    traces.put(activity, trace);
    }
    @Override
    public void onActivityStopped(Activity activity) {
    Trace trace = traces.remove(activity);
    trace.stop();
    }

    View Slide

  57. public Trace getTrace(Activity activity) {
    return traces.get(activity);
    }

    View Slide

  58. public class PerfInitContentProvider extends ContentProvider {
    @Override
    public boolean onCreate() {
    Context context = getContext();
    if (context != null) {
    Application app = (Application) context.getApplicationContext();
    app.registerActivityLifecycleCallbacks(
    PerfLifecycleCallbacks.getInstance());
    }
    return false;
    }
    // Overrides elided...
    }

    View Slide

  59. public class MyActivity extends Activity {
    private void loadStuff() {
    Trace trace = PerfLifecycleCallbacks.getInstance().getTrace(this);
    trace.incrementCounter("foo");
    }
    }

    View Slide

  60. Android Vitals - get clues for what to measure
    • New in the Google Play Console: goo.gl/rkztG1
    • Performance-related stats
    • Slow rendering (jank)
    • Frozen frames
    • Stuck WakeLocks

    View Slide

  61. Automatic HTTP/S transaction metrics
    • Response time
    • Payload size
    • Success rate
    • URL pattern globbing


    yourcdn.com/*.jpg

    api.yourdomain.com/v1/users/*

    api.yourdomain.com/v1/users/*/history/*

    View Slide

  62. HTTP/S transaction metrics breakdown
    • App version
    • Device
    • Country
    • OS level
    • Carrier
    • Radio

    View Slide

  63. How HTTP/S monitoring works
    Bytecode manipulation (with ASM)
    URL url = new URL("https://firebase.google.com");

    HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
    URL url = new URL(“https://firebase.google.com”);

    HttpsURLConnection conn = (HttpsURLConnection)

    FirebasePerfUrlConnection.instrument(url.openConnection());
    Decorator inside!

    View Slide

  64. How HTTP/S monitoring works
    dependencies {

    compile 'com.google.firebase:firebase-perf:11.0.2'

    }
    apply plugin: 'com.google.firebase.firebase-perf'
    Uses the Transform API: goo.gl/PwBcLE

    View Slide

  65. View Slide

  66. Firebase Remote Config
    • Use it to plan and stage rollouts of new features
    • Quickly back off the new feature if it has problems

    View Slide

  67. View Slide

  68. View Slide

  69. View Slide

  70. View Slide

  71. FirebaseRemoteConfig rc = FirebaseRemoteConfig.getInstance();
    if (rc.getBoolean("using_cool_new_feature")) {
    // do cool stuff
    }
    else {
    // do the boring old stuff
    }

    View Slide

  72. Firebase Remote Config
    • Find out what your users prefer by experimentation
    • Study the results in Google Analytics for Firebase

    View Slide

  73. View Slide

  74. Firebase Remote Config (for perf)
    • Compare CDN/server performance
    • Compare cache configuration strategies

    View Slide

  75. final FirebaseRemoteConfig rc = FirebaseRemoteConfig.getInstance();
    int cache_size = (int) rc.getLong("cache_size");
    // init cache with cache_size
    // later...
    Trace trace = FirebasePerformance.startTrace("my_trace");
    Item item = cache.get("some_resource");
    if (item != null) {
    trace.incrementCounter("cache_hit_size_" + cache_size);
    }
    else {
    trace.incrementCounter("cache_miss_size_" + cache_size);
    }
    trace.stop();

    View Slide

  76. Thank you!
    Confidential + Proprietary
    Doug Stevenson
    @CodingDoug
    youtube.com/firebase
    firebase.googleblog.com

    View Slide