Slide 1

Slide 1 text

Boost the Quality of Your App with Firebase Doug Stevenson @CodingDoug

Slide 2

Slide 2 text

50% Stability & bugs When leaving a 1 star review, 50% of the time the user mentions the app stability and bugs Google analysis on Play reviews, top 10 English speaking countries, last 365 days of reviews

Slide 3

Slide 3 text

Prevention After dev & test cycle, it’s too late!

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

Firebase Test Lab for Android Test Artifacts • Logcat (crashes) • Videos • Screenshots • Performance
 CPU
 memory
 network ingress/egress

Slide 7

Slide 7 text

Instrumented Tests • Specific to Android • Focus on the user story / use case • Espresso / UI Automator / Robotium • Adds extensions to JUnit • Android Testing Support Library: goo.gl/xSxzoj • Testing Codelab: goo.gl/RHdFBY

Slide 8

Slide 8 text

@RunWith(AndroidJUnit4.class) public class NotesScreenTest { @Rule public ActivityTestRule mNotesActivityTestRule = new ActivityTestRule<>(NotesActivity.class); @Test public void clickAddNoteButton_opensAddNoteUi() throws Exception { // Click on the add note button onView(withId(R.id.fab_add_notes)).perform(click()); // Check if the add note screen is displayed onView(withId(R.id.add_note_title)).check(matches(isDisplayed())); } }

Slide 9

Slide 9 text

(Local) Unit Tests • Runs fast on your computer • Focus on testing individual classes and methods • Dependency injection / object mocking

Slide 10

Slide 10 text

Android Instrumented Tests Local Unit Tests

Slide 11

Slide 11 text

Protip: Run your unit tests along with instrumented tests android {
 sourceSets {
 androidTest {
 java.srcDirs += "src/test/java"
 }
 }
 } testCompile 'org.mockito:mockito-core:2.8.47' androidTestCompile 'org.mockito:mockito-android:2.8.47'

Slide 12

Slide 12 text

Protip: Enable StrictMode when running in Test Lab Turn StrictMode violations into actionable crash reports in Test Lab.

Slide 13

Slide 13 text

private void enableStrictMode() { if (shouldEnableStrictMode()) { StrictMode.setThreadPolicy(new StrictMode.ThreadPolicy.Builder() .detectAll() .penaltyLog() .penaltyDeath() .build()); StrictMode.setVmPolicy(new StrictMode.VmPolicy.Builder() .detectAll() .penaltyLog() .penaltyDeath() .build()); } }

Slide 14

Slide 14 text

private boolean shouldEnableStrictMode() { return BuildConfig.DEBUG; }

Slide 15

Slide 15 text

private boolean shouldEnableStrictMode() { ContentResolver resolver = context.getContentResolver(); String isInTestLab = Settings.System.getString(resolver, "firebase.test.lab"); return BuildConfig.DEBUG || "true".equals(isInTestLab); }

Slide 16

Slide 16 text

// https://issuetracker.google.com/issues/36951662 private void workaroundEnableStrictMode() { if (Build.VERSION.SDK_INT >= 9) { enableStrictMode(); } if (Build.VERSION.SDK_INT >= 16) { // restore strict mode after onCreate() returns. new Handler().postAtFrontOfQueue(new Runnable() { @Override public void run() { enableStrictMode(); } }); } }

Slide 17

Slide 17 text

But I don’t want to write tests

Slide 18

Slide 18 text

Espresso Test Recorder • Android Studio
 Menu → Run → ⚫︎Record Espresso Test • Will generate Espresso code to match your actions

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

But I don’t even want to have tests

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

Robo Test • Only an APK with is required • Automatically logs in with Google Auth • Can pre-fill form fields (e.g. user/pass login, search) • Configure maximum crawl depth and timeout • Algorithm improves over time • Additional test artifact: Activity map

Slide 24

Slide 24 text

But I’m building a game

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

Game Loop Test (beta) • Sequence of scenarios driven by Android Intents • Performance += FPS • Game Loop Codelab: goo.gl/sn4RNp • “Mechahamster” sample game • Project home: goo.gl/8chQ8p • Test Lab Guide: goo.gl/16xSPn

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

gcloud firebase test android run \ --type instrumentation \ --app app-debug-unaligned.apk \ --test app-debug-test-unaligned.apk \ --device model=Nexus6,version=21,orientation=portrait \ --device model=Nexus7,version=19,orientation=landscape

Slide 32

Slide 32 text

Community Support for testing with CI • Walmart’s “Flank” for sharding tests
 goo.gl/5ApxVU • Circle CI integration
 goo.gl/xoPFqY

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

Billing for Firebase Test Lab • Unpaid projects: daily limits • Virtual devices: 10 tests/day • Physical devices: 5 tests/day • Paid projects: per-minute billing • Virtual devices: $1/device/hour • Physical devices: $5/device/hour

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

Confidential + Proprietary ● Unity support ● NDK support ● Search ● Webhooks Crashlytics Firebase Crash Reporting ● Tightly integrated with Analytics ○ Generates “app_exception” events ○ Analytics events in crash logs

Slide 41

Slide 41 text

public class MyHandler implements Thread.UncaughtExceptionHandler { private final Thread.UncaughtExceptionHandler mPriorExceptionHandler; public MyHandler(Thread.UncaughtExceptionHandler prior) { mPriorExceptionHandler = prior; } @Override public void uncaughtException(Thread thread, Throwable throwable) { // Deal with throwable here if (mPriorExceptionHandler != null) { mPriorExceptionHandler.uncaughtException(thread, throwable); } } }

Slide 42

Slide 42 text

Thread.UncaughtExceptionHandler prior = Thread.getDefaultUncaughtExceptionHandler(); Thread.setDefaultUncaughtExceptionHandler( new MyUncaughtExceptionHandler(prior));

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

Performance == Quality

Slide 45

Slide 45 text

60% Speed, design & usability When leaving a 5 star review, 60% of the time the user mentions the speed, design or usability Google analysis on Play reviews, top 10 English speaking countries, last 365 days of reviews

Slide 46

Slide 46 text

https://goo.gl/BnWcPK

Slide 47

Slide 47 text

Systrace Advantages • Tons of detail • Custom traces Disadvantages • App runs slow • You typically have to know what you're looking for • Can't collect data from the wild, only connected devices

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

Firebase Performance Monitoring • Runs in production • Negligible overhead • Automatic and Custom traces • Automatic HTTP/S traffic metrics

Slide 50

Slide 50 text

Trace • A report for a period of time with a well-defined beginning and end • Nominally has a duration • Also may contain "counters" for performance-related events during the trace

Slide 51

Slide 51 text

Automatic Traces - no coding necessary! • App Start (cold) • Time in foreground • Time in background

Slide 52

Slide 52 text

Automatic Traces - App Start ContentProvider.onCreate() Activity.onResume() But you might want something different!

Slide 53

Slide 53 text

Counters • Associate a string/count value to a Trace • Typically better to measure ratios than absolute values • Abs value if you want to compare similar counters between traces cache_hit++ cache_miss++ dropped_frames += 10

Slide 54

Slide 54 text

An idea for pseudo-automatic traces

Slide 55

Slide 55 text

public class PerfLifecycleCallbacks implements Application.ActivityLifecycleCallbacks { private static final PerfLifecycleCallbacks instance = new PerfLifecycleCallbacks(); private PerfLifecycleCallbacks() {} public static PerfLifecycleCallbacks getInstance() { return instance; } // Continued...

Slide 56

Slide 56 text

private final HashMap traces = new HashMap<>(); @Override public void onActivityStarted(Activity activity) { String name = activity.getClass().getSimpleName(); Trace trace = FirebasePerformance.startTrace(name); traces.put(activity, trace); } @Override public void onActivityStopped(Activity activity) { Trace trace = traces.remove(activity); trace.stop(); }

Slide 57

Slide 57 text

public Trace getTrace(Activity activity) { return traces.get(activity); }

Slide 58

Slide 58 text

public class PerfInitContentProvider extends ContentProvider { @Override public boolean onCreate() { Context context = getContext(); if (context != null) { Application app = (Application) context.getApplicationContext(); app.registerActivityLifecycleCallbacks( PerfLifecycleCallbacks.getInstance()); } return false; } // Overrides elided... }

Slide 59

Slide 59 text

public class MyActivity extends Activity { private void loadStuff() { Trace trace = PerfLifecycleCallbacks.getInstance().getTrace(this); trace.incrementCounter("foo"); } }

Slide 60

Slide 60 text

Android Vitals - get clues for what to measure • New in the Google Play Console: goo.gl/rkztG1 • Performance-related stats • Slow rendering (jank) • Frozen frames • Stuck WakeLocks

Slide 61

Slide 61 text

Automatic HTTP/S transaction metrics • Response time • Payload size • Success rate • URL pattern globbing
 
 yourcdn.com/*.jpg
 api.yourdomain.com/v1/users/*
 api.yourdomain.com/v1/users/*/history/*

Slide 62

Slide 62 text

HTTP/S transaction metrics breakdown • App version • Device • Country • OS level • Carrier • Radio

Slide 63

Slide 63 text

How HTTP/S monitoring works Bytecode manipulation (with ASM) URL url = new URL("https://firebase.google.com");
 HttpsURLConnection conn = (HttpsURLConnection) url.openConnection(); URL url = new URL(“https://firebase.google.com”);
 HttpsURLConnection conn = (HttpsURLConnection)
 FirebasePerfUrlConnection.instrument(url.openConnection()); Decorator inside!

Slide 64

Slide 64 text

How HTTP/S monitoring works dependencies {
 compile 'com.google.firebase:firebase-perf:11.0.2'
 } apply plugin: 'com.google.firebase.firebase-perf' Uses the Transform API: goo.gl/PwBcLE

Slide 65

Slide 65 text

No content

Slide 66

Slide 66 text

Firebase Remote Config • Use it to plan and stage rollouts of new features • Quickly back off the new feature if it has problems

Slide 67

Slide 67 text

No content

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

No content

Slide 70

Slide 70 text

No content

Slide 71

Slide 71 text

FirebaseRemoteConfig rc = FirebaseRemoteConfig.getInstance(); if (rc.getBoolean("using_cool_new_feature")) { // do cool stuff } else { // do the boring old stuff }

Slide 72

Slide 72 text

Firebase Remote Config • Find out what your users prefer by experimentation • Study the results in Google Analytics for Firebase

Slide 73

Slide 73 text

No content

Slide 74

Slide 74 text

Firebase Remote Config (for perf) • Compare CDN/server performance • Compare cache configuration strategies

Slide 75

Slide 75 text

final FirebaseRemoteConfig rc = FirebaseRemoteConfig.getInstance(); int cache_size = (int) rc.getLong("cache_size"); // init cache with cache_size // later... Trace trace = FirebasePerformance.startTrace("my_trace"); Item item = cache.get("some_resource"); if (item != null) { trace.incrementCounter("cache_hit_size_" + cache_size); } else { trace.incrementCounter("cache_miss_size_" + cache_size); } trace.stop();

Slide 76

Slide 76 text

Thank you! Confidential + Proprietary Doug Stevenson @CodingDoug youtube.com/firebase firebase.googleblog.com