Slide 1
Slide 1 text
Boost the Quality of Your App
with Firebase
Doug Stevenson
@CodingDoug
Slide 2
Slide 2 text
50%
Stability & bugs
When leaving a 1 star review, 50% of the time the user mentions
the app stability and bugs
Google analysis on Play reviews, top 10 English speaking countries, last 365 days of reviews
Slide 3
Slide 3 text
Prevention
After dev & test cycle, it’s too
late!
Slide 4
Slide 4 text
No content
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
Firebase Test
Lab for Android
Test Artifacts
• Logcat (crashes)
• Videos
• Screenshots
• Performance
CPU
memory
network ingress/egress
Slide 7
Slide 7 text
Instrumented Tests
• Specific to Android
• Focus on the user story / use case
• Espresso / UI Automator / Robotium
• Adds extensions to JUnit
• Android Testing Support Library: goo.gl/xSxzoj
• Testing Codelab: goo.gl/RHdFBY
Slide 8
Slide 8 text
@RunWith(AndroidJUnit4.class)
public class NotesScreenTest {
@Rule
public ActivityTestRule mNotesActivityTestRule =
new ActivityTestRule<>(NotesActivity.class);
@Test
public void clickAddNoteButton_opensAddNoteUi() throws Exception {
// Click on the add note button
onView(withId(R.id.fab_add_notes)).perform(click());
// Check if the add note screen is displayed
onView(withId(R.id.add_note_title)).check(matches(isDisplayed()));
}
}
Slide 9
Slide 9 text
(Local) Unit Tests
• Runs fast on your computer
• Focus on testing individual classes and methods
• Dependency injection / object mocking
Slide 10
Slide 10 text
Android Instrumented Tests
Local Unit Tests
Slide 11
Slide 11 text
Protip: Run your unit tests along with instrumented tests
android {
sourceSets {
androidTest {
java.srcDirs += "src/test/java"
}
}
}
testCompile 'org.mockito:mockito-core:2.8.47'
androidTestCompile 'org.mockito:mockito-android:2.8.47'
Slide 12
Slide 12 text
Protip: Enable StrictMode when running in Test Lab
Turn StrictMode violations into actionable crash reports in Test Lab.
Slide 13
Slide 13 text
private void enableStrictMode() {
if (shouldEnableStrictMode()) {
StrictMode.setThreadPolicy(new StrictMode.ThreadPolicy.Builder()
.detectAll()
.penaltyLog()
.penaltyDeath()
.build());
StrictMode.setVmPolicy(new StrictMode.VmPolicy.Builder()
.detectAll()
.penaltyLog()
.penaltyDeath()
.build());
}
}
Slide 14
Slide 14 text
private boolean shouldEnableStrictMode() {
return BuildConfig.DEBUG;
}
Slide 15
Slide 15 text
private boolean shouldEnableStrictMode() {
ContentResolver resolver = context.getContentResolver();
String isInTestLab =
Settings.System.getString(resolver, "firebase.test.lab");
return BuildConfig.DEBUG || "true".equals(isInTestLab);
}
Slide 16
Slide 16 text
// https://issuetracker.google.com/issues/36951662
private void workaroundEnableStrictMode() {
if (Build.VERSION.SDK_INT >= 9) {
enableStrictMode();
}
if (Build.VERSION.SDK_INT >= 16) {
// restore strict mode after onCreate() returns.
new Handler().postAtFrontOfQueue(new Runnable() {
@Override
public void run() {
enableStrictMode();
}
});
}
}
Slide 17
Slide 17 text
But I don’t want to write tests
Slide 18
Slide 18 text
Espresso Test Recorder
• Android Studio
Menu → Run → ⚫︎Record Espresso Test
• Will generate Espresso code to match your actions
Slide 19
Slide 19 text
No content
Slide 20
Slide 20 text
No content
Slide 21
Slide 21 text
But I don’t even want to have tests
Slide 22
Slide 22 text
No content
Slide 23
Slide 23 text
Robo Test
• Only an APK with is required
• Automatically logs in with Google Auth
• Can pre-fill form fields (e.g. user/pass login, search)
• Configure maximum crawl depth and timeout
• Algorithm improves over time
• Additional test artifact: Activity map
Slide 24
Slide 24 text
But I’m building a game
Slide 25
Slide 25 text
No content
Slide 26
Slide 26 text
Game Loop Test (beta)
• Sequence of scenarios driven by Android Intents
• Performance += FPS
• Game Loop Codelab: goo.gl/sn4RNp
• “Mechahamster” sample game
• Project home: goo.gl/8chQ8p
• Test Lab Guide: goo.gl/16xSPn
Slide 27
Slide 27 text
No content
Slide 28
Slide 28 text
No content
Slide 29
Slide 29 text
No content
Slide 30
Slide 30 text
No content
Slide 31
Slide 31 text
gcloud firebase test android run \
--type instrumentation \
--app app-debug-unaligned.apk \
--test app-debug-test-unaligned.apk \
--device model=Nexus6,version=21,orientation=portrait \
--device model=Nexus7,version=19,orientation=landscape
Slide 32
Slide 32 text
Community Support for testing with CI
• Walmart’s “Flank” for sharding tests
goo.gl/5ApxVU
• Circle CI integration
goo.gl/xoPFqY
Slide 33
Slide 33 text
No content
Slide 34
Slide 34 text
Billing for Firebase Test Lab
• Unpaid projects: daily limits
• Virtual devices: 10 tests/day
• Physical devices: 5 tests/day
• Paid projects: per-minute billing
• Virtual devices: $1/device/hour
• Physical devices: $5/device/hour
Slide 35
Slide 35 text
No content
Slide 36
Slide 36 text
No content
Slide 37
Slide 37 text
No content
Slide 38
Slide 38 text
No content
Slide 39
Slide 39 text
No content
Slide 40
Slide 40 text
Confidential + Proprietary
● Unity support
● NDK support
● Search
● Webhooks
Crashlytics Firebase Crash Reporting
● Tightly integrated with Analytics
○ Generates “app_exception”
events
○ Analytics events in crash logs
Slide 41
Slide 41 text
public class MyHandler implements Thread.UncaughtExceptionHandler {
private final Thread.UncaughtExceptionHandler mPriorExceptionHandler;
public MyHandler(Thread.UncaughtExceptionHandler prior) {
mPriorExceptionHandler = prior;
}
@Override
public void uncaughtException(Thread thread, Throwable throwable) {
// Deal with throwable here
if (mPriorExceptionHandler != null) {
mPriorExceptionHandler.uncaughtException(thread, throwable);
}
}
}
Slide 42
Slide 42 text
Thread.UncaughtExceptionHandler prior =
Thread.getDefaultUncaughtExceptionHandler();
Thread.setDefaultUncaughtExceptionHandler(
new MyUncaughtExceptionHandler(prior));
Slide 43
Slide 43 text
No content
Slide 44
Slide 44 text
Performance == Quality
Slide 45
Slide 45 text
60%
Speed, design & usability
When leaving a 5 star review, 60% of the time the user mentions
the speed, design or usability
Google analysis on Play reviews, top 10 English speaking countries, last 365 days of reviews
Slide 46
Slide 46 text
https://goo.gl/BnWcPK
Slide 47
Slide 47 text
Systrace
Advantages
• Tons of detail
• Custom traces
Disadvantages
• App runs slow
• You typically have to know what you're looking for
• Can't collect data from the wild, only connected devices
Slide 48
Slide 48 text
No content
Slide 49
Slide 49 text
Firebase Performance Monitoring
• Runs in production
• Negligible overhead
• Automatic and Custom traces
• Automatic HTTP/S traffic metrics
Slide 50
Slide 50 text
Trace
• A report for a period of time with a well-defined beginning and end
• Nominally has a duration
• Also may contain "counters" for performance-related events during the trace
Slide 51
Slide 51 text
Automatic Traces - no coding necessary!
• App Start (cold)
• Time in foreground
• Time in background
Slide 52
Slide 52 text
Automatic Traces - App Start
ContentProvider.onCreate() Activity.onResume()
But you might want something different!
Slide 53
Slide 53 text
Counters
• Associate a string/count value to a Trace
• Typically better to measure ratios than absolute values
• Abs value if you want to compare similar counters between traces
cache_hit++
cache_miss++
dropped_frames += 10
Slide 54
Slide 54 text
An idea for pseudo-automatic traces
Slide 55
Slide 55 text
public class PerfLifecycleCallbacks
implements Application.ActivityLifecycleCallbacks {
private static final PerfLifecycleCallbacks instance =
new PerfLifecycleCallbacks();
private PerfLifecycleCallbacks() {}
public static PerfLifecycleCallbacks getInstance() {
return instance;
}
// Continued...
Slide 56
Slide 56 text
private final HashMap traces =
new HashMap<>();
@Override
public void onActivityStarted(Activity activity) {
String name = activity.getClass().getSimpleName();
Trace trace = FirebasePerformance.startTrace(name);
traces.put(activity, trace);
}
@Override
public void onActivityStopped(Activity activity) {
Trace trace = traces.remove(activity);
trace.stop();
}
Slide 57
Slide 57 text
public Trace getTrace(Activity activity) {
return traces.get(activity);
}
Slide 58
Slide 58 text
public class PerfInitContentProvider extends ContentProvider {
@Override
public boolean onCreate() {
Context context = getContext();
if (context != null) {
Application app = (Application) context.getApplicationContext();
app.registerActivityLifecycleCallbacks(
PerfLifecycleCallbacks.getInstance());
}
return false;
}
// Overrides elided...
}
Slide 59
Slide 59 text
public class MyActivity extends Activity {
private void loadStuff() {
Trace trace = PerfLifecycleCallbacks.getInstance().getTrace(this);
trace.incrementCounter("foo");
}
}
Slide 60
Slide 60 text
Android Vitals - get clues for what to measure
• New in the Google Play Console: goo.gl/rkztG1
• Performance-related stats
• Slow rendering (jank)
• Frozen frames
• Stuck WakeLocks
Slide 61
Slide 61 text
Automatic HTTP/S transaction metrics
• Response time
• Payload size
• Success rate
• URL pattern globbing
yourcdn.com/*.jpg
api.yourdomain.com/v1/users/*
api.yourdomain.com/v1/users/*/history/*
Slide 62
Slide 62 text
HTTP/S transaction metrics breakdown
• App version
• Device
• Country
• OS level
• Carrier
• Radio
Slide 63
Slide 63 text
How HTTP/S monitoring works
Bytecode manipulation (with ASM)
URL url = new URL("https://firebase.google.com");
HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
URL url = new URL(“https://firebase.google.com”);
HttpsURLConnection conn = (HttpsURLConnection)
FirebasePerfUrlConnection.instrument(url.openConnection());
Decorator inside!
Slide 64
Slide 64 text
How HTTP/S monitoring works
dependencies {
compile 'com.google.firebase:firebase-perf:11.0.2'
}
apply plugin: 'com.google.firebase.firebase-perf'
Uses the Transform API: goo.gl/PwBcLE
Slide 65
Slide 65 text
No content
Slide 66
Slide 66 text
Firebase Remote Config
• Use it to plan and stage rollouts of new features
• Quickly back off the new feature if it has problems
Slide 67
Slide 67 text
No content
Slide 68
Slide 68 text
No content
Slide 69
Slide 69 text
No content
Slide 70
Slide 70 text
No content
Slide 71
Slide 71 text
FirebaseRemoteConfig rc = FirebaseRemoteConfig.getInstance();
if (rc.getBoolean("using_cool_new_feature")) {
// do cool stuff
}
else {
// do the boring old stuff
}
Slide 72
Slide 72 text
Firebase Remote Config
• Find out what your users prefer by experimentation
• Study the results in Google Analytics for Firebase
Slide 73
Slide 73 text
No content
Slide 74
Slide 74 text
Firebase Remote Config (for perf)
• Compare CDN/server performance
• Compare cache configuration strategies
Slide 75
Slide 75 text
final FirebaseRemoteConfig rc = FirebaseRemoteConfig.getInstance();
int cache_size = (int) rc.getLong("cache_size");
// init cache with cache_size
// later...
Trace trace = FirebasePerformance.startTrace("my_trace");
Item item = cache.get("some_resource");
if (item != null) {
trace.incrementCounter("cache_hit_size_" + cache_size);
}
else {
trace.incrementCounter("cache_miss_size_" + cache_size);
}
trace.stop();
Slide 76
Slide 76 text
Thank you!
Confidential + Proprietary
Doug Stevenson
@CodingDoug
youtube.com/firebase
firebase.googleblog.com