Unit Testing & Android

Lecture on Android unit testing and instrumentation testing, mDevCamp 2015. Overview of available libraries and frameworks, and their capabilities and drawbacks.

Transcript

1. Unit Testing & Android Tomáš Kypta
2. Unit Testing Experience on Android
4. Android apps are difficult to test
6. Types of Android tests
7. Types of Android tests Instrumentation tests Unit tests
8. Instrumentation tests
9. Instrumentation tests • running on physical device or emulator • gradle connectedCheck
11. Intstrumentation tests • Legacy instrumentation tests or • Testing Support Library
12. Legacy instrumentation tests • JUnit3 • Tests extend from TestCase • AndroidTestCase • ActivityInstrumentationTestCase2 • ServiceTestCase • …
13. Testing Support Library • AndroidJUnitRunner • JUnit4 compatible android { defaultConfig { testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner" } } androidTestCompile 'com.android.support.test:runner:0.3'
 androidTestCompile 'com.android.support.test:rules:0.3'
14. Testing Support Library • Test filtering • @RequiresDevice - run on physical device, no emulator • @SdkSupress - don’t run on lower Android API level
15. Test Sizes • @SmallTest • @MediumTest • @LargeTest
16. Test Sizes Feature Small Medium Large Network access No localhost only Yes Database No Yes Yes File system access No Yes Yes Use external systems No Discouraged Yes Multiple threads No Yes Yes Sleep statements No Yes Yes System properties No Yes Yes Time limit (seconds) 60 300 900+
17. Potential problems • Manifest merger • problems when testing libraries • aar dependency using manifest placeholders • e.g. ${applicationId}, ${localApplicationId} android { defaultConfig { manifestPlaceholders = [localApplicationId:”com.example.mylib”] } }
18. Problems • It affects the device! • Wiping contacts will wipe contacts! • You can’t prepare all test preconditions • e.g. you can’t dynamically change permissions
19. Problems Test failed to run to completion. Reason: 'Instrumentation run failed due to 'java.lang.IllegalStateException''. Check device logcat for details
21. • framework for functional UI tests • part of Android Testing Support Library androidTestCompile 'com.android.support.test.espresso:espresso-core:2.2'
22. @Test public void sayHello(){ onView(withId(R.id.editText)) .perform(typeText(STRING_TO_BE_TYPED), closeSoftKeyboard()); onView(withText("Say hello!”)) .perform(click()); String expectedText = "Hello, " + STRING_TO_BE_TYPED + "!"; onView(withId(R.id.textView)) .check(matches(withText(expectedText))); }
23. Problems • testing on device is not isolated • device state affects the result • e.g. screen on/off might affect test result onView(withId(R.id.my_view)) .check(matches(isDisplayed()));
24. Instrumentation tests are SLOOOOOW
25. java.lang.RuntimeException: Stub!
26. Unit Tests
27. Unit Tests • run on JVM • mockable android.jar • gradle test
28. Grade and Android Studio support • natively supported since Gradle plugin 1.1.0 • big problems in AS in previous versions • has issues • switching between unit & instrumentation tests • disabled type is not indexed
31. • the essential piece • alone can be used only for pure Java • don’t use on Android APIs!
32. Method ... not mocked. android {
 testOptions {
 unitTests.returnDefaultValues = true
 }
 } • Helps rarely • returns 0, false, null, …
34. • mocking framework • easy to use • compatible with Android unit testing testCompile ‘org.mockito:mockito-core:1.10.19'
35. • can be used in instrumentation tests • needs dexmaker androidTestCompile ‘org.mockito:mockito-core:1.10.19' androidTestCompile "com.google.dexmaker:dexmaker:1.2" androidTestCompile "com.google.dexmaker:dexmaker-mockito:1.2"
36. • @RunWith(MockitoJUnitRunner.class) • @Mock • Mockito.mock() • Mockito.when().thenReturn() • when(obj.getA()).thenReturn(aInstance)
37. • Mockito.spy() • wrapping a real object • Mockito.verify() • verify that special condition are met • e.g. method called, called twice, …
38. Limitations • final classes • anonymous classes • primitive types • static methods
40. • at first, might be difficult to use • the ultimate mock of Android APIs • allows custom shadows • @RunWith(RobolectricTestRunner.class)
41. • supports Android API level 18 • doesn’t support multiple users on the same machine!!! 2.4
42. • Robolectric class splits into • Robolectric • RuntimeEnvironment • Shadows • ShadowApplication • ShadowLooper 3.0-rc3
43. that’s the s**t you need for Android unit testing
44. Potential problems • difficult to search for solutions • long history of bigger changes • many obsolete posts
45. Potential problems • difficulties running on command line and in AS • different paths • difficulty working with resources • RobolectricGradleTestRunner • doesn’t work in AS
46. Code Coverage
47. Code Coverage • unit tests • JaCoCo • instrumentation tests • EMMA • obsolete • Google is supposedly working on JaCoCo support
48. JaCoCo
49. JaCoCo • enabled by default for unit tests • generates binary report in build/jacoco • build/jacoco/testDebugUnitTest.exec • gradle test
52. Code Coverage • don’t use it • generates coverage-instrumented-classes • for instrumentation tests • coverage for instrumentation tests is not ready buildTypes {
 debug {
 testCoverageEnabled true
 } }
53. Good tests
54. Good tests • run in any order • run in isolation • run consistently • run fast • are orthogonal
55. How to write testable apps?
56. Rules of thumb • prefer pure Java • abstract away from Android APIs • separate business logic and UI • don’t write business logic into activities and fragments • try avoid static methods • use dependency injection
57. Questions?
58. References • Espresso • https://code.google.com/p/android-test-kit/ • https://code.google.com/p/android-test-kit/wiki/ EspressoV2CheatSheet
59. References • Mockito • http://mockito.org/ • https://github.com/mockito/mockito • Dexmaker • https://github.com/crittercism/dexmaker
60. References • Robolectric • http://robolectric.org/ • https://github.com/robolectric/robolectric
61. References • code coverage • JaCoCo • Eclipse Public License v1.0 • http://www.eclemma.org/jacoco/trunk/index.html • https://github.com/jacoco/jacoco • EMMA • Common Public License v1.0 • http://emma.sourceforge.net/