Android testing - Do not trust, Test!

A presentation about testing on Android.

Transcript

1. Don't Trust Don't Trust, , Test! Test! Android Testing Android Testing @rolios o.gonthier@gmail.com
2. About Me About Me Gonthier Olivier Freelance Developer/Trainer in Paris Contributor infoq FR Work at Orange Vallee on libon
3. Testing Testing What we will talk about What we will talk about
4. Why Tests? Why Tests? Testing is not easy, it requires time BUT You need stability in your apps AND tests are there to detect regressions SO You need to practice it.
5. Why Tests? Why Tests? The red guy write tests and try to maintain them. The black guys will do everything to break his tests.
6. Unit vs Integration Testing Unit vs Integration Testing Unit Testing Test an isolated component: A class, or a method. Integration Testing Test a component in its environment: test how it interacts for real.
7. Problems Problems What to use? What to test? How to be efficient? When should I run Tests?
8. Android Test Framework Android Test Framework Start with basics. Start with basics.
9. Android Test Framework Android Test Framework Part of the SDK Based on jUnit3 (...) Instrument Android Components
10. How it works How it works
11. How to use it How to use it Create a test project android create test-project -m <path-app> -n <project-name> -p <test-path> Run tests adb shell am instrument -w <package>/<runner-class> or, with maven mvn android:instrument
12. Android Test Framework Android Test Framework Let's analyse the classes provided! Let's analyse the classes provided!
13. TestsCases TestsCases There are two families of Test Cases InstrumentationTestCase TestCases for test Activities and Account Sync Provides Context, and Instrumentation AndroidTestCase TestCases for test Providers, Services, Loaders and Applications Provides Context, and permissions asserts
14. TestCases TestCases
15. TestCases TestCases InstrumentationTestCases InstrumentationTestCases SyncBaseInstrumentation - Start/cancel Sync syncProvider(Uri uri, String accountName, String authority) cancelSyncsandDisableAutoSync() ActivityUnitTestCase - Isolated test setApplication(Application application) startActivity(Intent intent, Bundle savedInstanceState, Object lastNonConfigurationInst setActivityContext(Context activityContext) Inner Mock Activity: class MockParent extends Activity ActivityInstrumentationTestCase2 - Functional test getActivity() setActivityIntent(Intent i)
16. TestCases TestCases InstrumentationTestCases InstrumentationTestCases SingleLaunchActivity Functional Test of an activity, but launched one time for all tests methods in the TestCase. (ActivityUnitTestCase and ActivityInstrumentationTestCase launch activity each time setUp() is called)
17. TestCases TestCases AndroidTestCases AndroidTestCases ApplicationTestCase - Launch and terminate Application createApplication() terminateApplication() getSystemContext() LoaderTestCase - get data synchronously from Loader getLoaderResultSynchronously(final Loader<T> loader) ServiceTestCase - Start, bind, stop services startService(Intent intent) bindService(Intent intent) shutdownService()
18. TestCases TestCases AndroidTestCases AndroidTestCases ProviderTestCases2 Test Isolated Provider, using a MockContext, and a MockContentResolver. Data is not saved at the same place as usual! It can also be used from other TestCases newResolverWithContentProviderFromSql(Context targetContext, String filenamePrefix, Cla
19. Mocks Mocks We can find various mocks MockApplication MockContentProvider MockContentResolver MockContext MockCursor MockDialogInterface MockPackageManager MockResources However, most of them are not really useful: they are only classes ske leton, that throw UnsupportedOperationException... ...Excepted MockContentResolver
20. Mocks Mocks MockContentResolver MockContentResolver A ContentResolver that allows you to plug yourself your providers! addProvider(String name, ContentProvider provider) notifyChange(Uri uri, ContentObserver observer, boolean syncToNetwork) In case of ContentProviderTestCase2, only one provider is plugged.
21. Mocks Mocks There are two classes that takes benefits of ContextWrapper to mock Contexts A contextWrapper is a Context, that delegate all method execution t o another Context. Subclass it for... Mock Context!
22. Mocks Mocks IsolatedContext The class keep a list of broadcast Intents we tried to send. And it provides also a MockAccountManager! Prevent tests from talking to the device: checkUriPermission() return permission Granted getFilesDir() return File("/dev/null") getSystemService() return null getAndClearBroadcastIntents()
23. Mocks Mocks RenamingDelegatingContext Performs Database and File operations with a renamed database/filename! providerWithRenamedContext(Class<T> contentProvider, Context c, String filePrefix) makeExistingFilesAndDbsAccessible()
24. Asserts Asserts ViewAsserts ViewAsserts Assert View and ViewGroup Objects Test: Presence of a view in screen Location of a view in screen Alignement ViewGroup contains view or not assertGroupContains(ViewGroup parent, View child) assertLeftAligned(View first, View second) assertHasScreenCoordinates(View origin, View view, int x, int y) assertOnScreen(View origin, View view)
25. Asserts Asserts MoreAsserts MoreAsserts Assert Java objects Test: Inheritance Equality Regex validation Order of Collection content Collection content assertEmpty(Map<?,?> map) assertContentsInOrder(Iterable<?> actual, Object... expected) assertMatchesRegex(String expectedRegex, String actual) assertEquals(int[] expected, int[] actual) assertAssignableFrom(Class<?> expected, Object actual)
26. Utils Utils TouchUtils TouchUtils Set of static methods to generate Touch Events Apply in InstrumentationTestCases Manipulate views Different types: drag, tap, long-click, scroll Locate a view scrollToTop(ActivityInstrumentationTestCase test, ViewGroup v) dragViewToBottom(ActivityInstrumentationTestCase test, View v) tapView(InstrumentationTestCase test, View v) longClickView(ActivityInstrumentationTestCase test, View v) getStartLocation(View v, int gravity, int[] xy)
27. Annotations Annotations Some allow us to classify tests @SmallTest, @MediumTest, @LargeTest, @Smoke One for suppress a test @Suppress One for define tolerance @FlakyTest(tolerance = 2) And one that force test to execute on UI Thread @UiThreadTest
28. Annotations Annotations In sources, there are other classes, not part of sdk for now. Some of these looks promising! @TimedTest, @BandwidthTest, @RepetitiveTest
29. Android Test Framework Android Test Framework Now we can use it! Now we can use it!
30. TestCases in Action TestCases in Action public class TestCarProvider extends ProviderTestCase2<CarProvider>{ public CarProviderTest() { super(CarProvider.class, "com.roly.carseller.carprovider"); } public void testInsert() { //Given CarProvider provider = getProvider(); Car car = new Car("BMW"); provider.insert(car); //When Car fromDB = provider.query(car.getId()); //Then assertEquals(car.getBrand(),fromDB.getBrand()); } } ¡This is not real code!
31. TestCases in Action TestCases in Action public class TestServerResponseParsing extends AndroidTestCase{ ServerResponseParser parser; @Override protected void setUp() throws Exception { super.setUp(); parser = ((ClientApplication) getContext().getApplicationContext()).getParser(); parser.prepare(); } public void testErrorResponse(){ ServerResult result = parser.parse(JSON_ERROR_EXAMPLE); assertEquals(result.getCode(), ServerResult.ERROR); } @Override protected void tearDown() throws Exception { super.tearDown(); parser.clearAllTasks(); }
32. TestCases in Action TestCases in Action public class MainActivityTest extends ActivityInstrumentationTestCase2<MainActivity>{ public MainActivityTest() { super(MainActivity.class); } public void testIntentValueUpdateTextView() { Intent intent = new Intent(); intent.putExtra("userId",45); setActivityIntent(intent); getInstrumentation().callActivityOnCreate(getActivity(), null); TextView view = (TextView) getActivity().findViewById(R.id.userIdText); assertNotNull(view.getText()); } }
33. Classify Classify @SmallTest public void testActivityGetValueFromIntent() @MediumTest public class TestServer extends ServiceTestCase<RestService> @LargeTest public void testSyncAllContacts() adb shell am instrsument -w -e size small ... mvn android:instrument -Dandroid.test.testSize=small ...
34. Organize Organize Make your own TestSuite In Manifest <instrumentation android:name="com.roly.mycoolapp.test.CoolAppInstrumentation" android In a subclass of InstrumentationTestRunner public class CoolAppInstumentation extends InstrumentationTestRunner{ @Override public TestSuite getAllTests() { TestSuite testSuite = new TestSuite(); testSuite.addTest(new TestSuiteBuilder(CoolAppInstrumentation.class) .includePackages("com.roly.mycoolapp.test.sync") .build()); return testSuite; } }
35. Organize Organize So you want to make a TestSuite based on Test size? You have to define "predicates". Predicate<TestMethod> smallTests = new Predicate<TestMethod>(){ @Override public boolean apply(TestMethod testMethod) { return testMethod.getAnnotation(SmallTest.class) != null; } } new TestSuiteBuilder(MyInstrumentation.class).include(...) .addRequirements(smallTests) .build(); ...
36. Simplify Simplify class MyTest extends AndroidTestCase { @Override @Suppress public void testAndroidTestCaseSetupProperly() ... class MyServiceTest extends ServiceTestCase { @Override @Suppress public void testServiceTestCaseSetUpProperly() ... Keep only what you care about
37. Take control Take control Where can we introduce Mocks? Contexts are great objects to convey Mocks In all AndroidTestCases: public void testMeImFamous(){ setContext(myMockContext); ... } In ContentProvider constructor: public ContentProvider(Context context, String readPermission, String writePermission, And everywhere you pass a Context.
38. Take Control Take Control And then subclass your Context to deliver other Mocks mResolver = new MockContentResolver(); MockContext mockContext = new MockContext() { @Override public ContentResolver getContentResolver() { return mResolver; } }; mProvider = new MyProvider(); mProvider.attachInfo(mockContext, null); mResolver.addProvider("com.roly.myapp.coolnessprovider", mProvider);
39. Isolate Isolate Try to isolate as much as possible your tests Use RenamingDelegatingContext, and IsolatedContext mContextWrapper = new RenamingDelegatingContext(getContext(), "test-"); mContext = new IsolatedContext(mResolver, mContextWrapper); It will prevent you from: Erase file Modify database Add crap account on your phone
40. Tips for Testing Tips for Testing You'll be stronger You'll be stronger
41. Async becomes Sync Async becomes Sync When you write tests, you don't want to run async code! You can force you test to wait if you get any callback or event Use Java's CountDownLatch for that. final CountDownLatch latch = new CountDownLatch(1); final BroadcastReceiver syncReceiver = new BroadcastReceiver() { @Override public void onReceive(Context context, Intent intent) { latch.countDown(); } } try { latch.await(60, TimeUnit.SECONDS); } catch (InterruptedException e) { fail(); }
42. Reflection Reflection In tests parts, you can enjoy Reflection API It can facilitate your life! //Get private field Field privateField = myObject.getClass.getDeclaredField("mParser"); //Set it accessible privateField.setAccessible(true); //Inject a value privateField.set(myObject, new Parser()); //Then invoke a method parseMethod.invoke(myObject, "stuff to parse");
43. Builders Builders Make builders so that you can construct Objects easily public class ContactBuilder { private String firstName; public ContactBuilder firstName(String firstName) { this.firstName = firstName; return this; } public Contact build() { Contact contact = new Contact(); contact.setFirstname(firstname); return contact; } } new ContactBuilder().firstName("tata").build(); new ContactBuilder().firstName("toto").build();
44. Random Random Randomize your test data is a great thing. (imho) BUT Don't do it yourself Suggestions RandomStringUtils (Apache commons lang) Random (Java.utils) DataFactory Great library that generate random consistent data Example: getName() can return "Peter"
45. Make Utils Make Utils Make high level static methods that wrap code reusable. createUser() login() deleteUser() forceDeleteUser() Facilitate test writing to other developers is important.
46. Initialize/Revert Initialize/Revert The Framework uses jUnit3, so @BeforeClass and @AfterClass don't exist. There are (crap) solutions to get around this Use constructor Use static block boolean set to true when first setUp Count visited tests Note that the class TestSetup from jUnit3 is not part of the framework
47. Libraries Libraries As usual, many community projects! As usual, many community projects!
48. Robotium Robotium Most famous library for testing Extension of the base framework Facilitate a lot manipulation of UI public void testDisplayBlackBox() { //Enter 10 in first edit-field solo.enterText(0,"10"); //Enter 20 in second edit-field solo.enterText(1,"20"); //Click on Multiply button solo.clickOnButton("Multiply"); //Verify that resultant of 10 x 20 assertTrue(solo.searchText("200")); }
49. Robolectric Robolectric It allows to run tests on an usual jvm It uses jUnit4! No need to deploy on device = So much time saved! @RunWith(HttpTest.RobolectricTestRunner.class) public class HttpTest { @Before public void setup() { Robolectric.setDefaultHttpResponse(200, "OK"); } @Test public void testGet_shouldApplyCorrectHeaders(){ HashMap<String, String> headers=new HashMap<String, String>(); headers.put("foo", "bar"); http.get("www.example.com", headers, null, null); HttpRequest sent = Robolectric.getSentHttpRequest(0); assertThat(sent.getHeaders("foo")[0].getValue(), equalTo("bar")); } }
50. FestAndroid FestAndroid FEST Assertions for Android FEST Assertions for Android assertEquals(View.GONE, view.getVisibility()); Ὃ assertThat(view).isGone(); assertThat(layout).isVisible() .isVertical() .hasChildCount(4) .hasShowDividers(SHOW_DIVIDERS_MIDDLE); And it's also extensible, you can had your own assertions
51. Mockito Mockito A powerful Mock Framework mock(Class) create a mock instance of your class verify(myMock).someMethod() verify that a method has been called when(myMock.getUser()).thenReturn(new User()) specify what to return when a m Context context = Mockito.mock(Context.class); Mockito.when(context.getContentResolver()) .thenReturn(Mockito.mock(ContentResolver.class)); //Do something Mockito.verify(context).getContentResolver();
52. Spoon Spoon Great tool to automate test on multi-devices Generate a cool web interface to get tests result
53. Industrialization Industrialization Automate all these tests! Automate all these tests!
54. Coverage Coverage For test Coverage, Framework suggest to use Emma adb shell am instrsument -w -e emma true ... mvn android:instrument -Dandroid.test.testCoverage=true ...
55. Continuous Integration Continuous Integration On Jenkins, you can find an android emulator plugin. It provides all you need to automate tests. Additionnally, if you want to get the state 'unstable', you need to add a property with maven. mvn android:instrument -Dmaven.test.failure.ignore=true ...
56. Further more Further more There are so much things that we haven't talked about! Selenium Mock environment Monkey Cucumber? ...
57. End End - Thanks a lot! Thanks a lot! Any Questions? Any Questions? Get slides! slid.es/r0ly/android-testing/ @rolios @rolios o.gonthier@gmail.com