Tweet
Tweet

Slide 1

Slide 1 text

Modern Android Testing (Vincent) Aung Kyaw Paing Dev @ thoughtworks Android GDE

Slide 2

Slide 2 text

Android Testing Program Input Output

Slide 3

Slide 3 text

Android Testing Program Input Output [ Henry, Nicole, James ] “Henry, Nicole and James”

Slide 4

Slide 4 text

Android Testing Program Input Output [ Henry, Nicole, James ] “Henry, Nicole and James” Arrange Act Assert

Slide 5

Slide 5 text

@Test fun `test returns readbale string given a list of name` { // Arrange val input = listOf("Henry", "Nicole", "James") val expected = "Henry, Nicole and James" // Act val output = formatNames(input) // Assert assertEqual(expected, output) }

Slide 6

Slide 6 text

Type of testing Android Testing

Slide 7

Slide 7 text

Testing Pyramid Type of Testing closer to real world Slower, more $$$ E2E UI Test Integration Test Unit Test Manual Test

Slide 8

Slide 8 text

Testing Pyramid Type of Testing closer to real world Slower, more $$$ A11y Test E2E Test Exploratory Test Smoke Test Screenshot Test Contract Tests Unit Tests Form Factor Tests I18n test Performance Test

Slide 9

Slide 9 text

Unit Tests Type of Testing

Slide 10

Slide 10 text

Unit Tests Unit Test Test smallest unit of your software. A unit could be a function, a class, or a component Sign up Button Login Page Username Text Field Forgot password button Login Component Password Text Field Login Button Input Fields

Slide 11

Slide 11 text

Unit Tests Unit Test Test smallest unit of your software. A unit could be a function, a class, or a component Sign up Button Login Page Username Text Field Forgot password button Login Component Password Text Field Login Button Input Fields

Slide 12

Slide 12 text

Unit Tests Unit Test Test smallest unit of your software. A unit could be a function, a class, or a component Sign up Button Login Page Username Text Field Forgot password button Login Component Password Text Field Login Button Input Fields

Slide 13

Slide 13 text

class LoginButton( val viewModel: ViewModel ) { val status = state("unknown") Button( onClick= { viewModel.login().catch { status = "Failure" } status = "Success" } ) Text(status) }

Slide 14

Slide 14 text

@Test fun `test login press success state` { // TODO: Arrange // Act performClick() // Assert assertIsDisplayed("Success") }

Slide 15

Slide 15 text

Test Double Unit Test

Slide 16

Slide 16 text

Mocking Test Double Technique to replacing a implementation of its dependencies to achieve a behavior you want

Slide 17

Slide 17 text

class LoginViewModel() : ViewModel() { fun onLoginClick() { //... } }

Slide 18

Slide 18 text

interface LoginViewModel { fun onLoginClick() }

Slide 19

Slide 19 text

@Test fun `test login press success state` { val mockViewModel = object: LoginViewModel { fun onLoginClick() { // Do Nothing } } // Act performClick() // Assert assertIsDisplayed("Success") }

Slide 20

Slide 20 text

@Test fun `test login press success state` { val mockViewModel = object: LoginViewModel { fun onLoginClick() { // Do Nothing } } // Act performClick() // Assert assertIsDisplayed("Success") } class LoginButton( val viewModel: ViewModel ) { val status = state("unknown") Button( onClick= { viewModel.login().catch { status = "Failure" } status = "Success" } ) Text(status) }

Slide 21

Slide 21 text

@Test fun `test login press success state` { val mockViewModel = object: LoginViewModel { fun onLoginClick() { // Do Nothing } } // Act render(LoginButton(mockViewModel)) performClick() // Assert assertIsDisplayed("Success") }

Slide 22

Slide 22 text

interface LoginViewModel { fun onLoginClick() }

Slide 23

Slide 23 text

interface LoginViewModel { fun onLoginClick() fun onSignUpClick() fun onNavigateBack() fun onOpenTermsAndConditions() ... }

Slide 24

Slide 24 text

val mockViewModel = object: LoginViewModel { fun onLoginClick() { ... } fun onSignUpClick() { ... } fun onNavigateBack() { ... } fun onOpenTermsAndConditions() { ... } }

Slide 25

Slide 25 text

@Test fun `test login press success state` { val mockViewModel = object: LoginViewModel { fun onLoginClick() { ... } fun onSignUpClick() { ... } fun onNavigateBack() { ... } fun onOpenTermsAndConditions() { ... } // Act render(LoginButton(mockViewModel)) performClick() // Assert assertIsDisplayed("Success") }

Slide 26

Slide 26 text

@Test fun `test login press success state` { val mockViewModel = mockk() every { mockViewModel.onLoginClick() } answers { // Do Nothing } // Act render(LoginButton(mockViewModel)) performClick() // Assert assertIsDisplayed("Success") }

Slide 27

Slide 27 text

Mocking’s Drawbacks Test Double Every time a dependencies change the contract, the test breaks Brittle

Slide 28

Slide 28 text

class LoginViewModel { fun setUserName(username: String) { ... } fun setPassword(password: String) { ... } fun onLoginClick() { //... } }

Slide 29

Slide 29 text

@Test fun `test call login api on press` { val mockViewModel = object: LoginViewModel{ fun setUsername(username: String) { ... } fun setPassword(password: String) { ... } fun onLoginClick() { ... } } ... verify(times = 1) { mockLoginApi.execute(...) } }

Slide 30

Slide 30 text

class LoginViewModel { fun onLoginClick( username: String password: String ) { //... } }

Slide 31

Slide 31 text

@Test fun `test call login api on press` { val mockViewModel = object: LoginViewModel{ fun setUsername(username: String) { ... } fun setPassword(password: String) { ... } fun onLoginClick() { ... } } ... verify(times = 1) { mockLoginApi.execute(...) } }

Slide 32

Slide 32 text

Mocking Drawbacks Test Double Every time a dependencies change the contract, the test breaks Brittle Implementation Leak A test should avoid testing specific implementation but mocking do exactly that

Slide 33

Slide 33 text

@Test fun `test login press` { val mockViewModel = object: LoginViewModel{ fun setUsername(username: String) { ... } fun setPassword(password: String) { ... } fun onLoginClick() { ... } } ... verify(times = 1) { mockLoginApi.execute(...) } }

Slide 34

Slide 34 text

Mocking Drawbacks Unit Tests Every time a dependencies change the contract, the test breaks Brittle Implementation Leak Too much = Bad A test should avoid testing specific implementation but mocking do exactly that Mocking too much could leads to you not really testing anything at all since everything is a mock

Slide 35

Slide 35 text

class Auth( networkClient: NetworkClient, userStore: UserStore ) { fun login() { val res = networkClient.login() // Save locally userStore.put(res) } } @Test fun `test auth login`() { val auth = Auth( mockNetworkClient mockUserCache ) auth.login() // Assert verify { mockUserCache.put(mockRes) } }

Slide 36

Slide 36 text

Classicist Approach Unit Tests An approach to inside-out testing where we use test doubles(fake) instead of mocks to replicate actual behavior

Slide 37

Slide 37 text

class LoginButton( val viewModel: ViewModel ) { val status = state("unknown") Button( onClick= { viewModel.login().catch { status = "Failure" } status = "Success" } ) Text(status) }

Slide 38

Slide 38 text

class LoginButton( val viewModel: ViewModel ) { val status = state("unknown") Button( onClick= { viewModel.login().catch { status = "Failure" } status = "Success" } ) Text(status) }

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

class FakeLoginApi: LoginApi { val userList = listOf( User(id=0, name = "abc", pass="abc") ) fun login(name : String, pass: String) : User? { return userList.find { user -> user.name == name && user.pass == pass } } }

Slide 43

Slide 43 text

class FakeLoginApi: LoginApi { val userList = mutabeListOf( User(name = "abc", pass="abc") ) fun login(name : String, pass: String) : User? { return userList.find { user -> user.name == name && user.pass == pass } } fun signUp(name: Sring, pass: String) { userList.add(name, pass) } }

Slide 44

Slide 44 text

@Test fun `test login press success state` { val fakeLoginApi = FakeLoginApi() val viewModel = LoginViewModel(fakeLoginApi) // Act render(LoginButton(viewModel)) performClick() // Assert assertIsDisplayed("Success") }

Slide 45

Slide 45 text

Classicist Tradeoffs Unit Tests Test covers larger areas but missing out on details Larger test surface Flakiness Complex setup Fakes often replicate actual behavior, leading to unreliable tests if not used properly (e.g threading) Faking a part of your system is not easy and could take a chunk of your efforts

Slide 46

Slide 46 text

DEVICE FRAME - PIXELBOOK Mockist or Classicist It’s up to whatever approach make your team have confident in shipping your app Good reads: https://martinfowler.com/articles/mocksArentStubs.html https://martinfowler.com/articles/2021-test-shapes.html

Slide 47

Slide 47 text

Unit Tests Common Mistakes Unit Test More specific tests make the tests susceptible to breaking when production code changes with end result still being the same (e.g 5x2 = 10 and 7+3= 10) Testing specific implementation

Slide 48

Slide 48 text

Unit Tests Common Mistakes Unit Test More specific tests make the tests susceptible to breaking when production code changes with end result still being the same (e.g 5x2 = 10 and 7+3= 10) Testing specific implementation Testing libraries’ code Assume libraries are already tested. We should only test the code that we own.

Slide 49

Slide 49 text

Unit Tests Common Mistakes Unit Test More specific tests make the tests susceptible to breaking when production code changes with end result still being the same (e.g 5x2 = 10 and 7+3= 10) Testing specific implementation Testing libraries’ code Testing Configs Assume libraries are already tested. We should only test the code that we own. You are prone to making human mistakes the same way you’d make when writing a config file

Slide 50

Slide 50 text

Integration Tests Type of Testing

Slide 51

Slide 51 text

Integration Tests Type of Testing Test the interaction between your components and the Android system

Slide 52

Slide 52 text

Integration Tests Type of Testing Test the interaction between your components and the Android system Persistence Storage Notification Scheduler I18n Alarms

Slide 53

Slide 53 text

No content

Slide 54

Slide 54 text

@RunWith(AndroidJUnit4::class) @Config(sdk = { MARSHPMALLOW, OREO }) class UserCacheTest { ... }

Slide 55

Slide 55 text

@RunWith(AndroidJUnit4::class) class UserCacheTest { @Test fun `save cache in json"() { // Arrange val sharedPref = getContext().sharedPref("user") val user = User() // Act userCache.put(user) // Assert val expected = user.toJson() val actual = sharedPref.get(user) Assert.assertEquals(expected, actual) } }

Slide 56

Slide 56 text

UI Tests Type of Testing

Slide 57

Slide 57 text

UI Tests Type of Testing Test and verify your UI interactions are correct. Most of the time, these are higher level tests. You’d want to use fakes instead of mocks

Slide 58

Slide 58 text

@RunWith(AndroidJUnit4::class) class LoginScreenTest { @Test fun `show success on pressing Login`() { // Arrange composeTestRule.setContent { LoginScreen() } // Act composeTestRule.onNodeWithText("Login").performClick() // Assert composeTestRule.onNodeWithText("Login Success!").assertIsDisplayed() } }

Slide 59

Slide 59 text

@RunWith(AndroidJUnit4::class) class LoginScreenTest { @Test fun `show success on pressing Login`() { // Arrange composeTestRule.setContent { LoginScreen() } // Act composeTestRule.onNodeWithText("Login").performClick() // Assert Espresso.onView(withText("Login Success!")).check(matches(isDisplayed())) } }

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

No content

Slide 62

Slide 62 text

src/test/*** src/androidTest/*** Directory for Unit Tests & Robolectric Test Directory for instrumentation tests that execute on real devices (or emulators)

Slide 63

Slide 63 text

src/test/LoginScreenTest.kt @RunWith(AndroidJUnit4::class) class LoginScreenTest { @Test fun `show success on pressing Login`() { // Arrange composeTestRule.setContent { LoginScreen() } // Act composeTestRule.onNodeWithText("Login").performClick() // Assert composeTestRule.onNodeWithText("Login Success!").assertIsDisplayed() } }

Slide 64

Slide 64 text

src/androidTest/LoginScreenTest.kt class LoginScreenTest { @Test fun `show success on pressing Login`() { // Arrange composeTestRule.setContent { LoginScreen() } // Act composeTestRule.onNodeWithText("Login").performClick() // Assert composeTestRule.onNodeWithText("Login Success!").assertIsDisplayed() } }

Slide 65

Slide 65 text

UI Tests Type of Testing Real Device Fast to run Far from real world Slow to run Closet to real world

Slide 66

Slide 66 text

UI Tests Type of Testing Real Device Fast to run Far from real world Slow to run Closet to real world

Slide 67

Slide 67 text

Gradle Managed Devices Unit Tests Run tests in a scaleable way by utilizing gradle to manage the lifecycle of virtual devices. Available for API 27+ 🖥 PC 🐘 Gradle 📱Emulator

Slide 68

Slide 68 text

android { testOptions { managedDevices { devices { maybeCreate("pixel5api30").apply { device = "Pixel 5" apiLevel = 30 systemImageSource = "aosp" } } } } }

Slide 69

Slide 69 text

android { testOptions { managedDevices { devices { maybeCreate("pixel5api30").apply { device = "Pixel 5" apiLevel = 30 systemImageSource = "aosp" } } } } }

Slide 70

Slide 70 text

android { testOptions { managedDevices { devices { maybeCreate("pixel5api30").apply { device = "Pixel 5" apiLevel = 30 systemImageSource = "aosp" } } } } }

Slide 71

Slide 71 text

android { testOptions { managedDevices { devices { maybeCreate("pixel5api30").apply { device = "Pixel 5" apiLevel = 30 systemImageSource = "aosp" } } } } }

Slide 72

Slide 72 text

android { testOptions { managedDevices { devices { maybeCreate("pixel5api30").apply { device = "Pixel 5" apiLevel = 30 systemImageSource = "aosp" } } } } }

Slide 73

Slide 73 text

android { testOptions { managedDevices { devices { maybeCreate("pixel5api30").apply { device = "Pixel 5" apiLevel = 30 systemImageSource = "aosp" } } } } }

Slide 74

Slide 74 text

android { testOptions { managedDevices { devices { maybeCreate("pixel5api30").apply { device = "Pixel 5" apiLevel = 30 systemImageSource = "google" } } } } }

Slide 75

Slide 75 text

./gradlew {Name}{BuildType}AndroidTest ./gradlew pixel5api30DebugAndroidTest

Slide 76

Slide 76 text

managedDevices { devices { maybeCreate("pixel3api30").apply { ... } maybeCreate<.ManagedVirtualDevice>("pixelTabletApi30").apply { ... } } groups { maybeCreate("phoneAndTablet").apply { targetDevices.add(devices["pixel3api30"]) targetDevices.add(devices["pixelTabletApi30"]) } } }

Slide 77

Slide 77 text

managedDevices { devices { maybeCreate("pixel3api30").apply { ... } maybeCreate<.ManagedVirtualDevice>("pixelTabletApi30").apply { ... } } groups { maybeCreate("phoneAndTablet").apply { targetDevices.add(devices["pixel3api30"]) targetDevices.add(devices["pixelTabletApi30"]) } } }

Slide 78

Slide 78 text

managedDevices { devices { maybeCreate("pixel3api30").apply { ... } maybeCreate<.ManagedVirtualDevice>("pixelTabletApi30").apply { ... } } groups { maybeCreate("phoneAndTablet").apply { targetDevices.add(devices["pixel3api30"]) targetDevices.add(devices["pixelTabletApi30"]) } } }

Slide 79

Slide 79 text

./gradlew {Name}Group{BuildType}AndroidTest ./gradlew phoneAndTabletGroupDebugAndroidTest

Slide 80

Slide 80 text

Automated Test Device (ATD) UI Tests Reduces CPU and Memory usage by avoiding rendering to a screen. However, it still supports screenshot testing Lightweight Optimized Consistent Apps and services that are not likely to be used are removed to improve performance. It’s still an emulator device and built from the same images, that you use in Android Studio, so test results are consistent across local and CI.

Slide 81

Slide 81 text

UI Tests

Slide 82

Slide 82 text

Firebase Test Lab UI Tests Scale your instrumentation tests with Firebase test matrix Instrumentation Robo Test Game Loop Automatically run tests as how user would behave with no code Run test in your native game engine

Slide 83

Slide 83 text

Firebase Console Firebase Test Labs

Slide 84

Slide 84 text

From Android Studio Firebase Test Labs

Slide 85

Slide 85 text

With Gradle Managed Devices Firebase Test Labs An experimental API that runs the ATDs on Firebase Test Labs with real devices instead of emulators. Available on Android Studio Canary Build

Slide 86

Slide 86 text

plugins { ... id("com.google.firebase.testlab") }

Slide 87

Slide 87 text

firebaseTestLab { managedDevices { create("ftlDevice") { // Use device from Firebase device = "Pixel3" apiLevel = 30 } } }

Slide 88

Slide 88 text

./gradlew {Name}{BuildType}AndroidTest ./gradlew ftlDeviceDebugAndroidTest

Slide 89

Slide 89 text

Firebase Test Labs

Slide 90

Slide 90 text

Screenshot Test Type of Testing Also known as snapshot testing, these tests captures a copy of screenshot of the app and matches it against the copy to verify that UI has not broken yet. These tests allow you to iterate faster and make sure it cover edge cases like styling. screenshot-tests-for-android happo.io

Slide 91

Slide 91 text

@Test fun testScreenshot() { val scenario = launchActivity() scenario.onActivity { compareScreenshot(it) } }

Slide 92

Slide 92 text

https://developer.android.com/studio/preview Android Gradle Plugin 8.2

Slide 93

Slide 93 text

E2E Tests Type of Testing

Slide 94

Slide 94 text

E2E Tests Type of Testing Verify user flow is working as intended from beginning to end on a real device in a production or production-like environment. This is the most expensive form of testing and most often used as a form of regression testing. Black Box Testing

Slide 95

Slide 95 text

UIAutomator Type of Testing Testing framework for cross-app testing across the system and installed apps. Most modern end-to-end testing frameworks use UIAutomator API underneath the hood.

Slide 96

Slide 96 text

val device = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation()) // Go back to home device.pressHome() val gmail: UiObject2 = device.findObject(By.text("Gmail")) // Perform a click and wait until the app is opened. val opened: Boolean = gmail.clickAndWait(Until.newWindow(), 3000) assertThat(opened).isTrue()

Slide 97

Slide 97 text

E2E Tests Type of Testing wdio cypress detox maestro

Slide 98

Slide 98 text

appId: com.example.login --- - launchApp - tapOn: "username" - inputText: "Aung" - tapOn: "password" - inputText: "abcedfg" - tapOn: "Login"

Slide 99

Slide 99 text

Firebase Test Lab UI Tests Robo Test Automatically run tests as how user would behave with no code

Slide 100

Slide 100 text

E2E UI Test Integration Test Unit Test Manual Test

Slide 101

Slide 101 text

E2E UI Test Integration Test Unit Test Manual Test JUnit, Mockk, Mockito Local Location API

Slide 102

Slide 102 text

E2E UI Test Integration Test Unit Test Manual Test Robolectric Local JUnit, Mockk, Mockito Local Location API

Slide 103

Slide 103 text

E2E UI Test Integration Test Unit Test Manual Test Espresso, Compose UI Test Local through Gradle Managed Device / Firebase Test Lab Robolectric Local JUnit, Mockk, Mockito Local Location API

Slide 104

Slide 104 text

E2E UI Test Integration Test Unit Test Manual Test UIAutomator, Robo Script Real Device / Firebase Test Lab Espresso, Compose UI Test Local through Gradle Managed Device / Firebase Test Lab Robolectric Local JUnit, Mockk, Mockito Local Location API

Slide 105

Slide 105 text

Manual Testing Type of Testing

Slide 106

Slide 106 text

Firebase App Distribution Manual Testing Streamline your testing process by releasing builds through Firebase App Distribution and collect feedback easily

Slide 107

Slide 107 text

Accessibility test Manual Testing Accessibility, by nature, is a user experience and you can’t test user experience with a machine or an automated tool. These should be tested by an individual, preferably a person with disabilities A11y scanner

Slide 108

Slide 108 text

DEVICE FRAME - PIXELBOOK Checks Make sure your app stay compliant and up to date with policy changes in your market Sign up for early access at checks.google.com

Slide 109

Slide 109 text

Thank You aungkyawpaing.dev @vincentpaing