Testing, how hard can it be? (Droidcon Lisbon 2022)

When people start looking into the testing domain, very similar questions arise:
What to test? And more important, what not? What should I mock? What should I test with unit tests and what with Instrumentation? Do I need Robolectric? When? Do we need an in-memory database?
Let’s look at some examples.

Transcript

1. 1
   We lead what's next in music
   Testing how hard can it be?
   Danny Preussler, SoundCloud ¯\_(ツ)_/¯
   

   View Slide

2. 2
   We lead what's next in music
   

   View Slide

3. 3
   We lead what's next in music
   

   View Slide

4. 4
   We lead what's next in music
   

   View Slide

5. 5
   We lead what's next in music
   

   View Slide

6. 6
   We lead what's next in music
   

   View Slide

7. “Code without tests is bad code.”
   (Michael C. Feathers)
   

   View Slide

8. “any code without test
   is a legacy code.”
   (Michael C. Feathers)
   

   View Slide

9. “how do you know something works
   when you don’t have test for it?”
   (Robert ‘Uncle Bob’ Martin)
   

   View Slide

10. “Refactoring
    without good test coverage
    is changing shit”
    (Martin Fowler)
    

    View Slide

11. Android testing
    strategy
    Theory Unit tests
    Manual tests
    UI tests
    

    View Slide

12. Android testing
    strategy
    Reality
    Unit tests
    Manual tests
    UI tests
    Unit tests
    ¯\_(ツ)_/¯
    

    View Slide

13. Today’s task - Write the Login feature for a
    Wear OS client
    - Authentication via phone
    OAuth
    

    View Slide

14. @Composable
    fun LoginScreen() {
    val viewModel = hiltViewModel()
    Box(modifier = Modifier.fillMaxSize()) {
    Button(onClick = { viewModel.login() }) {
    Text(viewModel.loginStatus.name)
    }
    }
    }
    

    View Slide

15. @HiltViewModel
    class LoginViewModel
    @Inject constructor(
    private val operations: AuthOperations
    ) : ViewModel() {
    enum class LoginStatus {
    NOT_LOGGED_IN, LOGGED_IN, LOGGING_IN
    }
    var loginStatus: LoginStatus by mutableStateOf(NOT_LOGGED_IN)
    

    View Slide

16. A first test
    

    View Slide

17. @Test
    fun `initially its not logged in`() {
    tested.loginStatus `should be` NOT_LOGGED_IN
    }
    

    View Slide

18. fun login(scope: CoroutineScope = viewModelScope) {
    loginStatus = LOGGING_IN
    scope.launch {
    try {
    operations.login()
    loginStatus = LOGGED_IN
    } catch(e: Exception) {
    e.printStackTrace()
    loginStatus = NOT_LOGGED_IN
    }
    }
    }
    

    View Slide

19. Easy to test
    

    View Slide

20. @Nested
    inner class `when logging in` {
    @Test
    fun `should set loading status`() = runTest {
    tested.login(this)
    tested.loginStatus `should be` LOGGING_IN
    }
    

    View Slide

21. @Nested
    inner class `when logging in successful` {
    @Test
    fun `should set status to success`{
    runTest {
    tested.login(this)
    }
    tested.loginStatus `should be` LOGGED_IN
    }
    

    View Slide

22. @Nested
    inner class `when logging in successful` {
    init {
    runTest { tested.login(this) }
    }
    @Test
    fun `should set status to success`{
    tested.loginStatus `should be` LOGGED_IN
    }
    

    View Slide

23. authentication
    

    View Slide

24. class AuthOperations
    @Inject constructor(
    private val authClient: AuthClient,
    private val apiClient: ApiClient
    ) {
    suspend fun login(): AuthToken {
    val code = authClient.authenticate()
    return apiClient.registerCodeForToken(code)
    }
    }
    

    View Slide

25. val authClient = mock {
    onBlocking { authenticate(any()) } doReturn OAuthCode("123")
    }
    

    View Slide

26. scope.launch {
    try {
    val token = operations.login()
    settings.authToken = token
    loginStatus = LOGGED_IN
    } catch(e: Exception) {
    e.printStackTrace()
    loginStatus = NOT_LOGGED_IN
    }
    }
    

    View Slide

27. @Nested
    inner class `when logging in successful` {
    init {
    runTest { tested.login(this) }
    }
    @Test
    fun `should authenticate`() {
    verifyBlocking(authClient) { authenticate(any()) }
    }
    }
    

    View Slide

28. Lets write the
    ApiClient
    

    View Slide

29. interface ApiService {
    @FormUrlEncoded
    @POST("/oauth2/token")
    suspend fun getToken(
    @Field("code") code: String,
    @Field("grant_type") grantType: String,
    ): TokenResponse
    }
    

    View Slide

30. class ApiClient
    @Inject constructor(
    private val apiService: ApiService
    ) {
    suspend fun registerCodeForToken(code: OAuthCode): AuthToken {
    return apiService.getToken(
    grantType = "authorization_code",
    code = code.value
    ).toToken()
    }
    }
    

    View Slide

31. val authClient = mock {
    onBlocking { authenticate(any()) } doReturn OAuthCode("123")
    }
    

    View Slide

32. val authClient = mock {
    onBlocking { authenticate(any()) } doReturn OAuthCode("123")
    }
    val service = mock{
    onBlocking { getToken(any()) } doReturn TokenResponse("456")
    }
    val tested = LoginViewModel(
    AuthOperations(authClient, ApiClient(service))
    )
    )
    

    View Slide

33. val authClient = mock {
    onBlocking { authenticate(any()) } doReturn OAuthCode("123")
    }
    val service = mock{
    onBlocking { getToken(any()) } doReturn TokenResponse("456")
    }
    val tested = LoginViewModel(
    AuthOperations(authClient, ApiClient(service))
    )
    )
    

    View Slide

34. @Test
    fun `should authenticate`() {
    verifyBlocking(service) {
    token(
    code = "123",
    grantType = "authorization_code"
    )
    )
    

    View Slide

35. @Nested
    inner class `when logging in successful` {
    @Test
    fun `should authenticate`() {
    runTest { tested.login(this) }
    verifyBlocking(authClient) { authenticate(any()) }
    }
    }
    

    View Slide

36. val authClient = mock {
    onBlocking { authenticate(any()) } doReturn OAuthCode("123")
    }
    val service = mock{
    onBlocking { getToken(any()) } doReturn TokenResponse("456")
    }
    val tested = LoginViewModel(
    AuthOperations(authClient, ApiClient(service))
    )
    )
    

    View Slide

37. val authClient = object : OAuthClient {
    override suspend fun authenticate(clientId: ClientId) =
    OAuthCode("123")
    }
    val service = mock{
    onBlocking { getToken(any()) } doReturn TokenResponse("456")
    }
    val tested = LoginViewModel(
    AuthOperations(authClient, ApiClient(service))
    )
    )
    

    View Slide

38. Use “the real thing” where possible
    

    View Slide

39. What about the
    token?
    

    View Slide

40. scope.launch {
    try {
    operations.login()
    loginStatus = LOGGED_IN
    } catch(e: Exception) {
    e.printStackTrace()
    loginStatus = NOT_LOGGED_IN
    }
    }
    

    View Slide

41. interface Settings {
    var authToken: AuthToken?
    }
    

    View Slide

42. private class InMemorySettings: Settings {
    override var authToken: AuthToken? = null
    }
    val tested = LoginViewModel(
    InMemorySettings(),
    AuthOperations(authClient, ApiClient(service))
    )
    

    View Slide

43. @Nested
    inner class `when logging in successful` {
    init {
    runTest { tested.login(this) }
    }
    @Test
    fun `should save value`{
    settings.authToken `should be equal to` AuthToken("456")
    }
    

    View Slide

44. I don’t like this
    

    View Slide

45. class PersistingAuthOperations
    @Inject constructor(
    private val settings: Settings,
    private val authOperation: AuthOperations
    ){
    suspend fun login(): AuthToken {
    return authOperation.login().also {
    settings.authToken = it
    }
    }
    }
    

    View Slide

46. val tested = LoginViewModel(
    PersistingAuthOperations(
    settings,
    AuthOperations(authClient, ApiClient(service))
    )
    )
    ¯\_(ツ)_/¯
    

    View Slide

47. Refactoring should not lead to test changes
    

    View Slide

48. Let’s save the token
    

    View Slide

49. interface TokenStorage {
    var authToken: AuthToken?
    }
    interface Settings : TokenStorage
    

    View Slide

50. class SharedPreferenceTokenStorage(
    context: Application,
    val authPrefs: Lazy = encrypted(context)
    ) : TokenStorage {
    override var authToken: AuthToken?
    set(value) {
    authPrefs.value.edit {
    putString(KEY_TOKEN, value?.token)
    }
    }
    

    View Slide

51. @RestrictTo(RestrictTo.Scope.TESTS)
    class SharedPreferencesStub : SharedPreferences {
    private val editor: EditorStub = EditorStub(this)
    // ..
    }
    private class EditorStub(
    val parent: SharedPreferences,
    val items: HashMap = hashMapOf())
    : Editor
    

    View Slide

52. Prefer Stubs over mocks, reuse them!
    

    View Slide

53. The auth client
    

    View Slide

54. internal class WearOAuthClient(
    private val context: Application,
    private val authClient: RemoteAuthClient =
    RemoteAuthClient.create(context)
    ) : OAuthClient {
    

    View Slide

55. override suspend fun authenticate(): OAuthCode =
    suspendCoroutine { continuation ->
    authClient.sendAuthorizationRequest( // ..
    object : RemoteAuthClient.Callback() {
    override fun onAuthorizationResponse(response: OAuthResponse) {
    continuation.resumeWith(success(OAuthCode(response.code)))
    }
    override fun onAuthorizationError(errorCode: Int) {
    continuation.resumeWith(
    failure(IOException("failed: $errorCode")))
    }
    }
    }
    

    View Slide

56. override suspend fun authenticate(): OAuthCode =
    suspendCoroutine { continuation ->
    authClient.sendAuthorizationRequest( // ..
    object : RemoteAuthClient.Callback() {
    override fun onAuthorizationResponse(response: OAuthResponse) {
    continuation.resumeWith(success(OAuthCode(response.code)))
    }
    override fun onAuthorizationError(errorCode: Int) {
    continuation.resumeWith(
    failure(IOException("failed: $errorCode")))
    }
    }
    }
    

    View Slide

57. @RunWith(RobolectricTestRunner::class)
    @RobolectricNeededAsOf(RemoteAuthClient::class)
    class WearOAuthClientTest {
    val code = OAuthCode("789")
    val redirect = "http://test?code=$code"
    val successRequest = mock {
    on { sendAuthorizationRequest(any(), any(), any()) } doAnswer {
    val request = it.arguments[0] as OAuthRequest
    val response = Builder().setResponseUrl(redirect).build()
    (it.arguments[2] as Callback).onAuthorizationResponse(response)
    }
    } o_o
    

    View Slide

58. @Test
    fun `should extract the code when successful`() = runTest {
    tested = WearOAuthClient(
    application = RuntimeEnvironment.getApplication(),
    authClient = successRequest
    )
    tested.authenticate() `should be equal to` code
    }
    

    View Slide

59. For external dependencies
    Use Mocks and Robolectric
    

    View Slide

60. Back to api
    

    View Slide

61. Refactorings:
    ● AuthOperations -> AuthRepository
    ● ApiClient used in new ApiDataSource
    No test change ¯\_(ツ)_/¯
    

    View Slide

62. internal class ApiAuthDataSource
    @Inject
    constructor(
    val api: Lazy
    ) : AuthDataSource {
    

    View Slide

63. val service = JsonReadingAuthService(
    "/api/public_api_token.json",
    AppModule.providesMoshi()
    )
    val tested = ApiAuthDataSource { service }
    

    View Slide

64. @Test
    fun `should map token`() = runTest {
    response = tested.registerCodeForToken(Code(123))
    response.token `should be equal to`
    "TH1S-1S-JUST-A-T3ST-T0KEN"
    }
    

    View Slide

65. Test the JSON from your backend
    

    View Slide

66. Let’s test the UI
    

    View Slide

67. class LoginViewModelStub : LoginViewModel() {
    override fun login() {
    loginStatus = LOGGED_IN
    }
    }
    

    View Slide

68. abstract class LoginViewModel : ViewModel() {
    enum class LoginStatus {
    LOGGED_IN, LOGGING_IN, NOT_LOGGED_IN
    }
    var loginStatus: LoginStatus by mutableStateOf(NOT_LOGGED_IN)
    abstract fun login()
    // ..
    viewModel: LoginViewModel = hiltViewModel()
    

    View Slide

69. @RunWith(RobolectricTestRunner::class)
    @Config(instrumentedPackages =
    ["androidx.loader.content"])
    class LoginScreenTest {
    @get:Rule
    val composeTestRule = createComposeRule()
    val viewModel = LoginViewModelStub()
    

    View Slide

70. @Test
    fun `should login and update status`() =
    runComposeTest {
    setContent {
    ScTheme {
    LoginScreen(viewModel)
    }
    }
    

    View Slide

71. onNodeWithText(LOGGED_IN.name).assertDoesNotExist()
    onNodeWithText(NOT_LOGGED_IN.name).performClick()
    onNodeWithText(LOGGED_IN.name).assertExists()
    

    View Slide

72. Prefer Robolectric over Device for UI tests
    

    View Slide

73. What did we write?
    LoginScreen LoginViewModel
    LoginScreenTest LoginViewModelTest
    LoginRepository
    ApiDataSourceTest
    ApiDataSource
    WearOAuthClientTest
    SharedPreferenceTokenStorageTest
    

    View Slide

74. What did we write?
    LoginScreen LoginViewModel
    LoginScreenTest LoginViewModelTest
    LoginRepository
    WearOAuthClientTest
    SharedPreferenceTokenStorageTest
    ApiDataSource
    

    View Slide

75. This is not a unit test!
    (╯°□°)╯︵ ┻━┻
    

    View Slide

76. This is not
    a unit test!
    

    View Slide

77. The way most developers write their tests
    Test1 Test2
    Class1 Class2 Class3
    Test3
    

    View Slide

78. The way some developers write their tests
    Test1
    Class1 Class2 Class3
    

    View Slide

79. The way TDD developers write their tests
    Test1
    Class1
    

    View Slide

80. The way TDD developers write their tests
    Test1
    Class1 Class2 Class3
    

    View Slide

81. Solid tests: A true story
    Test1
    ViewModel Repository API
    

    View Slide

82. Solid tests: A true story
    Test1
    ViewModel
    Repository API
    Store Reducer
    SideEffects
    

    View Slide

83. WHAT your code does is stable
    HOW your code does it is unstable
    

    View Slide

84. Solid tests: A true story
    Test1
    ViewModel
    Repository API
    Store Reducer
    SideEffects
    

    View Slide

85. Tests should be coupled
    to the behavior of code and
    decoupled from the structure of code
    (Kent Beck)
    

    View Slide

86. True facts
    Fast and stable
    Flaky and slow
    

    View Slide

87. Can’t avoid flakyness at top Fast and stable
    Flaky and slow
    Your decision
    

    View Slide

88. Android testing
    strategy
    The real deal Junit (5)
    End to end
    Mocked endpoints
    Robolectric
    

    View Slide

89. The good pyramid
    ● Prefer screen tests over end to
    end tests
    ● Prefer mocked endpoint over
    api calls
    ● Prefer mocks/stubs over mock
    web servers
    ● Prefer Robolectric over Device
    tests
    ● Prefer JVM tests over
    Robolectric
    ● Do you really need that in
    memory database?
    Performance and flakyness
    

    View Slide

90. "Test only if you would
    want it to work.”
    Kent Beck
    

    View Slide

91. If it's worth building,
    it's worth testing
    If it's not worth testing,
    why are you wasting your time
    working on it?
    

    View Slide

92. If it's worth building,
    it's worth testing
    If it's not worth testing,
    why are you wasting your time
    working on it?
    

    View Slide

93. There is some
    ugliness in the test
    

    View Slide

94. val code = OAuthCode("789")
    

    View Slide

95. private var counter = 1
    object AuthTokenFixture {
    operator fun invoke(token: String = "${counter++}"): AuthToken {
    return AuthToken(token)
    }
    }
    object OAuthCodeFixture {
    operator fun invoke(code: String = "${counter++}"): OAuthCode {
    return OAuthCode(code)
    }
    }
    

    View Slide

96. val code = OAuthCodeFixture()
    

    View Slide

97. /module
    /src
    /main
    /testFixtures
    /
    Fixtures.java
    

    View Slide

98. Another Tip
    Avoiding Dispatchers.setMain
    

    View Slide

99. class CoroutineTaskExecutor :
    BeforeEachCallback, AfterEachCallback {
    override fun beforeEach(context: ExtensionContext?) {
    Dispatchers.setMain(TestCoroutineDispatcher())
    }
    override fun afterEach(context: ExtensionContext?) {
    Dispatchers.resetMain()
    }
    }
    

    View Slide

100. @HiltViewModel
     internal class HiltLoginViewModel(
     private val authRepository: PersistingAuthRepository,
     private val scope: ViewModel.() -> CoroutineScope
     ) : LoginViewModel() {
     @Inject
     constructor(
     authRepository: PersistingAuthRepository
     ) : this(settings, authRepository, { viewModelScope })
     

     View Slide

101. 101
     We lead what's next in music
     Thank You
     @PreusslerBerlin
     

     View Slide