Common Mistakes in UI Testing

Testing is a crucial part of software development, but it is also an area where developers put less effort into extensibility, maintainability and performance of UI tests.

In this talk, we will discuss the following topics:
– How to design a UI test?
– How to handle flaky tests?
– How to create fast and reliable tests?
– Best Practices in UI testing

Transcript

1. @alex_zhukovich
   https://alexzh.com/
   Common mistakes
   

   UI Testing
   

   View Slide

2. Interaction Pixel perfectness
   

   View Slide

3. INTERACTION
   PIXEL PERFECTNESS
   

   View Slide

4. INTERACTION
   PIXEL PERFECTNESS
   

   View Slide

5. Test Planning Test Data Device
   Preparation
   What to test
   

   View Slide

6. Test Planning
   

   View Slide

7. NOT DEFINING CLEAR TEST OBJECTIVES
   Design System
   Application
   Screen in
   isolation
   

   View Slide

8. NOT DEFINING CLEAR TEST OBJECTIVES
   Design System
   Application
   Screen in
   isolation
   END-TO-END
   
   
   TESTING
   

   View Slide

9. NOT DEFINING CLEAR TEST OBJECTIVES
   Design System
   Application
   Screen in
   isolation
   END-TO-END
   
   
   TESTING
   FUNCTIONAL
   

   TESTING
   

   View Slide

10. NOT DEFINING CLEAR TEST OBJECTIVES
    Design System
    Application
    Screen in
    isolation
    END-TO-END
    
    
    TESTING
    FUNCTIONAL
    

    TESTING
    SCREENSHOT
    

    TESTING
    

    View Slide

11. NOT SELECTING APPROPRIATE TEST SCENARIOS
    

    View Slide

12. NOT SELECTING APPROPRIATE TEST SCENARIOS
    

    View Slide

13. Test Data
    

    View Slide

14. NOT USING CORRECT TEST DATA
    @Test
    
    
    fun superImportantTest() {
    
    
    assertEquals(4, sum(2, 2))
    
    
    }
    
    
    fun sum(a: Int, b: Int): Int { ... }
    

    View Slide

15. @Test
    
    
    fun superImportantTest() {
    
    
    assertEquals(4, sum(2, 2))
    
    
    }
    
    
    fun sum(a: Int, b: Int): Int {
    
    
    return 4
    
    
    }
    NOT USING CORRECT TEST DATA
    

    View Slide

16. GLOBAL TEST DATA FOR A TEST SUITE
    TEST DATA
    TEST CASE TEST CASE TEST CASE TEST CASE
    

    View Slide

17. GLOBAL TEST DATA FOR A TEST SUITE
    TEST DATA
    TEST CASE TEST CASE TEST CASE TEST CASE
    

    View Slide

18. DO NOT CLEAR DATA AFTER TEST CASE EXECUTION
    Test Case
    Test Case
    Test Case
    TEST CASE
    TEST CASE
    TEST CASE
    

    View Slide

19. SETUP DATA VIA UI INSTEAD OF API, DB, MOCK
    +
    

    View Slide

20. SETUP DATA VIA UI INSTEAD OF API, DB, MOCK
    +
    ADD
    NOTE
    

    View Slide

21. SETUP DATA VIA UI INSTEAD OF API, DB, MOCK
    +
    Fake data
    

    View Slide

22. Device Preparation
    

    View Slide

23. INCORRECT SETUP OF DEVICE OR EMULATOR
    Tools
    requirements
    Noti
    fi
    cations Language
    

    View Slide

24. What to test
    

    View Slide

25. NOT TESTING VISUAL DESIGN GUIDELINES
    

    View Slide

26. NOT TESTING DIFFERENT FONT & DISPLAY SIZES
    Display Size: Large
    
    
    Font Size: Largest
    Display Size: Default
    
    
    Font Size: Largest
    Display Size: Default
    
    
    Font Size: Default
    

    View Slide

27. NOT TESTING INTERNALIZATION AND LOCALIZATION
    

    View Slide

28. NOT TESTING DIFFERENT USER WORKFLOWS
    PREMIUM
    PREMIUM
    PREMIUM
    PREMIUM
    

    View Slide

29. NOT TESTING NEGATIVE TEST CASES
    ERROR
    

    View Slide

30. NOT TESTING OFFLINE FUNCTIONALITY
    NO INTERNET
    

    View Slide

31. NOT TESTING DIFFERENT PERMISSIONS
    https://alexzh.com/ui-testing-of-android-runtime-permissions/
    

    View Slide

32. Tools & Techniques
    

    View Slide

33. HOW TO ACHIEVE EFFICIENCY IN TESTING
    END-TO-END TESTS
    FUNCTIONAL TESTS
    SCREENSHOT TESTS
    

    View Slide

34. PARAMETERIZED TEST CASES
    @RunWith(TestParameterInjector::class)
    
    
    class WeekCalendarParamScreenshotTest(
    
    
    @TestParameter val uiMode: UiMode,
    
    
    @TestParameter val fontSize: FontSize,
    
    
    @TestParameter val todayIsSelectedDate: Boolean
    
    
    ) : ScreenshotTest {
    
    
    ...
    
    
    @Test
    
    
    fun weekCalendar_paramUiModeAndFontSizeAndSelectedDate() {
    
    
    val todayDate = if (todayIsSelectedDate) testDate.minusDays(1) else testDate.minusDays(2)
    
    
    val todayIsSelectedDateDescription = if (todayIsSelectedDate) "todayIsSelectedDate" else "todayIsNotSelectedDate"
    
    
    val activityScenario = ActivityScenarioConfigurator.ForComposable()
    
    
    .setUiMode(uiMode)
    
    
    .setFontSize(fontSize)
    
    
    .launchConfiguredActivity()
    
    
    .onActivity {
    
    
    it.setContent {
    
    
    AppTheme {
    
    
    WeekCalendar(...)
    
    
    }
    
    
    }
    
    
    }
    
    
    activityScenario.waitForActivity()
    
    
    compareScreenshot(composeTestRule, "weekCalendar_${uiMode}_${fontSize}_${todayIsSelectedDateDescription}")
    
    
    activityScenario.close()
    
    
    }
    
    
    }
    

    View Slide

35. PARAMETERIZED TEST CASES
    

    View Slide

36. FIXING COMMON MISTAKES
    

    View Slide

37. HOW TO START ADDING UI TESTS
    Priorities User Flow Analytics
    

    View Slide

38. ?
    ?
    ?
    

    View Slide

39. @alex_zhukovich https://alexzh.com/
    Explore More
    
    
    UI Testing
    

    View Slide