Robolectric Native Graphics and Roborazzi

Slide 1

Slide 1 text

Robolectric Native Graphics and Roborazzi takahirom

Slide 2

Slide 2 text

1. Why now? 2. What exactly is Robolectric? 3. What is Robolectric Native Graphics? 4. Roborazzi a. What is Roborazzi? b. How to use Roborazzi c. How to Leverage Roborazzi Effectively i. Component-based screenshot testing ii. Screen-based screenshot testing iii. UI testing debugging 5. What to test 6. Practices

Slide 3

Slide 3 text

Why now? Tokyo

Slide 4

Slide 4 text

Number of detectable bugs Maintenance cost

Slide 5

Slide 5 text

Number of detectable bugs Maintenance cost This maintenance cost includes: ● Performance issues (such as slowness) ● False positives (false alerts, flakiness) ● Time consumed by human intervention ● The difficulty of running the test

Slide 6

Slide 6 text

Number of detectable bugs Maintenance cost A higher maintenance cost means fewer tests can be maintained.

Slide 7

Slide 7 text

Number of detectable bugs Good value for cost Maintenance cost

Slide 8

Slide 8 text

Number of detectable bugs Good value for cost Maintenance cost Manual Production test

Slide 9

Slide 9 text

Number of detectable bugs End to End test (E2E) Good value for cost Maintenance cost Manual Production test

Slide 10

Slide 10 text

Number of detectable bugs Manual Production test End to End test (E2E) Duration: A few hours to a few days 💰 Is there a way to make this more cost-effective? Good value for cost Maintenance cost

Slide 11

Slide 11 text

Number of detectable bugs End to End test (E2E) The new feature Robolectric Native Graphics feature may reduce the need for these tests ???? Good value for cost Maintenance cost Manual Production test

Slide 12

Slide 12 text

What exactly is Robolectric? Tokyo

Slide 13

Slide 13 text

What is Robolectric? > Running tests on an Android emulator or device is slow! Building, deploying, and launching the app often takes a minute or more. That’s no way to do TDD. There must be a better way. > Robolectric is a framework that brings fast and reliable unit tests to Android. Tests run inside the JVM on your workstation in seconds. https://robolectric.org/ A Test Framework for Android Apps. You can use Android classes in local test. Faster and More Reliable tests than Emulators or Physical Devices tests.

Slide 14

Slide 14 text

How fast is Robolectric? Emulator Robolectric Robolectric takes a few seconds to start up, but once it does, it is fast. Code: https://github.com/takahirom/roborazzi-usage-examples/pull/5 Robolectric: 4.11.1 Emulator: Pixel 6 API 32 CPU: Apple M1 Max

Slide 15

Slide 15 text

How to use Robolectric Tokyo

Slide 16

Slide 16 text

How to use Robolectric: Setup app/build.gradle.kts android { testOptions { unitTests { isIncludeAndroidResources = true } } Otherwise, you'll get an 'Unable to resolve activity for Intent' error. All changes from the new project template https://github.com/takahirom/roborazzi-usage-examples/ compare/b697...5c12

Slide 17

Slide 17 text

How to use Robolectric: Setup Add dependencies app/build.gradle.kts dependencies { ... testImplementation(libs.robolectric) testImplementation(libs.espresso.core) testImplementation(libs.ui.test.junit4) } All changes from the new project template https://github.com/takahirom/roborazzi-usage-examples/ compare/b697...5c12 gradle/libs.versions.toml robolectric = { group = "org.robolectric", name = "robolectric", version.ref = "robolectric" } espresso-core = { group = "androidx.test.espresso", name = "espresso-core", version.ref = "espresso-core" } ui-test-junit4 = { group = "androidx.compose.ui", name = "ui-test-junit4" } ... Robolectric is compatible with the Espresso testing library

Slide 18

Slide 18 text

Slide 19

Slide 19 text

How to use Robolectric: Test for Compose @RunWith(RobolectricTestRunner::class) class FirstRobolectricComposeTest { @get:Rule val composeRule = createComposeRule() @Test fun test() { composeRule.setContent {

Slide 20

Slide 20 text

Slide 21

Slide 21 text

@get:Rule val composeRule = createComposeRule() @Test fun test() { composeRule.setContent { Greeting(name = "Robolectric") } composeRule .onNode(hasText("Hello Robolectric!")) .assertExists() Setup compose test with createComposeRule()

Slide 22

Slide 22 text

@Test fun test() { composeRule.setContent { Greeting(name = "Robolectric") } composeRule .onNode(hasText("Hello Robolectric!")) .assertExists() } } Call a Composable function to test Assert what you want to check

Slide 23

Slide 23 text

What is Robolectric Native Graphics? Tokyo

Slide 24

Slide 24 text

What is Robolectric Native Graphics? > Robolectric 4.10 adds support for native Android graphics. It is currently disabled by default and can be enabled using @GraphicsMode(NATIVE). > When native graphics is enabled, interactions with Android graphics classes use real native Android graphics code and are much higher fidelity. https://github.com/robolectric/robole ctric/releases/tag/robolectric-4.10 Robolectric Native Graphics is a feature of Robolectric. You can use real native Android graphics code and are much higher fidelity

Slide 25

Slide 25 text

Try Robolectric Native Graphics @GraphicsMode(NATIVE) @Test fun test() { val bitmap = activity.findViewById(R.id.button_first) .drawToBitmap() println(bitmap) }

Slide 26

Slide 26 text

Verifying the Use of Actual Native Android Graphics Code Android's native implementation uses Skia. Therefore, I checked whether Robolectric Native Graphics also uses Skia, utilizing lldb for this purpose. https://twitter.com/new_runnable/st atus/1718191787163172986 Robolectric Native Graphics Utilize Real Native Android Graphics Classes.

Slide 27

Slide 27 text

What is Roborazzi? Tokyo

Slide 28

Slide 28 text

What is Roborazzi? Roborazzi is an Android Library that allows you to execute screenshot tests on your local machine without the need for real devices or emulators. It leverages Robolectric Native Graphics for rendering views. I, Takahirom, am the author of this library. It is utilized in Google's 'Now in Android' app and by companies such as Uber. https://twitter.com/ppvi/status/1694741201457418450 https://twitter.com/tsmith/status/1695095257044852823

Slide 29

Slide 29 text

Slide 30

Slide 30 text

What is Roborazzi? Robolectric Robolectric is a test Framework

Slide 31

Slide 31 text

What is Roborazzi? Robolectric Robolectric Native Graphics New feature Robolectric Native Graphics

Slide 32

Slide 32 text

What is Roborazzi? Robolectric Robolectric Native Graphics Roborazzi The new Roborazzi library uses Robolectric Native Graphics for screenshot testing

Slide 33

Slide 33 text

Roborazzi vs. Other Tools like Paparazzi: A Comparative Overview Paparazzi: Paparazzi is an Android library that enables you to take screenshots in a JVM environment, differing from Roborazzi which relies on Robolectric Native Graphics. It is built on Android Studio's Layout Lib

Slide 34

Slide 34 text

Roborazzi vs. Other Tools like Paparazzi: A Comparative Overview Roborazzi Android Studio Preview based tool Rendering Android's native graphics classes Android's native graphics classes Pros Robolectric support ● Dagger Hilt support ● Espresso support ● User interaction (click,scroll) support ● Animation, SideEffect support Shadow rendering support Cons Lack of Windows support You can check Sergio Sastre Flórez’s slide: https://speakerdeck.com/gio_sastre/screenshot-testing-for-android-which-library-is-right-for-you

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

How to use Roborazzi Tokyo

Slide 39

Slide 39 text

How to use Roborazzi: Setup All changes: https://github.com/takahirom/roborazzi-usage-examples/ commit/3a02 Just add a Gradle Plugin and dependencies

Slide 40

Slide 40 text

How to use Roborazzi @GraphicsMode(GraphicsMode.Mode.NATIVE) @RunWith(RobolectricTestRunner::class) class FirstRobolectricComposeTest { ... @Test fun roborazziTest() { composeRule.setContent { Greeting(name = "Roborazzi") All changes: https://github.com/takahirom/ roborazzi-usage-examples/ commit/3a02

Slide 41

Slide 41 text

Slide 42

Slide 42 text

Slide 43

Slide 43 text

composeRule .onNode(hasText("Hello Robolectric!")) .captureRoboImage() composeRule .onRoot() .captureRoboImage() } All changes: https://github.com/takahirom/ roborazzi-usage-examples/ commit/3a02 Just call .captureRoboImage() method in your test code.

Slide 44

Slide 44 text

composeRule .onNode(hasText("Hello Robolectric!")) .captureRoboImage() composeRule .onRoot() .captureRoboImage() } All changes: https://github.com/takahirom/ roborazzi-usage-examples/ commit/3a02 You can specify the compose node of screenshot Or you can take the root node screenshot.

Slide 45

Slide 45 text

You can take screenshots of anything you want in your app’s test https://github.com/takahirom/roborazzi

Slide 46

Slide 46 text

How to use Roborazzi build/output/roborazzi/test.png From JetNews Record the reference image ./gradlew recordRoborazziDebug Or You can use gradle.properties options roborazzi.test.record=true and ./gradlew testDebugUnitTest

Slide 47

Slide 47 text

How to use Roborazzi build/output/roborazzi/test_compare.png From JetNews Compare the new image with the reference image ./gradlew compareRoborazziDebug Or You can use gradle.properties roborazzi.test.compare=true and ./gradlew testDebugUnitTest

Slide 48

Slide 48 text

How to use Roborazzi Ensure that the new image remains unchanged compared to the reference image. ./gradlew verifyRoborazziDebug Or You can use gradle.properties roborazzi.test.verify=true and ./gradlew testDebugUnitTest Roborazzi throws an AssertionError if the image changes.

Slide 49

Slide 49 text

How to Leverage Roborazzi Effectively Tokyo

Slide 50

Slide 50 text

How to Leverage Roborazzi Effectively Roborazzi excels in three types of tests: 1) Component-based screenshot testing 2) Screen-based screenshot testing 3) UI test debugging

Slide 51

Slide 51 text

Component-based screenshot testing with Roborazzi ● What: Screenshot tests for component design like a Button component ● Why: Ensuring Consistency, Detecting Visual Regression, Cross-Setting and Cross-Form-Factor Consistency, Documentation, Enhancing User Experience ● How: You can utilize Jetpack Compose Preview for component-based screenshot testing. There are two methods: ・Utilizing Showkase, which employs Kotlin Symbol Processing to gather Preview functions. ・Using Reflection, with libraries such as ClassGraph Full changes https://github.com/takahirom/ro borazzi-usage-examples/pull/1

Slide 52

Slide 52 text

Component-based screenshot testing with Roborazzi Full changes https://github.com/takahirom/roborazzi-usage- examples/pull/1 @GraphicsMode(GraphicsMode.Mode.NATIVE) @RunWith(ParameterizedRobolectricTestRunner::class) class ComponentScreenshotTest( private val composablePreview: ComposablePreviewFunction, ) { @get:Rule val composeRule = createComposeRule() @Test fun test() { composeRule.setContent { composablePreview() } composeRule.onRoot().captureRoboImage( roboOutputName() + "_" + composablePreview.toString() + ".png" ) } ... Automated Collection of Composable Functions via Compose Preview — No More Writing Individual Tests

Slide 53

Slide 53 text

Component-based screenshot testing with Roborazzi https://github.com/takahirom/roborazzi Recommended article: Efficient Testing with Robolectric & Roborazzi Across Many UI States, Devices and Configurations You can use predefined device settings and configurations, such as dark mode.

Slide 54

Slide 54 text

Screen-based screenshot testing with Roborazzi What: Screenshot tests for screen How: Robolectric can launch your Activity using either Compose test's createAndroidComposeRule() or Espresso's ActivityScenario.launch() Why: ? Full changes https://github.com/takahirom/roborazzi-usage- examples/pull/4

Slide 55

Slide 55 text

Why do we do Screen-based Screenshot Testing Why: Identify and solve the problems faced by users, Ensuring Comprehensive Coverage, Detecting Layout and Integration Issues, Enhancing Test Fidelity Production

Slide 56

Slide 56 text

Slide 57

Slide 57 text

Obstacles in Implementing Screen-Based Tests ● The app will have dependencies that are difficult to test, such as network requests to the server. This makes the tests flaky because they can't be controlled. ● Additionally, there will be threads in the app that aren't controlled during testing, which also contributes to the tests' flakiness. Full changes https://github.com/takahir om/roborazzi-usage-exa mples/pull/4 Dependencies (Server APIs, Threads)

Slide 58

Slide 58 text

Obstacles in Implementing Screen-Based Tests ● The app will have dependencies that are difficult to test, such as network requests to the server. This makes the tests flaky because they can't be controlled. ● Additionally, there will be threads in the app that aren't controlled during testing, which also contributes to the tests' flakiness. Full changes https://github.com/takahirom/roborazzi-usage- examples/pull/4 We need to replace dependencies in tests.

Slide 59

Slide 59 text

How to replace dependencies in tests We can use Dagger Hilt to replace dependencies in tests. Full changes to introduce Dagger Hilt https://github.com/takahirom/roborazzi-usage- examples/pull/2 Fake Dependencies production test Dependencies (Server APIs, Threads)

Slide 60

Slide 60 text

How to replace dependencies in tests Full changes https://github.com/takahirom/roborazzi-usage- examples/pull/4 @Module @TestInstallIn( components = [SingletonComponent::class], replaces = [ArticleApiModule::class] ) object FakeArticleApiModule { @Provides fun provideArticleApi(): ArticleApi { return FakeArticleApi() }

Slide 61

Slide 61 text

Slide 62

Slide 62 text

Screen-based screenshot testing Full changes https://github.com/takahirom/roborazzi- usage-examples/pull/4 @HiltAndroidTest @RunWith(RobolectricTestRunner::class) @GraphicsMode(GraphicsMode.Mode.NATIVE) class ArticleScreenTest { private val composeTestRule = createAndroidComposeRule() @get:Rule val ruleChain = RuleChain .outerRule(HiltAndroidRule(this)) .around(composeTestRule) @Test fun checkScreenShot() { composeTestRule .onRoot() .captureRoboImage() }

Slide 63

Slide 63 text

Slide 64

Slide 64 text

Full changes https://github.com/takahirom/roborazzi- usage-examples/pull/4 @RunWith(RobolectricTestRunner::class) @GraphicsMode(GraphicsMode.Mode.NATIVE) class ArticleScreenTest { private val composeTestRule = createAndroidComposeRule() @get:Rule val ruleChain = RuleChain .outerRule(HiltAndroidRule(this)) .around(composeTestRule) @Test fun checkScreenShot() { composeTestRule .onRoot() .captureRoboImage() } } You can use captureRoboImage() extension function to take screenshot tests

Slide 65

Slide 65 text

UI test debugging What: Roborazzi can be used as a supplemental tool for debugging UI tests, such as verifying button visibility. Why: Writing tests becomes challenging when unable to visually assess the layout. How: It serves as an auxiliary tool used in conjunction with standard UI testing methods to enhance debugging capabilities.

Slide 66

Slide 66 text

Slide 67

Slide 67 text

UI test debugging composeTestRule .onNode(hasText("article title 1")) .assertExists() composeTestRule .onRoot() .captureRoboImage( roborazziOptions = RoborazziOptions( taskType = roborazziSystemPropertyTaskType() .convertVerifyingToComparing() ) ) Coming Soon: You can just take a screenshots during Roborazzi verify task Tell me your opinion! https://github.com/takahirom/ roborazzi/issues/215

Slide 68

Slide 68 text

What to test Tokyo

Slide 69

Slide 69 text

Number of detectable bugs Maintenance cost

Slide 70

Slide 70 text

Number of detectable bugs Good value for cost Maintenance cost

Slide 71

Slide 71 text

Number of detectable bugs End to End test (E2E) Good value for cost Maintenance cost Manual Production test

Slide 72

Slide 72 text

Number of detectable bugs Component-based Screenshot tests End to End test (E2E) You can automatically collect Preview functions and check the appearance of the components. Good value for cost Maintenance cost Manual Production test

Slide 73

Slide 73 text

Number of detectable bugs Component-based Screenshot tests End to End test (E2E) Robolectric UI tests Robolectric is faster and more reliable, but it is not the actual Android Framework Good value for cost Maintenance cost Manual Production test

Slide 74

Slide 74 text

Component-based Screenshot tests End to End test (E2E) Screenshot tests can capture what users see; however, they require checking after every change, which increases maintenance costs Robolectric Screen-based Screenshot tests Good value for cost Robolectric UI tests Number of detectable bugs Maintenance cost Manual Production test

Slide 75

Slide 75 text

Component-based Screenshot tests End to End test (E2E) Good value for cost Roborazzi Robolectric Screen-based Screenshot tests Robolectric UI tests Number of detectable bugs Maintenance cost Manual Production test

Slide 76

Slide 76 text

My recommendations for Testing fewer tests fewer tests

Slide 77

Slide 77 text

E2E My recommendations for Testing fewer tests fewer tests

Slide 78

Slide 78 text

E2E Robolectric Screen-based Screenshot tests My recommendations for Testing fewer tests fewer tests

Slide 79

Slide 79 text

E2E Robolectric Screen-based Screenshot tests My recommendations for Testing fewer tests fewer tests Maintenance costs may limit how many screen-based screenshot tests you can run, but starting them is easy.

Slide 80

Slide 80 text

E2E Robolectric Screen-based Screenshot tests Robolectric UI tests My recommendations for Testing fewer tests fewer tests

Slide 81

Slide 81 text

E2E Robolectric Screen-based Screenshot tests Robolectric UI tests My recommendations for Testing fewer tests fewer tests Screen-based screenshot tests and UI tests have almost identical code, so they can be migrated.

Slide 82

Slide 82 text

E2E Robolectric Screen-based Screenshot tests Robolectric UI tests Component-based Screenshot tests My recommendations for Testing fewer tests fewer tests

Slide 83

Slide 83 text

E2E Robolectric Screen-based Screenshot tests Robolectric UI tests Component-based Screenshot tests Unit tests for essential domain classes not covered by higher-level tests My recommendations for Testing fewer tests fewer tests

Slide 84

Slide 84 text

Practices Tokyo

Slide 85

Slide 85 text

Practices 1. Development Process a. Adopt a test-driven approach, beginning with screenshot tests of a blank screen. b. Set Roborazzi to be enabled by default. 2. Write Tests Effectively a. Employ Robot testing pattern or page object pattern for testing. For more information please read this article https://medium.com/@takahirom/roborazzi-elevating-android-visual-te sting-to-the-next-level-46ec56b24055 3. Debug Effectively a. Utilize dump mode for debugging. b. Implement code coverage analysis in debugging. 4. CI Effectively c. Define image storage locations in CI pipeline.

Slide 86

Slide 86 text

Summary 1. Robolectric: Streamlines Android unit tests within the JVM. 2. Roborazzi: Enables screenshot testing without real devices. 3. Benefits: Lowers maintenance costs and improves bug detection. 4. Usage: Integrates with Espresso and Dagger Hilt. 5. Strategy: Combine unit, component, and screen-based tests for optimal coverage

Slide 87

Slide 87 text

Thank you Tokyo