Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Robolectric Native Graphics and Roborazzi

takahirom
December 09, 2023
1.1k

Robolectric Native Graphics and Roborazzi

takahirom

December 09, 2023
Tweet

Transcript

  1. 1. Why now? 2. What exactly is Robolectric? 3. What

    is Robolectric Native Graphics? 4. Roborazzi a. What is Roborazzi? b. How to use Roborazzi c. How to Leverage Roborazzi Effectively i. Component-based screenshot testing ii. Screen-based screenshot testing iii. UI testing debugging 5. What to test 6. Practices
  2. Number of detectable bugs Maintenance cost This maintenance cost includes:

    • Performance issues (such as slowness) • False positives (false alerts, flakiness) • Time consumed by human intervention • The difficulty of running the test
  3. Number of detectable bugs End to End test (E2E) Good

    value for cost Maintenance cost Manual Production test
  4. Number of detectable bugs Manual Production test End to End

    test (E2E) Duration: A few hours to a few days 💰 Is there a way to make this more cost-effective? Good value for cost Maintenance cost
  5. Number of detectable bugs End to End test (E2E) The

    new feature Robolectric Native Graphics feature may reduce the need for these tests ???? Good value for cost Maintenance cost Manual Production test
  6. What is Robolectric? > Running tests on an Android emulator

    or device is slow! Building, deploying, and launching the app often takes a minute or more. That’s no way to do TDD. There must be a better way. > Robolectric is a framework that brings fast and reliable unit tests to Android. Tests run inside the JVM on your workstation in seconds. https://robolectric.org/ A Test Framework for Android Apps. You can use Android classes in local test. Faster and More Reliable tests than Emulators or Physical Devices tests.
  7. How fast is Robolectric? Emulator Robolectric Robolectric takes a few

    seconds to start up, but once it does, it is fast. Code: https://github.com/takahirom/roborazzi-usage-examples/pull/5 Robolectric: 4.11.1 Emulator: Pixel 6 API 32 CPU: Apple M1 Max
  8. How to use Robolectric: Setup app/build.gradle.kts android { testOptions {

    unitTests { isIncludeAndroidResources = true } } Otherwise, you'll get an 'Unable to resolve activity for Intent' error. All changes from the new project template https://github.com/takahirom/roborazzi-usage-examples/ compare/b697...5c12
  9. How to use Robolectric: Setup Add dependencies app/build.gradle.kts dependencies {

    ... testImplementation(libs.robolectric) testImplementation(libs.espresso.core) testImplementation(libs.ui.test.junit4) } All changes from the new project template https://github.com/takahirom/roborazzi-usage-examples/ compare/b697...5c12 gradle/libs.versions.toml robolectric = { group = "org.robolectric", name = "robolectric", version.ref = "robolectric" } espresso-core = { group = "androidx.test.espresso", name = "espresso-core", version.ref = "espresso-core" } ui-test-junit4 = { group = "androidx.compose.ui", name = "ui-test-junit4" } ... Robolectric is compatible with the Espresso testing library
  10. How to use Robolectric: Setup Add dependencies app/build.gradle.kts dependencies {

    ... testImplementation(libs.robolectric) testImplementation(libs.espresso.core) testImplementation(libs.ui.test.junit4) } All changes from the new project template https://github.com/takahirom/roborazzi-usage-examples/ compare/b697...5c12 gradle/libs.versions.toml robolectric = { group = "org.robolectric", name = "robolectric", version.ref = "robolectric" } espresso-core = { group = "androidx.test.espresso", name = "espresso-core", version.ref = "espresso-core" } ui-test-junit4 = { group = "androidx.compose.ui", name = "ui-test-junit4" } ... Robolectric is compatible with the Espresso testing library Robolectric is compatible with the Compose testing library.
  11. How to use Robolectric: Test for Compose @RunWith(RobolectricTestRunner::class) class FirstRobolectricComposeTest

    { @get:Rule val composeRule = createComposeRule() @Test fun test() { composeRule.setContent {
  12. How to use Robolectric: Test for Compose @RunWith(RobolectricTestRunner::class) class FirstRobolectricComposeTest

    { @get:Rule val composeRule = createComposeRule() @Test fun test() { composeRule.setContent { Use RobolectricTestRunner to Replace Android Classes with Robolectric
  13. @get:Rule val composeRule = createComposeRule() @Test fun test() { composeRule.setContent

    { Greeting(name = "Robolectric") } composeRule .onNode(hasText("Hello Robolectric!")) .assertExists() Setup compose test with createComposeRule()
  14. @Test fun test() { composeRule.setContent { Greeting(name = "Robolectric") }

    composeRule .onNode(hasText("Hello Robolectric!")) .assertExists() } } Call a Composable function to test Assert what you want to check
  15. What is Robolectric Native Graphics? > Robolectric 4.10 adds support

    for native Android graphics. It is currently disabled by default and can be enabled using @GraphicsMode(NATIVE). > When native graphics is enabled, interactions with Android graphics classes use real native Android graphics code and are much higher fidelity. https://github.com/robolectric/robole ctric/releases/tag/robolectric-4.10 Robolectric Native Graphics is a feature of Robolectric. You can use real native Android graphics code and are much higher fidelity
  16. Try Robolectric Native Graphics @GraphicsMode(NATIVE) @Test fun test() { val

    bitmap = activity.findViewById<View>(R.id.button_first) .drawToBitmap() println(bitmap) }
  17. Verifying the Use of Actual Native Android Graphics Code Android's

    native implementation uses Skia. Therefore, I checked whether Robolectric Native Graphics also uses Skia, utilizing lldb for this purpose. https://twitter.com/new_runnable/st atus/1718191787163172986 Robolectric Native Graphics Utilize Real Native Android Graphics Classes.
  18. What is Roborazzi? Roborazzi is an Android Library that allows

    you to execute screenshot tests on your local machine without the need for real devices or emulators. It leverages Robolectric Native Graphics for rendering views. I, Takahirom, am the author of this library. It is utilized in Google's 'Now in Android' app and by companies such as Uber. https://twitter.com/ppvi/status/1694741201457418450 https://twitter.com/tsmith/status/1695095257044852823
  19. What is Roborazzi? Roborazzi is an Android Library that allows

    you to execute screenshot tests on your local machine without the need for real devices or emulators. It leverages Robolectric Native Graphics for rendering views. I, Takahirom, am the author of this library. It is utilized in Google's 'Now in Android' app and by companies such as Uber. https://twitter.com/ppvi/status/1694741201457418450 https://twitter.com/tsmith/status/1695095257044852823 Screenshot library using Robolectric Native Graphics
  20. What is Roborazzi? Robolectric Robolectric Native Graphics Roborazzi The new

    Roborazzi library uses Robolectric Native Graphics for screenshot testing
  21. Roborazzi vs. Other Tools like Paparazzi: A Comparative Overview Paparazzi:

    Paparazzi is an Android library that enables you to take screenshots in a JVM environment, differing from Roborazzi which relies on Robolectric Native Graphics. It is built on Android Studio's Layout Lib
  22. Roborazzi vs. Other Tools like Paparazzi: A Comparative Overview Roborazzi

    Android Studio Preview based tool Rendering Android's native graphics classes Android's native graphics classes Pros Robolectric support • Dagger Hilt support • Espresso support • User interaction (click,scroll) support • Animation, SideEffect support Shadow rendering support Cons Lack of Windows support You can check Sergio Sastre Flórez’s slide: https://speakerdeck.com/gio_sastre/screenshot-testing-for-android-which-library-is-right-for-you
  23. Roborazzi vs. Other Tools like Paparazzi: A Comparative Overview Roborazzi

    Android Studio Preview based tool Rendering Android's native graphics classes Android's native graphics classes Pros Robolectric support • Dagger Hilt support • Espresso support • User interaction (click,scroll) support • Animation, SideEffect support Shadow rendering support Cons Lack of Windows support You can check Sergio Sastre Flórez’s slide: https://speakerdeck.com/gio_sastre/screenshot-testing-for-android-which-library-is-right-for-you Both use the same graphics classes
  24. Roborazzi vs. Other Tools like Paparazzi: A Comparative Overview Roborazzi

    Android Studio Preview based tool Rendering Android's native graphics classes Android's native graphics classes Pros Robolectric support • Dagger Hilt support • Espresso support • User interaction (click,scroll) support • Animation, SideEffect support Shadow rendering support Cons Lack of Windows support You can check Sergio Sastre Flórez’s slide: https://speakerdeck.com/gio_sastre/screenshot-testing-for-android-which-library-is-right-for-you Roborazzi can use Robolectric’s testing features
  25. Roborazzi vs. Other Tools like Paparazzi: A Comparative Overview Roborazzi

    Android Studio Preview based tool Rendering Android's native graphics classes Android's native graphics classes Pros Robolectric support • Dagger Hilt support • Espresso support • User interaction (click,scroll) support • Animation, SideEffect support Shadow rendering support Cons Lack of Windows support You can check Sergio Sastre Flórez’s slide: https://speakerdeck.com/gio_sastre/screenshot-testing-for-android-which-library-is-right-for-you Currently, Robolectric Native Graphics doesn’t support Windows You can run Roborazzi on CI
  26. How to use Roborazzi @GraphicsMode(GraphicsMode.Mode.NATIVE) @RunWith(RobolectricTestRunner::class) class FirstRobolectricComposeTest { ...

    @Test fun roborazziTest() { composeRule.setContent { Greeting(name = "Roborazzi") All changes: https://github.com/takahirom/ roborazzi-usage-examples/ commit/3a02
  27. How to use Roborazzi @GraphicsMode(GraphicsMode.Mode.NATIVE) @RunWith(RobolectricTestRunner::class) class FirstRobolectricComposeTest { ...

    @Test fun roborazziTest() { composeRule.setContent { Greeting(name = "Roborazzi") All changes: https://github.com/takahirom/ roborazzi-usage-examples/ commit/3a02 This test is nearly identical to the one used in Robolectric.
  28. How to use Roborazzi @GraphicsMode(GraphicsMode.Mode.NATIVE) @RunWith(RobolectricTestRunner::class) class FirstRobolectricComposeTest { ...

    @Test fun roborazziTest() { composeRule.setContent { Greeting(name = "Roborazzi") All changes: https://github.com/takahirom/ roborazzi-usage-examples/ commit/3a02 Enable Robolectric Native Graphics to take screenshot
  29. composeRule .onNode(hasText("Hello Robolectric!")) .captureRoboImage() composeRule .onRoot() .captureRoboImage() } All changes:

    https://github.com/takahirom/ roborazzi-usage-examples/ commit/3a02 Just call .captureRoboImage() method in your test code.
  30. composeRule .onNode(hasText("Hello Robolectric!")) .captureRoboImage() composeRule .onRoot() .captureRoboImage() } All changes:

    https://github.com/takahirom/ roborazzi-usage-examples/ commit/3a02 You can specify the compose node of screenshot Or you can take the root node screenshot.
  31. You can take screenshots of anything you want in your

    app’s test https://github.com/takahirom/roborazzi
  32. How to use Roborazzi build/output/roborazzi/test.png From JetNews Record the reference

    image ./gradlew recordRoborazziDebug Or You can use gradle.properties options roborazzi.test.record=true and ./gradlew testDebugUnitTest
  33. How to use Roborazzi build/output/roborazzi/test_compare.png From JetNews Compare the new

    image with the reference image ./gradlew compareRoborazziDebug Or You can use gradle.properties roborazzi.test.compare=true and ./gradlew testDebugUnitTest
  34. How to use Roborazzi Ensure that the new image remains

    unchanged compared to the reference image. ./gradlew verifyRoborazziDebug Or You can use gradle.properties roborazzi.test.verify=true and ./gradlew testDebugUnitTest Roborazzi throws an AssertionError if the image changes.
  35. How to Leverage Roborazzi Effectively Roborazzi excels in three types

    of tests: 1) Component-based screenshot testing 2) Screen-based screenshot testing 3) UI test debugging
  36. Component-based screenshot testing with Roborazzi • What: Screenshot tests for

    component design like a Button component • Why: Ensuring Consistency, Detecting Visual Regression, Cross-Setting and Cross-Form-Factor Consistency, Documentation, Enhancing User Experience • How: You can utilize Jetpack Compose Preview for component-based screenshot testing. There are two methods: ・Utilizing Showkase, which employs Kotlin Symbol Processing to gather Preview functions. ・Using Reflection, with libraries such as ClassGraph Full changes https://github.com/takahirom/ro borazzi-usage-examples/pull/1
  37. Component-based screenshot testing with Roborazzi Full changes https://github.com/takahirom/roborazzi-usage- examples/pull/1 @GraphicsMode(GraphicsMode.Mode.NATIVE)

    @RunWith(ParameterizedRobolectricTestRunner::class) class ComponentScreenshotTest( private val composablePreview: ComposablePreviewFunction, ) { @get:Rule val composeRule = createComposeRule() @Test fun test() { composeRule.setContent { composablePreview() } composeRule.onRoot().captureRoboImage( roboOutputName() + "_" + composablePreview.toString() + ".png" ) } ... Automated Collection of Composable Functions via Compose Preview — No More Writing Individual Tests
  38. Component-based screenshot testing with Roborazzi https://github.com/takahirom/roborazzi Recommended article: Efficient Testing

    with Robolectric & Roborazzi Across Many UI States, Devices and Configurations You can use predefined device settings and configurations, such as dark mode.
  39. Screen-based screenshot testing with Roborazzi What: Screenshot tests for screen

    How: Robolectric can launch your Activity using either Compose test's createAndroidComposeRule<ArticleActivity>() or Espresso's ActivityScenario.launch() Why: ? Full changes https://github.com/takahirom/roborazzi-usage- examples/pull/4
  40. Why do we do Screen-based Screenshot Testing Why: Identify and

    solve the problems faced by users, Ensuring Comprehensive Coverage, Detecting Layout and Integration Issues, Enhancing Test Fidelity Production
  41. Why do we do Screen-based Screenshot Testing Why: Identify and

    solve the problems faced by users, Ensuring Comprehensive Coverage, Detecting Layout and Integration Issues, Enhancing Test Fidelity Production Test
  42. Obstacles in Implementing Screen-Based Tests • The app will have

    dependencies that are difficult to test, such as network requests to the server. This makes the tests flaky because they can't be controlled. • Additionally, there will be threads in the app that aren't controlled during testing, which also contributes to the tests' flakiness. Full changes https://github.com/takahir om/roborazzi-usage-exa mples/pull/4 Dependencies (Server APIs, Threads)
  43. Obstacles in Implementing Screen-Based Tests • The app will have

    dependencies that are difficult to test, such as network requests to the server. This makes the tests flaky because they can't be controlled. • Additionally, there will be threads in the app that aren't controlled during testing, which also contributes to the tests' flakiness. Full changes https://github.com/takahirom/roborazzi-usage- examples/pull/4 We need to replace dependencies in tests.
  44. How to replace dependencies in tests We can use Dagger

    Hilt to replace dependencies in tests. Full changes to introduce Dagger Hilt https://github.com/takahirom/roborazzi-usage- examples/pull/2 Fake Dependencies production test Dependencies (Server APIs, Threads)
  45. How to replace dependencies in tests Full changes https://github.com/takahirom/roborazzi-usage- examples/pull/4

    @Module @TestInstallIn( components = [SingletonComponent::class], replaces = [ArticleApiModule::class] ) object FakeArticleApiModule { @Provides fun provideArticleApi(): ArticleApi { return FakeArticleApi() }
  46. How to replace dependencies in tests Full changes https://github.com/takahirom/roborazzi-usage- examples/pull/4

    @Module @TestInstallIn( components = [SingletonComponent::class], replaces = [ArticleApiModule::class] ) object FakeArticleApiModule { @Provides fun provideArticleApi(): ArticleApi { return FakeArticleApi() } You can replace dependencies in your test
  47. Screen-based screenshot testing Full changes https://github.com/takahirom/roborazzi- usage-examples/pull/4 @HiltAndroidTest @RunWith(RobolectricTestRunner::class) @GraphicsMode(GraphicsMode.Mode.NATIVE)

    class ArticleScreenTest { private val composeTestRule = createAndroidComposeRule<ArticleActivity>() @get:Rule val ruleChain = RuleChain .outerRule(HiltAndroidRule(this)) .around(composeTestRule) @Test fun checkScreenShot() { composeTestRule .onRoot() .captureRoboImage() }
  48. Screen-based screenshot testing Full changes https://github.com/takahirom/roborazzi- usage-examples/pull/4 @HiltAndroidTest @RunWith(RobolectricTestRunner::class) @GraphicsMode(GraphicsMode.Mode.NATIVE)

    class ArticleScreenTest { private val composeTestRule = createAndroidComposeRule<ArticleActivity>() @get:Rule val ruleChain = RuleChain .outerRule(HiltAndroidRule(this)) .around(composeTestRule) @Test fun checkScreenShot() { composeTestRule .onRoot() .captureRoboImage() } Add @HiltAndroidTest and HiltAndroidRule for Hilt
  49. Full changes https://github.com/takahirom/roborazzi- usage-examples/pull/4 @RunWith(RobolectricTestRunner::class) @GraphicsMode(GraphicsMode.Mode.NATIVE) class ArticleScreenTest { private

    val composeTestRule = createAndroidComposeRule<ArticleActivity>() @get:Rule val ruleChain = RuleChain .outerRule(HiltAndroidRule(this)) .around(composeTestRule) @Test fun checkScreenShot() { composeTestRule .onRoot() .captureRoboImage() } } You can use captureRoboImage() extension function to take screenshot tests
  50. UI test debugging What: Roborazzi can be used as a

    supplemental tool for debugging UI tests, such as verifying button visibility. Why: Writing tests becomes challenging when unable to visually assess the layout. How: It serves as an auxiliary tool used in conjunction with standard UI testing methods to enhance debugging capabilities.
  51. UI test debugging composeTestRule .onNode(hasText("article title 1")) .assertExists() composeTestRule .onRoot()

    .captureRoboImage( roborazziOptions = RoborazziOptions( taskType = roborazziSystemPropertyTaskType() .convertVerifyingToComparing() ) )
  52. UI test debugging composeTestRule .onNode(hasText("article title 1")) .assertExists() composeTestRule .onRoot()

    .captureRoboImage( roborazziOptions = RoborazziOptions( taskType = roborazziSystemPropertyTaskType() .convertVerifyingToComparing() ) ) Coming Soon: You can just take a screenshots during Roborazzi verify task Tell me your opinion! https://github.com/takahirom/ roborazzi/issues/215
  53. Number of detectable bugs End to End test (E2E) Good

    value for cost Maintenance cost Manual Production test
  54. Number of detectable bugs Component-based Screenshot tests End to End

    test (E2E) You can automatically collect Preview functions and check the appearance of the components. Good value for cost Maintenance cost Manual Production test
  55. Number of detectable bugs Component-based Screenshot tests End to End

    test (E2E) Robolectric UI tests Robolectric is faster and more reliable, but it is not the actual Android Framework Good value for cost Maintenance cost Manual Production test
  56. Component-based Screenshot tests End to End test (E2E) Screenshot tests

    can capture what users see; however, they require checking after every change, which increases maintenance costs Robolectric Screen-based Screenshot tests Good value for cost Robolectric UI tests Number of detectable bugs Maintenance cost Manual Production test
  57. Component-based Screenshot tests End to End test (E2E) Good value

    for cost Roborazzi Robolectric Screen-based Screenshot tests Robolectric UI tests Number of detectable bugs Maintenance cost Manual Production test
  58. E2E Robolectric Screen-based Screenshot tests My recommendations for Testing fewer

    tests fewer tests Maintenance costs may limit how many screen-based screenshot tests you can run, but starting them is easy.
  59. E2E Robolectric Screen-based Screenshot tests Robolectric UI tests My recommendations

    for Testing fewer tests fewer tests Screen-based screenshot tests and UI tests have almost identical code, so they can be migrated.
  60. E2E Robolectric Screen-based Screenshot tests Robolectric UI tests Component-based Screenshot

    tests My recommendations for Testing fewer tests fewer tests
  61. E2E Robolectric Screen-based Screenshot tests Robolectric UI tests Component-based Screenshot

    tests Unit tests for essential domain classes not covered by higher-level tests My recommendations for Testing fewer tests fewer tests
  62. Practices 1. Development Process a. Adopt a test-driven approach, beginning

    with screenshot tests of a blank screen. b. Set Roborazzi to be enabled by default. 2. Write Tests Effectively a. Employ Robot testing pattern or page object pattern for testing. For more information please read this article https://medium.com/@takahirom/roborazzi-elevating-android-visual-te sting-to-the-next-level-46ec56b24055 3. Debug Effectively a. Utilize dump mode for debugging. b. Implement code coverage analysis in debugging. 4. CI Effectively c. Define image storage locations in CI pipeline.
  63. Summary 1. Robolectric: Streamlines Android unit tests within the JVM.

    2. Roborazzi: Enables screenshot testing without real devices. 3. Benefits: Lowers maintenance costs and improves bug detection. 4. Usage: Integrates with Espresso and Dagger Hilt. 5. Strategy: Combine unit, component, and screen-based tests for optimal coverage