Mastering Screenshot Testing Libraries for Android

Slide 1

Slide 1 text

BY SERGIO SASTRE MASTERING SCREENSHOT TESTING LIBRARIES FOR ANDROID

Slide 2

Slide 2 text

Android Studio Electric Eel | 2022.1.1 AGP 7.4 JDK 11 Emulator API 30 - Pixel 3a Tools for the workshop 1 3 4 @Gio_Sastre Github Repo https://github.com/sergio-sastre/Android-screenshot-testing-playground 2 Screenshot testing companion library https://github.com/sergio-sastre/AndroidUiTestingUtils

Slide 3

Slide 3 text

Dropshot, Roborazzi & Paparazzi @Gio_Sastre Workshop’s agenda 1 2 3 Screenshot testing theory What it is & How it works Existing libraries Final thoughts on-device vs. jvm screenshot tests 5 Jvm screenshot tests Roborazzi & Paparazzi On-device screenshot tests Shot & Dropshots Tips & tricks Testing different configurations Which configurations to test 4

Slide 4

Slide 4 text

WHAT Screenshot testing is @Gio_Sastre

Slide 5

Slide 5 text

Special type of UI tests take a screenshot of it, compare it to its reference we inflate a UI component, where and @Gio_Sastre

Slide 6

Slide 6 text

WORKS HOW Screenshot testing @Gio_Sastre

Slide 7

Slide 7 text

record AND verify @Gio_Sastre

Slide 8

Slide 8 text

write test CI @Gio_Sastre

Slide 9

Slide 9 text

record CI @Gio_Sastre

Slide 10

Slide 10 text

= CI @Gio_Sastre

Slide 11

Slide 11 text

CI new


PR
+
@Gio_Sastre

Slide 12

Slide 12 text

approve CI @Gio_Sastre

Slide 13

Slide 13 text

merge CI @Gio_Sastre

Slide 14

Slide 14 text

CI new


PR
verify
@Gio_Sastre

Slide 15

Slide 15 text

CI new


PR
@Gio_Sastre

Slide 16

Slide 16 text

CI new


PR
@Gio_Sastre

Slide 17

Slide 17 text

CI new


PR
@Gio_Sastre

Slide 18

Slide 18 text

CI new


PR
verify
@Gio_Sastre

Slide 19

Slide 19 text

CI new


PR
@Gio_Sastre

Slide 20

Slide 20 text

CI new


PR
@Gio_Sastre

Slide 21

Slide 21 text

CI new


PR
@Gio_Sastre

Slide 22

Slide 22 text

CI Intentional change forgot to record regression bug YES NO @Gio_Sastre

Slide 23

Slide 23 text

@Gio_Sastre Verify CI CI

Slide 24

Slide 24 text

@Gio_Sastre Verify CI CI

Slide 25

Slide 25 text

@Gio_Sastre Verify CI CI

Slide 26

Slide 26 text

@Gio_Sastre Verify CI CI

Slide 27

Slide 27 text

@Gio_Sastre Verify CI CI

Slide 28

Slide 28 text

@Gio_Sastre Verify CI CI

Slide 29

Slide 29 text

@Gio_Sastre Verify CI CI

Slide 30

Slide 30 text

@Gio_Sastre Verify CI CI

Slide 31

Slide 31 text

@Gio_Sastre Verify CI CI

Slide 32

Slide 32 text

@Gio_Sastre Verify CI CI

Slide 33

Slide 33 text

@Gio_Sastre Verify CI CI

Slide 34

Slide 34 text

@Gio_Sastre Verify CI CI

Slide 35

Slide 35 text

@Gio_Sastre Verify CI CI

Slide 36

Slide 36 text

@Gio_Sastre Verify CI CI

Slide 37

Slide 37 text

@Gio_Sastre Verify CI CI

Slide 38

Slide 38 text

@Gio_Sastre Verify CI CI

Slide 39

Slide 39 text

@Gio_Sastre Verify Custom on top of java.awt Differ Differ Custom wrapper on top of external library comparison algorithm https://github.com/dropbox/differ * * *

Slide 40

Slide 40 text

EXIST? Which Screenshot testing libraries @Gio_Sastre

Slide 41

Slide 41 text

Screenshot testing libraries android-testify Paparazzi screenshot-tests for-android Shot Dropshots Snappy Roborazzi @Gio_Sastre

Slide 42

Slide 42 text

on-device Screenshot testing libraries @Gio_Sastre

Slide 43

Slide 43 text

jvm layoutlib robolectric robolectric * Needs to be con igured * Screenshot testing libraries @Gio_Sastre

Slide 44

Slide 44 text

in this workshop jvm jvm on-device on-device Screenshot testing libraries @Gio_Sastre

Slide 45

Slide 45 text

coding time it’s @Gio_Sastre

Slide 46

Slide 46 text

Dropshots Shot tolerance record & verify gradle managed devices test reports general value for all tests only from CLI @Gio_Sastre standard xml report configuration problematic Windows support Kotlin gradle scripts AGP 8.0 ⚠ ⚠ super easy per test from CLI & AS very nice HTML with diffs not supported not supported verify on device verify on local machine + diffs

Slide 47

Slide 47 text

Android Orchestrator without with 18 Tests tips & tricks

Slide 48

Slide 48 text

Android Orchestrator without with 18 Tests tips & tricks

Slide 49

Slide 49 text

coding time it’s @Gio_Sastre

Slide 50

Slide 50 text

Paparazzi complex animations (views) Side effects (compose) render elevation configuration easy but… easy only library Bazel support ⚠ not supported Roborazzi modules app & library modules Bazel support different API levels Activity & Fragment supported not supported not supported not fully supported not supported supported @Gio_Sastre supported2 1 2 1 Due to layoutlib 2 Due to Robolectric 2 supported supported2 2

Slide 51

Slide 51 text

@Gio_Sastre Configuration matters 1 3 Locale UI Mode 2 Orientation 4 6 Custom themes Display Size 5 Font Size

Slide 52

Slide 52 text

@Gio_Sastre Locale + Spanish German Chinese LTR RTL Arabic Urdu Hebrew … …

Slide 53

Slide 53 text

@Gio_Sastre Locale + en_XA LTR RTL ar_XB Pseudolocale Pseudolocale

Slide 54

Slide 54 text

@Gio_Sastre Locale

Slide 55

Slide 55 text

Settings Display Font size > > Font Size @Gio_Sastre

Slide 56

Slide 56 text

@Gio_Sastre Font Size

Slide 57

Slide 57 text

Display Size Settings Display Display size > > @Gio_Sastre

Slide 58

Slide 58 text

Display Size @Gio_Sastre

Slide 59

Slide 59 text

Font Size vs. Display Size @Gio_Sastre

Slide 60

Slide 60 text

Font Size + Display Size @Gio_Sastre

Slide 61

Slide 61 text

MANY CONFIGS? HOW to screenshot test so @Gio_Sastre

Slide 62

Slide 62 text

@Gio_Sastre possible Display size Font size + Locale UI Mode Orientation out of the box out of the box out of the box 1 via updateConfiguration() possible switch height & width 1 Activities under test must be open, otherwise still possible with AndroidUiTestingUtils out of the box Custom theme out of the box possible via updateConfiguration() out of the box out of the box out of the box robolectric based

Slide 63

Slide 63 text

coding time it’s @Gio_Sastre

Slide 64

Slide 64 text

JVM ? ON-DEVICE or @Gio_Sastre

Slide 65

Slide 65 text

supported supported supported supported supported complex animations (views) Side effects (compose) render elevation not supported different API levels Activity & Fragment not supported not supported supported not fully supported not supported supported @Gio_Sastre supported2 1 2 1 Due to layoutlib 2 Due to Robolectric supported supported 2 robolectric based 2 2

Slide 66

Slide 66 text

@Gio_Sastre * Only possible with Paparazzi and Roborazzi On-device on-device vs. jvm Best fidelity Foldable support Special configuration (adb) Jvm Very fast Less flaky Accessibility services * (no issues with emulators)

Slide 67

Slide 67 text

@Gio_Sastre on-device vs. jvm https://ubiratansoares.dev/posts/screenshot-testing-for-android-landscape/

Slide 68

Slide 68 text

Repo & Library recap 1 @Gio_Sastre Github Repo https://github.com/sergio-sastre/Android-screenshot-testing-playground 2 Screenshot testing companion library https://github.com/sergio-sastre/AndroidUiTestingUtils

Slide 69

Slide 69 text

attention for your THANKS

Slide 70

Slide 70 text

GitHub @SergioSastre sergio-sastre-florez @sergio-sastre @Gio_Sastre Hashnode blogs Sergio Sastre Flórez Lead & Senior Android developer appdev.de Linkedin