Android functional testing made easy with Kotlin by Marc-Antoine Sauvé

Android functional testing made easy with Kotlin
Level: Intermediate
by Marc-Antoine Sauvé, Hopper

This talk will walk you through how Hopper leveraged Kotlin to build clean DSL based functional testing that separates the “what” from the “how”.

https://gdgmontreal.com/2019/07/26/kotlin-everywhere/

1. Android functional testing made easy with Kotlin Marc-Antoine Sauvé
2. Hello to GDG Sofia! Special thanks to Boris Strandjev, organiser at GDG Sofia, for the help with this talk
3. #1 Mobile Travel App in North America Downloads to date 30M+ Trips planned in our app 60M+ Sold from 126 countries $900M
4. Functional testing according to Wikipedia Functional testing is [...] a type of black-box testing that bases its test cases on the specifications of the software component under test. Functional testing is conducted to evaluate the compliance of a system or component with specified functional requirements. Functional testing usually describes what the system does.
5. Why UI testing? Test their functionalities according to the specs of the different features Test main funnels We were not setup to test the business logic in isolation Not “testing ready” UI testing is not implementation dependant Legacy code
6. Goals Separate the what and the how Clean tests We want to improve our detection of regressions in our most used funnels Test features Generate clear reports to help identify the failures easily Clear failures
7. Concepts They know how do to the stuff, where the tests using them know what to do Robot pattern Its presence in a screen signify that the screen is ready to be used Proof object Allow simple logging to help understand where a test failed and allow to define optional steps in a funnel Steps
8. Lets see it in action
9. Test reports
10. Steps Starting test com.hopper.mountainview.debug.BookingTest#searchTripWhenLoggedOut Ensuring the user is logged out The page HomeScreenRobot is loaded Test step Open home page started Optional test step OnboardingRobot skipped The page HomeScreenRobot is loaded Test step Open home page finished ... Test step Loading after signin page started The page ChooseTravelersRobot is loaded Test step Loading after signin page finished The traveler exists Test step Select traveler started The page RunningBunnyRobot is loaded The page BookingChoosePaymentRobot is not loaded Failed to load page object BookingChoosePaymentRobot in time Taking screenshot of 'searchTripWhenLoggedOut' Screenshot taken
11. Steps Starting test com.hopper.mountainview.debug.BookingTest#searchTripWhenLoggedOut Ensuring the user is logged out The page HomeScreenRobot is loaded Test step Open home page started Optional test step OnboardingRobot skipped The page HomeScreenRobot is loaded Test step Open home page finished ... Test step Loading after signin page started The page ChooseTravelersRobot is loaded Test step Loading after signin page finished The traveler exists Test step Select traveler started The page RunningBunnyRobot is loaded The page BookingChoosePaymentRobot is not loaded Failed to load page object BookingChoosePaymentRobot in time Taking screenshot of 'searchTripWhenLoggedOut' Screenshot taken
12. Steps Starting test com.hopper.mountainview.debug.BookingTest#searchTripWhenLoggedOut Ensuring the user is logged out The page HomeScreenRobot is loaded Test step Open home page started Optional test step OnboardingRobot skipped The page HomeScreenRobot is loaded Test step Open home page finished ... Test step Loading after signin page started The page ChooseTravelersRobot is loaded Test step Loading after signin page finished The traveler exists Test step Select traveler started The page RunningBunnyRobot is loaded The page BookingChoosePaymentRobot is not loaded Failed to load page object BookingChoosePaymentRobot in time Taking screenshot of 'searchTripWhenLoggedOut' Screenshot taken
13. Steps Starting test com.hopper.mountainview.debug.BookingTest#searchTripWhenLoggedOut Ensuring the user is logged out The page HomeScreenRobot is loaded Test step Open home page started Optional test step OnboardingRobot skipped The page HomeScreenRobot is loaded Test step Open home page finished ... Test step Loading after signin page started The page ChooseTravelersRobot is loaded Test step Loading after signin page finished The traveler exists Test step Select traveler started The page RunningBunnyRobot is loaded The page BookingChoosePaymentRobot is not loaded Failed to load page object BookingChoosePaymentRobot in time Taking screenshot of 'searchTripWhenLoggedOut' Screenshot taken
14. Steps Starting test com.hopper.mountainview.debug.BookingTest#searchTripWhenLoggedOut Ensuring the user is logged out The page HomeScreenRobot is loaded Test step Open home page started Optional test step OnboardingRobot skipped The page HomeScreenRobot is loaded Test step Open home page finished ... Test step Loading after signin page started The page ChooseTravelersRobot is loaded Test step Loading after signin page finished The traveler exists Test step Select traveler started The page RunningBunnyRobot is loaded The page BookingChoosePaymentRobot is not loaded Failed to load page object BookingChoosePaymentRobot in time Taking screenshot of 'searchTripWhenLoggedOut' Screenshot taken
15. Steps Starting test com.hopper.mountainview.debug.BookingTest#searchTripWhenLoggedOut Ensuring the user is logged out The page HomeScreenRobot is loaded Test step Open home page started Optional test step OnboardingRobot skipped The page HomeScreenRobot is loaded Test step Open home page finished ... Test step Loading after signin page started The page ChooseTravelersRobot is loaded Test step Loading after signin page finished The traveler exists Test step Select traveler started The page RunningBunnyRobot is loaded The page BookingChoosePaymentRobot is not loaded Failed to load page object BookingChoosePaymentRobot in time Taking screenshot of 'searchTripWhenLoggedOut' Screenshot taken
16. How did we do it?
17. The Robot pattern ▪ Each screens have a Robot class ▪ Defines the elements on the screen you can interact with ▪ Defines the actions you can take on those elements ▪ Test data should be provided, not hardcoded
18. The Robot pattern open class NoProfileRobot(testContext: TestContext) : TestingRobot(testContext) { // 1- List all elements that can be interacted with private val signUpOrLoginButton = selectObject( viewIdSelector = ViewIdSelector(R.id.signup_button), textSelector = TextSelector(R.string.sign_up_log_in) ) // 2- Expose functions to interact with them fun clickSignUp(): LoginOnboardingRobot { signUpOrLoginButton.click() Return LoginOnboardingRobot(testContext) } fun fillInUsername(user: User): NoProfileRobot { usernameField.text = user.username } }
19. The Robot pattern @Test fun signIn() { HomeScreenRobot(testContext) .openTrips() val loginOnboardingRobot = NoProfileRobot(testContext) .clickSignUp() val signInRobot = loginOnboardingRobot .swipeThroughAllTabs() .goToSignin() signInRobot .enterPhoneNumber(DEFAULT_TEST_USER.userData) assertPageLoaded(TripsRobot(testContext)) }
20. The Robot pattern @Test fun signIn() { HomeScreenRobot(testContext) .openTrips() NoProfileRobot(testContext) .clickSignUp() .swipeThroughAllTabs() .goToSignin() .enterPhoneNumber(DEFAULT_TEST_USER.userData) assertPageLoaded(TripsRobot(testContext)) }
21. The Robot pattern fun noProfileRobot(testContext: TestContext, func: NoProfileRobot.() -> Unit = {}) = NoProfileRobot(testContext).apply(func) open class NoProfileRobot(testContext: TestContext) : TestingRobot(testContext) { [...] }
22. The Robot pattern @Test fun signIn() { // 1- Create the HomeScreenRobot homeScreenRobot(testContext) { // this: HomeScreenRobot // 2- Actions to do on the robot openTrips() } // 3- The robots can be chained noProfileRobot(testContext) .clickSignUp { // this: LoginOnboardingRobot swipeThroughAllTabs() }.goToSignin { // this: SignInRobot enterPhoneNumber(DEFAULT_TEST_USER.userData) sendVerificationText() } tripsRobot(testContext) }
23. The Robot pattern fun noProfileRobot(testContext: TestContext, func: NoProfileRobot.() -> Unit = {}) = NoProfileRobot(testContext).apply(func) open class NoProfileRobot(testContext: TestContext) : TestingRobot(testContext) { [...] }
24. The Robot pattern fun noProfileRobot(testContext: TestContext, func: NoProfileRobot.() -> Unit = {}) = NoProfileRobot(testContext).apply(func) open class NoProfileRobot(testContext: TestContext) : TestingRobot(testContext) { [...] }
25. The Robot pattern fun noProfileRobot(testContext: TestContext, func: NoProfileRobot.() -> Unit = {}) = NoProfileRobot(testContext).apply(func) open class NoProfileRobot(testContext: TestContext) : TestingRobot(testContext) { [...] }
26. The Robot pattern fun noProfileRobot(testContext: TestContext, func: NoProfileRobot.() -> Unit = {}) = NoProfileRobot(testContext).apply(func) open class NoProfileRobot(testContext: TestContext) : TestingRobot(testContext) { [...] } public inline fun <T> T.apply(block: T.() -> Unit): T { block() return this }
27. The Robot pattern @Test fun signIn() { homeScreenRobot(testContext) { // this: HomeScreenRobot openTrips() } [...] }
28. The Robot pattern @Test fun signIn() { homeScreenRobot(testContext) { // this: HomeScreenRobot this.openTrips() } [...] }
29. The Robot pattern open class NoProfileRobot(testContext: TestContext) : TestingRobot(testContext) { [...] fun clickSignUp(func: LoginOnboardingRobot.() -> Unit = {}): LoginOnboardingRobot { signUpOrLoginButton.click() return loginOnboardingRobot(testContext, func) } }
30. The Robot pattern open class NoProfileRobot(testContext: TestContext) : TestingRobot(testContext) { [...] fun clickSignUp(func: LoginOnboardingRobot.() -> Unit = {}): LoginOnboardingRobot { signUpOrLoginButton.click() return loginOnboardingRobot(testContext, func) } }
31. The Robot pattern open class NoProfileRobot(testContext: TestContext) : TestingRobot(testContext) { [...] fun clickSignUp(func: LoginOnboardingRobot.() -> Unit = {}): LoginOnboardingRobot { signUpOrLoginButton.click() return loginOnboardingRobot(testContext, func) } }
32. The Robot pattern open class NoProfileRobot(testContext: TestContext) : TestingRobot(testContext) { [...] fun clickSignUp(func: LoginOnboardingRobot.() -> Unit = {}): LoginOnboardingRobot { signUpOrLoginButton.click() return loginOnboardingRobot(testContext, func) } }
33. The Robot pattern @Test fun signIn() { homeScreenRobot(testContext) { // this: HomeScreenRobot openTrips() } noProfileRobot(testContext) .clickSignUp { // this: LoginOnboardingRobot swipeThroughAllTabs() }.goToSignin { // this: SignInRobot enterPhoneNumber(DEFAULT_TEST_USER.userData) sendVerificationText() } tripsRobot(testContext) }
34. The Proof object ▪ uiObjects are not immediately resolved, only when interacted with ▪ we defined annotation @ProofObject ▪ we defined the method TestingRobot.waitForPageToLoad(testContext) ▪ The work of fetching and validating the ProofObject is done by TestContext This allows us to ensure that the screens are loaded appropriately
35. The Proof object open class NoProfileRobot(testContext: TestContext) : TestingRobot(testContext) { @ProofObject private val signUpOrLoginButton = selectObject( viewIdSelector = ViewIdSelector(R.id.signup_button), textSelector = TextSelector(R.string.sign_up_log_in) ) fun clickSignUp(func: LoginOnboardingRobot.() -> Unit = {}): LoginOnboardingRobot { signUpOrLoginButton.click() return loginOnboardingRobot(testContext, func).waitForPageToLoad() } }
36. The TestContext ▪ Knows how to get the ProofObject for a screen ▪ Knows how to validate that the screen is loaded ▪ Knows if there is optional screens that can be skipped ▪ Each Robots have a TestContext that is shared for a given test ▪ The TestContext will timeout if the screen does not appear within reasonable time ▪ Holds the Android context ▪ Holds the Test logger ▪ Ui automator device ▪ Other things (e.g. whether ANR was encountered)
37. Optional screens ▪ define their own proof objects ▪ also define skip steps ▪ each Robot can define optional before and optional after screens ▪ our framework automatically skips these screens Sometimes a test can not determine if a page will be shown or not
38. Optional screens class OnboardingRobot(testContext: TestContext) : OptionalPage(testContext) { @ProofObject private val continueButton = selectObject([...]) private val letsGetStartedButton = selectObject([...]) override fun executeSkipSteps() { continueButton.click() letsGetStartedButton.click() loadThroughLoadingScreen(testContext, ::NavigationBarRobot) { openWatch() } } }
39. Optional screens class OnboardingRobot(testContext: TestContext) : OptionalPage(testContext) { @ProofObject private val continueButton = selectObject([...]) private val letsGetStartedButton = selectObject([...]) override fun executeSkipSteps() { continueButton.click() letsGetStartedButton.click() loadThroughLoadingScreen(testContext, ::NavigationBarRobot) { openWatch() } } }
40. Test structure
41. Test steps Steps logs whenever they start and finish, with their name and relevant information Better logs An optional test will not fail the test if it is there or not Optional steps Allows to add optional steps before or after a given step Hooks
42. Test step withTestStep(testContext, TestStep.OPEN_SETTINGS) { homeScreenRobot(testContext) { openTrips() } tripsRobot(testContext) { openSettings() } } A test is composed of multiple test steps
43. Test step // Extension function on any Robot or Test fun <T : TestContextHolder, U> T.withTestStep( testContext: TestContext, testStep: TestStep, stepBody: T.() -> U ): U { // Log & execute steps before (hooks) // Some more logs val returnValue = stepBody() // Some more logs & ANR check // Log & execute steps after (hooks) return returnValue }
44. Test step fun signIn() { withTestStep(TestStep.OPEN_HOME) { homeScreenRobot(testContext) }.withTestStep(TestStep.OPEN_TRIPS) { // this: HomeScreenRobot openTrips() } withTestStep(TestStep.CLICK_LOGIN) { noProfileRobot(testContext) .clickSignUp() }.withTestStep(TestStep.SCROLL_SIGNIN_INTRO) { // this: LoginOnboardingRobot swipeThroughAllTabs() goToSignin() } [...] }
45. And more
46. Prerequisites @UserLoggedOut @Test fun searchTripWhenLoggedOut() { Other examples of prerequisites: @CreditCardPresent(UserDataSample.NO_MATTER, CardDataSamples.MASTERCARD1) @CreditCardNotPresent(UserDataSample.NO_MATTER, CardDataSamples.MASTERCARD1) @UserLoggedIn(UserDataSample.NO_MATTER) These are things you expect to hold true before you run the test
47. Test against ANRs ▪ At one moment of time we had plenty ▪ We decided to use ANRWatchdog in our debug builds ▪ We also altered the tests to report failure on ANR
48. Thank you!
49. We’re on the Lookout for Talented People Check out current openings at hopper.com/jobs.