$30 off During Our Annual Pro Sale. View Details »

Introducing UI Testing to Your Team

Mustafa Berkay Mutlu
November 25, 2019
Programming
1
140

Introducing UI Testing to Your Team

Mustafa Berkay Mutlu

November 25, 2019
Tweet

More Decks by Mustafa Berkay Mutlu

See All by Mustafa Berkay Mutlu
Dependency Injection Using Dagger 2
mustafaberkaymutlu
1
95
UI Testing on Android
mustafaberkaymutlu
0
51
Fundamentals of React Native
mustafaberkaymutlu
0
44
Git Basics
mustafaberkaymutlu
0
50

Other Decks in Programming

See All in Programming
Open Source - A Journey of Contribution and Collaboration
ivargrimstad
0
1.1k
BigQuery のデータ品質やデータ活用を高める Dataplex 等の活用
mot_techtalk
5
1.1k
ブログのリアクションから始めるGoのパフォーマンス分析入門
always_allokay
0
110
第42回ロボティクス勉強会：実環境を手軽にシミュレータ環境に持ってくるpointcloud2gazebo
atinfinity
0
1k
fun HelloKMP(): GladToMeetYou
l2hyunwoo
0
150
Microservices: what does good look like? (Nov 2023)
cer
PRO
0
1.2k
Scroll-driven AnimationsとCSSの進化
degudegu2510
1
100
ニジエチューニング2023-12
nijieinfra
0
200
DIGGLEの分析基盤アーキテクチャ
zakky21
0
240
AWS IoT FleetWise のアップデートについて
wakatsuki
0
150
ABEMA モバイルアプリにおける 「アーキテクチャ」 / ABEMA Mobile Architecture (CA.swift #18)
akkyie
1
400
Firebase Apple SDK 年間ダイジェスト: 2023年 / Firebase Apple SDK Annual Digest: 2023
treastrain
0
460

Featured

See All Featured
How New CSS Is Changing Everything About Graphic Design on the Web
jensimmons
215
12k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
12
1.1k
Bootstrapping a Software Product
garrettdimon
PRO
299
110k
Art, The Web, and Tiny UX
lynnandtonic
286
19k
Build your cross-platform service in a week with App Engine
jlugia
223
17k
The World Runs on Bad Software
bkeepers
PRO
59
6.2k
Debugging Ruby Performance
tmm1
68
11k
Intergalactic Javascript Robots from Outer Space
tanoku
264
26k
Typedesign – Prime Four
hannesfritz
35
1.8k
Designing the Hi-DPI Web
ddemaree
274
33k
Optimizing for Happiness
mojombo
368
68k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
0
2.7k

Transcript

  1. Introducing UI Testing
    to Your Team
    Mustafa Mutlu

    View Slide

  2. @bekobird
    Android Engineer at N26
    Mustafa
    MoneyBeam, FPS and BACS Transactions for the UK

    View Slide

  3. 2014

    View Slide

  4. 2014
    Number of Android devs
    5

    View Slide

  5. 2016
    Number of Android devs
    5

    View Slide

  6. 2016
    Number of Android devs
    5

    View Slide

  7. 2017
    5
    Number of Android devs

    View Slide

  8. 5
    2018
    15
    Number of Android devs

    View Slide

  9. 32
    15
    Now
    Number of Android devs
    2019
    5

    View Slide

  10. E2E
    Tests
    Integration
    Tests
    Unit Tests
    Testing Strategy

    View Slide

  11. E2E
    Tests
    Integration
    Tests
    Unit Tests
    Testing Strategy

    View Slide

  12. E2E
    Tests
    Integration
    Tests
    Unit Tests
    Testing Strategy

    View Slide

  13. E2E
    Tests
    Integration
    Tests
    Unit Tests
    Testing Strategy

    View Slide

  14. E2E
    Tests
    Integration
    Tests
    Unit Tests
    Testing Strategy
    Slower
    Faster
    More Integration
    More Isolation

    View Slide

  15. E2E
    Tests
    Integration
    Tests
    Testing Strategy

    View Slide

  16. A Typical Test Case
    • ID and name of the test
    • Preconditions
    • Steps with expected result

    View Slide

  17. A Typical Test Case
    • Step: When I enter MoneyBeam and select a contact
    • Expected:
    Then I see the money amount entrance screen
    and the screen has these:
    • Top left: origin account's name and balance
    • Top right: destination account's name
    • Center: money input
    • Center: description text
    • Bottom: Continue button that is not in enabled/clickable state

    View Slide

  18. •
    3200+ test cases
    •
    Testing 500 to 850 cases, in each release
    •
    Releasing every once in 3 weeks
    •
    This is a lot of work!

    View Slide

  19. • Automated end-to-end UI tests
    • Both for Android and iOS
    • Black-box testing
    • Uses Appium
    Photo by ActionVance on Unsplash
    Batman Project

    View Slide

  20. •
    Slow test execution
    •
    Not included in our CI pipeline
    •
    Difficult to maintain
    •
    Flakiness caused by E2E
    Batman Project

    View Slide

  21. Espresso
    • Focused on testing the UI (not E2E)
    • Maintained by the same feature developers

    View Slide

  22. Robot Pattern
    https://jakewharton.com/testing-robots/
    Recommended talk by
    Jake Wharton

    View Slide

  23. @Test
    fun moneyBeamSample() {
    apiServer.registerJsonResponse(MoneyBeamApiRequests.MyApiRequest("a-param"), responseRaw().jsonify())
    val intent = MoneyBeamTransferActivity.newIntent(targetContext, "Namey McNameface")
    setupUser {
    firstName("John")
    lastName("Doe")
    currency("GBP")
    balance(10.toBigDecimal())
    }
    activityScenarioRule.launchActivity(intent)
    ...

    View Slide

  24. @Test
    fun moneyBeamSample() {
    apiServer.registerJsonResponse(MoneyBeamApiRequests.MyApiRequest("a-param"), responseRaw().jsonify())
    val intent = MoneyBeamTransferActivity.newIntent(targetContext, "Namey McNameface")
    setupUser {
    firstName("John")
    lastName("Doe")
    currency("GBP")
    balance(10.toBigDecimal())
    }
    activityScenarioRule.launchActivity(intent)
    ...

    View Slide

  25. @Test
    fun moneyBeamSample() {
    apiServer.registerJsonResponse(MoneyBeamApiRequests.MyApiRequest("a-param"), responseRaw().jsonify())
    val intent = MoneyBeamTransferActivity.newIntent(targetContext, "Namey McNameface")
    setupUser {
    firstName("John")
    lastName("Doe")
    currency("GBP")
    balance(10.toBigDecimal())
    }
    activityScenarioRule.launchActivity(intent)
    ...

    View Slide

  26. @Test
    fun moneyBeamSample() {
    apiServer.registerJsonResponse(MoneyBeamApiRequests.MyApiRequest("a-param"), responseRaw().jsonify())
    val intent = MoneyBeamTransferActivity.newIntent(targetContext, "Namey McNameface")
    setupUser {
    firstName("John")
    lastName("Doe")
    currency("GBP")
    balance(10.toBigDecimal())
    }
    activityScenarioRule.launchActivity(intent)
    ...

    View Slide

  27. @Test
    fun moneyBeamSample() {
    apiServer.registerJsonResponse(MoneyBeamApiRequests.MyApiRequest("a-param"), responseRaw().jsonify())
    val intent = MoneyBeamTransferActivity.newIntent(targetContext, "Namey McNameface")
    setupUser {
    firstName("John")
    lastName("Doe")
    currency("GBP")
    balance(10.toBigDecimal())
    }
    activityScenarioRule.launchActivity(intent)
    ...

    View Slide

  28. @Test
    fun moneyBeamSample() {
    apiServer.registerJsonResponse(MoneyBeamApiRequests.MyApiRequest("a-param"), responseRaw().jsonify())
    val intent = MoneyBeamTransferActivity.newIntent(targetContext, "Namey McNameface")
    setupUser {
    firstName("John")
    lastName("Doe")
    currency("GBP")
    balance(10.toBigDecimal())
    }
    activityScenarioRule.launchActivity(intent)
    ...

    View Slide

  29. ...
    assertClickableNavigateUpButton()
    onView(allOf(withParent(withId(originContainer)), withId(nameText)))
    .check(matches(allOf(isDisplayed(), withText("John Doe"))))
    onView(allOf(withParent(withId(originContainer)), withId(initialsText)))
    .check(matches(allOf(isDisplayed(), withText("JD"))))
    onView(allOf(withParent(withId(originContainer)), withId(balanceText)))
    .check(matches(allOf(isDisplayed(), withText("£10.00"))))
    assertDisplayed(descriptionText, "Send to Namey")
    assertNotDisplayed(errorText)
    isButtonEnabled(continueButton)
    writeTo(amountEditText, "1100")
    assertDisplayed(errorText, "You can only send MoneyBeams up to £1,000")
    assertNotDisplayed(descriptionText)
    isButtonDisabled(continueButton)
    }

    View Slide

  30. ...
    assertClickableNavigateUpButton()
    onView(allOf(withParent(withId(originContainer)), withId(nameText)))
    .check(matches(allOf(isDisplayed(), withText("John Doe"))))
    onView(allOf(withParent(withId(originContainer)), withId(initialsText)))
    .check(matches(allOf(isDisplayed(), withText("JD"))))
    onView(allOf(withParent(withId(originContainer)), withId(balanceText)))
    .check(matches(allOf(isDisplayed(), withText("£10.00"))))
    assertDisplayed(descriptionText, "Send to Namey")
    assertNotDisplayed(errorText)
    isButtonEnabled(continueButton)
    writeTo(amountEditText, "1100")
    assertDisplayed(errorText, "You can only send MoneyBeams up to £1,000")
    assertNotDisplayed(descriptionText)
    isButtonDisabled(continueButton)
    }

    View Slide

  31. ...
    assertClickableNavigateUpButton()
    onView(allOf(withParent(withId(originContainer)), withId(nameText)))
    .check(matches(allOf(isDisplayed(), withText("John Doe"))))
    onView(allOf(withParent(withId(originContainer)), withId(initialsText)))
    .check(matches(allOf(isDisplayed(), withText("JD"))))
    onView(allOf(withParent(withId(originContainer)), withId(balanceText)))
    .check(matches(allOf(isDisplayed(), withText("£10.00"))))
    assertDisplayed(descriptionText, "Send to Namey")
    assertNotDisplayed(errorText)
    isButtonEnabled(continueButton)
    writeTo(amountEditText, "1100")
    assertDisplayed(errorText, "You can only send MoneyBeams up to £1,000")
    assertNotDisplayed(descriptionText)
    isButtonDisabled(continueButton)
    }

    View Slide

  32. ...
    assertClickableNavigateUpButton()
    onView(allOf(withParent(withId(originContainer)), withId(nameText)))
    .check(matches(allOf(isDisplayed(), withText("John Doe"))))
    onView(allOf(withParent(withId(originContainer)), withId(initialsText)))
    .check(matches(allOf(isDisplayed(), withText("JD"))))
    onView(allOf(withParent(withId(originContainer)), withId(balanceText)))
    .check(matches(allOf(isDisplayed(), withText("£10.00"))))
    assertDisplayed(descriptionText, "Send to Namey")
    assertNotDisplayed(errorText)
    isButtonEnabled(continueButton)
    writeTo(amountEditText, "1100")
    assertDisplayed(errorText, "You can only send MoneyBeams up to £1,000")
    assertNotDisplayed(descriptionText)
    isButtonDisabled(continueButton)
    }

    View Slide

  33. Robot Pattern
    View ViewModel Domain Data
    What
    How
    How
    N26 Android App

    View Slide

  34. Robot Pattern
    View ViewModel Domain Data
    What
    How
    How
    N26 Android App
    Test
    Test
    What & How

    View Slide

  35. Robot Pattern
    View ViewModel Domain Data
    What
    How
    How
    N26 Android App
    Robot
    How
    Test
    What

    View Slide

  36. Robot Pattern
    View ViewModel Domain Data
    What
    How
    How
    N26 Android App
    Robot
    Test
    How
    What

    View Slide

  37. ...
    assertClickableNavigateUpButton()
    onView(allOf(withParent(withId(originContainer)), withId(nameText)))
    .check(matches(allOf(isDisplayed(), withText("John Doe"))))
    onView(allOf(withParent(withId(originContainer)), withId(initialsText)))
    .check(matches(allOf(isDisplayed(), withText("JD"))))
    onView(allOf(withParent(withId(originContainer)), withId(balanceText)))
    .check(matches(allOf(isDisplayed(), withText("£10.00"))))
    assertDisplayed(descriptionText, "Send to Namey")
    assertNotDisplayed(errorText)
    isButtonEnabled(continueButton)
    writeTo(amountEditText, "1100")
    assertDisplayed(errorText, "You can only send MoneyBeams up to £1,000")
    assertNotDisplayed(descriptionText)
    isButtonDisabled(continueButton)
    }

    View Slide

  38. ...
    moneyBeamSampleRobot {
    assertClickableNavigateUpButton()
    isOriginDisplayed("John Doe", "JD", "£10.00")
    isDestinationDisplayed("Namey McNameface", "NM")
    isDescriptionDisplayed("Send to Namey")
    isContinueButtonDisabled()
    enterAmount("1100")
    isErrorDisplayed("You can only send MoneyBeams up to £1,000")
    isContinueButtonDisabled()
    enterAmount("10")
    isDescriptionDisplayed("Send to Namey")
    isContinueButtonEnabled()
    }
    }

    View Slide

  39. ...
    moneyBeamSampleRobot {
    assertClickableNavigateUpButton()
    isOriginDisplayed("John Doe", "JD", "£10.00")
    isDestinationDisplayed("Namey McNameface", "NM")
    isDescriptionDisplayed("Send to Namey")
    isContinueButtonDisabled()
    enterAmount("1100")
    isErrorDisplayed("You can only send MoneyBeams up to £1,000")
    isContinueButtonDisabled()
    enterAmount("10")
    isDescriptionDisplayed("Send to Namey")
    isContinueButtonEnabled()
    }
    }

    View Slide

  40. ...
    moneyBeamSampleRobot {
    assertClickableNavigateUpButton()
    isOriginDisplayed("John Doe", "JD", "£10.00")
    isDestinationDisplayed("Namey McNameface", "NM")
    isDescriptionDisplayed("Send to Namey")
    isContinueButtonDisabled()
    enterAmount("1100")
    isErrorDisplayed("You can only send MoneyBeams up to £1,000")
    isContinueButtonDisabled()
    enterAmount("10")
    isDescriptionDisplayed("Send to Namey")
    isContinueButtonEnabled()
    }
    }

    View Slide

  41. ...
    moneyBeamSampleRobot {
    assertClickableNavigateUpButton()
    isOriginDisplayed("John Doe", "JD", "£10.00")
    isDestinationDisplayed("Namey McNameface", "NM")
    isDescriptionDisplayed("Send to Namey")
    isContinueButtonDisabled()
    enterAmount("1100")
    isErrorDisplayed("You can only send MoneyBeams up to £1,000")
    isContinueButtonDisabled()
    enterAmount("10")
    isDescriptionDisplayed("Send to Namey")
    isContinueButtonEnabled()
    }
    }

    View Slide

  42. class MoneyBeamSampleRobot : BaseRobot() {
    private val amountEditText = R.id.editMoneyBeamTransferAmount
    private val descriptionText = R.id.textMoneyBeamTransferAmountDescription
    private val errorText = R.id.textMoneyBeamTransferAmountError
    // other view IDs
    fun isDescriptionDisplayed(text: String) {
    assertDisplayed(descriptionText, text)
    assertNotDisplayed(errorText)
    }
    fun isErrorDisplayed(text: String) { ... }
    fun isOriginDisplayed(name: String, initials: String, balance: String) {
    onView(allOf(withParent(withId(originContainer)), withId(nameText))).check(matches(allOf(isDisplayed(), withText(name))))
    onView(allOf(withParent(withId(originContainer)), withId(initialsText))).check(matches(allOf(isDisplayed(), withText(initials))))
    onView(allOf(withParent(withId(originContainer)), withId(balanceText))).check(matches(allOf(isDisplayed(), withText(balance))))
    }
    fun isDestinationDisplayed(name: String, initials: String) { ... }
    fun isContinueButtonEnabled() = isButtonEnabled(continueButton)
    fun isContinueButtonDisabled() = isButtonDisabled(continueButton)
    fun enterAmount(amount: String) = writeTo(amountEditText, amount)
    fun clickOnContinueButton() = clickOn(continueButton)
    }

    View Slide

  43. class MoneyBeamSampleRobot : BaseRobot() {
    private val amountEditText = R.id.editMoneyBeamTransferAmount
    private val descriptionText = R.id.textMoneyBeamTransferAmountDescription
    private val errorText = R.id.textMoneyBeamTransferAmountError
    // other view IDs
    fun isDescriptionDisplayed(text: String) {
    assertDisplayed(descriptionText, text)
    assertNotDisplayed(errorText)
    }
    fun isErrorDisplayed(text: String) { ... }
    fun isOriginDisplayed(name: String, initials: String, balance: String) {
    onView(allOf(withParent(withId(originContainer)), withId(nameText))).check(matches(allOf(isDisplayed(), withText(name))))
    onView(allOf(withParent(withId(originContainer)), withId(initialsText))).check(matches(allOf(isDisplayed(), withText(initials))))
    onView(allOf(withParent(withId(originContainer)), withId(balanceText))).check(matches(allOf(isDisplayed(), withText(balance))))
    }
    fun isDestinationDisplayed(name: String, initials: String) { ... }
    fun isContinueButtonEnabled() = isButtonEnabled(continueButton)
    fun isContinueButtonDisabled() = isButtonDisabled(continueButton)
    fun enterAmount(amount: String) = writeTo(amountEditText, amount)
    fun clickOnContinueButton() = clickOn(continueButton)
    }

    View Slide

  44. class MoneyBeamSampleRobot : BaseRobot() {
    private val amountEditText = R.id.editMoneyBeamTransferAmount
    private val descriptionText = R.id.textMoneyBeamTransferAmountDescription
    private val errorText = R.id.textMoneyBeamTransferAmountError
    // other view IDs
    fun isDescriptionDisplayed(text: String) {
    assertDisplayed(descriptionText, text)
    assertNotDisplayed(errorText)
    }
    fun isErrorDisplayed(text: String) { ... }
    fun isOriginDisplayed(name: String, initials: String, balance: String) {
    onView(allOf(withParent(withId(originContainer)), withId(nameText))).check(matches(allOf(isDisplayed(), withText(name))))
    onView(allOf(withParent(withId(originContainer)), withId(initialsText))).check(matches(allOf(isDisplayed(), withText(initials))))
    onView(allOf(withParent(withId(originContainer)), withId(balanceText))).check(matches(allOf(isDisplayed(), withText(balance))))
    }
    fun isDestinationDisplayed(name: String, initials: String) { ... }
    fun isContinueButtonEnabled() = isButtonEnabled(continueButton)
    fun isContinueButtonDisabled() = isButtonDisabled(continueButton)
    fun enterAmount(amount: String) = writeTo(amountEditText, amount)
    fun clickOnContinueButton() = clickOn(continueButton)
    }

    View Slide

  45. class MoneyBeamSampleRobot : BaseRobot() {
    private val amountEditText = R.id.editMoneyBeamTransferAmount
    private val descriptionText = R.id.textMoneyBeamTransferAmountDescription
    private val errorText = R.id.textMoneyBeamTransferAmountError
    // other view IDs
    fun isDescriptionDisplayed(text: String) {
    assertDisplayed(descriptionText, text)
    assertNotDisplayed(errorText)
    }
    fun isErrorDisplayed(text: String) { ... }
    fun isOriginDisplayed(name: String, initials: String, balance: String) {
    onView(allOf(withParent(withId(originContainer)), withId(nameText))).check(matches(allOf(isDisplayed(), withText(name))))
    onView(allOf(withParent(withId(originContainer)), withId(initialsText))).check(matches(allOf(isDisplayed(), withText(initials))))
    onView(allOf(withParent(withId(originContainer)), withId(balanceText))).check(matches(allOf(isDisplayed(), withText(balance))))
    }
    fun isDestinationDisplayed(name: String, initials: String) { ... }
    fun isContinueButtonEnabled() = isButtonEnabled(continueButton)
    fun isContinueButtonDisabled() = isButtonDisabled(continueButton)
    fun enterAmount(amount: String) = writeTo(amountEditText, amount)
    fun clickOnContinueButton() = clickOn(continueButton)
    }

    View Slide

  46. class MoneyBeamSampleRobot : BaseRobot() {
    private val amountEditText = R.id.editMoneyBeamTransferAmount
    private val descriptionText = R.id.textMoneyBeamTransferAmountDescription
    private val errorText = R.id.textMoneyBeamTransferAmountError
    // other view IDs
    fun isDescriptionDisplayed(text: String) {
    assertDisplayed(descriptionText, text)
    assertNotDisplayed(errorText)
    }
    fun isErrorDisplayed(text: String) { ... }
    fun isOriginDisplayed(name: String, initials: String, balance: String) {
    onView(allOf(withParent(withId(originContainer)), withId(nameText))).check(matches(allOf(isDisplayed(), withText(name))))
    onView(allOf(withParent(withId(originContainer)), withId(initialsText))).check(matches(allOf(isDisplayed(), withText(initials))))
    onView(allOf(withParent(withId(originContainer)), withId(balanceText))).check(matches(allOf(isDisplayed(), withText(balance))))
    }
    fun isDestinationDisplayed(name: String, initials: String) { ... }
    fun isContinueButtonEnabled() = isButtonEnabled(continueButton)
    fun isContinueButtonDisabled() = isButtonDisabled(continueButton)
    fun enterAmount(amount: String) = writeTo(amountEditText, amount)
    fun clickOnContinueButton() = clickOn(continueButton)
    }

    View Slide

  47. Challenges

    View Slide

  48. Animations
    "To avoid flakiness, we highly recommend that you turn off system animations on the
    virtual or physical devices used for testing." - Espresso docs

    View Slide

  49. Animations
    • System animations
    • Dynamic Animations from AndroidX


    Our solution: add a switch to turn off custom animations

    View Slide

  50. Dealing Asynchronicity
    • Tests need to know when to wait and proceed
    • Causing flakiness
    Our solution: make use of Idling Resources
    • OkHttpIdlingResource
    • RxIdler

    View Slide

  51. Mocking Data Sources

    View Slide

  52. Mocking Data Sources
    Domain
    Data

    View Slide

  53. Mocking Data Sources
    Domain
    Data
    Repository
    Remote Data Source

    View Slide

  54. Mocking Data Sources
    Domain
    Data
    Repository
    Remote Data Source Framework Data Source

    View Slide

  55. Mocking Data Sources
    Domain
    Data
    Repository
    Remote Data Source Fake Data Source

    View Slide

  56. Dealing with Localization
    English (US) English (Canada)
    • Same account (United States)
    • Same currency (USD)
    • Same language (English)
    • Different device locale
    • Result: different money representation

    View Slide

  57. Dealing with Localization
    • Using device locale for formatting monetary amounts
    • Test device's locale is unpredictable
    • Causing flaky tests
    Our solution: add a test rule to define the device locale

    View Slide

  58. Dealing With Flakiness
    • Identify the cause and update the test
    • Add retry for failed tests (FlakyTestRule from the Barista library)

    View Slide

  59. Integration with CI Pipeline

    View Slide

  60. Integration with CI Pipeline
    3 min

    View Slide

  61. Integration with CI Pipeline
    5 min

    View Slide

  62. Integration with CI Pipeline
    10 min

    View Slide

  63. Integration with CI Pipeline
    20 min

    View Slide

  64. Integration with CI Pipeline
    Then what?

    View Slide

  65. Selectively Running the Tests
    • Find changed files from the Git history
    • Find which features are changed
    • Run the tests of the changed features
    • If there is a change in a common module, trigger the full run
    • See: goo.gle/androidx-dependency-tracker

    View Slide

  66. 20+ minutes
    UI Testing Pipeline

    View Slide

  67. 3 min
    UI Testing Pipeline

    View Slide

  68. android-project git:(my-branch) ./run-on-remote.sh -h
    |
    Running on Remote
    usage:
    -s | --skip-build Skip building tasks
    -n name | --test-name name Specify the name of the test. Default is branch name
    -d device | --device device Specify the name of the device on which the tests will be run. Default is random
    -p packages | --packages packages The list of packages to run the tests (comma separated). Default is all
    -l | --log Log the output to log.txt
    -h | --help Display help

    View Slide

  69. Running on Remote
    Test Name Device Name
    Execution Time
    Duration

    my-branch-name
    my-branch-name
    my-branch-name

    View Slide

  70. Reporting the Test Results
    • Link Espresso tests with test case IDs
    • Add a test rule to watch the test results
    • Post the test results to test management tool

    View Slide

  71. Sample Test Run
    MoneyBeam Spaces Premium

    View Slide

  72. Unit Tests
    ~500 UI Tests
    Functional UI
    Tests
    Oct 14 Jan 2016 Jan 2017 Jan 2018 Feb 2019
    ~9000 Unit tests
    Today

    View Slide

  73. Unit Tests
    ~500 UI Tests
    Oct 14 Jan 2016 Jan 2017 Jan 2018 Feb 2019 Today
    Functional UI Tests
    ~9000 Unit tests

    View Slide

  74. Unit Tests
    ~500 UI Tests
    Oct 14 Jan 2016 Jan 2017 Jan 2018 Feb 2019 Today
    Functional UI Tests
    Automated Regression
    10%
    ~9000 Unit tests

    View Slide

  75. Unit Tests
    ~500 UI Tests
    Oct 14 Jan 2016 Jan 2017 Jan 2018 Feb 2019 Today
    Functional UI Tests
    Automated Regression
    50%
    ~9000 Unit tests

    View Slide

  76. •
    Faster test execution
    •
    Part of our CI pipeline
    •
    Maintained together with the production code
    •
    ☑ Mostly trusty
    Espresso Project

    View Slide

  77. A Few Ideas

    View Slide

  78. Feature 1 Feature N
    Base
    App

    View Slide

  79. App
    Feature 1
    Base

    View Slide

  80. App Feature 1 App
    Feature 1
    Base

    View Slide

  81. App Feature 1 App
    Feature 1
    Base

    View Slide

  82. Feature 1 App
    Running Tests on Feature Apps
    Feature 1

    View Slide

  83. Is amount input displayed?
    ✅ Passed
    Is amount input displayed?
    ✅ Passed

    View Slide

  84. Snapshot Testing
    Verifies view attributes:
    • Position
    • Color
    • Size
    Is amount input displayed?
    ✅ Passed
    Is amount input displayed?
    ✅ Passed

    View Slide

  85. Snapshot Testing
    Verifies view attributes:
    • Position
    • Color
    • Size
    Is amount input displayed?
    ✅ Passed
    Is amount input displayed?
    ❌ Failed

    View Slide

  86. Thank You
    Questions?

    View Slide