ROBOTS, THE BEST TESTERS YOU WILL EVER BUILD

The talk will bring attendees on a journey from the current out of the box approach to UI automation testing, talk about the pitfalls and then introduce through Swift, the Robot pattern. The Robot pattern is a pattern commonly applied to test automation which allows an abstraction layer that is easy to write and easy to understand, so much so that even a product manager could write the tests. Moving beyond the basics of the pattern and show it running in action with a real app, we will evolve the pattern to leverage the latest function builders offered in Swift 5 to make our tests even more expressive, terse and easy to write.

Attendees will leave, having a new found love for test automation and how to put a strong pattern to use in their own code bases.

Transcript

1. Robots The Best Testers You Will Ever Build 1
3. Alan Cooke — Mobile Architect @ Zendesk 3
4. Test automation isn't without it's challenges — Me 4
5. Mobile test automation is hard 5
6. $$$ 6
7. low value 7
10. What makes a good UI test? 1. Easy to read 2. Easy to write 3. Easy to extend 4. Easy to debug 10
13. Let's write our first test 1. Select a location 2. Check the weather displays 3. Check some known informations is displayed 13
14. func testSelectSavedCityDisplaysCorrectly() { //Given let city = "Dublin" let application = XCUIApplication() //When let collectionViewsQuery = application.collectionViews collectionViewsQuery.scrollViews.otherElements.staticTexts[city].tap() collectionViewsQuery.cells.otherElements.containing(.image, identifier:"Arrow").element.tap() //Then let predicate = NSPredicate(format: "label CONTAINS[c] %@", "Feels Like") let elementQuery = collectionViewsQuery.staticTexts.containing(predicate) XCTAssertTrue(elementQuery.count == 1) } 14
15. func testSelectSavedCityDisplaysCorrectly() { //Given let city = "Dublin" let application = XCUIApplication() //When let collectionViewsQuery = application.collectionViews collectionViewsQuery.scrollViews.otherElements.staticTexts[city].tap() collectionViewsQuery.cells.otherElements.containing(.image, identifier:"Arrow").element.tap() //Then let predicate = NSPredicate(format: "label CONTAINS[c] %@", "Feels Like") let elementQuery = collectionViewsQuery.staticTexts.containing(predicate) XCTAssertTrue(elementQuery.count == 1) } 14
16. func testSelectSavedCityDisplaysCorrectly() { //Given let city = "Dublin" let application = XCUIApplication() //When let collectionViewsQuery = application.collectionViews collectionViewsQuery.scrollViews.otherElements.staticTexts[city].tap() collectionViewsQuery.cells.otherElements.containing(.image, identifier:"Arrow").element.tap() //Then let predicate = NSPredicate(format: "label CONTAINS[c] %@", "Feels Like") let elementQuery = collectionViewsQuery.staticTexts.containing(predicate) XCTAssertTrue(elementQuery.count == 1) } 14
17. func testSelectSavedCityDisplaysCorrectly() { //Given let city = "Dublin" let application = XCUIApplication() //When let collectionViewsQuery = application.collectionViews collectionViewsQuery.scrollViews.otherElements.staticTexts[city].tap() collectionViewsQuery.cells.otherElements.containing(.image, identifier:"Arrow").element.tap() //Then let predicate = NSPredicate(format: "label CONTAINS[c] %@", "Feels Like") let elementQuery = collectionViewsQuery.staticTexts.containing(predicate) XCTAssertTrue(elementQuery.count == 1) } 14
18. Let's write another test 1. Swipe on a location 2. Select delete 3. Verify it is gone 15
19. func testDeleteSavedCity() { //Given let city = "Dublin" let application = XCUIApplication() //When let scrollViewsQuery = application.collectionViews.scrollViews scrollViewsQuery.otherElements.staticTexts[city].swipeRight() scrollViewsQuery.otherElements .containing(.staticText, identifier:city) .children(matching: .other) .element .children(matching: .other) .element .children(matching: .map) .element.swipeLeft() application.collectionViews .children(matching: .cell) .element(boundBy: 3) .scrollViews.otherElements .containing(.staticText, identifier:"Delete") .element.tap() let collectionViewsQuery = application.collectionViews let predicate = NSPredicate(format: "label CONTAINS[c] %@", city) let elementQuery = collectionViewsQuery.scrollViews.otherElements.staticTexts.containing(predicate) //Assert that there is no city XCTAssertTrue(elementQuery.count == 0) } 16
20. func testDeleteSavedCity() { //Given let city = "Dublin" let application = XCUIApplication() //When let scrollViewsQuery = application.collectionViews.scrollViews scrollViewsQuery.otherElements.staticTexts[city].swipeRight() scrollViewsQuery.otherElements .containing(.staticText, identifier:city) .children(matching: .other) .element .children(matching: .other) .element .children(matching: .map) .element.swipeLeft() application.collectionViews .children(matching: .cell) .element(boundBy: 3) .scrollViews.otherElements .containing(.staticText, identifier:"Delete") .element.tap() let collectionViewsQuery = application.collectionViews let predicate = NSPredicate(format: "label CONTAINS[c] %@", city) let elementQuery = collectionViewsQuery.scrollViews.otherElements.staticTexts.containing(predicate) //Assert that there is no city XCTAssertTrue(elementQuery.count == 0) } 16
21. func testDeleteSavedCity() { //Given let city = "Dublin" let application = XCUIApplication() //When let scrollViewsQuery = application.collectionViews.scrollViews scrollViewsQuery.otherElements.staticTexts[city].swipeRight() scrollViewsQuery.otherElements .containing(.staticText, identifier:city) .children(matching: .other) .element .children(matching: .other) .element .children(matching: .map) .element.swipeLeft() application.collectionViews .children(matching: .cell) .element(boundBy: 3) .scrollViews.otherElements .containing(.staticText, identifier:"Delete") .element.tap() let collectionViewsQuery = application.collectionViews let predicate = NSPredicate(format: "label CONTAINS[c] %@", city) let elementQuery = collectionViewsQuery.scrollViews.otherElements.staticTexts.containing(predicate) //Assert that there is no city XCTAssertTrue(elementQuery.count == 0) } 16
22. func testDeleteSavedCity() { //Given let city = "Dublin" let application = XCUIApplication() //When let scrollViewsQuery = application.collectionViews.scrollViews scrollViewsQuery.otherElements.staticTexts[city].swipeRight() scrollViewsQuery.otherElements .containing(.staticText, identifier:city) .children(matching: .other) .element .children(matching: .other) .element .children(matching: .map) .element.swipeLeft() application.collectionViews .children(matching: .cell) .element(boundBy: 3) .scrollViews.otherElements .containing(.staticText, identifier:"Delete") .element.tap() let collectionViewsQuery = application.collectionViews let predicate = NSPredicate(format: "label CONTAINS[c] %@", city) let elementQuery = collectionViewsQuery.scrollViews.otherElements.staticTexts.containing(predicate) //Assert that there is no city XCTAssertTrue(elementQuery.count == 0) } 16
23. func testDeleteSavedCity() { //Given let city = "Dublin" let application = XCUIApplication() //When let scrollViewsQuery = application.collectionViews.scrollViews scrollViewsQuery.otherElements.staticTexts[city].swipeRight() scrollViewsQuery.otherElements .containing(.staticText, identifier:city) .children(matching: .other) .element .children(matching: .other) .element .children(matching: .map) .element.swipeLeft() application.collectionViews .children(matching: .cell) .element(boundBy: 3) .scrollViews.otherElements .containing(.staticText, identifier:"Delete") .element.tap() let collectionViewsQuery = application.collectionViews let predicate = NSPredicate(format: "label CONTAINS[c] %@", city) let elementQuery = collectionViewsQuery.scrollViews.otherElements.staticTexts.containing(predicate) //Assert that there is no city XCTAssertTrue(elementQuery.count == 0) } 16
24. Let's write more tests 1. Select add location 2. From Search screen, enter "Sofia, Bulgaria" 3. Confirm it displays on the list 17
25. func testSearchForCityAndSave() { //Given let citySearch = "Sofia, Bulgaria" let cityTitle = "Sofia" let application = XCUIApplication() //When let addButton = application.navigationBars["Locations"].buttons["Add"] addButton.tap() let searchForLocationsSearchField = application.navigationBars["SimpleWeather.SearchView"].searchFields["Search for locations"] searchForLocationsSearchField.tap() searchForLocationsSearchField.typeText(citySearch) let searchResult = application.collectionViews.staticTexts["Sofia, Bulgaria"] searchResult.tap() //Then let collectionViewsQuery = application.collectionViews let predicate = NSPredicate(format: "label CONTAINS[c] %@", cityTitle) let elementQuery = collectionViewsQuery.scrollViews.otherElements.staticTexts.containing(predicate) //Assert that there is no city XCTAssertTrue(elementQuery.count == 1) } 18
26. func testSearchForCityAndSave() { //Given let citySearch = "Sofia, Bulgaria" let cityTitle = "Sofia" let application = XCUIApplication() //When let addButton = application.navigationBars["Locations"].buttons["Add"] addButton.tap() let searchForLocationsSearchField = application.navigationBars["SimpleWeather.SearchView"].searchFields["Search for locations"] searchForLocationsSearchField.tap() searchForLocationsSearchField.typeText(citySearch) let searchResult = application.collectionViews.staticTexts["Sofia, Bulgaria"] searchResult.tap() //Then let collectionViewsQuery = application.collectionViews let predicate = NSPredicate(format: "label CONTAINS[c] %@", cityTitle) let elementQuery = collectionViewsQuery.scrollViews.otherElements.staticTexts.containing(predicate) //Assert that there is no city XCTAssertTrue(elementQuery.count == 1) } 18
27. func testSearchForCityAndSave() { //Given let citySearch = "Sofia, Bulgaria" let cityTitle = "Sofia" let application = XCUIApplication() //When let addButton = application.navigationBars["Locations"].buttons["Add"] addButton.tap() let searchForLocationsSearchField = application.navigationBars["SimpleWeather.SearchView"].searchFields["Search for locations"] searchForLocationsSearchField.tap() searchForLocationsSearchField.typeText(citySearch) let searchResult = application.collectionViews.staticTexts["Sofia, Bulgaria"] searchResult.tap() //Then let collectionViewsQuery = application.collectionViews let predicate = NSPredicate(format: "label CONTAINS[c] %@", cityTitle) let elementQuery = collectionViewsQuery.scrollViews.otherElements.staticTexts.containing(predicate) //Assert that there is no city XCTAssertTrue(elementQuery.count == 1) } 18
28. func testSearchForCityAndSave() { //Given let citySearch = "Sofia, Bulgaria" let cityTitle = "Sofia" let application = XCUIApplication() //When let addButton = application.navigationBars["Locations"].buttons["Add"] addButton.tap() let searchForLocationsSearchField = application.navigationBars["SimpleWeather.SearchView"].searchFields["Search for locations"] searchForLocationsSearchField.tap() searchForLocationsSearchField.typeText(citySearch) let searchResult = application.collectionViews.staticTexts["Sofia, Bulgaria"] searchResult.tap() //Then let collectionViewsQuery = application.collectionViews let predicate = NSPredicate(format: "label CONTAINS[c] %@", cityTitle) let elementQuery = collectionViewsQuery.scrollViews.otherElements.staticTexts.containing(predicate) //Assert that there is no city XCTAssertTrue(elementQuery.count == 1) } 18
29. func testSearchForCityAndSave() { //Given let citySearch = "Sofia, Bulgaria" let cityTitle = "Sofia" let application = XCUIApplication() //When let addButton = application.navigationBars["Locations"].buttons["Add"] addButton.tap() let searchForLocationsSearchField = application.navigationBars["SimpleWeather.SearchView"].searchFields["Search for locations"] searchForLocationsSearchField.tap() searchForLocationsSearchField.typeText(citySearch) let searchResult = application.collectionViews.staticTexts["Sofia, Bulgaria"] searchResult.tap() //Then let collectionViewsQuery = application.collectionViews let predicate = NSPredicate(format: "label CONTAINS[c] %@", cityTitle) let elementQuery = collectionViewsQuery.scrollViews.otherElements.staticTexts.containing(predicate) //Assert that there is no city XCTAssertTrue(elementQuery.count == 1) } 18
33. With proper design, the features come cheaply. — Dennis Ritchie 22
34. Pa!erns for writing UI Tests 1. No pattern 2. BDD style 3. Page Objects 4. Robot 23
35. Pa!erns for writing UI Tests 1. No pattern 2. BDD style 3. Page Objects 24
36. Pa!erns for writing UI Tests 1. No pattern 2. BDD style 3. Page Objects 25
37. Pa!erns for writing UI Tests 1. No pattern 2. BDD style 3. Page Objects 26
38. Pa!erns for writing UI Tests 1. No pattern 2. BDD style 3. Page Objects 27
39. Robots 28
44. Constructs complex objects using step-by-step approach — Builder Pattern 33
45. Constructs complex objects using step-by-step approach — Builder Pattern 34
46. Lets build a robot class WeatherListRobot { private var application: XCUIApplication { XCUIApplication() } } 35
47. Lets build a robot class WeatherListRobot { private var application: XCUIApplication { XCUIApplication() } @discardableResult func selectLocation(city: String) -> WeatherDetailsRobot { let collectionViewsQuery = XCUIApplication().collectionViews collectionViewsQuery.scrollViews.otherElements.staticTexts[city].tap() return WeatherDetailsRobot() } } 36
48. class WeatherListRobot { private var application: XCUIApplication { XCUIApplication() } //... @discardableResult func selectSearch() -> WeatherSearchRobot { let addButton = application.navigationBars["Locations"].buttons["Add"] addButton.tap() return WeatherSearchRobot() } //... } 37
49. class WeatherListRobot { private var application: XCUIApplication { XCUIApplication() } //... @discardableResult func deleteLocation(city: String) -> Self { let scrollViewsQuery = XCUIApplication().collectionViews.scrollViews scrollViewsQuery.otherElements.staticTexts[city].swipeRight() scrollViewsQuery.otherElements.staticTexts[city].swipeLeft() XCUIApplication().collectionViews.children(matching: .cell) .element(boundBy: 3) .scrollViews.otherElements .containing(.staticText, identifier:"Delete").element.tap() return self } //... } 38
50. class WeatherListRobot { private var application: XCUIApplication { XCUIApplication() } //... @discardableResult func deleteLocation(city: String) -> Self { let scrollViewsQuery = XCUIApplication().collectionViews.scrollViews scrollViewsQuery.otherElements.staticTexts[city].swipeRight() scrollViewsQuery.otherElements.staticTexts[city].swipeLeft() XCUIApplication().collectionViews.children(matching: .cell) .element(boundBy: 3) .scrollViews.otherElements .containing(.staticText, identifier:"Delete").element.tap() return self } //... } 38
51. class WeatherListRobot { private var application: XCUIApplication { XCUIApplication() } //... @discardableResult func deleteLocation(city: String) -> Self { let scrollViewsQuery = XCUIApplication().collectionViews.scrollViews scrollViewsQuery.otherElements.staticTexts[city].swipeRight() scrollViewsQuery.otherElements.staticTexts[city].swipeLeft() XCUIApplication().collectionViews.children(matching: .cell) .element(boundBy: 3) .scrollViews.otherElements .containing(.staticText, identifier:"Delete").element.tap() return self } //... } 38
52. Lets go back to the tests 39
53. Test 1: Select, Display & Verify func testSelectSavedCityDisplaysCorrectly() { //Given let listRobot = WeatherListRobot(); let verifyDetailsRobot = WeatherDetailsVerifierRobot() //When listRobot.selectLocation(city: "Dublin") .expandWeatherDetails() //Then verifyDetailsRobot.verifyWeatherDetailsExpanded() } 40
54. Test 1: Select, Display & Verify func testSelectSavedCityDisplaysCorrectly() { //Given let listRobot = WeatherListRobot(); let verifyDetailsRobot = WeatherDetailsVerifierRobot() //When listRobot.selectLocation(city: "Dublin") .expandWeatherDetails() //Then verifyDetailsRobot.verifyWeatherDetailsExpanded() } 40
55. Test 1: Select, Display & Verify func testSelectSavedCityDisplaysCorrectly() { //Given let listRobot = WeatherListRobot(); let verifyDetailsRobot = WeatherDetailsVerifierRobot() //When listRobot.selectLocation(city: "Dublin") .expandWeatherDetails() //Then verifyDetailsRobot.verifyWeatherDetailsExpanded() } 40
56. Test 1: Select, Display & Verify func testSelectSavedCityDisplaysCorrectly() { //Given let listRobot = WeatherListRobot(); let verifyDetailsRobot = WeatherDetailsVerifierRobot() //When listRobot.selectLocation(city: "Dublin") .expandWeatherDetails() //Then verifyDetailsRobot.verifyWeatherDetailsExpanded() } 40
57. Test 1: Select, Display & Verify func testSelectSavedCityDisplaysCorrectly() { //Given let listRobot = WeatherListRobot(); let verifyDetailsRobot = WeatherDetailsVerifierRobot() //When listRobot.selectLocation(city: "Dublin") .expandWeatherDetails() //Then verifyDetailsRobot.verifyWeatherDetailsExpanded() } 40
58. Test 2: Delete & Verify func testDeleteSavedCity() { //Given let listRobot = WeatherListRobot(); let verifierRobot = WeatherListVerifiierRobot(); //When listRobot.deleteLocation(city: "Dublin") //Then verifierRobot.verifyLocationGone(city: "Dublin") } 41
59. Test 2: Delete & Verify func testDeleteSavedCity() { //Given let listRobot = WeatherListRobot(); let verifierRobot = WeatherListVerifiierRobot(); //When listRobot.deleteLocation(city: "Dublin") //Then verifierRobot.verifyLocationGone(city: "Dublin") } 41
60. Test 3: Search, Select, Save & Verify func testSearchForCityAndSave() { //Given let listRobot = WeatherListRobot(); //When listRobot.selectSearch() .searchForLocation(query: "Sofia, Bulgaria") .selectLocation(location: "Sofia, Bulgaria") //Then let verifierRobot = WeatherListVerifiierRobot(); verifierRobot.verifyLocationsVisible(cities: "Sofia") } 42
63. Swi! 5 45
64. Let's apply some of Swift 5's language features and see how where go 46
65. Property Delegates Wrappers — Introduced in Swift 5.x — Offers the ability to extend the behaviour of properties — Similar to Java Annotations but solely focused to properties, for now.. 47
66. @propertyWrapper struct ConstructionWrapper<Value: XCUIApplication> { var wrappedValue: Value init() { wrappedValue = Value() } } 48
67. @propertyWrapper struct ConstructionWrapper<Value: XCUIApplication> { var wrappedValue: Value init() { wrappedValue = Value() } } 48
68. @propertyWrapper struct ConstructionWrapper<Value: XCUIApplication> { var wrappedValue: Value init() { wrappedValue = Value() } } 48
69. @propertyWrapper struct ConstructionWrapper<Value: XCUIApplication> { var wrappedValue: Value init() { wrappedValue = Value() } } 48
70. class WeatherListRobot { @ConstructionWrapper private var application: XCUIApplication } 49
71. class WeatherListRobot { @ConstructionWrapper private var application: XCUIApplication } 49
72. Trailing Closures extension EvolvedWeatherDetailsVerifierRobot { convenience init(_ closure: (EvolvedWeatherDetailsVerifierRobot) -> Void) { self.init() closure(self) } } 50
73. Trailing Closures extension EvolvedWeatherDetailsVerifierRobot { convenience init(_ closure: (EvolvedWeatherDetailsVerifierRobot) -> Void) { self.init() closure(self) } } 50
74. Trailing Closures extension EvolvedWeatherDetailsVerifierRobot { convenience init(_ closure: (EvolvedWeatherDetailsVerifierRobot) -> Void) { self.init() closure(self) } } 50
75. Trailing Closures extension EvolvedWeatherDetailsVerifierRobot { convenience init(_ closure: (EvolvedWeatherDetailsVerifierRobot) -> Void) { self.init() closure(self) } } 50
76. Revisit the tests func testSelectSavedCityDisplaysCorrectly() { EvolvedWeatherListRobot { $0.selectLocation(city: "Dublin") .expandWeatherDetails() } EvolvedWeatherDetailsVerifierRobot { $0.verifyWeatherDetailsExpanded() } } 51
77. What makes a good UI test? 1. Easy to read ✅ 2. Easy to write ✅ 3. Easy to extend ✅ 4. Easy to debug ✅ 52
78. Case study: Zendesk Support 1. Universal app (iOS & iPadOS) 2. Predominately written in Swift 3. EarlGrey 2 4. OHHTTPStubs 5. Run on every Pull Request 53
80. Kotlin tests follow same approach @Test fun viewIdShouldSelectView() { Preferences.setLastView("invalid view id") activity.launch() TicketListRobot .verifyViewSelected("Your unsolved tickets") .openViewSelector() .selectView(position = 1) val selectedViewId = Preferences.getLastView() assertEquals(selectedViewId, knownSelectedViewID) } 55
81. Open Source — https://github.com/a1cooke/codemonster2019 — Simple Weather - https://github.com/rnystrom/ SimpleWeather — Testing Robots 56
82. Thank you! Twitter: @a1cooke Github: a1cooke 57