Beyond UIAutomation

Slides from my CocoaConf PDX talk.

I covered some of the reasons that automated tests are valuable. I described some common challenges when doing automated UI tests, along with some solutions. I described why UIAutomation doesn't let us use all these solutions, and why we should look beyond UIAutomation.

Transcript

1. Beyond UIAutomation
2. Pete Hodgson @ph1 http://blog.thepete.net phodgson@thoughtworks.com
4. we’re hiring
6. enough about me
7. Topics - why do automated testing - challenges and solutions - UIAutomation and beyond - hands on with Frank
8. interrupt, ask questions, generally heckle Please
9. why do we test?
10. why do we test? interacting with the app to see what it does
11. “does it work?”
12. “does it still work?”
13. “what’s it supposed to do anyway?”
14. but computers are ... and we’re not very good at it ... manual testing is tedious ...
15. “Computers are designed to do simple repetitive tasks. The second you have humans doing repetitive tasks, all the computers get together late at night and laugh at you…” - Neal Ford
16. ... so automate your UI testing* * within reason
17. UIAutomation: Apple’s solution
18. UIAutomation - runs inside Instruments - write test scripts in javascript - drives simulator or device - lives outside of the wider test automation ecosystem
20. DEMO!
21. UI testing: challenges and solutions
22. a basic test script touch “textField marked:‘login’” type_into_keyboard “testuser” touch “textField marked:‘password’” type_into_keyboard ‘testpassword’ touch “button marked:‘Login’”
23. a basic test script touch “textField marked:‘login’” type_into_keyboard “testuser” touch “textField marked:‘password’” type_into_keyboard ‘testpassword’ touch “button marked:‘Login’”
24. a basic test script touch “textField marked:‘login’” type_into_keyboard “testuser” touch “textField marked:‘password’” type_into_keyboard ‘testpassword’ touch “button marked:‘Login’” PROBLEM! no abstractions
25. cucumber SOLUTION:
26. writes tests which describe behavior in terms of user value
27. Given, When, Then Scenario: prompted to log in Given I am on the home screen But I am not logged in When I tap the Friends button Then I should be on the log in screen
28. how does that magic work?
29. regular expressions* * oh the humanity!
30. Given I am on the home screen But I am not logged in When I tap the Friends button Then I should be on the log in screen Cucumber steps
31. Cucumber step definitions
32. Cucumber step definitions When /^I tap the Friends button$/ do touch( “button marked:‘Friends’” ) end
33. Cucumber step definitions When /^I tap the Friends button$/ do touch( “button marked:‘Friends’” ) end
34. Cucumber step definitions When /^I tap the Friends button$/ do touch( “button marked:‘Friends’” ) end
35. so, what does Cucumber do for us?
36. ... ... touch “textField marked:‘login’” type_into_keyboard “testuser” touch “textField marked:‘password’” type_into_keyboard ‘testpassword’ touch “button marked:‘Login’” ... ...
37. When I log in as “testuser”
38. When /^I log in as "(.*?)"$/ do |user| password = password_for_user(user) touch “textField marked:‘login’” type_into_keyboard user touch “textField marked:‘password’” type_into_keyboard password touch “button marked:‘Login’” end
39. When /^I log in as "(.*?)"$/ do |user| password = password_for_user(user) touch “textField marked:‘login’” type_into_keyboard user touch “textField marked:‘password’” type_into_keyboard password touch “button marked:‘Login’” end
40. When /^I log in as "(.*?)"$/ do |user| password = password_for_user(user) touch “textField marked:‘login’” type_into_keyboard user touch “textField marked:‘password’” type_into_keyboard password touch “button marked:‘Login’” end
41. When /^I log in as "(.*?)"$/ do |user| password = password_for_user(user) touch “textField marked:‘login’” type_into_keyboard user touch “textField marked:‘password’” type_into_keyboard password touch “button marked:‘Login’” end
42. When I log in as “tom”
43. When I log in as “dick”
44. When I log in as “harry”
45. cucumber isn’t the only option
46. selecting views
49. select the view labeled ‘Shopping List’
50. select the view labeled ‘Shopping List’
51. select the view labeled ‘Shopping List’
52. select the view labeled ‘Shopping List’ PROBLEM! too generic
53. select the view labeled ‘Shopping List’ SOLUTION: get specific
54. select the table cell labeled ‘Shopping List’ SOLUTION: get specific
56. select the table cell labeled “Ham”?
57. select the table cell labeled “Ham”? select the check box labeled “Ham”?
58. select the table cell labeled “Ham”? select the check box labeled “Ham”? select the 2nd check box?
59. PROBLEM! brittle tests select the table cell labeled “Ham”? select the check box labeled “Ham”? select the 2nd check box?
60. SOLUTION
61. SOLUTION
62. SOLUTION
63. SOLUTION
64. tableViewCell label marked:‘Ham’ parent tableViewCell button SOLUTION view selector
65. view selectors
66. view selectors button marked:‘Login’ tableView tableViewCell label view:‘MyCustomWidget’ textField
67. symbiote
69. race conditions fun with
70.     
71.   
72.  
     
73. did we log in? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”
74. did we log in? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”
75. did we log in? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”
76. what about the network?      touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”
77. what about the network?      touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”
78. what about the network?    touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”
79. what about the network?    touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”
80. what about the network?    touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’” fail
81. what about the network?  
      touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’” fail
82. a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view marked:‘Welcome back!’”
83. a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view marked:‘Welcome back!’”
84. a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view marked:‘Welcome back!’” PROBLEM slow tests
85. a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view marked:‘Welcome back!’” PROBLEM slow tests PROBLEM fragile tests
86. wait_until
87. wait_until SOLUTION:
88. wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’” end
89. wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’” end
90. wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’” end
91. wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’” end
92. wait_until touch “button marked:‘Login’” wait_for_element_to_exist do “view marked:‘Welcome back!’” end
93. wait_until touch “button marked:‘Login’” wait_for_element_to_exist do “view marked:‘Welcome back!’” end
94. those pesky services
96. services
97. services
98. tests services
99. services
100. loads of stuff services
101. loads of stuff PROBLEM slow services
102. loads of stuff PROBLEM slow PROBLEM flaky services
103. services SOLUTION:
104. services FAKE services Mustache designed by Dmitriy Lagunov from The Noun Project SOLUTION:
105. services FAKE services Mustache designed by Dmitriy Lagunov from The Noun Project SOLUTION: test-only back-channel
106. less stuff involved
107. when to do UI-based testing
108. UI Unit Service
109. UI Unit Service focused full-stack
110. the value in automated testing
111. feedback automated tests give us
112. protection from the worst kind of bugs ...
113. when cause and effect are separated by both space and time
114. confidence automated tests give us
115. Continuous Integration use to maximize the value
116. why go beyond UIAutomation
117. why go beyond UIAutomation automated tests are really valuable ... ... but tricky to get right
118. why go beyond UIAutomation lots of well-established solutions to these challenges ... ... but UIAutomation falls short.
119. there is plenty of good stuff beyond UIAutomation
120. UISpec NativeDriver KIF Bwoken Zucchini Frank Calabash tuneup.js FoneMonkey Telerik etc... etc... beyond UIAutomation
121. thanks. questions?
122. some resources www.testingwithfrank.com phodgson@thoughtworks.com @ph1 blog.thepete.net me, at the bar