Beyond UIAutomation

Fe6b81005d1553accd6b2a28f6a2bef1?s=47 Pete Hodgson
October 27, 2012

Beyond UIAutomation

Slides from my CocoaConf PDX talk.

I covered some of the reasons that automated tests are valuable. I described some common challenges when doing automated UI tests, along with some solutions. I described why UIAutomation doesn't let us use all these solutions, and why we should look beyond UIAutomation.

Fe6b81005d1553accd6b2a28f6a2bef1?s=128

Pete Hodgson

October 27, 2012
Tweet

Transcript

  1. Beyond UIAutomation

  2. Pete Hodgson @ph1 http://blog.thepete.net phodgson@thoughtworks.com

  3. None
  4. we’re hiring

  5. None
  6. enough about me

  7. Topics - why do automated testing - challenges and solutions

    - UIAutomation and beyond - hands on with Frank
  8. interrupt, ask questions, generally heckle Please

  9. why do we test?

  10. why do we test? interacting with the app to see

    what it does
  11. “does it work?”

  12. “does it still work?”

  13. “what’s it supposed to do anyway?”

  14. but computers are ... and we’re not very good at

    it ... manual testing is tedious ...
  15. “Computers are designed to do simple repetitive tasks. The second

    you have humans doing repetitive tasks, all the computers get together late at night and laugh at you…” - Neal Ford
  16. ... so automate your UI testing* * within reason

  17. UIAutomation: Apple’s solution

  18. UIAutomation - runs inside Instruments - write test scripts in

    javascript - drives simulator or device - lives outside of the wider test automation ecosystem
  19. None
  20. DEMO!

  21. UI testing: challenges and solutions

  22. a basic test script touch “textField marked:‘login’” type_into_keyboard “testuser” touch

    “textField marked:‘password’” type_into_keyboard ‘testpassword’ touch “button marked:‘Login’”
  23. a basic test script touch “textField marked:‘login’” type_into_keyboard “testuser” touch

    “textField marked:‘password’” type_into_keyboard ‘testpassword’ touch “button marked:‘Login’”
  24. a basic test script touch “textField marked:‘login’” type_into_keyboard “testuser” touch

    “textField marked:‘password’” type_into_keyboard ‘testpassword’ touch “button marked:‘Login’” PROBLEM! no abstractions
  25. cucumber SOLUTION:

  26. writes tests which describe behavior in terms of user value

  27. Given, When, Then Scenario: prompted to log in Given I

    am on the home screen But I am not logged in When I tap the Friends button Then I should be on the log in screen
  28. how does that magic work?

  29. regular expressions* * oh the humanity!

  30. Given I am on the home screen But I am

    not logged in When I tap the Friends button Then I should be on the log in screen Cucumber steps
  31. Cucumber step definitions

  32. Cucumber step definitions When /^I tap the Friends button$/ do

    touch( “button marked:‘Friends’” ) end
  33. Cucumber step definitions When /^I tap the Friends button$/ do

    touch( “button marked:‘Friends’” ) end
  34. Cucumber step definitions When /^I tap the Friends button$/ do

    touch( “button marked:‘Friends’” ) end
  35. so, what does Cucumber do for us?

  36. ... ... touch “textField marked:‘login’” type_into_keyboard “testuser” touch “textField marked:‘password’”

    type_into_keyboard ‘testpassword’ touch “button marked:‘Login’” ... ...
  37. When I log in as “testuser”

  38. When /^I log in as "(.*?)"$/ do |user| password =

    password_for_user(user) touch “textField marked:‘login’” type_into_keyboard user touch “textField marked:‘password’” type_into_keyboard password touch “button marked:‘Login’” end
  39. When /^I log in as "(.*?)"$/ do |user| password =

    password_for_user(user) touch “textField marked:‘login’” type_into_keyboard user touch “textField marked:‘password’” type_into_keyboard password touch “button marked:‘Login’” end
  40. When /^I log in as "(.*?)"$/ do |user| password =

    password_for_user(user) touch “textField marked:‘login’” type_into_keyboard user touch “textField marked:‘password’” type_into_keyboard password touch “button marked:‘Login’” end
  41. When /^I log in as "(.*?)"$/ do |user| password =

    password_for_user(user) touch “textField marked:‘login’” type_into_keyboard user touch “textField marked:‘password’” type_into_keyboard password touch “button marked:‘Login’” end
  42. When I log in as “tom”

  43. When I log in as “dick”

  44. When I log in as “harry”

  45. cucumber isn’t the only option

  46. selecting views

  47. None
  48. None
  49. select the view labeled ‘Shopping List’

  50. select the view labeled ‘Shopping List’

  51. select the view labeled ‘Shopping List’

  52. select the view labeled ‘Shopping List’ PROBLEM! too generic

  53. select the view labeled ‘Shopping List’ SOLUTION: get specific

  54. select the table cell labeled ‘Shopping List’ SOLUTION: get specific

  55. None
  56. select the table cell labeled “Ham”?

  57. select the table cell labeled “Ham”? select the check box

    labeled “Ham”?
  58. select the table cell labeled “Ham”? select the check box

    labeled “Ham”? select the 2nd check box?
  59. PROBLEM! brittle tests select the table cell labeled “Ham”? select

    the check box labeled “Ham”? select the 2nd check box?
  60. SOLUTION

  61. SOLUTION

  62. SOLUTION

  63. SOLUTION

  64. tableViewCell label marked:‘Ham’ parent tableViewCell button SOLUTION view selector

  65. view selectors

  66. view selectors button marked:‘Login’ tableView tableViewCell label view:‘MyCustomWidget’ textField

  67. symbiote

  68. None
  69. race conditions fun with

  70.     

  71.   

  72.     

  73. did we log in? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome

    back!’”
  74. did we log in? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome

    back!’”
  75. did we log in? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome

    back!’”
  76. what about the network?      touch

    “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”
  77. what about the network?      touch

    “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”
  78. what about the network?    touch “button marked:‘Login’”

    check_element_exists “view marked:‘Welcome back!’”
  79. what about the network?    touch “button marked:‘Login’”

    check_element_exists “view marked:‘Welcome back!’”
  80. what about the network?    touch “button marked:‘Login’”

    check_element_exists “view marked:‘Welcome back!’” fail
  81. what about the network?      touch

    “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’” fail
  82. a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view

    marked:‘Welcome back!’”
  83. a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view

    marked:‘Welcome back!’”
  84. a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view

    marked:‘Welcome back!’” PROBLEM slow tests
  85. a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view

    marked:‘Welcome back!’” PROBLEM slow tests PROBLEM fragile tests
  86. wait_until

  87. wait_until SOLUTION:

  88. wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’”

    end
  89. wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’”

    end
  90. wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’”

    end
  91. wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’”

    end
  92. wait_until touch “button marked:‘Login’” wait_for_element_to_exist do “view marked:‘Welcome back!’” end

  93. wait_until touch “button marked:‘Login’” wait_for_element_to_exist do “view marked:‘Welcome back!’” end

  94. those pesky services

  95. None
  96. services

  97. services

  98. tests services

  99. services

  100. loads of stuff services

  101. loads of stuff PROBLEM slow services

  102. loads of stuff PROBLEM slow PROBLEM flaky services

  103. services SOLUTION:

  104. services FAKE services Mustache designed by Dmitriy Lagunov from The

    Noun Project SOLUTION:
  105. services FAKE services Mustache designed by Dmitriy Lagunov from The

    Noun Project SOLUTION: test-only back-channel
  106. less stuff involved

  107. when to do UI-based testing

  108. UI Unit Service

  109. UI Unit Service focused full-stack

  110. the value in automated testing

  111. feedback automated tests give us

  112. protection from the worst kind of bugs ...

  113. when cause and effect are separated by both space and

    time
  114. confidence automated tests give us

  115. Continuous Integration use to maximize the value

  116. why go beyond UIAutomation

  117. why go beyond UIAutomation automated tests are really valuable ...

    ... but tricky to get right
  118. why go beyond UIAutomation lots of well-established solutions to these

    challenges ... ... but UIAutomation falls short.
  119. there is plenty of good stuff beyond UIAutomation

  120. UISpec NativeDriver KIF Bwoken Zucchini Frank Calabash tuneup.js FoneMonkey Telerik

    etc... etc... beyond UIAutomation
  121. thanks. questions?

  122. some resources www.testingwithfrank.com phodgson@thoughtworks.com @ph1 blog.thepete.net me, at the bar