Automated Mobile Acceptance Testing Workshop - mdevcon 2013

Fe6b81005d1553accd6b2a28f6a2bef1?s=47 Pete Hodgson
March 14, 2013
130

Automated Mobile Acceptance Testing Workshop - mdevcon 2013

Fe6b81005d1553accd6b2a28f6a2bef1?s=128

Pete Hodgson

March 14, 2013
Tweet

Transcript

  1. Mobile Acceptance Testing Workshop mdevcon 2013 did you do the

    setup steps? github.com/moredip/mdevcon-workshop Thursday, March 14, 13
  2. Pete Hodgson @ph1 thepete.net phodgson@thoughtworks.com Thursday, March 14, 13

  3. Thursday, March 14, 13

  4. Thursday, March 14, 13

  5. enough about me, what about you? Thursday, March 14, 13

  6. Thursday, March 14, 13

  7. The Plan Thursday, March 14, 13

  8. The Plan some theory, but centered on practical know-how lots

    of opportunities for hands on Thursday, March 14, 13
  9. The Plan focus will be iOS and Frank but almost

    all concepts are generally applicable Thursday, March 14, 13
  10. Thursday, March 14, 13

  11. Agenda the value of testing acceptance testing fundamentals selecting bits

    of UI interacting with bits of UI inspecting bits of UI “advanced stuff” Thursday, March 14, 13
  12. Thursday, March 14, 13

  13. The Goal leave here feeling empowered to write your first

    automated acceptance tests Thursday, March 14, 13
  14. Bonus Goal learn some acceptance testing ProTipstm Thursday, March 14,

    13
  15. interrupt, raise your hand, ask questions Please Thursday, March 14,

    13
  16. Thursday, March 14, 13

  17. Manual Testing Thursday, March 14, 13

  18. launch the app > cd your_app_directory > frank launch LAUNCHING

    IN THE SIMULATOR... Thursday, March 14, 13
  19. manual test script Scenario: adding a new counter - launch

    app - tap + to add a new counter - choose a CountUp counter - enter “My Group” as a group name - enter “My Counter” as a counter name - tap Save - check you are back on the main screen - check you see your group and counter name - check that tapping the counter makes it count Thursday, March 14, 13
  20. manual test script Scenario: adding a new counter - launch

    app - tap + to add a new counter - choose a CountUp counter - enter “My Group” as a group name - enter “My Counter” as a counter name - tap Save - check you are back on the main screen - check you see your group and counter name - check that tapping the counter makes it count again Thursday, March 14, 13
  21. Thursday, March 14, 13

  22. what have we discovered? - testing is a way to

    explore the app - testing can be tedious - humans tend to optimize - humans tend to make mistakes Thursday, March 14, 13
  23. what else? Thursday, March 14, 13

  24. did we find any bugs? Thursday, March 14, 13

  25. what else would you test? Thursday, March 14, 13

  26. Thursday, March 14, 13

  27. running our first automated test Thursday, March 14, 13

  28. running our first automated test > cd your_app_directory > frank

    build > cucumber Frank/features Thursday, March 14, 13
  29. what just happened? Thursday, March 14, 13

  30. iOS app acceptance tests Thursday, March 14, 13

  31. iOS app acceptance tests Frank server Frank driver HTTP test

    scripts Thursday, March 14, 13
  32. iOS app acceptance tests Frank server Frank driver HTTP test

    scripts Thursday, March 14, 13
  33. Thursday, March 14, 13

  34. why do we test? Thursday, March 14, 13

  35. why do we test? interacting with the app to see

    what it does Thursday, March 14, 13
  36. “does it work?” Thursday, March 14, 13

  37. “does it still work?” Thursday, March 14, 13

  38. “what’s it supposed to do anyway?” Thursday, March 14, 13

  39. manual testing is tedious ... Thursday, March 14, 13

  40. and we’re not very good at it ... manual testing

    is tedious ... Thursday, March 14, 13
  41. but computers are ... and we’re not very good at

    it ... manual testing is tedious ... Thursday, March 14, 13
  42. “Computers are designed to do simple repetitive tasks. The second

    you have humans doing repetitive tasks, all the computers get together late at night and laugh at you…” - Neal Ford Thursday, March 14, 13
  43. Thursday, March 14, 13

  44. why automate your tests Thursday, March 14, 13

  45. why automate your tests - more time for exploratory QA

    Thursday, March 14, 13
  46. why automate your tests - more time for exploratory QA

    - consistency in our testing Thursday, March 14, 13
  47. why automate your tests - more time for exploratory QA

    - consistency in our testing - drives focus on requirements Thursday, March 14, 13
  48. why automate your tests - more time for exploratory QA

    - consistency in our testing - drives focus on requirements - protect against regressions Thursday, March 14, 13
  49. why automate your tests - more time for exploratory QA

    - consistency in our testing - drives focus on requirements - protect against regressions - enable courageous change Thursday, March 14, 13
  50. why automate your tests - more time for exploratory QA

    - consistency in our testing - drives focus on requirements - protect against regressions - enable courageous change - what else? Thursday, March 14, 13
  51. Thursday, March 14, 13

  52. types of automated testing Thursday, March 14, 13

  53. Mobile App Thursday, March 14, 13

  54. Mobile App End User Thursday, March 14, 13

  55. Mobile App End User Backend Services Thursday, March 14, 13

  56. Unit Test Mobile App End User Backend Services Thursday, March

    14, 13
  57. Integration Test Mobile App End User Backend Services Thursday, March

    14, 13
  58. Acceptance Test Mobile App End User Backend Services Thursday, March

    14, 13
  59. Acceptance Test Mobile App End User Backend Services The App

    Thursday, March 14, 13
  60. Acceptance Unit Integration Thursday, March 14, 13

  61. a few reliable tests are better than lots of flakey

    tests ProTiptm Thursday, March 14, 13
  62. run your tests as frequently as you can ProTiptm Thursday,

    March 14, 13
  63. Thursday, March 14, 13

  64. the UI testing triad Thursday, March 14, 13

  65. select a bit o UI Thursday, March 14, 13

  66. select a bit o UI simulate interaction with it inspect

    state o it Thursday, March 14, 13
  67. UI select interact inspect Thursday, March 14, 13

  68. UI select interact inspect Thursday, March 14, 13

  69. symbiote > cd your_app_directory > frank launch > frank inspect

    Thursday, March 14, 13
  70. symbiote Thursday, March 14, 13

  71. demo! Thursday, March 14, 13

  72. Thursday, March 14, 13

  73. view selectors Thursday, March 14, 13

  74. view selectors - a little language to specify which views

    to inspect/ interact with - like CSS or XPath Thursday, March 14, 13
  75. Thursday, March 14, 13

  76. Thursday, March 14, 13

  77. Thursday, March 14, 13

  78. view marked:‘A Group’ by accessibility label Thursday, March 14, 13

  79. accessibility labels are your friend ProTiptm Thursday, March 14, 13

  80. view marked:‘A Group’ by accessibility label Thursday, March 14, 13

  81. tableViewCell by view class Thursday, March 14, 13

  82. tableViewCell label drill down Thursday, March 14, 13

  83. tableViewCell label drill down Thursday, March 14, 13

  84. tableViewCell label drill down Thursday, March 14, 13

  85. putting it together Thursday, March 14, 13

  86. tableViewCell label marked:’A Counter’ putting it together Thursday, March 14,

    13
  87. avoid brittle view selection ProTiptm Thursday, March 14, 13

  88. record & playback: not a sustainable approach ProTiptm Thursday, March

    14, 13
  89. Thursday, March 14, 13

  90. UI select interact inspect Thursday, March 14, 13

  91. UI select interact inspect Thursday, March 14, 13

  92. user simulation touch(...) type_into_keyboard(...) set_orientation(...) double_tap(...) tap_and_hold(...) drag_with_initial_delay(...) Thursday, March

    14, 13
  93. touch(...) Thursday, March 14, 13

  94. touch(...) view selector Thursday, March 14, 13

  95. frank console > cd your_app_directory > frank launch > frank

    console connecting to app... connected [1] pry(#<Frank::Console>)> Thursday, March 14, 13
  96. frank console > touch “button marked:‘Add’” Thursday, March 14, 13

  97. UI select interact inspect Thursday, March 14, 13

  98. UI select interact inspect Thursday, March 14, 13

  99. view introspection frankly_map(view_selector, method) runs an ObjectiveC method on all

    matching UIView instances Thursday, March 14, 13
  100. back in the console Thursday, March 14, 13

  101. back in the console > selector = “button marked:‘Edit’” >

    element_exists( selector ) => true > frankly_map( selector, ‘isEnabled’ ) => [true] > frankly_map( selector, ‘isHidden’ ) => [false] > frankly_map( selector, ‘accessibilityFrame’ ) => [{"size"=>{"width"=>50, "height"=>30}, "origin"=>{"x"=>5, "y"=>27}}] Thursday, March 14, 13
  102. interact inspect select UI Thursday, March 14, 13

  103. putting it all together Thursday, March 14, 13

  104. back in the console > touch “view marked:‘Add’” > touch

    “label marked:'CountUp'” > touch “view:'UITextFieldLabel' marked:'Group Name'” > type_into_keyboard “My Group” > touch “view:'UITextFieldLabel' marked:'Counter Name'” > type_into_keyboard “My Counter” > touch “button marked:'Save'” > element_exists “view marked:’My Group’” > element_exists “view marked:’My Counter’” > touch “view marked:’My Counter’” Thursday, March 14, 13
  105. Thursday, March 14, 13

  106. other platforms Thursday, March 14, 13

  107. Android Robotium Calabash Thursday, March 14, 13

  108. Mobile Web WebDriver (aka Selenium 2) with Capybara (if you’re

    using ruby) Thursday, March 14, 13
  109. which WebDriver? Thursday, March 14, 13

  110. which WebDriver? iPhoneDriver ~ or ~ a desktop WebDriver (e.g.

    chromedriver) Thursday, March 14, 13
  111. Headless capybara-webkit poltergeist ghostdriver htmlunitdriver Thursday, March 14, 13

  112. Thursday, March 14, 13

  113. cucumber Thursday, March 14, 13

  114. cucumber write tests which describe behavior in terms of user

    value Thursday, March 14, 13
  115. cucumber Given... When... Then... Thursday, March 14, 13

  116. cucumber Given I have a simple counter And I have

    counted to 10 When I reset the counter Then the counter should be at 0 Thursday, March 14, 13
  117. How does that work? Thursday, March 14, 13

  118. How does that work? Regular Expressions (!!!) Thursday, March 14,

    13
  119. Cucumber steps Given I have a simple counter And I

    have counted to 10 When I reset the counter Then the counter should be at 0 Thursday, March 14, 13
  120. Thursday, March 14, 13

  121. Cucumber step definitions Thursday, March 14, 13

  122. Cucumber step definitions When /^I reset the counter$/ do touch(

    “view marked:‘Zero Counts’” ) end Thursday, March 14, 13
  123. Cucumber step definitions When /^I reset the counter$/ do touch(

    “view marked:‘Zero Counts’” ) end Thursday, March 14, 13
  124. Cucumber step definitions When /^I reset the counter$/ do touch(

    “view marked:‘Zero Counts’” ) end Thursday, March 14, 13
  125. so, what does Cucumber do for us? Thursday, March 14,

    13
  126. ... ... touch “button marked:'CountUp'” touch “view marked:'Group Name'” type_into_keyboard

    “my group” touch “view marked:'Counter Name'” type_into_keyboard “my counter” touch “button marked:‘Save’” ... ... Thursday, March 14, 13
  127. When I create a “my counter” counter Thursday, March 14,

    13
  128. When /^I create a "(.*?)" counter$/ do |counter| touch “button

    marked:'CountUp'” touch “view marked:'Group Name'” type_into_keyboard DEFAULT_GROUP_NAME touch “view marked:'Counter Name'” type_into_keyboard counter touch “button marked:‘Save’” end Thursday, March 14, 13
  129. When /^I create a "(.*?)" counter$/ do |counter| touch “button

    marked:'CountUp'” touch “view marked:'Group Name'” type_into_keyboard DEFAULT_GROUP_NAME touch “view marked:'Counter Name'” type_into_keyboard counter touch “button marked:‘Save’” end Thursday, March 14, 13
  130. When /^I create a "(.*?)" counter$/ do |counter| touch “button

    marked:'CountUp'” touch “view marked:'Group Name'” type_into_keyboard DEFAULT_GROUP_NAME touch “view marked:'Counter Name'” type_into_keyboard counter touch “button marked:‘Save’” end Thursday, March 14, 13
  131. When /^I create a "(.*?)" counter$/ do |counter| touch “button

    marked:'CountUp'” touch “view marked:'Group Name'” type_into_keyboard DEFAULT_GROUP_NAME touch “view marked:'Counter Name'” type_into_keyboard counter touch “button marked:‘Save’” end Thursday, March 14, 13
  132. “foo” When I create a counter Thursday, March 14, 13

  133. “bar” When I create a counter Thursday, March 14, 13

  134. “baz” When I create a counter Thursday, March 14, 13

  135. Don’t write test scripts in cucumber ProTiptm Thursday, March 14,

    13
  136. Cucumber should be high-level and user-focussed ProTiptm Thursday, March 14,

    13
  137. embrace patterns like Page Object ProTiptm Thursday, March 14, 13

  138. Thursday, March 14, 13

  139. race conditions Thursday, March 14, 13

  140.      Thursday, March 14, 13

  141.    Thursday, March 14, 13

  142.      Thursday, March 14, 13

  143. did we log in? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome

    back!’” Thursday, March 14, 13
  144. did we log in? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome

    back!’” Thursday, March 14, 13
  145. did we log in? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome

    back!’” Thursday, March 14, 13
  146. did we log in? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome

    back!’” Thursday, March 14, 13
  147. what about the network?      touch

    “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’” Thursday, March 14, 13
  148. what about the network?      touch

    “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’” Thursday, March 14, 13
  149. what about the network?    touch “button marked:‘Login’”

    check_element_exists “view marked:‘Welcome back!’” Thursday, March 14, 13
  150. what about the network?    touch “button marked:‘Login’”

    check_element_exists “view marked:‘Welcome back!’” Thursday, March 14, 13
  151. what about the network?    touch “button marked:‘Login’”

    check_element_exists “view marked:‘Welcome back!’” fail Thursday, March 14, 13
  152. what about the network?      touch

    “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’” fail Thursday, March 14, 13
  153. a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view

    marked:‘Welcome back!’” Thursday, March 14, 13
  154. a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view

    marked:‘Welcome back!’” Thursday, March 14, 13
  155. a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view

    marked:‘Welcome back!’” PROBLEM slow tests Thursday, March 14, 13
  156. a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view

    marked:‘Welcome back!’” PROBLEM slow tests PROBLEM fragile tests Thursday, March 14, 13
  157. wait_until SOLUTION: Thursday, March 14, 13

  158. wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’”

    end Thursday, March 14, 13
  159. wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’”

    end Thursday, March 14, 13
  160. wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’”

    end Thursday, March 14, 13
  161. wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’”

    end Thursday, March 14, 13
  162. wait_until touch “button marked:‘Login’” wait_for_element_to_exist do “view marked:‘Welcome back!’” end

    Thursday, March 14, 13
  163. wait_until touch “button marked:‘Login’” wait_for_element_to_exist do “view marked:‘Welcome back!’” end

    Thursday, March 14, 13
  164. Spin Assert ProTiptm Thursday, March 14, 13

  165. Thursday, March 14, 13

  166. Thursday, March 14, 13

  167. wrapping up the value of testing (feedback) definition of an

    acceptance test selecting/interacting/inspecting frank console and symbiote “advanced stuff” Thursday, March 14, 13
  168. this slide deck will be at: http://slides.thepete.net Thursday, March 14,

    13
  169. further research - ‘Page Object’ pattern - Spin Asserts -

    Quarantine broken tests - acceptance testing Journeys - Continuous Integration - faking out external services Thursday, March 14, 13
  170. helpful resources testingwithfrank.com the cucumber book the rspec book scripting

    w. Ruby book specification by example book rubymine (good IDE for ruby/cuke dev.) eradicating non- determinism in tests (good blog post) creating maintainable acceptance tests (presentation slides and video) the example tests in the Frank repo Thursday, March 14, 13
  171. more questions? @ph1 http://blog.thepete.net phodgson@thoughtworks.com me, at the bar feedback

    plz! http://rate.thepete.net Thursday, March 14, 13