Slide 1

Slide 1 text

Beyond UIAutomation

Slide 2

Slide 2 text

Pete Hodgson @ph1 http://blog.thepete.net [email protected]

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

we’re hiring

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

enough about me

Slide 7

Slide 7 text

Topics - why do automated testing - challenges and solutions - UIAutomation and beyond - hands on with Frank

Slide 8

Slide 8 text

interrupt, ask questions, generally heckle Please

Slide 9

Slide 9 text

why do we test?

Slide 10

Slide 10 text

why do we test? interacting with the app to see what it does

Slide 11

Slide 11 text

“does it work?”

Slide 12

Slide 12 text

“does it still work?”

Slide 13

Slide 13 text

“what’s it supposed to do anyway?”

Slide 14

Slide 14 text

but computers are ... and we’re not very good at it ... manual testing is tedious ...

Slide 15

Slide 15 text

“Computers are designed to do simple repetitive tasks. The second you have humans doing repetitive tasks, all the computers get together late at night and laugh at you…” - Neal Ford

Slide 16

Slide 16 text

... so automate your UI testing* * within reason

Slide 17

Slide 17 text

UIAutomation: Apple’s solution

Slide 18

Slide 18 text

UIAutomation - runs inside Instruments - write test scripts in javascript - drives simulator or device - lives outside of the wider test automation ecosystem

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

DEMO!

Slide 21

Slide 21 text

UI testing: challenges and solutions

Slide 22

Slide 22 text

a basic test script touch “textField marked:‘login’” type_into_keyboard “testuser” touch “textField marked:‘password’” type_into_keyboard ‘testpassword’ touch “button marked:‘Login’”

Slide 23

Slide 23 text

a basic test script touch “textField marked:‘login’” type_into_keyboard “testuser” touch “textField marked:‘password’” type_into_keyboard ‘testpassword’ touch “button marked:‘Login’”

Slide 24

Slide 24 text

a basic test script touch “textField marked:‘login’” type_into_keyboard “testuser” touch “textField marked:‘password’” type_into_keyboard ‘testpassword’ touch “button marked:‘Login’” PROBLEM! no abstractions

Slide 25

Slide 25 text

cucumber SOLUTION:

Slide 26

Slide 26 text

writes tests which describe behavior in terms of user value

Slide 27

Slide 27 text

Given, When, Then Scenario: prompted to log in Given I am on the home screen But I am not logged in When I tap the Friends button Then I should be on the log in screen

Slide 28

Slide 28 text

how does that magic work?

Slide 29

Slide 29 text

regular expressions* * oh the humanity!

Slide 30

Slide 30 text

Given I am on the home screen But I am not logged in When I tap the Friends button Then I should be on the log in screen Cucumber steps

Slide 31

Slide 31 text

Cucumber step definitions

Slide 32

Slide 32 text

Cucumber step definitions When /^I tap the Friends button$/ do touch( “button marked:‘Friends’” ) end

Slide 33

Slide 33 text

Cucumber step definitions When /^I tap the Friends button$/ do touch( “button marked:‘Friends’” ) end

Slide 34

Slide 34 text

Cucumber step definitions When /^I tap the Friends button$/ do touch( “button marked:‘Friends’” ) end

Slide 35

Slide 35 text

so, what does Cucumber do for us?

Slide 36

Slide 36 text

... ... touch “textField marked:‘login’” type_into_keyboard “testuser” touch “textField marked:‘password’” type_into_keyboard ‘testpassword’ touch “button marked:‘Login’” ... ...

Slide 37

Slide 37 text

When I log in as “testuser”

Slide 38

Slide 38 text

When /^I log in as "(.*?)"$/ do |user| password = password_for_user(user) touch “textField marked:‘login’” type_into_keyboard user touch “textField marked:‘password’” type_into_keyboard password touch “button marked:‘Login’” end

Slide 39

Slide 39 text

When /^I log in as "(.*?)"$/ do |user| password = password_for_user(user) touch “textField marked:‘login’” type_into_keyboard user touch “textField marked:‘password’” type_into_keyboard password touch “button marked:‘Login’” end

Slide 40

Slide 40 text

When /^I log in as "(.*?)"$/ do |user| password = password_for_user(user) touch “textField marked:‘login’” type_into_keyboard user touch “textField marked:‘password’” type_into_keyboard password touch “button marked:‘Login’” end

Slide 41

Slide 41 text

When /^I log in as "(.*?)"$/ do |user| password = password_for_user(user) touch “textField marked:‘login’” type_into_keyboard user touch “textField marked:‘password’” type_into_keyboard password touch “button marked:‘Login’” end

Slide 42

Slide 42 text

When I log in as “tom”

Slide 43

Slide 43 text

When I log in as “dick”

Slide 44

Slide 44 text

When I log in as “harry”

Slide 45

Slide 45 text

cucumber isn’t the only option

Slide 46

Slide 46 text

selecting views

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

select the view labeled ‘Shopping List’

Slide 50

Slide 50 text

select the view labeled ‘Shopping List’

Slide 51

Slide 51 text

select the view labeled ‘Shopping List’

Slide 52

Slide 52 text

select the view labeled ‘Shopping List’ PROBLEM! too generic

Slide 53

Slide 53 text

select the view labeled ‘Shopping List’ SOLUTION: get specific

Slide 54

Slide 54 text

select the table cell labeled ‘Shopping List’ SOLUTION: get specific

Slide 55

Slide 55 text

No content

Slide 56

Slide 56 text

select the table cell labeled “Ham”?

Slide 57

Slide 57 text

select the table cell labeled “Ham”? select the check box labeled “Ham”?

Slide 58

Slide 58 text

select the table cell labeled “Ham”? select the check box labeled “Ham”? select the 2nd check box?

Slide 59

Slide 59 text

PROBLEM! brittle tests select the table cell labeled “Ham”? select the check box labeled “Ham”? select the 2nd check box?

Slide 60

Slide 60 text

SOLUTION

Slide 61

Slide 61 text

SOLUTION

Slide 62

Slide 62 text

SOLUTION

Slide 63

Slide 63 text

SOLUTION

Slide 64

Slide 64 text

tableViewCell label marked:‘Ham’ parent tableViewCell button SOLUTION view selector

Slide 65

Slide 65 text

view selectors

Slide 66

Slide 66 text

view selectors button marked:‘Login’ tableView tableViewCell label view:‘MyCustomWidget’ textField

Slide 67

Slide 67 text

symbiote

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

race conditions fun with

Slide 70

Slide 70 text

Slide 71

Slide 71 text

Slide 72

Slide 72 text

Slide 73

Slide 73 text

did we log in? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”

Slide 74

Slide 74 text

did we log in? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”

Slide 75

Slide 75 text

did we log in? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”

Slide 76

Slide 76 text

what about the network? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”

Slide 77

Slide 77 text

what about the network? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”

Slide 78

Slide 78 text

what about the network? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”

Slide 79

Slide 79 text

what about the network? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’”

Slide 80

Slide 80 text

what about the network? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’” fail

Slide 81

Slide 81 text

what about the network? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’” fail

Slide 82

Slide 82 text

a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view marked:‘Welcome back!’”

Slide 83

Slide 83 text

a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view marked:‘Welcome back!’”

Slide 84

Slide 84 text

a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view marked:‘Welcome back!’” PROBLEM slow tests

Slide 85

Slide 85 text

a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view marked:‘Welcome back!’” PROBLEM slow tests PROBLEM fragile tests

Slide 86

Slide 86 text

wait_until

Slide 87

Slide 87 text

wait_until SOLUTION:

Slide 88

Slide 88 text

wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’” end

Slide 89

Slide 89 text

wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’” end

Slide 90

Slide 90 text

wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’” end

Slide 91

Slide 91 text

wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’” end

Slide 92

Slide 92 text

wait_until touch “button marked:‘Login’” wait_for_element_to_exist do “view marked:‘Welcome back!’” end

Slide 93

Slide 93 text

wait_until touch “button marked:‘Login’” wait_for_element_to_exist do “view marked:‘Welcome back!’” end

Slide 94

Slide 94 text

those pesky services

Slide 95

Slide 95 text

No content

Slide 96

Slide 96 text

services

Slide 97

Slide 97 text

services

Slide 98

Slide 98 text

tests services

Slide 99

Slide 99 text

services

Slide 100

Slide 100 text

loads of stuff services

Slide 101

Slide 101 text

loads of stuff PROBLEM slow services

Slide 102

Slide 102 text

loads of stuff PROBLEM slow PROBLEM flaky services

Slide 103

Slide 103 text

services SOLUTION:

Slide 104

Slide 104 text

services FAKE services Mustache designed by Dmitriy Lagunov from The Noun Project SOLUTION:

Slide 105

Slide 105 text

services FAKE services Mustache designed by Dmitriy Lagunov from The Noun Project SOLUTION: test-only back-channel

Slide 106

Slide 106 text

less stuff involved

Slide 107

Slide 107 text

when to do UI-based testing

Slide 108

Slide 108 text

UI Unit Service

Slide 109

Slide 109 text

UI Unit Service focused full-stack

Slide 110

Slide 110 text

the value in automated testing

Slide 111

Slide 111 text

feedback automated tests give us

Slide 112

Slide 112 text

protection from the worst kind of bugs ...

Slide 113

Slide 113 text

when cause and effect are separated by both space and time

Slide 114

Slide 114 text

confidence automated tests give us

Slide 115

Slide 115 text

Continuous Integration use to maximize the value

Slide 116

Slide 116 text

why go beyond UIAutomation

Slide 117

Slide 117 text

why go beyond UIAutomation automated tests are really valuable ... ... but tricky to get right

Slide 118

Slide 118 text

why go beyond UIAutomation lots of well-established solutions to these challenges ... ... but UIAutomation falls short.

Slide 119

Slide 119 text

there is plenty of good stuff beyond UIAutomation

Slide 120

Slide 120 text

UISpec NativeDriver KIF Bwoken Zucchini Frank Calabash tuneup.js FoneMonkey Telerik etc... etc... beyond UIAutomation

Slide 121

Slide 121 text

thanks. questions?

Slide 122

Slide 122 text

some resources www.testingwithfrank.com [email protected] @ph1 blog.thepete.net me, at the bar