iOS Automated Testing with
Calabash: Tips and Tricks
Moncef Belyamani
@monfresh
Mobile Testing Summit / November 1, 2012 / San Francisco
Slide 2
Slide 2 text
No content
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
No content
Slide 5
Slide 5 text
iPad Apps
iPhone Apps
Slide 6
Slide 6 text
AUTOMATION TOOL
CHECKLIST
• Easy to set up and maintain
• Updated regularly
• Well-documented
• Supports iOS and Android
• Runs on devices untethered
• Readable tests
Slide 7
Slide 7 text
TOOLS EVALUATED
• Anteater
• Calabash
• FoneMonkey (before it became MonkeyTalk)
• Frank
• KIF
• TouchTest
• UI Automation
• Zucchini
Slide 8
Slide 8 text
AND THE WINNER IS...
Slide 9
Slide 9 text
No content
Slide 10
Slide 10 text
RECOMMENDED SETUP
• Ruby 1.9.3
• RVM
• Gemset
• How to install Xcode, Homebrew, Git, RVM, & Ruby 1.9.3 on
Snow Leopard, Lion, and Mountain Lion
• Enhance your Ruby development with RVM gemsets and .rvmrc
Slide 11
Slide 11 text
CALABASH TEST
FEATURE FILE + STEP DEFINITION
Slide 12
Slide 12 text
FEATURE EXAMPLE
Feature: Comments
In order to contribute to the discussion
As a Hacker News reader
I want to be able to add a comment
Scenario: User not logged in
Given I am not logged in
When I go to comment on a submission
Then I should be prompted to log in
Slide 13
Slide 13 text
DON’T DO THIS
Scenario: User logged in, but cancels comment form
Given the app is running
When I touch "Profile"
Then I wait to see "Login"
Then I fill in “Username” with "my_username"
Then I touch “Password”
Then I fill in “Password” with "my_password"
Then I touch done
Then I wait to see "Logout"
Then I touch "Home"
Then I touch list item number 1
Then I touch "reply"
Then I touch "Cancel"
Then I should see "Submission"
Slide 14
Slide 14 text
DO THIS
Scenario: User logged in, but cancels comment form
Given I am logged in as “username”
When I go to comment on a submission
But I cancel the comment form
Then I should see the submission
And a comment from "username" should not appear
Slide 15
Slide 15 text
STEP DEFINITION
Given /^I am logged in as "([^\"]*)"$/ do |username|
macro 'I touch "Profile"'
if element_exists("button marked:'Logout'")
sleep(1)
else
macro %Q[I fill in “Username” with "#{username}"]
macro 'I touch “Password”'
macro 'I fill in “Password” with "my_password"'
macro 'I touch done'
macro 'I wait to see "Logout"'
end
end
Slide 16
Slide 16 text
UPDATING IS EASY, BUT...
• gem update calabash-cucumber
• calabash-ios download (not with SVN)
• curl http://localhost:37265/version (to verify framework is
up to date)
• May need to “Reset Content and Settings” in Simulator
• When new files are added, update Target Membership
Slide 17
Slide 17 text
No content
Slide 18
Slide 18 text
KEEP IN MIND
Swiping depends on device orientation
Can only touch visible items
Slide 19
Slide 19 text
TIPS & TRICKS
Use .bash_profile aliases
self.myButton.accessibilityElementsHidden = YES;
Use console to experiment and “query” to find elements
Participate in the Google Group
Get commit notifications with GitHub for Mac
Slide 20
Slide 20 text
No content
Slide 21
Slide 21 text
RECOMMENDATIONS
• Full-time Automation Engineer
• Work with Dev to make apps accessible
• Continuous Integration
• Contribute fixes and custom steps
Slide 22
Slide 22 text
REFERENCES
• iOS Automated Testing with Calabash, Cucumber, and Ruby
• Create aliases in .bash_profile to assign shortcuts for common
Terminal commands
• A Calabash custom step to Swipe to delete all cells in a Table View
• Code for America
• AOL Mobile