Automated UI Testing: Xamarin TestCloud & Calabash

Xamarin's recently announced TestCloud provides amazing automation for UI testing in the cloud across the top mobile device hardware. Calabash is the core testing framework and you can use it locally today.

Presented at Interactive Developers of St. Louis. http://www.meetup.com/Interactive-Developers-of-St-Louis/

Transcript

1. Automated UI Testing:
   Xamarin TestCloud & Calabash
   Friday, May 31, 13
   

   View Slide

2. David Ortinau
   @davidortinau
   http://davidortinau.com
   17 yrs web, desktop, interactive, mobile.
   BA English,
   Maryville University
   Friday, May 31, 13
   

   View Slide

3. Friday, May 31, 13
   

   View Slide

4. Friday, May 31, 13
   

   View Slide

5. Friday, May 31, 13
   

   View Slide

6. Let’s Talk About
   • Xamarin TestCloud
   • Calabash
   • Demo
   Friday, May 31, 13
   

   View Slide

7. +=
   Friday, May 31, 13
   

   View Slide

8. Xamarin TestCloud
   Upload any IPA or APK
   Provide test scripts or use Xamarin Explorer
   Connect with continuous integration systems
   Results include screenshots of every test, step through events,
   stack trace output
   Reports performance, memory usage
   Slow is a bug
   Friday, May 31, 13
   

   View Slide

9. Friday, May 31, 13
   

   View Slide

10. How will it scale?
    Friday, May 31, 13
    

    View Slide

11. Friday, May 31, 13
    

    View Slide

12. Friday, May 31, 13
    

    View Slide

13. Calabash C#
    Calabash
    Android
    Calabash iOS
    Your App
    Calabash
    Server
    Calabash Clients
    Device, Emulator or Simulator
    Host computer
    High-level Architecture
    JSON
    over
    HTTP
    ............
    Friday, May 31, 13
    

    View Slide

14. Calabash Init
    • Ruby 1.9.3 and bundler
    • Recommend rvm or rbenv / ruby-build to manage Ruby and Gems
    • $ gem install calabash-cucumber
    • $ calabash-ios setup
    • $ calabash-ios gen
    • Add calabash.framework to your project directory and link it
    • XCode: -force_load "$(SRCROOT)/calabash.framework/calabash" -lstdc++
    • Xamarin: -force_load $(ProjectDir)/calabash.framework/calabash -lstdc++
    • $ bundle exec cucumber -p ios features/login.feature
    Friday, May 31, 13
    

    View Slide

15. Test Architecture
    • Features
    • Step Definitions
    • Page Objects / Interfaces
    Friday, May 31, 13
    

    View Slide

16. Login Feature
    Feature: Login
    Scenario: Login with valid user
    Given I am on the Login screen
    When I login as "Nat"
    Then I should go to the Assignments screen
    Scenario: Login with invalid user
    Given I am on the Login screen
    When I login as "Invalid"
    Then I should not be logged in
    And I should see a login error message
    Friday, May 31, 13
    

    View Slide

17. Login Step Definition
    Given /^I am on the Login screen$/ do
    await(LoginScreen)
    screenshot_embed(:label => 'Login')
    end
    When /^I login as "([^"]*)"$/ do |user_str|
    user = USERS[user_str.downcase.to_sym]
    login(:as => user)
    end
    Then /^I should go to the Assignments screen$/ do
    assert_screen(AssignmentsScreen)
    screenshot_embed(:label => 'Assignments')
    end
    https://github.com/calabash/calabash-ios/blob/master/calabash-cucumber/features/step_definitions/calabash_steps.rb
    Friday, May 31, 13
    

    View Slide

18. Login Page
    require 'calabash-cucumber/ibase'
    class LoginScreen < Calabash::IBase
    def trait
    "button marked:'Log In'"
    end
    def login(user)
    tap "Username"
    await_keyboard
    keyboard_enter_text user[:email]
    done
    keyboard_enter_text user[:password]
    tap('Log In')
    wait_for_elements_do_not_exist(['activityIndicatorView'])
    assignments = page(AssignmentsScreen)
    begin
    assignments.await
    rescue
    self
    end
    end
    Friday, May 31, 13
    

    View Slide

19. Friday, May 31, 13
    

    View Slide

20. Calabash Console
    $ calabash-ios console
    irb> query(“button”)
    irb> query(“button marked:‘Active’”)
    irb> query(“button marked:‘Accept’”)
    irb> query(“button marked:‘Accept’”)[0]
    irb> record_begin
    irb> record_end “feature_name”
    Friday, May 31, 13
    

    View Slide

21. Resources
    Friday, May 31, 13
    

    View Slide

22. Resources
    • Xamarin Evolve Videos
    • Intro to Calabash: http://xamarin.com/evolve/2013#session-xcjpj20d6s
    • TestCloud: http://xamarin.com/evolve/2013#keynote-80:44
    • Calabash
    • http://calaba.sh/
    • iOS: https://github.com/calabash/calabash-ios
    • Android: https://github.com/calabash/calabash-android
    • https://github.com/aduggin/calabash-ios-tutorial
    • Field Service Example with Calabash Tests
    • https://github.com/calabash/field-service-example
    • Cucumber
    • http://cukes.info/
    • http://pragprog.com/book/hwcuc/the-cucumber-book
    • Frank
    • http://www.testingwithfrank.com/
    Friday, May 31, 13
    

    View Slide

23. Thanks!
    @davidortinau
    http://davidortinau.com
    dave@davidortinau.com
    Friday, May 31, 13
    

    View Slide