Testing on iOS

A quick presentation on the testing frameworks in iOS

Transcript

1. Testing in iOS
   a hastily assembled presentation
   by bradley grzesiak
   

   View Slide

2. Frameworks
   • SenTest
   • OCHamcrest
   • Kiwi
   • Frank
   • UIAutomation
   • KIF
   

   View Slide

3. SenTest
   • The default
   • Two levels
   • File
   • - (void)test*
   • meh
   

   View Slide

4. Test Suite 'All tests' started at 2012-02-20 17:05:44 +0000
   Test Suite '/Users/.../ThrowawayTests.octest(Tests)' started at 2012-02-20 17:05:44 +0000
   Test Suite 'ThrowawayTests' started at 2012-02-20 17:05:44 +0000
   Test Case '-[ThrowawayTests testExample]' started.
   Test Case '-[ThrowawayTests testExample]' passed (0.000 seconds).
   Test Suite 'ThrowawayTests' finished at 2012-02-20 17:05:44 +0000.
   Executed 1 test, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
   Test Suite '/Users/.../ThrowawayTests.octest(Tests)' finished at 2012-02-20 17:05:44 +0000.
   Executed 1 test, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
   Test Suite 'All tests' finished at 2012-02-20 17:05:44 +0000.
   Executed 1 test, with 0 failures (0 unexpected) in 0.000 (0.010) seconds
   STAssertEqualObjects(@”a”, @”a”, @”my description”);
   

   View Slide

5. Test Suite 'All tests' started at 2012-02-20 17:02:01 +0000
   Test Suite '/Users/.../ThrowawayTests.octest(Tests)' started at 2012-02-20 17:02:01 +0000
   Test Suite 'ThrowawayTests' started at 2012-02-20 17:02:01 +0000
   Test Case '-[ThrowawayTests testExample]' started.
   /.../ThrowawayTests.m:29: error: -[ThrowawayTests testExample] : 'a' should be equal to 'b' my description
   Test Case '-[ThrowawayTests testExample]' failed (0.000 seconds).
   Test Suite 'ThrowawayTests' finished at 2012-02-20 17:02:01 +0000.
   Executed 1 test, with 1 failure (0 unexpected) in 0.000 (0.000) seconds
   Test Suite '/Users/.../ThrowawayTests.octest(Tests)' finished at 2012-02-20 17:02:01 +0000.
   Executed 1 test, with 1 failure (0 unexpected) in 0.000 (0.000) seconds
   Test Suite 'All tests' finished at 2012-02-20 17:02:01 +0000.
   Executed 1 test, with 1 failure (0 unexpected) in 0.000 (0.002) seconds
   STAssertEqualObjects(@”a”, @”b”, @”my description”);
   

   View Slide

6. OCHamcrest
   • matchers!
   • assertThat(foo, equalTo(bar)
   • poly-framework, polyglot
   

   View Slide

7. Kiwi
   • Like RSpec
   • describe, context, it, xit, etc.
   • remember the __block modifier!
   • infinitely nestable
   

   View Slide

8. Kiwi test file - top
   #import "Kiwi.h"
   #import "NSDictionary+BWTravisCI.h"
   

   View Slide

9. Kiwi test file - surround
   SPEC_BEGIN(NSDictionarySpec)
   ...
   SPEC_END
   

   View Slide

10. Kiwi test file - top-level
    describe(@"NSDictionary+BWTravisCI", ^{
    __block NSDictionary *subject = nil;
    beforeEach(^{
    subject = [NSDictionary dictionaryWithObjectsAndKeys:@"1",
    @"a", @"2", @"b", @"3", @"c", nil];
    });
    describe(@"subdictionaryUsingKeys", ^{
    ...
    });
    });
    

    View Slide

11. Kiwi test file - contexts & tests
    context(@"matching nothing", ^{
    it(@"returns an empty dictionary", ^{
    NSDictionary *result = [subject subdictionaryUsingKeys:@"foo", nil];
    [[theValue(result.count) should] equal:theValue(0)];
    });
    });
    context(@"matching something", ^{
    it(@"returns a subset dictionary", ^{
    NSDictionary *result = [subject subdictionaryUsingKeys:@"a", nil];
    [[result should] equal:[NSDictionary dictionaryWithObject:@"1" forKey:@"a"]];
    });
    });
    context(@"matching ALL THE THINGS", ^{
    it(@"returns a copy of the dictionary", ^{
    NSDictionary *result = [subject subdictionaryUsingKeys:@"a", @"b", @"c", nil];
    [[result should] equal:[subject copy]];
    });
    });
    

    View Slide

12. Frank
    • Ruby- and Cucumber-based
    • Gherkin language
    • “Selenium for native iOS apps”
    • Encourages accessibility
    

    View Slide

13. Frank - Feature File
    Feature: Login to the app
    Scenario: Successful login
    Given I launch the app
    When I log in with a valid userid and password
    Then I am on the start view
    

    View Slide

14. Frank - Simple Step
    Then /^I should see (.*) apples$/ do |count|
    apples = frankly_map( "label marked:'red apples'", 'tag' )
    apples.count.should == count.to_i
    end
    

    View Slide

15. Frank - Textfield Step
    When /^I use the keyboard to fill in the textfield marked
    "([^\\"]*)" with "([^\\"]*)"$/ do |text_field_mark, text_to_type|
    text_field_selector = "view marked:'#{text_field_mark}'"
    check_element_exists( text_field_selector )
    touch( text_field_selector )
    frankly_map( text_field_selector, 'setText:', text_to_type )
    frankly_map( text_field_selector, 'endEditing:', true )
    end
    

    View Slide

16. Frank - Screencapture Step
    Then /^I save a screenshot with prefix (\w+)$/ do |prefix|
    filename = prefix + Time.now.to_i.to_s
    %x[screencapture #{filename}.png]
    end
    

    View Slide

17. Frank - Videocapture
    Around( '@record' ) do | scenario, block |
    start_recording
    block.call
    stop_recording
    end
    

    View Slide

18. UIAutomation
    • Official Apple developer tool
    • Uses Instruments
    • Uses Javascript
    • Purely integration
    • Can run external scripts
    

    View Slide

19. UIAutomation
    function assertElementPresent(el, el_name) {
    el_name = el_name ? el_name : '';
    if (el.isValid()) {
    return UIALogger.logPass("Element " + el_name + " is present");
    } else {
    return UIALogger.logFail("Element " + el_name + " is not present");
    }
    }
    var window = UIATarget.localTarget().frontMostApp().mainWindow();
    var repository_list = window.tableViews().firstWithName("Repositories");
    var latest_repo = repository_list.cells()[0];
    window.navigationBar().leftButton().tap();
    latest_repo.tap();
    var build_list = window.tableViews().firstWithName("Builds");
    assertElementPresent(build_list.cells());
    

    View Slide

20. KIF
    • “Keep It Functional”
    • Objective-C
    • Integration/functional tests
    • Scenarios & Steps
    

    View Slide

21. KIF
    @interface EXTestController : KIFTestController
    @end
    @implementation EXTestController
    - (void)initializeScenarios;
    {
    [self addScenario:[KIFTestScenario scenarioToLogIn]];
    // Add additional scenarios you want to test here
    }
    @end
    

    View Slide

22. KIF
    #import "KIFTestScenario+EXAdditions.h"
    #import "KIFTestStep.h"
    #import "KIFTestStep+EXAdditions.h"
    @implementation KIFTestScenario (EXAdditions)
    + (id)scenarioToLogIn;
    {
    KIFTestScenario *scenario = [KIFTestScenario scenarioWithDescription:@"User can successfully login."];
    [scenario addStep:[KIFTestStep stepToReset]];
    [scenario addStepsFromArray:[KIFTestStep stepsToGoToLoginPage]];
    [scenario addStep:[KIFTestStep stepToEnterText:@"user@example.com" intoViewWithAccessibilityLabel:@"Username"]];
    [scenario addStep:[KIFTestStep stepToEnterText:@"thisismypassword" intoViewWithAccessibilityLabel:@"Password"]];
    [scenario addStep:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"Log In"]];
    // Verify that the login succeeded
    [scenario addStep:[KIFTestStep stepToWaitForTappableViewWithAccessibilityLabel:@"Welcome"]];
    return scenario;
    }
    @end
    

    View Slide

23. KIF
    #import "KIFTestStep+EXAdditions.h"
    @implementation KIFTestStep (EXAdditions)
    + (NSArray *)stepsToGoToLoginPage;
    {
    NSMutableArray *steps = [NSMutableArray array];
    // Dismiss the welcome message
    [steps addObject:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"That's awesome!"]];
    // Tap the "I already have an account" button
    [steps addObject:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"I already have an account."]];
    return steps;
    }
    @end
    

    View Slide

24. KIF
    - (BOOL)application:(UIApplication *)application
    didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
    {
    // ...
    #if RUN_KIF_TESTS
    [[EXTestController sharedInstance] startTestingWithCompletionBlock:^{
    // Exit after the tests complete so that CI knows we're done
    exit([[EXTestController sharedInstance] failureCount]);
    }];
    #endif
    return YES;
    }
    

    View Slide

25. that’s it!
    

    View Slide