Automated Testing with iOS

The talk gives an overview of automated testing with iOS. It discusses the issues of project construction, unit testing, view testing, integration testing, automated distribution of betas and the automated app build with a continuous integration system. Particular, it will address the pitfalls and best practices from experience gained during development at AutoScout24.

Transcript

1. Automated Testing with iOS Felix Schulze Teamlead Mobile Software Development
2. @x2on Every change can break your app! Picture: http://fav.me/d3g96mr
3. @x2on Break = Review
4. autoscout24.de
5. @x2on Continuous Integration • Tool • Minimize risk & Quality improvement • Automate! • Example: Xcode Bots, Jenkins, TeamCity, Bamboo, ...
6. @x2on Components • Build management • Unit tests • Integration tests • Automatic beta versions
7. @x2on Xcode Bots?
8. @x2on Beta App Internal App Store General Setup CI Server Changes Results Build Agent‘s
9. @x2on Build Unit-Tests Integration-Tests Every Hour Detail Setup Sourcecode Changes Alpha Version Every Hour
10. @x2on Hard & Software Setup • Server: • TeamCity Continuous Integration Server • Build Agents: • Mac Mini’s in 19’’ Racks • Mac OS X 10.9 / Xcode 5 / Homebrew / JDK 7
11. @x2on iOS Setup • Dependencies • CocoaPods • gradle-cocoapods-plugin • Build • xctool
12. @x2on iOS Setup • Tests • xctool • xctool teamcity reporter • Package • xctool • xcodebuild • gradle-hockeyapp-plugin
13. @x2on iOS Tests • Unit Tests: Apple XCTest • Mock: OCMock 3 • View Tests: ios-snapshot-test-case • PNG view rendering / compare • Integration Tests: KIF • Network Mocking: nocilla
14. @x2on XCTest - Assertion • XCTFail • XCTAssertNil • XCTAssertNotNil • XCTAssertTrue | XCTAssertFalse • XCTAssertEqualObjects • XCTAssertEquals • XCTAssertEqualsWithAccuracy
15. @x2on FBSnapshotTestCase • Setup: • pod ‚FBSnapshotTestCase'
16. @x2on FBSnapshotTestCase • Setup: • FB_REFERENCE_IMAGE_DIR
17. @x2on FBSnapshotTestCase
18. @x2on Reference Failed Diff
19. @x2on Mocking • Dummy Objects • Verify calls • OCMock Framework • http://ocmock.org/
20. @x2on Class/Protocol Mock id sample1 = OCMClassMock([Sample class]);! id sample2 = OCMProtocolMock(@protocol(SProtocol));! ! id object = [Sample new];! id sample3 = OCMPartialMock(object);
21. @x2on Stubbing Mock OCMStub([mock someMethod]).andReturn(aObject);! OCMStub([mock someValue]).andReturn(YES);! OCMStub([mock someLong]).andReturn(1); //FAIL! OCMStub([mock someLong]).andReturn(1.0L);! OCMStub([mock someMethod]).andForwardToRealObject();
22. @x2on Verify Mock id sample = OCMClassMock([SomeClass class]);! /* insert your test here */! OCMVerify([mock someMethod]);
23. @x2on Matching arguments Mock OCMStub([mock someMethod:[OCMArg isNil]]);! OCMStub([mock someMethod:[OCMArg isNotNil]]);! OCMStub([mock someMethod:[OCMArg isNotEqual:aValue]]);! OCMStub([mock someMethod:[OCMArg ! 
 ! ! ! ! ! checkWithBlock:^BOOL(id value) 
 ! ! ! ! ! ! ! ! ! ! ! {
 ! ! ! ! ! ! ! ! ! ! ! ! BOOL result;
 !! ! ! ! ! ! ! ! ! ! ! /*…*/
 ! ! ! ! ! ! ! ! ! ! ! ! return result; 
 ! ! ! ! ! ! ! ! ! ! ! }]);
24. @x2on Execute Block - Mock OCMStub(
 ! [mock someMethodWithCompletionBlock:OCMOCK_ANY]
 ! ! ).andDo(^(NSInvocation *invocation) {! ! ! void (^completion)(NSDictionary *someDict, NSError *error);! ! ! [invocation getArgument:&completion atIndex:2];! ! ! completion(@{@“key“:@“value“}, nil);! ! };
25. @x2on xctool xctool 
 ! -workspace DWX-2014-Demo.xcworkspace 
 ! -scheme Build
 ! -sdk iphonesimulator
 ! -destination "platform=iOS Simulator,OS=latest,name=iPhone Retina (4-inch)"
 ! -reporter pretty
 ! -reporter teamcity
 ! clean
 ! test!
26. @x2on xctool reporter • pretty • plain • phabricator • junit • json-stream • json-compilation-database • user-notifications • teamcity
27. @x2on Integration tests • „Automatic click test“ • KIF (Keep It Functional) • Tests in Objective-C • Test-Szenarios & Test-Steps • Accessibility Elements • XCTest Target —> Link App • https://github.com/kif-framework/KIF
28. @x2on KIF [tester enterText:@"user@example.com" intoViewWithAccessibilityLabel:@"Login"];! [tester enterText:@"mypassword" intoViewWithAccessibilityLabel:@"Password"];! [tester tapViewWithAccessibilityLabel:@"Login"];! [tester waitForTappableViewWithAccessibilityLabel:@"My User"];
29. @x2on Nocilla • HTTP Response Mocking • Supports AFNetworking, ASI-HTTP-Request, NSURLSession, NSURLConnection, etc. • https://github.com/luisobo/Nocilla
30. @x2on Nocilla stubRequest(@"GET", @„http://www.google.com");! stubRequest(@"GET", @"^http://(.*?)\\.example\\.com/ test\\.json".regex);! stubRequest(@"GET", @„http:// www.google.com“).andReturn(404);! stubRequest(@"GET", @"https://api.example.com/ test.json").andReturn(201).withHeaders(@{@"Content- Type": @"application/json"}).withBody(@"{\"ok \":true}");
31. @x2on Automatic Alpha Versions • iOS: Enterprise Dev Account • Hockey App • Automatic In-App-Updates • Crash Reports • Usage Statistics
32. Crash Analyzing
33. Crash Analyzing
34. @x2on Questions + Contact • Felix Schulze @x2on • github.com/AutoScout24 • github.com/x2on