A Primer to Unit Testing and Continuous Integration in iOS

I gave a talk at the April 2013 meetup event of iOS Dev Scout.

The talk began with my thoughts about testing. Then, I start describing the various techniques of testing from unit to user acceptance, specification-based to scenario-based, mocking object to matcher. After which I introduces command line tools of Xcode and how you can manage your tests with Makefile or Rakefile. Finally, with that you can set up a Continuous Integration Server to execute the tests at specific interval or on every commit. All these contents created with a focus on iOS. As I only had less than an hour of speaking time, my contents only touches the surface. Nevertheless, I hope you enjoy the slides.

iOS Dev Scout # April 2013 Meetup: https://www.facebook.com/events/160601547432676/
iOS Dev Scout: https://www.facebook.com/groups/iosdevscout/

Stan Chang Khin Boon
Blog: http://lxcid.com
Twitter: https://twitter.com/lxcid
Github: https://github.com/lxcid

Transcript

1. A Primer to Unit Testing and Continuous Integration in iOS Stan Chang Khin Boon @lxcid Thursday, 25 April, 13
2. README.md # Introduction # - Coded for over 7+ years. - Currently working for . - Founded d--buzz. Thursday, 25 April, 13 buUuk: http://www.buuuk.com/ d--buzz: http://www.d--buzz.com/
3. Notable Works for d--buzz Showtimezz Starbuckzz Polycliniczz Thursday, 25 April, 13 Showtimezz (SG) - https://itunes.apple.com/sg/app/showtimezz-singapore/id416536728? mt=8 Starbuckzz - https://itunes.apple.com/sg/app/starbuckzz-sg-starbucks-singapore/ id488736800?mt=8 Polycliniczz - https://itunes.apple.com/sg/app/polycliniczz-sg-singapore/id565285206? mt=8
4. Notable Works for buUuk Mana Rapid Transit Thursday, 25 April, 13 Mana Rapid Transit: https://itunes.apple.com/sg/app/mana-rapid-transit/id501961346? mt=8
5. What’s your job as a programmer? Thursday, 25 April, 13
6. “As a programmer, it is your job to put yourself out of business. What you do today can be automated tomorrow.” - Douglas McIlroy, developed Unix pipelines Thursday, 25 April, 13 References - http://en.wikipedia.org/wiki/Douglas_McIlroy
7. Which Development Methodology do you worship? Thursday, 25 April, 13
8. Waterfall? Thursday, 25 April, 13 References - http://en.wikipedia.org/wiki/Waterfall_model
9. Extreme Programming (Agile) References - http://dilbert.com/strips/comic/2003-01-09/ Thursday, 25 April, 13 References - http://dilbert.com/strips/comic/2003-01-09/
10. Scrum (Agile) References - http://dilbert.com/strips/comic/2008-04-15/ Thursday, 25 April, 13 References - http://dilbert.com/strips/comic/2008-04-15/
11. Agile? References - http://dilbert.com/strips/comic/2007-11-26/ Thursday, 25 April, 13 References - http://dilbert.com/strips/comic/2007-11-26/
12. Agile??? References - http://dilbert.com/strips/comic/2005-11-16/ Thursday, 25 April, 13 References - http://dilbert.com/strips/comic/2005-11-16/
13. Test Driven Development References - http://dilbert.com/strips/comic/2010-11-07/ Thursday, 25 April, 13 References - http://dilbert.com/strips/comic/2010-11-07/
14. Beer Driven Development Thursday, 25 April, 13 References - Dillion’s desk in buUuk office
15. But no matter which “religious” you “worship” ... Thursday, 25 April, 13
16. Bugs still looms large! References - http://dilbert.com/strips/comic/1995-11-13/ Thursday, 25 April, 13 References - http://dilbert.com/strips/comic/1995-11-13/
17. And we still have to test for every (alpha, beta, RC, 1.0, 2.0, etc) single releases! Thursday, 25 April, 13 References - http://en.wikipedia.org/wiki/Systems_development_life-cycle
18. Reasons for not testing • Counter-productive! • Waste of resources! • Doesn’t have time for it! • Slow down development effort! • Time writing tests could have been used to build more features! • blah, blah, blah... Thursday, 25 April, 13
19. Reasons for testing • Spaghetti code! • Bugs outgrew features! • Maintenance nightmare! • Adding features always end up breaking the codes! • blah, blah, blah? Thursday, 25 April, 13
20. Too much of either is bad. Thursday, 25 April, 13 References - http://www.mamalaughlin.com/2010/05/teeter-totter.html
21. When to test? Only when your company culture appreciates it. Thursday, 25 April, 13
22. More efforts into testing, less efforts into maintaining and vice versa. Thursday, 25 April, 13 References - http://www.mamalaughlin.com/2010/05/teeter-totter.html
23. How to test in iOS? Thursday, 25 April, 13
24. Built in Xcode! • Unit Testing - OCUnit #import <SenTestingKit/SenTestingKit.h> @interface MyAppTests : SenTestCase @end @implementation MyAppTests - (void)testArray { NSArray *arrayOfNumbers = @[ @1, @2, @3, @4, @5 ]; STAssertTrue(arrayOfNumbers.count == 5, @"arrayOfNumbers should have 5 elements."); } @end Thursday, 25 April, 13
25. Built in Xcode! • Unit Testing - OCUnit • Menu Bar > Product > Test • ⌘U Thursday, 25 April, 13
26. Built in Xcode! • User Acceptance Testing (UAT) - UI Automation • Uses accessibility var target = UIATarget.localTarget(); var appWindow = target.frontMostApp().mainWindow(); appWindow.tabBar().buttons()["Unit Conversion"].tap(); UIALogger.logStart("Logging element tree…"); element.logElementTree(); UIALogger.logPass(); Thursday, 25 April, 13
27. That’s it? Thursday, 25 April, 13
28. No Run Loop • Testing asynchronous network requests only works in a run loop environment. • OCUnit does not exercise its run loop. • Solution typedef void (^LXSResumeBlock)(void); typedef void (^LXSRunLoopBlock)(LXSResumeBlock resume); // Call resume() to stop the block from exercising the run loop and break out of the loop/block. // Failure to call resume() will cause the test to hang indefinitely. // This is useful to testing asynchronous actions like AFNetworking operations. See https:// gist.github.com/2215212 // With some reference from https://github.com/icanzilb/MTTestSemaphore, I was able to simplify this method. void LXS_exercisesRunLoopInBlock(LXSRunLoopBlock block) { __block BOOL keepRunning = YES; block(^{ keepRunning = NO; }); while (keepRunning && [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode beforeDate: [NSDate dateWithTimeIntervalSinceNow:0.03]]) { } } • Solution LXS_exercisesRunLoopInBlock(^(LXSResumeBlock resume) { // Do your async network connection here... resume(); } Thursday, 25 April, 13 References - A version with timeout available at https://github.com/lxcid/LXSupport/blob/master/src/ LXSupport/LXSupport/SenTestingKit%2BLXSupport.h
29. Other Unit Testing Frameworks • GHUnit • GTMUnit • Google Toolbox for Mac • Specta • Behavior Driven Development • Specification-Based • RSpec-like syntax • by Peter Jihoon Kim • UISpec • Kiwi Thursday, 25 April, 13 References - GHUnit: https://github.com/gabriel/gh-unit - GTMUnit: http://code.google.com/p/google-toolbox-for-mac/ - Specta: https://github.com/petejkim/specta - UISpec: http://code.google.com/p/uispec/ - Kiwi: https://github.com/allending/Kiwi
30. Mock object • OCMock • Stub - Verify expected results. id connection = [OCMockObject mockForClass:[TwitterConnection class]]; Tweet *testTweet = [Tweet tweetWithText:@”Hello World!”]; NSArray *tweetArray = @[ testTweet ]; [[[connection stub] andReturn:tweetArray] retrieveTweetsForSearchTerm:[OCMArg any]]; • Mock - Verify interactions and parameters. id view = [OCMockObject mockForClass:[TweetView class]]; controller.tweetView = view; [[view expect] addTweet:[OCMArg any]]; [controller displayTweets]; [view verify]; Thursday, 25 April, 13 References - http://ocmock.org
31. Mock object for networking • OHHTTPStubs • Uses NSURLProtocol to provides stub functionalities for networking. • http://nshipster.com/nsurlprotocol/ • Stub -Verify expected results [OHHTTPStubs addRequestHandler:^OHHTTPStubsResponse*(NSURLRequest *request, BOOL onlyCheck) { return [OHHTTPStubsResponse responseWithFile:@"response.json" contentType:@"text/json" responseTime:2.0]; }]; Thursday, 25 April, 13 References - https://github.com/AliSoftware/OHHTTPStubs
32. Matcher • Provides more expressive and flexible assertions. • OCHamcrest assertThat(@"foo", is(equalTo(@"foo"))); assertThatUnsignedInteger(foo, isNot(equalToUnsignedInteger(1))); assertThatBool([bar isBar], is(equalToBool(YES))); assertThatDouble(baz, is(equalToDouble(3.14159))); • Expecta expect(@"foo").to.equal(@"foo"); expect(foo).notTo.equal(1); expect([bar isBar]).to.equal(YES); expect(baz).to.equal(3.14159); Thursday, 25 April, 13 References - OCHamcrest: https://github.com/hamcrest/OCHamcrest - Expecta: https://github.com/petejkim/expecta
33. Integration/UAT • Most are much easier to use than UIAutomation. • KIF by Square • Calabash by LessPainful • Cross Platform • Frank by ThoughtWorks Thursday, 25 April, 13 References - KIF: http://github.com/square/KIF - Calabash: https://github.com/calabash/calabash-ios - Frank: http://testingwithfrank.com
34. Continuous Integration Thursday, 25 April, 13
35. Continuous Integration • Its all command line. • Manage tasks with Makefile, Rakefile or equivalent. • Xcode provides command line utility. • Xcode’s Preference > Downloads > Install Command Line Tools • xcodebuild • Execute tasks at interval or at events • every midnight or nightly • every commit Thursday, 25 April, 13 References - xcodebuild manual page: https://developer.apple.com/documentation/Darwin/Reference/ Manpages/man1/xcodebuild.1.html
36. Choice of CI • Jenkins • Travis CI Thursday, 25 April, 13 References - Jenkins: http://jenkins-ci.org - Travis CI: http://travis-ci.org
37. Running your own CI with Jenkins • Continuous Integration of iOS Projects using Jenkins, CocoaPods, and Kiwi http://9elements.com/io/index.php/ continuous-integration-of-ios-projects- using-jenkins-cocoapods-and-kiwi/ Thursday, 25 April, 13 References - http://9elements.com/io/index.php/continuous-integration-of-ios-projects-using- jenkins-cocoapods-and-kiwi/
38. Open Source with Travis CI • Defines .travis.yml • Enables the project for CI in Travis CI portal! • Add CI status ( , ) to your README.md! • Test exit with anything but 0 indicates error. Thursday, 25 April, 13
39. .travis.yml • Creates .travis.yml your repository root. language: objective-c script: make test Thursday, 25 April, 13
40. .travis.yml • You can use rake task too! • Uses Gemfile to manage your gem dependencies. language: objective-c before_install: "sudo gem update --system && sudo gem install bundler && bundle install" script: "bundle exec rake test --trace" Thursday, 25 April, 13
41. Makefile • Simple make file test: git submodule init git submodule update xcodebuild \ -project Project.xcodeproj \ -target ProjectTests \ -sdk iphonesimulator \ -configuration Debug \ TEST_AFTER_BUILD=YES Thursday, 25 April, 13
42. Trouble with Simulator? • If you have trouble with simulator, uses ios-sim test: git submodule init git submodule update which ios-sim || brew install ios-sim xcodebuild \ -project Project.xcodeproj \ -target ProjectTests \ -sdk iphonesimulator \ -configuration Debug \ RUN_UNIT_TEST_WITH_IOS_SIM=YES Thursday, 25 April, 13
43. Trouble with Simulator? Thursday, 25 April, 13
44. Rakefile • Wrap commands in Rakefile. def test_target(project, target, verbose = false) command = "xcodebuild -project #{project} -target #{target} -sdk iphonesimulator -configuration Debug TEST_AFTER_BUILD=YES 2>&1" IO.popen(command) do |io| . . . end end desc 'Run the tests' task :test do verbose = ENV['VERBOSE'] ios = test_target('src/LXSupport/LXSupport.xcodeproj', 'LXSupportTests', verbose) puts "\n\n\n" if verbose puts "iOS: #{ios == 0 ? 'PASSED'.green : 'FAILED'.red}" exit(ios) end Thursday, 25 April, 13
45. CI with UIAutomation Thursday, 25 April, 13
46. Notable Users • Many of the top companies adopts Testing • Square • Facebook • iOS SDK - OCUnit, OCMock, OHHTTPStubs and OCHamcrest. • R/GA • Uses frank • Any commit that break the CI have to do deployment until the next person break it. (Newbie beware!) Thursday, 25 April, 13
47. Questions? Thursday, 25 April, 13
48. Stay bug free Thank you @lxcid Thursday, 25 April, 13