CI for iOS

CodeMash version of my Continuous Integration for iPhone and iPad talk.

1. CI for iOS: do more while you sleep Kevin Munc – @muncman Method Up LLC [MU]
2. CI?
3. For iOS?
5. Pain
6. Pain & Suffering
7. It can be done • Start with the basics • Xcode and the command line • Version control and CI server • Script it, run it, schedule it
8. Tip #1
9. Jenkins • http://jenkins-ci.org/ • Mac installer or via homebrew • Many alternatives • Many plugins
11. Tip #1.1
12. How many Simulators does it take to break a build?
13. A CI server gives you:
14. A CI server gives you: Automation
15. A CI server gives you: Automation Feedback
16. A CI server gives you: Automation Feedback Wonderful!
17. A CI server gives you: Automation Feedback Wonderful! Peace of mind (for better naps)
18. Requirement: OS X
19. Requirement: • macminicolo.net • hosted-ci.com • cisimple.com • macincloud.com • hostmyapple.com • macminivault.com • xcloud.me OS X
20. Requirement: • macminicolo.net • hosted-ci.com • cisimple.com • macincloud.com • hostmyapple.com • macminivault.com • xcloud.me OS X Or schedule CI to run at night, on your own Mac, while you sleep...
21. Security sudo /usr/sbin/DevToolsSecurity --enable
22. Security sudo xcodebuild -license
23. Credentials • The CI server needs access to your repository • SSH key • The CI server needs Keychain credentials if you want to sign your builds • Developer certificate and profiles
24. Keychain • Export and Import using Keychain Access app • Keys and Certificates • Or use the command line: • sudo security import /path/to/distribution.cer -k /Library/Keychains/ System.keychain • sudo security import /path/to/distribution.p12 -k /Library/Keychains/ System.keychain • Don’t forget about provisioning profiles • Troubleshooting: http://tinyurl.com/iOSDevToolTroubles
25. Tip #2
26. Avoid this error from Xcode RunPlatformUnitTests: warning: Skipping tests; the iPhoneSimulator platform does not currently support application- hosted tests (TEST_HOST set).
27. The culprit /Applications/Xcode.app/ Contents/Developer/ Platforms/ iPhoneSimulator.platform/ Developer/Tools/ RunPlatformUnitTests
28. The workaround(s)
29. Xcode Plugin • https://wiki.jenkins-ci.org/display/JENKINS/Xcode+Plugin • Specify provisioning profile • Keychain access • Signed IPA
30. Xcode Plugin • Recommended xcodebuild arguments: • GCC_SYMBOLS_PRIVATE_EXTERN=NO • COPY_PHASE_STRIP=NO • Allows test bundle to link with Release build symbols
31. Signing with the Plugin
32. Signing with the Plugin
33. Signing with the Plugin
34. Tip #3
35. Prefer Scripts over CI Plugins • More flexibility • More resilient to Apple’s changes • More power to adapt and expand (specificity) • Less coupling to the specific CI server type • You can version control the settings easier
36. xcodebuild xcodebuild \ clean build
37. Tip #4
38. Be specific with xcodebuild options
39. xcodebuild xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
40. xcodebuild xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
41. xcodebuild xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
42. xcodebuild xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
43. xcodebuild xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
44. xcodebuild xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
45. xcodebuild xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
46. xcodebuild xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
47. Tip #5
48. Specify Alternate Output Locations • Don’t rely on Xcode’s cryptic locations • This will make your scripts easier to manage
49. # Build location for Instruments test runs. CI_BUILD_DIR=/tmp/SOLID_AUTOMATION mkdir -p $CI_BUILD_DIR xcodebuild \ ... \ CONFIGURATION_BUILD_DIR=$CI_BUILD_DIR \ clean build
50. # Build location for Instruments test runs. CI_BUILD_DIR=/tmp/SOLID_AUTOMATION mkdir -p $CI_BUILD_DIR xcodebuild \ ... \ CONFIGURATION_BUILD_DIR=$CI_BUILD_DIR \ clean build
51. # Build location for CI test runs. CI_BUILD_DIR=/tmp/SOLID_AUTOMATION mkdir -p $CI_BUILD_DIR xcodebuild \ ... \ CONFIGURATION_BUILD_DIR=$CI_BUILD_DIR \ clean build
52. Demo
53. Tip #6
54. Ensure Return Codes • If the tool you are invoking in your script doesn’t return an exit code indicating pass/fail, it’s up to you. • Unix-friendly 0 (success) or 1 (failure)
55. Tip #7
56. Test against multiple SDKs -sdk iphonesimulator5.0 -sdk iphonesimulator5.1 -sdk iphonesimulator6.0
58. Other Tools to Know About • xcode-select (manage xcode path) • Overridden by DEVELOPER_DIR • xcrun (find and run dev tools) • ibtool (for Interface Builder files) • /Applications/Xcode.app/Contents/Developer/usr/bin/ • Variables in Xcode • “Build Settings Reference”
60. OCUnit
61. Other Related Tools
62. Other Related Tools • OCMock • OCMockito • OCHamcrest • Expecta • TinyMock • LRMocky • OCUnit2JUnit • OCUnitReport • Kicker • xcodebuild-rb • xcodearchive • XcodeTest • AutoBuild • xcodeproj (CocoaPods) • Circle? • Simon? • ios-maven-plugin • Ceedling • XcodeCoverage • JMRTestTools • Nocilla • OHHTTPStubs
63. OCUnit Alternatives • Kiwi • Cedar • OCDSpec • C++ Automated Test Cases in Headers (CATCH) • Objective-Shoulda • Specta
64. Tip #8
65. OCUnit2JUnit for Test Results • ocunit2junit.rb • https://github.com/ciryon/OCUnit2JUnit • Converts output to JUnit format for easier transformation to HTML
67. UIAutomation
68. Other UIAutomation Tools • tunup_js • jasmine-iphone • uiautomation-jasmine- iphone • Bwoken • Zucchini Framework • WaxSim • ios-sim • ios-sim-locale • iphonesim • ui-auto-monkey • ui-screen-shooter • uiautomation-rb?
69. UIAutomation Alternatives • Frank • (iCuke) • KIF • Calabash • UISpec • Bromine • Objective C Slim (ocslim) • AutomationKit • Sikuli • MonkeyTalk (formerly FoneMonkey) • Plus other commercial tools, such as Telerik Test Studio for iOS
71. Tip #9
72. PListBuddy is Your Buddy /usr/libexec/PlistBuddy
74. Tip #10
75. Always quit the simulator • killall • killall -m -KILL "iPhone Simulator" • AppleScript • osascript -e 'tell app "iPhone Simulator" to quit'
76. Tip #11
77. Use Jonathan Penn’s AutomationExample https://github.com/jonathanpenn/AutomationExample
78. Demo
79. More?
80. Code Coverage “Risk coverage”
81. Coverage Tools • gcov • GCC’s coverage tool • GUIs for gcov • LCOV (HTML) • CoverStory (Mac app) • ocov?
82. Tip #12
83. gcovr for Coverage Automation Produces Cobertura-formatted output from gcov data
84. Tip #12.1
85. Enable Test Coverage
87. Static Analysis • Clang • http://clang-analyzer.llvm.org/ • scan-build
88. Tip #13
89. Symbolic links can be your ally
90. Avoid scan-build Pain sudo ln -s \ /Applications/Xcode.app/Contents/Developer/Toolchains/ XcodeDefault.xctoolchain/usr/lib/arc/ libarclite_iphonesimulator.a \ /usr/lib/arc/libarclite_iphonesimulator.a
93. What about test build deployment?
94. Tip #14
95. Use TestFlight • testflightapp.com • OTA deployment • SDK for more features • Free!!
96. TestFlight • Web UI • Desktop App • REST API • Jenkins Plugin
97. TestFlight Alternatives • HockeyApp • http://hockeyapp.net • Also has a Jenkins plugin (forked) • BetaBuilder for iOS • http://www.hanchorllc.com/betabuilder-for-ios/ • More players in this space, esp. for enterprises...
98. TestFlight Config • API Token and Team Token • Need a signed IPA • They get an email with a link for OTA installation!
99. Tip #15
100. Protect Your Tokens • API Token & Team Token • Keep them out of scripts • Instead, define them in Jenkins (and leverage Jenkins security) • 'Configure System' • 'Global properties' section • Or use the plugin (same parameters)
101. TestFlight Plugin
102. TestFlight Script
103. Deployment
104. Deployment
105. Q: Once you have a nightly job to build, unit test, sign an IPA and deploy it to QA, what’s next?
106. Q: Once you have a nightly job to build, unit test, sign an IPA and deploy it to QA, what’s next? A: Sleeping easy.
107. Documentation • Appledoc • Doxygen • HeaderDoc
108. DocSets
109. Appledoc
110. Doxygen
111. Tip #16
112. Don’t stop there!
113. Other Goodies
114. Other Goodies • Ensure your site or API is up • Scan your code for TODOs and FIXMEs • Get trend reports for lines of code (SLOCCount w/ sloc2html) • Scan for duplicated blocks of code (CPD, Simian, etc.) • Use agvtool (Apple-Generic Versioning Tool) to increment your build number
117. Still Missing • Cyclomatic Complexity • Coding convention/Style checker • Code Formatter • Uncrustify, UniversalIndentGUI • UML Generation • via CLI, as opposed to OmniGraffle (AppleScript?) • ER Diagram from Core Data schema
118. Tip #n
122. Sources • Slides: http://www.slideshare.net/muncman • http://blog.carbonfive.com/category/mobile/ • http://www.stewgleadow.com/ • http://lifeandcode.net/ • http://longweekendmobile.com/blog/ • http://longweekendmobile.com/2011/04/17/xcode4-running-application- tests-from-the-command-line-in-ios/ • http://qualitycoding.org • et cetera
123. Thank you! • CodeMash organizers • CodeMash sponsors • and YOU! [MU]
124. Thank you! • CodeMash organizers • CodeMash sponsors • and YOU! Questions? [MU]