Automated Testing & Processes for your iOS Agile Team Codemash 2.0.1.5

Explore how Continuous Integration combined with Automated Acceptance Testing can be integrated into an iOS Mobile Developer Team’s Agile process.

Transcript

1. Automated Testing & Processes for your iOS Agile Team Codemash 2.0.1.5 https://twitter.com/CodeCrushApp
2. Physarum polycephalum Codemash 2.0.1.5 Better known as plasmodial slime mold
3. The slime mold is single- minded in its goal to survive. The product team is single-minded to build sweet apps.
4. The slime mold just gets itself to the same place much more efficiently. The team needs to be nimble yet fast. Maybe even Agile.
5. The slime mold… "That fan pattern you see with the slime mold is entrenched in everything around us" Mobile. It’s everywhere. Some may see beauty in the patterns the mold creates. Others see beauty in code.
6. Slime survival "is by constantly being in a state of flux.”. DON’T BE THE SLIME. Development can be in flux each and every day, but it doesn’t need to be. Flux burns brain cells. Flux increases costs. Flux is bad. Beat the flux. Strive to be organized. Use the right tools for the right jobs.
7. “Issue Tracker” -> Jira We use Jira to track the information for our projects. Require ments Creative Implementation QA Ship it JIRA (/ˈdʒi.rə/ JEE-rə) is a proprietary issue tracking product, developed by Atlassian. It provides bug tracking, issue tracking, and project management functions. Although normally styled JIRA, the product name is not an acronym, but a truncation of Gojira, the Japanese name for Godzilla. https://www.atlassian.com/software/jira
8. Agile Plugin Board Example
9. Burn Down Chart A burn down chart is a graphical representation of work left to do versus time.
10. How we do it:Jira Flow App Development project management is different from a website. Backlog In Develop ment Development Complete Create a build Ready for QA Story(ies) Development Complete …Eventually Story/Issues Closed.
11. Git Workflow mention • /master for releases • /develop(ment) is mainline • /feature/{JiraTicketName}
12. Example flow Working in branch /feature/foo Working in branch /feature/bar Both merged into /develop(ment) Eventually everything merges into branch /qa (enterprise/adHoc, etc) •Git Workflow: nvie http://bit.ly/197szRP
13. Shipping to QA. Wha? This can be problematic. • What’s in the build? What’s not? • What was merged? When? Why? • What of the build was *tested*? The worst is build-archiving, making sure it’s packaged correctly, code signed, provisioned with the correct provisioning profile. Someone changed the Xcode project and now it won’t build? ugh.
14. Take the human out • Jenkins job monitors QA Branch, waits for a new push. • Runs all logical and acceptance tests. • Builds the IPA, provisions, code signs. • Distributes it; TestFlight, Hockey, OTA, etc. • Updates Jira Tickets • Benefits; the binary is always built exactly the same. All tests are always ran. If they fail, your QA’s time isn’t wasted. This helps maintain developer sanity.
15. The Big Benefits • A build is shipped by a simple Git merge to /QA. • All the Jira Tickets are updated automatically with build information. QA can match each ticket to the exact build the ticket should be tested against. • All the Jira Tickets in that this new build are changed to QA-Ready, no longer assigned to the Developer. • Shipping a build becomes easy to do and can be done as often as needed in less then a minute. • QA can be done in smaller, iterative chunks.
16. How we do it:CI Build Job on / development branch Build job on feature/ featurefoo Build job on / QA branch Jenkins updates Jira Tickets Status/ Workflow Ready for QA • Jenkins automatically creates build jobs when a new /feature branch is pushed to the repo. “Build Per Branch Plugin”
17. Building for the QA Branch 1. Logical Unit Tests 2. User Interface Acceptance Tests 3. Build the IPA. 4. Build & Sign the IPA. 5. Distribute the IPA (Hockey, Testflight, Crashlytics, etc) 6. Parse /CHANGELOG for Jira Tickets. 7. Update the ticket(s) in Jira
18. Acceptance Tests • Tests can be ran against all OS versions, all devices by the CI system. No humans needed. • Developer’s can leverage this to refactor code that’s already been QA’d. If the tests still pass after the refactor, you’re good (common sense and caution here always, of course). https://github.com/lottadot/JiraLottadotTools
19. Acceptance Tests (KIF) http://github.com/kif-framework/KIF @class KIFTestCase; @interface LDTJiraAuthenticationTests : KIFTestCase @end @implementation LDTJiraAuthenticationTests /// Verifies the username & password textfields are shown. - (void)test01ValidateInitialForm { [tester waitForViewWithAccessibilityLabel:@"userNameTextField"]; [tester waitForViewWithAccessibilityLabel:@"passwordTextField"]; [tester waitForViewWithAccessibilityLabel:@"authenticateButton"]; }
20. Acceptance Tests (KIF) /// Enters bogus username/password, submits, verifies alert is shown, dismisses alert - (void)test02AuthenticateInvalid { [tester clearTextFromAndThenEnterText:@"someome" intoViewWithAccessibilityLabel:@"userNameTextField"]; [tester clearTextFromAndThenEnterText:@"passwordfoo" intoViewWithAccessibilityLabel:@"passwordTextField"]; [tester tapViewWithAccessibilityLabel:@"authenticateButton"]; [tester waitForViewWithAccessibilityLabel:@"Login Failed"]; [tester waitForTappableViewWithAccessibilityLabel:@"OK"]; [tester tapViewWithAccessibilityLabel:@"OK"]; }
21. Acceptance Tests (KIF) /// Enters bogus username/password, submits, verifies no is shown - (void)test03AuthenticateValid { [tester clearTextFromAndThenEnterText:LDTJiraAuthenticationTestsValidUsername intoViewWithAccessibilityLabel:@"userNameTextField"]; [tester clearTextFromAndThenEnterText:LDTJiraAuthenticationTestsValidPassword intoViewWithAccessibilityLabel:@"passwordTextField"]; [tester tapViewWithAccessibilityLabel:@"authenticateButton"]; [tester waitForAbsenceOfViewWithAccessibilityLabel:@"Login Failed"]; }
22. Acceptance Tests Demo
23. Run tests from CLI run ./scripts/runAcceptanceTests.sh xcrun xcodebuild -workspace "${xcode_workspace}" \ -scheme "${xcode_acceptancetestsscheme}" \ -destination 'platform=iOS Simulator,name=iPhone 5s,OS=8.1' \ CONFIGURATION_BUILD_DIR="${WORKSPACE}/build" \ CONFIGURATION_TEMP_DIR="${WORKSPACE}/build" \ settings=GCC_TREAT_WARNINGS_AS_ERRORS=NO clean build test https://github.com/lottadot/JiraLottadotTools/tree/development/scripts
24. Jira Issue Updates: Changelog We parse a text file in the repo, which we update before commit (as needed) ./CHANGELOG 1.0.1 #30 2015-01-03 ======================= * TP-1 Issues List Shows Issues Jira's Restful API allows one to post comments, update Issue Status and update an Issue’s (Tech Task, Bug, Story) workflow. When a developer finishes a story, they update CHANGELOG with their last commit. Thereby, Jenkins will only update Jira tickets that are *in that build*.
25. Process Summary Developer’s Make Magic Build System Tests Developers Merge to / QA Build System Builds, Tests Deploys, Updates Jira QA Begins
26. CI Bonus Points 1.Bug analysis (Clang) 2.Unit test results. 3.TODO/BUG/FIXME Scan & Reports 4.App’s build icon version marking 5.Tweet Build Results 6.Post build results to your Company/Project’s IM 7.Email 8.Lines of Code 9.Much much more.
27. Retrospectives 1.Warnings/Bug analysis - What did your team learn from your past project? 2.Unit Test reports. 3.TODO/BUG/FIXME Scan & Reports. Why? Look at the Jira Tickets - do they have the requirements/info needed? Creative? Why can’t we finish the ticket?
28. CI Advice •The acceptance-test build job run-times will grow immensely. If your app uses any sort of web services you will want to Stub or Mock them out. •But didn’t you say the tests are on the UI layer? They can’t access objects. •Yes, I did. You cheat. Use an environment variable ie KIFTESTING = 1 if ([KIFTesting isEqualToString:@"1"]) { // Grab JSON from a file included in the Acceptance Test's target. } else { // Grab JSON from webservice. }
30. Jenkins Setup Tips security unlock-keychain -p jenkins login.keychain ssh-add rvm use 1.9.3 --default export MAVEN_OPTS="-Djava.awt.headless=true -XX: +UseParallelOldGC -XX:ParallelGCThreads=4 -XX:- UseGCOverheadLimit -Xms1280m -Xmx2048m - XX:NewSize=128m -XX:MaxPermSize=512m" java -Xms1024m -Xmx2048m -XX:MaxPermSize=512M -XX: +UseParallelOldGC -XX:ParallelGCThreads=4 - Djava.awt.headless=true -jar /Users/Jenkins/Applications/ Jenkins/jenkins.war •Run it on your local machine first to experiment. •SSH, Keychain, Ram & Startup
31. Edit your Build Scheme
32. Code/Project Advice •The more modular you make things, the easier it is to test. (see “less-coupled” design patterns). Some useful information: •WWDC 414 “Testing in Xcode 6” •WWDC 229 “Advanced iOS Application Architecture and Patterns” •WWDC 224 “Core iOS Application Architectural Patterns" •Mock & Stub when you can. •http://ocmock.org/ios/ •https://github.com/AliSoftware/OHHTTPStubs
33. How-to Get Started 1. An local-instance of Jira running on your machine. Follow the setup steps in the README. 2. iOS Objective-C app from this demonstration with Acceptance Tests that attaches to that Jira. 3. Some Ruby/Python/shell script examples to show communication that can be done from a Jenkins Build Job’s execute-script-build-step.OS Objective-C app from this demonstration with Acceptance Tests. In / scripts. https://github.com/lottadot/JiraLottadotTools
34. #endif My name is Shane Zatezalo. I help people figure out what they want. Then I make their app for them. I run the @BuckeyeCocoa user group. I own Lottadot LLC. I create sweet apps with my team at @Dynamit in Columbus, OH. And we’re hiring. Hit me up. @lottadot http://www.linkedin.com/in/lottadot http://dynamit.com/careers @buckeyecocoa @dynamit
35. Credits •Slime: http://bit.ly/1zXZjYK •Slime Photo: http://bit.ly/1zMWasW •Git Workflow: nvie http://bit.ly/197szRP •Atlassian Jira Board: http://bit.ly/L80P6Z •Jenkins Build Per Branch: http://entagen.github.io/ jenkins-build-per-branch/ •KIF http://github.com/kif-framework/KIF