Tips and Tricks to write reliable tests

Writing unit tests and UI Tests sometimes can be tricky. Many times, tests pass locally but fail in CI which makes them harder to debug. Other times, tests crash in the CI environment, but never on your Mac. Also, working with CoreData and creating mock objects and clearing them after each test requires some extra steps in the app design. This talk is going over these and talk about approaches that helped us at Zocdoc to resolve them.

Transcript

1. Tips and Tricks for Writing Reliable Tests Mani Ramezan @maniramezan
4. https://media.giphy.com/media/3oKIPnAiaMCws8nOsE/giphy.mp4
5. https://media.giphy.com/media/Px7FQJqhWTGaA/giphy.mp4
6. https://media.giphy.com/media/HCK7W8Qb52ifS/giphy.mp4
7. Agenda • Types of testing • Challenges with writing reliable tests • Some tips and tricks • Resources
8. Importance level
9. Integration vs. UI test • Integration Test: Aims to test integration between different parts and units of the app • Usually done against mock data • UI Test: Mainly refers to as functional testing, aims to test product integration with other services • Usually done against production code
10. Challenges • Tightly coupled with app architecture • Test interactions with server • UI Automation tests tends to be flaky • Local ≠ CI environment • Time consuming • Managers tends to check for code coverage • Test in production
11. Enable random testing
12. Separate AppDelegate • Useful for unit tests • Skip unnecessary setups • Customize data store and services • More details: https://marcosantadev.com/fake- appdelegate-unit-testing-swift/
 let isRunningTests = NSClassFromString("XCTestCase") != nil let appDelegateClass = isRunningTests ? NSStringFromClass(TestAppDelegate.self) : NSStringFromClass(AppDelegate.self) UIApplicationMain(CommandLine.argc, CommandLine.unsafeArgv, nil, appDelegateClass)
13. Data Store • Mainly a big issue for unit tests • Need a way to reset data after each unit test • Use in-memory persistent for core data • Be aware of NSUserDefaults
14. Network Stubbing • Use a third party library to mock API responses • OHHTTPStub • Useful for unit test and integration tests • Would need alternatives for production testing
15. Environment Variables • Can be used to customize tests • Test universal links and deeplinks • Feature flags • Camera testing
16. Reset warnings and content • Reset location & Privacy • “Reset Warning” on simulator and “Reset Settings” on device • Run before each UITest test case • Access Settings app let settings = XCUIApplication(bundleIdentifier: "com.apple.Preferences")
17. Delete the app • Helps with resetting NSUserDefaults • Explained https://stackoverflow.com/a/36168101/1450348 • Pros • Clear state • Cons • Increases test runs • Adds to test flakiness
18. You can’t trust a machine
19. Always check visibility
20. Keyboard interaction • Button title: • ”Search” • “Send” • “Route" • “Join" • “Go" • "Emergency call” • “Done" • “Continue"
21. More tips • Review crash logs • Increase wait interval • Always rely on CI results • Constantly monitoring • Re-visiting app architecture • Write tests for new bugs • Code coverage ≠ quality
22. Resources • John Sundell • Getting started with Xcode UI testing in Swift • Reduce Flakiness in Swift test • UIKonf 2018 - The Magic of UI Testing • Apple WWDC 2018 - Session 403
23. Questions? Mani Ramezan mani.ramezan@gmail.com @maniramezan