Snapshot testing. Testing the UI and beyond

What is snapshot testing? How can we use it to test the UI? How can it help us test quickly complex data structures or define a contract to an external system that we do not control?

Transcript

1. SNAPSHOT TESTING GEORGE SOTIROPOULOS MOBILE TEAM @XM TESTING THE UI AND BEYOND
2. SOFTWARE VERIFICATION • Assure that software fully satisfies all the expected requirements, equivalent to software testing • Static and dynamic verification
3. STATIC VERIFICATION SOFTWARE VERIFICATION • Type system (Swift, Kotlin) • Static analysis • Code review
4. DYNAMIC VERIFICATION SOFTWARE VERIFICATION • Unit tests • Snapshot tests • Characterisation tests • Integration tests • UI functional tests • Manual testing
5. WHAT IS SNAPSHOT TESTING? XCTAssertEqual(add(3,5), 8) assertSnapshot(add(3,5))
6. HOW DOES IT WORK? WHAT IS SNAPSHOT TESTING? func testAddition() { assertSnapshot(add(3,5)) } • Record mode Saves a snapshot of the output as an artifact on disk e.g. testAddition1.txt • Verify mode Compares produced snapshot with saved reference snapshot
7. VISUAL TESTING METHOD SCREENSHOT TESTING
8. COMPARING SCREENSHOTS SCREENSHOT TESTING Record Verify Diff
9. WHY DO THIS? SCREENSHOT TESTING • They provide far more coverage than a unit test normally allows • A fast way to see how our view looks while developing • Immediately detect UI breaking changes and avoid regression bugs
10. MULTIPLE UI CASES SCREENSHOT TESTING
11. TESTING LANGUAGES SCREENSHOT TESTING
12. TESTING DEVICES SIZES SCREENSHOT TESTING
13. TESTING THEMES SCREENSHOT TESTING
14. SCREENSHOT TESTING
15. THE PERFECT PULL REQUEST
16. TEXT INSTEAD OF IMAGE SNAPSHOT TESTING THE UI • Could also use a textual representation of the UI as a snapshot
17. COMPLEX DATA STRUCTURES SNAPSHOT TESTING let request = URLRequest(endpoint: .user) assertSnapshot(request) -POST https://xm.com/user +GET https://xm.com/user Accept: application/json { "name" : "George", "email" : “george@xm.com", +"type": “demo", "password" : "abc123" }
18. COMPLEX DATA STRUCTURES SNAPSHOT TESTING curl -X POST -H "Content-Type: application/json" -d '{"name":"George", "email":"george@xm.com", "password": #some_encrypted_password#}' http://xm.com/user
19. COMPLEX DATA STRUCTURES SNAPSHOT TESTING struct URLProvider: Codable { let webTraderHomeURL: String let loginURL: String let newsURL: String let serverTimeURL: String let userURL: String ... ... ... let watchlistURL: String }
20. COMPLEX DATA STRUCTURES SNAPSHOT TESTING _assertInlineSnapshot(matching: provider, as: .json, with: """ { "loginURL" : "https://xm.com/login", "newsURL" : "https://xm.com/news", "serverTimeURL" : "https://xm.com/serverTime", "userURL" : "https://xm.com/user/{userid}", "watchlistURL" : "https://xm.com/watchlist/ {watchlistid}", ... "webTraderHomeURL" : “https://xm.com/home" } """)
21. COMPLEX DATA FORMATS SNAPSHOT TESTING • PDF • XML • HTML • Code generation
22. AS A CHANGE DETECTOR SNAPSHOT TESTING • Snapshot tests are essentially change detectors • Ideal when using external systems that we do not control • ..or when using external libraries that we want to update from time-to-time V2 V1 ✅ ✅ ✅ ✅
23. SNAPSHOT TESTING • Interested whether the output has changed and less about whether it is correct • Lock the behaviour to a golden master version • Low effort method that tests multiple properties at once using a single assertion WHEN WRITING CHARACTERISATION TESTS
24. CONS SNAPSHOT TESTING • Effortless to update but easier to accept new snapshots that have problems • The context of what you test is lost • Might lead to fragile tests
25. MEDIUM BLOG POST https://medium.com/xmtrading
26. TOOLS • https://github.com/pointfreeco/swift-snapshot-testing • https://github.com/uber/ios-snapshot-test-case • https://github.com/Karumi/Shot • https://github.com/Karumi/KotlinSnapshot • https://jestjs.io
27. REFERENCES • https://medium.com/xmtrading/screenshot- testing-101-7dd24ca6b8ce • https://github.com/trading-point/screenshot-testing-android-demo • https://www.pointfree.co/episodes/ep41-a-tour-of-snapshot-testing • https://jestjs.io/docs/en/snapshot-testing • https://codeclimate.com/blog/gold-master-testing/
28. WE ARE HIRING IOS, ANDROID AND OTHER TEAMS https://www.xm.com/careers hr@xm.com
29. THANK YOU QUESTIONS?
30. WHO IS XM • Leading Forex broker • Offices in X countries • Athens X developers (Kotlin, Swift, Java, PHP, Go, C#, C++) • Multi-culture
31. THE MOBILE TEAM • Android / iOS • 7 people • Kotlin, Swift, Java