Visual Regression Testing in Mobile

A concise look at visual regression testing: comparing Flutter golden tests and native iOS/Android snapshot testing—how both use baseline images to catch unintended UI changes, where Flutter’s unified rendering brings speed and consistency, and where native approaches stay closer to real platform nuances. We’ll cover strengths, pitfalls (flakiness, baseline churn), choosing the right granularity, and practical tips.

Owain Brown is a seasoned mobile developer with over 10 years of experience, specialising in iOS and Swift development. Currently leading mobile engineering teams at hedgehog lab, he has expanded his expertise to encompass Android, Flutter, and React Native development across diverse platforms. Throughout his career, Owain has focused on software engineering excellence, with much of his work involving collaboration with financial companies managing large-scale codebases. He specialises in helping organisations evolve their mobile architecture to meet modern demands and scalability requirements.

Once a guest speaker at WWDC (it's less interesting than it sounds). Unashamed Apple fan boy, proud nerd, rugby watcher, father of two. Raised in Sunderland, North East resident (accent uncertain). Half Welsh, but using the English pronunciation.

Transcript

1. Visual regression testing in mobile From failing tests to shining UI — Gonna be, gonna be golden. X 23rd October 2025
2. Introduction Leeds mobile x hedgehog lab 2 Let’s learn something awesome together! Your presentation team today: Owain Brown - Engineering Manager Julien Le Pocher (in gallic spirit) - Flutter lead
3. Visual Regression Testing Leeds mobile x hedgehog lab 3 aka Snapshot Testing aka Golden Testing
4. Leeds mobile x hedgehog lab 4 Visual Regression Testing You're shipping mobile applications where visual bugs slip through manual testing and break user experiences. Visual regression testing automatically captures and compares UI snapshots to detect unintended changes.
5. Leeds mobile x hedgehog lab 5 End to End Integration Accessibility Performance Component Unit Testing approaches How do we achieve quality at every level?
6. Leeds mobile x hedgehog lab 6 Type AKA Contract Assert Unit iOS: XCTest Android: JUnit, Roboelectric Flutter: test package Individual functions, classes, or methods in isolation Return values match expected, Exceptions thrown correctly, State changes as intended Component iOS: XCTest Android: Roboelectric Flutter: Widget Tests Individual UI components in isolation with mocked external dependencies Component renders correctly, Props/inputs handled properly, Events fired on interaction Integration iOS: XCTest Android: Instrumented Tests Flutter: Integration Tests Multiple components working together, data flow between services Data transforms correctly, APIs return expected responses, State updates propagate End to End / UI iOS: XCUITest Android: UIAutomator, Espresso Flutter: Integration Tests + Driver Complete user journeys across the entire app with real backend/services User can complete workflows, Navigation works correctly, Real data integrates properly Low level High level
7. Leeds mobile x hedgehog lab 7 Test pyramid - Android UI Integration Espresso Instrumented Unit JUnit, Roboelectric
8. Leeds mobile x hedgehog lab 8 Test pyramid - iOS (or other Apple) UI Unit XCUITest XCTest, Swift Testing
9. Leeds mobile x hedgehog lab 9 Test pyramid - Flutter UI Component Integration Widget Unit
10. Leeds mobile x hedgehog lab 10 Quick aside - Android testing strategies
11. Leeds mobile x hedgehog lab 11 Type AKA Contract Assert Performance iOS: Instruments Profiling Android: Systrace, GPU Profiler Flutter: Observatory, DevTools App responsiveness, memory usage, battery consumption Execution time < threshold Memory usage < limit FPS > target Accessibility iOS: Accessibility Inspector Android: Accessibility Scanner Flutter: Semantics Testing App usability for users with disabilities Screen readers work, Contrast ratios meet standards, Touch targets are appropriately sized
12. Leeds mobile x hedgehog lab 12 Golden Testing: Flutter's Visual Regression Solution Flutter golden tests detect UI regressions automatically, maintaining quality during rapid development cycles Cross-platform - Ensure consistency across devices Fast iterations - Quick feedback during development Reliable framework - Built into Flutter's testing toolkit Active community - Rich ecosystem of supporting packages
13. Leeds Mobile x hedgehog lab 13 Goldens in the Test Pyramid UI Component Golden Unit
14. Leeds Mobile x hedgehog lab VRT Testing 14 Step by Step 1. Define 2. Arrange and inject 3. Perform required interactions 4. Capture 5. Persist 6. Compare 7. Review
15. Leeds Mobile x hedgehog lab Pexels 15 Tech test
16. Leeds Mobile x hedgehog lab Pexels 16 Example Golden 1. Define 2. Arrange and inject 3. Perform required interactions 4. Capture 5. Persist 6. Compare 7. Review
17. Leeds Mobile x hedgehog lab Pexels 17 Golden artifacts
18. Leeds Mobile x hedgehog lab Pexels 18 Resized Goldens
19. 19 Leeds Mobile x hedgehog lab Failure
20. 20 Leeds Mobile x hedgehog lab Masked vs Isolated
21. 21 Leeds Mobile x hedgehog lab Enhanced Goldens with Alchemist
22. Shadow rendering Multi-platform tests Leeds Mobile x hedgehog lab 22 22 Further Alchemy CI Tests Headless environment support Elevation accuracy Grouped Test Scenarios Customizable borders & backgrounds Test theming Multi-OS goldens Font rendering Typography consistency Gesture simulation Theme & locale permutations Test variants Grid layout organization Interactive state testing
23. Leeds Mobile x hedgehog lab Pexels 23 Again (SwiftUI)
24. 24 Leeds Mobile x hedgehog lab Snapshots
25. Leeds Mobile x hedgehog lab Pros 25 • Automatically catch UI regressions • Can be combined with interaction tests for deeper coverage • Rapid feedback • Low investment setup* • Living documentation, facilitates cross-team collaboration • Validates the UI as users see it • Ensures responsive, cross-device, and environment consistency • Changes are immediately visible in PRS
26. Leeds Mobile x hedgehog lab Cons 26 • Tests can suffer from noise • Large test artifacts in repository • Visual diffs are most effective with specialised tooling • Poor fit for dynamic interactions • Broad assertions • Must account for platform inconsistencies • Platforms use more indirect and adaptive rendering
27. Leeds Mobile x hedgehog lab Maximizing Value 27 • Consider your tolerance • Mirror tests, golden, and code organization • Invest in tooling • Use clear golden update and review processes
28. Leeds Mobile x hedgehog lab Maximizing Value 28 • Test multiple variants systematically • Consider careful fidelity vs consistency when designing tests ◦ Value in both whole-tree and component testing • Use whole test pyramid
29. A short story: Testing Expectations vs reality (Flutter) Leeds Mobile x hedgehog lab 29 Expected Effective
30. Q & A Leeds Mobile x hedgehog lab 30
31. Leeds Mobile x hedgehog lab 31