June Flutter Meetup - Golden Testing by Nicolas

Transcript

1. Hands on: Golden Testing in Flutter Montreal
2. Nicolas Cuillery Software engineer Has worked with: - Flutter - React Native - React - Angular.js - Flex - jQuery - JSP
3. Nicolas Cuillery Software engineer Has worked with: - Flutter - React Native - React - Angular.js - Flex - jQuery - JSP
4. Mobile Platform Specialized in eCommerce • Mobile development accelerator • Modular solution (catalog, search, i18n, message center, ...) • Multiple backend integration • Admin Console
5. What is Golden Testing?
6. What is Golden Testing? Widget
7. What is Golden Testing? Widget Image
8. What is Golden Testing? Widget Golden image
9. What is Golden Testing? Widget Golden image Updated widget
10. What is Golden Testing? Widget Golden image Updated widget Image
11. What is Golden Testing? Widget Golden image Updated widget Same image
12. What is Golden Testing? Widget Golden image Updated widget Different image
13. What is Golden Testing? Widget Golden image Updated widget Different image Unexpected
14. What is Golden Testing? Widget Golden image Updated widget Different image Unexpected Expected New golden image
15. Demo
16. Developer eXperience
17. Low verbosity
18. Visual feedback > Text feedback
19. Typical widget test error
20. Visual information > Text information
22. Icons reference Custom library of icons made with https://www.fluttericon.com Generates: • Dart class with static consts • Font file None of those files are self-descriptive
23. Icons reference
24. Resources
25. 3rd-party packages https://pub.dev/packages/golden_toolkit • Made and maintained by eBay • Widget wrapper to test different: ◦ parameters ◦ screen sizes ◦ Text scale • Font loading helpers https://tech.ebayinc.com/engineering/ebay-motors-screenshot-testing-with-flutter/
26. 3rd-party packages https://pub.dev/packages/alchemist • Directly inspired from the previous one • Advanced integration with Flutter SDK ◦ Theme injection • Platform aware ◦ CI-only test
27. Flutter SDK • Golden testing are at the core of the SDK testing workflow • ~1300 golden test cases • Guarantee of: ◦ Maturity ◦ Reliability ◦ Maintenance over time • Custom web app for results and difference analysis: https://flutter-gold.skia.org
28. Image loading • Asset images needs to be pre-cached ◦ https://github.com/flutter/flutter/issues/38997 • Network images ◦ Mock package: https://pub.dev/packages/network_image_mock ◦ Alternative solution with cached_network_image: https://medium.com/flutter-community/golden-testing-using-cachednetworkimage-1b488 c653af3
29. Use Cases
30. General purpose • UI regression testing • Responsiveness and adjustability • Test multiple variants at once
31. Edge case: business logic Golden test on the entire form
32. Edge case: business logic Golden test on the validation
33. Edge case: business logic • No longer a “unit test” • Still easy to debug • Quick to implement
34. Edge case: code gen Call the widget constructor from a JSON object
35. Edge case: code gen For each fixture file (xxxx.dart): 1. Read the golden xxxx.g.dart 2. Run the code generation on xxxx.dart 3. Compare the output w/ the golden file content 4. Eventually overwrite xxxx.g.dart (i.e. update golden)
36. Limitations
37. Limitations • Unique font weight for the same font family
38. Limitations • Text render differently across test execution platforms • Mitigation: ◦ Ahem font ◦ Custom threshold
39. Limitations • PlatformView need to be mocked (and display blank)
40. Thanks for listening ;)