June Flutter Meetup - Golden Testing by Nicolas

Transcript

1. Hands on:
   Golden Testing in
   Flutter
   Montreal
   

   View Slide

2. Nicolas Cuillery
   Software engineer
   Has worked with:
   - Flutter
   - React Native
   - React
   - Angular.js
   - Flex
   - jQuery
   - JSP
   

   View Slide

3. Nicolas Cuillery
   Software engineer
   Has worked with:
   - Flutter
   - React Native
   - React
   - Angular.js
   - Flex
   - jQuery
   - JSP
   

   View Slide

4. Mobile Platform
   Specialized in eCommerce
   ● Mobile development accelerator
   ● Modular solution (catalog, search, i18n,
   message center, ...)
   ● Multiple backend integration
   ● Admin Console
   

   View Slide

5. What is Golden
   Testing?
   

   View Slide

6. What is Golden Testing?
   Widget
   

   View Slide

7. What is Golden Testing?
   Widget Image
   

   View Slide

8. What is Golden Testing?
   Widget
   Golden
   image
   

   View Slide

9. What is Golden Testing?
   Widget
   Golden
   image
   Updated
   widget
   

   View Slide

10. What is Golden Testing?
    Widget
    Golden
    image
    Updated
    widget
    Image
    

    View Slide

11. What is Golden Testing?
    Widget
    Golden
    image
    Updated
    widget
    Same image
    

    View Slide

12. What is Golden Testing?
    Widget
    Golden
    image
    Updated
    widget
    Different
    image
    

    View Slide

13. What is Golden Testing?
    Widget
    Golden
    image
    Updated
    widget
    Different
    image
    Unexpected
    

    View Slide

14. What is Golden Testing?
    Widget
    Golden
    image
    Updated
    widget
    Different
    image
    Unexpected
    Expected
    New golden
    image
    

    View Slide

15. Demo
    

    View Slide

16. Developer
    eXperience
    

    View Slide

17. Low verbosity
    

    View Slide

18. Visual feedback > Text feedback
    

    View Slide

19. Typical widget test error
    

    View Slide

20. Visual information > Text information
    

    View Slide

21. View Slide

22. Icons reference
    Custom library of icons made with https://www.fluttericon.com
    Generates:
    ● Dart class with static consts
    ● Font file
    None of those files are self-descriptive
    

    View Slide

23. Icons reference
    

    View Slide

24. Resources
    

    View Slide

25. 3rd-party packages
    https://pub.dev/packages/golden_toolkit
    ● Made and maintained by eBay
    ● Widget wrapper to test different:
    ○ parameters
    ○ screen sizes
    ○ Text scale
    ● Font loading helpers
    https://tech.ebayinc.com/engineering/ebay-motors-screenshot-testing-with-flutter/
    

    View Slide

26. 3rd-party packages
    https://pub.dev/packages/alchemist
    ● Directly inspired from the previous one
    ● Advanced integration with Flutter SDK
    ○ Theme injection
    ● Platform aware
    ○ CI-only test
    

    View Slide

27. Flutter SDK
    ● Golden testing are at the core of the SDK testing workflow
    ● ~1300 golden test cases
    ● Guarantee of:
    ○ Maturity
    ○ Reliability
    ○ Maintenance over time
    ● Custom web app for results and difference analysis: https://flutter-gold.skia.org
    

    View Slide

28. Image loading
    ● Asset images needs to be pre-cached
    ○ https://github.com/flutter/flutter/issues/38997
    ● Network images
    ○ Mock package: https://pub.dev/packages/network_image_mock
    ○ Alternative solution with cached_network_image:
    https://medium.com/flutter-community/golden-testing-using-cachednetworkimage-1b488
    c653af3
    

    View Slide

29. Use Cases
    

    View Slide

30. General purpose
    ● UI regression testing
    ● Responsiveness and adjustability
    ● Test multiple variants at once
    

    View Slide

31. Edge case: business logic
    Golden test on the entire form
    

    View Slide

32. Edge case: business logic
    Golden test on the validation
    

    View Slide

33. Edge case: business logic
    ● No longer a “unit test”
    ● Still easy to debug
    ● Quick to implement
    

    View Slide

34. Edge case: code gen
    Call the widget constructor from a
    JSON object
    

    View Slide

35. Edge case: code gen
    For each fixture file (xxxx.dart):
    1. Read the golden xxxx.g.dart
    2. Run the code generation on
    xxxx.dart
    3. Compare the output w/ the
    golden file content
    4. Eventually overwrite
    xxxx.g.dart (i.e. update
    golden)
    

    View Slide

36. Limitations
    

    View Slide

37. Limitations
    ● Unique font weight for the same font family
    

    View Slide

38. Limitations
    ● Text render differently across test execution platforms
    ● Mitigation:
    ○ Ahem font
    ○ Custom threshold
    

    View Slide

39. Limitations
    ● PlatformView need to be mocked (and display blank)
    

    View Slide

40. Thanks for listening ;)
    

    View Slide