$30 off During Our Annual Pro Sale. View Details »

June Flutter Meetup - Golden Testing by Nicolas

June Flutter Meetup - Golden Testing by Nicolas

GDG Montreal

June 19, 2023
Tweet

More Decks by GDG Montreal

Other Decks in Programming

Transcript

  1. Hands on:
    Golden Testing in
    Flutter
    Montreal

    View Slide

  2. Nicolas Cuillery
    Software engineer
    Has worked with:
    - Flutter
    - React Native
    - React
    - Angular.js
    - Flex
    - jQuery
    - JSP

    View Slide

  3. Nicolas Cuillery
    Software engineer
    Has worked with:
    - Flutter
    - React Native
    - React
    - Angular.js
    - Flex
    - jQuery
    - JSP

    View Slide

  4. Mobile Platform
    Specialized in eCommerce
    ● Mobile development accelerator
    ● Modular solution (catalog, search, i18n,
    message center, ...)
    ● Multiple backend integration
    ● Admin Console

    View Slide

  5. What is Golden
    Testing?

    View Slide

  6. What is Golden Testing?
    Widget

    View Slide

  7. What is Golden Testing?
    Widget Image

    View Slide

  8. What is Golden Testing?
    Widget
    Golden
    image

    View Slide

  9. What is Golden Testing?
    Widget
    Golden
    image
    Updated
    widget

    View Slide

  10. What is Golden Testing?
    Widget
    Golden
    image
    Updated
    widget
    Image

    View Slide

  11. What is Golden Testing?
    Widget
    Golden
    image
    Updated
    widget
    Same image

    View Slide

  12. What is Golden Testing?
    Widget
    Golden
    image
    Updated
    widget
    Different
    image

    View Slide

  13. What is Golden Testing?
    Widget
    Golden
    image
    Updated
    widget
    Different
    image
    Unexpected

    View Slide

  14. What is Golden Testing?
    Widget
    Golden
    image
    Updated
    widget
    Different
    image
    Unexpected
    Expected
    New golden
    image

    View Slide

  15. Demo

    View Slide

  16. Developer
    eXperience

    View Slide

  17. Low verbosity

    View Slide

  18. Visual feedback > Text feedback

    View Slide

  19. Typical widget test error

    View Slide

  20. Visual information > Text information

    View Slide

  21. View Slide

  22. Icons reference
    Custom library of icons made with https://www.fluttericon.com
    Generates:
    ● Dart class with static consts
    ● Font file
    None of those files are self-descriptive

    View Slide

  23. Icons reference

    View Slide

  24. Resources

    View Slide

  25. 3rd-party packages
    https://pub.dev/packages/golden_toolkit
    ● Made and maintained by eBay
    ● Widget wrapper to test different:
    ○ parameters
    ○ screen sizes
    ○ Text scale
    ● Font loading helpers
    https://tech.ebayinc.com/engineering/ebay-motors-screenshot-testing-with-flutter/

    View Slide

  26. 3rd-party packages
    https://pub.dev/packages/alchemist
    ● Directly inspired from the previous one
    ● Advanced integration with Flutter SDK
    ○ Theme injection
    ● Platform aware
    ○ CI-only test

    View Slide

  27. Flutter SDK
    ● Golden testing are at the core of the SDK testing workflow
    ● ~1300 golden test cases
    ● Guarantee of:
    ○ Maturity
    ○ Reliability
    ○ Maintenance over time
    ● Custom web app for results and difference analysis: https://flutter-gold.skia.org

    View Slide

  28. Image loading
    ● Asset images needs to be pre-cached
    ○ https://github.com/flutter/flutter/issues/38997
    ● Network images
    ○ Mock package: https://pub.dev/packages/network_image_mock
    ○ Alternative solution with cached_network_image:
    https://medium.com/flutter-community/golden-testing-using-cachednetworkimage-1b488
    c653af3

    View Slide

  29. Use Cases

    View Slide

  30. General purpose
    ● UI regression testing
    ● Responsiveness and adjustability
    ● Test multiple variants at once

    View Slide

  31. Edge case: business logic
    Golden test on the entire form

    View Slide

  32. Edge case: business logic
    Golden test on the validation

    View Slide

  33. Edge case: business logic
    ● No longer a “unit test”
    ● Still easy to debug
    ● Quick to implement

    View Slide

  34. Edge case: code gen
    Call the widget constructor from a
    JSON object

    View Slide

  35. Edge case: code gen
    For each fixture file (xxxx.dart):
    1. Read the golden xxxx.g.dart
    2. Run the code generation on
    xxxx.dart
    3. Compare the output w/ the
    golden file content
    4. Eventually overwrite
    xxxx.g.dart (i.e. update
    golden)

    View Slide

  36. Limitations

    View Slide

  37. Limitations
    ● Unique font weight for the same font family

    View Slide

  38. Limitations
    ● Text render differently across test execution platforms
    ● Mitigation:
    ○ Ahem font
    ○ Custom threshold

    View Slide

  39. Limitations
    ● PlatformView need to be mocked (and display blank)

    View Slide

  40. Thanks for listening ;)

    View Slide