flutter_kaigi_2021.pdf

Transcript

1. UIΧλϩάΞϓϦͰ࣮ݱ͢Δ Visual Regression Testing FlutterKaigi 2021/11/30

2. About Me ҏ౻ ګฏ גࣜձࣾαΠόʔΤʔδΣϯτ Github: KyoheiG3 Twitter: KyoheiG3

3. • ֓ཁ • UI ΧλϩάΞϓϦ • ը໘Ωϟϓνϟ • ςετ ΞδΣϯμ

4. • ֓ཁ • UI ΧλϩάΞϓϦ • ը໘Ωϟϓνϟ • ςετ ΞδΣϯμ

5. • Unit Testing • UI Testing • Snapshot Testing •

   E2E Testing ςετͱ͸

6. Golden Testing • ༧Ί༻ҙ͓͍ͯͨ͠Ϛελʔը૾ͱͷࠩ෼Λൺֱ͢Δςετ • ࠩ෼͕͋Ε͹ςετࣦഊ • UI ͷมߋ఺Λ໨ࢹͰ֬ೝ͢Δ΋ͷͰ͸ͳ͍

7. Golden Testing void main() { testWidgets('Snapshot for MyApp', (tester) async

   { const app = MyApp(); // ΩϟϓνϟαΠζΛઃఆ await tester.binding.setSurfaceSize(const Size(414, 896)); await tester.pumpWidget(app); // Ϛελʔεφοϓγϣοτͱͷࠩ෼ൺֱ await expectLater( find.byWidget(app), matchesGoldenFile('MyApp.png'), ); }); }

8. Golden Testing $ flutter test 00:03 +1: All tests passed!

9. Golden Testing mainAxisAlignment: MainAxisAlignment.center, children: <Widget>[ const Text( 'You have

   pushed the button this many times:', ), Text( '$_counter', style: Theme.of(context).textTheme.headline4, ), ], ※ϓϩδΣΫτ࡞੒࣌ʹ࡞ΒΕΔίʔυ͔Βൈਮ

10. Golden Testing mainAxisAlignment: MainAxisAlignment.start, children: <Widget>[ const Text( 'You have

    pushed the button this many times:', ), Text( '$_counter', style: Theme.of(context).textTheme.headline4, ), ], ※ϓϩδΣΫτ࡞੒࣌ʹ࡞ΒΕΔίʔυ͔Βൈਮ DFOUFS
    Λ
    TUBSU
    ʹมߋͯ͠ςετͯ͠ΈΔ

11. Golden Testing $ flutter test 00:04 +0: Snapshot for MyApp

    00:04 +0 -1: Snapshot for MyApp [E] Test failed. See exception logs above. The test description was: Snapshot for MyApp The test description was: Snapshot for MyApp 00:04 +0 -1: Some tests failed.

12. Golden Testing ςετ࣮ߦը૾ ࠩ෼ը૾ Ϛελʔը૾

13. Golden Testing $ flutter test --update-goldens

14. Visual Regression Testing • ը૾ճؼςετ • ը૾ͷࠩ෼Λݕग़͢Δεφοϓγϣοτςετͷ ͻͱͭ • UI

    ͕༧ظͤͣมߋ͞Ε͍ͯͳ͍͔Λ͔֬ΊΔͷ ʹඇৗʹ༗༻

15. reg-suit

16. reg-suit Blend Toggle Slide

17. • reg-keygen-git-hash-plugin • reg-notify-github-plugin • reg-notify-slack-plugin • reg-publish-s3-plugin • reg-publish-gcs-plugin

    reg-suit

18. reg-suit ࠩ෼ͷαϚϦʔΛ௨஌ͯ͘͠ΕΔ "QQSPWF
    ͞ΕΔ·ͰϚʔδͰ͖ͳ͍

19. reg-suit $ yarn add reg-suit $ yarn reg-suit init --use-yarn

    ? Plugin(s) to install (bold: recommended) (Press <space> to select, <a> to toggle all, <i> to invert selection) ? Working directory of reg-suit. .reg ? Append ".reg" entry to your .gitignore file. Yes ? Directory contains actual images. catalog_app/test/screenshots ? Threshold, ranges from 0 to 1. Smaller value makes the comparison more sensitive. 0 ? notify-github plugin requires a client ID of reg-suit GitHub app. Open installation window in your browser Yes ? This repositoriy's client ID of reg-suit GitHub app ? Create a new GCS bucket No ? Existing bucket name ? Update configuration file Yes ? Copy sample images to working dir No

20. 🤔

21. • ֓ཁ • UI ΧλϩάΞϓϦ • ը໘Ωϟϓνϟ • ςετ ΞδΣϯμ

22. UI ΧλϩάΞϓϦ

23. • playbook-ui (playbook- fl utter) • ݩʑฐࣾͷ OSS ͱͯ͠ଘࡏ͍ͯ͠Δ iOS

    ൛ͷ΋ͷΛ Flutter Խ UI ΧλϩάΞϓϦ

24. • PlaybookGallery • Playbook • Story • Scenario playbook-ui (playbook-

    fl utter)

25. playbook-ui (playbook- fl utter) @override Widget build(BuildContext context) { return

    MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]), ]), ), ); }

26. playbook-ui (playbook- fl utter) @override Widget build(BuildContext context) { return

    MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]), ]), ), ); } Playbook ΠϯελϯεΛड͚औΓΧλϩάΞϓϦશମΛߏங͢Δ

27. playbook-ui (playbook- fl utter) @override Widget build(BuildContext context) { return

    MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]), ]), ), ); } ड͚औͬͨ Story ෼ͷϦετ͕ PlaybookGallery ʹදࣔ͞ΕΔ

28. playbook-ui (playbook- fl utter) @override Widget build(BuildContext context) { return

    MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]), ]), ), ); } ड͚औͬͨ Scenario ෼ͷϦετ͕ Story ͷ Row ʹදࣔ͞ΕΔ

29. playbook-ui (playbook- fl utter) @override Widget build(BuildContext context) { return

    MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]), ]), ), ); } ड͚औͬͨ Widget ͷදࣔʹؔ͢Δ৘ใΛઃఆͰ͖Δ

30. playbook-ui (playbook- fl utter) @override Widget build(BuildContext context) { return

    MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]), ]), ), ); } ಠࣗʹ࣮૷ͨ͠ Widget ͳͲΛ౉͢͜ͱͰදࣔ͞ΕΔ
31. None

32. playbook-ui (playbook- fl utter) name: simple_app dependencies: flutter: sdk: flutter

    ϓϩμΫτΞϓϦߏ੒ྫ

33. playbook-ui (playbook- fl utter) ΧλϩάΞϓϦΛ௥Ճ

34. playbook-ui (playbook- fl utter) name: simple_catalog_app dependencies: flutter: sdk: flutter

    simple_app: path: ../simple_app playbook_ui: playbook: UI ΧλϩάΞϓϦߏ੒ྫ ϓϩμΫτͷΞϓϦΛґଘʹ௥Ճ

35. ґଘΛݮΒ͢ํ๏ playbook-ui (playbook- fl utter)

36. playbook-ui (playbook- fl utter) ίϯϙʔωϯτΛ௥Ճ

37. playbook-ui (playbook- fl utter) name: component dependencies: flutter: sdk: flutter

    UI ίϯϙʔωϯτύοέʔδԽྫ දࣔʹඞཁͳॲཧ͚ͩͷύοέʔδΛ࡞੒

38. playbook-ui (playbook- fl utter) name: simple_app dependencies: flutter: sdk: flutter

    component: path: '../component' σόΠεઐ༻ͷґଘͳͲ΋ΞϓϦଆʹهड़͠ඞཁʹԠͯ͡ίϯϙʔωϯτʹ DI ࡞੒ͨ͠
    6*
    ίϯϙʔωϯτΛ௥Ճ

39. playbook-ui (playbook- fl utter) name: simple_catalog_app dependencies: flutter: sdk: flutter

    component: path: '../component' playbook_ui: playbook: ಉ͘͡
    6*
    ίϯϙʔωϯτΛ௥Ճ දࣔʹඞཁͳίϯϙʔωϯτ͚ͩΛ import ͢Δ͜ͱͰϏϧυ͕࣌ؒ୹ॖ

40. playbook-ui (playbook- fl utter) void main() { runApp(MyApp()); } class

    MyApp extends StatelessWidget { @override Widget build(BuildContext context) { return MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: ..., ), ]) ]), ), ); } } UI ΧλϩάΞϓϦ͕׬੒

41. • ֓ཁ • UI ΧλϩάΞϓϦ • ը໘Ωϟϓνϟ • ςετ ΞδΣϯμ

42. • ൺֱݩͷίϯϙʔωϯτͷΩϟϓνϟ • ൺֱର৅ͷίϯϙʔωϯτͷΩϟϓνϟ ը໘Ωϟϓνϟ

43. • fl utter_test matchesGoldenFile function • golden_toolkit (ebay) • playbook-snapshot

    (playbook- fl utter) ը໘Ωϟϓνϟ

44. • playbook-snapshot (playbook- fl utter) • playbook-ui Ͱ࡞ͬͨίϯϙʔωϯτ ͷΩϟϓνϟΛαΫοͱࡱΕΔ ը໘Ωϟϓνϟ

45. • Playbook extension • Snapshot playbook-snapshot (playbook- fl utter)

46. Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child:

    Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]).run( Snapshot( directoryPath: 'screenshots', devices: [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); playbook-snapshot (playbook- fl utter)

47. Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child:

    Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]).run( Snapshot( directoryPath: 'screenshots', devices: [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); playbook-snapshot (playbook- fl utter) 1MBZCPPL
    ʹ
    SVO
    ϝιου͕௥Ճ͞ΕΔ

48. Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child:

    Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]).run( Snapshot( directoryPath: 'screenshots', devices: [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); playbook-snapshot (playbook- fl utter) 4DFOBSJP
    ͕͍࣋ͬͯΔ
    XJEHFU
    ͕౉͞ΕΔ ΩϟϓνϟΛࡱΓ͍ͨ
    8JEHFU

49. Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child:

    Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]).run( Snapshot( directoryPath: 'screenshots', devices: [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); playbook-snapshot (playbook- fl utter) Snapshot ઃఆྫ

50. Future<void> main() async { await Playbook(stories: [ Story('Sample Widget', scenarios:

    [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]).run( Snapshot( directoryPath: 'screenshots', devices: [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); } playbook-snapshot (playbook- fl utter)

51. Future<void> main() async { await Playbook(stories: [ Story('Sample Widget', scenarios:

    [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]).run( Snapshot( directoryPath: 'screenshots', devices: [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); } playbook-snapshot (playbook- fl utter) ͜ͷ෦෼͸ΞϓϦ΋ςετ΋ڞ௨

52. Playbook playbook() { return Playbook(stories: [ Story('Sample Widget', scenarios: [

    Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]); } playbook-snapshot (playbook- fl utter)

53. void main() { runApp(MyApp()); } class MyApp extends StatelessWidget {

    @override Widget build(BuildContext context) { return MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: playbook(), ), ); } } playbook-snapshot (playbook- fl utter)

54. Future<void> main() async { await playbook().run( Snapshot( directoryPath: 'screenshots', devices:

    [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); } playbook-snapshot (playbook- fl utter)

55. • தͰ͸ fl utter_test ͷ matchesGoldenFile Λ࣮ߦ͍ͯ͠Δ • ϑΥϯτ΍ΞΠίϯ͕ಡΈࠐΊͳ͍໰୊ͷαϙʔτ playbook-snapshot

    (playbook- fl utter) ৄ͘͠͸ 
 https://speakerdeck.com/tomokitakahashi/shi-jian- fl utter-visual-regression-testing

56. name: simple_catalog_app flutter: fonts: - family: Roboto fonts: - asset:

    fonts/Roboto-Regular.ttf playbook-snapshot (playbook- fl utter) Ωϟϓνϟ࣌ͷϑΥϯτΛ௥Ճ͢Δ

57. • ඇಉظॲཧ • ௨৴σʔλʢը૾ʣ • ࣌ؒ • Ξχϝʔγϣϯ • etc…

    DI

58. Future<void> main() async { await playbook().run( Snapshot( directoryPath: 'screenshots', devices:

    [SnapshotDevice.iPhone8], ), (widget) { return ProviderScope( overrides: [], MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ), ); }, ); } DI ͜ͷ͋ͨΓͰ
    %*
    Մೳ

59. • Story ࡞੒ͷ؆ུԽ • Scenario ࡞੒༻ͷΞϊςʔγϣϯ • Playbook ͷΠϯελϯεࣗಈੜ੒ •

    build_runner Ͱ࣮ߦՄೳ playbook-generator (playbook- fl utter)

60. Playbook playbook() { return Playbook(stories: [ Story('Sample Widget', scenarios: [

    Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]); } playbook-generator (playbook- fl utter) Before

61. const storyTitle = 'Sample Widget'; @GenerateScenario(title: 'Sample Scenario') Widget sampleScenario()

    => Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ); playbook-generator (playbook- fl utter) After

62. void main() { runApp(MyApp()); } class MyApp extends StatelessWidget {

    @override Widget build(BuildContext context) { return MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: playbook, ), ); } } playbook-generator (playbook- fl utter) ม਺
    HFUUFS
    ͕ࣗಈੜ੒͞Ε͍ͯΔ

63. Future<void> main() async { await playbook.run( Snapshot( directoryPath: 'screenshots', devices:

    [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); } playbook-generator (playbook- fl utter) ม਺
    HFUUFS
    ͕ࣗಈੜ੒͞Ε͍ͯΔ

64. • ֓ཁ • UI ΧλϩάΞϓϦ • ը໘Ωϟϓνϟ • ςετ ΞδΣϯμ

65. • PR ࣌ʹ VRT ͕࣮ߦ͞ΕΔΑ͏ʹ CI Ͱઃఆ͢Δ • ௨஌ܥͷϓϥάΠϯΛ࢖͑͹ςετ݁ՌΛ௨஌ͯ͘͠ΕΔ •

    ϨϏϡʔޙʹ Approve ͢Δ·ͰϚʔδͰ͖ͳͨ͘͠ΓͰ͖Δ ςετ

66. { "dependencies": { "reg-suit": "^0.11.1" }, "devDependencies": { "reg-keygen-git-hash-plugin": "^0.11.1",

    "reg-notify-github-plugin": "^0.11.1", "reg-publish-gcs-plugin": "^0.11.1" }, "scripts": { "regression": "reg-suit run" } } CI SFHTVJU
    Λ࣮ߦ͢ΔͨΊͷઃఆΛ௥Ճ

67. version: 2.1 orbs: android: circleci/android@1.0.3 flutter: circleci/flutter@1.0.0 node: circleci/node@4.7.0 jobs:

    vrt: executor: android/android steps: - checkout - node/install: install-yarn: true - flutter/install_sdk: - run: name: Install melos command: | dart pub global activate melos melos run pub:get - run: name: Run Golden Testing command: melos run test:snapshot - run: name: Install dependencies command: yarn - run: name: Run VRT command: yarn regression workflows: vrt: jobs: - vrt CI

68. @override Widget build(BuildContext context) { return Container( color: Colors.amberAccent, child:

    Row( mainAxisAlignment: MainAxisAlignment.center, crossAxisAlignment: CrossAxisAlignment.center, children: [ Icon(Icons.star), SizedBox(width: 16), Text(text, style: Theme.of(context).textTheme.headline5) ], ), ); } ςετ

69. @override Widget build(BuildContext context) { return Container( color: Colors.amberAccent, child:

    Row( mainAxisAlignment: MainAxisAlignment.start, crossAxisAlignment: CrossAxisAlignment.center, children: [ Icon(Icons.star), SizedBox(width: 16), Text(text, style: Theme.of(context).textTheme.headline5) ], ), ); } ςετ ෆ۩߹͔൑அͮ͠Β͍

70. • ςετࣗମΛॻ࣌ؒ͘͸ͱʹ͔͘ݮΒ͍ͨ͠ • playbook_ui Λར༻ͯ͠ΩϟϓνϟͷࣗಈԽ • ࡞ͬͯյͯ͠Λ΋ͬͱؾܰʹߦ͑ΔΑ͏ʹͳΓ·͢ʂ ςετ

71. • https://github.com/playbook-ui/playbook- fl utter • https://github.com/Dropsource/monarch • https://github.com/widgetbook/widgetbook • https://github.com/ookami-kb/storybook_

    fl utter • https://github.com/eBay/ fl utter_glove_box/tree/master/packages/ golden_toolkit ࢀߟ URL

72. Thanks !