flutter_kaigi_2021.pdf

Transcript

1. UIΧλϩάΞϓϦͰ࣮ݱ͢Δ Visual Regression Testing FlutterKaigi 2021/11/30
2. About Me ҏ౻ ګฏ גࣜձࣾαΠόʔΤʔδΣϯτ Github: KyoheiG3 Twitter: KyoheiG3
3. • ֓ཁ • UI ΧλϩάΞϓϦ • ը໘Ωϟϓνϟ • ςετ ΞδΣϯμ
4. • ֓ཁ • UI ΧλϩάΞϓϦ • ը໘Ωϟϓνϟ • ςετ ΞδΣϯμ
5. • Unit Testing • UI Testing • Snapshot Testing • E2E Testing ςετͱ͸
6. Golden Testing • ༧Ί༻ҙ͓͍ͯͨ͠Ϛελʔը૾ͱͷࠩ෼Λൺֱ͢Δςετ • ࠩ෼͕͋Ε͹ςετࣦഊ • UI ͷมߋ఺Λ໨ࢹͰ֬ೝ͢Δ΋ͷͰ͸ͳ͍
7. Golden Testing void main() { testWidgets('Snapshot for MyApp', (tester) async { const app = MyApp(); // ΩϟϓνϟαΠζΛઃఆ await tester.binding.setSurfaceSize(const Size(414, 896)); await tester.pumpWidget(app); // Ϛελʔεφοϓγϣοτͱͷࠩ෼ൺֱ await expectLater( find.byWidget(app), matchesGoldenFile('MyApp.png'), ); }); }
8. Golden Testing $ flutter test 00:03 +1: All tests passed!
9. Golden Testing mainAxisAlignment: MainAxisAlignment.center, children: <Widget>[ const Text( 'You have pushed the button this many times:', ), Text( '$_counter', style: Theme.of(context).textTheme.headline4, ), ], ※ϓϩδΣΫτ࡞੒࣌ʹ࡞ΒΕΔίʔυ͔Βൈਮ
10. Golden Testing mainAxisAlignment: MainAxisAlignment.start, children: <Widget>[ const Text( 'You have pushed the button this many times:', ), Text( '$_counter', style: Theme.of(context).textTheme.headline4, ), ], ※ϓϩδΣΫτ࡞੒࣌ʹ࡞ΒΕΔίʔυ͔Βൈਮ DFOUFS
    Λ
    TUBSU
    ʹมߋͯ͠ςετͯ͠ΈΔ
11. Golden Testing $ flutter test 00:04 +0: Snapshot for MyApp 00:04 +0 -1: Snapshot for MyApp [E] Test failed. See exception logs above. The test description was: Snapshot for MyApp The test description was: Snapshot for MyApp 00:04 +0 -1: Some tests failed.
12. Golden Testing ςετ࣮ߦը૾ ࠩ෼ը૾ Ϛελʔը૾
13. Golden Testing $ flutter test --update-goldens
14. Visual Regression Testing • ը૾ճؼςετ • ը૾ͷࠩ෼Λݕग़͢Δεφοϓγϣοτςετͷ ͻͱͭ • UI ͕༧ظͤͣมߋ͞Ε͍ͯͳ͍͔Λ͔֬ΊΔͷ ʹඇৗʹ༗༻
15. reg-suit
16. reg-suit Blend Toggle Slide
17. • reg-keygen-git-hash-plugin • reg-notify-github-plugin • reg-notify-slack-plugin • reg-publish-s3-plugin • reg-publish-gcs-plugin reg-suit
18. reg-suit ࠩ෼ͷαϚϦʔΛ௨஌ͯ͘͠ΕΔ "QQSPWF
    ͞ΕΔ·ͰϚʔδͰ͖ͳ͍
19. reg-suit $ yarn add reg-suit $ yarn reg-suit init --use-yarn ? Plugin(s) to install (bold: recommended) (Press <space> to select, <a> to toggle all, <i> to invert selection) ? Working directory of reg-suit. .reg ? Append ".reg" entry to your .gitignore file. Yes ? Directory contains actual images. catalog_app/test/screenshots ? Threshold, ranges from 0 to 1. Smaller value makes the comparison more sensitive. 0 ? notify-github plugin requires a client ID of reg-suit GitHub app. Open installation window in your browser Yes ? This repositoriy's client ID of reg-suit GitHub app ? Create a new GCS bucket No ? Existing bucket name ? Update configuration file Yes ? Copy sample images to working dir No
20. 🤔
21. • ֓ཁ • UI ΧλϩάΞϓϦ • ը໘Ωϟϓνϟ • ςετ ΞδΣϯμ
22. UI ΧλϩάΞϓϦ
23. • playbook-ui (playbook- fl utter) • ݩʑฐࣾͷ OSS ͱͯ͠ଘࡏ͍ͯ͠Δ iOS ൛ͷ΋ͷΛ Flutter Խ UI ΧλϩάΞϓϦ
24. • PlaybookGallery • Playbook • Story • Scenario playbook-ui (playbook- fl utter)
25. playbook-ui (playbook- fl utter) @override Widget build(BuildContext context) { return MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]), ]), ), ); }
26. playbook-ui (playbook- fl utter) @override Widget build(BuildContext context) { return MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]), ]), ), ); } Playbook ΠϯελϯεΛड͚औΓΧλϩάΞϓϦશମΛߏங͢Δ
27. playbook-ui (playbook- fl utter) @override Widget build(BuildContext context) { return MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]), ]), ), ); } ड͚औͬͨ Story ෼ͷϦετ͕ PlaybookGallery ʹදࣔ͞ΕΔ
28. playbook-ui (playbook- fl utter) @override Widget build(BuildContext context) { return MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]), ]), ), ); } ड͚औͬͨ Scenario ෼ͷϦετ͕ Story ͷ Row ʹදࣔ͞ΕΔ
29. playbook-ui (playbook- fl utter) @override Widget build(BuildContext context) { return MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]), ]), ), ); } ड͚औͬͨ Widget ͷදࣔʹؔ͢Δ৘ใΛઃఆͰ͖Δ
30. playbook-ui (playbook- fl utter) @override Widget build(BuildContext context) { return MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]), ]), ), ); } ಠࣗʹ࣮૷ͨ͠ Widget ͳͲΛ౉͢͜ͱͰදࣔ͞ΕΔ
32. playbook-ui (playbook- fl utter) name: simple_app dependencies: flutter: sdk: flutter ϓϩμΫτΞϓϦߏ੒ྫ
33. playbook-ui (playbook- fl utter) ΧλϩάΞϓϦΛ௥Ճ
34. playbook-ui (playbook- fl utter) name: simple_catalog_app dependencies: flutter: sdk: flutter simple_app: path: ../simple_app playbook_ui: playbook: UI ΧλϩάΞϓϦߏ੒ྫ ϓϩμΫτͷΞϓϦΛґଘʹ௥Ճ
35. ґଘΛݮΒ͢ํ๏ playbook-ui (playbook- fl utter)
36. playbook-ui (playbook- fl utter) ίϯϙʔωϯτΛ௥Ճ
37. playbook-ui (playbook- fl utter) name: component dependencies: flutter: sdk: flutter UI ίϯϙʔωϯτύοέʔδԽྫ දࣔʹඞཁͳॲཧ͚ͩͷύοέʔδΛ࡞੒
38. playbook-ui (playbook- fl utter) name: simple_app dependencies: flutter: sdk: flutter component: path: '../component' σόΠεઐ༻ͷґଘͳͲ΋ΞϓϦଆʹهड़͠ඞཁʹԠͯ͡ίϯϙʔωϯτʹ DI ࡞੒ͨ͠
    6*
    ίϯϙʔωϯτΛ௥Ճ
39. playbook-ui (playbook- fl utter) name: simple_catalog_app dependencies: flutter: sdk: flutter component: path: '../component' playbook_ui: playbook: ಉ͘͡
    6*
    ίϯϙʔωϯτΛ௥Ճ දࣔʹඞཁͳίϯϙʔωϯτ͚ͩΛ import ͢Δ͜ͱͰϏϧυ͕࣌ؒ୹ॖ
40. playbook-ui (playbook- fl utter) void main() { runApp(MyApp()); } class MyApp extends StatelessWidget { @override Widget build(BuildContext context) { return MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: ..., ), ]) ]), ), ); } } UI ΧλϩάΞϓϦ͕׬੒
41. • ֓ཁ • UI ΧλϩάΞϓϦ • ը໘Ωϟϓνϟ • ςετ ΞδΣϯμ
42. • ൺֱݩͷίϯϙʔωϯτͷΩϟϓνϟ • ൺֱର৅ͷίϯϙʔωϯτͷΩϟϓνϟ ը໘Ωϟϓνϟ
43. • fl utter_test matchesGoldenFile function • golden_toolkit (ebay) • playbook-snapshot (playbook- fl utter) ը໘Ωϟϓνϟ
44. • playbook-snapshot (playbook- fl utter) • playbook-ui Ͱ࡞ͬͨίϯϙʔωϯτ ͷΩϟϓνϟΛαΫοͱࡱΕΔ ը໘Ωϟϓνϟ
45. • Playbook extension • Snapshot playbook-snapshot (playbook- fl utter)
46. Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]).run( Snapshot( directoryPath: 'screenshots', devices: [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); playbook-snapshot (playbook- fl utter)
47. Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]).run( Snapshot( directoryPath: 'screenshots', devices: [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); playbook-snapshot (playbook- fl utter) 1MBZCPPL
    ʹ
    SVO
    ϝιου͕௥Ճ͞ΕΔ
48. Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]).run( Snapshot( directoryPath: 'screenshots', devices: [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); playbook-snapshot (playbook- fl utter) 4DFOBSJP
    ͕͍࣋ͬͯΔ
    XJEHFU
    ͕౉͞ΕΔ ΩϟϓνϟΛࡱΓ͍ͨ
    8JEHFU
49. Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]).run( Snapshot( directoryPath: 'screenshots', devices: [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); playbook-snapshot (playbook- fl utter) Snapshot ઃఆྫ
50. Future<void> main() async { await Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]).run( Snapshot( directoryPath: 'screenshots', devices: [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); } playbook-snapshot (playbook- fl utter)
51. Future<void> main() async { await Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]).run( Snapshot( directoryPath: 'screenshots', devices: [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); } playbook-snapshot (playbook- fl utter) ͜ͷ෦෼͸ΞϓϦ΋ςετ΋ڞ௨
52. Playbook playbook() { return Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]); } playbook-snapshot (playbook- fl utter)
53. void main() { runApp(MyApp()); } class MyApp extends StatelessWidget { @override Widget build(BuildContext context) { return MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: playbook(), ), ); } } playbook-snapshot (playbook- fl utter)
54. Future<void> main() async { await playbook().run( Snapshot( directoryPath: 'screenshots', devices: [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); } playbook-snapshot (playbook- fl utter)
55. • தͰ͸ fl utter_test ͷ matchesGoldenFile Λ࣮ߦ͍ͯ͠Δ • ϑΥϯτ΍ΞΠίϯ͕ಡΈࠐΊͳ͍໰୊ͷαϙʔτ playbook-snapshot (playbook- fl utter) ৄ͘͠͸ 
 https://speakerdeck.com/tomokitakahashi/shi-jian- fl utter-visual-regression-testing
56. name: simple_catalog_app flutter: fonts: - family: Roboto fonts: - asset: fonts/Roboto-Regular.ttf playbook-snapshot (playbook- fl utter) Ωϟϓνϟ࣌ͷϑΥϯτΛ௥Ճ͢Δ
57. • ඇಉظॲཧ • ௨৴σʔλʢը૾ʣ • ࣌ؒ • Ξχϝʔγϣϯ • etc… DI
58. Future<void> main() async { await playbook().run( Snapshot( directoryPath: 'screenshots', devices: [SnapshotDevice.iPhone8], ), (widget) { return ProviderScope( overrides: [], MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ), ); }, ); } DI ͜ͷ͋ͨΓͰ
    %*
    Մೳ
59. • Story ࡞੒ͷ؆ུԽ • Scenario ࡞੒༻ͷΞϊςʔγϣϯ • Playbook ͷΠϯελϯεࣗಈੜ੒ • build_runner Ͱ࣮ߦՄೳ playbook-generator (playbook- fl utter)
60. Playbook playbook() { return Playbook(stories: [ Story('Sample Widget', scenarios: [ Scenario( 'Sample Scenario', child: Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ), ), ]) ]); } playbook-generator (playbook- fl utter) Before
61. const storyTitle = 'Sample Widget'; @GenerateScenario(title: 'Sample Scenario') Widget sampleScenario() => Center( child: Container( width: 200, height: 250, color: Colors.amber, alignment: Alignment.center, child: const Text('Hello World'), ), ); playbook-generator (playbook- fl utter) After
62. void main() { runApp(MyApp()); } class MyApp extends StatelessWidget { @override Widget build(BuildContext context) { return MaterialApp( title: 'Playbook Demo', theme: ThemeData.light(), home: PlaybookGallery( title: 'Sample App', playbook: playbook, ), ); } } playbook-generator (playbook- fl utter) ม਺
    HFUUFS
    ͕ࣗಈੜ੒͞Ε͍ͯΔ
63. Future<void> main() async { await playbook.run( Snapshot( directoryPath: 'screenshots', devices: [SnapshotDevice.iPhone8], ), (widget) { return MaterialApp( debugShowCheckedModeBanner: false, home: Material(child: widget), ); }, ); } playbook-generator (playbook- fl utter) ม਺
    HFUUFS
    ͕ࣗಈੜ੒͞Ε͍ͯΔ
64. • ֓ཁ • UI ΧλϩάΞϓϦ • ը໘Ωϟϓνϟ • ςετ ΞδΣϯμ
65. • PR ࣌ʹ VRT ͕࣮ߦ͞ΕΔΑ͏ʹ CI Ͱઃఆ͢Δ • ௨஌ܥͷϓϥάΠϯΛ࢖͑͹ςετ݁ՌΛ௨஌ͯ͘͠ΕΔ • ϨϏϡʔޙʹ Approve ͢Δ·ͰϚʔδͰ͖ͳͨ͘͠ΓͰ͖Δ ςετ
66. { "dependencies": { "reg-suit": "^0.11.1" }, "devDependencies": { "reg-keygen-git-hash-plugin": "^0.11.1", "reg-notify-github-plugin": "^0.11.1", "reg-publish-gcs-plugin": "^0.11.1" }, "scripts": { "regression": "reg-suit run" } } CI SFHTVJU
    Λ࣮ߦ͢ΔͨΊͷઃఆΛ௥Ճ
67. version: 2.1 orbs: android: circleci/android@1.0.3 flutter: circleci/flutter@1.0.0 node: circleci/node@4.7.0 jobs: vrt: executor: android/android steps: - checkout - node/install: install-yarn: true - flutter/install_sdk: - run: name: Install melos command: | dart pub global activate melos melos run pub:get - run: name: Run Golden Testing command: melos run test:snapshot - run: name: Install dependencies command: yarn - run: name: Run VRT command: yarn regression workflows: vrt: jobs: - vrt CI
68. @override Widget build(BuildContext context) { return Container( color: Colors.amberAccent, child: Row( mainAxisAlignment: MainAxisAlignment.center, crossAxisAlignment: CrossAxisAlignment.center, children: [ Icon(Icons.star), SizedBox(width: 16), Text(text, style: Theme.of(context).textTheme.headline5) ], ), ); } ςετ
69. @override Widget build(BuildContext context) { return Container( color: Colors.amberAccent, child: Row( mainAxisAlignment: MainAxisAlignment.start, crossAxisAlignment: CrossAxisAlignment.center, children: [ Icon(Icons.star), SizedBox(width: 16), Text(text, style: Theme.of(context).textTheme.headline5) ], ), ); } ςετ ෆ۩߹͔൑அͮ͠Β͍
70. • ςετࣗମΛॻ࣌ؒ͘͸ͱʹ͔͘ݮΒ͍ͨ͠ • playbook_ui Λར༻ͯ͠ΩϟϓνϟͷࣗಈԽ • ࡞ͬͯյͯ͠Λ΋ͬͱؾܰʹߦ͑ΔΑ͏ʹͳΓ·͢ʂ ςετ
71. • https://github.com/playbook-ui/playbook- fl utter • https://github.com/Dropsource/monarch • https://github.com/widgetbook/widgetbook • https://github.com/ookami-kb/storybook_ fl utter • https://github.com/eBay/ fl utter_glove_box/tree/master/packages/ golden_toolkit ࢀߟ URL
72. Thanks !