flutter_kaigi_2021.pdf

UIΧλϩάΞϓϦͰ࣮ݱ͢Δ

Visual Regression Testing
FlutterKaigi 2021/11/30

View Slide

About Me
ҏ౻ ګฏ

גࣜձࣾαΠόʔΤʔδΣϯτ

Github: KyoheiG3

Twitter: KyoheiG3

View Slide

• ֓ཁ

• UI ΧλϩάΞϓϦ

• ը໘Ωϟϓνϟ

• ςετ
ΞδΣϯμ

View Slide

• Unit Testing

• UI Testing

• Snapshot Testing

• E2E Testing
ςετͱ͸

View Slide

Golden Testing
• ༧Ί༻ҙ͓͍ͯͨ͠Ϛελʔը૾ͱͷࠩ෼Λൺֱ͢Δςετ

• ࠩ෼͕͋Ε͹ςετࣦഊ

• UI ͷมߋ఺Λ໨ࢹͰ֬ೝ͢Δ΋ͷͰ͸ͳ͍

View Slide

Golden Testing
void main() {

testWidgets('Snapshot for MyApp', (tester) async {

const app = MyApp();

// ΩϟϓνϟαΠζΛઃఆ

await tester.binding.setSurfaceSize(const Size(414, 896));

await tester.pumpWidget(app);

// Ϛελʔεφοϓγϣοτͱͷࠩ෼ൺֱ

await expectLater(

find.byWidget(app),

matchesGoldenFile('MyApp.png'),

);

});

}

View Slide

Golden Testing
$ flutter test

00:03 +1: All tests passed!

View Slide

Golden Testing
mainAxisAlignment: MainAxisAlignment.center,

children: [

const Text(

'You have pushed the button this many times:',

),

Text(

'$_counter',

style: Theme.of(context).textTheme.headline4,

),

],
※ϓϩδΣΫτ࡞੒࣌ʹ࡞ΒΕΔίʔυ͔Βൈਮ

View Slide

Golden Testing
mainAxisAlignment: MainAxisAlignment.start,

children: [

const Text(

'You have pushed the button this many times:',

),

Text(

'$_counter',

style: Theme.of(context).textTheme.headline4,

),

],
※ϓϩδΣΫτ࡞੒࣌ʹ࡞ΒΕΔίʔυ͔Βൈਮ
DFOUFSΛTUBSUʹมߋͯ͠ςετͯ͠ΈΔ

View Slide

Golden Testing
$ flutter test

00:04 +0: Snapshot for MyApp

00:04 +0 -1: Snapshot for MyApp [E]

Test failed. See exception logs above.

The test description was: Snapshot for MyApp

The test description was: Snapshot for MyApp

00:04 +0 -1: Some tests failed.

View Slide

Golden Testing
ςετ࣮ߦը૾ ࠩ෼ը૾
Ϛελʔը૾

View Slide

Golden Testing
$ flutter test --update-goldens

View Slide

Visual Regression Testing
• ը૾ճؼςετ

• ը૾ͷࠩ෼Λݕग़͢Δεφοϓγϣοτςετͷ
ͻͱͭ

• UI ͕༧ظͤͣมߋ͞Ε͍ͯͳ͍͔Λ͔֬ΊΔͷ
ʹඇৗʹ༗༻

View Slide

reg-suit

View Slide

reg-suit
Blend Toggle
Slide

View Slide

• reg-keygen-git-hash-plugin

• reg-notify-github-plugin

• reg-notify-slack-plugin

• reg-publish-s3-plugin

• reg-publish-gcs-plugin
reg-suit

View Slide

reg-suit
ࠩ෼ͷαϚϦʔΛ௨஌ͯ͘͠ΕΔ
"QQSPWF͞ΕΔ·ͰϚʔδͰ͖ͳ͍

View Slide

reg-suit
$ yarn add reg-suit

$ yarn reg-suit init --use-yarn

? Plugin(s) to install (bold: recommended) (Press to select, to toggle all, to invert selection)

? Working directory of reg-suit. .reg

? Append ".reg" entry to your .gitignore file. Yes

? Directory contains actual images. catalog_app/test/screenshots

? Threshold, ranges from 0 to 1. Smaller value makes the comparison more sensitive. 0

? notify-github plugin requires a client ID of reg-suit GitHub app. Open installation window in your browser Yes

? This repositoriy's client ID of reg-suit GitHub app

? Create a new GCS bucket No

? Existing bucket name

? Update configuration file Yes

? Copy sample images to working dir No

View Slide

🤔

View Slide

• ֓ཁ


• ը໘Ωϟϓνϟ

• ςετ
ΞδΣϯμ

View Slide

UI ΧλϩάΞϓϦ

View Slide

• playbook-ui (playbook-
fl
utter)

• ݩʑฐࣾͷ OSS ͱͯ͠ଘࡏ͍ͯ͠Δ
iOS ൛ͷ΋ͷΛ Flutter Խ
UI ΧλϩάΞϓϦ

View Slide

• PlaybookGallery

• Playbook

• Story

• Scenario
playbook-ui (playbook-
fl
utter)

View Slide

fl
utter)
@override

Widget build(BuildContext context) {

return MaterialApp(

title: 'Playbook Demo',

theme: ThemeData.light(),

home: PlaybookGallery(

title: 'Sample App',

playbook: Playbook(stories: [

Story('Sample Widget', scenarios: [

Scenario(

'Sample Scenario',

child: Center(

child: Container(

width: 200,

height: 250,

color: Colors.amber,

alignment: Alignment.center,

child: const Text('Hello World'),

),

),

),

]),

]),

),

);

}

View Slide

fl
utter)
@override


return MaterialApp(







Scenario(

'Sample Scenario',

child: Center(

child: Container(

width: 200,

height: 250,




),

),

),

]),

]),

),

);

}
Playbook ΠϯελϯεΛड͚औΓΧλϩάΞϓϦશମΛߏங͢Δ

View Slide

fl
utter)
@override


return MaterialApp(







Scenario(

'Sample Scenario',

child: Center(

child: Container(

width: 200,

height: 250,




),

),

),

]),

]),

),

);

}
ड͚औͬͨ Story ෼ͷϦετ͕ PlaybookGallery ʹදࣔ͞ΕΔ

View Slide

fl
utter)
@override


return MaterialApp(







Scenario(

'Sample Scenario',

child: Center(

child: Container(

width: 200,

height: 250,




),

),

),

]),

]),

),

);

}
ड͚औͬͨ Scenario ෼ͷϦετ͕ Story ͷ Row ʹදࣔ͞ΕΔ

View Slide

fl
utter)
@override


return MaterialApp(







Scenario(

'Sample Scenario',

child: Center(

child: Container(

width: 200,

height: 250,




),

),

),

]),

]),

),

);

}
ड͚औͬͨ Widget ͷදࣔʹؔ͢Δ৘ใΛઃఆͰ͖Δ

View Slide

fl
utter)
@override


return MaterialApp(







Scenario(

'Sample Scenario',

child: Center(

child: Container(

width: 200,

height: 250,




),

),

),

]),

]),

),

);

}
ಠࣗʹ࣮૷ͨ͠ Widget ͳͲΛ౉͢͜ͱͰදࣔ͞ΕΔ

View Slide

View Slide

fl
utter)
name: simple_app

dependencies:

flutter:

sdk: flutter
ϓϩμΫτΞϓϦߏ੒ྫ

View Slide

fl
utter)
ΧλϩάΞϓϦΛ௥Ճ

View Slide

fl
utter)
name: simple_catalog_app

dependencies:

flutter:

sdk: flutter

simple_app:

path: ../simple_app

playbook_ui:

playbook:
UI ΧλϩάΞϓϦߏ੒ྫ
ϓϩμΫτͷΞϓϦΛґଘʹ௥Ճ

View Slide

ґଘΛݮΒ͢ํ๏
fl
utter)

View Slide

fl
utter)
ίϯϙʔωϯτΛ௥Ճ

View Slide

fl
utter)
name: component

dependencies:

flutter:

sdk: flutter
UI ίϯϙʔωϯτύοέʔδԽྫ
දࣔʹඞཁͳॲཧ͚ͩͷύοέʔδΛ࡞੒

View Slide

fl
utter)
name: simple_app

dependencies:

flutter:

sdk: flutter

component:

path: '../component'

σόΠεઐ༻ͷґଘͳͲ΋ΞϓϦଆʹهड़͠ඞཁʹԠͯ͡ίϯϙʔωϯτʹ DI
࡞੒ͨ͠6*ίϯϙʔωϯτΛ௥Ճ

View Slide

fl
utter)

dependencies:

flutter:

sdk: flutter

component:

path: '../component'

playbook_ui:

playbook:
ಉ͘͡6*ίϯϙʔωϯτΛ௥Ճ
දࣔʹඞཁͳίϯϙʔωϯτ͚ͩΛ import ͢Δ͜ͱͰϏϧυ͕࣌ؒ୹ॖ

View Slide

fl
utter)
void main() {

runApp(MyApp());

}

class MyApp extends StatelessWidget {

@override


return MaterialApp(







Scenario(

'Sample Scenario',

child: ...,

),

])

]),

),

);

}

}
UI ΧλϩάΞϓϦ͕׬੒

View Slide

• ֓ཁ


• ը໘Ωϟϓνϟ

• ςετ
ΞδΣϯμ

View Slide

• ൺֱݩͷίϯϙʔωϯτͷΩϟϓνϟ

• ൺֱର৅ͷίϯϙʔωϯτͷΩϟϓνϟ
ը໘Ωϟϓνϟ

View Slide

•
fl
utter_test matchesGoldenFile function

• golden_toolkit (ebay)

• playbook-snapshot (playbook-
fl
utter)
ը໘Ωϟϓνϟ

View Slide

• playbook-snapshot (playbook-
fl
utter)

• playbook-ui Ͱ࡞ͬͨίϯϙʔωϯτ
ͷΩϟϓνϟΛαΫοͱࡱΕΔ
ը໘Ωϟϓνϟ

View Slide

• Playbook extension

• Snapshot
playbook-snapshot (playbook-
fl
utter)

View Slide

Playbook(stories: [


Scenario(

'Sample Scenario',

child: Center(

child: Container(

width: 200,

height: 250,




),

),

),

])

]).run(

Snapshot(

directoryPath: 'screenshots',

devices: [SnapshotDevice.iPhone8],

),

(widget) {

return MaterialApp(

debugShowCheckedModeBanner: false,

home: Material(child: widget),

);

},

);
fl
utter)

View Slide

Playbook(stories: [


Scenario(

'Sample Scenario',

child: Center(

child: Container(

width: 200,

height: 250,




),

),

),

])

]).run(

Snapshot(



),

(widget) {

return MaterialApp(



);

},

);
fl
utter)
1MBZCPPLʹSVOϝιου͕௥Ճ͞ΕΔ

View Slide

Playbook(stories: [


Scenario(

'Sample Scenario',

child: Center(

child: Container(

width: 200,

height: 250,




),

),

),

])

]).run(

Snapshot(



),

(widget) {

return MaterialApp(



);

},

);
fl
utter)
4DFOBSJP͕͍࣋ͬͯΔXJEHFU͕౉͞ΕΔ
ΩϟϓνϟΛࡱΓ͍ͨ8JEHFU

View Slide

Playbook(stories: [


Scenario(

'Sample Scenario',

child: Center(

child: Container(

width: 200,

height: 250,




),

),

),

])

]).run(

Snapshot(



),

(widget) {

return MaterialApp(



);

},

);
fl
utter)
Snapshot ઃఆྫ

View Slide

Future main() async {

await Playbook(stories: [


Scenario(

'Sample Scenario',

child: Center(

child: Container(

width: 200,

height: 250,




),

),

),

])

]).run(

Snapshot(



),

(widget) {

return MaterialApp(



);

},

);

}
fl
utter)

View Slide


await Playbook(stories: [


Scenario(

'Sample Scenario',

child: Center(

child: Container(

width: 200,

height: 250,




),

),

),

])

]).run(

Snapshot(



),

(widget) {

return MaterialApp(



);

},

);

}
fl
utter)
͜ͷ෦෼͸ΞϓϦ΋ςετ΋ڞ௨

View Slide

Playbook playbook() {

return Playbook(stories: [


Scenario(

'Sample Scenario',

child: Center(

child: Container(

width: 200,

height: 250,




),

),

),

])

]);

}
fl
utter)

View Slide

void main() {

runApp(MyApp());

}


@override


return MaterialApp(





playbook: playbook(),

),

);

}

}
fl
utter)

View Slide


await playbook().run(

Snapshot(



),

(widget) {

return MaterialApp(



);

},

);

}
fl
utter)

View Slide

• தͰ͸
fl
utter_test ͷ matchesGoldenFile Λ࣮ߦ͍ͯ͠Δ

• ϑΥϯτ΍ΞΠίϯ͕ಡΈࠐΊͳ͍໰୊ͷαϙʔτ
fl
utter)
ৄ͘͠͸
 
https://speakerdeck.com/tomokitakahashi/shi-jian-
fl
utter-visual-regression-testing

View Slide


flutter:

fonts:

- family: Roboto

fonts:

- asset: fonts/Roboto-Regular.ttf
fl
utter)
Ωϟϓνϟ࣌ͷϑΥϯτΛ௥Ճ͢Δ

View Slide

• ඇಉظॲཧ

• ௨৴σʔλʢը૾ʣ

• ࣌ؒ

• Ξχϝʔγϣϯ

• etc…
DI

View Slide


await playbook().run(

Snapshot(



),

(widget) {

return ProviderScope(

overrides: [],

MaterialApp(



),

);

},

);

}
DI
͜ͷ͋ͨΓͰ%*Մೳ

View Slide

• Story ࡞੒ͷ؆ུԽ

• Scenario ࡞੒༻ͷΞϊςʔγϣϯ

• Playbook ͷΠϯελϯεࣗಈੜ੒

• build_runner Ͱ࣮ߦՄೳ
playbook-generator (playbook-
fl
utter)

View Slide

Playbook playbook() {

return Playbook(stories: [


Scenario(

'Sample Scenario',

child: Center(

child: Container(

width: 200,

height: 250,




),

),

),

])

]);

}
fl
utter)
Before

View Slide

const storyTitle = 'Sample Widget';

@GenerateScenario(title: 'Sample Scenario')

Widget sampleScenario() => Center(

child: Container(

width: 200,

height: 250,




),

);
fl
utter)
After

View Slide

void main() {

runApp(MyApp());

}


@override


return MaterialApp(





playbook: playbook,

),

);

}

}
fl
utter)
ม਺ HFUUFS
͕ࣗಈੜ੒͞Ε͍ͯΔ

View Slide


await playbook.run(

Snapshot(



),

(widget) {

return MaterialApp(



);

},

);

}
fl
utter)
ม਺ HFUUFS
͕ࣗಈੜ੒͞Ε͍ͯΔ

View Slide

• ֓ཁ


• ը໘Ωϟϓνϟ

• ςετ
ΞδΣϯμ

View Slide

• PR ࣌ʹ VRT ͕࣮ߦ͞ΕΔΑ͏ʹ CI Ͱઃఆ͢Δ

• ௨஌ܥͷϓϥάΠϯΛ࢖͑͹ςετ݁ՌΛ௨஌ͯ͘͠ΕΔ

• ϨϏϡʔޙʹ Approve ͢Δ·ͰϚʔδͰ͖ͳͨ͘͠ΓͰ͖Δ
ςετ

View Slide

{

"dependencies": {

"reg-suit": "^0.11.1"

},

"devDependencies": {

"reg-keygen-git-hash-plugin": "^0.11.1",

"reg-notify-github-plugin": "^0.11.1",

"reg-publish-gcs-plugin": "^0.11.1"

},

"scripts": {

"regression": "reg-suit run"

}

}
CI
SFHTVJUΛ࣮ߦ͢ΔͨΊͷઃఆΛ௥Ճ

View Slide

version: 2.1

orbs:

android: circleci/[email protected]

flutter: circleci/[email protected]

node: circleci/[email protected]

jobs:

vrt:

executor: android/android

steps:

- checkout

- node/install:

install-yarn: true

- flutter/install_sdk:

- run:

name: Install melos

command: |

dart pub global activate melos

melos run pub:get

- run:

name: Run Golden Testing

command: melos run test:snapshot

- run:

name: Install dependencies

command: yarn

- run:

name: Run VRT

command: yarn regression

workflows:

vrt:

jobs:

- vrt
CI

View Slide

@override


return Container(

color: Colors.amberAccent,

child: Row(

mainAxisAlignment: MainAxisAlignment.center,

crossAxisAlignment: CrossAxisAlignment.center,

children: [

Icon(Icons.star),

SizedBox(width: 16),

Text(text, style: Theme.of(context).textTheme.headline5)

],

),

);

}
ςετ

View Slide

@override


return Container(

color: Colors.amberAccent,

child: Row(

mainAxisAlignment: MainAxisAlignment.start,

crossAxisAlignment: CrossAxisAlignment.center,

children: [

Icon(Icons.star),

SizedBox(width: 16),

Text(text, style: Theme.of(context).textTheme.headline5)

],

),

);

}
ςετ
ෆ۩߹͔൑அͮ͠Β͍

View Slide

• ςετࣗମΛॻ࣌ؒ͘͸ͱʹ͔͘ݮΒ͍ͨ͠

• playbook_ui Λར༻ͯ͠ΩϟϓνϟͷࣗಈԽ

• ࡞ͬͯյͯ͠Λ΋ͬͱؾܰʹߦ͑ΔΑ͏ʹͳΓ·͢ʂ
ςετ

View Slide

• https://github.com/playbook-ui/playbook-
fl
utter

• https://github.com/Dropsource/monarch

• https://github.com/widgetbook/widgetbook

• https://github.com/ookami-kb/storybook_
fl
utter

• https://github.com/eBay/
fl
utter_glove_box/tree/master/packages/
golden_toolkit
ࢀߟ URL

View Slide

Thanks !

View Slide

flutter_kaigi_2021.pdf

flutter_kaigi_2021.pdf

Kyohei Ito

More Decks by Kyohei Ito

Other Decks in Programming

Featured

Transcript