Testing Everything with Flutter

Slide 1

Slide 1 text

Testing everything with Flutter Miguel Beltran Freelancer - beltran.work - @MiBLT

Slide 2

Slide 2 text

Freelance Developer, specialized in Mobile (Android and Flutter) More than 15 years of professional developer experience Co-Host of Code Cafeteria Podcast Reach me out: beltran.work About Me @MiBLT

Slide 3

Slide 3 text

Testing: Dart, Widgets, Driver Mocks and Fakes in Dart Architecture + testing: DI and State Mgmt Limits of Flutter testing In this talk... @MiBLT

Slide 4

Slide 4 text

Sample app: GIF REVIEW Purpose: Let users rate GIFs Screens: Main and detail Code: miquelbeltran/flutter_testing_talk The test subject @MiBLT

Slide 5

Slide 5 text

Why do we need to test at all? “The more features your app has, the harder it is to test manually. Automated tests help ensure that your app performs correctly before you publish it, while retaining your feature and bug fix velocity.” - Flutter docs. @MiBLT

Slide 6

Slide 6 text

Pure Dart tests that don’t load Widgets Dart Test Tests that load Widgets Widget Test Tests that run on a real device Driver @MiBLT

Slide 7

Slide 7 text

Writing a Dart test @MiBLT

Slide 8

Slide 8 text

Simple Dart test double calculateAverage(List ratings) { return // calculate average } @MiBLT

Slide 9

Slide 9 text

@MiBLT

Slide 10

Slide 10 text

Simple Dart test import 'package:flutter_test/flutter_test.dart'; void main() { test('calculate ratings', () { // body of test }); } @MiBLT

Slide 11

Slide 11 text

Simple Dart test void main() { final ratings = [ Rating('Miguel', 5), Rating('Lara', 4), Rating('Lily', 3), ]; test('calculate ratings', () { final average = calculateAverage(ratings); expect(average, 4); }); } @MiBLT

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Assert that “actual” matches “matcher”. expect(actual, matcher) @MiBLT

Slide 15

Slide 15 text

Expect expect(average, 4); // same as expect(average, equals(4)); @MiBLT

Slide 16

Slide 16 text

Expect expect(myArray, isNotEmpty); expect(myWidget, findsOneWidget); expect(myBool, isTrue); @MiBLT

Slide 17

Slide 17 text

Also with right-click -> Run test in…. or $ flutter test @MiBLT

Slide 18

Slide 18 text

Writing a Widget test @MiBLT

Slide 19

Slide 19 text

Testing “GifScore” Text Image.network SmoothStarRating @MiBLT

Slide 20

Slide 20 text

Testing “GifScore” Text @MiBLT

Slide 21

Slide 21 text

GifScore Widget class GifScore extends StatelessWidget { Gif gif; GifScore(this.gif); @override Widget build(BuildContext context) { return ...; } } @MiBLT

Slide 22

Slide 22 text

Writing testWidgets testWidgets('display GifScore', (WidgetTester tester) async { /// ... }); }); @MiBLT

Slide 23

Slide 23 text

Writing testWidgets testWidgets('display GifScore', (WidgetTester tester) async { /// ... }); }); @MiBLT

Slide 24

Slide 24 text

pumpWidget final gif = Gif( id: '1', title: 'Kermit Sipping Tea', url: 'https://...', rating: 3.5, ); await tester.pumpWidget( GifScore( gif, ), ); @MiBLT

Slide 25

Slide 25 text

pumpWidget final gif = Gif( id: '1', title: 'Kermit Sipping Tea' , url: 'https://...', rating: 3.5, ); await tester.pumpWidget( GifScore( gif, ), ); @MiBLT

Slide 26

Slide 26 text

pumpWidget await tester.pumpWidget( Directionality( textDirection: TextDirection.rtl, child: GifScore( gif, ), ), ); @MiBLT

Slide 27

Slide 27 text

Writing testWidgets expect( find.text('Kermit Sipping Tea'), findsOneWidget ); @MiBLT

Slide 28

Slide 28 text

Writing testWidgets find.byKey(key) find.bySemanticsLabel(label) find.descendant( of: find.byKey(key), matching: find.text(text), ) @MiBLT

Slide 29

Slide 29 text

Writing testWidgets testWidgets('display GifScore', (WidgetTester tester) async { mockNetworkImagesFor(() async { final gif = Gif( id: '1', title: 'Kermit Sipping Tea', url:'...', rating: 3.5, ); await tester.pumpWidget( Directionality( textDirection: TextDirection.rtl, child: GifScore( gif, ), ), ); expect(find.text('Kermit Sipping Tea'), findsOneWidget); }); }); @MiBLT

Slide 30

Slide 30 text

Writing a Driver test @MiBLT

Slide 31

Slide 31 text

Driver Tests Run on a real device (also simulators) Test full running app: Compile + Run + Test Let’s check the title is displayed Flutter 1.22: Android and iOS -> slower Beta/Master: Web, Desktop, etc -> faster tests! @MiBLT

Slide 32

Slide 32 text

@MiBLT

Slide 33

Slide 33 text

Driver Tests: app.dart import 'package:../main.dart' as app; void main() { enableFlutterDriverExtension(); app.main(); } @MiBLT

Slide 34

Slide 34 text

Driver Tests: app_test.dart FlutterDriver driver; setUpAll(() async { driver = await FlutterDriver.connect(); }); tearDownAll(() async { if (driver != null) { driver.close(); } }); test('title is displayed', () async { expect(await driver.getText(titleFinder), "Kermit Sipping Tea"); }); @MiBLT

Slide 35

Slide 35 text

Driver Tests: app_test.dart final gifFinder = find.byValueKey('gif-score'); final titleFinder = find.descendant( of: gifFinder, matching: find.byValueKey('gif-title'), ); @MiBLT

Slide 36

Slide 36 text

Driver Tests: Running @MiBLT

Slide 37

Slide 37 text

Driver Tests: failed? @MiBLT

Slide 38

Slide 38 text

Driver Tests: Failed? @MiBLT

Slide 39

Slide 39 text

Driver Tests: Success @MiBLT

Slide 40

Slide 40 text

Complex to Setup Require a device Hard to debug Real user experience Driver Tests @MiBLT

Slide 41

Slide 41 text

SLOW UNRELIABLE MOST COMPLEX REAL USER Driver FAST RELIABLE EASY NOT REAL USER Dart FAST RELIABLE SOMEHOW COMPLEX SOMEHOW REAL USER Widget Testing Options Summary SPEED: STABLE: CODE: USER: @MiBLT

Slide 42

Slide 42 text

Dependency Injection, Fakes, Mocks, and making your app “testable” @MiBLT

Slide 43

Slide 43 text

Tests so far final average = calculateAverage(ratings); await tester.pumpWidget(GifScore(gif)); No internal dependencies! @MiBLT

Slide 44

Slide 44 text

HomePage Widget class HomePage extends StatelessWidget { @override Widget build(BuildContext context) { final api = ApiService(); return Scaffold( appBar: AppBar( title: Text('GifReview'), ), body: FutureBuilder>( future: api.getGifs(), builder: (context, snap) { ... }, ), ); } } @MiBLT

Slide 45

Slide 45 text

Pass our ApiService Fake the ApiService Challenges @MiBLT

Slide 46

Slide 46 text

Pass our ApiService: Dep. Inj. Fake the ApiService: Fake or Mock Challenges @MiBLT

Slide 47

Slide 47 text

Dependencies in tests Code that depends on other components is hard to test. Provide those components externally → Dependency Injection. DI in Flutter: - via constructor - Provider (through Widget tree) - Get_it (as Service Locator) @MiBLT

Slide 48

Slide 48 text

DI via constructor class UseCase { final ApiService service; UseCase(this.service); Gif run() { ... } } @MiBLT

Slide 49

Slide 49 text

DI via constructor void main() { final service = FakeApiService(); final usecase = UseCase(service); test('test my UseCase', () { final result = usecase.run(); expect(result, expected); }); } @MiBLT

Slide 50

Slide 50 text

DI via Provider class MyApp extends StatelessWidget { @override Widget build(BuildContext context) { return Provider( create: (context) => RealApiService(), child: MaterialApp(...), ); } } // later in code: service = Provider.of(context) @MiBLT

Slide 51

Slide 51 text

DI via Provider await tester.pumpWidget ( Provider( create: (context) => FakeApiService() , child: MyWidget(...), ), ); @MiBLT

Slide 52

Slide 52 text

Which State Management do you recommend? “Whatever allows you to test your code!” - Me @MiBLT

Slide 53

Slide 53 text

Fakes and Mocks Custom implementation of a class that fakes its functionality Fake Object that allows to configure answers, capture parameters and verify those calls Mock @MiBLT

Slide 54

Slide 54 text

FakeApiService class _FakeApiService extends ApiService { final gif = Gif( id: '1', title: 'Kermit Sipping Tea' , Url: '...', rating: 3.5, ); @override Future> getGifs() { return SynchronousFuture ([gif]); } } @MiBLT

Slide 55

Slide 55 text

FakeApiService testWidgets('load homescreen with fakes', (WidgetTester tester) async { mockNetworkImagesFor(() async { final fakeApiService = _FakeApiService(); await tester.pumpWidget( Provider( create: (context) => fakeApiService, child: MaterialApp(home: HomePage()), ), ); expect(find.text('Kermit Sipping Tea'), findsOneWidget); }); }); @MiBLT

Slide 56

Slide 56 text

MockApiService https://pub.dev/packages/mockito 1 → class MockApiService extends Mock implements ApiService {} 2 → final mockApiService = MockApiService(); 3 → when(mockApiService.getGifs()) .thenAnswer((realInvocation) => SynchronousFuture([gif])); @MiBLT

Slide 57

Slide 57 text

MockApiService testWidgets('load homescreen with mocks', (tester) async { mockNetworkImagesFor(() async { //... await tester.pumpWidget ( Provider( create: (context) => mockApiService , child: MaterialApp (home: HomePage()), ), ); expect(find.text('Kermit Sipping Tea' ), findsOneWidget ); verify(mockApiService .getGifs()); }); }); @MiBLT

Slide 58

Slide 58 text

Fakes and Mocks NO PACKAGE EASIER TO WRITE NEEDS EXTRA WORK TO CAPTURE AND VERIFY CALLS Fake NEEDS PACKAGE HARDER TO WRITE CAN CAPTURE AND VERIFY CALLS BY DEFAULT Mock @MiBLT

Slide 59

Slide 59 text

Coverage and having “enough” tests @MiBLT

Slide 60

Slide 60 text

Test Pyramid Unit Tests Integration Tests E2E https://testing.googleblog.com/2015/04/just-say-no-to-more-end-to-end-tests.html @MiBLT

Slide 61

Slide 61 text

Test Pyramid https://testing.googleblog.com/2015/04/just-say-no-to-more-end-to-end-tests.html Unit test “take a small piece of the product and test that piece in isolation.” Unit Tests Integration Tests E2E @MiBLT

Slide 62

Slide 62 text

Test Pyramid https://testing.googleblog.com/2015/04/just-say-no-to-more-end-to-end-tests.html Integration Tests “takes a small group of units, and tests their behavior as a whole, verifying that they coherently work together.” Unit Tests Integration Tests E2E @MiBLT

Slide 63

Slide 63 text

Test Pyramid https://testing.googleblog.com/2015/04/just-say-no-to-more-end-to-end-tests.html End to End “Build and run a real app, run automated tests on it” Unit Tests Integration Tests E2E @MiBLT

Slide 64

Slide 64 text

Test Pyramid Unit Tests Integration Tests E2E https://testing.googleblog.com/2015/04/just-say-no-to-more-end-to-end-tests.html 10% E2E 20% Integration Tests 70% Unit Tests @MiBLT

Slide 65

Slide 65 text

Test Pyramid and Flutter Unit Tests Integration Tests E2E Flutter Driver Widget Tests Dart Tests @MiBLT

Slide 66

Slide 66 text

Test Pyramid and Flutter Unit Tests Integration Tests E2E Flutter Driver Widget Tests Dart Tests @MiBLT

Slide 67

Slide 67 text

Test Pyramid and Flutter Unit Tests Integration Tests E2E Flutter Driver Complex Dart & Widget Tests Simple Dart & Widget Tests @MiBLT

Slide 68

Slide 68 text

https://flutter.dev/docs/testing @MiBLT

Slide 69

Slide 69 text

“See Widget tests as Unit Tests” @MiBLT @MiBLT

Slide 70

Slide 70 text

% of app code executed when tests run. Supported by Dart and Widget tests. Run with flutter test --coverage Running tests with coverage @MiBLT @MiBLT

Slide 71

Slide 71 text

Coverage reports @MiBLT

Slide 72

Slide 72 text

Coverage reports $ lcov --summary lcov.info Reading tracefile lcov.info Summary coverage rate: lines......: 25.4% (18 of 71 lines) functions..: no data found branches...: no data found $ lcov -l lcov.info Also compatible with codecov.io! @MiBLT

Slide 73

Slide 73 text

Coverage reports @MiBLT

Slide 74

Slide 74 text

Recommended test coverage in Flutter by me 70% @MiBLT

Slide 75

Slide 75 text

Edge cases you should know @MiBLT

Slide 76

Slide 76 text

mockNetworkImagesFor testWidgets('..', (tester) async { mockNetworkImagesFor (() async { }); }); @MiBLT

Slide 77

Slide 77 text

mockNetworkImagesFor Text Image.network SmoothStarRating @MiBLT

Slide 78

Slide 78 text

mockNetworkImagesFor Flutter testing framework creates a fake HTTP Client Always returns HTTP Error 400 Why? Networking in tests is bad How can we have images? Override the default mock HTTP Client Provide empty images on network calls Package: network_image_mock @MiBLT

Slide 79

Slide 79 text

tester.pump(duration) Testing and animations WidgetTester is not rebuilding and advancing the clock How to wait them to complete? Call tester.pump(duration) Call tester.pumpAndSettle @MiBLT

Slide 80

Slide 80 text

Network calls in Dart test (and skipping tests) test('network call', () async { String url = 'http://example.com/api/items/1' ; Http.Response response = await Http.get(url); expect(response.statusCode, 200); }, skip: true); @MiBLT

Slide 81

Slide 81 text

SynchronousFuture Fake Future that responds immediately Part of the Flutter framework Use it in Widget tests with mocked network calls when(mockApiService .getGifs()) .thenAnswer((realInvocation ) => SynchronousFuture ([gif])); @MiBLT

Slide 82

Slide 82 text

More testing approaches integration_test package Allows you to run Widget tests on devices/emulators Same API as WidgetTester Used in some Flutter plugins Can run on Firebase Test Lab espresso package Test Flutter code from Espresso Android tests @MiBLT

Slide 83

Slide 83 text

RECAP 3 ways of testing Dart tests: Fast and reliable, no Widget testing Widget tests: Also fast and can test Widgets Flutter Driver: Run on a real device Architecture is important! → Dependency Injection Mocks and Fakes, use them! @MiBLT

Slide 84

Slide 84 text

Miguel Beltran Freelancer Twitter: @MiBLT Reach me out: beltran.work Repo: miquelbeltran/flutter_testing_talk THANK YOU!