Testing as a Culture - Speaker Deck

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

©2020 Wantedly, Inc. Follow scenarios and validate the outcomes Time consuming Not very scalable Hard to test different devices and features Hard to simulate edge cases How to simulate network errors? Who should test your app? Manual QA team

Slide 13

Slide 13 text

Slide 14

Slide 14 text

©2020 Wantedly, Inc. Know the code best Manual tests are necessary Is it productive? Still difficult to simulate edge cases Need to build and ship product to the users Who should test your app? Engineers

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

©2020 Wantedly, Inc. Consider this screen EditText for a message and a “Send” Button Type into the EditText and when the “Send” Button is pressed, send the message inside the EditText. If EditText is empty, the Button will be disabled. Benefits of automated tests Scalable

Slide 23

Slide 23 text

©2020 Wantedly, Inc. @Test fun givenMessageIsEmpty_ThenButtonIsDisabled() {!!...} @Test fun givenMessageHasText_WhenClickSend_ThenSendTextToAPI() {!!...} @Test fun givenMessageHasText_WhenDeleteAllText_ThenButtonIsDisabled() {!!...} Benefits of automated tests Scalable

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Slide 39

Slide 39 text

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Slide 44

Slide 44 text

Slide 45

Slide 45 text

Slide 46

Slide 46 text

©2020 Wantedly, Inc. Located in src/androidTest/java Runs on a device or an emulator Combined with Espresso Tests and validates UI components Relatively slow Automated tests for Android projects Instrumented test

Slide 47

Slide 47 text

Slide 48

Slide 48 text

©2020 Wantedly, Inc. Uses Android framework’s API in local JVM environment 4.0+ supports JetPack’s androidx.tests APIs Shadow implementation of Android Framework APIs See: https://github.com/robolectric/robolectric Automated tests for Android projects Robolectric

Slide 49

Slide 49 text

Slide 50

Slide 50 text

Slide 51

Slide 51 text

©2020 Wantedly, Inc. Project Nitrogen Setting up shared tests https://www.youtube.com/watch?v=VJi2vmaQe6w Automated tests for Android projects Project Nitrogen build.gradle android { sourceSets { String sharedTestDir = ‘src/sharedTest/java’ test { java.srcDir sharedTestDir } androidTest { java.srcDir sharedTestDir } } }

Slide 52

Slide 52 text

©2020 Wantedly, Inc. import androidx.test.ext.junit.runners.AndroidJUnit4 !!... @RunWith(AndroidJUnit4!::class) class SampleFragmentTest { !!... } Automated tests for Android projects Project Nitrogen See: https://github.com/android/testing-samples/blob/master/ui/espresso/FragmentScenarioSample

Slide 53

Slide 53 text

Slide 54

Slide 54 text

©2020 Wantedly, Inc. High level, black-box testing Set of instructions to interact with the device Access device’s state, click at views, press the back button, rotate screen, etc. Tests a device as a whole Access home screen and app drawers, or interact with other apps Automated tests for Android projects UIAutomator

Slide 55

Slide 55 text

Slide 56

Slide 56 text

©2020 Wantedly, Inc. Simulates real user interactions Tests “flows” of an application Guide the interactions with roboscript and/or robo directives Automated tests for Android projects Firebase Robo Test https://medium.com/wantedly-engineering/ android-application-testing-with-ﬁrebase-robo-test-c674e1754298

Slide 57

Slide 57 text

Slide 58

Slide 58 text

Slide 59

Slide 59 text

Slide 60

Slide 60 text

Slide 61

Slide 61 text

Slide 62

Slide 62 text

Slide 63

Slide 63 text

Slide 64

Slide 64 text

Slide 65

Slide 65 text

Slide 66

Slide 66 text

©2020 Wantedly, Inc. Determine effort and type of tests Create a balanced test suite Increase developer’s productivity Find the correct amount of tests Less exhaustive tests at the top How? Structuring test suite

Slide 67

Slide 67 text

Slide 68

Slide 68 text

Slide 69

Slide 69 text

Slide 70

Slide 70 text

©2020 Wantedly, Inc. Fake classes Easiest to create manually Actual object with “shortcut” implementations to act like the real implementation. interface UserRepository { suspend fun getUser(userId: Long): User } class FakeUserRepository: UserRepository { override suspend fun getUser(userId: Long): User { return User( id = userId, firstName = "John", lastName = "Smith" ) } } Structuring code for testability Test doubles

Slide 71

Slide 71 text

©2020 Wantedly, Inc. Mock classes Answer to set calls Created using Mockito or MockK Respond to a set of method calls whose answers have been deﬁned. Throw an exception if it encounters any unexpected calls that have not been set. Structuring code for testability Test doubles interface UserRepository { suspend fun getUser(userId: Long): User suspend fun deleteUser(userId: Long) } "// Create a mock UserRepository with MockK val mockRepo = mockk() coEvery { mockRepo.getUser(any()) } answers { User( id = firstArg(), firstName = "John", lastName = "Smith" ) }

Slide 72

Slide 72 text

©2020 Wantedly, Inc. Structuring code for testability Test doubles - Mock classes !//Usage launch { mockRepo.getUser(1L) "// User(1L, "John", "Smith") mockRepo.deleteUser(1L) "// Throws MockKException: no answer found for UserRepository.deleteUser }

Slide 73

Slide 73 text

Slide 74

Slide 74 text

©2020 Wantedly, Inc. Decouple classes Allow for replacement with test doubles Use constructor’s arguments to pass in other classes or interfaces Avoid static getInstance methods Use Dependency Injection Framework Dagger Structuring code for testability Independent and isolated classes

Slide 75

Slide 75 text

©2020 Wantedly, Inc. Structuring code for testability Independent and isolated classes val navController = mockk() ""... fun launchFragment(): FragmentScenario { return launchFragmentInContainer(themeResId = R.style.AppTheme) { SampleFragment() }.onFragment { Navigation.setViewNavController(it.requireView(), navController) } } ""... verify { navController.navigate(R.id.toOtherFragment) } Test in isolation

Slide 76

Slide 76 text

Slide 77

Slide 77 text

©2020 Wantedly, Inc. Start and finish at the same state Clean up DB, SharedPreferences, value flags, etc. Prevent side-effects and flakiness Might affect other test’s outcomes Properly set up and tear down tests Structuring code for testability Stateless tests

Slide 78

Slide 78 text

©2020 Wantedly, Inc. class SampleFragmentTest { @Before fun setUp() {""...} @After fun tearDown() {""...} @Test fun givenMessageIsEmpty_ThenButtonIsDisabled() {""...} @Test fun givenMessageHasText_WhenClickSend_ThenSendTextToAPI() {""...} } Structuring code for testability Stateless tests }

Slide 79

Slide 79 text

Slide 80

Slide 80 text

©2020 Wantedly, Inc. class TestSchedulerRule : TestRule { val testScheduler = TestScheduler() override fun apply(base: Statement, description: Description): Statement { return object : Statement() { override fun evaluate() { RxJavaPlugins.setIoSchedulerHandler { testScheduler } !!... try { base.evaluate() "// Test code will be run here. } finally { RxJavaPlugins.reset() ""... } } } } } Structuring code for testability Stateless tests Set up Test Tear down

Slide 81

Slide 81 text

Slide 82

Slide 82 text

©2020 Wantedly, Inc. Avoid multi-threading Can cause side-effects and increase execution time significantly Testing with 3rd party libraries TestScheduler for RxJava InstantTaskExecutorRule for Architecture Components TestDispatcher and runBlockingTest for coroutines Structuring code for testability Synchronous tests

Slide 83

Slide 83 text

Slide 84

Slide 84 text

Slide 85

Slide 85 text

Slide 86

Slide 86 text

Slide 87

Slide 87 text

Slide 88

Slide 88 text

Slide 89

Slide 89 text

Slide 90

Slide 90 text

Slide 91

Slide 91 text

Slide 92

Slide 92 text

Slide 93

Slide 93 text

Slide 94

Slide 94 text

Slide 95

Slide 95 text

Slide 96

Slide 96 text

Slide 97

Slide 97 text

Slide 98

Slide 98 text

©2020 Wantedly, Inc. Slow test can decrease productivity Long CI/CD pipeline execution time Can have financial impact Don’t try aim for 100% coverage Focus on testing the most important parts Other considerations Time

Slide 99

Slide 99 text

Slide 100

Slide 100 text

Slide 101

Slide 101 text

Slide 102

Slide 102 text

Slide 103

Slide 103 text

©2020 Wantedly, Inc. Difficult to maintain Too complex, each part of the app required different test setup A lot of “magic” to inject test doubles Tightly coupled Hard to test in isolation Rewrite is not possible Making legacy code testable Understanding the problems

Slide 104

Slide 104 text

Slide 105

Slide 105 text

Slide 106

Slide 106 text

©2020 Wantedly, Inc. Descriptive test method name Develop branch is always shippable Every Pull Request needs to have added/removed test code Test libraries for easier setup AndroidX Test, Robolectric, MockK, Custom JUnit rules, etc. Making legacy code testable Create a plan

Slide 107

Slide 107 text

Slide 108

Slide 108 text

Slide 109

Slide 109 text

Slide 110

Slide 110 text

Slide 111

Slide 111 text

Slide 112

Slide 112 text

Slide 113

Slide 113 text

Slide 114

Slide 114 text

Slide 115

Slide 115 text

Slide 116

Slide 116 text

Slide 117

Slide 117 text

Slide 118

Slide 118 text

©2020 Wantedly, Inc. Automated tests make development faster Makes team and project scalable Create test strategy and test conventions Everybody follows the same strategy and conventions Focus on writing maintainable tests Plan testable architecture early or create a migration plan. Projects will always get more complicated! Summary

Slide 119

Slide 119 text