Testing Kotlin at Scale — Spek #KotlinConf2017

Slide 1

Slide 1 text

@artem_zin Artem Zinnatullin Testing Kotlin at Scale: Spek

Slide 2

Slide 2 text

- Productivity

Slide 3

Slide 3 text

- Productivity - Reviewability

Slide 4

Slide 4 text

- Productivity - Reviewability - Maintainability

Slide 5

Slide 5 text

- Patterns - Principles - OOP/FP - Common Sense

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

But

Slide 8

Slide 8 text

We focus on production code

Slide 9

Slide 9 text

- Productivity - Reviewability - Maintainability - Developer Happiness When it comes to test code

Slide 10

Slide 10 text

- Patterns - Principles - OOP/FP - Common Sense When it comes to test code

Slide 11

Slide 11 text

Aren’t we being hypocrite to ourselves?

Slide 12

Slide 12 text

Do we really understand how much Effort do we put in Tests?

Slide 13

Slide 13 text

There is a simple metric though

Slide 14

Slide 14 text

Tests can actually take more LOCs than production code

Slide 15

Slide 15 text

RxJava (library): - $ cloc src/main — 82 K LOC November 2017 https://github.com/ReactiveX/RxJava

Slide 16

Slide 16 text

RxJava (library): - $ cloc src/main — 82 K LOC - $ cloc src/test — 159 K LOC November 2017 https://github.com/ReactiveX/RxJava

Slide 17

Slide 17 text

159 / 82 = 1,94 x RxJava

Slide 18

Slide 18 text

RxJava: 159 / 82 = 1,94 x OkHttp: 27 / 15 = 1,8 x

Slide 19

Slide 19 text

RxJava: 159 / 82 = 1,94 x OkHttp: 27 / 15 = 1,8 x Retroﬁt: 4,9 / 2,8= 1,75 x

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Is there something common between these projects?

Slide 22

Slide 22 text

- Assertion libraries

Slide 23

Slide 23 text

- Assertion libraries ✅

Slide 24

Slide 24 text

- Assertion libraries ✅ - Mocking libraries

Slide 25

Slide 25 text

- Assertion libraries ✅ - Mocking libraries ✅

Slide 26

Slide 26 text

Test framework though?

Slide 27

Slide 27 text

They all use JUnit (4) And it works

Slide 28

Slide 28 text

JUnit 4 ❤ - It’s robust

Slide 29

Slide 29 text

JUnit 4 ❤ - It’s robust - It’s straightforward

Slide 30

Slide 30 text

JUnit 4 ❤ - It’s robust - It’s straightforward - You don’t have to debug it

Slide 31

Slide 31 text

JUnit 4 ❤ - It’s robust - It’s straightforward - You don’t have to debug it - All build systems and IDEs support it

Slide 32

Slide 32 text

JUnit 4 ❤ - It’s robust - It’s straightforward - You don’t have to debug it - All build systems and IDEs support it - Everybody is familiar with it

Slide 33

Slide 33 text

JUnit 4 ❤ - It’s robust - It’s straightforward - You don’t have to debug it - All build systems and IDEs support it - Everybody is familiar with it - It’s a standard.

Slide 34

Slide 34 text

But it has problems

Slide 35

Slide 35 text

“Code Repetition”

Slide 36

Slide 36 text

@Test fun updateCoarseLocationSync_Foregrounded() { `when`(appForegroundDetector.isForegrounded).thenReturn(true) val first = AndroidLocationBuilder() .withProvider(AndroidLocation.Provider.FUSED) .withLat(0.0) .withLng(0.0) .withTime(1L) .build() locationIngestService.updateCoarseLocationSync(first) val argumentCaptor = ArgumentCaptor.forClass(IngestLocationsRequestDTO::class.java) verify(locationIngestApi, times(1)).postLocations(argumentCaptor.capture()) val value = argumentCaptor.value assertThat(value).isNotNull() assertThat(value.locations).isNotNull().hasSize(1) assertThat(value.locations[0]).isNotNull() assertThat(value.locations[0].source).isEqualTo(Location.SIGNIFICANT_LOCATION_CHANGE_FG) } JUnit 4: “Code Repetition” Problem

Slide 37

Slide 37 text

@Test fun updateCoarseLocationSync_NotForegrounded() { `when`(appForegroundDetector.isForegrounded).thenReturn(false) val first = AndroidLocationBuilder() .withProvider(AndroidLocation.Provider.FUSED) .withLat(0.0) .withLng(0.0) .withTime(1L) .build() locationIngestService.updateCoarseLocationSync(first) val argumentCaptor = ArgumentCaptor.forClass(IngestLocationsRequestDTO::class.java) verify(locationIngestApi, times(1)).postLocations(argumentCaptor.capture()) val value = argumentCaptor.value assertThat(value).isNotNull() assertThat(value.locations).isNotNull().hasSize(1) assertThat(value.locations[0]).isNotNull() assertThat(value.locations[0].source).isEqualTo(Location.SIGNIFICANT_LOCATION_CHANGE_BG) } JUnit 4: “Code Repetition” Problem

Slide 38

Slide 38 text

@Test fun updateCoarseLocationSync_ NotForegrounded() { `when`(appForegroundDetector.isForegrounded).thenReturn( false) val first = AndroidLocationBuilder() .withProvider(AndroidLocation.Provider.FUSED) .withLat(0.0) .withLng(0.0) .withTime(1L) .build() locationIngestService.updateCoarseLocationSync(first) val argumentCaptor = ArgumentCaptor.forClass(IngestLocationsRequestDTO::class.java) verify(locationIngestApi, times(1)).postLocations(argumentCaptor.capture()) val value = argumentCaptor.value assertThat(value).isNotNull() assertThat(value.locations).isNotNull().hasSize(1) assertThat(value.locations[0]).isNotNull() assertThat(value.locations[0].source).isEqualTo(Location.SIGNIFICANT_LOCATION_CHANGE_BG) } JUnit 4: “Code Repetition” Problem

Slide 39

Slide 39 text

Slide 40

Slide 40 text

“Test case/context naming”

Slide 41

Slide 41 text

@Test fun removeAccessTokenShouldStopLoggedinScopeAndStartLoggedOutScope() { accessTokenRepository.removeAccessToken() val scopeChange1 = scopeManager.scopeChanges[0] assertThat(scopeChange1.scope).isEqualTo(PassengerScopes.LOGGED_IN) assertThat(scopeChange1.started).isFalse() val scopeChange2 = scopeManager.scopeChanges[1] assertThat(scopeChange2.scope).isEqualTo(PassengerScopes.LOGGED_OUT) assertThat(scopeChange2.started).isTrue() } JUnit4: “Test case/context naming” Problem

Slide 42

Slide 42 text

@Test fun `remove access token should stop logged in scope and start logged out scope`() { JUnit4: “Test case/context naming” Problem accessTokenRepository.removeAccessToken() val scopeChange1 = scopeManager.scopeChanges[0] assertThat(scopeChange1.scope).isEqualTo(PassengerScopes.LOGGED_IN) assertThat(scopeChange1.started).isFalse() val scopeChange2 = scopeManager.scopeChanges[1] assertThat(scopeChange2.scope).isEqualTo(PassengerScopes.LOGGED_OUT) assertThat(scopeChange2.started).isTrue() }

Slide 43

Slide 43 text

@Test fun `remove access token should stop logged in scope and start logged out scope`() JUnit4: “Test case/context naming” Problem Still bad

Slide 44

Slide 44 text

“What does this test test?”

Slide 45

Slide 45 text

@Test fun `remove access token should stop logged in scope and start logged out scope`() { accessTokenRepository.removeAccessToken() val scopeChange1 = scopeManager.scopeChanges[0] assertThat(scopeChange1.scope).isEqualTo(PassengerScopes.LOGGED_IN) assertThat(scopeChange1.started).isFalse() val scopeChange2 = scopeManager.scopeChanges[1] assertThat(scopeChange2.scope).isEqualTo(PassengerScopes.LOGGED_OUT) assertThat(scopeChange2.started).isTrue() } JUnit 4: “What does this test test” Problem

Slide 46

Slide 46 text

Slide 47

Slide 47 text

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

Some motivation speech here

Slide 50

Slide 50 text

Spek

Slide 51

Slide 51 text

Hadi Hariri @JetBrains Spek v0, 2012

Slide 52

Slide 52 text

Hadi Hariri @JetBrains Spek v0, 2012

Slide 53

Slide 53 text

Hadi Hariri @JetBrains Spek v0, 2012 Kotlin 0.4.297

Slide 54

Slide 54 text

Hadi Hariri @JetBrains Spek v0, 2012

Slide 55

Slide 55 text

Hadi Hariri @JetBrains Spek v0, 2012

Slide 56

Slide 56 text

2016

Slide 57

Slide 57 text

Spek v1.0.25, 2016 We started to feel JUnit 4 problems really badly

Slide 58

Slide 58 text

Spek v1.0.25, 2016

Slide 59

Slide 59 text

Spek v1.0.25, 2016

Slide 60

Slide 60 text

Spek v1.0.25, 2016 We started using it

Slide 61

Slide 61 text

Spek v1.0.25, 2016

Slide 62

Slide 62 text

Spek v1.0.25, 2016

Slide 63

Slide 63 text

No content

Slide 64

Slide 64 text

No content

Slide 65

Slide 65 text

I see what you did there, Hadi

Slide 66

Slide 66 text

Spek 2.x

Slide 67

Slide 67 text

: BetterTestingFuture Spek 2.x

Slide 68

Slide 68 text

class SuperTypicalJUnitTest { val calculator = Calculator() @Test fun `2 + 4 = 6`() { val result = calculator.add(2, 4) assertEquals(6, result) } } Typical Spec

Slide 69

Slide 69 text

class CalculatorSpec : Spek({ val calculator by memoized { Calculator() } context("2 + 4") { val result by memoized { calculator.add(2, 4) } it("equals 6") { assertEquals(6, result) } } }) Typical Spec

Slide 70

Slide 70 text

Spek: Basic API

Slide 71

Slide 71 text

context(“2 + 4") { }x Spek: Basic API

Slide 72

Slide 72 text

context("2 + 4") { }x describe(“2 + 4") { }x Spek: Basic API

Slide 73

Slide 73 text

context("2 + 4") { }x describe("2 + 4") { }x given(“2 + 4") { }x Spek: Basic API

Slide 74

Slide 74 text

describe("2 + 4") { }x context("2 + 4") { }x given("2 + 4") { }x group(“2 + 4") Spek: Basic API

Slide 75

Slide 75 text

group("") ~= Test class in JUnit Spek: Basic API

Slide 76

Slide 76 text

group("") ~= Test class in JUnit Spek: Basic API You can nest groups naturally

Slide 77

Slide 77 text

it("equals 6") { assertThat(result).isEqualTo(6) } Spek: Basic API

Slide 78

Slide 78 text

it("equals 6") { assertThat(result).isEqualTo(6) } @Test fun `2 + 4 = 6`() { val result = calculator.add(2, 4) assertEquals(6, result) } Spek: Basic API

Slide 79

Slide 79 text

it("equals 6") { assertThat(result).isEqualTo(6) } @Test fun `2 + 4 = 6`() { val result = calculator.add(2, 4) assertEquals(6, result) } Spek: Basic API

Slide 80

Slide 80 text

it("equals 6") { assertThat(result).isEqualTo(6) } @Test fun `2 + 4 = 6`() { val result = calculator.add(2, 4) assertEquals(6, result) } Spek: Basic API

Slide 81

Slide 81 text

it("equals 6") { assertThat(result).isEqualTo(6) } @Test fun `2 + 4 = 6`() { val result = calculator.add(2, 4) assertEquals(6, result) } Spek: Basic API

Slide 82

Slide 82 text

Spek: Basic API it("") = @Test in JUnit

Slide 83

Slide 83 text

it("") = @Test in JUnit Spek: Basic API You can have as many `it` in a `group` as needed

Slide 84

Slide 84 text

Spek: Basic API Groups and Tests create natural structure that scales very well

Slide 85

Slide 85 text

val calculator by memoized { Calculator() } Spek: Basic API

Slide 86

Slide 86 text

Spek: Basic API You avoid state sharing betwen tests with `memoized`

Slide 87

Slide 87 text

Let’s rewrite real JUnit test with Spek

Slide 88

Slide 88 text

Slide 89

Slide 89 text

Slide 90

Slide 90 text

context("remove access token") { } Let’s rewrite real JUnit test with Spek

Slide 91

Slide 91 text

context("remove access token") { beforeEachTest { accessTokenRepository.removeAccessToken() } } Let’s rewrite real JUnit test with Spek

Slide 92

Slide 92 text

context("remove access token") { beforeEachTest { accessTokenRepository.removeAccessToken() } } Let’s rewrite real JUnit test with Spek

Slide 93

Slide 93 text

Slide 94

Slide 94 text

Slide 95

Slide 95 text

describe("first scope change") { } Let’s rewrite real JUnit test with Spek

Slide 96

Slide 96 text

describe("first scope change") { val firstScopeChange by memoized { scopeManager.scopeChanges[0] } } Let’s rewrite real JUnit test with Spek

Slide 97

Slide 97 text

Slide 98

Slide 98 text

describe("first scope change") { val firstScopeChange by memoized { scopeManager.scopeChanges[0] } it("is 'logged in' scope") { assertThat(firstScopeChange.scope).isEqualTo(LOGGED_IN) } it("is not started") { assertThat(firstScopeChange.started).isFalse() } } Let’s rewrite real JUnit test with Spek

Slide 99

Slide 99 text

class JUnitTest { private var scopeManager = MockScopeManager() private val accessTokenRepository = AccessTokenRepository( RuntimeEnvironment.application, scopeManager ) lateinit var scopeChange1: MockScopeManager.ScopeChange lateinit var scopeChange2: MockScopeManager.ScopeChange @Before fun `remove access token`() { accessTokenRepository.removeAccessToken() scopeChange1 = scopeManager.scopeChanges[0] scopeChange2 = scopeManager.scopeChanges[1] } @Test fun `first scope change is 'logged in' scope`() { assertThat(scopeChange1.scope).isEqualTo(LOGGED_IN) } @Test fun `first scope change is not started`() { assertThat(scopeChange1.started).isFalse() } @Test fun `second scope change is 'logged out' scope`() { assertThat(scopeChange2.scope).isEqualTo(LOGGED_OUT) } @Test fun `first scope change is started`() { assertThat(scopeChange2.started).isTrue() } } Let’s rewrite real JUnit test with Spek JUnit is not structured, it’s flat

Slide 100

Slide 100 text

class Spec : Spek({ val scopeManager by memoized { MockScopeManager() } val accessTokenRepository by memoized { AccessTokenRepository(RuntimeEnvironment.application, scopeManager) } context("remove access token") { beforeEachTest { accessTokenRepository.removeAccessToken() } describe("first scope change") { val firstScopeChange by memoized { scopeManager.scopeChanges[0] } it("is 'logged in' scope") { assertThat(firstScopeChange.scope).isEqualTo(LOGGED_IN) } it("is not started") { assertThat(firstScopeChange.started).isFalse() } } describe("second scope change") { val secondScopeChange by memoized { scopeManager.scopeChanges[1] } it("is 'logged out' scope") { assertThat(secondScopeChange.scope).isEqualTo(LOGGED_OUT) } it("is started") { assertThat(secondScopeChange.started).isTrue() } } } }) Let’s rewrite real JUnit test with Spek Spek has structure

Slide 101

Slide 101 text

Test code can be structured with Spek

Slide 102

Slide 102 text

Let’s rewrite real JUnit test with Spek Hierarchical Report in IntelliJ

Slide 103

Slide 103 text

The missing documentation & samples Spek Tips

Slide 104

Slide 104 text

Spek Tips You can iterate stuff

Slide 105

Slide 105 text

Spek Tips Iterate things, Spek is just a code class StringExtensionsSpec : Spek({ listOf(null, "", " ", "\t").forEach { string -> context("null or blank string '$string'") { val isNullOrBlank by memoized { string.isNullOrBlank() } it("is indeed null or blank") { assertThat(isNullOrBlank, equalTo(true)) } } } })

Slide 106

Slide 106 text

when (spek) ?

Slide 107

Slide 107 text

Hopefully Spek 2.x release plans Developer Preview by the end of November

Slide 108

Slide 108 text

Most likely Spek 2.x release plans ~month is exactly enough to get sick of how we typically write tests

Slide 109

Slide 109 text

Spek 2.x release plans github.com/spekframework/spek

Slide 110

Slide 110 text

Spek 2.x release plans github.com/spekframework/spek twitter.com/artem_zin

Slide 111

Slide 111 text

Spek 2.x release plans github.com/spekframework/spek twitter.com/artem_zin

Slide 112

Slide 112 text

Spek 2.x release plans github.com/spekframework/spek twitter.com/artem_zin End of November

Slide 113

Slide 113 text

#kotlinconf17 @artem_zin Artem Zinnatullin Thank you! @hhariri Hadi Hariri @raniejade Ranie Jade Ramiso @arturdryomov Artur Dryomov