Pragmatic Testing - Speaker Deck

Slide 1

Slide 1 text

Pragmatic Testing Zac Sweers - Uber @pandanomic

Slide 2

Slide 2 text

Testing?

Slide 3

Slide 3 text

Testing!

Slide 4

Slide 4 text

@Test

Slide 5

Slide 5 text

@Test

Slide 6

Slide 6 text

Pragmatic Testing

Slide 7

Slide 7 text

Pragmatic? Testing

Slide 8

Slide 8 text

Goals ⚽

Slide 9

Slide 9 text

Non-Goals

Slide 10

Slide 10 text

Non-Goals • Convince you testing is worthwhile

Slide 11

Slide 11 text

Non-Goals • Convince you testing is worthwhile • UI Testing (Espresso, layouts, etc)

Slide 12

Slide 12 text

Non-Goals • Convince you testing is worthwhile • UI Testing (Espresso, layouts, etc) • Knock specific libraries

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

Boundaries

Slide 15

Slide 15 text

Calculator

Slide 16

Slide 16 text

Calculator add() subtract() multiply() divide() equal() clear()

Slide 17

Slide 17 text

Calculator InputManager DisplayManager CPU RAM add() subtract() multiply() divide() equal() clear()

Slide 18

Slide 18 text

InputManager DisplayManager CPU RAM add() subtract() multiply() divide() equal() clear()

Slide 19

Slide 19 text

InputManager DisplayManager CPU RAM add() subtract() multiply() divide() equal() clear()

Slide 20

Slide 20 text

InputManager DisplayManager CPU RAM add() subtract() multiply() divide() equal() clear()

Slide 21

Slide 21 text

InputManager DisplayManager CPU RAM add() subtract() multiply() divide() equal() clear()

Slide 22

Slide 22 text

InputManager DisplayManager CPU RAM

Slide 23

Slide 23 text

InputManager DisplayManager CPU RAM

Slide 24

Slide 24 text

@Test fun testAdd() { }a InputManager DisplayManager CPU RAM

Slide 25

Slide 25 text

@Test fun testAdd() { val calculator = Calculator() }a InputManager DisplayManager CPU RAM

Slide 26

Slide 26 text

@Test fun testAdd() { val cpu = mock(CPU::class.java) val calculator = Calculator() }a InputManager DisplayManager CPU RAM

Slide 27

Slide 27 text

@Test fun testAdd() { val cpu = mock(CPU::class.java) val ram = mock(RAM::class.java) val calculator = Calculator() }a InputManager DisplayManager CPU RAM

Slide 28

Slide 28 text

@Test fun testAdd() { val cpu = mock(CPU::class.java) val ram = mock(RAM::class.java) val display = mock(DisplayManager::class.java) val calculator = Calculator() }a InputManager DisplayManager CPU RAM

Slide 29

Slide 29 text

@Test fun testAdd() { val cpu = mock(CPU::class.java) val ram = mock(RAM::class.java) val display = mock(DisplayManager::class.java) val input = mock(InputManager::class.java) val calculator = Calculator() }a InputManager DisplayManager CPU RAM

Slide 30

Slide 30 text

Slide 31

Slide 31 text

@Test fun testAdd() { // Mocks... val calculator = Calculator(cpu, ram, display, input) }a InputManager DisplayManager CPU RAM

Slide 32

Slide 32 text

@Test fun testAdd() { // Mocks... val calculator = Calculator(cpu, ram, display, input) input.add(2, 2) }a InputManager DisplayManager CPU RAM

Slide 33

Slide 33 text

@Test fun testAdd() { // Mocks... val calculator = Calculator(cpu, ram, display, input) input.add(2, 2) verify(cpu).setRam(ram) }a InputManager DisplayManager CPU RAM

Slide 34

Slide 34 text

@Test fun testAdd() { // Mocks... val calculator = Calculator(cpu, ram, display, input) input.add(2, 2) verify(cpu).setRam(ram) verify(input).onInputReceived(2) }a InputManager DisplayManager CPU RAM

Slide 35

Slide 35 text

Slide 36

Slide 36 text

@Test fun testAdd() { // Mocks... val calculator = Calculator(cpu, ram, display, input) input.add(2, 2) verify(cpu).setRam(ram) verify(input).onInputReceived(2) verify(cpu).loadInputToCpu(2) verify(input).onInputReceived("+") }a InputManager DisplayManager CPU RAM

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Slide 39

Slide 39 text

Slide 40

Slide 40 text

InputManager DisplayManager CPU RAM @Test fun testAdd() { // Mocks... val calculator = Calculator(cpu, ram, display, input) input.add(2, 2) verify(cpu).setRam(ram) verify(input).onInputReceived(2) verify(cpu).loadInputToCpu(2) verify(input).onInputReceived("+") verify(ram).storeInRAM(2) verify(cpu).loadInputToCpu("+") verify(input).onInputReceived(2) verify(cpu).loadInputToCpu(2) }a

Slide 41

Slide 41 text

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Slide 44

Slide 44 text

Slide 45

Slide 45 text

@Test fun testAdd() { input.add(2, 2) }a InputManager DisplayManager CPU RAM

Slide 46

Slide 46 text

@Test fun testAdd() { input.add(2, 2) }a registerInputListener(listener: Listener) InputManager DisplayManager CPU RAM

Slide 47

Slide 47 text

val captor = ArgumentCaptor.forClass(Listener::class.java) InputManager DisplayManager CPU RAM

Slide 48

Slide 48 text

val captor = ArgumentCaptor.forClass(Listener::class.java) verify(input).registerInputListener(captor.capture()) InputManager DisplayManager CPU RAM

Slide 49

Slide 49 text

val captor = ArgumentCaptor.forClass(Listener::class.java) verify(input).registerInputListener(captor.capture()) val listener = captor.value InputManager DisplayManager CPU RAM

Slide 50

Slide 50 text

@Test fun testAdd() { val captor = ArgumentCaptor.forClass(Listener::class.java) verify(input).registerInputListener(captor.capture()) val listener = captor.value input.add(2, 2) }a InputManager DisplayManager CPU RAM

Slide 51

Slide 51 text

@Test fun testAdd() { // Captor stuff input.add(2, 2) }a InputManager DisplayManager CPU RAM

Slide 52

Slide 52 text

InputManager DisplayManager CPU RAM @Test fun testAdd() { // Mocks... val calculator = Calculator(cpu, ram, display, input) // Captor stuff... input.add(2, 2) verify(cpu).setRam(ram) listener.onNewInput(2) verify(input).onInputReceived(2) verify(cpu).loadInputToCpu(2) listener.onNewInput("+") verify(input).onInputReceived("+") verify(ram).storeInRAM(2) verify(cpu).loadInputToCpu("+") listener.onNewInput(2) verify(input).onInputReceived(2) verify(cpu).loadInputToCpu(2) verify(ram).loadFromRam() verify(cpu).calculateSum(2, 2) verify(display).displayResult(4) }a

Slide 53

Slide 53 text

Slide 54

Slide 54 text

Slide 55

Slide 55 text

Slide 56

Slide 56 text

Slide 57

Slide 57 text

@Test fun testAdd() { verify(cpu).calculateSum(2, 2) verify(display).displayResult(4) }a InputManager DisplayManager CPU RAM

Slide 58

Slide 58 text

@Test fun testAdd() { verify(cpu).calculateSum(2, 2) // Actually 0 verify(display).displayResult(4) }a InputManager DisplayManager CPU RAM

Slide 59

Slide 59 text

@Test fun testAdd() { when(cpu.calculateSum(2, 2)).thenReturn(4) verify(cpu).calculateSum(2, 2) verify(display).displayResult(4) }a InputManager DisplayManager CPU RAM

Slide 60

Slide 60 text

InputManager DisplayManager CPU RAM @Test fun testAdd() { // Mocks... val calculator = Calculator(cpu, ram, display, input) // Captor stuff... input.add(2, 2) verify(cpu).setRam(ram) listener.onNewInput(2) verify(input).onInputReceived(2) verify(cpu).loadInputToCpu(2) listener.onNewInput("+") verify(input).onInputReceived("+") verify(ram).storeInRAM(2) verify(cpu).loadInputToCpu("+") listener.onNewInput(2) verify(input).onInputReceived(2) verify(cpu).loadInputToCpu(2) verify(ram).loadFromRam() when(cpu.calculateSum(2, 2)).thenReturn(4) verify(cpu).calculateSum(2, 2) verify(display).displayResult(4)

Slide 61

Slide 61 text

InputManager DisplayManager CPU RAM @Test fun testAdd() { // Mocks... val calculator = Calculator(cpu, ram, display, input) // Captor stuff... input.add(2, 2) verify(cpu).setRam(ram) listener.onNewInput(2) verify(input).onInputReceived(2) verify(cpu).loadInputToCpu(2) listener.onNewInput("+") verify(input).onInputReceived("+") verify(ram).storeInRAM(2) verify(cpu).loadInputToCpu("+") listener.onNewInput(2) verify(input).onInputReceived(2) verify(cpu).loadInputToCpu(2) verify(ram).loadFromRam() when(cpu.calculateSum(2, 2)).thenReturn(4) verify(cpu).calculateSum(2, 2) verify(display).displayResult(4)

Slide 62

Slide 62 text

InputManager DisplayManager CPU RAM @Test fun testAdd() { // Mocks... val calculator = Calculator(cpu, ram, display, input) // Captor stuff... input.add(2, 2) verify(cpu).setRam(ram) listener.onNewInput(2) verify(input).onInputReceived(2) verify(cpu).loadInputToCpu(2) listener.onNewInput("+") verify(input).onInputReceived("+") verify(ram).storeInRAM(2) verify(cpu).loadInputToCpu("+") listener.onNewInput(2) verify(input).onInputReceived(2) verify(cpu).loadInputToCpu(2) verify(ram).loadFromRam() when(cpu.calculateSum(2, 2)).thenReturn(4) verify(cpu).calculateSum(2, 2) verify(display).displayResult(4)

Slide 63

Slide 63 text

@Test fun testAdd() { // Mocks... val calculator = Calculator(cpu, ram, display, input) // Captor stuff... input.add(2, 2) listener.onNewInput(2) listener.onNewInput("+") listener.onNewInput(2) when(cpu.calculateSum(2, 2)).thenReturn(4) verify(cpu).calculateSum(2, 2) verify(display).displayResult(4) }a InputManager DisplayManager CPU RAM

Slide 64

Slide 64 text

InputManager DisplayManager CPU RAM @Test fun testAdd() { val calculator = Calculator(cpu, ram, display, input) }a

Slide 65

Slide 65 text

InputManager DisplayManager CPU RAM @Test fun testAdd() { val calculator = Calculator(cpu, ram, display, input) assertThat(calculator.add(2, 2)).isEqualTo(4) }a

Slide 66

Slide 66 text

InputManager DisplayManager CPU RAM InputManager DisplayManager CPU RAM @Test fun testAdd() { val calculator = Calculator(cpu, ram, display, input) assertThat(calculator.add(2, 2)).isEqualTo(4) }a

Slide 67

Slide 67 text

DisplayManager CPU RAM InputManager DisplayManager CPU RAM @Test fun testAdd() { val calculator = Calculator(cpu, ram, display, input) assertThat(calculator.add(2, 2)).isEqualTo(4) }a

Slide 68

Slide 68 text

DisplayManager CPU RAM DisplayManager CPU RAM @Test fun testAdd() { val calculator = Calculator(cpu, ram, display) assertThat(calculator.add(2, 2)).isEqualTo(4) }a

Slide 69

Slide 69 text

DisplayManager RAM DisplayManager CPU RAM @Test fun testAdd() { val calculator = Calculator(cpu, ram, display) assertThat(calculator.add(2, 2)).isEqualTo(4) }a

Slide 70

Slide 70 text

DisplayManager CPU RAM @Test fun testAdd() { val calculator = Calculator(CPU(), ram, display) assertThat(calculator.add(2, 2)).isEqualTo(4) }a

Slide 71

Slide 71 text

@Test fun testAdd() { val calculator = Calculator(CPU(), ram, display) assertThat(calculator.add(2, 2)).isEqualTo(4) }a DisplayManager CPU RAM DisplayManager CPU RAM

Slide 72

Slide 72 text

DisplayManager CPU DisplayManager CPU RAM @Test fun testAdd() { val calculator = Calculator(CPU(), ram, display) assertThat(calculator.add(2, 2)).isEqualTo(4) }a

Slide 73

Slide 73 text

RAM

Slide 74

Slide 74 text

RAM interface RAMa{ afunastoreInRAM(key:aString,avalue:aT) afunaloadFromRam(key:aString):aT? }

Slide 75

Slide 75 text

class InMemoryRAM : RAMa{ private val store = mutableMapOf() override afunastoreInRAM(key:aString,avalue:aT) { return store[key] = value } override afunaloadFromRam(key:aString):aT? { return store[key] } }

Slide 76

Slide 76 text

InMemoryRAM

Slide 77

Slide 77 text

DisplayManager CPU InMemoryRAM @Test fun testAdd() { val ram = mock(RAM::class.java) val calculator = Calculator(CPU(), ram, display) assertThat(calculator.add(2, 2)).isEqualTo(4) }a

Slide 78

Slide 78 text

DisplayManager CPU InMemoryRAM @Test fun testAdd() { val ram = InMemoryRAM() val calculator = Calculator(CPU(), ram, display) assertThat(calculator.add(2, 2)).isEqualTo(4) }a

Slide 79

Slide 79 text

CPU DisplayManager CPU @Test fun testAdd() { val ram = InMemoryRAM() val calculator = Calculator(CPU(), ram, display) assertThat(calculator.add(2, 2)).isEqualTo(4) }a

Slide 80

Slide 80 text

CPU DisplayManager CPU @Test fun testAdd() { val cpu = CPU(InMemoryRAM()) val calculator = Calculator(cpu, display) assertThat(calculator.add(2, 2)).isEqualTo(4) }a

Slide 81

Slide 81 text

CPU CPU DisplayManager @Test fun testAdd() { val cpu = CPU(InMemoryRAM()) val calculator = Calculator(cpu, display) assertThat(calculator.add(2, 2)).isEqualTo(4) }a DisplayManager

Slide 82

Slide 82 text

CPU CPU DisplayManager verify(display).displayResult(4) DisplayManager

Slide 83

Slide 83 text

CPU CPU DisplayManager verify(display).displayResult(4) assertThat(display.getCurrentResult()).isEqualTo(4) DisplayManager

Slide 84

Slide 84 text

CPU CPU DisplayManager verify(display).displayResult(4) assertThat(display.getCurrentResult()).isEqualTo(4) assertThat(display.recordedResults[0]).isEqualTo(4) DisplayManager

Slide 85

Slide 85 text

CPU CPU val resultHandler = RecordingResultHandler() // Calc assertThat(resultHandler.results[0]).isEqualTo(4) ResultHandler

Slide 86

Slide 86 text

CPU ResultHandler CPU @Test fun testAdd() { val cpu = CPU(InMemoryRAM()) val resultHandler = RecordingResultHandler() val calculator = Calculator(cpu, resultHandler) calculator.add(2, 2) assertThat(resultHandler.results[0]).isEqualTo(4) }a

Slide 87

Slide 87 text

CPU @Test fun testAdd() { val cpu = CPU(InMemoryRAM()) val calculator = Calculator(cpu) assertThat(calculator.add(2, 2)).isEqualTo(4) }a

Slide 88

Slide 88 text

val cpu = CPU(InMemoryRAM()) val calculator = Calculator(cpu) @Test fun testAdd() { assertThat(calculator.add(2, 2)).isEqualTo(4) }a

Slide 89

Slide 89 text

val cpu = CPU(InMemoryRAM()) val calculator = Calculator(cpu) @Testa fun testAdd() { assertThat(calculator.add(2, 2)).isEqualTo(4) }a @Testb fun testSub() { assertThat(calculator.subtract(2, 2)).isEqualTo(0) }b @Testc fun testMultiply() { assertThat(calculator.multiply(2, 2)).isEqualTo(4) }c

Slide 90

Slide 90 text

val cpu = CPU(InMemoryRAM()) val calculator = Calculator(cpu) @Testa fun testAdd() { assertThat(calculator.add(2, 2)).isEqualTo(4) }a

Slide 91

Slide 91 text

val cpu = CPU(InMemoryRAM()) val resultHandler = RecordingResultHandler() val calculator = Calculator(cpu, resultHandler) @Test fun testAdd() { calculator.add(2, 2) calculator.add(2, 3) calculator.add(2, -1) assertThat(resultHandler.results) .containsExactly(4, 5, 1) }a

Slide 92

Slide 92 text

Failure

Slide 93

Slide 93 text

@Test fun foo() { try { foo.bar() } catch (e: Exception) { // Expected } }a

Slide 94

Slide 94 text

@Test(expected = Exception::class) fun foo() { foo.bar() }a

Slide 95

Slide 95 text

@Test(expected = ExpectedException::class) fun foo() { foo.bar() }a

Slide 96

Slide 96 text

@Test() fun foo() { try { foo.bar() } catch (e: ExpectedException) { assertThat(e) .hasMessageThat() .contains("Oh no") }b }a

Slide 97

Slide 97 text

@Test() fun foo() { try { foo.bar() throw AssertionError("No") } catch (e: ExpectedException) { assertThat(e) .hasMessageThat() .contains("Oh no") }b }a

Slide 98

Slide 98 text

RxJava

Slide 99

Slide 99 text

@Test() fun rx() { Observable.error() .subscribe() }a

Slide 100

Slide 100 text

@Test() fun rx() { Observable.error() .subscribe() }a

Slide 101

Slide 101 text

throw error

Slide 102

Slide 102 text

// throw error Thread currentThread = Thread.currentThread(); UncaughtExceptionHandler handler = currentThread.getUncaughtExceptionHandler(); handler.uncaughtException(currentThread, error);

Slide 103

Slide 103 text

// throw error Thread currentThread = Thread.currentThread(); UncaughtExceptionHandler handler = currentThread.getUncaughtExceptionHandler(); handler.uncaughtException(currentThread, error); https://github.com/uber/AutoDispose/blob/master/autodispose/src/test/java/com/uber/ autodispose/RxErrorsRule.java

Slide 104

Slide 104 text

TestObserver

Slide 105

Slide 105 text

TestObserver Observable.just(1) .test()

Slide 106

Slide 106 text

TestObserver Observable.just(1) .test() .assertValue(...)

Slide 107

Slide 107 text

TestObserver Observable.just(1) .test() .assertValue(...) .assertError(...)

Slide 108

Slide 108 text

TestObserver Observer o = mock(Observer::class.java) Observable.just(1) .subscribe(o) verify(o).onNext(...) verify(o).onError(...)

Slide 109

Slide 109 text

TestObserver Observable.just(1) .test() .assertValue(...) .assertError(...) .awaitDone(...)

Slide 110

Slide 110 text

Some others quickly because ⏰

Slide 111

Slide 111 text

What about my architecture?

Slide 112

Slide 112 text

class Calculator(val cpu: CPU, val resultHandler: ResultHandler, val inputManager: InputManager)

Slide 113

Slide 113 text

class CalculatorPresenter(val cpu: CPU, val resultHandler: ResultHandler, val inputManager: InputManager)

Slide 114

Slide 114 text

class CalculatorPresenter(val cpu: CPU, val resultHandler: ResultHandler, val inputManager: InputManager) Courtesy of Nick Butcher (@crafty)

Slide 115

Slide 115 text

class CalculatorPresenter(val cpu: CPU, val resultHandler: ResultHandler, val inputManager: InputManager) 0.0 7 8 9 4 5 6 1 2 3 C 0 ⏎

Slide 116

Slide 116 text

class CurrentValueView : ResultHandler 0.0 7 8 9 4 5 6 1 2 3 C 0 ⏎

Slide 117

Slide 117 text

class NumberInputManager : InputManager class CurrentValueView : ResultHandler 0.0 7 8 9 4 5 6 1 2 3 C 0 ⏎

Slide 118

Slide 118 text

class NumberInputManager : InputManager class CurrentValueView : ResultHandler 0.0 7 8 9 4 5 6 1 2 3 C 0 ⏎ class ImeInputManager : InputManager

Slide 119

Slide 119 text

class NumberInputManager : InputManager class CurrentValueView : ResultHandler 7 8 9 4 5 6 1 2 3 C 0 ⏎ 0.0

Slide 120

Slide 120 text

What about DI?

Slide 121

Slide 121 text

class MainActivity : Activity { @Inject lateinit var foo: Foo @Inject lateinit var bar: Bar }

Slide 122

Slide 122 text

@Test fun testActivity() { val activity = ... activity.foo = TestFoo() activity.bar = MockBar() }

Slide 123

Slide 123 text

@Test fun testActivity() { val activity = MainActivityTestHelper.create( TestFoo(), MockBar() )a }b

Slide 124

Slide 124 text

@Test fun testActivity() { val activity = MainActivityTestHelper.create( TestFoo(), MockBar() )a }b RIBS: https://github.com/uber/RIBs/blob/master/android/libraries/rib- compiler-test/src/main/java/com/uber/rib/compiler/ InteractorTestGenerator.java Injection Helper: https://github.com/dinosaurwithakatana/injection- helper

Slide 125

Slide 125 text

What about state?

Slide 126

Slide 126 text

State machines!

Slide 127

Slide 127 text

In short

Slide 128

Slide 128 text

Value is the boundary

Slide 129

Slide 129 text

Value is the boundary assertThat(calculator.add(2, 2)).isEqualTo(4) assertThat(resultHandler.results[0]).isEqualTo(4) Observable.just(1) .test() .assertValue(...) .assertError(...) .awaitDone(...)

Slide 130

Slide 130 text

Fakes are worth writing

Slide 131

Slide 131 text

Fakes are worth writing class InMemoryRAM : RAM { private val store = mutableMapOf() override fun storeInRAM(key: String, value: T) { return store[key] = value } override fun loadFromRam(key: String): T? { return store[key] } }

Slide 132

Slide 132 text

Ship your fakes

Slide 133

Slide 133 text

Ship your fakes fun test(): TestObserver TestScopeProvider Room.inMemoryDatabaseBuilder(...)

Slide 134

Slide 134 text

Tests should test the public API

Slide 135

Slide 135 text

Tests should test the public API Semver your tests!

Slide 136

Slide 136 text

Test are prod code

Slide 137

Slide 137 text

Failures should be designed for

Slide 138

Slide 138 text

Failures should be designed for throw AssertionError("Should have crashed before this") catch (e: ExpectedException) { assertThat(e).isWhatWeExpected() } @Rule val errorsRule = RxErrorsRule()

Slide 139

Slide 139 text

More than one right way

Slide 140

Slide 140 text

Keep it simple.

Slide 141

Slide 141 text

Keep it pragmatic.

Slide 142

Slide 142 text

Further reading • Boundaries: https:// www.destroyallsoftware.com/talks/boundaries • Square libraries • RxJava • AutoDispose

Slide 143

Slide 143 text

Fini, merci! Zac Sweers - Uber @pandanomic