TESTING IOS APP WITH EXTERNAL HARDWARE THE PROBLEM Bluetooth LE WiFi Cable Connection Bluetooth Classic v.1.0.0 v.1.1.0 v.2.0.0 v.2.1.0 Firmware RESPONSIBLE FOR VERSIONING
TESTING IOS APP WITH EXTERNAL HARDWARE RESOLUTION: ARCHITECTURE Connectivity Layer Bluetooth LE WiFi Cable Connection Bluetooth Classic • System layer, e.g. Core Bluetooth
TESTING IOS APP WITH EXTERNAL HARDWARE RESOLUTION: ARCHITECTURE Connectivity Layer Bluetooth LE WiFi Cable Connection Bluetooth Classic Transport Layer • StateLESS layer • Request/Response • Device specific components • System layer, e.g. Core Bluetooth
TESTING IOS APP WITH EXTERNAL HARDWARE RESOLUTION: ARCHITECTURE Connectivity Layer Bluetooth LE WiFi Cable Connection Bluetooth Classic Transport Layer • StateLESS layer • Request/Response • Device specific components • System layer, e.g. Core Bluetooth Business Logic Layer • StateFULL layer • Device agnostic components
Unit Tests Cases TESTING IOS APP WITH EXTERNAL HARDWARE UNIT TESTING Business Logic Layer • StateFULL layer • Device agnostic components Mock Transport Layer
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITH REAL DEVICE Connectivity Layer Connectivity Layer Abstraction Transport Layer • StateLESS layer • Request/Response • Device specific components • System layer, e.g. Core Bluetooth Integration Tests
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITH REAL DEVICE ▸ Guarantee to work with Real device ▸ Nothing to mock: faster than Unit Tests PROS CONS
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITH REAL DEVICE ▸ Guarantee to work with Real device ▸ Nothing to mock: faster than Unit Tests PROS ▸ Hard to cover corner cases CONS
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITH REAL DEVICE ▸ Guarantee to work with Real device ▸ Nothing to mock: faster than Unit Tests PROS ▸ Hard to cover corner cases ▸ Not always stable CONS
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITH REAL DEVICE ▸ Guarantee to work with Real device ▸ Nothing to mock: faster than Unit Tests PROS ▸ Hard to cover corner cases ▸ Not always stable ▸ Hard to automate (CI) CONS
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITH REAL DEVICE ▸ Guarantee to work with Real device ▸ Nothing to mock: faster than Unit Tests PROS ▸ Hard to cover corner cases ▸ Not always stable ▸ Hard to automate (CI) ▸ Run takes more time CONS
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITH DEVICE SIMULATOR ▸ Limited guarantee to work with Real device ▸ No device prototype needed PROS CONS
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITH DEVICE SIMULATOR ▸ Limited guarantee to work with Real device ▸ No device prototype needed ▸ Easy to cover edge cases PROS CONS
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITH DEVICE SIMULATOR ▸ Limited guarantee to work with Real device ▸ No device prototype needed ▸ Easy to cover edge cases PROS ▸ Need to follow specs very carefully CONS
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITH DEVICE SIMULATOR ▸ Limited guarantee to work with Real device ▸ No device prototype needed ▸ Easy to cover edge cases PROS ▸ Need to follow specs very carefully ▸ An additional codebase to support CONS
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITH DEVICE SIMULATOR ▸ Limited guarantee to work with Real device ▸ No device prototype needed ▸ Easy to cover edge cases PROS ▸ Need to follow specs very carefully ▸ An additional codebase to support ▸ Codebase grows with new Firmware/ Devices CONS
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITHIN TEST HARNESS Connectivity Layer Connectivity Layer Abstraction Transport Layer • StateLESS layer • Request/Response • Device specific components • System layer, e.g. Core Bluetooth Integration Tests
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITHIN TEST HARNESS Connectivity Layer Connectivity Layer Abstraction Transport Layer • StateLESS layer • Request/Response • Device specific components • System layer, e.g. Core Bluetooth Integration Tests
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITHIN TEST HARNESS Connectivity Layer Connectivity Layer Abstraction Transport Layer • StateLESS layer • Request/Response • Device specific components • System layer, e.g. Core Bluetooth Integration Tests Test Harness iOS App
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITHIN TEST HARNESS ▸ Can share build with QA-, FW- Engineers ▸ Easier to test new firmware/hardware PROS CONS
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITHIN TEST HARNESS ▸ Can share build with QA-, FW- Engineers ▸ Easier to test new firmware/hardware ▸ Android: easy to test on exotic device PROS CONS
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITHIN TEST HARNESS ▸ Can share build with QA-, FW- Engineers ▸ Easier to test new firmware/hardware ▸ Android: easy to test on exotic device PROS ▸ Need to reimplement some XCTest tools CONS
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITHIN TEST HARNESS ▸ Can share build with QA-, FW- Engineers ▸ Easier to test new firmware/hardware ▸ Android: easy to test on exotic device PROS ▸ Need to reimplement some XCTest tools ▸ Need to keep transport layer in Framework CONS
TESTING IOS APP WITH EXTERNAL HARDWARE INTEGRATION TESTING WITHIN TEST HARNESS ▸ Can share build with QA-, FW- Engineers ▸ Easier to test new firmware/hardware ▸ Android: easy to test on exotic device PROS ▸ Need to reimplement some XCTest tools ▸ Need to keep transport layer in Framework ▸ Need to support another codebase CONS
TESTING IOS APP WITH EXTERNAL HARDWARE BEST PRACTICES ▸ Cover happy path + corner cases via Unit Tests CI - must-have ▸ Cover happy path ONLY via Integration Tests CI-optional
TESTING IOS APP WITH EXTERNAL HARDWARE BEST PRACTICES ▸ Cover happy path + corner cases via Unit Tests CI - must-have ▸ Cover happy path ONLY via Integration Tests CI-optional ▸ No device - use Simulator
TESTING IOS APP WITH EXTERNAL HARDWARE SHOWCASE 1: COFFEE ROASTING MACHINE Coffee roasting machine Hardware Unit Hardware Unit Hardware Unit Hardware Unit
TESTING IOS APP WITH EXTERNAL HARDWARE SHOWCASE 1: COFFEE ROASTING MACHINE Coffee roasting machine Hardware Unit Hardware Unit Hardware Unit Hardware Unit USB
TESTING IOS APP WITH EXTERNAL HARDWARE SHOWCASE 1: COFFEE ROASTING MACHINE Coffee roasting machine Hardware Unit Hardware Unit Hardware Unit Hardware Unit USB
Simulator TESTING IOS APP WITH EXTERNAL HARDWARE SHOWCASE 1: COFFEE ROASTING MACHINE Coffee roasting machine Bluetooth LE WiFi USB Arduino Uno Integration Tests
TESTING IOS APP WITH EXTERNAL HARDWARE SHOWCASE 2: MEDICAL WEARABLE Data synchronization algorithm Transport Layer Integration Tests Cases Test Harness iOS App
TESTING IOS APP WITH EXTERNAL HARDWARE INSTEAD OF CONCLUSIONS ▸ Don’t sell your customer auto tests - Include those to estimates ▸ 100% code coverage is never needed - Find level needed by your app
TESTING IOS APP WITH EXTERNAL HARDWARE INSTEAD OF CONCLUSIONS ▸ Don’t sell your customer auto tests - Include those to estimates ▸ 100% code coverage is never needed - Find level needed by your app ▸ Try both sides of the force: Both under-testing and over-testing
cocoaheadsukraine
lovyan03
takuyay0ne
cer
katsyoshi
little_rubyist
track3jyo
punchdrunker
m_seki
hitokun
naokimrmt
kishida
masatoshiitoh
notwaldorf
speakerdeck
chrislema
kneath
morganepeng
rasmusluckow
lara
tmm1
shlominoach
paulrobertlloyd
keathley