Slide 1

Slide 1 text

DATA MANAGEMENT HIGHLY RELIABLE TESTS @EmanuilSlavov [email protected]

Slide 2

Slide 2 text

High Level Automated Tests Problems Slow Unreliable @EmanuilSlavov

Slide 3

Slide 3 text

3 hours 3 minutes *Need for Speed: Accelerate Tests from 3 Hours to 3 Minutes

Slide 4

Slide 4 text

Falcon’s flaky test rate: 0.13% Google’s flaky test rate: 1.5%* *Flaky Tests at Google and How We Mitigate Them @EmanuilSlavov

Slide 5

Slide 5 text

Each test creates all the data that it needs. The way we achieved this @EmanuilSlavov

Slide 6

Slide 6 text

@EmanuilSlavov

Slide 7

Slide 7 text

@EmanuilSlavov

Slide 8

Slide 8 text

The time needed to create data for one test And then the test starts Call 12 API endpoints Modify data in 11 tables Takes about 1.2 seconds @EmanuilSlavov

Slide 9

Slide 9 text

Static vs Dynamic Data

Slide 10

Slide 10 text

@EmanuilSlavov

Slide 11

Slide 11 text

@EmanuilSlavov

Slide 12

Slide 12 text

Eum odit omnis impedit officia adipisci id non. random tweet '' Random Sentence Constant String Special Character random tweet Provident ipsa dolor excepturi quo asperiores animi. @someMention & random tweet Dignissimos eos accusamus aut ratione [email protected] random tweet Ut optio illum libero. Natus accusantium aliquam dolore atque voluptatum et a. http://ryanpacocha.biz/nikita random tweet @EmanuilSlavov

Slide 13

Slide 13 text

Service Virtualization Application Facebook Paypal Amazon S3 @EmanuilSlavov

Slide 14

Slide 14 text

Facebook Application Paypal Amazon S3 Proxy* Service Virtualization *github.com/emanuil/nagual

Slide 15

Slide 15 text

Existing Tools (March 2016) Transparent Fake SSL certs Dynamic Responses Persist State Return Binary Data Regex URL match Stubby4J WireMock Wilma soapUI MockServer mounteback Hoverfly Mirage

Slide 16

Slide 16 text

@EmanuilSlavov

Slide 17

Slide 17 text

Independent (run in isolation) Run in random order (do all the state setting) Run in parallel (to bring speed) Run on any database (only schema is needed) Easy to investigate (independent data per test) Catch more bugs (using realistic generators) @EmanuilSlavov Advantages

Slide 18

Slide 18 text

Tips & Tricks

Slide 19

Slide 19 text

Use an official interface to insert the test data Careful when testing in production - write operations Don’t expose test-only endpoints to the outside world @EmanuilSlavov

Slide 20

Slide 20 text

Test Data Cleaning

Slide 21

Slide 21 text

Each Test Deletes its Data @EmanuilSlavov

Slide 22

Slide 22 text

Tag Test Data @EmanuilSlavov

Slide 23

Slide 23 text

In case of a Dedicated Test Environment @EmanuilSlavov

Slide 24

Slide 24 text

Other Test Data Strategies

Slide 25

Slide 25 text

Use a dedicated test data set Use (sanitized) production data Seed a DB with test data before all tests start @EmanuilSlavov

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

FALCON.IO WE’RE HIRING. Sofia · Copenhagen · Budapest

Slide 28

Slide 28 text

[email protected] @EmanuilSlavov EmanuilSlavov.com