Flaky test - not on my watch

Presentation about tooling which we are using to collect meta data to be aware about flaky tests

Transcript

1. Flaky tests - not on my watch Andrejs Kalnačs Scout RFP TAPOST 2019
2. 2 who am i QA Automation Lead at Scout RFP 10+ years of industry experience in QA Scout RFP, Nokia, Evolution Gaming, Accenture, Nokia Networks Sharing experience in conferences for 2 years Riga Testing Automation club meet-up co-organizer TesterFromRiga
3. 3 Scout RFP San Francisco based late stage start-up Sourcing and procurement softwware Top reverse auctions technology provider 250+ clients
4. 4 Flaky tests = Non deterministic ✅ ✅ ❌ ✅
5. 5 Tests should fail expected failure false negative
6. 6 Scale 8 diffeerent test types 247758 tests
7. 7 Scale
8. 8 retry() Insanity Is Doing the Same Thing Over and Over Again and Expecting Diffeerent Results Albert Einstein
9. 9 Retry once!
10. 10 Not to mention
11. 11 Be informed
12. 12 How to retry logic test executor listener capability slack bindings database bindings
13. 13 Time series database Open Source* SQL-like (InfluxQL) or jq-like (Flux) syntax
14. 14 Supports many data sources Open Source* Packed with features Extendable
15. 15 Execution dashboard
16. 16 Test dashboard
17. 17 More dashboards
18. 18 Our future Merge train Advanced solutions
19. 19 Key takeaways Retry is dangerous but it is not evil Collect meta data Always evolve
20. 20 Thank you TesterFromRiga Andrejs Kalnačs Queestions? andrejska