Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Flaky test - not on my watch

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Andrejs Kalnačs Andrejs Kalnačs
October 10, 2019
Programming
0
120

Flaky test - not on my watch

Presentation about tooling which we are using to collect meta data to be aware about flaky tests

Avatar for Andrejs Kalnačs

Andrejs Kalnačs

October 10, 2019
Tweet

More Decks by Andrejs Kalnačs

See All by Andrejs Kalnačs
Loading time testing of web games – set-up, execution and visualization of results @ TestingStage '18
Avatar for Andrejs Kalnačs andrejska
0
230
From numbers to graphs: visualisation as a tool for better understanding of testing @ ITEM 2018 Kiev
Avatar for Andrejs Kalnačs andrejska
0
66
Grafana usage in the Quality Assurance Process @ GrafanaCon 2018
Avatar for Andrejs Kalnačs andrejska
0
250
Loading time testing and results visualisation of web games @ Heisenbug Moscow 2017
Avatar for Andrejs Kalnačs andrejska
0
120
Loading time testing and results visualisation of web games @ SQA Days 22
Avatar for Andrejs Kalnačs andrejska
0
60
Loading time testing and results visualisation of web games @ TAPOST 2017
Avatar for Andrejs Kalnačs andrejska
0
75

Other Decks in Programming

See All in Programming
CSC307 Lecture 10
Avatar for Javier Gonzalez-Sanchez javiergs
PRO
1
660
Architectural Extensions
Avatar for Denys Poltorak denyspoltorak
0
290
CSC307 Lecture 03
Avatar for Javier Gonzalez-Sanchez javiergs
PRO
1
490
Fluid Templating in TYPO3 14
Avatar for Simon Praetorius s2b
0
130
AIエージェント、”どう作るか”で差は出るか？ / AI Agents: Does the "How" Make a Difference?
Avatar for r-kagaya rkaga
4
2k
Patterns of Patterns
Avatar for Denys Poltorak denyspoltorak
0
1.4k
AIによる高速開発をどう制御するか？ ガードレール設置で開発速度と品質を両立させたチームの事例
Avatar for tonkotsuboy_com tonkotsuboy_com
7
2.4k
AI巻き込み型コードレビューのススメ
Avatar for Nealle nealle
2
420
AWS re:Invent 2025参加 直前 Seattle-Tacoma Airport(SEA)におけるハードウェア紛失インシデントLT
Avatar for tetutetu214 tetutetu214
2
120
Apache Iceberg V3 and migration to V3
Avatar for Tomohiro Tanaka tomtanaka
0
160
OSSとなったswift-buildで Xcodeのビルドを差し替えられるため 自分でXcodeを直せる時代になっている ダイアモンド問題編
Avatar for yimajo yimajo
3
620
MDN Web Docs に日本語翻訳でコントリビュート
Avatar for uutan1108 ohmori_yusuke
0
650

Featured

See All Featured
The Organizational Zoo: Understanding Human Behavior Agility Through Metaphoric Constructive Conversations (based on the works of Arthur Shelley, Ph.D)
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
240
Typedesign – Prime Four
Avatar for Hannes Fritz hannesfritz
42
2.9k
SEO in 2025: How to Prepare for the Future of Search
Avatar for Michael King ipullrank
3
3.3k
Bootstrapping a Software Product
Avatar for Garrett Dimon garrettdimon
PRO
307
120k
Everyday Curiosity
Avatar for Cassini Nazir cassininazir
0
130
Odyssey Design
Avatar for Ryan Kendrick rkendrick25
PRO
1
500
Designing for Performance
Avatar for Lara Hogan lara
610
70k
Avoiding the “Bad Training, Faster” Trap in the Age of AI
Avatar for Mike Taylor tmiket
0
77
How to Get Subject Matter Experts Bought In and Actively Contributing to SEO & PR Initiatives.
Avatar for Liv Day livdayseo
0
67
Jess Joyce - The Pitfalls of Following Frameworks
Avatar for Tech SEO Connect techseoconnect
PRO
1
66
B2B Lead Gen: Tactics, Traps & Triumph
Avatar for Sophie Logan marketingsoph
0
55
The #1 spot is gone: here's how to win anyway
Avatar for Tamara Novitovic tamaranovitovic
2
940

Transcript

  1. Flaky tests - not on my watch Andrejs Kalnačs Scout

    RFP TAPOST 2019

  2. 2 who am i QA Automation Lead at Scout RFP

    10+ years of industry experience in QA Scout RFP, Nokia, Evolution Gaming, Accenture, Nokia Networks Sharing experience in conferences for 2 years Riga Testing Automation club meet-up co-organizer TesterFromRiga

  3. 3 Scout RFP San Francisco based late stage start-up Sourcing

    and procurement softwware Top reverse auctions technology provider 250+ clients

  4. 4 Flaky tests = Non deterministic ✅ ✅ ❌ ✅

  5. 5 Tests should fail expected failure false negative

  6. 6 Scale 8 diffeerent test types 247758 tests

  7. 7 Scale

  8. 8 retry() Insanity Is Doing the Same Thing Over and

    Over Again and Expecting Diffeerent Results Albert Einstein

  9. 9 Retry once!

  10. 10 Not to mention

  11. 11 Be informed

  12. 12 How to retry logic test executor listener capability slack

    bindings database bindings

  13. 13 Time series database Open Source* SQL-like (InfluxQL) or jq-like

    (Flux) syntax

  14. 14 Supports many data sources Open Source* Packed with features

    Extendable

  15. 15 Execution dashboard

  16. 16 Test dashboard

  17. 17 More dashboards

  18. 18 Our future Merge train Advanced solutions

  19. 19 Key takeaways Retry is dangerous but it is not

    evil Collect meta data Always evolve

  20. 20 Thank you TesterFromRiga Andrejs Kalnačs Queestions? andrejska