FILIPE FREIRE – JUNE 2019 ADIDAS TECH SUMMIT DEEP TESTING

ABOUT ME JUN-19 DEEP TESTING 2 Filipe Freire – Software Tester. Developer. Husband. Learner. Open- source software lover and contributor. Adidas projects: worked in .com’s Embellishment & Customization, Mobile App Backend APIs and currently Hype 2.0 Visitmypersonal blog: filfreire.com

“WE ARE IN THE BUSINESS OF WINNING WARS (…)” KELSEY GRAMMER AS “GEN. PARTRIDGE” -“THE PENTAGON WARS” 1998

Testing a laser-guided bomb: place BOMB above the target

Testing heat seeking missile: GLUE tons of hotplates the vehicle

THE PENTAGON WARS JUN-19 DEEP TESTING 6 “(I wanted the Army to run realistic tests) to see what would happen to it (the Bradley) and the people inside when it was hit by weapons we could expect on the battlefield. The army did not want to run these tests.” – James Burton Shallow testing example

DEEPSEA CHALLENGER JUN-19 DEEP TESTING 7 Ron Allum:“Our solutions draw upon thousands of hours of practical experience at sea and at depth.” Deep testing example

“DEEP SEA” ATTITUDE JUN-19 DEEP TESTING 8 deep testing: whatever it is I’m testing I expect to find every single important bug that is out there to find

JUN-19 DEEP TESTING 9 “If you don't pursue the questions beneath the surface, you generally won't obtain your money's worth out of your investment in testing.” - Gerald M. Weinberg

GOING DEEP JUN-19 DEEP TESTING 10 IDENTITY POSTURE OWNERSHIP NO BULL$%1t

IDENTITY JUN-19 DEEP TESTING 11 We trust doctors, mechanics, plumbers… We don’t tell them what/how to do. As a tester… do I own my identity? or I’m owned by it?

PRECONCEIVED IDENTITY JUN-19 DEEP TESTING 12 Imposed/preconceived - Gatekeeper - I create tons of test cases or “Automate all the testing and test cases” - I decide and “give the go” for the manager/owner Self-imposed/self-owned - Trustworthy - I hunt for meaningful risks and bugs, using any means necessary - I inform the manager/owner who is paid to decide “go”

POSTURE JUN-19 DEEP TESTING 13 Am I ready to test… Anything? Anytime? Right now? Complex stuff? Using any tools?

OWNERSHIP JUN-19 DEEP TESTING 14 “I am the go-to tester to talk about testing done in the project I’m with.” Do I own: the tools? Envs? Code reviews? Bug backlog? … Example heuristic: “I get logs to help Devs with debug info?”

NO BULL$%1T JUN-19 DEEP TESTING 15 Do I use: the process, the bureaucracy, the test cases, the time (or lack of it), the environment… As an excuse to take zero action? Example heuristic: Can I prepare, without previous notice, a load test for my entire API in under 3 hours?

“WHAT BLINDS MY SIGHT?” JUN-19 DEEP TESTING 16 - I don’t have test cases! Is that bad? - I don’t have all the information! So? - I don’t have time! Why? - … I’m blocked until… I 1) understand and 2) do something about it!

JUN-19 DEEP TESTING 17 “Sound decisions cannot be made in an environment where truth is hard to find.” - James G. Burton

EATING MY DOG‘S FOOD JUN-19 DEEP TESTING 18 Q1 (& Q2) 2019 . Hardcore project and timelines. . Complex architecture & business logic (tons of malicious checks). . 25 business weeks. . Every second counts. test strategy: mindmap test plan: 5 paragraphs 0 test cases >100 meaningful bugs reported >400 avg http req/day 2-4 merges to “master“ / day 7 diff api services to test Release to prod (few hours)= $ >6 figures

NUMBER OF BUGS? JUN-19 DEEP TESTING 19 “Counting beans = pointless” Every single bug tells a story through time. “Bugs are not beans” Our prio (testing perspective): find and report the most important ones in the time given… How can I direct my testing to do this? Eating my Dog‘s food

JUN-19 DEEP TESTING 20 “(Be careful about) falling victim of equating information quality with data quantity (bug counts, pages of test reports, number of test cases run, and so on)” - Gerald M. Weinberg

CONTEXT MATTERS! JUN-19 DEEP TESTING 21 - More critical exploit paths than regular purchase flow; - Persistence tech changed halfway; - Architecture changed halfway; - Lost documentation at one point; - Constant changes and improvements; Eating my Dog‘s food Underlying need to be adaptable. (Tools: postman, curl, wget, kubectl, aws cli, jenkins, docker, bash, charles, sublime, git… … and our “minds”)

Meaningful Testing WRAPPING UP JUN-19 DEEP TESTING 22 NO BULL$%1T IDENTITY POSTURE OWNERSHIP Structured approach to testing Context-adapted approach to testing

THANK YOU! QUESTIONS?