Share
Embed
Deep Testing
Filipe Freire
June 27, 2019
Transcript
23 slides
-
ABOUT ME JUN-19 DEEP TESTING 2 Filipe Freire – Software
Tester. Developer. Husband. Learner. Open- source software lover and contributor. Adidas projects: worked in .com’s Embellishment & Customization, Mobile App Backend APIs and currently Hype 2.0 Visitmypersonal blog: filfreire.com -
“WE ARE IN THE BUSINESS OF WINNING WARS (…)” KELSEY
GRAMMER AS “GEN. PARTRIDGE” -“THE PENTAGON WARS” 1998 -
THE PENTAGON WARS JUN-19 DEEP TESTING 6 “(I wanted the
Army to run realistic tests) to see what would happen to it (the Bradley) and the people inside when it was hit by weapons we could expect on the battlefield. The army did not want to run these tests.” – James Burton Shallow testing example -
DEEPSEA CHALLENGER JUN-19 DEEP TESTING 7 Ron Allum:“Our solutions draw
upon thousands of hours of practical experience at sea and at depth.” Deep testing example -
“DEEP SEA” ATTITUDE JUN-19 DEEP TESTING 8 deep testing: whatever
it is I’m testing I expect to find every single important bug that is out there to find -
JUN-19 DEEP TESTING 9 “If you don't pursue the questions
beneath the surface, you generally won't obtain your money's worth out of your investment in testing.” - Gerald M. Weinberg -
IDENTITY JUN-19 DEEP TESTING 11 We trust doctors, mechanics, plumbers…
We don’t tell them what/how to do. As a tester… do I own my identity? or I’m owned by it? -
PRECONCEIVED IDENTITY JUN-19 DEEP TESTING 12 Imposed/preconceived - Gatekeeper -
I create tons of test cases or “Automate all the testing and test cases” - I decide and “give the go” for the manager/owner Self-imposed/self-owned - Trustworthy - I hunt for meaningful risks and bugs, using any means necessary - I inform the manager/owner who is paid to decide “go” -
POSTURE JUN-19 DEEP TESTING 13 Am I ready to test…
Anything? Anytime? Right now? Complex stuff? Using any tools? -
OWNERSHIP JUN-19 DEEP TESTING 14 “I am the go-to tester
to talk about testing done in the project I’m with.” Do I own: the tools? Envs? Code reviews? Bug backlog? … Example heuristic: “I get logs to help Devs with debug info?” -
NO BULL$%1T JUN-19 DEEP TESTING 15 Do I use: the
process, the bureaucracy, the test cases, the time (or lack of it), the environment… As an excuse to take zero action? Example heuristic: Can I prepare, without previous notice, a load test for my entire API in under 3 hours? -
“WHAT BLINDS MY SIGHT?” JUN-19 DEEP TESTING 16 - I
don’t have test cases! Is that bad? - I don’t have all the information! So? - I don’t have time! Why? - … I’m blocked until… I 1) understand and 2) do something about it! -
JUN-19 DEEP TESTING 17 “Sound decisions cannot be made in
an environment where truth is hard to find.” - James G. Burton -
EATING MY DOG‘S FOOD JUN-19 DEEP TESTING 18 Q1 (&
Q2) 2019 . Hardcore project and timelines. . Complex architecture & business logic (tons of malicious checks). . 25 business weeks. . Every second counts. test strategy: mindmap test plan: 5 paragraphs 0 test cases >100 meaningful bugs reported >400 avg http req/day 2-4 merges to “master“ / day 7 diff api services to test Release to prod (few hours)= $ >6 figures -
NUMBER OF BUGS? JUN-19 DEEP TESTING 19 “Counting beans =
pointless” Every single bug tells a story through time. “Bugs are not beans” Our prio (testing perspective): find and report the most important ones in the time given… How can I direct my testing to do this? Eating my Dog‘s food -
JUN-19 DEEP TESTING 20 “(Be careful about) falling victim of
equating information quality with data quantity (bug counts, pages of test reports, number of test cases run, and so on)” - Gerald M. Weinberg -
CONTEXT MATTERS! JUN-19 DEEP TESTING 21 - More critical exploit
paths than regular purchase flow; - Persistence tech changed halfway; - Architecture changed halfway; - Lost documentation at one point; - Constant changes and improvements; Eating my Dog‘s food Underlying need to be adaptable. (Tools: postman, curl, wget, kubectl, aws cli, jenkins, docker, bash, charles, sublime, git… … and our “minds”) -
Meaningful Testing WRAPPING UP JUN-19 DEEP TESTING 22 NO BULL$%1T
IDENTITY POSTURE OWNERSHIP Structured approach to testing Context-adapted approach to testing