Deep Testing

Transcript

1. FILIPE FREIRE – JUNE 2019 ADIDAS TECH SUMMIT DEEP TESTING

2. ABOUT ME JUN-19 DEEP TESTING 2 Filipe Freire – Software

   Tester. Developer. Husband. Learner. Open- source software lover and contributor. Adidas projects: worked in .com’s Embellishment & Customization, Mobile App Backend APIs and currently Hype 2.0 Visitmypersonal blog: filfreire.com

3. “WE ARE IN THE BUSINESS OF WINNING WARS (…)” KELSEY

   GRAMMER AS “GEN. PARTRIDGE” -“THE PENTAGON WARS” 1998

4. Testing a laser-guided bomb: place BOMB above the target

5. Testing heat seeking missile: GLUE tons of hotplates the vehicle

6. THE PENTAGON WARS JUN-19 DEEP TESTING 6 “(I wanted the

   Army to run realistic tests) to see what would happen to it (the Bradley) and the people inside when it was hit by weapons we could expect on the battlefield. The army did not want to run these tests.” – James Burton Shallow testing example

7. DEEPSEA CHALLENGER JUN-19 DEEP TESTING 7 Ron Allum:“Our solutions draw

   upon thousands of hours of practical experience at sea and at depth.” Deep testing example

8. “DEEP SEA” ATTITUDE JUN-19 DEEP TESTING 8 deep testing: whatever

   it is I’m testing I expect to find every single important bug that is out there to find

9. JUN-19 DEEP TESTING 9 “If you don't pursue the questions

   beneath the surface, you generally won't obtain your money's worth out of your investment in testing.” - Gerald M. Weinberg

10. GOING DEEP JUN-19 DEEP TESTING 10 IDENTITY POSTURE OWNERSHIP NO

    BULL$%1t

11. IDENTITY JUN-19 DEEP TESTING 11 We trust doctors, mechanics, plumbers…

    We don’t tell them what/how to do. As a tester… do I own my identity? or I’m owned by it?

12. PRECONCEIVED IDENTITY JUN-19 DEEP TESTING 12 Imposed/preconceived - Gatekeeper -

    I create tons of test cases or “Automate all the testing and test cases” - I decide and “give the go” for the manager/owner Self-imposed/self-owned - Trustworthy - I hunt for meaningful risks and bugs, using any means necessary - I inform the manager/owner who is paid to decide “go”

13. POSTURE JUN-19 DEEP TESTING 13 Am I ready to test…

    Anything? Anytime? Right now? Complex stuff? Using any tools?

14. OWNERSHIP JUN-19 DEEP TESTING 14 “I am the go-to tester

    to talk about testing done in the project I’m with.” Do I own: the tools? Envs? Code reviews? Bug backlog? … Example heuristic: “I get logs to help Devs with debug info?”

15. NO BULL$%1T JUN-19 DEEP TESTING 15 Do I use: the

    process, the bureaucracy, the test cases, the time (or lack of it), the environment… As an excuse to take zero action? Example heuristic: Can I prepare, without previous notice, a load test for my entire API in under 3 hours?

16. “WHAT BLINDS MY SIGHT?” JUN-19 DEEP TESTING 16 - I

    don’t have test cases! Is that bad? - I don’t have all the information! So? - I don’t have time! Why? - … I’m blocked until… I 1) understand and 2) do something about it!

17. JUN-19 DEEP TESTING 17 “Sound decisions cannot be made in

    an environment where truth is hard to find.” - James G. Burton

18. EATING MY DOG‘S FOOD JUN-19 DEEP TESTING 18 Q1 (&

    Q2) 2019 . Hardcore project and timelines. . Complex architecture & business logic (tons of malicious checks). . 25 business weeks. . Every second counts. test strategy: mindmap test plan: 5 paragraphs 0 test cases >100 meaningful bugs reported >400 avg http req/day 2-4 merges to “master“ / day 7 diff api services to test Release to prod (few hours)= $ >6 figures

19. NUMBER OF BUGS? JUN-19 DEEP TESTING 19 “Counting beans =

    pointless” Every single bug tells a story through time. “Bugs are not beans” Our prio (testing perspective): find and report the most important ones in the time given… How can I direct my testing to do this? Eating my Dog‘s food

20. JUN-19 DEEP TESTING 20 “(Be careful about) falling victim of

    equating information quality with data quantity (bug counts, pages of test reports, number of test cases run, and so on)” - Gerald M. Weinberg

21. CONTEXT MATTERS! JUN-19 DEEP TESTING 21 - More critical exploit

    paths than regular purchase flow; - Persistence tech changed halfway; - Architecture changed halfway; - Lost documentation at one point; - Constant changes and improvements; Eating my Dog‘s food Underlying need to be adaptable. (Tools: postman, curl, wget, kubectl, aws cli, jenkins, docker, bash, charles, sublime, git… … and our “minds”)

22. Meaningful Testing WRAPPING UP JUN-19 DEEP TESTING 22 NO BULL$%1T

    IDENTITY POSTURE OWNERSHIP Structured approach to testing Context-adapted approach to testing

23. THANK YOU! QUESTIONS?