Dipping the toes into the sea of testing

An overview of testing world, basically to draw a map and define a language to talk about testing.

Transcript

1. 09.11.2017 1 Dipping the toes into the sea of testing by Michael Pershyn
2. 09.11.2017 2 Motivation • My experience with Testing • Testing, SOLID and Unit Testing • Testing is a big thing – Area complexity – Efort & Cost • Let‘s walk through and make a map
3. 09.11.2017 3 Agenda • Defnitions – What is Testing – And some other defnitions • Goals of Testing • Objectives of Testing • Quality Traits of Software • Methods of Testing • Levels of Testing • Artifacts of Testing • Small Remark about DevOps
4. 09.11.2017 4 Defnitions SStandards„ • SWEBOK – Software Engineering Body of Knowledge – aka ISO/IEC TR 19759 2005 – From IEEE • ISTQB – International Software Testing Qualifcations Board – Certify Test Engineers
5. 09.11.2017 5 Defnitions Software Testing A set of processes aimed at investigating, evaluating and determining the completeness, quality and purpose ftting of software
6. 09.11.2017 6 Defnitions 啤alidation and 啤erifcation Validation and Verifiation is the process of checking that a software system meets specifcations and that it fulflls its intended purpose 啤alidation Are we building the right product? 啤erifcation Are we building the product right?
7. 09.11.2017 7 Defnitions Defect and Failure • Defeit vs bug vs fault – When actual result deviates from the expected (specifed) result – Also known as Bug or Fault • Failure – Application Crash
8. 09.11.2017 8 Defnitions Component, Use Case and Test • Test - an experiment to fnd answers to the question "Does the software in general or specifc function works as expected" • Component is an identifable part of a larger program or construction – Unit is the smallest compilable component – A unit is a component – The integration of one or more components is a component • Use Case - the specifcation of tests that are conducted from the end-user perspective
9. 09.11.2017 9 Defnitions QA, QC and Testing • Quality Assurance • Implementation of proiesses, proiedures and standards in context to verifcation of developed software and intended requirements • Subset of SDLC (Software Development Live Cycle) • Quality Control • Activities that ensure the verifcation of a developed software with respeit to doiumented (or not in some cases) requirements • Subset of Quality Assurance • Testing • Activities that ensure the identifiation of bugs/error/defeits in a software • Subset of Quality Control
10. 09.11.2017 10 Defnitions Test Engineer vs Tester
11. 09.11.2017 11 Agenda (Reminder) • [x] Defnitions – What is Testing – And some other defnitions • [ ] Goals of Testing • [ ] Objectives of Testing • [ ] Quality Traits of Software • [ ] Methods of Testing • [ ] Levels of Testing • [ ] Artifacts of Testing • [ ] Small Remark about DevOps
12. 09.11.2017 12 Goals of Testing • Find defects • Measure and improve quality • Prevent defects • Ensure that the end result meets the requirements
13. 09.11.2017 13 Agenda (Reminder) • [x] Defnitions – What is Testing – And some other defnitions • [x] Goals of Testing • [ ] Objeitives of Testing • [ ] Quality Traits of Software • [ ] Methods of Testing • [ ] Levels of Testing • [ ] Artifacts of Testing • [ ] Small Remark about DevOps
14. 09.11.2017 14 Objectives of Testing • Installation • Compatibility – Is software compatible with other components (e.g. browsers) • Smoke – Run a set of test cases to check the stability of entire system – Wide and shallow • Sanity – After fxing a bug ensure 1 or 2 features are working – Narrow and deep – Usually performed after smoke test
15. 09.11.2017 15 Objectives of Testing 2 • Regression – Run all of the tests to check how the proportion of green test changed. Are there side efeits of any new ihange or enhancements done in the application? • Retesting – Check that partiiular bug has been fxed • Ad Hoc Testing – QA Engineer tries to 'break' the system by randomly trying the system • Acceptance, Apha and Beta – See the Levels of Testing
16. 09.11.2017 16 Objectives of Testing 3 • Positive vs Negative – Positive Testing - to show the software works. Aka "test to pass" – Negative Testing. Aka "test to fail" • Functional with non-Functional – Functional (refers to Levels) • Unit, Integration, System, Acceptance, Alpha, Beta – Non-Functional (group) • Performance, Load, Stress, Usability, Security, Portability
17. 09.11.2017 17 Objectives of Testing 4 • Stress – Testing on overloads to fnd weak spots – System is expected to fail in a decent manner • Performance – Load Testing • Testing on diferent load, gradually increased from 0 to 100% – Endurance • Normal load on long time (days) – Soak Testing • High performance during a long time (days) • Ramp Testing – Raising input signal till the system breaks down
18. 09.11.2017 18 Objectives of Testing 5 • A/B • Usability • Accessibility • Recovery • Security • Internationalization and localization • Conformance (aka Compliance) • ...
19. 09.11.2017 19 Agenda (Reminder) • [x] Defnitions – What is Testing – And some other defnitions • [x] Goals of Testing • [x] Objectives of Testing • [ ] Quality Traits of Software • [ ] Methods of Testing • [ ] Levels of Testing • [ ] Artifacts of Testing • [ ] Small Remark about DevOps
20. 09.11.2017 20 Quality Traits of Software • Abstraction • Cohesion • Completeness • Extencibility • Fault tolerance • Maintainability • Reliability • Robustness • Scalability • Testability • ...
21. 09.11.2017 21 Agenda (Reminder) • [x] Defnitions – What is Testing – And some other defnitions • [x] Goals of Testing • [x] Objectives of Testing • [x] Quality Traits of Software • [ ] Methods of Testing • [ ] Levels of Testing • [ ] Artifacts of Testing • [ ] Small Remark about DevOps
22. 09.11.2017 22 Methods of Testing Overview • The Box Approach – Black Box – White Box – Grey Box • Static vs Dynamic • 啤isual
23. 09.11.2017 23 Methods of Testing Black Box • Design Techiques – Decision Table – All-pairs Testing – State Transition Reference – Equivalence Partitioning – Boundary 啤alue Analysis • Penetration Hacking
24. 09.11.2017 24 Methods of Testing White Box • Design Techiques – Statement Coverage – Decision Coverage – Flow of Data – Flow of Control – Branch Testing – Path Testing – Mutation Testing • Penetration Hacking
25. 09.11.2017 25 Methods of Testing Grey Box • Design Techiques – Matrix Testing – Regression Testing – Pattern Testing – Orthogonal Array Testing
26. 09.11.2017 26 Methods of Testing Static vs Dynamic • Static – Code Review – Code Walkthrough – Code Inspection • Dynamic – Executing program with a set of test cases
27. 09.11.2017 27 Methods of Testing 啤isual • 啤isual Inspection with a set of heuristics
28. 09.11.2017 28 Agenda (Reminder) • [x] Defnitions – What is Testing – And some other defnitions • [x] Goals of Testing • [x] Objectives of Testing • [x] Quality Traits of Software • [x] Methods of Testing • [ ] Levels of Testing • [ ] Artifacts of Testing • [ ] Small Remark about DevOps
29. 09.11.2017 29 Levels of Testing • There are lots of diferent views on this topic • Generally it depends on product • Let‘s go through the SWEBOK and ISTQB common case • And also look at 啤-Model
30. 09.11.2017 30 Levels of Testing SWEBOK (IEEE) • Unit • Integration – top-down – bottom-up • System
31. 09.11.2017 31 Levels of Testing ISTQB • Unit • Integration – Approach • top-down • bottom-up – Levels • Component • Component Integration • System Integration • System • Acceptance • Alpha • Beta
32. 09.11.2017 32 Levels of Testing 啤-Model
33. 09.11.2017 33 Agenda (Reminder) • [x] Defnitions – What is Testing – And some other defnitions • [x] Goals of Testing • [x] Objectives of Testing • [x] Quality Traits of Software • [x] Methods of Testing • [x] Levels of Testing • [ ] Artifaits of Testing • [ ] Small Remark about DevOps
34. 09.11.2017 34 Artifacts of Testing • Test plan – scope, approach, resources, and schedule of intended testing – identifes test items, the features to be tested, the testing tasks, who will do each task, and any risks to concider • Test case – requirement references – preconditions – events – steps (actions) to follow – input – output – expected result – actual result
35. 09.11.2017 35 Artifacts of Testing 2 • Traceability matrix • Test script • Test suite – Collection of test cases • Test fxture – Test data used with test cases, but separated • Test harness – Software – Tools – Confgurations
36. 09.11.2017 36 Agenda (Reminder) • [x] Defnitions – What is Testing – And some other defnitions • [x] Goals of Testing • [x] Objectives of Testing • [x] Quality Traits of Software • [x] Methods of Testing • [x] Levels of Testing • [x] Artifacts of Testing • [ ] Small Remark about DevOps
37. 09.11.2017 37 Small Remark about DevOps
38. 09.11.2017 38 Small Remark about DevOps 2
39. 09.11.2017 39 Small Remark about DevOps 3
40. 09.11.2017 40 Questions and Thanks