Share
Embed
Dipping the toes into the sea of testing
An overview of testing world, basically to draw a map and define a language to talk about testing.
Michael Pershyn
November 09, 2017
Transcript
40 slides
-
09.11.2017 1 Dipping the toes into the sea of testing
by Michael Pershyn -
09.11.2017 2 Motivation • My experience with Testing • Testing,
SOLID and Unit Testing • Testing is a big thing – Area complexity – Efort & Cost • Let‘s walk through and make a map -
09.11.2017 3 Agenda • Defnitions – What is Testing –
And some other defnitions • Goals of Testing • Objectives of Testing • Quality Traits of Software • Methods of Testing • Levels of Testing • Artifacts of Testing • Small Remark about DevOps -
09.11.2017 4 Defnitions SStandards„ • SWEBOK – Software Engineering Body
of Knowledge – aka ISO/IEC TR 19759 2005 – From IEEE • ISTQB – International Software Testing Qualifcations Board – Certify Test Engineers -
09.11.2017 5 Defnitions Software Testing A set of processes aimed
at investigating, evaluating and determining the completeness, quality and purpose ftting of software -
09.11.2017 6 Defnitions 啤alidation and 啤erifcation Validation and Verifiation is
the process of checking that a software system meets specifcations and that it fulflls its intended purpose 啤alidation Are we building the right product? 啤erifcation Are we building the product right? -
09.11.2017 7 Defnitions Defect and Failure • Defeit vs bug
vs fault – When actual result deviates from the expected (specifed) result – Also known as Bug or Fault • Failure – Application Crash -
09.11.2017 8 Defnitions Component, Use Case and Test • Test
- an experiment to fnd answers to the question "Does the software in general or specifc function works as expected" • Component is an identifable part of a larger program or construction – Unit is the smallest compilable component – A unit is a component – The integration of one or more components is a component • Use Case - the specifcation of tests that are conducted from the end-user perspective -
09.11.2017 9 Defnitions QA, QC and Testing • Quality Assurance
• Implementation of proiesses, proiedures and standards in context to verifcation of developed software and intended requirements • Subset of SDLC (Software Development Live Cycle) • Quality Control • Activities that ensure the verifcation of a developed software with respeit to doiumented (or not in some cases) requirements • Subset of Quality Assurance • Testing • Activities that ensure the identifiation of bugs/error/defeits in a software • Subset of Quality Control -
09.11.2017 11 Agenda (Reminder) • [x] Defnitions – What is
Testing – And some other defnitions • [ ] Goals of Testing • [ ] Objectives of Testing • [ ] Quality Traits of Software • [ ] Methods of Testing • [ ] Levels of Testing • [ ] Artifacts of Testing • [ ] Small Remark about DevOps -
09.11.2017 12 Goals of Testing • Find defects • Measure
and improve quality • Prevent defects • Ensure that the end result meets the requirements -
09.11.2017 13 Agenda (Reminder) • [x] Defnitions – What is
Testing – And some other defnitions • [x] Goals of Testing • [ ] Objeitives of Testing • [ ] Quality Traits of Software • [ ] Methods of Testing • [ ] Levels of Testing • [ ] Artifacts of Testing • [ ] Small Remark about DevOps -
09.11.2017 14 Objectives of Testing • Installation • Compatibility –
Is software compatible with other components (e.g. browsers) • Smoke – Run a set of test cases to check the stability of entire system – Wide and shallow • Sanity – After fxing a bug ensure 1 or 2 features are working – Narrow and deep – Usually performed after smoke test -
09.11.2017 15 Objectives of Testing 2 • Regression – Run
all of the tests to check how the proportion of green test changed. Are there side efeits of any new ihange or enhancements done in the application? • Retesting – Check that partiiular bug has been fxed • Ad Hoc Testing – QA Engineer tries to 'break' the system by randomly trying the system • Acceptance, Apha and Beta – See the Levels of Testing -
09.11.2017 16 Objectives of Testing 3 • Positive vs Negative
– Positive Testing - to show the software works. Aka "test to pass" – Negative Testing. Aka "test to fail" • Functional with non-Functional – Functional (refers to Levels) • Unit, Integration, System, Acceptance, Alpha, Beta – Non-Functional (group) • Performance, Load, Stress, Usability, Security, Portability -
09.11.2017 17 Objectives of Testing 4 • Stress – Testing
on overloads to fnd weak spots – System is expected to fail in a decent manner • Performance – Load Testing • Testing on diferent load, gradually increased from 0 to 100% – Endurance • Normal load on long time (days) – Soak Testing • High performance during a long time (days) • Ramp Testing – Raising input signal till the system breaks down -
09.11.2017 18 Objectives of Testing 5 • A/B • Usability
• Accessibility • Recovery • Security • Internationalization and localization • Conformance (aka Compliance) • ... -
09.11.2017 19 Agenda (Reminder) • [x] Defnitions – What is
Testing – And some other defnitions • [x] Goals of Testing • [x] Objectives of Testing • [ ] Quality Traits of Software • [ ] Methods of Testing • [ ] Levels of Testing • [ ] Artifacts of Testing • [ ] Small Remark about DevOps -
09.11.2017 20 Quality Traits of Software • Abstraction • Cohesion
• Completeness • Extencibility • Fault tolerance • Maintainability • Reliability • Robustness • Scalability • Testability • ... -
09.11.2017 21 Agenda (Reminder) • [x] Defnitions – What is
Testing – And some other defnitions • [x] Goals of Testing • [x] Objectives of Testing • [x] Quality Traits of Software • [ ] Methods of Testing • [ ] Levels of Testing • [ ] Artifacts of Testing • [ ] Small Remark about DevOps -
09.11.2017 22 Methods of Testing Overview • The Box Approach
– Black Box – White Box – Grey Box • Static vs Dynamic • 啤isual -
09.11.2017 23 Methods of Testing Black Box • Design Techiques
– Decision Table – All-pairs Testing – State Transition Reference – Equivalence Partitioning – Boundary 啤alue Analysis • Penetration Hacking -
09.11.2017 24 Methods of Testing White Box • Design Techiques
– Statement Coverage – Decision Coverage – Flow of Data – Flow of Control – Branch Testing – Path Testing – Mutation Testing • Penetration Hacking -
09.11.2017 25 Methods of Testing Grey Box • Design Techiques
– Matrix Testing – Regression Testing – Pattern Testing – Orthogonal Array Testing -
09.11.2017 26 Methods of Testing Static vs Dynamic • Static
– Code Review – Code Walkthrough – Code Inspection • Dynamic – Executing program with a set of test cases -
09.11.2017 27 Methods of Testing 啤isual • 啤isual Inspection with
a set of heuristics -
09.11.2017 28 Agenda (Reminder) • [x] Defnitions – What is
Testing – And some other defnitions • [x] Goals of Testing • [x] Objectives of Testing • [x] Quality Traits of Software • [x] Methods of Testing • [ ] Levels of Testing • [ ] Artifacts of Testing • [ ] Small Remark about DevOps -
09.11.2017 29 Levels of Testing • There are lots of
diferent views on this topic • Generally it depends on product • Let‘s go through the SWEBOK and ISTQB common case • And also look at 啤-Model -
09.11.2017 30 Levels of Testing SWEBOK (IEEE) • Unit •
Integration – top-down – bottom-up • System -
09.11.2017 31 Levels of Testing ISTQB • Unit • Integration
– Approach • top-down • bottom-up – Levels • Component • Component Integration • System Integration • System • Acceptance • Alpha • Beta -
09.11.2017 33 Agenda (Reminder) • [x] Defnitions – What is
Testing – And some other defnitions • [x] Goals of Testing • [x] Objectives of Testing • [x] Quality Traits of Software • [x] Methods of Testing • [x] Levels of Testing • [ ] Artifaits of Testing • [ ] Small Remark about DevOps -
09.11.2017 34 Artifacts of Testing • Test plan – scope,
approach, resources, and schedule of intended testing – identifes test items, the features to be tested, the testing tasks, who will do each task, and any risks to concider • Test case – requirement references – preconditions – events – steps (actions) to follow – input – output – expected result – actual result -
09.11.2017 35 Artifacts of Testing 2 • Traceability matrix •
Test script • Test suite – Collection of test cases • Test fxture – Test data used with test cases, but separated • Test harness – Software – Tools – Confgurations -
09.11.2017 36 Agenda (Reminder) • [x] Defnitions – What is
Testing – And some other defnitions • [x] Goals of Testing • [x] Objectives of Testing • [x] Quality Traits of Software • [x] Methods of Testing • [x] Levels of Testing • [x] Artifacts of Testing • [ ] Small Remark about DevOps