Dipping the toes into the sea of testing

Dipping the toes into the sea of testing

An overview of testing world, basically to draw a map and define a language to talk about testing.

3997614cf9ef044e5f569318fbfc64d5?s=128

Michael Pershyn

November 09, 2017
Tweet

Transcript

  1. 09.11.2017 1 Dipping the toes into the sea of testing

    by Michael Pershyn
  2. 09.11.2017 2 Motivation • My experience with Testing • Testing,

    SOLID and Unit Testing • Testing is a big thing – Area complexity – Efort & Cost • Let‘s walk through and make a map
  3. 09.11.2017 3 Agenda • Defnitions – What is Testing –

    And some other defnitions • Goals of Testing • Objectives of Testing • Quality Traits of Software • Methods of Testing • Levels of Testing • Artifacts of Testing • Small Remark about DevOps
  4. 09.11.2017 4 Defnitions SStandards„ • SWEBOK – Software Engineering Body

    of Knowledge – aka ISO/IEC TR 19759 2005 – From IEEE • ISTQB – International Software Testing Qualifcations Board – Certify Test Engineers
  5. 09.11.2017 5 Defnitions Software Testing A set of processes aimed

    at investigating, evaluating and determining the completeness, quality and purpose ftting of software
  6. 09.11.2017 6 Defnitions 啤alidation and 啤erifcation Validation and Verifiation is

    the process of checking that a software system meets specifcations and that it fulflls its intended purpose 啤alidation Are we building the right product? 啤erifcation Are we building the product right?
  7. 09.11.2017 7 Defnitions Defect and Failure • Defeit vs bug

    vs fault – When actual result deviates from the expected (specifed) result – Also known as Bug or Fault • Failure – Application Crash
  8. 09.11.2017 8 Defnitions Component, Use Case and Test • Test

    - an experiment to fnd answers to the question "Does the software in general or specifc function works as expected" • Component is an identifable part of a larger program or construction – Unit is the smallest compilable component – A unit is a component – The integration of one or more components is a component • Use Case - the specifcation of tests that are conducted from the end-user perspective
  9. 09.11.2017 9 Defnitions QA, QC and Testing • Quality Assurance

    • Implementation of proiesses, proiedures and standards in context to verifcation of developed software and intended requirements • Subset of SDLC (Software Development Live Cycle) • Quality Control • Activities that ensure the verifcation of a developed software with respeit to doiumented (or not in some cases) requirements • Subset of Quality Assurance • Testing • Activities that ensure the identifiation of bugs/error/defeits in a software • Subset of Quality Control
  10. 09.11.2017 10 Defnitions Test Engineer vs Tester

  11. 09.11.2017 11 Agenda (Reminder) • [x] Defnitions – What is

    Testing – And some other defnitions • [ ] Goals of Testing • [ ] Objectives of Testing • [ ] Quality Traits of Software • [ ] Methods of Testing • [ ] Levels of Testing • [ ] Artifacts of Testing • [ ] Small Remark about DevOps
  12. 09.11.2017 12 Goals of Testing • Find defects • Measure

    and improve quality • Prevent defects • Ensure that the end result meets the requirements
  13. 09.11.2017 13 Agenda (Reminder) • [x] Defnitions – What is

    Testing – And some other defnitions • [x] Goals of Testing • [ ] Objeitives of Testing • [ ] Quality Traits of Software • [ ] Methods of Testing • [ ] Levels of Testing • [ ] Artifacts of Testing • [ ] Small Remark about DevOps
  14. 09.11.2017 14 Objectives of Testing • Installation • Compatibility –

    Is software compatible with other components (e.g. browsers) • Smoke – Run a set of test cases to check the stability of entire system – Wide and shallow • Sanity – After fxing a bug ensure 1 or 2 features are working – Narrow and deep – Usually performed after smoke test
  15. 09.11.2017 15 Objectives of Testing 2 • Regression – Run

    all of the tests to check how the proportion of green test changed. Are there side efeits of any new ihange or enhancements done in the application? • Retesting – Check that partiiular bug has been fxed • Ad Hoc Testing – QA Engineer tries to 'break' the system by randomly trying the system • Acceptance, Apha and Beta – See the Levels of Testing
  16. 09.11.2017 16 Objectives of Testing 3 • Positive vs Negative

    – Positive Testing - to show the software works. Aka "test to pass" – Negative Testing. Aka "test to fail" • Functional with non-Functional – Functional (refers to Levels) • Unit, Integration, System, Acceptance, Alpha, Beta – Non-Functional (group) • Performance, Load, Stress, Usability, Security, Portability
  17. 09.11.2017 17 Objectives of Testing 4 • Stress – Testing

    on overloads to fnd weak spots – System is expected to fail in a decent manner • Performance – Load Testing • Testing on diferent load, gradually increased from 0 to 100% – Endurance • Normal load on long time (days) – Soak Testing • High performance during a long time (days) • Ramp Testing – Raising input signal till the system breaks down
  18. 09.11.2017 18 Objectives of Testing 5 • A/B • Usability

    • Accessibility • Recovery • Security • Internationalization and localization • Conformance (aka Compliance) • ...
  19. 09.11.2017 19 Agenda (Reminder) • [x] Defnitions – What is

    Testing – And some other defnitions • [x] Goals of Testing • [x] Objectives of Testing • [ ] Quality Traits of Software • [ ] Methods of Testing • [ ] Levels of Testing • [ ] Artifacts of Testing • [ ] Small Remark about DevOps
  20. 09.11.2017 20 Quality Traits of Software • Abstraction • Cohesion

    • Completeness • Extencibility • Fault tolerance • Maintainability • Reliability • Robustness • Scalability • Testability • ...
  21. 09.11.2017 21 Agenda (Reminder) • [x] Defnitions – What is

    Testing – And some other defnitions • [x] Goals of Testing • [x] Objectives of Testing • [x] Quality Traits of Software • [ ] Methods of Testing • [ ] Levels of Testing • [ ] Artifacts of Testing • [ ] Small Remark about DevOps
  22. 09.11.2017 22 Methods of Testing Overview • The Box Approach

    – Black Box – White Box – Grey Box • Static vs Dynamic • 啤isual
  23. 09.11.2017 23 Methods of Testing Black Box • Design Techiques

    – Decision Table – All-pairs Testing – State Transition Reference – Equivalence Partitioning – Boundary 啤alue Analysis • Penetration Hacking
  24. 09.11.2017 24 Methods of Testing White Box • Design Techiques

    – Statement Coverage – Decision Coverage – Flow of Data – Flow of Control – Branch Testing – Path Testing – Mutation Testing • Penetration Hacking
  25. 09.11.2017 25 Methods of Testing Grey Box • Design Techiques

    – Matrix Testing – Regression Testing – Pattern Testing – Orthogonal Array Testing
  26. 09.11.2017 26 Methods of Testing Static vs Dynamic • Static

    – Code Review – Code Walkthrough – Code Inspection • Dynamic – Executing program with a set of test cases
  27. 09.11.2017 27 Methods of Testing 啤isual • 啤isual Inspection with

    a set of heuristics
  28. 09.11.2017 28 Agenda (Reminder) • [x] Defnitions – What is

    Testing – And some other defnitions • [x] Goals of Testing • [x] Objectives of Testing • [x] Quality Traits of Software • [x] Methods of Testing • [ ] Levels of Testing • [ ] Artifacts of Testing • [ ] Small Remark about DevOps
  29. 09.11.2017 29 Levels of Testing • There are lots of

    diferent views on this topic • Generally it depends on product • Let‘s go through the SWEBOK and ISTQB common case • And also look at 啤-Model
  30. 09.11.2017 30 Levels of Testing SWEBOK (IEEE) • Unit •

    Integration – top-down – bottom-up • System
  31. 09.11.2017 31 Levels of Testing ISTQB • Unit • Integration

    – Approach • top-down • bottom-up – Levels • Component • Component Integration • System Integration • System • Acceptance • Alpha • Beta
  32. 09.11.2017 32 Levels of Testing 啤-Model

  33. 09.11.2017 33 Agenda (Reminder) • [x] Defnitions – What is

    Testing – And some other defnitions • [x] Goals of Testing • [x] Objectives of Testing • [x] Quality Traits of Software • [x] Methods of Testing • [x] Levels of Testing • [ ] Artifaits of Testing • [ ] Small Remark about DevOps
  34. 09.11.2017 34 Artifacts of Testing • Test plan – scope,

    approach, resources, and schedule of intended testing – identifes test items, the features to be tested, the testing tasks, who will do each task, and any risks to concider • Test case – requirement references – preconditions – events – steps (actions) to follow – input – output – expected result – actual result
  35. 09.11.2017 35 Artifacts of Testing 2 • Traceability matrix •

    Test script • Test suite – Collection of test cases • Test fxture – Test data used with test cases, but separated • Test harness – Software – Tools – Confgurations
  36. 09.11.2017 36 Agenda (Reminder) • [x] Defnitions – What is

    Testing – And some other defnitions • [x] Goals of Testing • [x] Objectives of Testing • [x] Quality Traits of Software • [x] Methods of Testing • [x] Levels of Testing • [x] Artifacts of Testing • [ ] Small Remark about DevOps
  37. 09.11.2017 37 Small Remark about DevOps

  38. 09.11.2017 38 Small Remark about DevOps 2

  39. 09.11.2017 39 Small Remark about DevOps 3

  40. 09.11.2017 40 Questions and Thanks