An Introduction to Software Testing

Vakhtang Muskhelishvili, Software Tester, Exactpro

Exactpro Webinar for Tbilisi State University | A Webinar on Software Testing and AI
16.06.2020

To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/
Vkontakte https://vk.com/exactpro_llc

Subscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

Transcript

1. 1 An introduction to Software Testing Vakhtang Muskhelishvili, Exactpro Tbilisi State University, 2020-06-16
2. 2 Build Software to Test Software exactpro.com First, we need to check if everything is ok . What is software testing?
3. 3 Build Software to Test Software exactpro.com The process of running a software program with the purpose of detecting errors [The art of Software Testing, Glenford J. Myers, 1979] The process of observing a software program running in specific conditions and assessing some of its aspects based on the run [ANSI/IEEE standard 610.12-1990: Glossary of SE Terminology. NY:IEEE, 1987] A technical exploration of a software program aimed at getting information on its quality from the point of view of the stakeholders involved [С. Kaner, 1999] The verification of whether the actual program behaviour matches the expected behaviour, performed against a finite set of tests selected in a specific way [IEEE Guide to Software Engineering Body of Knowledge, SWEBOK, 2004] What is Software Testing? 1980 1987 1999 2004
4. 4 Build Software to Test Software exactpro.com Software testing is an integral part of the software development life cycle because people can make mistakes/errors. Some errors are insignificant, others may have costly repercussions. Since everyone makes mistakes, we have to thoroughly verify the result of our work, as well as the work of others in everything that we make. Why is Software Testing Important?
5. 5 Build Software to Test Software exactpro.com Software Testing is of Utmost Importance Therac-25 Knight Capital Group Ariane 5 Patriot Missile Interceptor The destroyed rocket and its cargo were valued at $500 million Radiation therapy machine caused injuries to 6 patients, 3 people died Loss of $440 million, plus caused a major disruption of 148 companies pricing 28 U.S. Soldiers died and almost 100 were injured due to failure to intercept a hostile missile 1996 1984 2012 1991
6. 6 Build Software to Test Software exactpro.com What is a Software Bug?
7. 7 Build Software to Test Software exactpro.com The Goals of Software Testing
8. 8 Build Software to Test Software exactpro.com The Goals of Software Testing • Providing information for decision-making • Increasing the confidence in the level of quality • Defect detection • Defect prevention
9. 9 Build Software to Test Software exactpro.com The Goals of Software Testing • Providing information for decision-making • Increasing the confidence in the level of quality • Defect detection • Defect prevention Testing helps decrease the general risk level in the system as a result of detecting defects and debugging, and instills confidence in the software quality.
10. 10 Build Software to Test Software exactpro.com Types of software testing • Functional Testing • Non-functional Testing • White-box Testing • Change-related Testing
11. 11 Build Software to Test Software exactpro.com Functional testing Testing performed to evaluate if a component or system satisfies functional requirements. This type of testing answers the question “what” the system should do.
12. 12 Build Software to Test Software exactpro.com Non-functional testing Testing performed to evaluate that a component or system complies with non-functional requirements. 5 Usability Testing 4 M aintenance Testing 3 Stress Testing Load Testing 2 1 Perform ance Testing 6 Reliability Testing 7 Portability Testing e t c
13. 13 Build Software to Test Software exactpro.com White-box (Structure-based) Testing Testing based on the analysis of the internal structure of a component or system. The thoroughness of white-box testing can be measured through structural coverage. Structural coverage is the extent to which some type of structural element has been exercised by tests and is expressed as a percentage of the type of element being covered. if x>0: print(“Number “ + a + “ is greater than” + b) elif a==b: print(“Number “ + a + “ and “ + b + “ are equal”) else: print(“Number “ + b + “ is greater than ” + a)
14. 14 Build Software to Test Software exactpro.com Confirmation and Regression Testing Confirmation testing (re-testing) — testing performed after fixing a defect to confirm that a failure caused by that defect does not reoccur. Regression testing — testing testing to detect whether defects have been introduced or uncovered in unchanged areas of the software.
15. 15 Build Software to Test Software exactpro.com Test Documentation • Test plan • Test case • Bug report • Check-list • Traceability matrix • Test summary report
16. 16 Build Software to Test Software exactpro.com Test Plan The documentation describing the test objectives to be achieved as well as the means and the schedule for achieving them, organized to coordinate testing activities. • Determine the scope and risks and identify the testing objectives • Determine the test approach (techniques, test items, coverage, identifying and interfacing with the teams involved in testing, testware) • Implement the test policy and/or the test strategy • Determine the required test resources (e.g. people, test environment, PCs) • Schedule test analysis and design tasks, test implementation, execution and evaluation • Determine the exit criteria
17. 17 Build Software to Test Software exactpro.com Test Case A set of preconditions, inputs, actions (where applicable), expected results and postconditions developed based on test conditions.
18. 18 Build Software to Test Software exactpro.com The Structure and Contents of a Test Scenario Test case ID Test case priority Test case title Preconditions Test case description Test data Postconditions Expected result The ID required for conveniently storing and navigating the test suites. • critical • high • medium • low The main objective/idea of the test case and a brief description of its essence. The description of conditions that are not directly related to the functionality under test, but must be fulfilled. The sequence of actions that should lead to the expected result. The data required to run a test case. The list of actions that return the system to its original state. What we expect to see as a result of executing the steps.
19. 19 Build Software to Test Software exactpro.com Bug Report A Bug report is a technical document that contains a full description of the bug, including the information about the bug itself (a brief description, severity and priority levels, etc.) and the description of the conditions in which the bug occurred. A bug report should contain the correct commonly-accepted terminology describing the user interface elements and their events leading to the bug occurrence.
20. 20 Build Software to Test Software exactpro.com The Structure and Contents of a Bug Report Bug ID A unique bug ID Author The bug report’s author Status Displays the bug status in its life cycle Priority This is the priority of fixing the issue. Severity The gradation of how much the bug influences the application Assignee The person assigned to fix the defect Bug title A brief description of the problem Test environment The Operating system and it Bit depth, the service pack, the browser and its version, etc Preconditions The description of conditions that are not directly related to the functionality under test, but must be fulfilled Steps to reproduce A detailed description of the path leading to the defect occurrence Postconditions The list of actions that return the system to its original state Expected result The result that must satisfy the requirements Actual result The result received after executing all the steps to reproduce Attachments Logs, screenshots and other documents that can help reproduce the problem or solve it
21. 21 Build Software to Test Software exactpro.com Bug Tracking Systems • Redmine; • Atlassian JIRA; • Bugzilla; • Launchpad; • etc.
22. 22 Build Software to Test Software exactpro.com Financial infrastructures • Exchanges • Broker systems • Clearing agencies • Banks • CSD’s Specialized software and hardware systems • Ticker plants • Surveillance systems Risks associated with financial infrastructure outages: • Lost profit • Lost data • Damaged reputation
23. 23 Build Software to Test Software exactpro.com Typical requirements for an Exchange system Daily capacity - 100+ mln transactions Peak rates - 40k+ transactions per second Average round-trip latency - <100 microseconds Availability - 100% 3000 trx 2.5 cm <1 mm
24. 24 Build Software to Test Software exactpro.com Exchange complexity
25. 25 Build Software to Test Software exactpro.com 75K messages / second from a single CPU core Measures latencies in microsecond range We have built unique software to test our clients’ software A variety of algoes simulating end-clients will run and see how the system performs Automated testing Compares expected and actual results Automated monitoring, analysis and reporting Automated end-to-end clearing system testing
26. 26 Build Software to Test Software exactpro.com Thank you! Questions?