An introduction to software testing

1
An introduction to
Software Testing
Vakhtang Muskhelishvili, Exactpro
Software Testing Webinar, 2020-11-19

View Slide

2 Build Software to Test Software
exactpro.com
First, we need to check if everything is ok .
What is software testing?

View Slide

exactpro.com
The process of running a software program with the purpose of detecting errors
[The art of Software Testing, Glenford J. Myers, 1979]
The process of observing a software program running in specific conditions and
assessing some of its aspects based on the run
[ANSI/IEEE standard 610.12-1990: Glossary of SE Terminology. NY:IEEE, 1987]
A technical exploration of a software program aimed at getting information on
its quality from the point of view of the stakeholders involved [С. Kaner, 1999]
The verification of whether the actual program behaviour matches the expected
behaviour, performed against a finite set of tests selected in a specific way
[IEEE Guide to Software Engineering Body of Knowledge, SWEBOK, 2004]
What is Software Testing?
1980
1987
1999
2004

View Slide

exactpro.com
Software testing is an integral part of the software development life
cycle because people can make mistakes/errors. Some errors are
insignificant, others may have costly repercussions.
Since everyone makes mistakes, we have to thoroughly verify the result
of our work, as well as the work of others in everything that we make.
Why is Software Testing Important?

View Slide

exactpro.com
Software Testing is of Utmost Importance
Therac-25 Knight Capital Group
Ariane 5 Patriot Missile Interceptor
The destroyed rocket and
its cargo were valued at
$500 million
Radiation therapy
machine caused injuries
to 6 patients,
3 people died
Loss of $440 million, plus
caused a major
disruption of 148
companies pricing
28 U.S. Soldiers died and
almost 100 were injured due
to failure to intercept a
hostile missile
1996 1984 2012 1991

View Slide

exactpro.com
What is a Software Bug?

View Slide

exactpro.com
The Goals of Software Testing

View Slide

exactpro.com
● Providing information for decision-making
● Increasing the confidence in the level of quality
● Defect detection
● Defect prevention

View Slide

exactpro.com
● Providing information for decision-making
● Increasing the confidence in the level of quality
● Defect detection
● Defect prevention
Testing helps decrease the general risk level in the system
as a result of detecting defects and debugging, and instills
confidence in the software quality.

View Slide

exactpro.com
Types of software testing
● Functional Testing
● Non-functional Testing
● White-box Testing
● Change-related Testing

View Slide

exactpro.com
Functional testing
Testing performed to evaluate if a
component or system satisfies functional
requirements. This type of testing
answers the question “what” the system
should do.

View Slide

exactpro.com
Non-functional testing
Testing performed to evaluate that a component or system complies with non-functional
requirements.
5
Usability Testing
4
M
aintenance
Testing
3
Stress
Testing
Load
Testing
2
1
Perform
ance
Testing
6
Reliability Testing
7
Portability Testing
e t c

View Slide

exactpro.com
White-box (Structure-based) Testing
Testing based on the analysis of the internal structure of a component or system. The
thoroughness of white-box testing can be measured through structural coverage. Structural
coverage is the extent to which some type of structural element has been exercised by tests
and is expressed as a percentage of the type of element being covered.
if x>0:
print(“Number “ + a + “ is greater than” + b)
elif a==b:
print(“Number “ + a + “ and “ + b + “ are equal”)
else:
print(“Number “ + b + “ is greater than ” + a)

View Slide

exactpro.com
Confirmation and Regression Testing
Confirmation testing (re-testing) —
testing performed after fixing a defect to
confirm that a failure caused by that
defect does not reoccur.
Regression testing — testing testing
to detect whether defects have been
introduced or uncovered in unchanged
areas of the software.

View Slide

exactpro.com
Test Documentation
● Test plan
● Test case
● Bug report
● Check-list
● Traceability matrix
● Test summary report

View Slide

exactpro.com
Test Plan
The documentation describing the test objectives to be achieved as well as the
means and the schedule for achieving them, organized to coordinate testing
activities.
● Determine the scope and risks and identify the testing objectives
● Determine the test approach (techniques, test items, coverage,
identifying and interfacing with the teams involved in testing, testware)
● Implement the test policy and/or the test strategy
● Determine the required test resources (e.g. people, test environment,
PCs)
● Schedule test analysis and design tasks, test implementation,
execution and evaluation
● Determine the exit criteria

View Slide

exactpro.com
Test Case
A set of preconditions, inputs, actions (where applicable), expected
results and postconditions developed based on test conditions.

View Slide

exactpro.com
The Structure and Contents of a Test Scenario
Test case ID
Test case
priority
Test case title Preconditions
Test case
description
Test data Postconditions Expected result
The ID required for
conveniently
storing and
navigating the test
suites.
● critical
● high
● medium
● low
The main
objective/idea
of the test case
and a brief
description of
its essence.
The description
of conditions
that are not
directly related
to the
functionality
under test, but
must be fulfilled.
The sequence of
actions that
should lead to
the expected
result.
The data
required to run
a test case.
The list of
actions that
return the
system to its
original state.
What we expect
to see as a
result of
executing the
steps.

View Slide

exactpro.com
Bug Tracking Systems
● Redmine;
● Atlassian JIRA;
● Bugzilla;
● Launchpad;
● etc.

View Slide

exactpro.com
Bug Report
A Bug report is a technical document
that contains a full description of the
bug, including the information about
the bug itself (a brief description,
severity and priority levels, etc.) and
the description of the conditions in
which the bug occurred.
A bug report should contain the correct
commonly-accepted terminology
describing the user interface elements
and their events leading to the bug
occurrence.

View Slide

exactpro.com
The Structure and Contents of a Bug Report
Bug ID A unique bug ID
Author The bug report’s author
Status Displays the bug status in its life cycle
Priority This is the priority of fixing the issue.
Severity The gradation of how much the bug influences the application
Assignee The person assigned to fix the defect
Bug title A brief description of the problem
Test environment The Operating system and it Bit depth, the service pack, the browser and its version, etc
Preconditions
The description of conditions that are not directly related to the functionality under test, but
must be fulfilled
Steps to reproduce A detailed description of the path leading to the defect occurrence
Postconditions The list of actions that return the system to its original state
Expected result The result that must satisfy the requirements
Actual result The result received after executing all the steps to reproduce
Attachments Logs, screenshots and other documents that can help reproduce the problem or solve it

View Slide

exactpro.com
Financial infrastructures
• Exchanges
• Broker systems
• Clearing agencies
• Banks
• CSD’s
Specialized software and hardware systems
• Ticker plants
• Surveillance systems
Risks associated with financial infrastructure
outages:
• Lost profit
• Lost data
• Damaged reputation

View Slide

exactpro.com
Typical requirements for an Exchange system
Daily capacity - 100+ mln transactions
Peak rates - 40k+ transactions per second
Average round-trip latency - <100 microseconds
Availability - 100%
3000 trx 2.5 cm <1 mm

View Slide

exactpro.com
Exchange complexity

View Slide

exactpro.com
75K messages / second from
a single CPU core
Measures latencies in
microsecond range
We have built unique software to test our clients’ software
A variety of algoes simulating
end-clients will run and see how the
system performs
Automated testing
Compares expected and actual
results
Automated monitoring,
analysis and reporting
Automated end-to-end
clearing system testing

View Slide

exactpro.com
Thank you! Questions?

View Slide

An introduction to software testing

An introduction to software testing

Exactpro
PRO

More Decks by Exactpro

Other Decks in Education

Featured

Transcript

An introduction to software testing

An introduction to software testing

Exactpro PRO

More Decks by Exactpro

Other Decks in Education

Featured

Transcript

Exactpro
PRO