Upgrade to Pro — share decks privately, control downloads, hide ads and more …

An Introduction to Software Testing

An Introduction to Software Testing

Vakhtang Muskhelishvili, Software Tester, Exactpro

Exactpro Webinar for Tbilisi State University | A Webinar on Software Testing and AI
16.06.2020

To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/
Vkontakte https://vk.com/exactpro_llc

Subscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

Exactpro
PRO

June 16, 2020
Tweet

More Decks by Exactpro

Other Decks in Technology

Transcript

  1. 1
    An introduction to
    Software Testing
    Vakhtang Muskhelishvili, Exactpro
    Tbilisi State University, 2020-06-16

    View Slide

  2. 2 Build Software to Test Software
    exactpro.com
    First, we need to check if everything is ok .
    What is software testing?

    View Slide

  3. 3 Build Software to Test Software
    exactpro.com
    The process of running a software program with the purpose of detecting errors
    [The art of Software Testing, Glenford J. Myers, 1979]
    The process of observing a software program running in specific conditions and
    assessing some of its aspects based on the run
    [ANSI/IEEE standard 610.12-1990: Glossary of SE Terminology. NY:IEEE, 1987]
    A technical exploration of a software program aimed at getting information on
    its quality from the point of view of the stakeholders involved [С. Kaner, 1999]
    The verification of whether the actual program behaviour matches the expected
    behaviour, performed against a finite set of tests selected in a specific way
    [IEEE Guide to Software Engineering Body of Knowledge, SWEBOK, 2004]
    What is Software Testing?
    1980
    1987
    1999
    2004

    View Slide

  4. 4 Build Software to Test Software
    exactpro.com
    Software testing is an integral part of the software development life
    cycle because people can make mistakes/errors. Some errors are
    insignificant, others may have costly repercussions.
    Since everyone makes mistakes, we have to thoroughly verify the result
    of our work, as well as the work of others in everything that we make.
    Why is Software Testing Important?

    View Slide

  5. 5 Build Software to Test Software
    exactpro.com
    Software Testing is of Utmost Importance
    Therac-25 Knight Capital Group
    Ariane 5 Patriot Missile Interceptor
    The destroyed rocket and
    its cargo were valued at
    $500 million
    Radiation therapy
    machine caused injuries
    to 6 patients,
    3 people died
    Loss of $440 million, plus
    caused a major
    disruption of 148
    companies pricing
    28 U.S. Soldiers died and
    almost 100 were injured due
    to failure to intercept a
    hostile missile
    1996 1984 2012 1991

    View Slide

  6. 6 Build Software to Test Software
    exactpro.com
    What is a Software Bug?

    View Slide

  7. 7 Build Software to Test Software
    exactpro.com
    The Goals of Software Testing

    View Slide

  8. 8 Build Software to Test Software
    exactpro.com
    The Goals of Software Testing
    ● Providing information for decision-making
    ● Increasing the confidence in the level of quality
    ● Defect detection
    ● Defect prevention

    View Slide

  9. 9 Build Software to Test Software
    exactpro.com
    The Goals of Software Testing
    ● Providing information for decision-making
    ● Increasing the confidence in the level of quality
    ● Defect detection
    ● Defect prevention
    Testing helps decrease the general risk level in the system
    as a result of detecting defects and debugging, and instills
    confidence in the software quality.

    View Slide

  10. 10 Build Software to Test Software
    exactpro.com
    Types of software testing
    ● Functional Testing
    ● Non-functional Testing
    ● White-box Testing
    ● Change-related Testing

    View Slide

  11. 11 Build Software to Test Software
    exactpro.com
    Functional testing
    Testing performed to evaluate if a
    component or system satisfies functional
    requirements. This type of testing
    answers the question “what” the system
    should do.

    View Slide

  12. 12 Build Software to Test Software
    exactpro.com
    Non-functional testing
    Testing performed to evaluate that a component or system complies with non-functional
    requirements.
    5
    Usability Testing
    4
    M
    aintenance
    Testing
    3
    Stress
    Testing
    Load
    Testing
    2
    1
    Perform
    ance
    Testing
    6
    Reliability Testing
    7
    Portability Testing
    e t c

    View Slide

  13. 13 Build Software to Test Software
    exactpro.com
    White-box (Structure-based) Testing
    Testing based on the analysis of the internal structure of a component or system. The
    thoroughness of white-box testing can be measured through structural coverage. Structural
    coverage is the extent to which some type of structural element has been exercised by tests
    and is expressed as a percentage of the type of element being covered.
    if x>0:
    print(“Number “ + a + “ is greater than” + b)
    elif a==b:
    print(“Number “ + a + “ and “ + b + “ are equal”)
    else:
    print(“Number “ + b + “ is greater than ” + a)

    View Slide

  14. 14 Build Software to Test Software
    exactpro.com
    Confirmation and Regression Testing
    Confirmation testing (re-testing) —
    testing performed after fixing a defect to
    confirm that a failure caused by that
    defect does not reoccur.
    Regression testing — testing testing
    to detect whether defects have been
    introduced or uncovered in unchanged
    areas of the software.

    View Slide

  15. 15 Build Software to Test Software
    exactpro.com
    Test Documentation
    ● Test plan
    ● Test case
    ● Bug report
    ● Check-list
    ● Traceability matrix
    ● Test summary report

    View Slide

  16. 16 Build Software to Test Software
    exactpro.com
    Test Plan
    The documentation describing the test objectives to be achieved as well as the
    means and the schedule for achieving them, organized to coordinate testing
    activities.
    ● Determine the scope and risks and identify the testing objectives
    ● Determine the test approach (techniques, test items, coverage,
    identifying and interfacing with the teams involved in testing, testware)
    ● Implement the test policy and/or the test strategy
    ● Determine the required test resources (e.g. people, test environment,
    PCs)
    ● Schedule test analysis and design tasks, test implementation,
    execution and evaluation
    ● Determine the exit criteria

    View Slide

  17. 17 Build Software to Test Software
    exactpro.com
    Test Case
    A set of preconditions, inputs, actions (where applicable), expected
    results and postconditions developed based on test conditions.

    View Slide

  18. 18 Build Software to Test Software
    exactpro.com
    The Structure and Contents of a Test Scenario
    Test case ID
    Test case
    priority
    Test case title Preconditions
    Test case
    description
    Test data Postconditions Expected result
    The ID required for
    conveniently
    storing and
    navigating the test
    suites.
    ● critical
    ● high
    ● medium
    ● low
    The main
    objective/idea
    of the test case
    and a brief
    description of
    its essence.
    The description
    of conditions
    that are not
    directly related
    to the
    functionality
    under test, but
    must be fulfilled.
    The sequence of
    actions that
    should lead to
    the expected
    result.
    The data
    required to run
    a test case.
    The list of
    actions that
    return the
    system to its
    original state.
    What we expect
    to see as a
    result of
    executing the
    steps.

    View Slide

  19. 19 Build Software to Test Software
    exactpro.com
    Bug Report
    A Bug report is a technical document
    that contains a full description of the
    bug, including the information about
    the bug itself (a brief description,
    severity and priority levels, etc.) and
    the description of the conditions in
    which the bug occurred.
    A bug report should contain the correct
    commonly-accepted terminology
    describing the user interface elements
    and their events leading to the bug
    occurrence.

    View Slide

  20. 20 Build Software to Test Software
    exactpro.com
    The Structure and Contents of a Bug Report
    Bug ID A unique bug ID
    Author The bug report’s author
    Status Displays the bug status in its life cycle
    Priority This is the priority of fixing the issue.
    Severity The gradation of how much the bug influences the application
    Assignee The person assigned to fix the defect
    Bug title A brief description of the problem
    Test environment The Operating system and it Bit depth, the service pack, the browser and its version, etc
    Preconditions
    The description of conditions that are not directly related to the functionality under test, but
    must be fulfilled
    Steps to reproduce A detailed description of the path leading to the defect occurrence
    Postconditions The list of actions that return the system to its original state
    Expected result The result that must satisfy the requirements
    Actual result The result received after executing all the steps to reproduce
    Attachments Logs, screenshots and other documents that can help reproduce the problem or solve it

    View Slide

  21. 21 Build Software to Test Software
    exactpro.com
    Bug Tracking Systems
    ● Redmine;
    ● Atlassian JIRA;
    ● Bugzilla;
    ● Launchpad;
    ● etc.

    View Slide

  22. 22 Build Software to Test Software
    exactpro.com
    Financial infrastructures
    • Exchanges
    • Broker systems
    • Clearing agencies
    • Banks
    • CSD’s
    Specialized software and hardware systems
    • Ticker plants
    • Surveillance systems
    Risks associated with financial infrastructure
    outages:
    • Lost profit
    • Lost data
    • Damaged reputation

    View Slide

  23. 23 Build Software to Test Software
    exactpro.com
    Typical requirements for an Exchange system
    Daily capacity - 100+ mln transactions
    Peak rates - 40k+ transactions per second
    Average round-trip latency - <100 microseconds
    Availability - 100%
    3000 trx 2.5 cm <1 mm

    View Slide

  24. 24 Build Software to Test Software
    exactpro.com
    Exchange complexity

    View Slide

  25. 25 Build Software to Test Software
    exactpro.com
    75K messages / second from
    a single CPU core
    Measures latencies in
    microsecond range
    We have built unique software to test our clients’ software
    A variety of algoes simulating
    end-clients will run and see how the
    system performs
    Automated testing
    Compares expected and actual
    results
    Automated monitoring,
    analysis and reporting
    Automated end-to-end
    clearing system testing

    View Slide

  26. 26 Build Software to Test Software
    exactpro.com
    Thank you! Questions?

    View Slide