Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AI Testing Talks – Negative Testing

Exactpro
December 22, 2022
33

AI Testing Talks – Negative Testing

On 22 December, we held an online lecture on Negative Testing, which was led by Julia Emelianova, PhD, Researcher at Exactpro.

In this session, Julia gave an overview of the characteristics of negative testing and its scenarios, and discussed how negative testing can help identify the defects that can potentially result in major failures or outages, data corruption or security violations, while also improving the quality and stability of software applications. Julia also reviewed the benefits and disadvantages of negative testing.

Exactpro

December 22, 2022
Tweet

Transcript

  1. 1 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    BUILD SOFTWARE TO TEST SOFTWARE
    exactpro.com
    Negative Testing
    Julia Emelianova
    PhD, Researcher, Exactpro
    22 DECEMBER | 11:30 AM SLST

    View full-size slide

  2. 2 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Negative Testing is a type of software testing used to check software applications for unexpected input
    data and conditions, that might range from wrong data type to a strong hacking attack. The purpose of
    negative testing is to prevent software systems from crashing due to negative inputs and improve the
    quality and stability, to ensure the software is built in such a way that it will not crash or perform abnormally
    in negative scenarios. When performing negative testing, exceptions and appropriate errors are expected.
    This shows that the application is able to handle improper user behaviour. During negative testing, users
    input values that do not work in the system to test its ability to handle incorrect values or system failure.
    https://www.guru99.com/negative-testing.html
    https://en.wikipedia.org/wiki/Negative_testing
    https://www.mstsolutions.com/technical/why-testing-process-needs-negative-testing/
    https://www.javatpoint.com/negative-testing
    Synonyms for ”Negative Testing”
    ● Error Path Testing
    ● Failure Testing
    ● Dirty Testing
    ● Invalid Testing
    ● Destructive Testing
    Invalid
    Input
    Error
    Negative Test
    System
    1. What is Negative Testing?

    View full-size slide

  3. 3 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    ● is used to analyse the stability of an application or a software product against failed
    conditions, invalid input data or input values, outside the scope or limit;
    ● is executed to break the system and achieve failure in the operation of a software product
    by feeding harmful data;
    ● is performed to identify the defects which can result in major failures or outages, data
    corruption or security violations;
    ● allows to evaluate the potential security breaches and special handling procedures;
    ● is implemented to find the critical loopholes or bugs and weak areas resulting in failures;
    ● usually, is performed by a test engineer.
    2. Characteristics of Negative Testing

    View full-size slide

  4. 4 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    To improve the testing coverage of the application.
    To improve the quality of the application by detecting defects.
    To make the application more stable and reliable.
    To prevent the application from crashing.
    3. Purposes of Negative Testing

    View full-size slide

  5. 5 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    https://www.javatpoint.com/negative-testing
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    4. Negative Testing Scenarios

    View full-size slide

  6. 6 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Some software programmes include input fields that only receive data that
    falls under a particular range of numbers or text.
    Negative Testing Scenario 1 – Allowed data bounds and limits.
    Data Bounds Test – Test all the lower and upper bounds for data fields, and
    also limits of possible data values.

    View full-size slide

  7. 7 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Test Case Examples:
    1. Enter “Birth Year” equal to 1899.
    2. Enter “Birth Year” equal to 2017.
    3. Enter “Birth Year” equal to -2000.
    4. Enter “Your Name” equal to “John123”.
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Only numbers, no letters or special characters,
    values from 1900 to 2016)
    Birth Year*
    Comments Save
    Some software programmes include input fields that only receive data that
    falls under a particular range of numbers or text.
    Negative Testing Scenario 1 – Allowed data bounds and limits.
    Data Bounds Test – Test all the lower and upper bounds for data fields, and
    also limits of possible data values.

    View full-size slide

  8. 8 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Examples:
    1. Enter “Birth Year” equal to 1899.
    2. Enter “Birth Year” equal to 2017.
    3. Enter “Birth Year” equal to -2000.
    4. Enter “Your Name” equal to “John123”.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    An error occurred.
    Please enter valid data.
    Retry
    Your Name*
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Only numbers, no letters or special characters,
    values from 1900 to 2016)
    Birth Year*
    Comments Save
    Some software programmes include input fields that only receive data that
    falls under a particular range of numbers or text.
    Negative Testing Scenario 1 – Allowed data bounds and limits.
    Data Bounds Test – Test all the lower and upper bounds for data fields, and
    also limits of possible data values.

    View full-size slide

  9. 9 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    There are web pages and applications with fields that only permit a limited
    number of characters to be entered by the user.
    Negative Testing Scenario 2 – Allowed number of characters.
    Field Size Test – Test the number of characters one can enter into a field
    which exceeds the limit specified in the functional specifications.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing

    View full-size slide

  10. 10 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Enter “Your Name” equal to “Maximilianus” (12 characters long).
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Only numbers, no letters or special characters,
    values from 1900 to 2016)
    Birth Year*
    Comments Save
    There are web pages and applications with fields that only permit a limited
    number of characters to be entered by the user.
    Negative Testing Scenario 2 – Allowed number of characters.
    Field Size Test – Test the number of characters one can enter into a field
    which exceeds the limit specified in the functional specifications.

    View full-size slide

  11. 11 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Enter “Your Name” equal to “Maximilianus” (12 characters long).
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    An error occurred.
    The number of characters is
    too large.
    Retry
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Only numbers, no letters or special characters,
    values from 1900 to 2016)
    Birth Year*
    Comments Save
    There are web pages and applications with fields that only permit a limited
    number of characters to be entered by the user.
    Negative Testing Scenario 2 – Allowed number of characters.
    Field Size Test – Test the number of characters one can enter into a field
    which exceeds the limit specified in the functional specifications.

    View full-size slide

  12. 12 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    There are certain software and web pages which showcase fields that are
    required to be filled up by the user (marked with “*”).
    Negative Testing Scenario 3 – Populating required fields.
    Necessary Data Entry Test – Skip the required data entry and try to proceed.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing

    View full-size slide

  13. 13 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Examples:
    1. Leave the field “Birth Year” blank. Enter valid “Your Name”
    and some text in “Comments”.
    2. Leave the fields “Your Name” and “Comments” blank. Enter valid
    “Birth Year”.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Your Name*
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Only numbers, no letters or special characters,
    values from 1900 to 2016)
    Birth Year*
    Comments Save
    3. Leave the fields “Your
    Name” and “Birth Year”
    blank. Enter some text
    in “Comments”.
    There are certain software and web pages which showcase fields that are
    required to be filled up by the user (marked with “*”).
    Negative Testing Scenario 3 – Populating required fields.
    Necessary Data Entry Test – Skip the required data entry and try to proceed.

    View full-size slide

  14. 14 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Examples:
    1. Leave the field “Birth Year” blank. Enter valid “Your Name”
    and some text in “Comments”.
    2. Leave the fields “Your Name” and “Comments” blank. Enter valid
    “Birth Year”.
    3. Leave the fields “Your
    Name” and “Birth Year”
    blank. Enter some text
    in “Comments”.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    An error occurred.
    Not all the required fields
    are populated.
    Retry
    Your Name*
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Only numbers, no letters or special characters,
    values from 1900 to 2016)
    Birth Year*
    Comments Save
    There are certain software and web pages which showcase fields that are
    required to be filled up by the user (marked with “*”).
    Negative Testing Scenario 3 – Populating required fields.
    Necessary Data Entry Test – Skip the required data entry and try to proceed.

    View full-size slide

  15. 15 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Most digital forms and dialog boxes are capable of receiving data in a certain
    form, such as text, number, date, and time.
    Negative Testing Scenario 4 – Correspondence between data and field types.
    Data Format/Type Test – Verify each field type, date and time format and its
    validity.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing

    View full-size slide

  16. 16 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Examples:
    1. Enter “1987/02/17” in the
    field “Birth Date”.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Use the date format YYYY-MM-DD)
    Birth Date*
    Comments Save
    Most digital forms and dialog boxes are capable of receiving data in a certain
    form, such as text, number, date, and time.
    Negative Testing Scenario 4 – Correspondence between data and field types.
    Data Format/Type Test – Verify each field type, date and time format and its
    validity.

    View full-size slide

  17. 17 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Examples:
    1. Enter “1987/02/17” in the
    field “Birth Date”.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    An error occurred.
    The date format is incorrect.
    Retry
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Use the date format YYYY-MM-DD)
    Birth Date*
    Comments Save
    Most digital forms and dialog boxes are capable of receiving data in a certain
    form, such as text, number, date, and time.
    Negative Testing Scenario 4 – Correspondence between data and field types.
    Data Format/Type Test – Verify each field type, date and time format and its
    validity.

    View full-size slide

  18. 18 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Examples:
    1. Enter “1987/02/17” in the
    field “Birth Date”.
    2. Enter “1987-02-45” in the field
    “Birth Date”.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    An error occurred.
    The date format is incorrect.
    Retry
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Use the date format YYYY-MM-DD)
    Birth Date*
    Comments Save
    Most digital forms and dialog boxes are capable of receiving data in a certain
    form, such as text, number, date, and time.
    Negative Testing Scenario 4 – Correspondence between data and field types.
    Data Format/Type Test – Verify each field type, date and time format and its
    validity.

    View full-size slide

  19. 19 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Examples:
    1. Enter “1987/02/17” in the
    field “Birth Date”.
    2. Enter “1987-02-45” in the field
    “Birth Date”.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    An error occurred.
    The date format is incorrect.
    Retry
    An error occurred.
    The specified day is invalid.
    Retry
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Use the date format YYYY-MM-DD)
    Birth Date*
    Comments Save
    Most digital forms and dialog boxes are capable of receiving data in a certain
    form, such as text, number, date, and time.
    Negative Testing Scenario 4 – Correspondence between data and field types.
    Data Format/Type Test – Verify each field type, date and time format and its
    validity.

    View full-size slide

  20. 20 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Some fields are required to receive alphanumeric data entry containing a
    single quote.
    Negative Testing Scenario 5 – Single Quote Testing.
    Implanted Single Quote Test – Try to store information containing a single
    quote: embed a single quote on URL when it tries to query the database or
    enter text involving one or more single quotes in the text fields.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing

    View full-size slide

  21. 21 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Enter “It’s my name in the field ‘Your Name’ above” in the field “Comments”
    and click the “Save” button.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Use the date format YYYY-MM-DD)
    Birth Date*
    Comments Save
    Some fields are required to receive alphanumeric data entry containing a
    single quote.
    Negative Testing Scenario 5 – Single Quote Testing.
    Implanted Single Quote Test – Try to store information containing a single
    quote: embed a single quote on URL when it tries to query the database or
    enter text involving one or more single quotes in the text fields.

    View full-size slide

  22. 22 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Enter “It’s my name in the field ‘Your Name’ above” in the field “Comments”
    and click the “Save” button.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Use the date format YYYY-MM-DD)
    Birth Date*
    Comments Save
    The following comment is saved:
    Its my name in the field Your Name above
    Ok
    !
    Some fields are required to receive alphanumeric data entry containing a
    single quote.
    Negative Testing Scenario 5 – Single Quote Testing.
    Implanted Single Quote Test – Try to store information containing a single
    quote: embed a single quote on URL when it tries to query the database or
    enter text involving one or more single quotes in the text fields.

    View full-size slide

  23. 23 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Enter “It’s my name in the field ‘Your Name’ above” in the field “Comments”
    and click the “Save” button.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Use the date format YYYY-MM-DD)
    Birth Date*
    Comments Save
    The following comment is saved:
    Its my name in the field Your Name above
    Ok
    !
    = =
    =
    Some fields are required to receive alphanumeric data entry containing a
    single quote.
    Negative Testing Scenario 5 – Single Quote Testing.
    Implanted Single Quote Test – Try to store information containing a single
    quote: embed a single quote on URL when it tries to query the database or
    enter text involving one or more single quotes in the text fields.

    View full-size slide

  24. 24 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Negative Testing Scenario 6 – Previous and current release performance.
    Performance Modification Test – The test suite should contain test cases that
    compare previous and current release performance statistics, which could help
    in categorising potential performance problems.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing

    View full-size slide

  25. 25 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Compare round-trip latency distribution for the previous and
    current releases.
    Previous release
    Current release
    Latency is the time required for a data packet to travel from the sending endpoint to the receiving endpoint.
    The two maximums correspond to the two types of messages sent.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Negative Testing Scenario 6 – Previous and current release performance.
    Performance Modification Test – The test suite should contain test cases that
    compare previous and current release performance statistics, which could help
    in categorising potential performance problems.

    View full-size slide

  26. 26 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Compare round-trip latency distribution for the previous and
    current releases.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Previous release
    Current release
    Latency is the time required for a data packet to travel from the sending endpoint to the receiving endpoint.
    The two maximums correspond to the two types of messages sent.
    The following problem is observed:
    Round-trip latency gets globally worse.
    Ok
    !
    Negative Testing Scenario 6 – Previous and current release performance.
    Performance Modification Test – The test suite should contain test cases that
    compare previous and current release performance statistics, which could help
    in categorising potential performance problems.

    View full-size slide

  27. 27 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Certain web browsers require the user to log in first before it loads web pages.
    Negative Testing Scenario 7 – Web Session Testing.
    Web Session Test – Attempt to open web pages in the application without the
    user logging in beforehand.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing

    View full-size slide

  28. 28 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Try to download boards for the user “johnsmith123” in the Trello application.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Certain web browsers require the user to log in first before it loads web pages.
    Negative Testing Scenario 7 – Web Session Testing.
    Web Session Test – Attempt to open web pages in the application without the
    user logging in beforehand.

    View full-size slide

  29. 29 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    !
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Certain web browsers require the user to log in first before it loads web pages.
    Negative Testing Scenario 7 – Web Session Testing.
    Web Session Test – Attempt to open web pages in the application without the
    user logging in beforehand.
    Test Case Example:
    Try to download boards for the user “johnsmith123” in the Trello application.

    View full-size slide

  30. 30 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    Testers need:
    ● to try every input which is invalid;
    ● check the behaviour of the application against it.
    If an application crashes or shuts down without any warning or message, it needs:
    ● to be fixed;
    ● to give a proper message or warning.
    So that the user would know what not to do while using the application.
    Consider some of the smart techniques that you can utilise to perform negative
    testing using an AI chatbot as an example.

    View full-size slide

  31. 31 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.1. Boundary Value Analysis
    This method involves writing the Test Cases for values that are outside the boundary limits.

    View full-size slide

  32. 32 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.1. Boundary Value Analysis
    This method involves writing the Test Cases for values that are outside the boundary limits.
    Test Case Example:
    Boundary values on the character length of a question for the chatbot.

    View full-size slide

  33. 33 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.1. Boundary Value Analysis
    This method involves writing the Test Cases for values that are outside the boundary limits.
    Test Case Example:
    Boundary values on the character length of a question for the chatbot.
    Short question:
    Julia: Do you love cats?
    ChatBot: Yes.

    View full-size slide

  34. 34 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.1. Boundary Value Analysis
    This method involves writing the Test Cases for values that are outside the boundary limits.
    Test Case Example:
    Boundary values on the character length of a question for the chatbot.
    Short question:
    Long question:
    Julia: The cat is similar in anatomy to the other felid species: it
    has a strong flexible body, quick reflexes, sharp teeth, and
    retractable claws adapted to killing small prey. Its night vision and
    sense of smell are well developed. Cat communication includes
    vocalisations like meowing, purring, trilling, hissing, growling, and
    grunting as well as cat-specific body language. Do you love cats?
    ChatBot: Sorry, friend. Something went wrong and now I can not
    answer you.
    Julia: Do you love cats?
    ChatBot: Yes.

    View full-size slide

  35. 35 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.1. Boundary Value Analysis
    This method involves writing the Test Cases for values that are outside the boundary limits.
    Test Case Example:
    Boundary values on the character length of a question for the chatbot.
    Long question:
    Julia: The cat is similar in anatomy to the other field species: it
    has a strong flexible body, quick reflexes, sharp teeth, and
    retractable claws adapted to killing small prey. Its night vision and
    sense of smell are well developed. Cat communication includes
    vocalizations like meowing, purring, trilling, hissing, growling, and
    grunting as well as cat-specific body language. Do you love cats?
    ChatBot: Sorry, friend. Something went wrong and now I can not
    answer you.
    Short question:
    Julia: Do you love cats?
    ChatBot: Yes.
    Negative testing

    View full-size slide

  36. 36 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.2. Equivalence Partitioning
    This is a software testing technique which divides the input data into many partitions.
    Values from each partition must be tested at least once.
    Partitions with valid values are used for positive testing.
    Partitions with invalid values are used for negative testing.

    View full-size slide

  37. 37 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.2. Equivalence Partitioning
    This is a software testing technique which divides the input data into many partitions.
    Values from each partition must be tested at least once.
    Partitions with valid values are used for positive testing.
    Partitions with invalid values are used for negative testing.
    Test Case Example:
    Two types of languages used for dialog with the chatbot = Two equivalence partitions
    ● Supported languages (English, German, French)
    ● Unsupported languages (Korean, Chinese, Japan, etc.)

    View full-size slide

  38. 38 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.2. Equivalence Partitioning
    This is a software testing technique which divides the input data into many partitions.
    Values from each partition must be tested at least once.
    Partitions with valid values are used for positive testing.
    Partitions with invalid values are used for negative testing.
    Test Case Example:
    Two types of languages used for dialog with the chatbot = Two equivalence partitions
    ● Supported languages (English, German, French)
    ● Unsupported languages (Korean, Chinese, Japan, etc.)
    Julia: 잘 지내고 있나요?
    ChatBot: I do not even know what to answer...
    잘 지내고 있나요?
    Means “How are you?”
    in Korean.

    View full-size slide

  39. 39 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.3. Error Guessing
    This procedure pinpoints the specific conditions which trigger failure or error messages.
    If possible, try to identify and correct the issue without a system crash.
    In some cases, an incorrect result may lead to an extreme “downstream” failure if the error condition
    is not immediately noticed and addressed.
    https://botsurfer.com/learn/8-ways-to-break-chatbot

    View full-size slide

  40. 40 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.3. Error Guessing
    This procedure pinpoints the specific conditions which trigger failure or error messages.
    If possible, try to identify and correct the issue without a system crash.
    In some cases, an incorrect result may lead to an extreme “downstream” failure if the error condition
    is not immediately noticed and addressed.
    Test Case Examples:
    There are several known issues which may break a chatbot.
    So if you need to test the chatbot, you may guess that your chatbot can also be broken in similar ways.
    https://botsurfer.com/learn/8-ways-to-break-chatbot

    View full-size slide

  41. 41 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.3. Error Guessing
    This procedure pinpoints the specific conditions which trigger failure or error messages.
    If possible, try to identify and correct the issue without a system crash.
    In some cases, an incorrect result may lead to an extreme “downstream” failure if the error condition
    is not immediately noticed and addressed.
    Test Case Examples:
    There are several known issues which may break a chatbot.
    So if you need to test the chatbot, you may guess that your chatbot can also be broken in similar ways.
    Julia: Reset
    ChatBot: I do not know what
    you want. Do you have any other
    questions?
    1. Tell the chatbot to reset or start over
    when you came to the end of your line
    of questioning and you wanted to look
    at other information.
    https://botsurfer.com/learn/8-ways-to-break-chatbot

    View full-size slide

  42. 42 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.3. Error Guessing
    This procedure pinpoints the specific conditions which trigger failure or error messages.
    If possible, try to identify and correct the issue without a system crash.
    In some cases, an incorrect result may lead to an extreme “downstream” failure if the error condition
    is not immediately noticed and addressed.
    Test Case Examples:
    There are several known issues which may break a chatbot.
    So if you need to test the chatbot, you may guess that your chatbot can also be broken in similar ways.
    Julia: What can I do?
    ChatBot: You have to do what
    you are good at. What can you
    do?
    2. Ask for Help or Assistance
    Many chatbots don’t know what to do
    when you ask them for assistance or
    help.
    https://botsurfer.com/learn/8-ways-to-break-chatbot

    View full-size slide

  43. 43 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.4. Checklist
    It is a basic method that documents error conditions
    that you are planning to test.
    A software testing checklist is commonly used in
    tandem with error guessing.
    https://www.qable.io/the-chatbot-testing-checklist-top-tools-techniques/

    View full-size slide

  44. 44 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.4. Checklist
    It is a basic method that documents error conditions
    that you are planning to test.
    A software testing checklist is commonly used in
    tandem with error guessing.
    Example:
    https://www.qable.io/the-chatbot-testing-checklist-top-tools-techniques/
    ❏ Slow or bad navigation
    ❏ Poor User-Interface
    ❏ Lack of word knowledge
    ❏ Lack of intelligence
    ❏ Take too much time to
    respond
    ❏ Same or wrong answers
    ❏ Poor connection timing

    View full-size slide

  45. 45 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.5. Exploratory Testing
    This technique is used to increase your knowledge about the application while the test is ongoing.
    It may be conducted simultaneously with other tests. Exploratory testing can show you a clear picture of
    which aspects of the application work and which ones do not.

    View full-size slide

  46. 46 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.5. Exploratory Testing
    Test Case Examples:
    1. If a chatbot shares links to other
    resources such as websites, email ids,
    images, etc., make sure all these
    entities are functional.
    This technique is used to increase your knowledge about the application while the test is ongoing.
    It may be conducted simultaneously with other tests. Exploratory testing can show you a clear picture of
    which aspects of the application work and which ones do not.

    View full-size slide

  47. 47 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.5. Exploratory Testing
    Test Case Examples:
    Julia: 000111000011001010000111
    ChatBot: Unfortunately, I do not
    know what to answer...
    2. Enter the incorrect question as a sequence
    of numbers and check the result.
    1. If a chatbot shares links to other
    resources such as websites, email ids,
    images, etc., make sure all these
    entities are functional.
    This technique is used to increase your knowledge about the application while the test is ongoing.
    It may be conducted simultaneously with other tests. Exploratory testing can show you a clear picture of
    which aspects of the application work and which ones do not.

    View full-size slide

  48. 48 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.6. Small Scale Test Automation
    It is a method that uncovers:
    ● memory errors,
    ● code defects,
    ● other issues
    that are occasionally revealed in production use.

    View full-size slide

  49. 49 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.6. Small Scale Test Automation
    It is a method that uncovers:
    ● memory errors,
    ● code defects,
    ● other issues
    that are occasionally revealed in production use.
    It entails performing the same action thousands of times to see its effect on the program.

    View full-size slide

  50. 50 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.7. Fuzz Testing
    This procedure involves the input of:
    ● invalid data,
    ● random data,
    ● semi random data
    which may cause unexpected failures, crashes, and issues.
    In this case, there are no preset expected results, unlike other negative test cases.
    It is simply an observation of what may potentially occur because of arbitrary inputs.
    Fuzzing looks only for crashes or some other undesirable behaviour, and millions of test cases might be
    executed before a crash.
    Moreover, many bugs such as logic errors do not crash the software system but instead produce incorrect
    output – a failure type that is much more difficult to detect.

    View full-size slide

  51. 51 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.7. Fuzz Testing
    Test Case Example:
    Julia: fjdhdsjkfsdkj
    ChatBot: thinking…
    The chatbot is not active for a long time
    Step 1

    View full-size slide

  52. 52 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.7. Fuzz Testing
    Test Case Example:
    Julia: fjdhdsjkfsdkj
    ChatBot: Sorry, friend. Something
    went wrong and now I can not
    answer you.
    Step 1

    View full-size slide

  53. 53 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.7. Fuzz Testing
    Test Case Example:
    Julia: fjdhdsjkfsdkj
    ChatBot: Sorry, friend. Something
    went wrong and now I can not
    answer you.
    Step 1
    Julia: 4357845hjfd ??? sd iufd @@^%?
    ChatBot: thinking…
    Step 2
    The chatbot is not active for a long time

    View full-size slide

  54. 54 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.7. Fuzz Testing
    Test Case Example:
    Julia: fjdhdsjkfsdkj
    ChatBot: Sorry, friend. Something
    went wrong and now I can not
    answer you.
    Step 1
    Julia: 4357845hjfd ??? sd iufd @@^%?
    ChatBot: Sorry, friend. Something went
    wrong and now I can not answer you.
    Step 2

    View full-size slide

  55. 55 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.7. Fuzz Testing
    Test Case Example:
    Julia: fjdhdsjkfsdkj
    ChatBot: Sorry, friend. Something
    went wrong and now I can not
    answer you.
    Step 1
    Julia: 4357845hjfd ??? sd iufd @@^%?
    ChatBot: Sorry, friend. Something went
    wrong and now I can not answer you.
    Step 2
    Julia: What oh yes ring do help horse?
    ChatBot: thinking…
    Step N

    The chatbot is not active for a long time

    View full-size slide

  56. 56 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Sorry, the ChatBot just crashed!
    Ok
    !
    5. Negative Testing Techniques
    5.7. Fuzz Testing
    Test Case Example:
    Julia: fjdhdsjkfsdkj
    ChatBot: Sorry, friend. Something
    went wrong and now I can not
    answer you.
    Step 1
    Julia: 4357845hjfd ??? sd iufd @@^%?
    ChatBot: Sorry, friend. Something went
    wrong and now I can not answer you.
    Step 2
    Step N

    View full-size slide

  57. 57 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.8. State Transition Testing
    It originates from the concept that software can only exist in one particular state at one point in time.
    For instance, a program is considered to be in its normal state until an issue reveals itself.
    Test Case Examples:
    State transition diagram for logging into the chatbot application.
    https://www.guru99.com/state-transition-testing.html
    START 3rd Try
    ACCESS
    GRANTED
    2nd Try
    1st Try
    Correct
    Password
    Correct
    Password
    Correct
    Password
    ACCOUNT
    BLOCKED
    Incorrect
    Password
    Incorrect
    Password
    Incorrect
    Password

    View full-size slide

  58. 58 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.8. State Transition Testing
    It originates from the concept that software can only exist in one particular state at one point in time.
    For instance, a program is considered to be in its normal state until an issue reveals itself.
    Test Case Examples:
    State transition diagram for logging into the chatbot application.
    If you enter the incorrect password
    three times, your account will be
    blocked with a warning message.
    https://www.guru99.com/state-transition-testing.html
    Sorry, your account is blocked!
    Ok
    !
    START 3rd Try
    ACCESS
    GRANTED
    2nd Try
    1st Try
    Correct
    Password
    Correct
    Password
    Correct
    Password
    ACCOUNT
    BLOCKED
    Incorrect
    Password
    Incorrect
    Password
    Incorrect
    Password

    View full-size slide

  59. 59 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    ● Increasing software application protection from cybercrime
    and unethical hacks.
    ● Increasing guarantee of a product's quality, stability and
    pleasant to use.
    https://www.javatpoint.com/negative-testing
    https://www.educba.com/negative-testing/
    6. Benefits of Negative Testing

    View full-size slide

  60. 60 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    ● Increasing software application protection from cybercrime
    and unethical hacks.
    ● Increasing guarantee of a product's quality, stability and
    pleasant to use.
    https://www.javatpoint.com/negative-testing
    https://www.educba.com/negative-testing/
    ● Enhance the possibility of covering all bases and each
    type of error, which can arise because of inconsistent human behaviour.
    ● The user will know the limitations beforehand. Knowledge of the application
    limitations allows us to use it accordingly.
    6. Benefits of Negative Testing

    View full-size slide

  61. 61 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    ● Increasing software application protection from cybercrime
    and unethical hacks.
    ● Increasing guarantee of a product's quality, stability and
    pleasant to use.
    https://www.javatpoint.com/negative-testing
    https://www.educba.com/negative-testing/
    ● Enhance the possibility of covering all bases and each
    type of error, which can arise because of inconsistent human behaviour.
    ● The user will know the limitations beforehand. Knowledge of the application
    limitations allows us to use it accordingly.
    ● Give more confidence to the client before going live.
    6. Benefits of Negative Testing

    View full-size slide

  62. 62 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    ● The execution of
    negative testing needs
    extra time, money,
    and effort.
    https://www.javatpoint.com/negative-testing
    https://www.educba.com/negative-testing/
    7. Disadvantages of Negative Testing

    View full-size slide

  63. 63 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    ● The execution of
    negative testing needs
    extra time, money,
    and effort.
    https://www.javatpoint.com/negative-testing
    https://www.educba.com/negative-testing/
    7. Disadvantages of Negative Testing
    ● For the customer, it
    causes extreme delays in a
    software product
    or an application release.

    View full-size slide

  64. 64 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    ● The execution of
    negative testing needs
    extra time, money,
    and effort.
    https://www.javatpoint.com/negative-testing
    https://www.educba.com/negative-testing/
    7. Disadvantages of Negative Testing
    ● For the customer, it
    causes extreme delays in a
    software product
    or an application release.
    ● The implementation of negative testing
    needs a trained and experienced test
    engineer to develop negative test cases.
    Highly experienced people are required
    for negative testing.

    View full-size slide

  65. 65 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Which test cases are related to negative testing?
    ❏ Upload invalid files like XML, SQL, etc. files in image upload field.
    ❏ Try to enter a valid email id in the email field, a valid password in the password
    field and try to login.
    ❏ Verify whether payment is done for an invalid member, provider or broker ID by
    creating a respective record in the feed.
    ❏ Test system with expired card details.
    ❏ Test with a valid PIN for the credit card.
    Question

    View full-size slide

  66. 66 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Please scan
    the QR code and choose
    the right answer(s):
    Join at slido.com
    #7237 487

    View full-size slide

  67. 67 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Which test cases are related to negative testing?
    ❏ Upload invalid files like XML, SQL, etc. files in image upload field.
    ❏ Try to enter a valid email id in the email field, a valid password in the password
    field and try to login.
    ❏ Verify whether payment is done for an invalid member, provider or broker ID by
    creating a respective record in the feed.
    ❏ Test system with expired card details.
    ❏ Test with a valid PIN for the credit card.
    Question

    View full-size slide

  68. 68 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    AI Testing Talks
    Thank You!
    Questions?

    View full-size slide