AI Testing Talks – Negative Testing

On 22 December, we held an online lecture on Negative Testing, which was led by Julia Emelianova, PhD, Researcher at Exactpro.

In this session, Julia gave an overview of the characteristics of negative testing and its scenarios, and discussed how negative testing can help identify the defects that can potentially result in major failures or outages, data corruption or security violations, while also improving the quality and stability of software applications. Julia also reviewed the benefits and disadvantages of negative testing.

Transcript

1. 1 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   BUILD SOFTWARE TO TEST SOFTWARE
   exactpro.com
   Negative Testing
   Julia Emelianova
   PhD, Researcher, Exactpro
   22 DECEMBER | 11:30 AM SLST
   

   View Slide

2. 2 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   Negative Testing is a type of software testing used to check software applications for unexpected input
   data and conditions, that might range from wrong data type to a strong hacking attack. The purpose of
   negative testing is to prevent software systems from crashing due to negative inputs and improve the
   quality and stability, to ensure the software is built in such a way that it will not crash or perform abnormally
   in negative scenarios. When performing negative testing, exceptions and appropriate errors are expected.
   This shows that the application is able to handle improper user behaviour. During negative testing, users
   input values that do not work in the system to test its ability to handle incorrect values or system failure.
   https://www.guru99.com/negative-testing.html
   https://en.wikipedia.org/wiki/Negative_testing
   https://www.mstsolutions.com/technical/why-testing-process-needs-negative-testing/
   https://www.javatpoint.com/negative-testing
   Synonyms for ”Negative Testing”
   ● Error Path Testing
   ● Failure Testing
   ● Dirty Testing
   ● Invalid Testing
   ● Destructive Testing
   Invalid
   Input
   Error
   Negative Test
   System
   1. What is Negative Testing?
   

   View Slide

3. 3 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   ● is used to analyse the stability of an application or a software product against failed
   conditions, invalid input data or input values, outside the scope or limit;
   ● is executed to break the system and achieve failure in the operation of a software product
   by feeding harmful data;
   ● is performed to identify the defects which can result in major failures or outages, data
   corruption or security violations;
   ● allows to evaluate the potential security breaches and special handling procedures;
   ● is implemented to find the critical loopholes or bugs and weak areas resulting in failures;
   ● usually, is performed by a test engineer.
   2. Characteristics of Negative Testing
   

   View Slide

4. 4 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   To improve the testing coverage of the application.
   To improve the quality of the application by detecting defects.
   To make the application more stable and reliable.
   To prevent the application from crashing.
   3. Purposes of Negative Testing
   

   View Slide

5. 5 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   https://www.javatpoint.com/negative-testing
   What are
   Negative
   Test Cases?
   Data Bounds
   Test
   Field Size Test
   Necessary
   Data Entry
   Data Format/
   Type Test
   Implanted
   Single
   Quote
   Performance
   Modification
   Web Session
   Testing
   4. Negative Testing Scenarios
   

   View Slide

6. 6 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   4. Negative Testing Scenarios
   What are
   Negative
   Test Cases?
   Data Bounds
   Test
   Field Size
   Test
   Necessary
   Data Entry
   Data Format/
   Type Test
   Implanted
   Single
   Quote
   Performance
   Modification
   Web Session
   Testing
   Some software programmes include input fields that only receive data that
   falls under a particular range of numbers or text.
   Negative Testing Scenario 1 – Allowed data bounds and limits.
   Data Bounds Test – Test all the lower and upper bounds for data fields, and
   also limits of possible data values.
   

   View Slide

7. 7 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   4. Negative Testing Scenarios
   What are
   Negative
   Test Cases?
   Data Bounds
   Test
   Field Size
   Test
   Necessary
   Data Entry
   Data Format/
   Type Test
   Implanted
   Single
   Quote
   Performance
   Modification
   Web Session
   Testing
   Test Case Examples:
   1. Enter “Birth Year” equal to 1899.
   2. Enter “Birth Year” equal to 2017.
   3. Enter “Birth Year” equal to -2000.
   4. Enter “Your Name” equal to “John123”.
   Your Name*
   (Only a-z and A-Z characters, no numbers
   or special characters, maximum 10 characters long)
   (Only numbers, no letters or special characters,
   values from 1900 to 2016)
   Birth Year*
   Comments Save
   Some software programmes include input fields that only receive data that
   falls under a particular range of numbers or text.
   Negative Testing Scenario 1 – Allowed data bounds and limits.
   Data Bounds Test – Test all the lower and upper bounds for data fields, and
   also limits of possible data values.
   

   View Slide

8. 8 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   Test Case Examples:
   1. Enter “Birth Year” equal to 1899.
   2. Enter “Birth Year” equal to 2017.
   3. Enter “Birth Year” equal to -2000.
   4. Enter “Your Name” equal to “John123”.
   4. Negative Testing Scenarios
   What are
   Negative
   Test Cases?
   Data Bounds
   Test
   Field Size
   Test
   Necessary
   Data Entry
   Data Format/
   Type Test
   Implanted
   Single
   Quote
   Performance
   Modification
   Web Session
   Testing
   An error occurred.
   Please enter valid data.
   Retry
   Your Name*
   Your Name*
   (Only a-z and A-Z characters, no numbers
   or special characters, maximum 10 characters long)
   (Only numbers, no letters or special characters,
   values from 1900 to 2016)
   Birth Year*
   Comments Save
   Some software programmes include input fields that only receive data that
   falls under a particular range of numbers or text.
   Negative Testing Scenario 1 – Allowed data bounds and limits.
   Data Bounds Test – Test all the lower and upper bounds for data fields, and
   also limits of possible data values.
   

   View Slide

9. 9 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   There are web pages and applications with fields that only permit a limited
   number of characters to be entered by the user.
   Negative Testing Scenario 2 – Allowed number of characters.
   Field Size Test – Test the number of characters one can enter into a field
   which exceeds the limit specified in the functional specifications.
   4. Negative Testing Scenarios
   What are
   Negative
   Test Cases?
   Data Bounds
   Test
   Field Size
   Test
   Necessary
   Data Entry
   Data Format/
   Type Test
   Implanted
   Single
   Quote
   Performance
   Modification
   Web Session
   Testing
   

   View Slide

10. 10 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Enter “Your Name” equal to “Maximilianus” (12 characters long).
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Only numbers, no letters or special characters,
    values from 1900 to 2016)
    Birth Year*
    Comments Save
    There are web pages and applications with fields that only permit a limited
    number of characters to be entered by the user.
    Negative Testing Scenario 2 – Allowed number of characters.
    Field Size Test – Test the number of characters one can enter into a field
    which exceeds the limit specified in the functional specifications.
    

    View Slide

11. 11 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Enter “Your Name” equal to “Maximilianus” (12 characters long).
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    An error occurred.
    The number of characters is
    too large.
    Retry
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Only numbers, no letters or special characters,
    values from 1900 to 2016)
    Birth Year*
    Comments Save
    There are web pages and applications with fields that only permit a limited
    number of characters to be entered by the user.
    Negative Testing Scenario 2 – Allowed number of characters.
    Field Size Test – Test the number of characters one can enter into a field
    which exceeds the limit specified in the functional specifications.
    

    View Slide

12. 12 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    There are certain software and web pages which showcase fields that are
    required to be filled up by the user (marked with “*”).
    Negative Testing Scenario 3 – Populating required fields.
    Necessary Data Entry Test – Skip the required data entry and try to proceed.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    

    View Slide

13. 13 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Examples:
    1. Leave the field “Birth Year” blank. Enter valid “Your Name”
    and some text in “Comments”.
    2. Leave the fields “Your Name” and “Comments” blank. Enter valid
    “Birth Year”.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Your Name*
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Only numbers, no letters or special characters,
    values from 1900 to 2016)
    Birth Year*
    Comments Save
    3. Leave the fields “Your
    Name” and “Birth Year”
    blank. Enter some text
    in “Comments”.
    There are certain software and web pages which showcase fields that are
    required to be filled up by the user (marked with “*”).
    Negative Testing Scenario 3 – Populating required fields.
    Necessary Data Entry Test – Skip the required data entry and try to proceed.
    

    View Slide

14. 14 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Examples:
    1. Leave the field “Birth Year” blank. Enter valid “Your Name”
    and some text in “Comments”.
    2. Leave the fields “Your Name” and “Comments” blank. Enter valid
    “Birth Year”.
    3. Leave the fields “Your
    Name” and “Birth Year”
    blank. Enter some text
    in “Comments”.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    An error occurred.
    Not all the required fields
    are populated.
    Retry
    Your Name*
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Only numbers, no letters or special characters,
    values from 1900 to 2016)
    Birth Year*
    Comments Save
    There are certain software and web pages which showcase fields that are
    required to be filled up by the user (marked with “*”).
    Negative Testing Scenario 3 – Populating required fields.
    Necessary Data Entry Test – Skip the required data entry and try to proceed.
    

    View Slide

15. 15 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Most digital forms and dialog boxes are capable of receiving data in a certain
    form, such as text, number, date, and time.
    Negative Testing Scenario 4 – Correspondence between data and field types.
    Data Format/Type Test – Verify each field type, date and time format and its
    validity.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    

    View Slide

16. 16 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Examples:
    1. Enter “1987/02/17” in the
    field “Birth Date”.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Use the date format YYYY-MM-DD)
    Birth Date*
    Comments Save
    Most digital forms and dialog boxes are capable of receiving data in a certain
    form, such as text, number, date, and time.
    Negative Testing Scenario 4 – Correspondence between data and field types.
    Data Format/Type Test – Verify each field type, date and time format and its
    validity.
    

    View Slide

17. 17 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Examples:
    1. Enter “1987/02/17” in the
    field “Birth Date”.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    An error occurred.
    The date format is incorrect.
    Retry
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Use the date format YYYY-MM-DD)
    Birth Date*
    Comments Save
    Most digital forms and dialog boxes are capable of receiving data in a certain
    form, such as text, number, date, and time.
    Negative Testing Scenario 4 – Correspondence between data and field types.
    Data Format/Type Test – Verify each field type, date and time format and its
    validity.
    

    View Slide

18. 18 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Examples:
    1. Enter “1987/02/17” in the
    field “Birth Date”.
    2. Enter “1987-02-45” in the field
    “Birth Date”.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    An error occurred.
    The date format is incorrect.
    Retry
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Use the date format YYYY-MM-DD)
    Birth Date*
    Comments Save
    Most digital forms and dialog boxes are capable of receiving data in a certain
    form, such as text, number, date, and time.
    Negative Testing Scenario 4 – Correspondence between data and field types.
    Data Format/Type Test – Verify each field type, date and time format and its
    validity.
    

    View Slide

19. 19 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Examples:
    1. Enter “1987/02/17” in the
    field “Birth Date”.
    2. Enter “1987-02-45” in the field
    “Birth Date”.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    An error occurred.
    The date format is incorrect.
    Retry
    An error occurred.
    The specified day is invalid.
    Retry
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Use the date format YYYY-MM-DD)
    Birth Date*
    Comments Save
    Most digital forms and dialog boxes are capable of receiving data in a certain
    form, such as text, number, date, and time.
    Negative Testing Scenario 4 – Correspondence between data and field types.
    Data Format/Type Test – Verify each field type, date and time format and its
    validity.
    

    View Slide

20. 20 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Some fields are required to receive alphanumeric data entry containing a
    single quote.
    Negative Testing Scenario 5 – Single Quote Testing.
    Implanted Single Quote Test – Try to store information containing a single
    quote: embed a single quote on URL when it tries to query the database or
    enter text involving one or more single quotes in the text fields.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    

    View Slide

21. 21 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Enter “It’s my name in the field ‘Your Name’ above” in the field “Comments”
    and click the “Save” button.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Use the date format YYYY-MM-DD)
    Birth Date*
    Comments Save
    Some fields are required to receive alphanumeric data entry containing a
    single quote.
    Negative Testing Scenario 5 – Single Quote Testing.
    Implanted Single Quote Test – Try to store information containing a single
    quote: embed a single quote on URL when it tries to query the database or
    enter text involving one or more single quotes in the text fields.
    

    View Slide

22. 22 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Enter “It’s my name in the field ‘Your Name’ above” in the field “Comments”
    and click the “Save” button.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Use the date format YYYY-MM-DD)
    Birth Date*
    Comments Save
    The following comment is saved:
    Its my name in the field Your Name above
    Ok
    !
    Some fields are required to receive alphanumeric data entry containing a
    single quote.
    Negative Testing Scenario 5 – Single Quote Testing.
    Implanted Single Quote Test – Try to store information containing a single
    quote: embed a single quote on URL when it tries to query the database or
    enter text involving one or more single quotes in the text fields.
    

    View Slide

23. 23 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Enter “It’s my name in the field ‘Your Name’ above” in the field “Comments”
    and click the “Save” button.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Your Name*
    (Only a-z and A-Z characters, no numbers
    or special characters, maximum 10 characters long)
    (Use the date format YYYY-MM-DD)
    Birth Date*
    Comments Save
    The following comment is saved:
    Its my name in the field Your Name above
    Ok
    !
    = =
    =
    Some fields are required to receive alphanumeric data entry containing a
    single quote.
    Negative Testing Scenario 5 – Single Quote Testing.
    Implanted Single Quote Test – Try to store information containing a single
    quote: embed a single quote on URL when it tries to query the database or
    enter text involving one or more single quotes in the text fields.
    

    View Slide

24. 24 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Negative Testing Scenario 6 – Previous and current release performance.
    Performance Modification Test – The test suite should contain test cases that
    compare previous and current release performance statistics, which could help
    in categorising potential performance problems.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    

    View Slide

25. 25 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Compare round-trip latency distribution for the previous and
    current releases.
    Previous release
    Current release
    Latency is the time required for a data packet to travel from the sending endpoint to the receiving endpoint.
    The two maximums correspond to the two types of messages sent.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Negative Testing Scenario 6 – Previous and current release performance.
    Performance Modification Test – The test suite should contain test cases that
    compare previous and current release performance statistics, which could help
    in categorising potential performance problems.
    

    View Slide

26. 26 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Compare round-trip latency distribution for the previous and
    current releases.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Previous release
    Current release
    Latency is the time required for a data packet to travel from the sending endpoint to the receiving endpoint.
    The two maximums correspond to the two types of messages sent.
    The following problem is observed:
    Round-trip latency gets globally worse.
    Ok
    !
    Negative Testing Scenario 6 – Previous and current release performance.
    Performance Modification Test – The test suite should contain test cases that
    compare previous and current release performance statistics, which could help
    in categorising potential performance problems.
    

    View Slide

27. 27 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Certain web browsers require the user to log in first before it loads web pages.
    Negative Testing Scenario 7 – Web Session Testing.
    Web Session Test – Attempt to open web pages in the application without the
    user logging in beforehand.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    

    View Slide

28. 28 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Test Case Example:
    Try to download boards for the user “johnsmith123” in the Trello application.
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Certain web browsers require the user to log in first before it loads web pages.
    Negative Testing Scenario 7 – Web Session Testing.
    Web Session Test – Attempt to open web pages in the application without the
    user logging in beforehand.
    

    View Slide

29. 29 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    !
    4. Negative Testing Scenarios
    What are
    Negative
    Test Cases?
    Data Bounds
    Test
    Field Size
    Test
    Necessary
    Data Entry
    Data Format/
    Type Test
    Implanted
    Single
    Quote
    Performance
    Modification
    Web Session
    Testing
    Certain web browsers require the user to log in first before it loads web pages.
    Negative Testing Scenario 7 – Web Session Testing.
    Web Session Test – Attempt to open web pages in the application without the
    user logging in beforehand.
    Test Case Example:
    Try to download boards for the user “johnsmith123” in the Trello application.
    

    View Slide

30. 30 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    Testers need:
    ● to try every input which is invalid;
    ● check the behaviour of the application against it.
    If an application crashes or shuts down without any warning or message, it needs:
    ● to be fixed;
    ● to give a proper message or warning.
    So that the user would know what not to do while using the application.
    Consider some of the smart techniques that you can utilise to perform negative
    testing using an AI chatbot as an example.
    

    View Slide

31. 31 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.1. Boundary Value Analysis
    This method involves writing the Test Cases for values that are outside the boundary limits.
    

    View Slide

32. 32 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.1. Boundary Value Analysis
    This method involves writing the Test Cases for values that are outside the boundary limits.
    Test Case Example:
    Boundary values on the character length of a question for the chatbot.
    

    View Slide

33. 33 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.1. Boundary Value Analysis
    This method involves writing the Test Cases for values that are outside the boundary limits.
    Test Case Example:
    Boundary values on the character length of a question for the chatbot.
    Short question:
    Julia: Do you love cats?
    ChatBot: Yes.
    

    View Slide

34. 34 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.1. Boundary Value Analysis
    This method involves writing the Test Cases for values that are outside the boundary limits.
    Test Case Example:
    Boundary values on the character length of a question for the chatbot.
    Short question:
    Long question:
    Julia: The cat is similar in anatomy to the other felid species: it
    has a strong flexible body, quick reflexes, sharp teeth, and
    retractable claws adapted to killing small prey. Its night vision and
    sense of smell are well developed. Cat communication includes
    vocalisations like meowing, purring, trilling, hissing, growling, and
    grunting as well as cat-specific body language. Do you love cats?
    ChatBot: Sorry, friend. Something went wrong and now I can not
    answer you.
    Julia: Do you love cats?
    ChatBot: Yes.
    

    View Slide

35. 35 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.1. Boundary Value Analysis
    This method involves writing the Test Cases for values that are outside the boundary limits.
    Test Case Example:
    Boundary values on the character length of a question for the chatbot.
    Long question:
    Julia: The cat is similar in anatomy to the other field species: it
    has a strong flexible body, quick reflexes, sharp teeth, and
    retractable claws adapted to killing small prey. Its night vision and
    sense of smell are well developed. Cat communication includes
    vocalizations like meowing, purring, trilling, hissing, growling, and
    grunting as well as cat-specific body language. Do you love cats?
    ChatBot: Sorry, friend. Something went wrong and now I can not
    answer you.
    Short question:
    Julia: Do you love cats?
    ChatBot: Yes.
    Negative testing
    

    View Slide

36. 36 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.2. Equivalence Partitioning
    This is a software testing technique which divides the input data into many partitions.
    Values from each partition must be tested at least once.
    Partitions with valid values are used for positive testing.
    Partitions with invalid values are used for negative testing.
    

    View Slide

37. 37 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.2. Equivalence Partitioning
    This is a software testing technique which divides the input data into many partitions.
    Values from each partition must be tested at least once.
    Partitions with valid values are used for positive testing.
    Partitions with invalid values are used for negative testing.
    Test Case Example:
    Two types of languages used for dialog with the chatbot = Two equivalence partitions
    ● Supported languages (English, German, French)
    ● Unsupported languages (Korean, Chinese, Japan, etc.)
    

    View Slide

38. 38 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.2. Equivalence Partitioning
    This is a software testing technique which divides the input data into many partitions.
    Values from each partition must be tested at least once.
    Partitions with valid values are used for positive testing.
    Partitions with invalid values are used for negative testing.
    Test Case Example:
    Two types of languages used for dialog with the chatbot = Two equivalence partitions
    ● Supported languages (English, German, French)
    ● Unsupported languages (Korean, Chinese, Japan, etc.)
    Julia: 잘 지내고 있나요?
    ChatBot: I do not even know what to answer...
    잘 지내고 있나요?
    Means “How are you?”
    in Korean.
    

    View Slide

39. 39 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.3. Error Guessing
    This procedure pinpoints the specific conditions which trigger failure or error messages.
    If possible, try to identify and correct the issue without a system crash.
    In some cases, an incorrect result may lead to an extreme “downstream” failure if the error condition
    is not immediately noticed and addressed.
    https://botsurfer.com/learn/8-ways-to-break-chatbot
    

    View Slide

40. 40 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.3. Error Guessing
    This procedure pinpoints the specific conditions which trigger failure or error messages.
    If possible, try to identify and correct the issue without a system crash.
    In some cases, an incorrect result may lead to an extreme “downstream” failure if the error condition
    is not immediately noticed and addressed.
    Test Case Examples:
    There are several known issues which may break a chatbot.
    So if you need to test the chatbot, you may guess that your chatbot can also be broken in similar ways.
    https://botsurfer.com/learn/8-ways-to-break-chatbot
    

    View Slide

41. 41 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.3. Error Guessing
    This procedure pinpoints the specific conditions which trigger failure or error messages.
    If possible, try to identify and correct the issue without a system crash.
    In some cases, an incorrect result may lead to an extreme “downstream” failure if the error condition
    is not immediately noticed and addressed.
    Test Case Examples:
    There are several known issues which may break a chatbot.
    So if you need to test the chatbot, you may guess that your chatbot can also be broken in similar ways.
    Julia: Reset
    ChatBot: I do not know what
    you want. Do you have any other
    questions?
    1. Tell the chatbot to reset or start over
    when you came to the end of your line
    of questioning and you wanted to look
    at other information.
    https://botsurfer.com/learn/8-ways-to-break-chatbot
    

    View Slide

42. 42 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.3. Error Guessing
    This procedure pinpoints the specific conditions which trigger failure or error messages.
    If possible, try to identify and correct the issue without a system crash.
    In some cases, an incorrect result may lead to an extreme “downstream” failure if the error condition
    is not immediately noticed and addressed.
    Test Case Examples:
    There are several known issues which may break a chatbot.
    So if you need to test the chatbot, you may guess that your chatbot can also be broken in similar ways.
    Julia: What can I do?
    ChatBot: You have to do what
    you are good at. What can you
    do?
    2. Ask for Help or Assistance
    Many chatbots don’t know what to do
    when you ask them for assistance or
    help.
    https://botsurfer.com/learn/8-ways-to-break-chatbot
    

    View Slide

43. 43 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.4. Checklist
    It is a basic method that documents error conditions
    that you are planning to test.
    A software testing checklist is commonly used in
    tandem with error guessing.
    https://www.qable.io/the-chatbot-testing-checklist-top-tools-techniques/
    

    View Slide

44. 44 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.4. Checklist
    It is a basic method that documents error conditions
    that you are planning to test.
    A software testing checklist is commonly used in
    tandem with error guessing.
    Example:
    https://www.qable.io/the-chatbot-testing-checklist-top-tools-techniques/
    ❏ Slow or bad navigation
    ❏ Poor User-Interface
    ❏ Lack of word knowledge
    ❏ Lack of intelligence
    ❏ Take too much time to
    respond
    ❏ Same or wrong answers
    ❏ Poor connection timing
    

    View Slide

45. 45 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.5. Exploratory Testing
    This technique is used to increase your knowledge about the application while the test is ongoing.
    It may be conducted simultaneously with other tests. Exploratory testing can show you a clear picture of
    which aspects of the application work and which ones do not.
    

    View Slide

46. 46 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.5. Exploratory Testing
    Test Case Examples:
    1. If a chatbot shares links to other
    resources such as websites, email ids,
    images, etc., make sure all these
    entities are functional.
    This technique is used to increase your knowledge about the application while the test is ongoing.
    It may be conducted simultaneously with other tests. Exploratory testing can show you a clear picture of
    which aspects of the application work and which ones do not.
    

    View Slide

47. 47 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.5. Exploratory Testing
    Test Case Examples:
    Julia: 000111000011001010000111
    ChatBot: Unfortunately, I do not
    know what to answer...
    2. Enter the incorrect question as a sequence
    of numbers and check the result.
    1. If a chatbot shares links to other
    resources such as websites, email ids,
    images, etc., make sure all these
    entities are functional.
    This technique is used to increase your knowledge about the application while the test is ongoing.
    It may be conducted simultaneously with other tests. Exploratory testing can show you a clear picture of
    which aspects of the application work and which ones do not.
    

    View Slide

48. 48 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.6. Small Scale Test Automation
    It is a method that uncovers:
    ● memory errors,
    ● code defects,
    ● other issues
    that are occasionally revealed in production use.
    

    View Slide

49. 49 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.6. Small Scale Test Automation
    It is a method that uncovers:
    ● memory errors,
    ● code defects,
    ● other issues
    that are occasionally revealed in production use.
    It entails performing the same action thousands of times to see its effect on the program.
    

    View Slide

50. 50 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.7. Fuzz Testing
    This procedure involves the input of:
    ● invalid data,
    ● random data,
    ● semi random data
    which may cause unexpected failures, crashes, and issues.
    In this case, there are no preset expected results, unlike other negative test cases.
    It is simply an observation of what may potentially occur because of arbitrary inputs.
    Fuzzing looks only for crashes or some other undesirable behaviour, and millions of test cases might be
    executed before a crash.
    Moreover, many bugs such as logic errors do not crash the software system but instead produce incorrect
    output – a failure type that is much more difficult to detect.
    

    View Slide

51. 51 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.7. Fuzz Testing
    Test Case Example:
    Julia: fjdhdsjkfsdkj
    ChatBot: thinking…
    The chatbot is not active for a long time
    Step 1
    

    View Slide

52. 52 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.7. Fuzz Testing
    Test Case Example:
    Julia: fjdhdsjkfsdkj
    ChatBot: Sorry, friend. Something
    went wrong and now I can not
    answer you.
    Step 1
    

    View Slide

53. 53 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.7. Fuzz Testing
    Test Case Example:
    Julia: fjdhdsjkfsdkj
    ChatBot: Sorry, friend. Something
    went wrong and now I can not
    answer you.
    Step 1
    Julia: 4357845hjfd ??? sd iufd @@^%?
    ChatBot: thinking…
    Step 2
    The chatbot is not active for a long time
    

    View Slide

54. 54 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.7. Fuzz Testing
    Test Case Example:
    Julia: fjdhdsjkfsdkj
    ChatBot: Sorry, friend. Something
    went wrong and now I can not
    answer you.
    Step 1
    Julia: 4357845hjfd ??? sd iufd @@^%?
    ChatBot: Sorry, friend. Something went
    wrong and now I can not answer you.
    Step 2
    

    View Slide

55. 55 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.7. Fuzz Testing
    Test Case Example:
    Julia: fjdhdsjkfsdkj
    ChatBot: Sorry, friend. Something
    went wrong and now I can not
    answer you.
    Step 1
    Julia: 4357845hjfd ??? sd iufd @@^%?
    ChatBot: Sorry, friend. Something went
    wrong and now I can not answer you.
    Step 2
    Julia: What oh yes ring do help horse?
    ChatBot: thinking…
    Step N
    …
    The chatbot is not active for a long time
    

    View Slide

56. 56 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Sorry, the ChatBot just crashed!
    Ok
    !
    5. Negative Testing Techniques
    5.7. Fuzz Testing
    Test Case Example:
    Julia: fjdhdsjkfsdkj
    ChatBot: Sorry, friend. Something
    went wrong and now I can not
    answer you.
    Step 1
    Julia: 4357845hjfd ??? sd iufd @@^%?
    ChatBot: Sorry, friend. Something went
    wrong and now I can not answer you.
    Step 2
    Step N
    …
    

    View Slide

57. 57 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.8. State Transition Testing
    It originates from the concept that software can only exist in one particular state at one point in time.
    For instance, a program is considered to be in its normal state until an issue reveals itself.
    Test Case Examples:
    State transition diagram for logging into the chatbot application.
    https://www.guru99.com/state-transition-testing.html
    START 3rd Try
    ACCESS
    GRANTED
    2nd Try
    1st Try
    Correct
    Password
    Correct
    Password
    Correct
    Password
    ACCOUNT
    BLOCKED
    Incorrect
    Password
    Incorrect
    Password
    Incorrect
    Password
    

    View Slide

58. 58 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    5. Negative Testing Techniques
    5.8. State Transition Testing
    It originates from the concept that software can only exist in one particular state at one point in time.
    For instance, a program is considered to be in its normal state until an issue reveals itself.
    Test Case Examples:
    State transition diagram for logging into the chatbot application.
    If you enter the incorrect password
    three times, your account will be
    blocked with a warning message.
    https://www.guru99.com/state-transition-testing.html
    Sorry, your account is blocked!
    Ok
    !
    START 3rd Try
    ACCESS
    GRANTED
    2nd Try
    1st Try
    Correct
    Password
    Correct
    Password
    Correct
    Password
    ACCOUNT
    BLOCKED
    Incorrect
    Password
    Incorrect
    Password
    Incorrect
    Password
    

    View Slide

59. 59 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    ● Increasing software application protection from cybercrime
    and unethical hacks.
    ● Increasing guarantee of a product's quality, stability and
    pleasant to use.
    https://www.javatpoint.com/negative-testing
    https://www.educba.com/negative-testing/
    6. Benefits of Negative Testing
    

    View Slide

60. 60 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    ● Increasing software application protection from cybercrime
    and unethical hacks.
    ● Increasing guarantee of a product's quality, stability and
    pleasant to use.
    https://www.javatpoint.com/negative-testing
    https://www.educba.com/negative-testing/
    ● Enhance the possibility of covering all bases and each
    type of error, which can arise because of inconsistent human behaviour.
    ● The user will know the limitations beforehand. Knowledge of the application
    limitations allows us to use it accordingly.
    6. Benefits of Negative Testing
    

    View Slide

61. 61 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    ● Increasing software application protection from cybercrime
    and unethical hacks.
    ● Increasing guarantee of a product's quality, stability and
    pleasant to use.
    https://www.javatpoint.com/negative-testing
    https://www.educba.com/negative-testing/
    ● Enhance the possibility of covering all bases and each
    type of error, which can arise because of inconsistent human behaviour.
    ● The user will know the limitations beforehand. Knowledge of the application
    limitations allows us to use it accordingly.
    ● Give more confidence to the client before going live.
    6. Benefits of Negative Testing
    

    View Slide

62. 62 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    ● The execution of
    negative testing needs
    extra time, money,
    and effort.
    https://www.javatpoint.com/negative-testing
    https://www.educba.com/negative-testing/
    7. Disadvantages of Negative Testing
    

    View Slide

63. 63 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    ● The execution of
    negative testing needs
    extra time, money,
    and effort.
    https://www.javatpoint.com/negative-testing
    https://www.educba.com/negative-testing/
    7. Disadvantages of Negative Testing
    ● For the customer, it
    causes extreme delays in a
    software product
    or an application release.
    

    View Slide

64. 64 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    ● The execution of
    negative testing needs
    extra time, money,
    and effort.
    https://www.javatpoint.com/negative-testing
    https://www.educba.com/negative-testing/
    7. Disadvantages of Negative Testing
    ● For the customer, it
    causes extreme delays in a
    software product
    or an application release.
    ● The implementation of negative testing
    needs a trained and experienced test
    engineer to develop negative test cases.
    Highly experienced people are required
    for negative testing.
    

    View Slide

65. 65 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Which test cases are related to negative testing?
    ❏ Upload invalid files like XML, SQL, etc. files in image upload field.
    ❏ Try to enter a valid email id in the email field, a valid password in the password
    field and try to login.
    ❏ Verify whether payment is done for an invalid member, provider or broker ID by
    creating a respective record in the feed.
    ❏ Test system with expired card details.
    ❏ Test with a valid PIN for the credit card.
    Question
    

    View Slide

66. 66 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Please scan
    the QR code and choose
    the right answer(s):
    Join at slido.com
    #7237 487
    

    View Slide

67. 67 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Which test cases are related to negative testing?
    ❏ Upload invalid files like XML, SQL, etc. files in image upload field.
    ❏ Try to enter a valid email id in the email field, a valid password in the password
    field and try to login.
    ❏ Verify whether payment is done for an invalid member, provider or broker ID by
    creating a respective record in the feed.
    ❏ Test system with expired card details.
    ❏ Test with a valid PIN for the credit card.
    Question
    

    View Slide

68. 68 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    AI Testing Talks
    Thank You!
    Questions?
    

    View Slide