API Resiliency Testing - Speaker Deck

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

© 2025 All Rights Reserved Dependencies System Under Test Test Test Mode – Contract Test Setup Request Response Request Response 1 3 4 5 Message Contract Test Http Stub Kafka mock Broker (In-memory) Schema Validation Topic BFF Set Expectations 2 Set Expectations Verify message count and schema 7 8 6

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

© 2025 All Rights Reserved Property Based Testing • Define rules or properties that your system should always satisfy • Instead of writing individual test cases • Example: “Addition is commutative: a + b = b + a.” https://scalacheck.org/ QuickCheck https://hackage.haskell.org/package/QuickCheck

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

© 2025 All Rights Reserved Mutation Testing • Introduce small deliberate changes called mutations in your code • See if your tests can identify them • Thereby killing the mutation • Mutation Score = Mutants Killed / Total Mutants • Examples: • Java PIT • Python Mutmut • JS / TS Stryker

Slide 16

Slide 16 text

© 2025 All Rights Reserved Mutation Testing as applied API Specification • Instead of mutating code, introduce mutations in inputs. Examples: • Skip sending a mandatory field • Send the wrong data type • Expect the application to handle these spurious inputs

Slide 17

Slide 17 text

© 2025 All Rights Reserved Negative Scenario Testing • Deliberately send invalid inputs • And expect that the response is not a ”200 OK ” • Thereby validating that the input mutation has been successfully killed by input validation in your Component / Service (System Under Test)

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

© 2025 All Rights Reserved Issues with 500 response for invalid input • Conveys poor health of application • Thereby even bad input can lead to middleware considering service to be unavailable • Exposes unnecessary technical details

Slide 21

Slide 21 text

Slide 22

Slide 22 text

© 2025 All Rights Reserved Input validation is non-negotiable • “Insufficient Input/Output Validation” is one of the top risks listed by OWASP - https://owasp.org/www-project-mobile-top-10/2023- risks/m4-insufficient-input-output-validation • Can lead to incorrect data being introduced into your data store • Compromise other system and services that depend on this data • And many more high-risk issues

Slide 23

Slide 23 text

Slide 24

Slide 24 text

© 2025 All Rights Reserved Why 77 tests for POST /products ?? Fields Data Type Required Constraints Values name string Yes - - type string Yes Enum book, food, gadget & others inventory number Yes min & max min = 1 & max = 101 Positive Variations Negative Variations Variation Count string int, bool, null 4 4 enum values int, bool, null 7 1, 1