Software Testing

Software testing ➢ One of the most important phase in
software life cycle ➢ Required to point out defects and errors that have made during the development ➢ Make sure customers satisfaction and reliability of the product ➢ Guarantees that the product/solution fulfills all the required functional and nonfunctional requirements ➢ Decides the stability and quality of the product/solution ➢ Necessary while delivering the product to makes sure it requires lower maintenance cost and it produces more accurate, consistent and reliable results.

Developers Belief on Software Testing Developers believe that responsibility of
the testing completely holds the QA team And, ➢ their responsibility is to give a functional bundle to the QA team which covers all the happy day scenarios ➢ if there any issues assigned to them, fix those issues after the QA round and then go for another QA round ➢ implementing a new functionality to the product is more important than spending their time on testing the implemented features

Do you agree with that belief? Absolutely not…. why? ➢
Developers and QAs have two different views about the product ➢ They are testing the functionalities of the product/solution based on their views ➢ Issues which can be found by developers, will not be found by QAs. Other way around is also true ➢ Both teams should contribute to improve the test coverage of the product/solution

Developers Responsibility for testing ➢ Should write unit test cases
(and integration tests also if possible) to cover all the important functionalities of the product ➢ Test cases should cover almost all the paths in the product ➢ As developers we do have a clear idea about the most of the error paths of the system than the QA team because they will check error paths based on the user experience and user inputs, but we can test the error paths based on the implementation ➢ Test cases should be extensible if it needs to improve/extend at the QA cycle ➢ Developers should be able to be confident that the product is fully functional in their point of view (Still QAs will be able to find issues which is ok, since developers tried their best to make the product is reliable)

Test writing methods There are two types of test writing
approaches, 1. State based testing 2. Behavioural/interaction based testing

State Based Testing... verifying that the system under the test(SUT)
returns correct results. In other words code under the test ended up at the expected state with respect to the given input values Ex : testSort () { Sorter sorter = new Sorter(quicksort, bubbleSort); //Verifying the input number list is sorted regardless of the sorting algorithm //which used to sort the list assertEquals(new ArrayList(1, 2, 3), sorter.sort(new ArrayList(3, 1, 2))); }

Behavioural/Interaction Based Testing verifying that the SUT goes through the
expected flow, it calls expected methods required number of times and with expected arguments etc Ex : public void testSortQuicksortIsUsed() { // Passing the mocks to the class and call the method under test. Sorter sorter = new Sorter(mockQuicksort, mockBubbleSort); sorter.sort(new ArrayList(3, 1, 2)); // Verify that sorter.sort() used Quicksort. The test should // fail if mockQuicksort.sort() is never called or if it's called with the // wrong arguments (e.g. if mockBubbleSort is used to sort the numbers). verify(mockQuicksort, times(1)).sort(new ArrayList(3, 1, 2)); }

There is another important responsibility for developers

What is it? Writing a Testable Code

Writing a Testable Code Why is this important??? ➢ Writing
test cases should not be an overhead ➢ Test writer shouldn’t have to spend much time on setting up the environment for testing → Focus only about the SUT(system under the test) → Setting up required collaborators should be easy and straightforward → Should be able to easily identify all the dependencies for the test, by looking at the unit of code(API) that you are going to test There are some Anti-patterns(flaws) that you should be aware of to write a testable code

Flaw 1 Constructor does Real Work

Constructor does Real Work Code in the constructor should be
simple and straightforward. It should not, ➢ Create/Initialize collaborators ➢ Communicate with other services ➢ Have the logic to set up its own state

Constructor does Real Work How to identify this flaw… ➢
“new” keyword in a constructor or at field declaration ➢ “static” method calls in a constructor or at field declaration ➢ Anything more than field assignment in constructors ➢ Object not fully initialized after the constructor finishes (watch out for initialize methods) ➢ Control flow (conditional or looping logic) in a constructor ➢ CL does complex object graph construction inside a constructor rather than using a factory or builder ➢ Adding or using an initialization block

Why this is not a good practice? ➢ Difficult to
test directly ➢ Forces collaborators on test writer ➢ Violates the Single Responsibility Principle

Constructor does Real Work Work around?? ➢ Subclassing and overriding
✓ Delegates collaborator creation logic to a method which is expected to overriden in test sub classes ✓ This may work around the problem, but you will fail to test the method which is overridden and it does lots of work ✓ This work around can keep the last option if you really couldn't get rid of collaborator creation at the constructor of the SUT

Constructor does Real Work Work around?? ➢ Keep separate constructor
for testing ✓ This is also same as the previous work around ✓ You are going to miss the first few steps in the path which is expected to test. ✓ Later changes in the actual constructor will not be captured from your test cases

How to fix/avoid this flaw? Do not create collaborators, pass
them into the constructor Don't look for things! Ask for things! ➢ Move the object graph construction responsibility to some other object like a builder or factory object and pass these collaborators to your constructor ➢ Use DI to pass relevant implementation of collaborators for the construction of SUT

Examples of the flaw and fixed version of them Example
1: SUT class AccountView { User user; AccountView() { user = PCClient.getInstance() .getUser(); } } Test class ACcountViewTest extends TestCase { public void testUnfortunatelyWithRealRPC() { AccountView view = new AccountView(); // Shucks! We just had to connect to a real //RPCClient. This test is now slow. }

1: SUT class AccountView { User user; @Inject AccountView(User user) { this.user = user; } } User getUser(RPCClient rpcClient) { return rpcClient.getUser(); } RPCClient getRPCClient() { return new RPCClient(); } Test class AccountViewTest extends TestCase { public void testLightweightAndFlexible() { User user = new DummyUser(); AccountView view = new AccountView(user); // Easy to test and fast with test-double user. } }

2: SUT class Car { Engine engine; Car(File file) { String model = readEngineModel(file); engine = new EngineFactory() .create(model); } } Test class CarTest extends TestCase { public void testNoSeamForFakeEngine() { // Aggh! I hate using files in unit tests File file = new File("engine.config"); Car car = new Car(file); // I want to test with a fake engine but I can't since the //EngineFactory only knows how to make real engines. } }

2: SUT class Car { Engine engine; Car(Engine engine) { this.engine = engine; } } // Have a provider in the Module to give you the //Engine Engine getEngine(EngineFactory engineFactory, String model) { return engineFactory.create(model); } Test // Now we can see a flexible, injectible design class CarTest extends TestCase { public void testShowsWeHaveCleanDesign() { Engine fakeEngine = new FakeEngine(); Car car = new Car(fakeEngine); // Now testing is easy, with the car taking exactly what //it needs. } }

Flaw 2 API lies about it's real dependencies

API lies about it's real dependencies ➢ Your API lies
about it’s real dependencies, if it ✓ uses context or holder objects in method signatures/APIs ✓ pass the the context objects as parameters instead of passing specific property/object in the context object ✓ look for the object of interest by walking through the object graph ➢ If you are going through the object graph more than once, then you are looking right at an example of this flaw

API lies about it's real dependencies How to identify this
flaw ➢ Objects are passed in but never used directly (only used to get access to other objects) ➢ Method call chain walks an object graph with more than one dot (.) ➢ Suspicious names: context, environment, principal, container, or manager in method signatures/APIs ➢ Have to create mocks that return mocks in tests ➢ Difficult to write tests, due to complex fixture setup

Why this is not a good practice? ➢ Deceitful APIs
➢ Makes your code brittle and reduce the understandability ➢ Makes your code difficult to test

How to fix/avoid this flaw? Instead of looking for things,
simply ask for the objects that you need Don't look for things! Ask for things! ➢ By asking required objects you will delegate the object creation responsibility to the factory which is supposed to create objects and not to do any other things ➢ So you should, ✓ Only talk to your immediate friends. ✓ Inject (pass in) the more specific object that you really need. ✓ Leave the object location and configuration responsibility to the caller

1: SUT class SalesTaxCalculator { TaxTable taxTable; SalesTaxCalculator(TaxTable taxTable) { this.taxTable = taxTable; } float computeSalesTax(User user, Invoice invoice) { // note that "user" is never used directly Address address = user.getAddress(); float amount = invoice.getSubTotal(); return amount*taxTable.getTaxRate(address); } } Test class SalesTaxCalculatorTest extends TestCase { SalesTaxCalculator calc = new SalesTaxCalculator(new TaxTable()); Address address = new Address("1600 Amphitheatre Parkway..."); User user = new User(address); Invoice invoice = new Invoice(1, new ProductX(95.00)); // So much work wiring together all the objects //needed //…. assertEquals(0.09, calc.computeSalesTax(user, invoice), 0.05); }

1: SUT class SalesTaxCalculator { TaxTable taxTable; SalesTaxCalculator(TaxTable taxTable) { this.taxTable = taxTable; } // Note that we no longer use User, nor do we // dig inside the address. (Note: We would // use a Money, BigDecimal, etc. in reality). float computeSalesTax(Address address, float amount) { return amount * taxTable.getTaxRate(address); } } Test class SalesTaxCalculatorTest extends TestCase { SalesTaxCalculator calc = new SalesTaxCalculator(new TaxTable()); // Only wire together the objects that are needed Address address = new Address("1600 Amphitheatre Parkway..."); // … assertEquals(0.09, calc.computeSalesTax(address, 95.00), 0.05); } }

2: SUT class LoginPage { RPCClient client; HttpRequest request; LoginPage(RPCClient client, HttpServletRequest request) { this.client = client; this.request = request; } boolean login() { String cookie = request.getCookie(); return client.getAuthenticator() .authenticate(cookie); } } Test class LoginPageTest extends TestCase { public void testTooComplicatedThanItNeedsToBe() { Authenticator authenticator = new FakeAuthenticator(); IMocksControl control = EasyMock.createControl(); RPCClient client = control.createMock(RPCClient.class); EasyMock.expect(client.getAuthenticator()).andReturn(authentica or); HttpServletRequest request = control.createMock(HttpServletRequest.class); Cookie[] cookies = new Cookie[]{new Cookie("g", "xyz123")}; EasyMock.expect(request.getCookies()) .andReturn(cookies); control.replay(); LoginPage page = new LoginPage(client, request); // … assertTrue(page.login());

2: SUT class LoginPage { LoginPage(String cookie, Authenticator authenticator) { this.cookie = cookie; this.authenticator = authenticator; } boolean login() { return authenticator.authenticate(cookie); } } Test class LoginPageTest extends TestCase { public void testMuchEasier() { Cookie cookie = new Cookie("g", "xyz123"); Authenticator authenticator = new FakeAuthenticator(); LoginPage page = new LoginPage(cookie, authenticator); // … assertTrue(page.login()); }

Flaw 3 Brittle Global State & Singletons

Brittle Global State & Singletons ➢ Global State and Singletons
make APIs lie about their true dependencies ➢ developers must read every line of code to understand the real dependencies ➢ Spooky Action at a Distance ✓ when running test suites, global state mutated in one test can cause a subsequent or parallel test to fail unexpectedly

Brittle Global State & Singletons How to identify this flaw…
➢ Adding or using singletons ➢ Adding or using static fields or static methods ➢ Adding or using static initialization blocks ➢ Tests fail if you change the order of execution ➢ Tests fail when run in a suite, but pass individually or vice versa

Why this is not a good practice? Global State Dirties
your Design ➢ Global state is that it is globally accessible ➢ An object should be able to interact only with other objects which were directly passed into it. But this practice violates it. ✓ if I instantiate two objects A and B, and I never pass a reference from A to B, then neither A nor B can get hold of the other or modify the other’s state. This is a very desirable property of code ✓ However, object A could(unknown to developers) get hold of singleton C and modify it. If, when object B gets instantiated, it too grabs singleton C, then A and B can affect each other through C.

Why this is not a good practice? Global State enables
Spooky Action at a Distance ➢ When we run one thing that we believe is isolated (since we did not pass any references in), there will be unexpected interactions and state changes happen in distant locations of the system which we did not tell the object about ➢ This can only happen via global state ➢ Whenever you use static state, you’re creating secret communication channels and not making them clear in the API ➢ Spooky Action at a Distance forces developers to read every line of code to understand the potential interactions, lowers developer productivity, and confuses new team members

How to fix/avoid this flaw? ➢ If you need a
collaborator, use Dependency Injection ➢ If you need shared state, use DI framework which can manage Application Scope singletons in a way that is still entirely testable ➢ If you’re stuck with a library class’ static methods, wrap it in an object that implements an interface. Pass in the object where it is needed. You can stub the interface for testing, and cut out the static dependency Dependency Injection is your Friend

1: SUT class LoginService { private static LoginService instance; private LoginService() {}; static LoginService getInstance() { if (instance == null) { instance = new RealLoginService(); } return instance;} static setForTest(LoginService testDouble) { instance = testDouble; } static resetForTest() { instance = null;} SUT // Elsewhere... //A method uses the singleton class AdminDashboard { //… boolean isAuthenticatedAdminUser(User user) { LoginService loginService = LoginService.getInstance(); return loginService.isAuthenticatedAdmin(user); } }

1: Test // Trying to write a test is painful! class AdminDashboardTest extends TestCase { public void testForcedToUseRealLoginService() { // … assertTrue(adminDashboard.isAuthenticatedAdminUser(user)); // Arghh! Because of the Singleton, this is // forced to use the RealLoginService() }

1: SUT //Dependency Injected into where it is needed, //making tests very easy to create and run. class LoginService { // removed the static instance // removed the private constructor // removed the static getInstance() // ... keep the rest } // Use dependency injection to inject required //implementation of the login service // eg: bind(LoginService.class).to(RealLoginService.class) .in(Scopes.SINGLETON); SUT // Elsewhere... // Where the single instance is needed class AdminDashboard { LoginService loginService; // This is all we need to do, and the right // LoginService is injected. AdminDashboard(LoginService loginService) { this.loginService = loginService; } boolean isAuthenticatedAdminUser(User user) { return loginService.isAuthenticatedAdmin(user); } }

1: Test // With DI, the test is now easy to write. class AdminDashboardTest extends TestCase { public void testUsingMockLoginService() { // Testing is now easy, we just pass in a test- // double LoginService in the constructor. AdminDashboard dashboard = new AdminDashboard(new MockLoginService()); // ... now all tests will be small and fast } }

These are very simple facts. But if you don’t focus
about the testing when you write the code, definitely you will do at least one of the above mistakes accidentally. You may have your own list of guidelines to follow to write a quality code. Make a note about these points and add them to your list if you really need to write a testable code

Testing Frameworks and tools for Java... ✓ JUnit ✓ TestNG
✓ Mockito ✓ PowerMock ✓ Arquillian ✓ The Grinder ✓ JTest

Mockito and PowerMock... In Real-Life Units are NOT Totally Self
Contained ✓ Dependencies – In order for a function to run it often requires Files, DB, JNDI or generally – other units ✓ Side effects – When a function runs we are often interested in data written to files, saved to db or generally - passed to other units. So how can we test a single unit, without being affected by other units How can we test a single unit, without being affected by other units – their errors, set-up complexities and performance issues?

You have two options…. Stubs or Mocked Objects

Testing Models There are two type of testing models for
unit testing, 1. Stub based testing model ➢ Stubs provide pre-programmed sample values to calls made to it during the test. They may record the information about the calls such as number of search queries, number of success queries etc. They are not responding anything outside what's programmed in for the test 2. Mock based testing model ➢ Mocks are preprogrammed objects which can be trained according to the requirement of the test. The trained behaviour can be changed according to your requirement and same mock object can be trained to behave in two different ways in two different test cases

Stubs Based Testing Model Stubs provide pre-programmed sample values to
calls made to it during the test. They may record the information about the calls such as number of search queries, number of success queries etc. They are not responding anything outside what's programmed in for the test

Stubs Based Testing Model - eg: public interface ConnectivityEngineFacade {
String search(String priceRequest); }

Stubs Based Testing Model - eg: cntd... public class ConnectivityEngineFacadeMock
implements ConnectivityEngineFacade { private String ceStubFile; @Override public String search(String priceRequest) { return readFile(); } private String readFile() { //reading the content of the pre configured file(ceStubFile) and return the content as string }

Mocked Objects Based Testing Model Mocks are preprogrammed objects which
can be trained according to the requirement of the test. The trained behaviour can be changed according to your requirement and same mock object can be trained to behave in two different ways in two different test cases

Mocked Objects Based Testing Model - eg: @RunWith(PowerMockRunner.class) @PowerMockIgnore("javax.management.*") @PrepareForTest({
Util.class, XMLAdapterContext.class}) //classes to be used within the testing process public class ConnectivityEngineTest { }

Mocked Objects Based Testing Model - eg: @Mock private ConnectivityEngineFacade
connectivityEngineFacadeMock; @Before public void setUp() throws Exception { MockitoAnnotations.initMocks(this); .... when(connectivityEngineFacadeMock.search(Mockito.anyString())).thenReturn("This is the sample value to be returned within the whole test class"); } @Test public void validateConnectivityEnginePath() throws Exception { when(connectivityEngineFacadeMock.search(Mockito.anyString())).thenReturn("This is the sample value to be returned within the validateConnectivityEnginePath test case"); xmlAdaptorRequestSequence.execute(msg, mediation); //execute the required sequence to test …

Mockito and PowerMock... Mockito and PowerMock are mocking framework that
will help you to create mock object to replace your collaborators and make your testing process easier ✓ Help you to focus on the SUT ✓ Give ability verify behavioral/interaction based testing ✓ Testing result of SUT doesn’t depend on collaborator's issues and performance ✓ Low setup costs

Mockito and PowerMock... Be Careful while using mocking frameworks, if
you are using mocked objects unnecessarily then you will face some other problems such as slowing down your test suites, extremely complicated set up code etc. Read more : http://googletesting.blogspot.com/2013/05/testing-on-toilet-dont-overuse-mocks.html https://blog.8thlight.com/uncle-bob/2014/05/10/WhenToMock.html

JUit 4.+ and TestNG Feature TestNG JUnit Run test method
@Test @Test Run method before first method in class @BeforeClass @BeforeClass Run method after last method in class @AfterClass @AfterClass Run method before each method in class @BeforeMethod @Before Run method after each method in class @AfterMethod @After Ignore a test method @Ignore @Ignore

JUnit 4.+ and TestNG Feature TestNG JUnit Expected exception @Test(expected=Exception.class
) @Test(expected=Exception.class) Timeout @Test(timeout=100) @Test(timeout=100) Run method before first method in suite @BeforeSuite You have to create a suite class and add @BeforeClass to it Run method after last method in suite @AfterSuite You have to create a suite class and add @AfterClass to it Run method before first method in group @BeforeGroup JUnit has similar concept of Category which can be included or excluded in a suite Run method after last method in group @AfterGroup JUnit has similar concept of Category which can be included or excluded in a suite

JUnit 4.+ vs TestNG Ordering… Ordering is needed when some
test cases are supposed to be run before/after other test cases eg: running loginTest before updateNameTest or updateEmailTest TestNG - test methods can be ordered via dependsOnMethods @Test(dependsOnMethods={“login”}) JUnit - Doesn’t have straightforward ordering mechanism. Can be ordered only by the test method names’ alphabetical order @FixMethodOrder(MethodSorters.NAME_ASCENDING)

JUnit 4.+ vs TestNG Test Suites… You might want to,
✓ Create a suite of unit tests and another suite of integration tests and run these two suites separately ➢ Both TestNG and JUnit have support for creating test suites ➢ TestNG - you can easily do that in a simple xml file ➢ JUnit - you have to create a class and add @RunWith, @SuiteClasses etc do the same thing ➢ Suites also allow you to run specific types of test methods, TestNG calls them as group and JUnit calls them as category.

JUnit 4.+ vs TestNG Test Suites… Test Suite example in
with TestNG <suite name=“unit tests” > <test name=“manager tests”> <groups> <run><include name=“slow”/></run> </groups> <classes><class name=“com.manager”/><classes> </test> </suite>

JUnit 4.+ vs TestNG Test Suites… Test Suite example with
JUnit4 @RunWith(Suite.class)@Suite.SuiteClasses({AdminManager.class, UserManager.class}) @IncludeCategory(SlowTest.class) public class UnitTestSuite { // the class remains empty, used only as a holder for the above // annotations // add @BeforeClass or @AfterClass to run methods before/after first/last // method in this suite }

JUnit 4.+ vs TestNG Data providers and parameterised tests… JUnit4…
✓ provides @Parameters that can be added to a static method that contains data, this method is called while instantiating the test class. ➢ return value is passed to the test class as an argument, ➢ test class constructor has to accept arguments. ➢ JUnit also provides @Parameter that can be added to any member field to set the data directly to the field instead of constructor.

JUnit 4.+ - Parameterized Test Example @RunWith(value = Parameterized.class) public
class JunitTest { private int number; public JunitTest(int number) { this.number = number; } @Parameters public static Collection<Object[]> data() { Object[][] data = new Object[][] { { 1 }, { 2 }, { 3 }, { 4 } }; return Arrays.asList(data); } @Test public void pushTest() { System.out.println("Parameterized Number is : " + number); }

JUnit 4.+ vs TestNG Data providers and parameterised tests… TestNG…
✓ provides better features, ➢ it allows you to add parameters directly at the test method ➢ if input data is not a complex type, you can configure input data with the xml configuration file. ➢ allows you to assign a method as data provider which any test method can use to get data dynamically.

TestNG - Parameterized Test Example(Primitive Types) public class TestNGTest6_1_0 {
@Test @Parameters(value="number") public void parameterIntTest(int number) { System.out.println("Parameterized Number is : " + number); } } <suite name="My test suite"> <test name="testing"> <parameter name="number" value="2"/> <classes><class name="com.fsecure.demo.testng.TestNGTest6_0" /></classes> </test> </suite>

TestNG - Parameterized Test Example(Compex Types) @Test(dataProvider = "Data-Provider-Function") public
void parameterIntTest(TestNGTest6_3_0 clzz) { System.out.println("Parameterized Number is : " + clzz.getMsg()); System.out.println("Parameterized Number is : " + clzz.getNumber()); } @DataProvider(name = "Data-Provider-Function") public Object[][] parameterIntTestProvider() { TestNGTest6_3_0 obj = new TestNGTest6_3_0(); obj.setMsg("Hello"); obj.setNumber(123); return new Object[][]{ {obj} }; }

JUnit 4.+ vs TestNG - conclusion ✓ From the features
point of view both frameworks are almost same ✓ Syntaxes are also same or at least with the same pattern ✓ But, ➢ JUnit introduces some unnecessary constraints while using its features while TestNG provides more convenient approach of configuration ➢ Eg: • For parameterized test, JUnit only allows to provide data to the complete test class not for each test method • JUnit expects most of the methods as static(@Parameters, @BeforeClass methods) • JUnit expects to configure test suites at class level. while TestNG allows to configure them via xml file ➢ TestNG allows to configure dependent test methods while JUnit doesn’t provide a flexible solution for that

JUnit 4.+ vs TestNG - conclusion If you aren’t already
using any framework, you can go with TestNG as it’s easy to configure and maintain. However, if you are already using JUnit, I would suggest you to upgrade to latest version of JUnit that has more features for grouping, parallelism, assertions etc

Software Testing

Software Testing

More Decks by AdroitLogic

Other Decks in Programming

Featured

Transcript