Improved software testing using Visual Studio and TFS 2010

Transcript

1. Improved software testing using Visual Studio and TFS 2010 Alexander Vanwynsberghe Software Engineer @vanwynsberghea
2. What is this presentation about? • Software testing, why? • Use structured testing • The capabilities of TFS 2010 • Summary Alexander Vanwynsberghe – 09/05/2011
3. Software testing, why? • It’s all about the quality • Lets you measure how well software is designed • Find the bugs! • Checks how the product meets the requirements • Testing does not make software perfect Alexander Vanwynsberghe – 09/05/2011
4. Use structured testing What does that mean? http://www.flickr.com/photos/59068359@N04/5563821201
5. 1. Make test plans • A summary of the test activities that will be performed • Shows how the tests will be organized • A list of tests which, if performed, will identify all of the requirements that have been set Alexander Vanwynsberghe – 09/05/2011
6. 2. Define test cases • Description of a specific interaction that a tester will have in order to test a single behavior of the software • Step-by-step guide for the tester • Consists of: • Preconditions • To-do • Result Alexander Vanwynsberghe – 09/05/2011
7. 3. Execute tests • Run all test cases from the appropriate test plan • First iteration test focuses on new features • Regression testing usually involves executing all test cases which have previously been executed • Test parallel with the developers, not only just before a release Alexander Vanwynsberghe – 09/05/2011
8. 4. Automate tests • User interfaces can be test-automated • Eliminate repetitive tasks • Can save the testers a lot of time if many iterations of testing are required • Do not automate one-time-only tests Alexander Vanwynsberghe – 09/05/2011
9. http://www.flickr.com/photos/harmony19490/362411276/ 5. Report bugs And assign them to developer(s)
10. http://http://www.flickr.com/photos/frau_k/3984560746// The ‘how-not-to-do-it’ process
11. The developer.. • All coding done • Some small „debugger-tests‟ • Check-in/commit • Deploy to a development environment • “Test please” to tester… Alexander Vanwynsberghe – 09/05/2011
12. The tester tests.. • No plan available (no user stories) • Lucky to find bugs • Waste a lot of time on irrelevant things • This is called “Exploratory testing” Alexander Vanwynsberghe – 09/05/2011
13. ..and reports the bug.. Alexander Vanwynsberghe – 09/05/2011 Email subject: Mayor bug found!! Email content: “I can’t quite remember what I did, but my data is gone !?” Where is my data?
14. The dev tests and replies.. Alexander Vanwynsberghe – 09/05/2011 Email subject: RE: mayor bug found!!
15. Problems Alexander Vanwynsberghe – 09/05/2011 • Tester had no plan – Exploratory testing • Tester tests everything • “Unable to reproduce” = “Works on my machine” • Bug never gets fixed • Ship a buggy product
16. http://http://www.flickr.com/photos/frau_k/3984560746// There is a better way
17. Use the power of TFS 2010 Alexander Vanwynsberghe – 09/05/2011 All data is stored in TFS
18. http://www.flickr.com/photos/avalsitarb/403407215/ Manual testing
19. Manual testing with MTM 2010 Alexander Vanwynsberghe – 09/05/2011
20. What can a tester do with MTM? Alexander Vanwynsberghe – 09/05/2011 • Create test plans • Define test cases • Execute test plans • Step-by-step execution • Screen recording • IntelliTrace logging • Inline bug creation with attachments
21. What does the developer receive? Alexander Vanwynsberghe – 09/05/2011 • A work-item of type ‘bug’ assigned to him/her • An email to inform about the bug (configured using TFS Alert Explorer) • The performed testcase with completed steps • Necessary information to reproduce the bug • IntelliTrace to get to the ‘bug point’ in code • Video recording • Comments of the tester
22. Ok, bug solved, what’s next? Alexander Vanwynsberghe – 09/05/2011 • Check-in the changes • Link the bug to the changeset • The changes for a particular bug are traceable • Developer sets the bug work-item to ‘Resolved’ • Tester can test again and check the influence of the code change on other tests (Test Impact Analysis)
23. Some interesting things Alexander Vanwynsberghe – 09/05/2011 • When the developer finds the bug: • Don’t fix it at first • Create an automated test from the recorded test • Let the test fail • Fix the bug • Run the test again • TDD Will be explained later
24. http://www.flickr.com/photos/avalsitarb/403407215/ Demo MTM 2010
25. http://www.flickr.com/photos/slimwhitman/2317727338 Automatic testing
26. Automatic testing with VS2010 Alexander Vanwynsberghe – 09/05/2011
27. What can be tested with VS2010? Alexander Vanwynsberghe – 09/05/2011 • Code from the developer • Unit testing • Integration testing • User Interface = Coded UI testing • Database (stored procedures) = Database testing • Web Performance testing • Load testing
28. Unit tests Alexander Vanwynsberghe – 09/05/2011 • A piece of code that tests another piece of code • Readable • Understand! • What? Scenario? What will happen? • Maintainable • Trustworthy • If all unit tests passed = bug free
29. The structure of a unit test Alexander Vanwynsberghe – 09/05/2011 1. Arrange 2. Act 3. Assert [TestMethod] public void Add_TwoPlusThree_EqualsFive() { // arrange int expected = 5; int first = 2; int second = 3; var calculator = new Calculator(); // act int actual = calculator.Add(first, second); // assert Assert.AreEqual(expected, actual, “addition failed”); }
30. Integration test Alexander Vanwynsberghe – 09/05/2011 • Integrates different pieces: • UI • Business Logic • Database • Touches: • File system • Database • Other shared resources • Not in memory
31. Coded UI test Alexander Vanwynsberghe – 09/05/2011 • Test that the user interface functions correctly • Created using recording, or using code • For web, Win-forms and Silverlight • Can check control behaviors (like label values)
32. Database testing Alexander Vanwynsberghe – 09/05/2011 • Unit test for database objects like stored procedures • Can execute T-SQL statements like: • SELECT * FROM Content.Workspaces WHERE WorkspaceId = 1 --will fail, because it exists
33. Web performance testing Alexander Vanwynsberghe – 09/05/2011 • Simulate a user performing a set of operations • Validate the responses to see if the application is working as expected • Created by recording a human user clicking through a Web application • Web tests can be made smarter by using Validation Rules, Extraction Rules, and Test Context
34. Web performance testing Alexander Vanwynsberghe – 09/05/2011
35. Load testing Alexander Vanwynsberghe – 09/05/2011 • Contains web performance tests and unit tests • Simulate virtual users by setting the number of iterations • Collects all sorts of information like: • Response times of the executed requests, • Performance counters of the tested application infrastructure (like CPU, Memory, I/O, …) • The health of your load-testing infrastructure (load controller and agents)
36. Load testing Alexander Vanwynsberghe – 09/05/2011
37. http://www.flickr.com/photos/avalsitarb/403407215/ Demo automatic testing
38. http://www.flickr.com/photos/niffgurd/3324865930 What you may not forget
39. Summary Alexander Vanwynsberghe – 09/05/2011 • Testing is important, and easy with TFS • It gives confidence to those selling the product • The earlier a defect is found, the less development rework and re-test is required • Use proven high performance as a selling point • Testing needs discipline
40. Engelse Wandeling 2, K18 B-8500 Kortrijk, Belgium alexander.vanwynsberghe@cognistreamer.com www.cognistreamer.com Alexander Vanwynsberghe @vanwynsberghea