Software Testing - Peer Testing - Ways to improve the quality of a software product by 90%

Peer Testing is a software testing process that is done during the development of the software by a fellow developer on a daily basis for a short duration much before the software is handed off to the testing Team. This presentation explains the lessons we learnt , results achieved , what worked for us and did not work for us , how to make this work for a distributed team across different continents .

The results have been extremely positive with around 80% to 90% of reduction in the overall bugs reported.

The main benefit is the drastic improvement in the communication within the team , and the interest shown by a fellow developer in testing the code is reciprocated by his team mate which ends up in gelling the team together.

Have implemented this in multiple teams and also teams that are split across continents and this works like a charm.

Transcript

1. Peer Testing Presenter : Magesh Gangadharan Director of Software – Game Development Magesh.gangadharan@gmail.com www.linkedin.com/in/mageshgangadharan/
2. Peer Testing • What is Peer testing ? Peer testing is a simple process introduced within development life cycle / iteratio here o e de eloper tests a other de eloper’s soft are module / story much before being handed off to Quality Assurance (QA). • Does Peer testing replace actual testing by Quality Assurance ? No Peer testing does not replace actual testing. It compliments the QA testing process and helps in finding bugs early in the product life cycle.
3. How did this process start in the first place? • We kick started a super aggressive program within our company where there was a need to hire a lot of contractors from outside to test. There was a lot of tribal knowledge about the products we make and hence a concern arose on quality . • We in the development team discussed internally and more of a trial basis de ided to test ea h other’s produ t o a dail asis for a fe i utes during development i.e. much before being handed off to QA for testing. • Yes this process purely started out of a need basis and a genuine willingness from the developers to improve the quality of the product going out as they took great pride in what they deliver.
4. Peer Testing Results Review • Results – What did we measure against and how? We decided NOT to consider the no of bugs identified during peer testing rather once we have finished peer testing and handed off to QA waited to see how many bugs they are able to find during their testing phase and measure the numbers against similar projects in the past where we have not peer tested before. • Results Overview As a result of peer testing what we found was bugs reduced somewhere consistently between 85% to 90 % across multiple projects where it was applied depending on how rigorously the process was adopted by the team. Lets say that previously in a project around 100 bugs were reported and now after peer testing the bugs found got reduced to somewhere between 15 to 20.
5. Benefits of Peer testing • The communication between the team improves tremendously when identified bugs are being discussed among the team. This creates an awareness among the developers on the nature of the bug and sometimes requirements are interpreted wrongly by a developer and that gets sorted out much earlier in the life cycle. A better quality product is delivered to Product Assurance. This is one big beneficiary you will see as a result of peer testing. • Developers are good testers as they clearly understand what the expected behavior of a module / story is. • When a team member finds bugs the developer who owns the module takes it well and is also provided with necessary information like core dumps / call stacks by the other developer. • When a bug is reported by QA sometimes additional information like core dumps / stack traces which are not provided in the pid. The developer asks for those details and the tester tries to reproduce the problem and then based on when he can reproduce it / not, provides the asked info. All of this takes time.
6. Benefits of Peer testing Contd. • What we have noticed is that once the developers are fully involved in peer testing , it becomes more of a sport between developers and they take it as fun. • Developers take pride in the Quality of their work and when bugs are reported by their fellow team mate they start to believe in the value of peer testing . • As there is zero to minimal documentation involved in co-located teams the developers are not troubled in writing bugs.
7. Who and How ? • Who does peer testing ? • All de elopers i ludi g leads parti ipate i testi g other de eloper’s ode. • Wh should Leads perfor peer testi g? • Leads need to set an example for the rest of the team and hence when the lead tests another developers code in the team others follow. • How is peer testing implemented ? • Pair of two developers who are working on two different themes/games are formed within a tea a d ea h de eloper ithi a pair tests the other perso ’s odule /stor . Whe a ug is found the developer who found the bug just walks to the other developers cube and let him know in person of the issue. • Do leads / managers need to plan for peer testing in their schedule ? • Yes time need to be allocated in the schedule of each individual for peer testing. • For e er 2 eek iteratio of ork i a de eloper’s s hedule add 4 to 8 hours for peer testing. • We are not expecting to re-forecast the testing time but we expect that less time will be spent in QA phase. • We are alread ehi d s hedule ? We do ’t ha e ti e to do/ allo ate for peer testi g , hat do we do ? • If executed the bugs are going to be found early during the development phase or else they would be found during the QA testing phase. When its found late during QA testing the overall time spent on the product is more . The net result is that the product gets shipped early if Peer testing is done. If planned and executed well the quality improves.
8. When ? When does peer testing start ? • Peer testing should start as early as the deliverable is in a stable / runnable state . what has worked for us is that we allocate 3 weeks of time before handoff to QA just for Peer testing as sometimes we have seen that just by finishing a story the deliverable might not even be in a stable playable state for his team mate to test as there might be a lot of other dependencies pending. So we leave it to the developer to let know his team mate when the deliverable is ready for peer testing. Ho lo g i a da does a de eloper test other perso ’s odule ? – This time is anywhere between 15 minutes to 30 minutes in a day is hat is e pe ted a de eloper to test other perso ’s odule. – One example is when the developer is compiling the code he can go to the dev la / olleague’s u e a d test the produ t. – When the developer wants to take a break from coding he goes to the dev la a d test his part er’s soft are.
9. Peer Testing Results- Discussion When and where are peer testing results discussed ? • When a bug is identified the developer reports it to his team mate right away. Daily stand ups are where each developer discusses peer testing activity so that the rest of the team becomes aware of the time spent by the developer in peer testing and the no of issues found . What happe s if a lead / Ma ager does ’t hear a out a peer testi g results fro a particular team member for a while ? • The lead needs to ask the developer on his findings . Most probably the reasoning /e use ould e that the de eloper ould ’t fi d ti e for peer testi g as he as busy with his current work. • The lead needs to insist on the importance of peer testing to that developer and a flatl dis iss the reaso i g or if it’s a alid reaso i g the if there is a other developer within the team who is less occupied then the peer testing work for a particular module / story / for a week can be moved to that developer until the original developer who was assigned becomes free.
10. What to test ?and NOT to test? • The idea here is to ha e the de eloper test his olleague’s soft are o a dail asis for a fe i utes. A thi g that’s goi g to take a lot of ti e to just have the setup / test is not going to sustain the process of testing everyday example would be power recovery scenarios which takes a lot of time to test. The process has to sustain. • So the team needs to decide on what they would be testing and what NOT to test.
11. Where to Peer Test Where is peer testing done ? In target or in dev box ? Peer testing can be done both in target or in the dev box wherever it could be applicable. We prefer the test be done in target as it is a better system to test without interrupting the developer and also from a performance standpoint .
12. Documentation • Very little or No documentation is needed due to personal interaction and immediate action taken • The issues found in peer testing are NOT reported in any bug tracking tool and this is intentionally done to keep less documentation . Why no documentation ? The process of peer testing is suppose to be of less burden / fun to a developer and is more concentrated on the old school format of when a bug is found the tester walks to the developer in person and tells him about the issue. Adding details in a bug tracking tool can be time consuming and hence not entertained. Then what little documentation is involved ? – The tester maintains a simple issue list of bugs found during peer testing. – If we have two teams working in two different geographical different areas then developer A sends an email simply mentioning the different issues to developer B.
13. Challenges faced ? • Relu ta e a o g de elopers to test. I a a de eloper a d ot a tester . • Sustaining the process of testing everyday . The developer needs to believe that there is a lot of value in testing early. How do we handle the developer reluctance to test? Response : Everyone has to test and this is a norm going forward . This will be part of tea ’s est pra ti es. When do you start to see difference among developers w.r.to adoption of the process ? As ugs get reported their peers a d as ell he the do fi d ugs i other’s module the behavior slowly starts to change and they start recognizing the value in the process.