Agile Testing at Grange Insurance

We may be a "boring insurance company," but we're pushing the envelope for software development and testing. Leveraging ATDD and Agile practices with Cucumber and Watir, we have seen a dramatic increase in team productivity and quality.

Transcript

1. Page 1 November 7, 2012 Presentation Title Agile Testing at Grange Insurance Mark Harris, QA Director, Grange Insurance
2. What we do Partnering with Development (IT) and Business to consistently produce quality software that enables business success. Who we are Dynamic, creative, engaged team of software professionals, who bring value to every conversation. How we do it Small, integrated teams with transparent results and continuous improvement.
3. Page 3 November 7, 2012 Presentation Title Dynamic, creative, engaged team of software professionals, who bring value to every conversation.
4. 1. Do I know what is expected of me at work? 2. Do I have the materials and equipment I need to do my work right? 3. At work, do I have the opportunity to do what I do best every day? 4. In the last seven days, have I received recognition or praise for doing good work? 5. Does my supervisor, or someone at work, seem to care about me as a person? 6. Is there someone at work who encourages my development? 1. At work, do my opinions count? 2. Does QA’s mission make me feel my job is important? 3. Are my co-workers committed to doing quality work? 1. In the last six months, has someone at work talked to me about my progress? 2. This last year, have I had opportunities at work to learn and grow?
5. Proc/QA Architect, 2 Sr. Analyst, 2 Manager, 3 Test Lead, 3 Tester II, 4 Test Engineer, 6 Analyst II, 12 Analyst I, 16 QA Staff by Role Page 5 November 7, 2012 Software QA for Grange IT 52 people total
6. New Roles • Process Architect – Provide analytics and implement SDLC … process improvements based on the trends identified • QA Architect – Lead the architecture, … implementation, … of systems which … drive quality throughout … the SDLC • Test Engineer (Senior, Architect) – Support the delivery of high quality software from a technical perspective while providing guidance to the team … will write code and has technical skills … • Test Lead – Lead team to provide a planned approach to … ensure system functionality, adherence to filed eligibility rules and guidelines established by our business partners. Page 6 November 7, 2012 Presentation Title
7. Training • 70% On the Job – Side-by-Side work with external partners who’ve done it – Accentient for Test Case Manager 2010 – Quality Push: 6 week integrated effort • 20% Formal – Webinars, STAR East, QA or Highway, CodeMash – Degrees & Certs (Franklin, CSTP, CSTE) • 10% Self Directed – Test Automation User Group, COQAA – Books, Blogs, MSDN Magazine, Code Katas Page 7 November 7, 2012 Software QA for Grange IT
8. Building Team Sharepoint is our central communication point for the team. All documents, announcements, resources, and contacts kept in the site. Page 8 November 7, 2012 Software QA for Grange IT
9. Building Team The Blog is our forum for team communication. Editors are in charge of content, publishing, editing, but ANYONE is welcome to author a post. Regular columns: •Bug of the Month •Management Monday •Tester Tuesday Page 9 November 7, 2012 Software QA for Grange IT
10. Come Aboard! • Explicitly hiring people with software background: – 9 people since 2010 • Results-Oriented Interviews – Ask task-based questions during the interview – Interview with several test managers • Greater mix of Contractors and staff augmentation – 6 contractors ~ 11% Page 10 November 7, 2012 Software QA for Grange IT
11. Page 11 November 7, 2012 Presentation Title Move quality to the left for true partnership
12. Partnership & Transparency Page 12 November 7, 2012 Software QA for Grange IT
13. Requirements Design Code Test Release! Test-Driven Write tests with requirements Tests are specifications of the system’s behavior – Failing tests show a difference between specification and implementation, i.e. a BUG Communication is REQUIRED
14. Feature: Amend Driver with an accident prevention course completed on a BP Policy As a user, I want to amend a BOP with a CA policy driver who has completed an accident prevention class, so that I can give them a more appropriate rate Scenario: Add a driver who is an armed forces member with accident prevention course completed Given I am an authenticated "Grange" "inhouse" And I have an existing policy "1117060" And I set the Effective Date of Change to "9/1/2012" When I add the following driver with the specified information | driver_name | birth_date | gender | license_number | license_state | marital_status | | Driver Two | 07/23/1964 | Male | RR168996 | Ohio | Single | Then I should only receive the following IL16 messages: | message | | Adding Driver 2: Driver Two | | Adding an Approved Accident Prevention Course for Driver Two | Page 14 November 7, 2012 Software QA for Grange IT
15. Page 15 November 7, 2012 Presentation Title Use tools to leverage the best capabilities of our team
16. Page 16 November 7, 2012 Software QA for Grange IT
17. Page 17 November 7, 2012 Software QA for Grange IT
18. Test Case Management Page 18 November 7, 2012 Software QA for Grange IT
19. Page 19 November 7, 2012 Presentation Title Tangible results show our success
20. Page 20 November 7, 2012 Software QA for Grange IT
21. Q1 Q2 Q3 Q4 Customer Reported Production Issues 2010-2012 2010 2011 2012 Average same-quarter improvement: 38% since 2010
22. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 2011 Q1 2011 Q2 2011 Q3 2011 Q4 2012 Q1 2012 Q2 2012 Q3 Model-to-Production 2011-2012 Production Model -28% in prod bugs +156% in model bugs
23. 1 2 3 4 Days to Resolve Critical Issues (by Quarter) 2010 2011 2012
24. 1 2 3 4 Critical Issues Opened (By Quarter) 2010 2011 2012 Page 24 November 7, 2012 Software QA for Grange IT
25. Page 25 November 7, 2012 Presentation Title Continue the Journey
26. • Business Intelligence QA – custom made for ATDD • Better, quicker customer feedback • Acceptance Test Driven Development & other automation for: – Business Systems (like Guidewire) – Mainframe systems – Thick-client desktop systems – Mobile applications – 3rd party integrations Page 26 November 7, 2012 Software QA for Grange IT
27. Success is Infectious Page 27 November 7, 2012 Software QA for Grange IT
28. Page 28 November 7, 2012 Presentation Title THANK YOU!