Ingo Philip Evangelist Product Manager Tricentis The Future of Software Testing

© 2016 by Ingo Philipp © 2016 by . © 2016 by . The Future of Testing

© 2016 by General Testing Relativity Your future depends on what you do in your present. Mahatma Gandhi

© 2016 by 80% Manual Testing 20% Automated Testing UI API Testing Present The reliance on manual testing is the top technical challenge in app development. World Quality Report, 2015/16 Test automation requires developers. Testing Present

© 2016 by Testing Present Testing Future Exploratory Tests Automated UI Tests API Tests Orchestrated Service Virtualization 80% Manual Testing +85% 20% Automated Testing - key enabler for high automation rates - Testing Future UI API

© 2016 by 100% 0% Sprint Start Sprint End API UI Start testing early. Shift left test automation. API Testing Degree of Completion Time Efficiency Gain + 4x Creation + 6x Maintenance + 20x Execution

© 2016 by *voke, market snapshot report on service virtualization - 2012 On average, organizations require access to 33systems for development or testing. 18 Average # of systems with unrestricted access 96 % of testers have restricted test lab access Orchestrated Service Virtualization Testing Present Enterprise system landscapes are alike disease gene networks. Welcome to the tester’s hell. Identify 60% to 90% more defects at least one step earlier in the development process. *Service Virtualization – Window into Advanced DevOps - 2014

© 2016 by Do the things right. Automation is the dictate of today. Automation doesn’t make testing easy, it makes testing possible.

© 2016 by It’s me, a problem! Start End software delivery cycle 67% average level of redundancy in enterprise test portfolios 40% average risk coverage achieved in enterprise test portfolios 90% of all test automation is UI test automation 80% overall testing effort goes into manual testing 55% of systems only partially accessible by Dev/Test 56% of overall test effort goes into test case maintenance 30% of bugs found in acceptance & production stage 50% of manual testing goes into test data preparation & organization Albert Einstein, 1921 “The first step to solve a problem is to accept that you have one.” Is it just about speed?

© 2016 by By compressing the delivery cycle, do problems just move closer to each other? Is it just about speed? Start End software delivery cycle

© 2016 by Start End software delivery cycle No, they mutually reinforce each other! The devil is in the combination! It’s about speed@quality. Hence, the biggest strength of DevOps is not solving problems, but rather exposing buried problems.

© 2016 by Do the right things right. Testing harder isn’t the answer, testing smarter is. Wolfgang Platz

© 2016 by Low Risk High Risk Medium Risk 80% 20% Business Risk Coverage Test Cases The time needed for testing is infinitely larger than the time available. Critical Limit Risk-Coverage Optimization Have the right test cases. Do the right things.

© 2016 by Risks Your system under test. Performance Issue Usability Issue Functional Issue Stability Issue Reliability Issue Scalability Issue Coherence Issue Understandability Issue Testability Issue Convenience Issue Security Issue Accessibility Issue

© 2016 by Specification Based Testing Exploratory Testing Exploratory Testing Analyze Potential Risks Problem vs. No Problem Monitor Known Risks Pass Result vs. Fail Result Effective testing results from the combination of both. *That’s an illustration. Don’t confuse it with reality. Increase your testing cross section. Actively attack risk. Go beyond the obvious. Diversify your testing. Enrich your test design. Find more critical bugs. Fast error detection. Provide rapid feedback. Make your testing intellectually rich. Core Benefits

© 2016 by 100% Manual Testing 0% Automated Testing Testing Present Customer Case Study There’s a way to do it better. Find it! Project Charter 11 Manual Testers 4755 Manual Test Cases ? Unknown Coverage 10 Weeks Execution

© 2016 by Risk Coverage Optimization Test Data Management 48 Hours Execution Single Agent 8 Hours Execution Multiple Agents Customer Case Study 11 Manual Testers 4755 Manual Test Cases ? Unknown Coverage 10 Weeks Execution 89% Business Risk Coverage 1193 Manual Test Cases 37% Business Risk 5 Weeks Execution 2.5 Weeks Execution 92% Automated Test Cases 53% Business Risk Coverage Smoke Testing 34 Minutes Execution Smoke Testing Test Case Count 7% Automation (UI & API) & OSV Continuous Integration & Distributed Execution 75% Redundancy 50% Effort Test Data 64% API Tests

© 2016 by 8 Hours Execution Multiple Agents Customer Case Study 72% Critical Defect Reduction Production Defects 89% Business Risk Coverage 1193 Manual Test Cases 5 Weeks Execution 2.5 Weeks Execution 92% Automated Test Cases 34 Minutes Execution Smoke Testing 53% Business Risk Coverage Smoke Testing Test Case Count 7% Test Case Design & Exploratory Testing Risk Coverage Optimization Test Data Management Automation (UI & API) & OSV Continuous Integration & Distributed Execution 64% API Tests

© 2016 by Bottom Line The future starts today, not tomorrow. Start remodeling your traditional approaches for greater agility today.