Testing 3.0 - Testing 3.0 - Contributing to Sutainable Success of Enterprise 3.0

Presentation by Laurent Gaudy, Plenary Keynote Speaker @STC 2012.

Presentation Abstract

The modern enterprise is no longer a single, isolated, monolithic organization. Customers, Partners, Suppliers, Outsourcers, Distributors, Resellers, . all kinds of entities extend and expand the boundaries of the enterprise, and make "collaboration" and "sharing" important. This new reality has been dubbed "Enterprise 3.0".

Testing should be based on constant dialogue and cooperation with the different stakeholders in the production, development and commercialization of higher quality software. Tomorrow, testing will not only suggest the necessary fine tuning of the application to the peculiar environment of the customer but, beyond that, the inevitable adaptations incumbent upon the customer to accomodate change and innovation.

The object of the talk is to envision how testing will lead to ever more demanding but productive communication and dialogue between developers, networks, customers and various experts or consultants in the different fields.. Thus in the diversification of its relevant services to firms, "Software testing 3.0" will meet, point after point, the challenge of " Enterprise 3.0".

About Laurent Gaudy

He is VP Partners & Alliance at Neotys, has a 17-year experience of international business development across several sectors such as luxury goods, retail and IT. Laurent served at Microsoft for 10 years where he held various sales & marketing responsibilities across three continents (Europe, Asia, Middle-East/Africa). His expertise covers IT channels, alliance and key accounts development, complex project and multi-cultural management. Laurent graduated from HEC Paris, ESADE Barcelona and the Chartered Institute for Management Accountant (CIMA, UK).

Transcript

1. TESTING 3.0 – CONTRIBUTING TO TESTING 3.0 – CONTRIBUTING TO SUSTAINABLE SUCCESS OF ENTERPRISE 3.0 Laurent GAUDY, Vice-President Partners & Alliances
2. AGENDA 1. The new face of Enterprise 2. A (short) history of Software Testing 3. Time for Maturity 4. Software Testing 3.0: What will it take? © 2011 Neotys - All Rights Reserved. 2 4. Software Testing 3.0: What will it take? 5. New environment, new challenges: Is Testing up to it? 6. Creating the Right Conditions for Sustainability 7. Future is on its way
3. "It is tough to make predictions, especially about the future.“ —Yogi Berra © 2011 Neotys - All Rights Reserved. 3 about the future.“ —Yogi Berra
4. WHO REMEMBERS E1.0? © 2011 Neotys - All Rights Reserved. 4
7. George Clair Tooker, Jr. (1920 – 2011)
8. THE NEW FACE OF ENTERPRISE © 2011 Neotys - All Rights Reserved. 8
9. THE NEW FACE OF ENTERPRISE More than just a push toward integrating the social and collaborative tools of Web 2.0 into the office environment. Enterprise 2.0 also represents a fundamental change in how © 2011 Neotys - All Rights Reserved. 9 a fundamental change in how businesses operate. Customers, Partners, Suppliers, Outsourcers, Distributors, Resellers, … all kinds of entities extend and expand the boundaries of the enterprise, and make “collaboration” and “sharing” important.
10. COLLABORATE & SHARE © 2011 Neotys - All Rights Reserved. 10 Akbar, Mughal emperor, 16th Century, receiving Jesuits as well as Muslim & Hindu theologians.
11. COLLABORATE & SHARE © 2011 Neotys - All Rights Reserved. 11 Utagawa Hiroshige Vincent Van Gogh
12. COLLABORATE & SHARE The cruel proprietor © 2011 Neotys - All Rights Reserved. 12 Jean-Claude Bonnefond, 1824 Wong Hoy Cheong, 2009
13. A REVOLUTION IN THE MAKING Significant organizational disruptions ahead! 1. Globalization 2. IT complexity 3. Software defined networks 4. Bigger data and storage 5. Hybrid cloud services © 2011 Neotys - All Rights Reserved. 13 5. Hybrid cloud services 6. Client and server architectures 7. Internet of things 8. IT appliance madness 9. Operational complexity 10.Virtual data centers 11.Exploding IT capacity
14. WHETHER E3.0 IS ON… Is E2.0 over yet? If so, what would E3,0 be? « The Mobile Enterprise »….. Enterprise 3.0 = (SaaS + Extended Enterprise)…? © 2011 Neotys - All Rights Reserved. 14
15. …OR NOT: ENTERPRISES NEED TO ADAPT Companies are facing the full impact of globalization today, and yet, their IT systems were designed long time back. Prepare for organizational disruptions – externally and internally generated. Software networks are coming: begin updating skills now. © 2011 Neotys - All Rights Reserved. 15 Software networks are coming: begin updating skills now. Hybrid cloud services are evolving rapidly. Internet of things will be here…
16. RISE OF SOFTWARE…TESTING Software expands into newer spaces and virtually all walks of life. World is getting revolutionized around ‘quality © 2011 Neotys - All Rights Reserved. 16 ‘quality New technologies (RIA, mobile, cloud…) give a new dimension for testing Software Quality Testing is becoming a business-led activity.
17. THE FUTURE OF THE TESTING WORLD The Time-to-Market Revolution – More pressure from customers & competitors drive delivery cycles down. The commoditization of technology – reducing or eliminating the cost of most of the technological products needed for our work today. © 2011 Neotys - All Rights Reserved. 17 products needed for our work today. Globalization - where international and cultural boundaries have been blured and are slowly disapearing. The Cloud / Virtualization Revolution – access cheap & unlimited computational resources
18. A (SHORT) HISTORY OF SOFTWARE TESTING In the beginning there were no testers. © 2011 Neotys - All Rights Reserved. 18 In the beginning there were no testers.
19. Programmers did it all. Then.. a BREAKTHROUGH: “Developers cannot be good testers!” First testers appeared but suffered lack of consideration, SOFTWARE TESTING 1.0 © 2011 Neotys - All Rights Reserved. 19 visibility and recognition: software testing was misunderstood. Not a profession Few, useful but cumbersome testing tools and methods Disconnection from executive management
20. AND THEN CAME SOFTWARE TESTING 2.0 A first recognition: yes, software testing was an important part of the development process. Apparition of ownership issues: where testing fits organizationally, how it is budgeted for, and from where testing should receive its direction?... © 2011 Neotys - All Rights Reserved. 20 Explosion of available tools. These tools often further distracted the testing efforts, turning the process into one of tool selection without clear goals, architecture or direction. Better understanding from executive management but still low involvement.
21. SOFTWARE TESTING 2.0 FALLS SHORT 1. Lack of involvement & understanding from senior management. 2. Emphasis on testing for testing’s sake, and tools and tool selection without any The importance of testing did not come across © 2011 Neotys - All Rights Reserved. 21 tools and tool selection without any strategy for guidance. Ill-equipped testing programs to actually meet organizations’ needs for better software made faster and cheaper. Led to damaging perceptions and complaints towards software testing: “Our organization struggles with test automation and offshoring and we do not see significant gains in coverage or quality, or reduced costs…” “Software testing requires too much hands-on…” “Software quality problems are as pervasive as ever in spite of advancements in development techniques and technologies…”
22. TIME FOR MATURITY After taking one step forward in the 60's and 70's, Testing took two steps backwards in the 80’s and 90’s! In reaction, the "agile movement“ early 2000’s has succeeded in establishing testing as an integral part of software development, along with coding. Testing has been emerging into a healthy industry and an established discipline. Yet, lack of visibility on key functions and challenges of software testing denies its well- © 2011 Neotys - All Rights Reserved. 22 Yet, lack of visibility on key functions and challenges of software testing denies its well- deserved “strategic” importance. Why Treat Software Testing as a Strategic Initiative? Software testing typically represents 40% of the development budget. Helps the organization produce and release higher quality software. Helps the development team to produce higher quality code. Should provide senior management with information about Product quality. Treating Software Testing as a Strategic initiative is what Software Testing 3.0 is all about!
23. BUILDING THE CASE FOR SOFTWARE TESTING 3.0 To be convinced of the value of this proposition, it is important for senior management to understand the “whys” of Software Testing 3.0. Software Testing produces key metrics or data that provides visibility into the status and quality of the software under test. Development & Senior management should use this data: Development can deliver better code to testing. © 2011 Neotys - All Rights Reserved. 23 Senior management can make critical decisions about product schedules and delivery, as well as resource allocations. With visibility comes: Organizational improvement Better confidence in the quality (consistency and dependability) of the software Better decision making, and better confidence in the decision making because it is based on data Fewer “surprises” Better allocation and utilization of resources and budget Ability to spend more time and money on development and less on maintenance Delivering higher quality software
24. THE EVOLUTION TO SOFTWARE TESTING 3.0 Software Testing 3.0, treating software testing as a strategic effort, enables an organization to turn quality into a strategic advantage. Software Testing 3.0 starts with the understanding that testing is valuable to the organization. The foundation for Software Testing 3.0 is a strategic end-to-end framework for change based upon a strategy to drive testing activities, tool selection, and people development. © 2011 Neotys - All Rights Reserved. 24 tool selection, and people development. Fundamental to its success is the realization that testing is an integral part of the development process. Testing must have its own budget to build the necessary infrastructure, as well as clear leadership and ownership. It must give executive management visibility for smart decision-making & confidence about customers’ expectations being met.
25. FROM SOFTWARE TESTING 2.0 TO SOFTWARE TESTING 3.0 Starts with a strategic commitment A transformation of culture, methodology, beliefs, and techniques An organization must emerge to address the needs of people, practice, and process: 1. Methodology to drive tool selection (tools that support the methodology) © 2011 Neotys - All Rights Reserved. 25 2. People development (training, tools, and software testing techniques that support the methodology), 3. Metrics to provide management with visibility into the status and quality of the software under test. A new organization fostering communication & interactions: move away from Organic Testing Teams and seek integration within Development Teams, or belong to a QA team that oversees the quality and testing processes for the whole organization: emergence of Testing Centers Of Excellence. Source: World Quality Report 2012-2013
26. 1.METHODOLOGY: EMPOWERING NOT ISOLATING Software Testing cannot be discussed in isolation: It always has to be in conjunction with the Development phase. If we think Software Testing as a separate line of business, we will always be in a mode of code-test-code-test. Improving quality significantly at the source is KEY: Quality In, Quality Out! © 2011 Neotys - All Rights Reserved. 26 Tester (Software Design Engineer in Test) must focus on getting good End-2-End scenarios, complex integration, scale, performance testing, etc. 1. Finding bugs early in the cycle 2. Prevent bugs from showing up 3. Automation for majority of the test cases (except some E2E and some UI tests) 4. Active participation in Spec review, Design Review, Code Reviews, etc. 5. Regression test cases (90-95% automated) ran everyday 6. Contribute to tendencial reduction in the cycle time 7. Reduction in Dev:Test ratio year over year 8. Implementation of Model-Based-Testing 9. All aspects of testing (Accessibility, Globalization, Performance, Scale, Security, etc.) done on time
27. FROM SOFTWARE TESTING 2.0 TO SOFTWARE TESTING 3.0 Face New Challenges emerging from the sophisticated and interconnected environment of the computing world. Build a comprehensive Software Testing Program aiming at testing as a whole rather than in parts (feature reliability, usability, interoperability, performance acceptability, scalability, security, accessibility…) © 2011 Neotys - All Rights Reserved. 27 Expected benefits: • improves quality by increasing testing coverage and effectiveness • decreases testing time • decreases costs • provides the necessary information for informed decision making Expected outcomes: - tested, high quality software - reusable software tests that codify organization knowledge about the software under test and provide the basis for the testing of future generations of the software • focuses on test design and agile test development practices, and • extends test automation to multiple platforms including non-GUI platforms
28. MAPPING TESTING WITH SDLC: PERFORMANCE ENGINEERING Get things right the first time Low Impact & Cost of Change early in the cycle Minimize re-work required to meeting NFR's down the line © 2011 Neotys - All Rights Reserved. 28 Prevent things from going pear shaped and band aid fixes in production Strengthen & enhance the relationship with your client Focus on the things that are important and prevent fire fighting Spend wisely and optimize ROI
29. “TAAT”: TESTING-AS-A-TEAM! • Spec Writers Help Testers • Testers Help Spec Writers Specs Test Models © 2011 Neotys - All Rights Reserved. 29 • Testers Help Developers • Testers Help Testers Code Overall Quality
30. 2. PEOPLE: THE ROLE OF THE TESTER Boundaries Blur Between Testing and Development Testing mission becomes to make the development process faster and to complete products and deliverable faster. A new objective for the tester To ensure the stability of the product throughout the development process: © 2011 Neotys - All Rights Reserved. 30 process: Testers Get Respect A new title, even: “Software Design Engineer in Test” Testing Gets Trendy Breaking stuff is at least as much fun as building it! Adrenaline Junkies Move On Away from late-night, last-minute, firefighting sessions
31. THE ROLE OF THE TESTER IN THE EXTENDED ENTERPRISE A new role? Ambassadors of the development to external teams. Leveraging requirements for Testers: connection to the “business aspects” of the process, © 2011 Neotys - All Rights Reserved. 31 communication, awareness of all external factors that may influence the product and the team. Coordinating the communication with other teams (product, support, other dev teams, customers, suppliers, etc)
32. 3. METRICS FOR MANAGEMENT Not only deliver traditional software quality metrics focused on the product, process and projects (# of developers, skill levels, etc.).. But also metrics to measure and validate the end customer experience. Effectively measure the added-value of testing and clearly communicate this to business stakeholders. © 2011 Neotys - All Rights Reserved. 32 communicate this to business stakeholders. Important customer performance metrics across the Software Delivery Life Cycle should be identified, documented and tracked. A key factor for customer satisfaction.. And ROI!
33. DEVIL IS IN THE DETAIL.. Great strategies are those surviving implementation © 2011 Neotys - All Rights Reserved. 33 Louis XIVth by Bernini 17th Century
34. WHAT IT WILL TAKE TO SUCCEED: REALITY CHECK! Understand what may be realistically accomplished with testing vs what cannot.. Build a team of software testers that can perform the different types of testing. Find the RIGHT people: right EDUCATION program out there? Right understanding of SDLC and can LEARN to understand the purpose of the application to be tested. It has become a competitive imperative! © 2011 Neotys - All Rights Reserved. 34 Change the ROLES Build and document robust and comprehensive software testing programs Integrate feedback from customers with Testing Foster a culture of RESPECT & COLLABORATION Source: World Quality Report 2012-2013
35. George Catlin, 19th Century
39. TESTING-AS-A-TEAM © 2011 Neotys - All Rights Reserved. 39 Franz Hals The Meagre Company, 17th Century
40. TESTING-AS-A-TEAM © 2011 Neotys - All Rights Reserved. 40 Franz Hals The regents of St-Elizabeth’s hospital , 17th Century
41. MANAGEMENT IMPLICATIONS © 2011 Neotys - All Rights Reserved. 41
44. E3.0 MEETS SOFTWARE TESTING 3.0 Rapidly evolving IT landscape calls for Software Testing 3.0 implementation: greater influence over SDLC, alignment with business needs New operating models emerge: Cloud-based models to achieve unprecedented © 2011 Neotys - All Rights Reserved. 44 Cloud-based models to achieve unprecedented effectiveness and productivity Testing-As-A-Service, as a complement to Testing COEs Mobility back on the QA priority list: Only 31% of WQR respondents test mobile applications Performance is the key to success over functionality and security Source: World Quality Report 2012-2013
45. WITH NEOTYS, FUTURE IS HERE ALREADY Harnessing the power of the cloud: Testing in production: hybrid campaigns © 2011 Neotys - All Rights Reserved. 45 High-scale tests – Cost effective model (pay per use) – Global coverage
46. WITH NEOTYS, FUTURE IS HERE ALREADY © 2011 Neotys - All Rights Reserved. 46 Performance testing for mobile applications Simulates a lot of concurrent mobile users Ensures the server won’t have problems under load Ensures that all your users will get acceptable response times Deploy with confidence!
47. OUR VISION We, at Neotys, aim at eradicating e-failure, wherever it is simply © 2011 Neotys - All Rights Reserved. 47 e-failure, wherever it is simply not an option.
48. PREPARE FOR THE FUTURE TODAY 1. Get Actively Dissatisfied Don't accept the current state of testing. Look around and think, "What are we doing that makes no sense?" 2. Push the Envelope Figure out how to test better and share that knowledge. 3. Learn More about Testing The industry is exploding with innovative software testing ideas. See what is © 2011 Neotys - All Rights Reserved. 48 The industry is exploding with innovative software testing ideas. See what is happening on the cutting edge of testing. 4. Learn More about Development Take a programming class as a reconnaissance flight over “bug territory”. 5. Change the World! "The best way to predict the future is to invent it.“ PC pioneer Alan Kay
49. THANK YOU! laurent.gaudy@neotys.com © 2011 Neotys - All Rights Reserved. 49
50. ABSTRACT The modern enterprise is no longer a single, isolated, monolithic organization. Customers, Partners, Suppliers, Outsourcers, Distributors, Resellers, … all kinds of entities extend and expand the boundaries of the enterprise, and make “collaboration” and “sharing” important. This new reality has been dubbed “Enterprise 3.0”. Testing should be based on constant dialogue and cooperation with the different stakeholders in the production, development and commercialization of higher quality software. Tomorrow, testing will not © 2011 Neotys - All Rights Reserved. 50 commercialization of higher quality software. Tomorrow, testing will not only suggest the necessary fine tuning of the application to the peculiar environment of the customer but, beyond that, the inevitable adaptations incumbent upon the customer to accommodate change and innovation. The object of the talk is to envision how testing will lead to ever more demanding but productive communication and dialogue between developers, networks, customers and various experts or consultants in the different fields.. Thus in the diversification of its relevant services to firms, “Software testing 3.0” will meet, point after point, the challenge of “ Enterprise 3.0”...
51. WITH NEOTYS, FUTURE IS HERE ALREADY Neotys ISV Partner Program For Software Development organizations : Optimize deployments and excel at customer satisfaction A product with higher quality means a reduced need for © 2011 Neotys - All Rights Reserved. 51 A product with higher quality means a reduced need for Support Staff there by bringing a greater saving to the business. Safeguarding quality ensures the total cost of ownership is reduced A well tested product is a better investment.