How to Save Time and Money with Automated Testing

Learn how a consistent and sustainable automated testing process can not only help gain efficiencies and reduce labor costs in your release cycle, but ensure a quality deployment by effectively mitigating risk.

http://www.youtube.com/watch?v=7jr1fou8G5c

Transcript

1. How to Save Time & Money with Automated Testing Gerlinde Weiglein , HP; Testing Automation Program Lead Vaughn Paladin, Arcsona; Co-founder and CEO Greg Wester, salesforce.com ; Quality Engineering Lead Debbie Guy, salesforce.com ; Sr. Principal Cloud Success Technologist Randy Goh, salesforce.com; Sr. Cloud Success Technologist
2. Safe harbor Safe harbor statement under the Private Securities Litigation Reform Act of 1995: This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services. The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site. Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.
3. Agenda •Intro •Getting Started with Test Automation • Greg Wester, salesforce.com •Testing Automation a Real Time Saver: The HP Story • Gerlinde Weiglein, HP and Vaughn Paladin, Arcsona •Q&A
4. What is Automated Testing? Test automation is the use of programmatic tools to run a set of controlled commands with a predictable outcome that is easily repeatable over time with the same predictable outcomes during each release cycle.
5. What percentage of companies are doing manual testing today? 60% Worksoft Market Research Report: 2013 Trends in Automated Testing For Enterprise Systems
6. What percentage of companies will invest in automated testing within the next year? 34.3% Worksoft Market Research Report: 2013 Trends in Automated Testing For Enterprise Systems
7. Example of how much you can reduce costs $78 <$18 SkyIT Group whitepaper: Automated testing ROI: fact or fiction?
8. Post your questions to the session: http://bit.ly/1aqk2XZ @Randy Goh @Debbie Guy Let’s Make This Interactive – Start Posting Your Questions
9. Greg Wester lead engineer, salesforce.com @gwestr
10. Quality Engineering 101 ▪ Version control • tests and application code versioned together ▪ Commit code minimum 1x per day ▪ Continuous, remote builds of check-ins ▪ Automated validation of builds ▪ No broken builds - ever
11. Portfolio of Automation ▪ Balance of unit and functional tests ▪ Diversify tools and tests ▪ Measure by coverage; not test counts
12. Exemplary Application ▪ 500,000 lines of code ▪ 10,000 branches (cyclomatic complexity) ▪ 9,000 unit tests ▪ 500 functional tests (API tests) ▪ 100 UI tests
13. cloudevent.secure.force.com Demo
14. /* * https://github.com/gwestersfdc/WebDriver-Sample */ public class DreamforceTest { private final WebDriver driver; @Test public void RegistrationTest() { LoginPage page = new LoginPage(driver); page.doLogin(username, password); assertTrue(driver.getTitle().equals(“DF13”)); } }
15. Business Value ▪ Cost of finding a defect in production ▪ Engineering velocity ▪ Continuously releasable
16. Pro Tips ▪ Long running tests are of low business value ▪ Unit testing allows for refactoring ▪ WebDriver is the standard for browser testing ▪ Continuous delivery is not continuous deployment ▪ When code is deleted, tests should fail
17. Gerlinde Weiglein HP Sales Process & Platform Capabilities @GWeiglein Vaughn Paladin CIO Arcsona @twitter
18. Session Outline The objective of this session is to share ….. ▪ HP’s regression and release testing approach • Improving quality and reduce risk of highly integrated Salesforce implementations • Testing strategy and future plans • Business and IT collaboration ▪ Testing automation Demo • Showcase (recorded) ▪ Q&A
19. Accelerating Regression Testing Through Automation • Sales Transformation in HP • Short, Mid and Long Term Goals • Demo • Testing Automation Project • The Transformation Working Landscape • Current vs. Future Testing Methodology
20. Testing Automation Project ❑ Delivers high quality outcome ❑ Supports HP’s Sales Transformation Program ❑ Improves collaboration between Business and IT, Operations and the Business Groups ❑ Mitigates risks aligning across multiple organizations, programs and timelines ❑ Reduces time to build, execute, and maintain the regression and E2E test suites ❑ Scales down on manual testing and expensive resource involvement ❑ Increases testing coverage across all capabilities and systems
21. Test Phases in HP Application Lifecycle Management CRP ‘0’ SIT & UAT MTP Build CRP Design CRP Scope CRP PRE-UAT (User Acceptance Testing) System Integration Testing User Acceptance Testing Regression Testing & E2E S M O K E Test Toda y Automated testing HP ALM & ARCSONA Futur e MTP & Deployment; Warranty & Hotfixes IT Biz Biz Biz BIZ & IT IT & Biz IT - automated
22. Projected timeline for comparison Current & Future Methodology ROI within 6 test cycles HP ALM Manual testing Manual test script creation Manual runtime Defect analysis Script modification HP ALM & QTP Standard automation Script automation Automated runtime Defect analysis Script modification Time saving Past & today ROI within 2 test cycles Today & go forward strategy HP ALM & ARCSONA Accelerator Automate compone nts Creat e scripts Runti me Defect analysi s Modi fy Time saving
23. Reducing the Time, Resources and Cost of Testing # of Test Cycles Aggregated Effort in hours Manual Record & Replay HP ALM & Arcsona Accelerator Record & Replay ROI within 6 Test Cycles ROI within 2 Test Cycles
24. Goals Short Term •Automate regression test scripts from previous releases and applicable E2E scenarios Mid Term •In addition to short term goal testing automation for newly onboarded projects including E2E scenarios Long Term •Automation of all testing for regression, E2E and new functionality
25. A day in the life of a UAT tester....... Start with a review of the components and tests that are already created. Show the list of components Enter 1 or two components and show how they are built Show the list of tests Show how the tests are created by components Execute a test quickly Should show the system bouncing between 2 applications Test should fail and create a bug Show the execution of the test Show the bug and screenshot Show all of the history of the executions
26. Greg Wester Quality Engineering Lead salesforce.com @gwestr Gerlinde Weiglein Vaughn Paladin Testing Automation Program Lead HP @GWeiglein Co-founder and CEO Arcsona Post your questions to the session: http://bit.ly/1aqk2XZ