Testing to reduce support calls

For years we have been talking about testing to improve the quality of applications, but there are still many support calls coming in and bug reports submitted. Is it because our code is faulty and our tests are not covering the right components?

The answer lies in the perception of users and customers of the product which differs significantly from what developers and managers believe the user wants or needs. Welcome to user experience testing! In this talk I'll highlight some of the steps we took to address the ever growing user dissatisfaction of the software we thought was awesome and well tested.

Transcript

1. TESTING TO REDUCE SUPPORT CALLS Michelangelo van Dam
2. MICHELANGELO VAN DAM I'm a senior #php architect, co-founder and #ceo of @in2itvof, #community leader at @phpbenelux, coach at @CoderDojoBelgium, #MVP, #digitalnomad, likes #coffee. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
3. I’M ACTIVELY INVOLVED IN THE COMMUNITY … DevOps
 Community Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
4. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
5. To be successful at testing you need to be able to look beyond the walls of the imaginary room! Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
6. WHY TESTING? ➤ Ensure business/regulatory requirements are met ➤ Prevent (serious) issues when used by customers ➤ Protect against loss of revenue or reputational damage ➤ Gain competitive advantage ➤ Receive trust and respect Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
7. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
8. HOW DO I START? Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
9. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
10. Even though F5 testing is easy to do, please adopt a more robust form of testing that can be shared with your team members Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
11. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
12. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
13. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
14. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
15. Be mindful that automated acceptance tests are sensitive for changes in layout of your application! Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
16. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
17. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
18. Feature: Meetup group has events In order to participate in an event As a participant A meetup group must have public events Scenario: Search for events of a user group Given the meetup group "GroningenPHP" exists When I search for meetup group "GroningenPHP" Then I should see at least "1" event listed Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
19. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
20. Feature: Meetup group has events In order to participate in an event As a participant A meetup group must have public events Scenario: Search for events of a user group Given the meetup group "GroningenFoo" exists When I search for meetup group "GroningenFoo" Then I should see at least "1" event listed Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
21. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
22. Behat tests that business requirements are met, not that correct logic is implemented! Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
23. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
24. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
25. LEGACY APPLICATIONS Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
26. If you measure code coverage, ensure that the core of your application is 100% covered with tests, even though it only is 1% of the whole code base! Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
27. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
28. USER EXPERIENCE TESTING Keep in touch with your users/customers Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
29. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
30. UNDERSTAND THE “BUSINESS VALUE” Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
31. WRITE OUT THE REQUIREMENTS Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
32. DRAW OR USE WIREFRAMES Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
33. START WITH A POC Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
34. CODE IN PROD FROM DAY 0 Use feature toggles to perform A/B testing immediately Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
35. GIVE USERS A WAY TO PROVIDE FEEDBACK EASILY Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
36. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
37. KEEP DEVELOPMENT CYCLES SHORT AND AGILE Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
38. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe) Agile Lean Kanban
39. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
40. PROTIP: KEEP TRACK OF… Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
41. OTHER FORMS OF TESTING Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
42. PERFORMANCE TESTING Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
43. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
44. SECURITY TESTING Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
45. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe) Sqlmap®
46. RESILIENCE TESTING Chaos Engineering Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
47. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
48. MANUAL TESTING Not all things can be automated Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
49. EXPLORATORY TESTING Looking at the application from every angle to discover flaws in functionality, design or behaviour. A.K.A. BlackBox Testing Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
50. PENETRATION TESTING Finding and exploiting weaknesses, flaws or poor configurations in your application to gain privileged access or syphon data out of your business. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
51. LEGAL TESTING & AUDITING In regulatory industries, legal testing and auditing is a must Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
52. Use the right tool for your testing requirements. Nothing worse than to waste time configuring a testing tool that provides no value! Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
53. AUTOMATED TESTING? Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
54. AUTOMATE WHAT’S POSSIBLE ➤ If it’s repetitive, automate it ➤ Prevents someone’s “bad day” to cause issues ➤ Helps when under stress or pressure ➤ Provides documentation and insights to everyone ➤ Can be fine-tuned over time Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
55. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
56. If I have to do the same thing more than once, I automate it! Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
57. WILL IT PREVENT FAILURE? Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
58. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
59. WHEN FAILURE DOES HAPPEN… FAILURE RESOLUTION REVIEW REMEDIATION ANALYSIS BEST PRACTICES AUTOMATION ADOPTION Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe) Credits: Josh Evans, Mastering Chaos at Netflix
60. WILL THIS REDUCE SUPPORT CALLS? Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
61. LET’S SUMMARISE ➤ We test functional requirements with Selenium ➤ We test business requirements with Behat ➤ We test code requirements with unit and mutation tests ➤ We let our target audience test our work early ➤ We test performance, security and resilience of our application ➤ We run all this in a continuous integration tool ➤ And we have regular manual exploratory and penetration tests ➤ We understand failures can still occur, but we address them as a team and adopt a culture of continuous learning & automation Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
62. Bring tests closer to the point where code is created and you already win time and money! SHIFT LEFT Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
63. WILL THIS REDUCE SUPPORT CALLS? YES Between 40% and 65% less support calls or bug reports* Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe) (*) Review of in2it customers in 2018
64. RESOURCES 1/3 ➤ PHP User Groups: php.ug ➤ Ministry of Testing: ministryoftesting.com ➤ Belgium DevOps Community: meetup.com/Devops-Belgium ➤ Belgium Cyber Security Coalition: cybersecuritycoalition.be ➤ Azure User Group Belgium: azug.be ➤ Global Open Source Initiative: opensource.org ➤ Selenium: seleniumhq.org ➤ Katalon Studio: katalon.com ➤ SauceLabs: saucelabs.com ➤ Behat: behat.org ➤ PHPUnit: phpunit.de ➤ Infection: infection.github.io ➤ Usabilla: usabilla.com ➤ Apache Bench: httpd.apache.org ➤ Apache JMeter: jmeter.apache.org ➤ Joe Dog Siege: joedog.org/siege-home ➤ MySQL Slap: dev.mysql.com Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
65. RESOURCES 2/3 ➤ NeoLoad: neotys.com/neoload ➤ LoadNinja: loadninja.com ➤ OWASP Zed Attack Proxy: owasp.org ➤ Vega Vulnerability Scanner: subgraph.com/ vega ➤ Burp Suite: portswigger.net/burp ➤ SonarQube: sonarqube.org ➤ RIPS Tech: ripstech.com ➤ SQLMap: sqlmap.org ➤ Arachni: arachni-scanner.com ➤ Netflix Chaos Monkey: techblog.netflix.com ➤ Jenkins: jenkins.io ➤ JetBrains TeamCity: jetbrains.com/ teamcity ➤ Atlassian Bamboo: atlassian.com/ software/bamboo ➤ GitHub: github.com ➤ GitLab: gitlab.com ➤ Atlassian BitBucket: bitbucket.org ➤ Phabricator: phacility.com/phabricator Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
66. RESOURCES 3/3 ➤ Microsoft Azure: azure.microsoft.com ➤ Amazon Web Services: aws.amazon.com ➤ Google Cloud Platform: cloud.google.com ➤ Travis CI: travis-ci.org ➤ CodeShip: codeship.com ➤ Semaphore: semaphoreci.com ➤ Buddy: buddy.works ➤ Go CD: gocd.org ➤ NeverCode: nevercode.io ➤ Strider CD: strider-cd.github.io Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
67. Improving quality one test at a $me Slides available at in2.se/reduce-support-calls Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)