$30 off During Our Annual Pro Sale. View Details »

Testing to reduce support calls

Testing to reduce support calls

For years we have been talking about testing to improve the quality of applications, but there are still many support calls coming in and bug reports submitted. Is it because our code is faulty and our tests are not covering the right components?

The answer lies in the perception of users and customers of the product which differs significantly from what developers and managers believe the user wants or needs. Welcome to user experience testing! In this talk I'll highlight some of the steps we took to address the ever growing user dissatisfaction of the software we thought was awesome and well tested.

DragonBe

July 04, 2019
Tweet

More Decks by DragonBe

Other Decks in Programming

Transcript

  1. TESTING TO REDUCE SUPPORT CALLS
    Michelangelo van Dam

    View Slide

  2. MICHELANGELO VAN DAM
    I'm a senior #php architect, co-founder and #ceo of @in2itvof, #community leader
    at @phpbenelux, coach at @CoderDojoBelgium, #MVP, #digitalnomad,
    likes #coffee.
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  3. I’M ACTIVELY INVOLVED IN THE COMMUNITY …
    DevOps

    Community
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  4. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  5. To be successful at testing you need to
    be able to look beyond the walls of the
    imaginary room!
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  6. WHY TESTING?
    ➤ Ensure business/regulatory requirements are met
    ➤ Prevent (serious) issues when used by customers
    ➤ Protect against loss of revenue or reputational damage
    ➤ Gain competitive advantage
    ➤ Receive trust and respect
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  7. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  8. HOW DO I START?
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  9. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  10. Even though F5 testing is easy to
    do, please adopt a more robust form of
    testing that can be shared with your
    team members
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  11. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  12. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  13. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  14. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  15. Be mindful that automated
    acceptance tests are sensitive for
    changes in layout of your application!
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  16. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  17. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  18. Feature: Meetup group has events
    In order to participate in an event
    As a participant
    A meetup group must have public events
    Scenario: Search for events of a user group
    Given the meetup group "GroningenPHP" exists
    When I search for meetup group "GroningenPHP"
    Then I should see at least "1" event listed
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  19. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  20. Feature: Meetup group has events
    In order to participate in an event
    As a participant
    A meetup group must have public events
    Scenario: Search for events of a user group
    Given the meetup group "GroningenFoo" exists
    When I search for meetup group "GroningenFoo"
    Then I should see at least "1" event listed
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  21. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  22. Behat tests that business
    requirements are met,
    not that correct logic is implemented!
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  23. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  24. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  25. LEGACY APPLICATIONS
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  26. If you measure code coverage,
    ensure that the core of your application
    is 100% covered with tests, even though
    it only is 1% of the whole code
    base!
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  27. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  28. USER EXPERIENCE TESTING
    Keep in touch with your users/customers
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  29. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  30. UNDERSTAND THE
    “BUSINESS VALUE”
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  31. WRITE OUT THE REQUIREMENTS
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  32. DRAW OR USE WIREFRAMES
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  33. START WITH A POC
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  34. CODE IN PROD FROM DAY 0
    Use feature toggles to perform A/B testing immediately
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  35. GIVE USERS A WAY TO PROVIDE
    FEEDBACK EASILY
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  36. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  37. KEEP DEVELOPMENT CYCLES
    SHORT AND AGILE
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  38. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
    Agile Lean Kanban

    View Slide

  39. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  40. PROTIP: KEEP TRACK OF…
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  41. OTHER FORMS OF
    TESTING
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  42. PERFORMANCE TESTING
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  43. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  44. SECURITY TESTING
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  45. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
    Sqlmap®

    View Slide

  46. RESILIENCE TESTING
    Chaos Engineering
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  47. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  48. MANUAL TESTING
    Not all things can be automated
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  49. EXPLORATORY TESTING
    Looking at the application from every angle to discover flaws in
    functionality, design or behaviour.
    A.K.A. BlackBox Testing
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  50. PENETRATION TESTING
    Finding and exploiting weaknesses, flaws or poor configurations
    in your application to gain privileged access or syphon data out
    of your business.
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  51. LEGAL TESTING & AUDITING
    In regulatory industries, legal testing and auditing is a must
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  52. Use the right tool for your testing
    requirements. Nothing worse than to
    waste time configuring a testing tool
    that provides no value!
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  53. AUTOMATED TESTING?
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  54. AUTOMATE WHAT’S POSSIBLE
    ➤ If it’s repetitive, automate it
    ➤ Prevents someone’s “bad day” to cause issues
    ➤ Helps when under stress or pressure
    ➤ Provides documentation and insights to everyone
    ➤ Can be fine-tuned over time
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  55. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  56. If I have to do the same thing more than
    once, I automate it!
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  57. WILL IT PREVENT FAILURE?
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  58. Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  59. WHEN FAILURE DOES HAPPEN…
    FAILURE
    RESOLUTION
    REVIEW
    REMEDIATION
    ANALYSIS
    BEST PRACTICES
    AUTOMATION
    ADOPTION
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
    Credits: Josh Evans, Mastering Chaos at Netflix

    View Slide

  60. WILL THIS REDUCE
    SUPPORT CALLS?
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  61. LET’S SUMMARISE
    ➤ We test functional requirements with Selenium
    ➤ We test business requirements with Behat
    ➤ We test code requirements with unit and mutation tests
    ➤ We let our target audience test our work early
    ➤ We test performance, security and resilience of our application
    ➤ We run all this in a continuous integration tool
    ➤ And we have regular manual exploratory and penetration tests
    ➤ We understand failures can still occur, but we address them as a
    team and adopt a culture of continuous learning & automation
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  62. Bring tests closer to the point
    where code is created and you
    already win time and money!
    SHIFT LEFT
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  63. WILL THIS REDUCE
    SUPPORT CALLS?
    YES
    Between 40% and 65% less
    support calls or bug reports*
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)
    (*) Review of in2it customers in 2018

    View Slide

  64. RESOURCES 1/3
    ➤ PHP User Groups: php.ug
    ➤ Ministry of Testing:
    ministryoftesting.com
    ➤ Belgium DevOps Community:
    meetup.com/Devops-Belgium
    ➤ Belgium Cyber Security Coalition:
    cybersecuritycoalition.be
    ➤ Azure User Group Belgium: azug.be
    ➤ Global Open Source Initiative:
    opensource.org
    ➤ Selenium: seleniumhq.org
    ➤ Katalon Studio: katalon.com
    ➤ SauceLabs: saucelabs.com
    ➤ Behat: behat.org
    ➤ PHPUnit: phpunit.de
    ➤ Infection: infection.github.io
    ➤ Usabilla: usabilla.com
    ➤ Apache Bench: httpd.apache.org
    ➤ Apache JMeter: jmeter.apache.org
    ➤ Joe Dog Siege: joedog.org/siege-home
    ➤ MySQL Slap: dev.mysql.com
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  65. RESOURCES 2/3
    ➤ NeoLoad: neotys.com/neoload
    ➤ LoadNinja: loadninja.com
    ➤ OWASP Zed Attack Proxy: owasp.org
    ➤ Vega Vulnerability Scanner: subgraph.com/
    vega
    ➤ Burp Suite: portswigger.net/burp
    ➤ SonarQube: sonarqube.org
    ➤ RIPS Tech: ripstech.com
    ➤ SQLMap: sqlmap.org
    ➤ Arachni: arachni-scanner.com
    ➤ Netflix Chaos Monkey:
    techblog.netflix.com
    ➤ Jenkins: jenkins.io
    ➤ JetBrains TeamCity: jetbrains.com/
    teamcity
    ➤ Atlassian Bamboo: atlassian.com/
    software/bamboo
    ➤ GitHub: github.com
    ➤ GitLab: gitlab.com
    ➤ Atlassian BitBucket: bitbucket.org
    ➤ Phabricator: phacility.com/phabricator
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  66. RESOURCES 3/3
    ➤ Microsoft Azure: azure.microsoft.com
    ➤ Amazon Web Services:
    aws.amazon.com
    ➤ Google Cloud Platform:
    cloud.google.com
    ➤ Travis CI: travis-ci.org
    ➤ CodeShip: codeship.com
    ➤ Semaphore: semaphoreci.com
    ➤ Buddy: buddy.works
    ➤ Go CD: gocd.org
    ➤ NeverCode: nevercode.io
    ➤ Strider CD: strider-cd.github.io
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide

  67. Improving quality one test at a $me
    Slides available at
    in2.se/reduce-support-calls
    Tes$ng to reduce support calls by Michelangelo van Dam (@DragonBe)

    View Slide