Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Test Dive conference: Cognitive testing in shift-left mindset

Test Dive conference: Cognitive testing in shift-left mindset

Aleksandra Kornecka

October 18, 2018

More Decks by Aleksandra Kornecka

Other Decks in Technology


  1. Agenda 1. Quality in SDLC. 2. Shift-left approach. 3. Case

    studies. 4. Cognitive testing approach. 5. Science behind. 6. Tools, tips & tricks. 7. Summary.
  2. Software quality in general • technical & business values •

    dimensions like functionality, reliability, usability, measurability, maintainability, -other abilities • standards like ISO, IEEE, GDPR (...)
  3. Shift-left approach definition Integrating the quality assurance practices and testing

    with entire software development life cycle: - from the very beginning e.g. requirements testing, prototype testing etc. - into every role e.g. developer, product owner, tester
  4. Other shift-left approach factors • test the requirements, both technical

    and business • work closely with stakeholders, validate ideas • research often (A/B testing, users insights, tracked data) • no sanctuary code, refactor on daily basis • consider test automation as business value • automate monitoring and alerting to keep focus into goal
  5. Anti-pattern: never ending business request - one feature developed by

    3 teams - requirements verification in the very end of development - unwanted feature from developer invention due to lack of business communication - 2 days of delay for 7 days marketing campaign Lesson learned: communicate with business teams before coding, create common calendar, update each other, squash branches NOT OK: shift left next time!
  6. Success story: migrating to cloud case 4 countries, 2 months,

    5 DevOps, server > cloud - after 1st failure: recovery plan, checklists, backup scenario - management supporting the migration team - active email, chat and oral communication - pair-programming to teach the new technology stack Good practice: requirements testing, learn by experience, create basic checklists, communicate with business stakeholders not only tech team OK: attention to quality shifted left on time
  7. “Cognitive” means ... • respecting human mind and brain capabilities

    & limitations • respecting sensual processes like seeing, hearing, touching • respecting various interfaces (haptic/tactile, visual, brain) • dependent to perception
  8. Perceptual limitations • we are humans • we make mistakes,

    we miss things We cannot see the entire site despite the web browser sees it all the time.
  9. Cognitive testing approach manifesto • Developers, testers, QAs, business stakeholders,

    end-users are humans. • Humans have some perceptual limitations we should mind when developing software for humans. • There are cultural differences between human using software.
  10. Working memory capacity • just 2+-7 information items at once

    in 15-30 seconds duration (Miller’s law)
  11. Change blindness Strong focusing makes us blind to changes. Strong

    focusing on testing one feature can make us blind to bugs in another feature.
  12. Real-time monitoring • distributed communication • health-checks in releases (CI,

    versioning) • real-time monitoring (e.g. DataDog, New Relic, Fabric) • incident reports
  13. Cognitive + shift left = future quality manifesto • we

    are humans, we need to communicate • we craft the software, you contribute to the quality, no matter what role you have • we want other humans to use our software • quality is a continuous thing • the earlier you shift attention to the quality, the better for you
  14. Inspirations • cognitive science, cognitive psychology • clinical psychology research

    (EEG, fMRI, ERP, experiments) • user experience research • information architecture • anthropology • personal experiences of mates in software development :-)