$30 off During Our Annual Pro Sale. View Details »

Test Dive conference: Cognitive testing in shift-left mindset

Test Dive conference: Cognitive testing in shift-left mindset

Aleksandra Kornecka

October 18, 2018

More Decks by Aleksandra Kornecka

Other Decks in Technology


  1. Cognitive testing in shift-left mindset Aleksandra Kornecka OLX Group Test

    Dive conference, 18.10.2018 Kraków, Poland
  2. whoami • software quality assurance engineer • MSc. in cognitive

    science • sprinter
  3. Agenda 1. Quality in SDLC. 2. Shift-left approach. 3. Case

    studies. 4. Cognitive testing approach. 5. Science behind. 6. Tools, tips & tricks. 7. Summary.
  4. Quality in Software Development Life Cycle The true story

  5. Software quality in general • technical & business values •

    dimensions like functionality, reliability, usability, measurability, maintainability, -other abilities • standards like ISO, IEEE, GDPR (...)
  6. SDLC

  7. Where the quality starts for you?

  8. Where the quality starts for you?

  9. Where the quality starts for you?

  10. Where the quality starts for you?

  11. The quality never starts and never ends

  12. … we should care all the time ...

  13. … so it’s time to SHIFT-LEFT!

  14. Shift-left approach explained

  15. “Shift-left” is a buzz word, but ...

  16. ..but the stats prove some benefits

  17. Enable awareness & attention early

  18. Shift-left approach definition Integrating the quality assurance practices and testing

    with entire software development life cycle: - from the very beginning e.g. requirements testing, prototype testing etc. - into every role e.g. developer, product owner, tester
  19. Source: webinar by Experitest

  20. Source: webinar by Experitest

  21. Other shift-left approach factors • test the requirements, both technical

    and business • work closely with stakeholders, validate ideas • research often (A/B testing, users insights, tracked data) • no sanctuary code, refactor on daily basis • consider test automation as business value • automate monitoring and alerting to keep focus into goal
  22. Case studies shift-left by example

  23. Source: webinar by Experitest

  24. Anti-pattern: never ending business request - one feature developed by

    3 teams - requirements verification in the very end of development - unwanted feature from developer invention due to lack of business communication - 2 days of delay for 7 days marketing campaign Lesson learned: communicate with business teams before coding, create common calendar, update each other, squash branches NOT OK: shift left next time!
  25. Success story: migrating to cloud case 4 countries, 2 months,

    5 DevOps, server > cloud - after 1st failure: recovery plan, checklists, backup scenario - management supporting the migration team - active email, chat and oral communication - pair-programming to teach the new technology stack Good practice: requirements testing, learn by experience, create basic checklists, communicate with business stakeholders not only tech team OK: attention to quality shifted left on time
  26. Cognitive testing approach explained

  27. “Cognitive” means ... • respecting human mind and brain capabilities

    & limitations • respecting sensual processes like seeing, hearing, touching • respecting various interfaces (haptic/tactile, visual, brain) • dependent to perception
  28. Cognitive approach goal = human-friendly & working software humans =

    testers, developers, end-users (...)
  29. Perceptual limitations • we are humans • we make mistakes,

    we miss things We cannot see the entire site despite the web browser sees it all the time.
  30. Biases-prone

  31. None
  32. Cognitive testing approach manifesto • Developers, testers, QAs, business stakeholders,

    end-users are humans. • Humans have some perceptual limitations we should mind when developing software for humans. • There are cultural differences between human using software.
  33. Science behind cognitive approach

  34. Attention span Source: “Attention spans. Consumer Insights.” Report by Microsoft

    Canada, 2015
  35. Working memory capacity • just 2+-7 information items at once

    in 15-30 seconds duration (Miller’s law)
  36. Change blindness Strong focusing makes us blind to changes. Strong

    focusing on testing one feature can make us blind to bugs in another feature.
  37. Attention gap Brain constructs the view step by step. (Marr’s

  38. Schemas reasoning (Gestalt theory laws) Brain is working on schemas.

  39. Schemas reasoning (Gestalt laws) Law of proximity

  40. Schemas reasoning (Gestalt laws) Law of similarity

  41. Schemas reasoning (Gestalt laws) Law of common fate: drag-and-drop interaction

  42. Schemas reasoning (Gestalt laws) Law of symmetry

  43. Schemas reasoning (Gestalt laws) Law of Prägnanz (figure-ground)

  44. Gibson affordances Where the bug can be?

  45. Tools, tips & tricks for daily use

  46. Awareness is a key

  47. Heuristics by Jakob Nielsen 47

  48. Definition of Ready, Definition of Done

  49. Incident analysis

  50. Real-time monitoring • distributed communication • health-checks in releases (CI,

    versioning) • real-time monitoring (e.g. DataDog, New Relic, Fabric) • incident reports
  51. Summary

  52. Cognitive + shift left = future quality manifesto • we

    are humans, we need to communicate • we craft the software, you contribute to the quality, no matter what role you have • we want other humans to use our software • quality is a continuous thing • the earlier you shift attention to the quality, the better for you
  53. Inspirations • cognitive science, cognitive psychology • clinical psychology research

    (EEG, fMRI, ERP, experiments) • user experience research • information architecture • anthropology • personal experiences of mates in software development :-)
  54. Thank you for attention! aleksandrakornecka.com girlswhotest.pl joinOLX.com