Test Dive conference: Cognitive testing in shift-left mindset

http://www.testdive.pl/program
https://www.facebook.com/TestDiveConference/
https://www.facebook.com/rdkrakow/

Transcript

1. Cognitive testing in shift-left mindset Aleksandra Kornecka OLX Group Test Dive conference, 18.10.2018 Kraków, Poland
2. whoami • software quality assurance engineer • MSc. in cognitive science • sprinter
3. Agenda 1. Quality in SDLC. 2. Shift-left approach. 3. Case studies. 4. Cognitive testing approach. 5. Science behind. 6. Tools, tips & tricks. 7. Summary.
4. Quality in Software Development Life Cycle The true story
5. Software quality in general • technical & business values • dimensions like functionality, reliability, usability, measurability, maintainability, -other abilities • standards like ISO, IEEE, GDPR (...)
6. SDLC
7. Where the quality starts for you?
8. Where the quality starts for you?
9. Where the quality starts for you?
10. Where the quality starts for you?
11. The quality never starts and never ends
12. … we should care all the time ...
13. … so it’s time to SHIFT-LEFT!
14. Shift-left approach explained
15. “Shift-left” is a buzz word, but ...
16. ..but the stats prove some benefits
17. Enable awareness & attention early
18. Shift-left approach definition Integrating the quality assurance practices and testing with entire software development life cycle: - from the very beginning e.g. requirements testing, prototype testing etc. - into every role e.g. developer, product owner, tester
19. Source: webinar by Experitest
20. Source: webinar by Experitest
21. Other shift-left approach factors • test the requirements, both technical and business • work closely with stakeholders, validate ideas • research often (A/B testing, users insights, tracked data) • no sanctuary code, refactor on daily basis • consider test automation as business value • automate monitoring and alerting to keep focus into goal
22. Case studies shift-left by example
23. Source: webinar by Experitest
24. Anti-pattern: never ending business request - one feature developed by 3 teams - requirements verification in the very end of development - unwanted feature from developer invention due to lack of business communication - 2 days of delay for 7 days marketing campaign Lesson learned: communicate with business teams before coding, create common calendar, update each other, squash branches NOT OK: shift left next time!
25. Success story: migrating to cloud case 4 countries, 2 months, 5 DevOps, server > cloud - after 1st failure: recovery plan, checklists, backup scenario - management supporting the migration team - active email, chat and oral communication - pair-programming to teach the new technology stack Good practice: requirements testing, learn by experience, create basic checklists, communicate with business stakeholders not only tech team OK: attention to quality shifted left on time
26. Cognitive testing approach explained
27. “Cognitive” means ... • respecting human mind and brain capabilities & limitations • respecting sensual processes like seeing, hearing, touching • respecting various interfaces (haptic/tactile, visual, brain) • dependent to perception
28. Cognitive approach goal = human-friendly & working software humans = testers, developers, end-users (...)
29. Perceptual limitations • we are humans • we make mistakes, we miss things We cannot see the entire site despite the web browser sees it all the time.
30. Biases-prone
32. Cognitive testing approach manifesto • Developers, testers, QAs, business stakeholders, end-users are humans. • Humans have some perceptual limitations we should mind when developing software for humans. • There are cultural differences between human using software.
33. Science behind cognitive approach
34. Attention span Source: “Attention spans. Consumer Insights.” Report by Microsoft Canada, 2015
35. Working memory capacity • just 2+-7 information items at once in 15-30 seconds duration (Miller’s law)
36. Change blindness Strong focusing makes us blind to changes. Strong focusing on testing one feature can make us blind to bugs in another feature.
37. Attention gap Brain constructs the view step by step. (Marr’s theory)
38. Schemas reasoning (Gestalt theory laws) Brain is working on schemas.
39. Schemas reasoning (Gestalt laws) Law of proximity
40. Schemas reasoning (Gestalt laws) Law of similarity
41. Schemas reasoning (Gestalt laws) Law of common fate: drag-and-drop interaction
42. Schemas reasoning (Gestalt laws) Law of symmetry
43. Schemas reasoning (Gestalt laws) Law of Prägnanz (figure-ground)
44. Gibson affordances Where the bug can be?
45. Tools, tips & tricks for daily use
46. Awareness is a key
47. Heuristics by Jakob Nielsen 47
48. Definition of Ready, Definition of Done
49. Incident analysis
50. Real-time monitoring • distributed communication • health-checks in releases (CI, versioning) • real-time monitoring (e.g. DataDog, New Relic, Fabric) • incident reports
51. Summary
52. Cognitive + shift left = future quality manifesto • we are humans, we need to communicate • we craft the software, you contribute to the quality, no matter what role you have • we want other humans to use our software • quality is a continuous thing • the earlier you shift attention to the quality, the better for you
53. Inspirations • cognitive science, cognitive psychology • clinical psychology research (EEG, fMRI, ERP, experiments) • user experience research • information architecture • anthropology • personal experiences of mates in software development :-)
54. Thank you for attention! aleksandrakornecka.com girlswhotest.pl joinOLX.com