Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Test Dive conference: Cognitive testing in shift-left mindset

Test Dive conference: Cognitive testing in shift-left mindset

Aleksandra Kornecka

October 18, 2018
Tweet

More Decks by Aleksandra Kornecka

Other Decks in Technology

Transcript

  1. Cognitive testing
    in shift-left mindset
    Aleksandra Kornecka
    OLX Group
    Test Dive conference, 18.10.2018 Kraków, Poland

    View full-size slide

  2. whoami
    ● software quality assurance engineer
    ● MSc. in cognitive science
    ● sprinter

    View full-size slide

  3. Agenda
    1. Quality in SDLC.
    2. Shift-left approach.
    3. Case studies.
    4. Cognitive testing approach.
    5. Science behind.
    6. Tools, tips & tricks.
    7. Summary.

    View full-size slide

  4. Quality in Software
    Development Life Cycle
    The true story

    View full-size slide

  5. Software quality in general
    ● technical & business values
    ● dimensions like functionality, reliability, usability, measurability,
    maintainability, -other abilities
    ● standards like ISO, IEEE, GDPR (...)

    View full-size slide

  6. Where the quality starts for you?

    View full-size slide

  7. Where the quality starts for you?

    View full-size slide

  8. Where the quality starts for you?

    View full-size slide

  9. Where the quality starts for you?

    View full-size slide

  10. The quality never starts and never ends

    View full-size slide

  11. … we should care all the time ...

    View full-size slide

  12. … so it’s time to SHIFT-LEFT!

    View full-size slide

  13. Shift-left approach
    explained

    View full-size slide

  14. “Shift-left” is a buzz word, but ...

    View full-size slide

  15. ..but the stats prove some benefits

    View full-size slide

  16. Enable awareness & attention early

    View full-size slide

  17. Shift-left approach definition
    Integrating the quality assurance practices and testing with
    entire software development life cycle:
    - from the very beginning e.g. requirements testing,
    prototype testing etc.
    - into every role e.g. developer, product owner, tester

    View full-size slide

  18. Source: webinar by Experitest

    View full-size slide

  19. Source: webinar by Experitest

    View full-size slide

  20. Other shift-left approach factors
    ● test the requirements, both technical and business
    ● work closely with stakeholders, validate ideas
    ● research often (A/B testing, users insights, tracked data)
    ● no sanctuary code, refactor on daily basis
    ● consider test automation as business value
    ● automate monitoring and alerting to keep focus into goal

    View full-size slide

  21. Case studies
    shift-left by example

    View full-size slide

  22. Source: webinar by Experitest

    View full-size slide

  23. Anti-pattern: never ending business request
    - one feature developed by 3 teams
    - requirements verification in the very end of development
    - unwanted feature from developer invention due to lack of business
    communication
    - 2 days of delay for 7 days marketing campaign
    Lesson learned: communicate with business teams before coding, create
    common calendar, update each other, squash branches
    NOT OK: shift left next time!

    View full-size slide

  24. Success story: migrating to cloud case
    4 countries, 2 months, 5 DevOps, server > cloud
    - after 1st failure: recovery plan, checklists, backup scenario
    - management supporting the migration team
    - active email, chat and oral communication
    - pair-programming to teach the new technology stack
    Good practice: requirements testing, learn by experience, create
    basic checklists, communicate with business stakeholders not only
    tech team
    OK: attention to quality shifted left on time

    View full-size slide

  25. Cognitive testing
    approach
    explained

    View full-size slide

  26. “Cognitive” means ...
    ● respecting human mind and brain capabilities & limitations
    ● respecting sensual processes like seeing, hearing, touching
    ● respecting various interfaces (haptic/tactile, visual, brain)
    ● dependent to perception

    View full-size slide

  27. Cognitive approach goal = human-friendly &
    working software
    humans = testers, developers, end-users (...)

    View full-size slide

  28. Perceptual limitations
    ● we are humans
    ● we make mistakes, we miss things
    We cannot see the entire site despite the
    web browser sees it all the time.

    View full-size slide

  29. Biases-prone

    View full-size slide

  30. Cognitive testing approach manifesto
    ● Developers, testers, QAs, business stakeholders, end-users are
    humans.
    ● Humans have some perceptual limitations we should mind when
    developing software for humans.
    ● There are cultural differences between human using software.

    View full-size slide

  31. Science behind cognitive
    approach

    View full-size slide

  32. Attention span
    Source: “Attention spans. Consumer Insights.” Report by Microsoft Canada, 2015

    View full-size slide

  33. Working memory capacity
    ● just 2+-7 information items at once in 15-30 seconds duration
    (Miller’s law)

    View full-size slide

  34. Change blindness
    Strong focusing makes us blind to changes.
    Strong focusing on testing one feature can make us blind to bugs in
    another feature.

    View full-size slide

  35. Attention gap
    Brain constructs the view step by step. (Marr’s theory)

    View full-size slide

  36. Schemas reasoning (Gestalt theory laws)
    Brain is working on schemas.

    View full-size slide

  37. Schemas reasoning (Gestalt laws)
    Law of proximity

    View full-size slide

  38. Schemas reasoning
    (Gestalt laws)
    Law of similarity

    View full-size slide

  39. Schemas reasoning (Gestalt laws)
    Law of common fate: drag-and-drop interaction

    View full-size slide

  40. Schemas reasoning
    (Gestalt laws)
    Law of symmetry

    View full-size slide

  41. Schemas reasoning (Gestalt laws)
    Law of Prägnanz
    (figure-ground)

    View full-size slide

  42. Gibson affordances
    Where the bug can be?

    View full-size slide

  43. Tools, tips & tricks
    for daily use

    View full-size slide

  44. Awareness is a key

    View full-size slide

  45. Heuristics by Jakob Nielsen
    47

    View full-size slide

  46. Definition of Ready, Definition of Done

    View full-size slide

  47. Incident analysis

    View full-size slide

  48. Real-time monitoring
    ● distributed communication
    ● health-checks in releases
    (CI, versioning)
    ● real-time monitoring (e.g.
    DataDog, New Relic, Fabric)
    ● incident reports

    View full-size slide

  49. Cognitive + shift left = future
    quality manifesto
    ● we are humans, we need to communicate
    ● we craft the software, you contribute to the
    quality, no matter what role you have
    ● we want other humans to use our software
    ● quality is a continuous thing
    ● the earlier you shift attention to the quality,
    the better for you

    View full-size slide

  50. Inspirations
    ● cognitive science, cognitive psychology
    ● clinical psychology research (EEG, fMRI, ERP, experiments)
    ● user experience research
    ● information architecture
    ● anthropology
    ● personal experiences of mates in software development :-)

    View full-size slide

  51. Thank you for attention!
    aleksandrakornecka.com
    girlswhotest.pl
    joinOLX.com

    View full-size slide