Ensemble Testing - 10 years of Mindshare

Ten years ago I started a journey of facing one of my personal demons: I did not like programming, I loved system testing. Not manual testing, but exploratory testing. However, I learned about a practice of ensemble (mob) testing, and I volunteered to sit with my all-developers team and program together using one computer and taking turns on the keyboard. Since then, I have been in a lot of ensemble testing sessions, and facilitated even more of them. It has become my primary way of learning and teaching for both programming and testing.

Ensemble testing - a group testing activity utilizing one computer - creates mindshare by giving voice of the knowledge in the group of individuals on a shared task with a testing perspective. It makes a great mechanism not just for passing knowledge but also for building habits and transferring skills. Working on the same thing, at the same time, in the same space and at the same computer allows testers to contribute more and be appreciated for their contributions. Lessons specific to skillsets rub in both ways, leaving everyone better off after the experience. The task at hand can involve programming, but also it can start as a typical recon exploratory testing session, building a plan as we work with an application.

This talk shares my core lessons of ensemble testing, both as a participant and a facilitator. It gives you guidance on how and why to get started, how to avoid the common pitfalls and how to create varied experiences of testing beyond the first day of ensembling.

Transcript

1. © 2024 CGI Inc. 1 Ensemble Testing 10 Years of Mindshare Maaret Pyhäjärvi
2. © 2024 CGI Inc. 2 2 Brains + Voice NAVIGATOR Voices ENSEMBLE Hands DRIVER ROTATE
3. © 2024 CGI Inc. 3 Variation over time, variation over skill 3 Ensemble Programming Solo Programming
4. © 2024 CGI Inc. 4 Why? 4
5. © 2024 CGI Inc. 5 Timeline 2014: Woody Zuill “Whole team approach – a ridiculous idea. Must experience 1st hand.” 2015: Ensemble Exploring Moving from pairs to ensembles in teaching hands-on exploratory testing. 2017: Learning by Osmosis Personal transformation from ‘tester to developer’ acknowledged 2020: Community rename Cultivated options and choosing new name by Maaret Pyhäjärvi & Denise Yu. Emily Bache, Lisi Hocke & Lisa Crispin pioneered. 2024: Teaching and learning Go-to approach for hands-on teaching and learning. Reveals unknown unknowns.
6. © 2024 CGI Inc. 6 Ensemble lessons on testing 6
7. © 2024 CGI Inc. 7 7 Everyone can test
8. © 2024 CGI Inc. 8 Testing Gives Us… Unit / Exploratory / Contemporary Exploratory 8 Spec Feedback Regression Granularity Guidance Understanding Models Serendipity Documenting Extending reach Alerting to attend Guiding to detail
9. © 2024 CGI Inc. 9 Ensemble testing is great for… • Larger organizations where functional silos continue to exist (Testers get together to test) • Agile teams where we need to share testing skills (Teams get together to test) • Teaching anything testing related to a group (Training group gets together to test) • Introducing testing perspectives to programming activities (Testers joining ensemble programming)
10. © 2024 CGI Inc. 10 Ensemble lessons on programming 10
11. © 2024 CGI Inc. 11 Programming is like writing. Getting started is easy and it takes a lifetime to get good at. 11
12. © 2024 CGI Inc. 12 Key Lessons Correcting mistakes without egos in play Do both! Holding space for what is relevant Slow allows for thoughtful thinking Opportunities to inject exploration Moving from technical debt to technical assets Best face forward in a group Best ideas win when you care about work over credit 12
13. © 2024 CGI Inc. 13 Becoming polyglot programmer / contemporary exploratory tester 13 I had great ideas even when I did not know how to turn them into code – removing the distance is worth the struggle. Cognitive dissonance
14. © 2024 CGI Inc. 14 Ensemble lessons on ensembling 14
15. © 2024 CGI Inc. 15 Ensemble Working Ensemble Programming Applying Ensemble Working technique on development purposes and tasks including whole team testing. Ensembling Ensemble Testing Applying Ensemble Working technique on testing purposes and tasks. NEW Building sufficient space for all necessary activities, and bring all people along for the change. Scoping + TDD + Review + All levels of test automation + Exploring
16. © 2024 CGI Inc. 16 Ensemble Working - Examples Ensemble Programming Refactor code with ‘extract method’ and ‘rename’ Make reports feature testable Implement a feature Test-Driven Development Ensembling Ensemble Testing Create Selenium tests Explore a feature Explore an application with a mindmap Explore an application with programmatic tests NEW
17. © 2024 CGI Inc. 17 Heuristics of keep going 17 Learning Hard tasks by sourcing best of everyone Contributing Easy tasks by automating to oblivion Holding space
18. © 2024 CGI Inc. 18 Rules Bias to action Keep the group moving. Not a meeting. Decisions away from keyboard Accepting actions, correcting actions. Time to retro Improve the way the group works early and often. Highest level of abstraction Intent – Location – Details in instruction. Listen to more than words. Solo Breaks Step out when you need a break. Group can continue uninterrupted.
19. © 2024 CGI Inc. 19 Dynamics 19 Ensemble size 2 … 23 Rotation 4 min … 1 min … 10 min *mindshare* Roles RPG (Role Playing Game) by Willem Larsen Navigation Intent – Location - Details
20. © 2024 CGI Inc. 20 How to think about control Strong Style Traditional I have an idea… Please take the keyboard I have an idea… Give me the keyboard “For an idea to go from your head to the computer it must go through someone else’s hands”
21. © 2024 CGI Inc. 21 21 Step in to navigate In-middle retro Artifacts over conversations
22. © 2024 CGI Inc. 22 Varied experiences Testing is framed by • Charter • Target – Need of skill to learn and to test – Size – Quality – Interface for programmatic testing Target applications Text field Web element collections ePrimer Freemind, Xmind Dark Function Editor ToDoMVC ParkCalc Conduit Juice shop Restful Booker Zippopotamus Weather App, OpenWeather API Sentiment API Gilded rose Roman numerals Raster Reveal 22
23. © 2024 CGI Inc. 23 https://www.linkedin.com/in/maar et/ @maaretp@mas.to Temporary working spaces differ from permanent ones
24. © 2024 CGI Inc. 24 24 Even better remotely.
25. © 2024 CGI Inc. 25 25 Mindshare
26. © 2024 CGI Inc. 26 26 Allows for contributing on different skills profiles.
27. © 2024 CGI Inc. 27 Observe Ideate Let Go Be Impressed Support others Do Something Yes, and… 1. Observe: notice more 2. Ideate: stop censuring 3. Let go: trust the process 4. Be Impressed: react 5. Support others: make them shine 6. Do something: experiment culture 7. Yes, and: combine * Metaskills of Improv, Simo Routarinne, proimpro.fi
28. © 2024 CGI Inc. 28 28 https://ensembleprogramming.xyz
29. © 2024 CGI Inc. 29 Insights you can act on Founded in 1976, CGI is among the largest IT and business consulting services firms in the world. We are insights-driven and outcomes-based to help accelerate returns on your investments. Across hundreds of locations worldwide, we provide comprehensive, scalable and sustainable IT and business consulting services that are informed globally and delivered locally. cgi.com