Cognitive Driven Development: A Research Agenda

1. Cognitive Driven Development: A Research Agenda 1 Gustavo Pinto Assistant professor at UFPA.br Head of research at ZUP.com.br @gustavopinto
2. @zupinnovation zup.com.br <> @zupinnovation Software will grow and get old! But people still have to maintain these programs
3. https://github.com/spring-projects/spring-framework/blob/b595dc1dfad9db534ca7b9e8f46bb9926b88ab5a/spring-core/src/main/java/org/springframework/asm/ClassReader.java (~4k LOC) How to avoid this?
4. @zupinnovation zup.com.br <> @zupinnovation If we want to keep making changes … https://twitter.com/kentbeck/status/1354418068869398538
5. Keep chunking and slicing differently
6. <> @zupinnovation zup.com.br Cognitive-Driven Development (CDD) • CDD aims to reduce the developers’ cognitive load during coding activities • CDD does so by posing a limit on the number of items devs could use at once
7. <> @zupinnovation zup.com.br Cognitive-Driven Development (CDD) • CDD is based on two psychological theories The Magical Number 7 The Cognitive Load Theory
8. <> @zupinnovation zup.com.br The Magical Number 7 (+/- 2) (Miller, 1956) • We are only able to process 7 (+/- 2) units of information in short-term memory. • As we receive more information simultaneously, we lose the ability to process it (and we tend to make mistakes). Units of information Comprehension
9. <> @zupinnovation zup.com.br Cognitive Load Theory (Sweller, 1988; Sweller, 2010) • Each material has its own intrinsic complexity, and requires some cognitive effort ◦ Sometimes big, sometimes small • Certain types of items can hinder our understanding
10. @zupinnovation zup.com.br <> @zupinnovation CDD in a nutshell • CDD provides a limit indicating how much a code unit could grow • CDD defines the code elements that hinder our understanding (and then should be limited)
11. @zupinnovation zup.com.br <> @zupinnovation CDD in a nutshell • CDD provides a limit indicating how much a code unit could grow • CDD defines the code elements that hinder our understanding (and then should be limited) • Every class over the limit must be refactored
12. @zupinnovation zup.com.br <> @zupinnovation The team is responsible for defining the items of complexities
13. @zupinnovation zup.com.br <> @zupinnovation Early works on CDD SBES 2021 ICSME 2020 ESEM 2022 JSS (Under review) ICEIS 2022
14. Does CDD impact code readability?
15. @zupinnovation zup.com.br <> 18 Zuppers 2 OSS projects 10s refactorings
16. We curated 10 pairs of code 10 CDD-Driven Refactorings 10 Original Versions CDD-Driven Refactorings The source code quality league
17. We asked 133 devs to answer (and justify) their preferences Java Experience Does CDD improve code readability?
18. Does CDD improve code readability? Description Original CDD % Original % CDD Encapsulating error handling 67 66 50% 50% Encapsulating business rules 19 114 14% 86% Concatenating logical expressions 9 124 7% 93% Extracting classes 14 119 11% 89% Listing all imports 45 88 34% 66% Functional checking style 95 38 71% 29% Extracting classes 27 106 20% 80% Encapsulating for loops 10 123 8% 92%
19. Does CDD improve code readability? Description Original CDD % Original % CDD Encapsulating error handling 67 66 50% 50% Encapsulating business rules 19 114 14% 86% Concatenating logical expressions 9 124 7% 93% Extracting classes 14 119 11% 89% Listing all imports 45 88 34% 66% Functional checking style 95 38 71% 29% Extracting classes 27 106 20% 80% Encapsulating for loops 10 123 8% 92%
20. Encapsulating error handling Original (50%) CDD (50%) “Logging with try-catch is so common that I didn’t even need to read the code to understand its purpose." “the code is smaller and sounds like English"
21. Does CDD improve code readability? Description Original CDD % Original % CDD Encapsulating error handling 67 66 50% 50% Encapsulating business rules 19 114 14% 86% Concatenating logical expressions 9 124 7% 93% Extracting classes 14 119 11% 89% Listing all imports 45 88 34% 66% Functional checking style 95 38 71% 29% Extracting classes 27 106 20% 80% Encapsulating for loops 10 123 8% 92%
22. Encapsulating error handling “It is important to know which exception could be raised” “The method signature is clear and it makes it easier to understand its behavior” Original (14%) CDD (86%)
23. Description Original CDD % Original % CDD Encapsulating error handling 67 66 50% 50% Encapsulating business rules 19 114 14% 86% Concatenating logical expressions 9 124 7% 93% Extracting classes 14 119 11% 89% Listing all imports 45 88 34% 66% Functional checking style 95 38 71% 29% Extracting classes 27 106 20% 80% Encapsulating for loops 10 123 8% 92% Does CDD improve code readability?
24. Functional checking style “The ternary is an easy resource and there was no complex logic to justify a .map” Original (71%) CDD (29%)
25. Does CDD improve code quality?
26. @zupinnovation zup.com.br <> handora.zup.com.br • Online training platform by Zup • Composed by 5 services: ◦ Core service (Java) ◦ Search service (Java) ◦ Menu service (Java) ◦ Frontend Service (TypeScript) ◦ ML service (Python)
27. @zupinnovation zup.com.br <> handora.zup.com.br • Online training platform by Zup • Composed by 5 services: ◦ Core service (Java) ◦ Search service (Java) ◦ Menu service (Java) ◦ Frontend Service (TypeScript) ◦ ML service (Python) • Method ◦ Mining repository data ◦ Cross-validation with the devs
28. @zupinnovation zup.com.br <> @zupinnovation CDD impacts the size of the classes • CDD seems to help keep the classes small, even as the product evolves (almost linearly) • Developers concur that the size of the classes are probably due to CDD
29. @zupinnovation zup.com.br <> @zupinnovation CDD impacts the size of the methods • Maintenance effort is positively correlated with method length. • “Developers should strive to keep their methods within 24 SLOC”
30. @zupinnovation zup.com.br <> @zupinnovation CDD impacts the size of the methods • 92% of the handora’s methods are under the limit (24 SLOC) • On average, a method has 6.8 SLOC • “Every unit of code is impacted, because we know what the limit is and what goes into that limit.”
31. @zupinnovation zup.com.br <> @zupinnovation It is not always possible to impose the limit • “Core” classes • Classes with rich contracts
32. @zupinnovation zup.com.br <> @zupinnovation It is not always possible to impose the limit • “Core” classes • Classes with rich contracts
33. @zupinnovation zup.com.br <> @zupinnovation It is not always possible to impose the limit • “Core” classes • Classes with rich contracts
34. @zupinnovation zup.com.br <> @zupinnovation CDD impacts testing code # SLOC Methods Coverage S1 7.6k 215 71% S2 1.3k 41 61% S3 5.2k 128 64%
35. @zupinnovation zup.com.br <> @zupinnovation CDD impacts testing code • On average, a testing method has ~8 SLOC • “I think there is a relationship because the complexity of the test can be seen as a proxy of the complexity of the code under test”. • There are ~1.3 assertions per method (no method without assertion) # SLOC Methods Coverage S1 7.6k 215 71% S2 1.3k 41 61% S3 5.2k 128 64%
36. What are we up to next?
37. @zupinnovation zup.com.br <> @zupinnovation 1. Are small classes better than large ones? • CDD forces developers to create smaller classes • But this might require them to navigate more between classes.
38. @zupinnovation zup.com.br <> @zupinnovation 2. Could CDD help devs to find items of complexity? • Do devs agree with the CDD-suggested items of complexity? • When they do not agree? • Would devs refactor these suggested items of complexity?
39. @zupinnovation zup.com.br <> 3. How to ease CDD adoption? • Our experience in advocating for CDD at Zup wasn’t successful • Tools are not everything • How to support technology adoption?
40. @zupinnovation zup.com.br <> Cognitive Driven Development: A Research Agenda • CDD-driven refactorings seems to be more readable • CDD seems to help designing small classes • CDD seems to help designing small testing methods Findings
41. @zupinnovation zup.com.br <> Cognitive Driven Development: A Research Agenda • CDD-driven refactorings seems to be more readable • CDD seems to help designing small classes • CDD seems to help designing small testing methods Findings • Are smaller classes better? • Does CDD help in finding items of complexity? • How to ease CDD adoption? Questions