Challenges in Software Development

Gives a glimpse of that Software Development is about, illustrated by 3 challenges:
- Avoiding misunderstandings using "Agile" methods
- Avoiding techical debt with clean code
- Delivering quality fast with automated tests

1. Challenges in Software Development Sebastian Nozzi * - no, this is not me climbing
2. 3 Challenges
3. Challenge 1 Communication
4. The classical Process Analyse Design Implement Deliver customer's problem big final solution
5. The classical Process Analyse Design Implement Deliver big final solution 1 year customer's problem
6. The classical Process Analyse Design Implement Deliver time, complexity, assumptions big final solution 1 year customer's problem
7. Problems with this approach • Not all details available at the beginning ➡ Many assumptions need to be made (possibly wrong ones) • Difficult to do accurate resource planing and time estimations for the whole process • Customer does not want what he/she needs until seeing final solution • Once design is made, it's difficult / costly to change
8. Often the result is this...
9. What the customer wanted
10. What the customer got
11. ”Agile” Process to the rescue...
12. Short Feedback Cycles As soon as there is something to show to the customer, show it, gather feedback and repeat cycle. Approach final solution gradually and iteratively.
13. happy customer =
14. But even if the customer is happy...
15. ”I have this great idea for a new feature!” The Customer
16. Challenge 2 Adapt to Change
17. Constant effort per version Version1.0 Version1.1 Version1.2 (In the ideal world)
18. Reality Check!
19. Increasing effort per version Version1.0 Version1.1 Version1.2
20. What is one main cause of this?
21. Legacy Code
22. What is Legacy Code?
23. Legacy Code •Unreadable •Not reusable •Fragile (*) * small changes break the code unexpectedly
24. Increasing effort per version... Version1.0 Version1.1 Version1.2 LC LC LC LC LC LC ... thanks to accumulated legacy code ... also called "technical debt"
25. The alternative...?
26. Clean Code!
27. Clean Code •Readable •Re-Usable •Adaptable (*) * you can make changes with confidence
28. Examples (in pseudo-Python)
29. def f(x): data = [] ... return data Names too abstract, impossible to know what they mean:
30. def process(my_list): data = [] ... return data Names too ambiguous
31. def find_phd_students(ist_user_list): phd_students = [] ... return phd_students Clear, meaningful, intention-revealing names:
32. def find_phd_students(ist_user_list): ... if(ist_user.user_type == 24): ... A "magic number". What does it mean?
33. def find_phd_students(ist_user_list): ... if(ist_user.user_type == UserType.PHD): ... Better with a meaningful constant:
34. def find_phd_students(ist_user_list): ... def find_postdocs(ist_user_list): ... def find_professors(ist_user_list): ... def find_staff(ist_user_list): ... def find_ssu_heads(ist_user_list): ... Re-writing similar functionality over and over again:
35. def find_users(ist_user_list, user_type): ... Generalise into re-usable code:
36. def find_users(ist_user_list, user_type): ... phd_students = find_users(users, UserType.PHD) ssu_heads = find_users(users, UserType.SSU_HEAD) Generalise into re-usable code:
37. def fetch_ist_users(only_active): # open configuration file # get database name, user and password # connect to the database # construct query # perform query for all users # iterate over result # if "only_active" is set, keep only active # return list -Doing too much -Mixing different levels of abstraction
38. object Database: def connect(): # get configuration ... def get_all_users(): ... object Configuration: def get_datbase_user(): ... def get_database_password(): ... def fetch_ist_users(only_active): # Obtain all users from Database # iterate over result # if "only_active" is set, keep only active # return list Separated concerns into re-usable components
39. generic specific generic specific generic specific generic re-usable not re-usable not re-usable In general, de-compose your problem into specific and generic aspects. Otherwise the whole is not re-usable. You benefit in the future, by being able to leverage your re- usable components and cut cost&time...
40. Clean Code •Readable •Re-Usable •Adaptable what about this one?
41. Challenge 3 Grow with Confidence ... and peace of mind
42. Goal: Version 1.1 Version1.0 Version1.1 Version1.2 We were given the task to implement Version 1.1, and we want time and costs not to increase...
43. Feature Feature Fix Feature Version 1.0 Version 1.1 These are the features/fixes to implement... goal
44. breaks? breaks? breaks? Feature Feature Fix Feature Version 1.0 Version 1.1 But by each introduced change we might have broken something...
45. It could even be like this... Feature Feature Fix Feature breaks? breaks? breaks? breaks? breaks?
46. How to be sure?
47. Possible Approaches • Prevent (static analysis, compile-time checking, etc.) • Analyse (review code / changes, hope to spot bugs) • Manual Tests (time-intensive, prone to human error) • Automated Tests...?
48. Automated Tests • Write once (has some initial cost) • Run repeatedly (running is cheap / quick) • No need to manually track dependencies ➡ Developer productivity and program robustness
49. One possible approach...
50. Write a failing Test Make it Pass Improve your Code Start here Test Driven Development
51. Benefits of "Test-first" • Forces one to state the requirements clearly, in advance+ • Focuses on one problem at a time ➡ Avoids designing too early for inexistent problems • Leads to better, re-usable, testable code • Leads to high test-coverage
52. A Case Study Greeter (*) * - Just a function that greets you
53. greeter_test.py greeter.py Here comes the test-code Here comes the implementation
54. def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) greeter_test.py greeter.py We begin by writing the test first
55. def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) greeter_test.py greeter.py Initially, the test fails because there is no implementation
56. def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) def greet(name): return "Hello, " + name greeter_test.py greeter.py Once there is a correct implementation, the test passes
57. def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) def test("greet the world when no name"): expected = "Hello, World" result = greet("") assert(result == expected) def greet(name): return "Hello, " + name greeter_test.py greeter.py We write a test for a further requirement, which initially fails
58. def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) def test("greet the world when no name"): expected = "Hello, World" result = greet("") assert(result == expected) def greet(name): if(name == ""): return "Hello, World" else return "Hello, " + name greeter_test.py greeter.py Once the implementation has been corrected to comply with this requirement, the test passes
59. def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) def test("greet the world when no name"): expected = "Hello, World" result = greet("") assert(result == expected) def greet(name): if(name == ""): return "Hello, World" else return "Hello, " + name greeter_test.py greeter.py def test("greet the world when nil"): expected = "Hello, World" result = greet(nil) assert(result == expected) One last requirement, and an initially failing test
60. def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) def test("greet the world when no name"): expected = "Hello, World" result = greet("") assert(result == expected) def greet(name): if(name == nil): return "Hello, World" else return "Hello, " + name greeter_test.py greeter.py def test("greet the world when nil"): expected = "Hello, World" result = greet(nil) assert(result == expected) ? Does this new implementation make all tests pass? ? ?
61. def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) def test("greet the world when no name"): expected = "Hello, World" result = greet("") assert(result == expected) def greet(name): if(name == nil): return "Hello, World" else return "Hello, " + name greeter_test.py greeter.py def test("greet the world when nil"): expected = "Hello, World" result = greet(nil) assert(result == expected) No, it actually breaks the second test. Note that we didn't need to analyse the whole problem. The testing framework does it for us.
62. def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) def test("greet the world when no name"): expected = "Hello, World" result = greet("") assert(result == expected) def greet(name): if(name == nil || name == ""): return "Hello, World" else return "Hello, " + name greeter_test.py greeter.py def test("greet the world when nil"): expected = "Hello, World" result = greet(nil) assert(result == expected) Once fixed, all tests pass
63. Metrics of a Production App • Very complex workflows, lots of dependencies • 65 Testing Scenarios • 742 Automated Interactions • 117 Automated non-interactive Tests • Time for all tests: 1 minute • Manually: a whole day? two? Real time- saver!
64. Recap
65. Recap •Short Feedback Cycles ... to avoid unpleasant surprises •Readable, Re-Usable Code ... to reduce cost per version •Automated Tests ... to introduce changes quickly and with confidence
66. Thanks