Challenges in Software Development

Slide 1

Slide 1 text

Challenges in Software Development Sebastian Nozzi * - no, this is not me climbing

Slide 2

Slide 2 text

3 Challenges

Slide 3

Slide 3 text

Challenge 1 Communication

Slide 4

Slide 4 text

The classical Process Analyse Design Implement Deliver customer's problem big ﬁnal solution

Slide 5

Slide 5 text

The classical Process Analyse Design Implement Deliver big ﬁnal solution 1 year customer's problem

Slide 6

Slide 6 text

The classical Process Analyse Design Implement Deliver time, complexity, assumptions big ﬁnal solution 1 year customer's problem

Slide 7

Slide 7 text

Problems with this approach • Not all details available at the beginning ➡ Many assumptions need to be made (possibly wrong ones) • Difficult to do accurate resource planing and time estimations for the whole process • Customer does not want what he/she needs until seeing final solution • Once design is made, it's difficult / costly to change

Slide 8

Slide 8 text

Often the result is this...

Slide 9

Slide 9 text

What the customer wanted

Slide 10

Slide 10 text

What the customer got

Slide 11

Slide 11 text

”Agile” Process to the rescue...

Slide 12

Slide 12 text

Short Feedback Cycles As soon as there is something to show to the customer, show it, gather feedback and repeat cycle. Approach ﬁnal solution gradually and iteratively.

Slide 13

Slide 13 text

happy customer =

Slide 14

Slide 14 text

But even if the customer is happy...

Slide 15

Slide 15 text

”I have this great idea for a new feature!” The Customer

Slide 16

Slide 16 text

Challenge 2 Adapt to Change

Slide 17

Slide 17 text

Constant effort per version Version1.0 Version1.1 Version1.2 (In the ideal world)

Slide 18

Slide 18 text

Reality Check!

Slide 19

Slide 19 text

Increasing effort per version Version1.0 Version1.1 Version1.2

Slide 20

Slide 20 text

What is one main cause of this?

Slide 21

Slide 21 text

Legacy Code

Slide 22

Slide 22 text

What is Legacy Code?

Slide 23

Slide 23 text

Legacy Code •Unreadable •Not reusable •Fragile (*) * small changes break the code unexpectedly

Slide 24

Slide 24 text

Increasing effort per version... Version1.0 Version1.1 Version1.2 LC LC LC LC LC LC ... thanks to accumulated legacy code ... also called "technical debt"

Slide 25

Slide 25 text

The alternative...?

Slide 26

Slide 26 text

Clean Code!

Slide 27

Slide 27 text

Clean Code •Readable •Re-Usable •Adaptable (*) * you can make changes with conﬁdence

Slide 28

Slide 28 text

Examples (in pseudo-Python)

Slide 29

Slide 29 text

def f(x): data = [] ... return data Names too abstract, impossible to know what they mean:

Slide 30

Slide 30 text

def process(my_list): data = [] ... return data Names too ambiguous

Slide 31

Slide 31 text

def find_phd_students(ist_user_list): phd_students = [] ... return phd_students Clear, meaningful, intention-revealing names:

Slide 32

Slide 32 text

def find_phd_students(ist_user_list): ... if(ist_user.user_type == 24): ... A "magic number". What does it mean?

Slide 33

Slide 33 text

def find_phd_students(ist_user_list): ... if(ist_user.user_type == UserType.PHD): ... Better with a meaningful constant:

Slide 34

Slide 34 text

def find_phd_students(ist_user_list): ... def find_postdocs(ist_user_list): ... def find_professors(ist_user_list): ... def find_staff(ist_user_list): ... def find_ssu_heads(ist_user_list): ... Re-writing similar functionality over and over again:

Slide 35

Slide 35 text

def find_users(ist_user_list, user_type): ... Generalise into re-usable code:

Slide 36

Slide 36 text

def find_users(ist_user_list, user_type): ... phd_students = find_users(users, UserType.PHD) ssu_heads = find_users(users, UserType.SSU_HEAD) Generalise into re-usable code:

Slide 37

Slide 37 text

def fetch_ist_users(only_active): # open configuration file # get database name, user and password # connect to the database # construct query # perform query for all users # iterate over result # if "only_active" is set, keep only active # return list -Doing too much -Mixing different levels of abstraction

Slide 38

Slide 38 text

object Database: def connect(): # get configuration ... def get_all_users(): ... object Configuration: def get_datbase_user(): ... def get_database_password(): ... def fetch_ist_users(only_active): # Obtain all users from Database # iterate over result # if "only_active" is set, keep only active # return list Separated concerns into re-usable components

Slide 39

Slide 39 text

generic specific generic specific generic specific generic re-usable not re-usable not re-usable In general, de-compose your problem into specific and generic aspects. Otherwise the whole is not re-usable. You benefit in the future, by being able to leverage your re- usable components and cut cost&time...

Slide 40

Slide 40 text

Clean Code •Readable •Re-Usable •Adaptable what about this one?

Slide 41

Slide 41 text

Challenge 3 Grow with Conﬁdence ... and peace of mind

Slide 42

Slide 42 text

Goal: Version 1.1 Version1.0 Version1.1 Version1.2 We were given the task to implement Version 1.1, and we want time and costs not to increase...

Slide 43

Slide 43 text

Feature Feature Fix Feature Version 1.0 Version 1.1 These are the features/ﬁxes to implement... goal

Slide 44

Slide 44 text

breaks? breaks? breaks? Feature Feature Fix Feature Version 1.0 Version 1.1 But by each introduced change we might have broken something...

Slide 45

Slide 45 text

It could even be like this... Feature Feature Fix Feature breaks? breaks? breaks? breaks? breaks?

Slide 46

Slide 46 text

How to be sure?

Slide 47

Slide 47 text

Possible Approaches • Prevent (static analysis, compile-time checking, etc.) • Analyse (review code / changes, hope to spot bugs) • Manual Tests (time-intensive, prone to human error) • Automated Tests...?

Slide 48

Slide 48 text

Automated Tests • Write once (has some initial cost) • Run repeatedly (running is cheap / quick) • No need to manually track dependencies ➡ Developer productivity and program robustness

Slide 49

Slide 49 text

One possible approach...

Slide 50

Slide 50 text

Write a failing Test Make it Pass Improve your Code Start here Test Driven Development

Slide 51

Slide 51 text

Beneﬁts of "Test-ﬁrst" • Forces one to state the requirements clearly, in advance+ • Focuses on one problem at a time ➡ Avoids designing too early for inexistent problems • Leads to better, re-usable, testable code • Leads to high test-coverage

Slide 52

Slide 52 text

A Case Study Greeter (*) * - Just a function that greets you

Slide 53

Slide 53 text

greeter_test.py greeter.py Here comes the test-code Here comes the implementation

Slide 54

Slide 54 text

def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) greeter_test.py greeter.py We begin by writing the test ﬁrst

Slide 55

Slide 55 text

def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) greeter_test.py greeter.py Initially, the test fails because there is no implementation

Slide 56

Slide 56 text

def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) def greet(name): return "Hello, " + name greeter_test.py greeter.py Once there is a correct implementation, the test passes

Slide 57

Slide 57 text

def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) def test("greet the world when no name"): expected = "Hello, World" result = greet("") assert(result == expected) def greet(name): return "Hello, " + name greeter_test.py greeter.py We write a test for a further requirement, which initially fails

Slide 58

Slide 58 text

def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) def test("greet the world when no name"): expected = "Hello, World" result = greet("") assert(result == expected) def greet(name): if(name == ""): return "Hello, World" else return "Hello, " + name greeter_test.py greeter.py Once the implementation has been corrected to comply with this requirement, the test passes

Slide 59

Slide 59 text

def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) def test("greet the world when no name"): expected = "Hello, World" result = greet("") assert(result == expected) def greet(name): if(name == ""): return "Hello, World" else return "Hello, " + name greeter_test.py greeter.py def test("greet the world when nil"): expected = "Hello, World" result = greet(nil) assert(result == expected) One last requirement, and an initially failing test

Slide 60

Slide 60 text

def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) def test("greet the world when no name"): expected = "Hello, World" result = greet("") assert(result == expected) def greet(name): if(name == nil): return "Hello, World" else return "Hello, " + name greeter_test.py greeter.py def test("greet the world when nil"): expected = "Hello, World" result = greet(nil) assert(result == expected) ? Does this new implementation make all tests pass? ? ?

Slide 61

Slide 61 text

def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) def test("greet the world when no name"): expected = "Hello, World" result = greet("") assert(result == expected) def greet(name): if(name == nil): return "Hello, World" else return "Hello, " + name greeter_test.py greeter.py def test("greet the world when nil"): expected = "Hello, World" result = greet(nil) assert(result == expected) No, it actually breaks the second test. Note that we didn't need to analyse the whole problem. The testing framework does it for us.

Slide 62

Slide 62 text

def test("greet Dave"): expected = "Hello, Dave" result = greet("Dave") assert(result == expected) def test("greet the world when no name"): expected = "Hello, World" result = greet("") assert(result == expected) def greet(name): if(name == nil || name == ""): return "Hello, World" else return "Hello, " + name greeter_test.py greeter.py def test("greet the world when nil"): expected = "Hello, World" result = greet(nil) assert(result == expected) Once ﬁxed, all tests pass

Slide 63

Slide 63 text

Metrics of a Production App • Very complex workﬂows, lots of dependencies • 65 Testing Scenarios • 742 Automated Interactions • 117 Automated non-interactive Tests • Time for all tests: 1 minute • Manually: a whole day? two? Real time- saver!

Slide 64

Slide 64 text

Recap

Slide 65

Slide 65 text

Recap •Short Feedback Cycles ... to avoid unpleasant surprises •Readable, Re-Usable Code ... to reduce cost per version •Automated Tests ... to introduce changes quickly and with conﬁdence

Slide 66

Slide 66 text

Thanks