Searching for the best tests

Interested in learning more about this topic? Visit this web site to see related software: https://www.gregorykapfhammer.com/software/

Transcript

1. “Searching” for the Best Tests An Introduction to Automated Software Testing with Search-Based Techniques Gregory M. Kapfhammer Department of Computer Science Allegheny College March 31, 2015
2. Software is Everywhere Software is pervasive — and so it must be thoroughly tested! Computer Server Program
3. Software is Everywhere Software is pervasive — and so it must be thoroughly tested! Computer Server Program Computer Server Program Desktop Computer Program
4. Software is Everywhere Software is pervasive — and so it must be thoroughly tested! Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program
5. Software is Everywhere Software is pervasive — and so it must be thoroughly tested! Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program
6. Software is Everywhere Software is pervasive — and so it must be thoroughly tested! Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scienti c Device Program Household Appliance Program
7. Software is Everywhere Software is pervasive — and so it must be thoroughly tested! Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scienti c Device Program Household Appliance Program Network Router Program Scienti c Device Program
8. Software is Complex Even simple programs are intricate — and difficult to test! Computer Software
9. Software is Complex Even simple programs are intricate — and difficult to test! Computer Software Lines of Code
10. Software is Complex Even simple programs are intricate — and difficult to test! Computer Software Lines of Code Numerous Features
11. Software is Complex Even simple programs are intricate — and difficult to test! Computer Software Lines of Code Numerous Features Feature Interactions
12. Software is Complex Even simple programs are intricate — and difficult to test! Computer Software Lines of Code Numerous Features Feature Interactions Runtime Environs
13. Software is Complex Even simple programs are intricate — and difficult to test! Computer Software Lines of Code Numerous Features Feature Interactions Runtime Environs “Software entities are more complex for their size than perhaps any other hu- man construct” — Frederick P. Brooks, Jr.
14. Software is Evolving Software is continuously updated — making testing critical! Execution Environment Program
15. Software is Evolving Software is continuously updated — making testing critical! Execution Environment Program Execution Environment Program
16. Software is Evolving Software is continuously updated — making testing critical! Execution Environment Program Execution Environment Program Program Changed because of the addition of a new feature or the correction of a defect
17. Software is Evolving Software is continuously updated — making testing critical! Execution Environment Program Execution Environment Program
18. Software is Evolving Software is continuously updated — making testing critical! Execution Environment Program Execution Environment Program Execution Environment Program
19. Software is Evolving Software is continuously updated — making testing critical! Execution Environment Program Execution Environment Program Execution Environment Program Execution Environment Changed due to an up- grade in a kernel, device driver, or virtual machine
20. Software is Evolving Software is continuously updated — making testing critical! Execution Environment Program Execution Environment Program Execution Environment Program Execution Environment Changed due to an up- grade in a kernel, device driver, or virtual machine “Release early, release often” means that programs are regularly updated
21. Motivating Example The computation of an object’s velocity presents challenges! K = 1 2 m × v2
22. Motivating Example The computation of an object’s velocity presents challenges! K = 1 2 m × v2
23. Motivating Example The computation of an object’s velocity presents challenges! K = 1 2 m × v2
24. Motivating Example The computation of an object’s velocity presents challenges! K = 1 2 m × v2
25. Computing Velocity public static String computeVelocity(int kinetic, int mass) { int velocitySquared = 0; int velocity = 0; StringBuffer finalVelocity = new StringBuffer(); if( mass != 0 ) { velocitySquared = 3 * (kinetic / mass); velocity = (int)Math.sqrt(velocitySquared); finalVelocity.append(velocity); } else { finalVelocity.append("Undefined"); } return finalVelocity.toString(); }
26. Important Questions Finding software defects is a challenging and rewarding task Can you nd the defect in this program?
27. Important Questions Finding software defects is a challenging and rewarding task Are there general purpose strategies for defect isolation?
28. The Challenges of Software Development Pervasiveness of Software Complexity of Software Evolving Nature of Software Motivating Example Important Questions Bene ts of Software Testing Test Cases Test Suites Examples of Tests The PIE Model Test Case E ectiveness Search-Based Software Testing Testing Methods Random Testing Testing with EvoSuite Conclusion
29. The Challenges of Software Development Pervasiveness of Software Complexity of Software Evolving Nature of Software Motivating Example Important Questions Bene ts of Software Testing Test Cases Test Suites Examples of Tests The PIE Model Test Case E ectiveness Search-Based Software Testing Testing Methods Random Testing Testing with EvoSuite Conclusion
30. The Challenges of Software Development Pervasiveness of Software Complexity of Software Evolving Nature of Software Motivating Example Important Questions Bene ts of Software Testing Test Cases Test Suites Examples of Tests The PIE Model Test Case E ectiveness Search-Based Software Testing Testing Methods Random Testing Testing with EvoSuite Conclusion
31. The Challenges of Software Development Pervasiveness of Software Complexity of Software Evolving Nature of Software Motivating Example Important Questions Bene ts of Software Testing Test Cases Test Suites Examples of Tests The PIE Model Test Case E ectiveness Search-Based Software Testing Testing Methods Random Testing Testing with EvoSuite Conclusion
32. What is a Test Case? A test case calls a method and checks its output with an oracle Method Un- der Test
33. What is a Test Case? A test case calls a method and checks its output with an oracle Method Un- der Test Input
34. What is a Test Case? A test case calls a method and checks its output with an oracle Method Un- der Test Input Output
35. What is a Test Case? A test case calls a method and checks its output with an oracle Method Un- der Test Test Set Up
36. What is a Test Case? A test case calls a method and checks its output with an oracle Method Un- der Test Test Set Up Input
37. What is a Test Case? A test case calls a method and checks its output with an oracle Method Un- der Test Test Set Up Input Output
38. What is a Test Case? A test case calls a method and checks its output with an oracle Method Un- der Test Test Set Up Input Output Test Clean Up
39. What is a Test Case? A test case calls a method and checks its output with an oracle Method Un- der Test Test Set Up Input Output Test Clean Up Test Oracle
40. What is a Test Case? A test case calls a method and checks its output with an oracle Method Un- der Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output
41. What is a Test Case? A test case calls a method and checks its output with an oracle Method Un- der Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict
42. What is a Test Case? A test case calls a method and checks its output with an oracle Method Un- der Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output
43. What is a Test Case? A test case calls a method and checks its output with an oracle Method Un- der Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output Test Verdict The test case passes and the code is correct!
44. What is a Test Case? A test case calls a method and checks its output with an oracle Method Un- der Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output
45. What is a Test Case? A test case calls a method and checks its output with an oracle Method Un- der Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output Test Verdict The test case fails and a defect is found!
46. What is a Test Suite? A test suite is an organized collection of test cases T1
47. What is a Test Suite? A test suite is an organized collection of test cases T1 T2
48. What is a Test Suite? A test suite is an organized collection of test cases T1 T2 T3
49. What is a Test Suite? A test suite is an organized collection of test cases T1 T2 T3 T4
50. What is a Test Suite? A test suite is an organized collection of test cases T1 T2 T3 T4 . . .
51. What is a Test Suite? A test suite is an organized collection of test cases T1 T2 T3 T4 . . . Tn
52. What is a Test Suite? A test suite is an organized collection of test cases T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite
53. What is a Test Suite? A test suite is an organized collection of test cases T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite Tool Support for Software Testing?
54. What is a Test Suite? A test suite is an organized collection of test cases T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite Tool Support for Software Testing? JUnit
55. What is a Test Suite? A test suite is an organized collection of test cases T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite Tool Support for Software Testing? JUnit Apache Ant
56. What is a Test Suite? A test suite is an organized collection of test cases T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite Tool Support for Software Testing? JUnit Apache Ant Eclipse
57. A JUnit Test Case @Test public void testOne() { String expected = new String("Undefined"); String actual = Kinetic. computeVelocity(5,0); assertEquals(expected, actual); }
58. Another JUnit Test @Test public void testTwo() { String expected = new String("0"); String actual = Kinetic. computeVelocity(0,5); assertEquals(expected, actual); }
59. Important Questions Not all tests have the same fault detection effectiveness! Will these test cases nd the fault in the example program?
60. Important Questions Not all tests have the same fault detection effectiveness! T1 assigns K = 5, m = 0 — Pass or fail?
61. Important Questions Not all tests have the same fault detection effectiveness! T2 assigns K = 0, m = 5 — Pass or fail?
62. Important Questions Not all tests have the same fault detection effectiveness! How do we study the e ectiveness of di erent test cases?
63. The PIE Model There are necessary and sufficient conditions for fault detection Execute the faulty source code Infect the program’s data state Propagate to the program’s output All of these must occur before the fault manifests itself as a failure! Using PIE, will the test cases nd the defect in the program?
64. The PIE Model There are necessary and sufficient conditions for fault detection Execute the faulty source code Infect the program’s data state Propagate to the program’s output All of these must occur before the fault manifests itself as a failure! Using PIE, will the test cases nd the defect in the program?
65. The PIE Model There are necessary and sufficient conditions for fault detection Execute the faulty source code Infect the program’s data state Propagate to the program’s output All of these must occur before the fault manifests itself as a failure! Using PIE, will the test cases nd the defect in the program?
66. The PIE Model There are necessary and sufficient conditions for fault detection Execute the faulty source code Infect the program’s data state Propagate to the program’s output All of these must occur before the fault manifests itself as a failure! Using PIE, will the test cases nd the defect in the program?
67. The PIE Model There are necessary and sufficient conditions for fault detection Execute the faulty source code Infect the program’s data state Propagate to the program’s output All of these must occur before the fault manifests itself as a failure! Using PIE, will the test cases nd the defect in the program?
68. The PIE Model There are necessary and sufficient conditions for fault detection Execute the faulty source code Infect the program’s data state Propagate to the program’s output All of these must occur before the fault manifests itself as a failure! Using the PIE model, will the test cases nd the defect in the program?
69. A JUnit Test Case — T1 @Test public void testOne() { String expected = new String("Undefined"); String actual = Kinetic. computeVelocity(5,0); assertEquals(expected, actual); } E I P   
70. A JUnit Test Case — T1 @Test public void testOne() { String expected = new String("Undefined"); String actual = Kinetic. computeVelocity(5,0); assertEquals(expected, actual); } E I P   
71. A JUnit Test Case — T2 @Test public void testTwo() { String expected = new String("0"); String actual = Kinetic. computeVelocity(0,5); assertEquals(expected, actual); } E I P   
72. A JUnit Test Case — T2 @Test public void testTwo() { String expected = new String("0"); String actual = Kinetic. computeVelocity(0,5); assertEquals(expected, actual); } E I P   
73. A JUnit Test Case — T3 @Test public void testThree() { String expected = new String("4"); String actual = Kinetic. computeVelocity(8,1); assertEquals(expected, actual); } E I P   
74. A JUnit Test Case — T3 @Test public void testThree() { String expected = new String("4"); String actual = Kinetic. computeVelocity(8,1); assertEquals(expected, actual); } E I P   
75. A JUnit Test Case — T4 @Test public void testFour() { String expected = new String("20"); String actual = Kinetic. computeVelocity(1000,5); assertEquals(expected, actual); } E I P   
76. A JUnit Test Case — T4 @Test public void testFour() { String expected = new String("20"); String actual = Kinetic. computeVelocity(1000,5); assertEquals(expected, actual); } E I P   
77. Test Suite Summary A test case must create specific inputs in order to cause failure! Test Case Status T1 Pass T2 Pass T3 Pass T4 Fail
78. Important Insights Software testing is fundamentally challenging — is there help? I shall not deny that the construction of these testing programs has been a major intellectual e ort: to convince oneself that one has not overlooked “a relevant state” and to convince oneself that the testing programs generate them all is no simple matter. Edsger W. Dijkstra, Communications of the ACM, 1968
79. The Challenges of Software Development Pervasiveness of Software Complexity of Software Evolving Nature of Software Motivating Example Important Questions Bene ts of Software Testing Test Cases Test Suites Examples of Tests The PIE Model Test Case E ectiveness Search-Based Software Testing Testing Methods Random Testing Testing with EvoSuite Conclusion
80. The Challenges of Software Development Pervasiveness of Software Complexity of Software Evolving Nature of Software Motivating Example Important Questions Bene ts of Software Testing Test Cases Test Suites Examples of Tests The PIE Model Test Case E ectiveness Search-Based Software Testing Testing Methods Random Testing Testing with EvoSuite Conclusion
81. The Challenges of Software Development Pervasiveness of Software Complexity of Software Evolving Nature of Software Motivating Example Important Questions Bene ts of Software Testing Test Cases Test Suites Examples of Tests The PIE Model Test Case E ectiveness Search-Based Software Testing Testing Methods Random Testing Testing with EvoSuite Conclusion
82. The Challenges of Software Development Pervasiveness of Software Complexity of Software Evolving Nature of Software Motivating Example Important Questions Bene ts of Software Testing Test Cases Test Suites Examples of Tests The PIE Model Test Case E ectiveness Search-Based Software Testing Testing Methods Random Testing Testing with EvoSuite Conclusion
83. Manual Testing While it has benefits, this industry standard may be limited! Manual Testing
84. Manual Testing While it has benefits, this industry standard may be limited! Manual Testing Laborious
85. Manual Testing While it has benefits, this industry standard may be limited! Manual Testing Laborious Time Consuming
86. Manual Testing While it has benefits, this industry standard may be limited! Manual Testing Laborious Time Consuming Very Tedious
87. Manual Testing While it has benefits, this industry standard may be limited! Manual Testing Laborious Time Consuming Very Tedious Di cult
88. Manual Testing While it has benefits, this industry standard may be limited! Manual Testing Laborious Time Consuming Very Tedious Di cult Can we develop and employ methods that will automatically generate high- quality test cases for real-world software?
89. Automated Testing Automatically generating tests is amazing — but does it work? Automated Testing
90. Automated Testing Automatically generating tests is amazing — but does it work? Automated Testing Laborious
91. Automated Testing Automatically generating tests is amazing — but does it work? Automated Testing Laborious Time Consuming
92. Automated Testing Automatically generating tests is amazing — but does it work? Automated Testing Laborious Time Consuming Very Tedious
93. Automated Testing Automatically generating tests is amazing — but does it work? Automated Testing Laborious Time Consuming Very Tedious Di cult
94. Automated Testing Automatically generating tests is amazing — but does it work? Automated Testing Laborious Time Consuming Very Tedious Di cult Laborious Time Consuming Very Tedious Testing is less laborious and tedious because an algorithm generates the tests. While computational time is needed, a human can be less involved!
95. Automated Testing Automatically generating tests is amazing — but does it work? Automated Testing Laborious Time Consuming Very Tedious Di cult Laborious Time Consuming Very Tedious Laborious Time Consuming Very Tedious Di cult Automated testing is less di cult since a good tness function can guide the algorithm to inputs that nd the faults
96. Random Testing It is easy to randomly generate tests — but how good are they? −6 −4 −2 0 2 4 6 −1 −0.5 0 0.5 1 0 2 4 6 8 10 12 14
97. Search-Based Testing Use a fitness function to guide the search to “good” values 0 0.2 0.4 0.6 0.8 1 0 0.5 1 0 0.5 1
98. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2
99. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4
100. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6
101. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8
102. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
103. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10
104. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 M1 M2
105. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 M1 M2 M3 M4
106. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 M1 M2 M3 M4 M5 M6
107. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 M1 M2 M3 M4 M5 M6 M7 M8
108. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10
109. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12
110. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 Set of Program Methods M = {M1, M2, . . . , M11, M12}
111. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 Set of Program Methods M = {M1, M2, . . . , M11, M12} M12 M12 M12 M12
112. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 Set of Program Methods M = {M1, M2, . . . , M11, M12} M12 M12 M12 M12 M12 M12
113. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 Set of Program Methods M = {M1, M2, . . . , M11, M12} M12 M12 M12 M12 M12 M12 M12 M12 M12 M12
114. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 Set of Program Methods M = {M1, M2, . . . , M11, M12} M12 M12 M12 M12 M12 M12 M12 M12 M12 M12 M4
115. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 Set of Program Methods M = {M1, M2, . . . , M11, M12} M12 M12 M12 M12 M12 M12 M12 M12 M12 M12 M4 T1
116. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 Set of Program Methods M = {M1, M2, . . . , M11, M12} M12 M12 M12 M12 M12 M12 M12 M12 M12 M12 M4 T1
117. Mutation Testing Let’s purposefully insert faults into the program under test! T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 Set of Program Methods M = {M1, M2, . . . , M11, M12} M12 M12 M12 M12 M12 M12 M12 M12 M12 M12 M4 T1 M8
118. Testing with EvoSuite This prototype can automatically generate real JUnit test suites! Evolutionary Testing
119. Testing with EvoSuite This prototype can automatically generate real JUnit test suites! Evolutionary Testing Representation
120. Testing with EvoSuite This prototype can automatically generate real JUnit test suites! Evolutionary Testing Representation Fitness Function
121. Testing with EvoSuite This prototype can automatically generate real JUnit test suites! Evolutionary Testing Representation Fitness Function Modify Program
122. Testing with EvoSuite This prototype can automatically generate real JUnit test suites! Evolutionary Testing Representation Fitness Function Modify Program Operators
123. Testing with EvoSuite This prototype can automatically generate real JUnit test suites! Evolutionary Testing Representation Fitness Function Modify Program Operators “1600 Faults in 100 Projects: Automatically Finding Faults While Achieving High Coverage with EvoSuite”, Empirical Software Engineering
124. Configuring EvoSuite This tool has many unique configurations — which are best? EvoSuite’s Con gurations
125. Configuring EvoSuite This tool has many unique configurations — which are best? EvoSuite’s Con gurations Random
126. Configuring EvoSuite This tool has many unique configurations — which are best? EvoSuite’s Con gurations Random Fixed-Random
127. Configuring EvoSuite This tool has many unique configurations — which are best? EvoSuite’s Con gurations Random Fixed-Random Genetic
128. Configuring EvoSuite This tool has many unique configurations — which are best? EvoSuite’s Con gurations Random Fixed-Random Genetic Regression
129. Configuring EvoSuite This tool has many unique configurations — which are best? EvoSuite’s Con gurations Random Fixed-Random Genetic Regression The tness function computes branch coverage, weak mutation score, or strong mutation score to guide the search to the “best” test cases
130. A Test from EvoSuite @Test public void test001() throws Throwable { int int0 = 0; String string0 = Kinetic.computeVelocity(int0, int0); assertEquals("Undefined", string0); assertNotNull(string0); int int1 = 1262; String string1 = Kinetic.computeVelocity(int0, int0); int int2 = 5349; String string2 = Kinetic.computeVelocity(int1, int2); int int3 = 0; int int4 = 3; String string3 = Kinetic.computeVelocity(int3, int4); Kinetic kinetic0 = new Kinetic(); }
131. Important Questions EvoSuite is an advanced, yet sometimes limited, testing tool Will EvoSuite’s test cases nd the fault in the example program?
132. Important Questions EvoSuite is an advanced, yet sometimes limited, testing tool What is missing from the test cases that EvoSuite generates?
133. Important Questions EvoSuite is an advanced, yet sometimes limited, testing tool How does the oracle problem in uence the e ectiveness of EvoSuite?
134. Important Questions EvoSuite is an advanced, yet sometimes limited, testing tool What are the fundamental limitations of automated testing?
135. No Silver Bullet Software tools are fundamentally limited — what is our hope? There is no single development, in either technology or management technique, which by itself promises even one order-of-magnitude improvement within a decade in productivity, in reliability, in simplicity. Frederick P. Brooks, Jr., Proceedings of the IFIP Tenth World Computing Conference, 1986
136. The Challenges of Software Development Pervasiveness of Software Complexity of Software Evolving Nature of Software Motivating Example Important Questions Bene ts of Software Testing Test Cases Test Suites Examples of Tests The PIE Model Test Case E ectiveness Search-Based Software Testing Testing Methods Random Testing Testing with EvoSuite Conclusion
137. The Challenges of Software Development Pervasiveness of Software Complexity of Software Evolving Nature of Software Motivating Example Important Questions Bene ts of Software Testing Test Cases Test Suites Examples of Tests The PIE Model Test Case E ectiveness Search-Based Software Testing Testing Methods Random Testing Testing with EvoSuite Conclusion
138. The Challenges of Software Development Pervasiveness of Software Complexity of Software Evolving Nature of Software Motivating Example Important Questions Bene ts of Software Testing Test Cases Test Suites Examples of Tests The PIE Model Test Case E ectiveness Search-Based Software Testing Testing Methods Random Testing Testing with EvoSuite Conclusion
139. The Challenges of Software Development Pervasiveness of Software Complexity of Software Evolving Nature of Software Motivating Example Important Questions Bene ts of Software Testing Test Cases Test Suites Examples of Tests The PIE Model Test Case E ectiveness Search-Based Software Testing Testing Methods Random Testing Testing with EvoSuite Conclusion
140. The Software Crisis Solutions are available — but not always obvious or popular What are the solutions to the software crisis?
141. The Software Crisis Solutions are available — but not always obvious or popular Great Designers
142. The Software Crisis Solutions are available — but not always obvious or popular Unremitting Care
143. The Software Crisis Solutions are available — but not always obvious or popular Incremental Advances
144. Final Admonishment An epilogue called “Fifty Years of Wonder, Excitement, and Joy” Too many interests, too many exciting opportunities for learning, research, and thought. What a marvelous predicament! Not only is the end not in sight, but the pace is not slackening. We have many future joys. Frederick P. Brooks, Jr., The Mythical Man-Month, 1995