Tweet
Tweet

Slide 1

Slide 1 text

adrn ! adrianprw " Adrian Price-Whelan (Columbia University) Software Testing

Slide 2

Slide 2 text

github.com/adrn/SoftwareTesting

Slide 3

Slide 3 text

Disclaimer: This is Python-focused (but applies to any language) github.com/adrn/SoftwareTesting

Slide 4

Slide 4 text

Why

Slide 5

Slide 5 text

• Complex code is impossible to debug by eye Why

Slide 6

Slide 6 text

• Complex code is impossible to debug by eye • Testing is a robust way to validate software Why

Slide 7

Slide 7 text

• Complex code is impossible to debug by eye • Testing is a robust way to validate software - Does it work as I expect? Why

Slide 8

Slide 8 text

• Complex code is impossible to debug by eye • Testing is a robust way to validate software - Does it work as I expect? - Does it work as the users expect? Why

Slide 9

Slide 9 text

• Complex code is impossible to debug by eye • Testing is a robust way to validate software - Does it work as I expect? - Does it work as the users expect? - Does this update break existing code? Why

Slide 10

Slide 10 text

• Complex code is impossible to debug by eye • Testing is a robust way to validate software - Does it work as I expect? - Does it work as the users expect? - Does this update break existing code? • Testing also helps refine structure / intent Why

Slide 11

Slide 11 text

• Complex code is impossible to debug by eye • Testing is a robust way to validate software - Does it work as I expect? - Does it work as the users expect? - Does this update break existing code? • Testing also helps refine structure / intent - Is this code modular and testable? Why

Slide 12

Slide 12 text

What

Slide 13

Slide 13 text

What •Unit tests - Check small “units” of code independent of other code - Is the code doing what the programmer expects?

Slide 14

Slide 14 text

What •Unit tests - Check small “units” of code independent of other code - Is the code doing what the programmer expects? •Functional tests - Check that units of code interact as expected - Tests may use multiple classes, functions, modules… - Is the programmer doing what the user expects?

Slide 15

Slide 15 text

Hypothetical software

Slide 16

Slide 16 text

Hypothetical software • Read an optical spectrum from a text file

Slide 17

Slide 17 text

Hypothetical software • Read an optical spectrum from a text file • Integrate flux over some wavelength range

Slide 18

Slide 18 text

Unit tests • Read an optical spectrum from a text file - Does it succeed for all anticipated input types? - Does it fail if you have invalid data? • Integrate flux over some wavelength range - Is integral positive for physical input values?

Slide 19

Slide 19 text

Functional tests

Slide 20

Slide 20 text

Functional tests • Read an optical spectrum from a text file • Integrate flux over some wavelength range - Generate valid temp. file on the fly with known integral - Read from temporary file, pass to integration routine - Check output

Slide 21

Slide 21 text

Functional tests • Read an optical spectrum from a text file • Integrate flux over some wavelength range - Generate valid temp. file on the fly with known integral - Read from temporary file, pass to integration routine - Check output ➡ Tests end-to-end functionality

Slide 22

Slide 22 text

Test-driven development

Slide 23

Slide 23 text

1. Write tests / API Test-driven development

Slide 24

Slide 24 text

1. Write tests / API 2. Write skeleton of code Test-driven development

Slide 25

Slide 25 text

1. Write tests / API 2. Write skeleton of code 3. Run tests and debug Test-driven development

Slide 26

Slide 26 text

1. Write tests / API 2. Write skeleton of code 3. Run tests and debug 4. Write code until tests pass Test-driven development

Slide 27

Slide 27 text

1. Write tests / API 2. Write skeleton of code 3. Run tests and debug 4. Write code until tests pass Test-driven development

Slide 28

Slide 28 text

Tools: testing frameworks

Slide 29

Slide 29 text

Tools: testing frameworks • unittest - Standard library - Requires some boiler-plate code to make simple tests

Slide 30

Slide 30 text

Tools: testing frameworks • unittest - Standard library - Requires some boiler-plate code to make simple tests • py.test - Third-party package - Easy to write simple tests - Plugin architecture (e.g., doctests, coverage, pep8, mpl, …) - (used by Astropy)

Slide 31

Slide 31 text

Tools: testing frameworks • unittest - Standard library - Requires some boiler-plate code to make simple tests • py.test - Third-party package - Easy to write simple tests - Plugin architecture (e.g., doctests, coverage, pep8, mpl, …) - (used by Astropy) • nose - Similar to py.test — pick one and roll with it

Slide 32

Slide 32 text

Tools: testing frameworks • Discovers & runs functions starting with test_… classes starting with Test…

Slide 33

Slide 33 text

Tools: testing frameworks

Slide 34

Slide 34 text

Tools: testing frameworks

Slide 35

Slide 35 text

Tools: continuous integration • When new code is integrated or is proposed (e.g., pull request), verify that tests pass • See: Travis-CI, Jenkins

Slide 36

Slide 36 text

Tools: continuous integration • Travis-CI — “matrix” builds

Slide 37

Slide 37 text

Tools: continuous integration • Travis-CI — “matrix” builds

Slide 38

Slide 38 text

Tools: continuous integration comments, commits, etc…

Slide 39

Slide 39 text

Conclusions

Slide 40

Slide 40 text

Conclusions • Unit tests by themselves are almost useless — balance unit and functional tests

Slide 41

Slide 41 text

Conclusions • Unit tests by themselves are almost useless — balance unit and functional tests • Try out test-driven development — could save you time in the long run

Slide 42

Slide 42 text

Conclusions • Unit tests by themselves are almost useless — balance unit and functional tests • Try out test-driven development — could save you time in the long run • There are many tools out there that make writing and running tests easy — use them!

Slide 43

Slide 43 text

Conclusions • Unit tests by themselves are almost useless — balance unit and functional tests • Try out test-driven development — could save you time in the long run • There are many tools out there that make writing and running tests easy — use them! (try: Py.test + GitHub + Travis-CI)