Software Testing

An overview of why software testing is important, what methods can be used to test software, and how to integrate testing with development practices.

Some of the code got cut off, visit the original presentation to see it in full: http://nickmccurdy.com/testing-talk/

Transcript

1. SOFTWARE TESTING BY NICOLAS MCCURDY
2. WHAT IS SOFTWARE TESTING? The process of verifying software to ensure it works as expected Can be manual or automated Whenever it is possible, automated testing is prefered because of its repeatability and ease of use. In more complex scenarios, some things may only be easy to test manually.
3. WHY SHOULD I TEST? To prove that your code satisfies requirements To catch bugs faster, before software is out in the field To have an unambiguous way of determining if a unit of code still works after it is modified (regression testing) To add flexibility and efficiency to your project's development Continuous integration is a great example.
4. TESTING METHODS There are many testing methods that define what aspects of a system should be tested and how. It is common for specific software projects to use multiple testing methods to cover different kinds of defects (bugs, performance issues, security vulnerabilities, etc.). Many test frameworks and other testing tools are available for most popular testing methods.
5. UNIT TESTING The process of testing individual units of code separately from each other By running unit tests, it is easier to isolate what units of code are to blame for bugs. Unit tests are considered "white box" tests because they involve looking at the internals of an application's code and runtime.
6. WHAT DO WE NEED TO KNOW TO WRITE A UNIT TEST? The unit we are testing (usually a function or method) The test cases we will create to prove the unit functions as expected Inputs Equivalance classes: groups of inputs that should result in similar behaviors Boundary cases: values on the edges of these classes Expected outputs and side effects
7. UNIT TESTING TIPS Try to test with as many equivalance classes of inputs as possible. Start by focusing on tests for individual methods and functions. Share common setup between unit tests.
8. TEST CASE EXAMPLE What should our test cases be? d e f p a l i n d r o m e ? ( w o r d ) w o r d = = w o r d . r e v e r s e e n d TEST CASES When the word is empty, it should be a palindrome When the word has one character, it should be a palindrome When the word has multiple characters, we should ensure the right words are marked as palindromes When the word is null, it should not be a palindrome
9. UNIT TESTING WITH MINITEST r e q u i r e ' m i n i t e s t / a u t o r u n ' c l a s s T e s t P a l i n d r o m e < M i n i t e s t : : T e s t d e f t e s t _ p a l i n d r o m e a s s e r t _ t r u e p a l i n d r o m e ? ' ' a s s e r t _ t r u e p a l i n d r o m e ? ' c ' a s s e r t _ f a l s e p a l i n d r o m e ? ' c a r ' a s s e r t _ t r u e p a l i n d r o m e ? ' r a c e c a r ' a s s e r t _ f a l s e p a l i n d r o m e ? n i l e n d e n d
10. MINITEST RESULTS $ r u b y t e s t _ p a l i n d r o m e . r b R u n o p t i o n s : - - s e e d 6 5 3 1 0 # R u n n i n g : E F i n i s h e d i n 0 . 0 0 1 6 5 3 s , 6 0 4 . 7 7 9 6 r u n s / s , 2 4 1 9 . 1 1 8 3 a s s e r t i o n s / s . 1 ) E r r o r : T e s t P a l i n d r o m e # t e s t _ p a l i n d r o m e : N o M e t h o d E r r o r : u n d e f i n e d m e t h o d ` r e v e r s e ' f o r n i l : N i l C l a s s / U s e r s / n i c k / t e s t i n g - t a l k / p a l i n d r o m e . r b : 2 : i n ` p a l i n d r o m e ? ' t e s t _ p a l i n d r o m e . r b : 1 0 : i n ` t e s t _ p a l i n d r o m e ' 1 r u n s , 4 a s s e r t i o n s , 0 f a i l u r e s , 1 e r r o r s , 0 s k i p s
11. UNIT TESTING WITH RSPEC d e s c r i b e ' p a l i n d r o m e ? ' d o c o n t e x t ' g i v e n a n e m p t y s t r i n g ' d o i t ' r e t u r n s t r u e ' d o e x p e c t ( p a l i n d r o m e ? ' ' ) . t o b e t r u e e n d e n d c o n t e x t ' g i v e n a s t r i n g w i t h o n l y o n e c h a r a c t e r ' d o i t ' r e t u r n s t r u e ' d o e x p e c t ( p a l i n d r o m e ? ' c ' ) . t o b e t r u e e n d e n d c o n t e x t ' g i v e n a l o n g e r n o n - p a l i n d r o m e ' d o i t ' r e t u r n s t r u e ' d o e x p e c t ( p a l i n d r o m e ? ' c a r ' ) . t o b e f a l s e e n d e n d c o n t e x t ' g i v e n a l o n g e r p a l i n d r o m e ' d o i t ' r e t u r n s t r u e ' d o e x p e c t ( p a l i n d r o m e ? ' r a c e c a r ' ) . t o b e t r u e e n d e n d
12. c o n t e x t ' g i v e n a n u l l v a l u e ' d o i t ' r e t u r n s f a l s e ' d o e x p e c t ( p a l i n d r o m e ? n i l ) . t o b e f a l s e e n d e n d e n d
13. INTEGRATION TESTING Multiple software modules are tested together to verify their communications with each other. Spies can be used to wrap interfaces and inspect what they are called with. Integration tests do not need to know the internals of the individual modules.
14. ACCEPTANCE TESTING The process of testing a program's features to ensure that they meet customer requirements Acceptance tests give us a way to prove that software meets needs and works from a user's perspective. Acceptance tests are considered "black box" tests because they simulate user actions without requiring access to an application's internals (however, they can still be automated).
15. SMOKE TESTING The process of performing quick checks to see if there are any basic problems with the code that would prevent it from running. Smoke testing is generally not enough for testing an entire application, though it can be useful at earlier phases of the testing process. EXAMPLES Check if there are any compiler errors Check if the application launches Perform a request on an important API endpoint and assert that it returns a 200 (success)
16. MOCK OBJECTS "Fake" objects that replace the real implementations of code Very useful for dealing with slow, unreliable, and hard to test code that you aren't testing directly. Can make tests more flexible, but can also increase the amount of complexity in test setup. Especially useful for services and HTTP requests. Example: Tests for a weather application that relies on a third party API.
17. TESTING AND PROCESS Depending on a project's process model and personal development practices, testing may happen after, during, or even before development. Waiting to test until after all development is done (waterfall) can cause difficulties with integrating work and verifying if systems themselves or their dependencies are broken.
18. TEST DRIVEN DEVELOPMENT The process of writing test code before implementation code. 1. Plan what you are going to implement. 2. Write tests (usually unit tests or acceptance tests) for what you will implement. 3. Run the tests (before implementing). 4. Write the implementation. 5. Run tests and ensure that the implementation's tests pass. 6. Rinse and repeat until all tests are passing.
19. BEHAVIORAL DRIVEN DEVELOPMENT Similar to test driven development, but involving more process and abstraction. Tests (usually acceptance tests) are written to verify specific user stories as they are implemented. BDD TOOLS Test frameworks that focus on heirarchal specifications (like RSpec) or executable user stories (like Cucumber) are designed for and frequently used for BDD.
20. ADDITIONAL RESOURCES RSpec Documentation: MiniTest Documentation: Pair Columbus Challenges: CI and CI Talk: rspec.info github.com/seattlerb/minitest paircolumbus.org/challenges bbesmanoff.github.io/ci-cd-talk
21. QUESTIONS