Automated tests - facts and myths

The discussion on automated tests is hot topic. The approach has same number of advocates and skeptics. More and more tools eases testing, but also introduces a fundamental question: what, when and how to test? Practise and experience let's answer those questions or guide in the right direction.

In this talk usage examples of unit, functional and behavioral tests will be shown. Importance of properly handling dependencies and mocking them will be discussed as well. But most of important part will be hints on how to write code, that could be tested automaticaly.

Slides are available in interactive mode here: http://tdd.sznapka.pl/

Transcript

1. Automated tests Facts and myths / Wojciech Sznapka @sznapka
2. Cześć
3. An introduction I work in software industry for about 9 years Care a lot about robust and testable architectures Loves software craftsmanship, sophisticated architectures, Big Data and ice hockey
4. Test Driven Development What is this?
5. Add a test
6. Write an implementation
7. Refactor code
8. Repeat!
9. What's important here?
10. ... to have tests
11. You won’t go to hell if you’ll write a test after declaring an interface or prototyping a class ...
12. ... but you’ll surely end up in hell, if there won’t be a test coverage
13. Myths about automated tests
14. Boss doesn't pay for automated tests
15. It's a myth! A 2005 study found that using TDD meant writing more tests and, in turn, programmers who wrote more tests tended to be more productive by American Scientists
16. But this ain't that hard ... c l a s s R o m a n C o n v e r t e r { p r o t e c t e d $ c o n v e r s i o n s = [ 1 0 0 0 = > ' M ' , 9 0 0 = > ' C M ' , 5 0 0 = > ' D ' , 4 0 0 = > ' C D ' , 1 0 0 = > ' C ' , 9 0 = > ' X C ' , 5 0 = > ' L ' , 4 0 = > ' X L ' , 1 0 = > ' X ' , 9 = > ' I X ' , 5 = > ' V ' , 4 = > ' I V ' , 1 = > ' I ' ] ; p u b l i c f u n c t i o n c o n v e r t ( $ i n A r a b i c ) { i f ( ! i s _ n u m e r i c ( $ i n A r a b i c ) ) { t h r o w n e w \ I n v a l i d A r g u m e n t E x c e p t i o n ( ' I c o n v e r t n u m e r i c s ' ) ; } i f ( $ i n A r a b i c < = 0 ) { r e t u r n ' ' ; } l i s t ( $ a r a b i c , $ r o m a n ) = $ t h i s - > c o n v e r s i o n F a c t o r F o r ( $ i n A r a b i c ) ; r e t u r n $ r o m a n . $ t h i s - > c o n v e r t ( $ i n A r a b i c - $ a r a b i c ) ; } p r o t e c t e d f u n c t i o n c o n v e r s i o n F a c t o r F o r ( $ i n A r a b i c ) { f o r e a c h ( $ t h i s - > c o n v e r s i o n s a s $ a r a b i c = > $ r o m a n ) { i f ( $ a r a b i c < = $ i n A r a b i c ) { r e t u r n [ $ a r a b i c , $ r o m a n ] ; } } } }
17. ... start from obvious things c l a s s R o m a n C o n v e r t e r T e s t e x t e n d s P H P U n i t _ F r a m e w o r k _ T e s t C a s e { p u b l i c f u n c t i o n t e s t E m p t y ( ) { $ t h i s - > a s s e r t E q u a l s ( ' ' , ( n e w R o m a n C o n v e r t e r ) - > c o n v e r t ( ' ' ) ) ; } }
18. ... provide some meat ... c l a s s R o m a n C o n v e r t e r T e s t e x t e n d s P H P U n i t _ F r a m e w o r k _ T e s t C a s e { / * * @ d a t a P r o v i d e r p r o v i d e T e s t D a t a * / p u b l i c f u n c t i o n t e s t C o n v e r s i o n s ( $ a r a b i c , $ r o m a n ) { $ c o n v e r t e r = n e w R o m a n C o n v e r t e r ( ) ; $ t h i s - > a s s e r t E q u a l s ( $ r o m a n , $ c o n v e r t e r - > c o n v e r t ( $ a r a b i c ) ) ; } p u b l i c f u n c t i o n p r o v i d e T e s t D a t a ( ) { r e t u r n a r r a y [ [ 3 4 9 7 , ' M M M C D X C V I I ' ] , [ 1 , ' I ' ] , [ 2 , ' I I ' ] , [ 6 , ' V I ' ] , [ 9 , ' I X ' ] , [ 4 0 , ' X L ' ] , [ 4 5 , ' X L V ' ] , [ 9 0 , ' X C ' ] , [ 1 0 0 , ' C ' ] , [ 4 0 0 , ' C D ' ] ] ; } }
19. ... test edge cases c l a s s R o m a n C o n v e r t e r T e s t e x t e n d s P H P U n i t _ F r a m e w o r k _ T e s t C a s e { / * * @ e x p e c t e d E x c e p t i o n \ I n v a l i d A r g u m e n t E x c e p t i o n * / p u b l i c f u n c t i o n t e s t E m p t y ( ) { ( n e w R o m a n C o n v e r t e r ) - > c o n v e r t ( ' w t f I a m p a s s i n g h e r e ' ) ) ; } }
20. Help yourself with tests generation p h p u n i t - s k e l g e n - - t e s t R o m a n C o n v e r t e r
21. ... or using Symfony's XSolveUnitSkelgenBundle . / a p p / c o n s o l e x s o l v e : s k e l g e n : t e s t A c m e / E x a m p l e B u n d l e / S e r v i c e / . . / a p p / c o n s o l e x s o l v e : s k e l g e n : t e s t A c m e / * / C o n t r o l l e r / *
22. We are building too complicated system for an automated tests
23. TDD forces you to write cleaner code
24. It's easier to test smaller units of code
25. so you write smaller classes which are less coupled and that makes your system more stable and open for an extension in the future
26. Your Object Oriented design becomes S.O.L.I.D. compilant and this is awesome!
27. You will of course tackle costly dependencies
28. Use PHPUnit's mock framework or Mockery
29. Fake it till you make it c l a s s M o c k e d T e s t e x t e n d s P H P U n i t _ F r a m e w o r k _ T e s t C a s e { p u b l i c f u n c t i o n t e s t N o n E x i s t i n g V a l u e O b j e c t s ( ) { $ c o n f i g u r a t i o n = \ M o c k e r y : : m o c k ( ' \ C o n f i g u r a t i o n V a l u e O b j e c t ' , [ ' g e t U r l ' = > ' h t t p : / / t d d . s z n a p k a . p l ' , ' g e t F o r m a t ' = > ' x m l ' ] ) ; $ t h i s - > a s s e r t E q u a l s ( ' x m l ' , $ c o n f i g u r a t i o n - > g e t F o r m a t ( ) ) ; / / O K } }
30. Mock things that can't be tested quickly or non- reproducable c l a s s M o c k e d T e s t e x t e n d s P H P U n i t _ F r a m e w o r k _ T e s t C a s e { p u b l i c f u n c t i o n t e s t A p i C a l l s ( ) { $ b u z z M o c k = \ M o c k e r y : : m o c k ( ' \ B u z z \ B r o w s e r ' ) ; $ b u z z M o c k - > s h o u l d R e c e i v e ( ' g e t ' ) - > a n d R e t u r n ( ' < r e s p o n s e > < m o o d > A w e s o m i t y < / m o o d > < / r e s p o n s e > ' ) ; / / t h i s a l s o i s w i s e s o l u t i o n , t o w r i t e x m l f i x t u r e s i n f i l e / / $ b u z z M o c k - > s h o u l d R e c e i v e ( ' g e t ' ) - > o n c e ( ) / / - > a n d R e t u r n ( f i l e _ g e t _ c o n t e n t s ( _ _ D I R _ _ . ' / f i x t u r e s . x m l ' ) ) ; $ a p i = n e w \ A p i C o n s u m e r ( $ b u z z M o c k ) ; $ a p i - > a s s e r t E q u a l s ( ' A w e s o m i t y ' , $ a p i - > g e t C u r r e n t M o o d ( ) ) ; / / O K } }
31. Expect declared behaviors c l a s s M o c k e d T e s t e x t e n d s P H P U n i t _ F r a m e w o r k _ T e s t C a s e { p u b l i c f u n c t i o n t e s t E x p e c t a t i o n s D e c l a r a t i o n s ( ) { $ b u z z M o c k = \ M o c k e r y : : m o c k ( ' \ B u z z \ B r o w s e r ' ) ; $ b u z z M o c k - > s h o u l d R e c e i v e ( ' g e t ' ) - > a n d R e t u r n ( ' < r e s p o n s e > < m o o d > A w e s o m i t y < / m o o d > < / r e s p o n s e > ' ) ; $ l o g g e r M o c k = \ M o c k e r y : : m o c k ( ' \ M o n o l o g \ L o g g e r ' ) ; / / w e j u s t w a n t t o b e s u r e t h a t L o g g e r : : i n f o w a s c a l l e d o n l y o n c e $ l o g g e r M o c k - > s h o u l d R e c e i v e ( ' i n f o ' ) - > o n c e ( ) ; $ a p i = n e w \ A p i C o n s u m e r ( $ b u z z M o c k , $ l o g g e r M o c k ) ; $ a p i - > a s s e r t E q u a l s ( ' A w e s o m i t y ' , $ a p i - > g e t C u r r e n t M o o d ( ) ) ; / / O K } }
32. Be prepared for failures and check if you prepared for unexpected situations c l a s s M o c k e d T e s t e x t e n d s P H P U n i t _ F r a m e w o r k _ T e s t C a s e { / * * @ e x p e c t e d E x c e p t i o n \ M y E x c e p t i o n W r a p p e r * / p u b l i c f u n c t i o n t e s t F a i l e d C o n n e c t i o n ( ) { $ b u z z M o c k = \ M o c k e r y : : m o c k ( ' \ B u z z \ B r o w s e r ' ) ; $ b u z z M o c k - > s h o u l d R e c e i v e ( ' g e t ' ) - > a n d T h r o w ( ' \ B u z z \ E x c e p t i o n \ C l i e n t E x c e p t i o n ' ) ; $ l o g g e r M o c k = \ M o c k e r y : : m o c k ( ' \ M o n o l o g \ L o g g e r ' ) ; $ l o g g e r M o c k - > s h o u l d R e c e i v e ( ' i n f o ' ) - > n e v e r ( ) ; $ l o g g e r M o c k - > s h o u l d R e c e i v e ( ' e r r ' ) - > o n c e ( ) ; ( n e w \ A p i C o n s u m e r ( $ b u z z M o c k , $ l o g g e r M o c k ) ) - > g e t C u r r e n t M o o d ( ) ; } }
33. We providing API for external consumers it can't be tested...
34. Use Symfony's WebTestCase to test your API I call it integration tests
35. Call your API and check if it returns prepared data c l a s s E x p e n d i t u r e C o n t r o l l e r T e s t e x t e n d s W e b T e s t C a s e { u s e I s o l a t e d T e s t s T r a i t ; / / i t r e s e t s t e s t e n v i r o n m e n t p u b l i c f u n c t i o n t e s t G e t L i s t I n J s o n ( ) { $ c l i e n t = s t a t i c : : c r e a t e C l i e n t ( ) ; $ c l i e n t - > r e q u e s t ( ' G E T ' , ' / e x p e n d i t u r e s . j s o n ' ) ; $ j s o n = j s o n _ d e c o d e ( $ c l i e n t - > g e t R e s p o n s e ( ) - > g e t C o n t e n t ( ) ) ; $ t h i s - > a s s e r t T r u e ( $ c l i e n t - > g e t R e s p o n s e ( ) - > i s S u c c e s s f u l ( ) ) ; $ t h i s - > a s s e r t C o u n t ( 8 0 , $ j s o n ) ; $ t h i s - > a s s e r t G r e a t e r T h a n O r E q u a l ( n e w \ D a t e T i m e ( $ j s o n [ 7 9 ] - > c r e a t e d _ a t ) , n e w \ D a t e T i m e ( $ j s o n [ 0 ] - > c r e a t e d _ a t ) ) ; } }
36. Use fixtures and reset environment IsolatedTestsTrait should do the trick
37. Steps required to effectively run in isolation 1. configure PDO SQLite in file 2. create database 3. drop schema 4. load fixtres 5. copy database as a backup 6. copy database from backup for every test 7. delete database backup after test suite
38. Requirements changes frequently we can't keep up with unit tests
39. Use behavioral approach Behat in PHP
40. Describe features as scenarios It will be readable for: business, developers and machines
41. Perfectly fits into Agile process
42. Simple example F e a t u r e : l o o k f o r a j o b I n o r d e r t o f i n d c o o l j o b A s a n a s p i r i n g p r o g r a m m e r I n e e d t o b e a b l e t o l i s t j o b o f f e r s S c e n a r i o : l i s t o f f e r s f o r P L v e r s i o n G i v e n I a m o n " / " T h e n I s h o u l d s e e " D o ł ą c z d o t e a m u " A n d c l i c k " D o ł ą c z d o t e a m u " T h e n I s h o u l d b e o n " / k a r i e r a " A n d I s h o u l d s e e " P H P S e n i o r D e v e l o p e r ( G l i w i c e ) " S c e n a r i o : n o o f f e r s f o r E N s i t e G i v e n I a m o n " / e n " T h e n I s h o u l d n o t s e e " D o ł ą c z d o t e a m u " A n d I s h o u l d n o t s e e a n " # j o i n - u s " e l e m e n t
43. Our tests are slow!
44. This could be true ...
45. Always use in-memory sqlite database Or create clean sqlite database and copy it for every test
46. Group your tests Use PHPUnit's @group or Behat's @tag
47. Create "smoke tests" groups Those should be fast test, which ensures your system is most likely stable
48. Slower tests should run during night
49. Facts
50. It gives you confidence about changes and your code
51. Team is able to rapidly experiment with code
52. TDD enforces better Object Oriented design Smaller units of code and lower coupling always leads to better understanding of the codebase by future devs
53. End user experiences better quality lower 500 error ratio in the production
54. Happier users ↓ more $$$ in future
55. Conclusion
56. Setting up an working environment for automated tests is timely costly at the begining, but it pays off in the future
57. System without any kind of automated tests has big potential to be a big ball of mud
58. Good coverage can be easily achieved with mix of unit, functional and behavioral tests
59. You need to build and cultivate TDD culture in your surrounding
60. Thank you so much for attending! Feedback is much appreciated wojciech@sznapka.pl Twitter: @sznapka GitHub: @wowo
61. Join team :-) XSolve