Automated tests - facts and myths

Automated tests - facts and myths

The discussion on automated tests is hot topic. The approach has same number of advocates and skeptics. More and more tools eases testing, but also introduces a fundamental question: what, when and how to test? Practise and experience let's answer those questions or guide in the right direction.

In this talk usage examples of unit, functional and behavioral tests will be shown. Importance of properly handling dependencies and mocking them will be discussed as well. But most of important part will be hints on how to write code, that could be tested automaticaly.

Slides are available in interactive mode here: http://tdd.sznapka.pl/

5bbc7b79e04d8e8b1212c934ff2e2831?s=128

Wojciech Sznapka

October 26, 2013
Tweet

Transcript

  1. Automated tests Facts and myths / Wojciech Sznapka @sznapka

  2. Cześć

  3. An introduction I work in software industry for about 9

    years Care a lot about robust and testable architectures Loves software craftsmanship, sophisticated architectures, Big Data and ice hockey
  4. Test Driven Development What is this?

  5. Add a test

  6. Write an implementation

  7. Refactor code

  8. Repeat!

  9. What's important here?

  10. ... to have tests

  11. You won’t go to hell if you’ll write a test

    after declaring an interface or prototyping a class ...
  12. ... but you’ll surely end up in hell, if there

    won’t be a test coverage
  13. Myths about automated tests

  14. Boss doesn't pay for automated tests

  15. It's a myth! A 2005 study found that using TDD

    meant writing more tests and, in turn, programmers who wrote more tests tended to be more productive by American Scientists
  16. But this ain't that hard ... c l a s

    s R o m a n C o n v e r t e r { p r o t e c t e d $ c o n v e r s i o n s = [ 1 0 0 0 = > ' M ' , 9 0 0 = > ' C M ' , 5 0 0 = > ' D ' , 4 0 0 = > ' C D ' , 1 0 0 = > ' C ' , 9 0 = > ' X C ' , 5 0 = > ' L ' , 4 0 = > ' X L ' , 1 0 = > ' X ' , 9 = > ' I X ' , 5 = > ' V ' , 4 = > ' I V ' , 1 = > ' I ' ] ; p u b l i c f u n c t i o n c o n v e r t ( $ i n A r a b i c ) { i f ( ! i s _ n u m e r i c ( $ i n A r a b i c ) ) { t h r o w n e w \ I n v a l i d A r g u m e n t E x c e p t i o n ( ' I c o n v e r t n u m e r i c s ' ) ; } i f ( $ i n A r a b i c < = 0 ) { r e t u r n ' ' ; } l i s t ( $ a r a b i c , $ r o m a n ) = $ t h i s - > c o n v e r s i o n F a c t o r F o r ( $ i n A r a b i c ) ; r e t u r n $ r o m a n . $ t h i s - > c o n v e r t ( $ i n A r a b i c - $ a r a b i c ) ; } p r o t e c t e d f u n c t i o n c o n v e r s i o n F a c t o r F o r ( $ i n A r a b i c ) { f o r e a c h ( $ t h i s - > c o n v e r s i o n s a s $ a r a b i c = > $ r o m a n ) { i f ( $ a r a b i c < = $ i n A r a b i c ) { r e t u r n [ $ a r a b i c , $ r o m a n ] ; } } } }
  17. ... start from obvious things c l a s s

    R o m a n C o n v e r t e r T e s t e x t e n d s P H P U n i t _ F r a m e w o r k _ T e s t C a s e { p u b l i c f u n c t i o n t e s t E m p t y ( ) { $ t h i s - > a s s e r t E q u a l s ( ' ' , ( n e w R o m a n C o n v e r t e r ) - > c o n v e r t ( ' ' ) ) ; } }
  18. ... provide some meat ... c l a s s

    R o m a n C o n v e r t e r T e s t e x t e n d s P H P U n i t _ F r a m e w o r k _ T e s t C a s e { / * * @ d a t a P r o v i d e r p r o v i d e T e s t D a t a * / p u b l i c f u n c t i o n t e s t C o n v e r s i o n s ( $ a r a b i c , $ r o m a n ) { $ c o n v e r t e r = n e w R o m a n C o n v e r t e r ( ) ; $ t h i s - > a s s e r t E q u a l s ( $ r o m a n , $ c o n v e r t e r - > c o n v e r t ( $ a r a b i c ) ) ; } p u b l i c f u n c t i o n p r o v i d e T e s t D a t a ( ) { r e t u r n a r r a y [ [ 3 4 9 7 , ' M M M C D X C V I I ' ] , [ 1 , ' I ' ] , [ 2 , ' I I ' ] , [ 6 , ' V I ' ] , [ 9 , ' I X ' ] , [ 4 0 , ' X L ' ] , [ 4 5 , ' X L V ' ] , [ 9 0 , ' X C ' ] , [ 1 0 0 , ' C ' ] , [ 4 0 0 , ' C D ' ] ] ; } }
  19. ... test edge cases c l a s s R

    o m a n C o n v e r t e r T e s t e x t e n d s P H P U n i t _ F r a m e w o r k _ T e s t C a s e { / * * @ e x p e c t e d E x c e p t i o n \ I n v a l i d A r g u m e n t E x c e p t i o n * / p u b l i c f u n c t i o n t e s t E m p t y ( ) { ( n e w R o m a n C o n v e r t e r ) - > c o n v e r t ( ' w t f I a m p a s s i n g h e r e ' ) ) ; } }
  20. Help yourself with tests generation p h p u n

    i t - s k e l g e n - - t e s t R o m a n C o n v e r t e r
  21. ... or using Symfony's XSolveUnitSkelgenBundle . / a p p

    / c o n s o l e x s o l v e : s k e l g e n : t e s t A c m e / E x a m p l e B u n d l e / S e r v i c e / . . / a p p / c o n s o l e x s o l v e : s k e l g e n : t e s t A c m e / * / C o n t r o l l e r / *
  22. We are building too complicated system for an automated tests

  23. TDD forces you to write cleaner code

  24. It's easier to test smaller units of code

  25. so you write smaller classes which are less coupled and

    that makes your system more stable and open for an extension in the future
  26. Your Object Oriented design becomes S.O.L.I.D. compilant and this is

    awesome!
  27. You will of course tackle costly dependencies

  28. Use PHPUnit's mock framework or Mockery

  29. Fake it till you make it c l a s

    s M o c k e d T e s t e x t e n d s P H P U n i t _ F r a m e w o r k _ T e s t C a s e { p u b l i c f u n c t i o n t e s t N o n E x i s t i n g V a l u e O b j e c t s ( ) { $ c o n f i g u r a t i o n = \ M o c k e r y : : m o c k ( ' \ C o n f i g u r a t i o n V a l u e O b j e c t ' , [ ' g e t U r l ' = > ' h t t p : / / t d d . s z n a p k a . p l ' , ' g e t F o r m a t ' = > ' x m l ' ] ) ; $ t h i s - > a s s e r t E q u a l s ( ' x m l ' , $ c o n f i g u r a t i o n - > g e t F o r m a t ( ) ) ; / / O K } }
  30. Mock things that can't be tested quickly or non- reproducable

    c l a s s M o c k e d T e s t e x t e n d s P H P U n i t _ F r a m e w o r k _ T e s t C a s e { p u b l i c f u n c t i o n t e s t A p i C a l l s ( ) { $ b u z z M o c k = \ M o c k e r y : : m o c k ( ' \ B u z z \ B r o w s e r ' ) ; $ b u z z M o c k - > s h o u l d R e c e i v e ( ' g e t ' ) - > a n d R e t u r n ( ' < r e s p o n s e > < m o o d > A w e s o m i t y < / m o o d > < / r e s p o n s e > ' ) ; / / t h i s a l s o i s w i s e s o l u t i o n , t o w r i t e x m l f i x t u r e s i n f i l e / / $ b u z z M o c k - > s h o u l d R e c e i v e ( ' g e t ' ) - > o n c e ( ) / / - > a n d R e t u r n ( f i l e _ g e t _ c o n t e n t s ( _ _ D I R _ _ . ' / f i x t u r e s . x m l ' ) ) ; $ a p i = n e w \ A p i C o n s u m e r ( $ b u z z M o c k ) ; $ a p i - > a s s e r t E q u a l s ( ' A w e s o m i t y ' , $ a p i - > g e t C u r r e n t M o o d ( ) ) ; / / O K } }
  31. Expect declared behaviors c l a s s M o

    c k e d T e s t e x t e n d s P H P U n i t _ F r a m e w o r k _ T e s t C a s e { p u b l i c f u n c t i o n t e s t E x p e c t a t i o n s D e c l a r a t i o n s ( ) { $ b u z z M o c k = \ M o c k e r y : : m o c k ( ' \ B u z z \ B r o w s e r ' ) ; $ b u z z M o c k - > s h o u l d R e c e i v e ( ' g e t ' ) - > a n d R e t u r n ( ' < r e s p o n s e > < m o o d > A w e s o m i t y < / m o o d > < / r e s p o n s e > ' ) ; $ l o g g e r M o c k = \ M o c k e r y : : m o c k ( ' \ M o n o l o g \ L o g g e r ' ) ; / / w e j u s t w a n t t o b e s u r e t h a t L o g g e r : : i n f o w a s c a l l e d o n l y o n c e $ l o g g e r M o c k - > s h o u l d R e c e i v e ( ' i n f o ' ) - > o n c e ( ) ; $ a p i = n e w \ A p i C o n s u m e r ( $ b u z z M o c k , $ l o g g e r M o c k ) ; $ a p i - > a s s e r t E q u a l s ( ' A w e s o m i t y ' , $ a p i - > g e t C u r r e n t M o o d ( ) ) ; / / O K } }
  32. Be prepared for failures and check if you prepared for

    unexpected situations c l a s s M o c k e d T e s t e x t e n d s P H P U n i t _ F r a m e w o r k _ T e s t C a s e { / * * @ e x p e c t e d E x c e p t i o n \ M y E x c e p t i o n W r a p p e r * / p u b l i c f u n c t i o n t e s t F a i l e d C o n n e c t i o n ( ) { $ b u z z M o c k = \ M o c k e r y : : m o c k ( ' \ B u z z \ B r o w s e r ' ) ; $ b u z z M o c k - > s h o u l d R e c e i v e ( ' g e t ' ) - > a n d T h r o w ( ' \ B u z z \ E x c e p t i o n \ C l i e n t E x c e p t i o n ' ) ; $ l o g g e r M o c k = \ M o c k e r y : : m o c k ( ' \ M o n o l o g \ L o g g e r ' ) ; $ l o g g e r M o c k - > s h o u l d R e c e i v e ( ' i n f o ' ) - > n e v e r ( ) ; $ l o g g e r M o c k - > s h o u l d R e c e i v e ( ' e r r ' ) - > o n c e ( ) ; ( n e w \ A p i C o n s u m e r ( $ b u z z M o c k , $ l o g g e r M o c k ) ) - > g e t C u r r e n t M o o d ( ) ; } }
  33. We providing API for external consumers it can't be tested...

  34. Use Symfony's WebTestCase to test your API I call it

    integration tests
  35. Call your API and check if it returns prepared data

    c l a s s E x p e n d i t u r e C o n t r o l l e r T e s t e x t e n d s W e b T e s t C a s e { u s e I s o l a t e d T e s t s T r a i t ; / / i t r e s e t s t e s t e n v i r o n m e n t p u b l i c f u n c t i o n t e s t G e t L i s t I n J s o n ( ) { $ c l i e n t = s t a t i c : : c r e a t e C l i e n t ( ) ; $ c l i e n t - > r e q u e s t ( ' G E T ' , ' / e x p e n d i t u r e s . j s o n ' ) ; $ j s o n = j s o n _ d e c o d e ( $ c l i e n t - > g e t R e s p o n s e ( ) - > g e t C o n t e n t ( ) ) ; $ t h i s - > a s s e r t T r u e ( $ c l i e n t - > g e t R e s p o n s e ( ) - > i s S u c c e s s f u l ( ) ) ; $ t h i s - > a s s e r t C o u n t ( 8 0 , $ j s o n ) ; $ t h i s - > a s s e r t G r e a t e r T h a n O r E q u a l ( n e w \ D a t e T i m e ( $ j s o n [ 7 9 ] - > c r e a t e d _ a t ) , n e w \ D a t e T i m e ( $ j s o n [ 0 ] - > c r e a t e d _ a t ) ) ; } }
  36. Use fixtures and reset environment IsolatedTestsTrait should do the trick

  37. Steps required to effectively run in isolation 1. configure PDO

    SQLite in file 2. create database 3. drop schema 4. load fixtres 5. copy database as a backup 6. copy database from backup for every test 7. delete database backup after test suite
  38. Requirements changes frequently we can't keep up with unit tests

  39. Use behavioral approach Behat in PHP

  40. Describe features as scenarios It will be readable for: business,

    developers and machines
  41. Perfectly fits into Agile process

  42. Simple example F e a t u r e :

    l o o k f o r a j o b I n o r d e r t o f i n d c o o l j o b A s a n a s p i r i n g p r o g r a m m e r I n e e d t o b e a b l e t o l i s t j o b o f f e r s S c e n a r i o : l i s t o f f e r s f o r P L v e r s i o n G i v e n I a m o n " / " T h e n I s h o u l d s e e " D o ł ą c z d o t e a m u " A n d c l i c k " D o ł ą c z d o t e a m u " T h e n I s h o u l d b e o n " / k a r i e r a " A n d I s h o u l d s e e " P H P S e n i o r D e v e l o p e r ( G l i w i c e ) " S c e n a r i o : n o o f f e r s f o r E N s i t e G i v e n I a m o n " / e n " T h e n I s h o u l d n o t s e e " D o ł ą c z d o t e a m u " A n d I s h o u l d n o t s e e a n " # j o i n - u s " e l e m e n t
  43. Our tests are slow!

  44. This could be true ...

  45. Always use in-memory sqlite database Or create clean sqlite database

    and copy it for every test
  46. Group your tests Use PHPUnit's @group or Behat's @tag

  47. Create "smoke tests" groups Those should be fast test, which

    ensures your system is most likely stable
  48. Slower tests should run during night

  49. Facts

  50. It gives you confidence about changes and your code

  51. Team is able to rapidly experiment with code

  52. TDD enforces better Object Oriented design Smaller units of code

    and lower coupling always leads to better understanding of the codebase by future devs
  53. End user experiences better quality lower 500 error ratio in

    the production
  54. Happier users ↓ more $$$ in future

  55. Conclusion

  56. Setting up an working environment for automated tests is timely

    costly at the begining, but it pays off in the future
  57. System without any kind of automated tests has big potential

    to be a big ball of mud
  58. Good coverage can be easily achieved with mix of unit,

    functional and behavioral tests
  59. You need to build and cultivate TDD culture in your

    surrounding
  60. Thank you so much for attending! Feedback is much appreciated

    wojciech@sznapka.pl Twitter: @sznapka GitHub: @wowo
  61. Join team :-) XSolve