Writing tests that less sucks

Introduction to how respec-given and it's Given/When/Then syntax can lead to better Javascript tests(specs).

Transcript

1. Writing tests that less sucks
2. Motivation Justin Searls' talk How to stop hating your tests RubyConf 2015 - How to Stop Hating your Test Suite by Justi... 8:42 ‒ 11:14: What test should do 13:54 ‒ 15:49: Large v.s. Small API
3. Test code is code quality matters need maintainance if not carefully designed, easy to become hard to read lots of duplication
4. Test code should only do 3 things Arrange Act Assert
5. rspec‒given (a ruby gem by Jim Weirich) Strongly opinionated on How to write a test self documented test code Natural assertion expose API quality clearly AAA style Given/When/Then effectively reduce duplication Lazy Given execution order
6. let's take a look at API
7. XUnit TestSuite TestCase rspec/jasmine/mocha d e s c r i b e / c o n t e x t b e f o r e / b e f o r e E a c h / a f t e r / a f t e r E a c h i t / s p e c i f y l e t / l e t ! (powerful feature, rspec only) rspec‒given d e s c r i b e / c o n t e x t G i v e n / G i v e n ! / W h e n / T h e n / A n d / I n v a r i a n t
8. GWT's responsibility Given preparation When resolve action to result if error occurs, capture it & assign to result handle async (in JS) Then a T h e n forms a test case verify result And/Invariant minor verifacation
9. not only syntax sugar conscious testing API force AAA style 1 line per action not repeat action in English again
10. How about JavaScript?
11. respec‒given an test framework extension that encourages cleaner, readable, and maintainable specs respec‒given (mocha) respec‒given‒lab (Lab)
12. it's a tribute rspec‒given by Jim Weirich jasmine‒given by Justin Searls mocha‒given mocha‒gwt given.js power‒assert‒js
13. Example #1 AAA style
14. bdd # f o r s i m p l i c i t y w e s k i p a s y n c s t u f f d e s c r i b e ' a d s e r v e r c a m p a i g n l i s t A P I ' , ‐ > i t ' s h o u l d w o r k s ' , ‐ > r o u t e = ' / v 3 / c a m p a i g n s ' # A r r a n g e r e s p = r e q u e s t . p o s t a p i _ h o s t + r o u t e # A c t e x p e c t ( r e s p . s t a t u s C o d e ) . t o . b e ( 2 0 0 ) # A s s e r t given d e s c r i b e ' a d s e r v e r c a m p a i g n l i s t A P I ' , ‐ > G i v e n r o u t e : ‐ > ' / v 3 / c a m p a i g n s ' # A r r a n g e W h e n r e s p : ‐ > r e q u e s t . p o s t a p i _ h o s t + @ r o u t e # A c t T h e n ‐ > @ r e s p . s t a t u s C o d e = = 2 0 0 # A s s e r t
15. Example #2 Override behavior at later stage
16. before jump into code we need to understand the Execution Order
17. in BDD 1. all b e f o r e of this block (when enter a block) 2. all b e f o r e E a c h of all block 3. i t 4. all a f t e r E a c h of all block 5. all a f t e r of this block (when exit a block) in Given 1. all G i v e n and G i v e n I of all block 2. all W h e n of all block 3. all I n v a r i a n t of all block 4. T h e n 5. all A n d of this block 6. all C l e a n u p of all block (respec‒given only)
18. Override behavior at later stage bdd style (works) d e s c r i b e ' a d s e r v e r A P I ' , ‐ > r e s p = n u l l r o u t e = ' ' b e f o r e E a c h ‐ > r e s p = r e q u e s t . p o s t a p i _ h o s t + r o u t e d e s c r i b e ' c a m p a i g n l i s t A P I ' , ‐ > b e f o r e ‐ > r o u t e = ' / v 3 / c a m p a i g n s ' i t ' w o r k s ' , ‐ > e x p e c t ( r e s p . s t a t u s C o d e ) . t o . b e ( 2 0 0 ) i t ' w o r k s ' , ‐ > e x p e c t ( r e s p . b o d y ) . t o . b e . a n ( ' a r r a y ' )
19. Override behavior at later stage bdd style (broken) d e s c r i b e ' a d s e r v e r A P I ' , ‐ > r e s p = n u l l r o u t e = ' ' b e f o r e E a c h ‐ > r e s p = r e q u e s t . p o s t a p i _ h o s t + r o u t e a f t e r E a c h ‐ > # r e s p = n u l l # r o u t e = ' ' # r e s e t r o u t e a f t e r E V E R Y t e s t d e s c r i b e ' c a m p a i g n l i s t A P I ' , ‐ > b e f o r e ‐ > r o u t e = ' / v 3 / c a m p a i g n s ' i t ' w o r k s ' , ‐ > # t r u e , i t w o r k s . e x p e c t ( r e s p . s t a t u s C o d e ) . t o . b e ( 2 0 0 ) i t ' w o r k s ' , ‐ > # n o . i t b r e a k s . e x p e c t ( r e s p . b o d y ) . t o . b e . a n ( ' a r r a y ' )
20. How about don't reset? then everything is OK
21. Not really. we don't want to leave dirty state to next test. and sometimes reset after each test is unavoidable reset testdouble library or dependencies
22. Override behavior at later stage (given) d e s c r i b e ' a d s e r v e r A P I ' , ‐ > W h e n r e s p : ‐ > r e q u e s t . p o s t a p i _ h o s t + r o u t e d e s c r i b e ' c a m p a i g n l i s t A P I ' , ‐ > G i v e n r o u t e : ‐ > ' / v 3 / c a m p a i g n s ' T h e n ‐ > r e s p . s t a t u s C o d e = = 2 0 0 # O K T h e n ‐ > A r r a y . i s A r r a y ( r e s p . b o d y ) # O K
23. How Given solve this problem execution order use t h i s as context object every test case has it's own fresh context object no state sharing no pollution C l e a n u p same as a f t e r E a c h silent ignore error
24. warning: = > is not f u n c t i o n note ES6 arrow function will bound outer t h i s so you cannot access context object inside = >
25. Example #3 not repeat action in English again e.g, Naturall Assertion
26. bdd i t ' o n e p l u s o n e e q u a l s t w o ' , ‐ > e x p e c t ( 1 + 1 ) . t o . e q u a l 2 given T h e n ‐ > 1 + 1 = = 2
27. What a report looks like?
28. bdd ✓ o n e p l u s o n e e q u a l s t w o 1 p a s s i n g ( 9 m s ) given ✓ T h e n { 1 + 1 = = = 2 } 2 p a s s i n g ( 1 2 m s )
29. Real test report from AdServer G E T / v 3 / c a m p a i g n s / : a d _ i d w i t h d e f a u l t ( T o k e n ‐ v 1 ) a u t h m e t h o d ✓ T h e n { t h i s . s t a t u s = = = 2 0 0 } ✓ T h e n { t y p e o f t h i s . b o d y = = = ' o b j e c t ' } ✓ T h e n { t y p e o f t h i s . b o d y . a d _ i d = = = ' s t r i n g ' } ✓ T h e n { t y p e o f t h i s . b o d y . l a n g = = = ' s t r i n g ' } ✓ T h e n { t y p e o f t h i s . b o d y . o r i e n t a t i o n = = = ' s t r i n g ' } ✓ T h e n { t y p e o f t h i s . b o d y . s t o r e _ u r l = = = ' u n d e f i n e d ' } b u t l a c k o f r e q u i r e d h e a d e r V M 5 ‐ A P I ‐ T o k e n ✓ T h e n { t h i s . s t a t u s = = = 4 0 1 } ✓ T h e n { t h i s . b o d y . e r r o r = = = t r u e } ✓ T h e n { / a u t h f a i l e d / . t e s t ( t h i s . b o d y . m e s s a g e ) } b u t a d i s e x p i r e d ✓ T h e n { t h i s . s t a t u s = = = 4 0 4 } ✓ T h e n { t h i s . b o d y . e r r o r }
30. And what about error report? judge an assertion library by its error message, not its API ‒‒ Justin Searls good error message make debug easier
31. rspec‒given's error message is nice. It evaluates every sub‒ expression for you! 1 ) t e s t T h e n { 1 + 1 = = 3 } F a i l u r e / E r r o r : T h e n { 1 + 1 = = 3 } T h e n e x p r e s s i o n f a i l e d a t s p e c / t e s t _ s p e c . r b : 1 3 e x p e c t e d : 2 t o e q u a l : 3 f a l s e < ‐ 1 + 1 = = 3 2 < ‐ 1 + 1
32. But I found power‒assert‒js is even better: A s s e r t i o n E r r o r : # a t l i n e : 4 a s s e r t ( 1 + 1 = = = 3 ) | | 2 f a l s e [ n u m b e r ] 3 = > 3 [ n u m b e r ] 1 + 1 = > 2
33. bdd 1 ) o n e p l u s o n e e q u a l s t w o : E r r o r : e x p e c t e d 2 t o e q u a l 3 given 1 ) T h e n { 1 + 1 = = = 3 } : E x p e c t t r u t h y v a l u e b u t g o t f a l s e : T h e n { 1 + 1 = = = 3 } T h e n e x p r e s s i o n f a i l e d a t t e s t / p l u s _ s p e c . c o f f e e : 1 1 : 1 1 T h e n { 1 + 1 = = = 3 } | | 2 f a l s e e x p e c t e d : 2 t o e q u a l : 3
34. That's it. Thank you for listening. Wish all of us can write test that less sucks.
35. References How to stop hating your tests by Justin Searls rspec‒given's README & wiki tests wiki by testdouble.com testdouble.com blog respec‒given respec‒given‒lab