Testing-Darwinismus

Agiler Stammtisch Frankfurt 2016

Transcript

1. 1 32. Agiler Stammtisch Frankfurt T esting -Darwinismus Code Defekte mit Mutation T esting aufdecken Qaiser Abbasi
2. 2 . 1 T oday ... Who is who ? Mutation T esting Lean Coffee Panel Pi zza :) Feedback
3. 3 . 1 A long time ago in a galaxy far , far away ...
4. 3 . 2 The untested side effect p u b l i c s t a t i c S t r i n g f o o ( b o o l e a n b ) { i f ( b ) { p e r f o r m V i t a l l y I m p o r t a n t B u s i n e s s F u n c t i o n ( ) ; r e t u r n " O K " ; } r e t u r n " F A I L " ; } @ T e s t p u b l i c v o i d s h o u l d F a i l W h e n G i v e n F a l s e ( ) { a s s e r t E q u a l s ( " F A I L " , f o o ( f a l s e ) ) ; } @ T e s t p u b l i c v o i d s h o u l d B e O k W h e n G i v e n T r u e ( ) { a s s e r t E q u a l s ( " O K " , f o o ( t r u e ) ) ; }
5. 3 . 3 The missing boundary test p u b l i c s t a t i c S t r i n g f o o ( i n t i ) { i f ( i > = 0 ) { r e t u r n " f o o " ; } e l s e { r e t u r n " b a r " ; } } @ T e s t p u b l i c v o i d s h o u l d R e t u r n B a r W h e n G i v e n 1 ( ) { a s s e r t E q u a l s ( " b a r " , f o o ( 1 ) ) ; } @ T e s t p u b l i c v o i d s h o u l d R e t u r n F o o W h e n G i v e n M i n u s 1 ( ) { a s s e r t E q u a l s ( " f o o " , f o o ( - 1 ) ) ; }
6. 3 . 4 The myopic mockist p u b l i c s t a t i c S t r i n g f o o ( C o l l a b o r a t o r c , b o o l e a n b ) { i f ( b ) { r e t u r n c . p e r f o r m A c t i o n ( ) ; } r e t u r n " F O O " ; } @ T e s t p u b l i c v o i d s h o u l d P e r f o r m A c t i o n W h e n G i v e n T r u e ( ) { f o o ( m o c k C o l l a b o r a t o r , t r u e ) ; v e r i f y ( m o c k C o l l a b o r a t o r ) . p e r f o r m A c t i o n ( ) ; } @ T e s t p u b l i c v o i d s h o u l d N o t P e r f o r m A c t i o n W h e n G i v e n F a l s e ( ) { f o o ( m o c k C o l l a b o r a t o r , f a l s e ) ; v e r i f y ( n e v e r ( ) , m o c k C o l l a b o r a t o r ) . p e r f o r m A c t i o n ( ) ; }
7. 3 . 5 T ry to describe the very nature of the problem here
8. 3 . 6 How do we ensure that our test suite copes well with the relevant stuff ?
9. 4 . 1 Mutation T esting Idea : Mutate the code and the tests should fail
10. 5 . 1 Pitest – „Real world mutation testing “ pitest .org Most popular framework for Java Active development & support Works with Ant , Maven , Gradle and others Human -readable reports containing line and mutation coverage information
11. 5 . 2 Mutators Apply mutation operators to the byte code
12. 5 . 3 Conditionals Boundary Mutator i f ( i > = 0 ) { r e t u r n " f o o " ; } e l s e { r e t u r n " b a r " ; } mutated into i f ( i > 0 ) { r e t u r n " f o o " ; } e l s e { r e t u r n " b a r " ; }
13. 5 . 4 Mutants Java classes containing the injected fault
14. 5 . 5 Available Mutators http ://pitest .org /quickstart /mutators / Conditionals Boundary Mutator Increments Mutator Invert Negatives Mutator Math Mutator Return Values Mutator Void Method Calls Mutator ...
15. 5 . 6 Return Values Mutator p u b l i c O b j e c t f o o ( ) { r e t u r n n e w O b j e c t ( ) ; } mutated into p u b l i c O b j e c t f o o ( ) { n e w O b j e c t ( ) ; r e t u r n n u l l ; }
16. 5 . 7 Void Method Calls Mutator p u b l i c v o i d s o m e V o i d M e t h o d ( i n t i ) { / / s e r i o u s b u s i n e s s l o g i c } p u b l i c i n t f o o ( ) { i n t i = 5 ; d o S o m e t h i n g ( i ) ; r e t u r n i ; } mutated into p u b l i c v o i d s o m e V o i d M e t h o d ( i n t i ) { / / s e r i o u s b u s i n e s s l o g i c } p u b l i c i n t f o o ( ) { i n t i = 5 ; r e t u r n i ; }
17. 5 . 8 Running The T ests Attack the suspicious ones !
18. 6 . 1 Summing up T raditional test coverage only addresses which code is executed By contrast , Mutation testing detects whether the code is meaningfully tested
19. 6 . 2 Thank you for your attention ! Any questions left ? Slides @github .com /qabbasi