Testing-Darwinismus

Efa70d43d8e57623d0d5fafaeef1109f?s=47 Qaiser Abbasi
July 02, 2016
23

 Testing-Darwinismus

Agiler Stammtisch Frankfurt 2016

Efa70d43d8e57623d0d5fafaeef1109f?s=128

Qaiser Abbasi

July 02, 2016
Tweet

Transcript

  1. 1 32. Agiler Stammtisch Frankfurt T esting -Darwinismus Code Defekte

    mit Mutation T esting aufdecken Qaiser Abbasi
  2. 2 . 1 T oday ... Who is who ?

    Mutation T esting Lean Coffee Panel Pi zza :) Feedback
  3. 3 . 1 A long time ago in a galaxy

    far , far away ...
  4. 3 . 2 The untested side effect p u b

    l i c s t a t i c S t r i n g f o o ( b o o l e a n b ) { i f ( b ) { p e r f o r m V i t a l l y I m p o r t a n t B u s i n e s s F u n c t i o n ( ) ; r e t u r n " O K " ; } r e t u r n " F A I L " ; } @ T e s t p u b l i c v o i d s h o u l d F a i l W h e n G i v e n F a l s e ( ) { a s s e r t E q u a l s ( " F A I L " , f o o ( f a l s e ) ) ; } @ T e s t p u b l i c v o i d s h o u l d B e O k W h e n G i v e n T r u e ( ) { a s s e r t E q u a l s ( " O K " , f o o ( t r u e ) ) ; }
  5. 3 . 3 The missing boundary test p u b

    l i c s t a t i c S t r i n g f o o ( i n t i ) { i f ( i > = 0 ) { r e t u r n " f o o " ; } e l s e { r e t u r n " b a r " ; } } @ T e s t p u b l i c v o i d s h o u l d R e t u r n B a r W h e n G i v e n 1 ( ) { a s s e r t E q u a l s ( " b a r " , f o o ( 1 ) ) ; } @ T e s t p u b l i c v o i d s h o u l d R e t u r n F o o W h e n G i v e n M i n u s 1 ( ) { a s s e r t E q u a l s ( " f o o " , f o o ( - 1 ) ) ; }
  6. 3 . 4 The myopic mockist p u b l

    i c s t a t i c S t r i n g f o o ( C o l l a b o r a t o r c , b o o l e a n b ) { i f ( b ) { r e t u r n c . p e r f o r m A c t i o n ( ) ; } r e t u r n " F O O " ; } @ T e s t p u b l i c v o i d s h o u l d P e r f o r m A c t i o n W h e n G i v e n T r u e ( ) { f o o ( m o c k C o l l a b o r a t o r , t r u e ) ; v e r i f y ( m o c k C o l l a b o r a t o r ) . p e r f o r m A c t i o n ( ) ; } @ T e s t p u b l i c v o i d s h o u l d N o t P e r f o r m A c t i o n W h e n G i v e n F a l s e ( ) { f o o ( m o c k C o l l a b o r a t o r , f a l s e ) ; v e r i f y ( n e v e r ( ) , m o c k C o l l a b o r a t o r ) . p e r f o r m A c t i o n ( ) ; }
  7. 3 . 5 T ry to describe the very nature

    of the problem here
  8. 3 . 6 How do we ensure that our test

    suite copes well with the relevant stuff ?
  9. 4 . 1 Mutation T esting Idea : Mutate the

    code and the tests should fail
  10. 5 . 1 Pitest – „Real world mutation testing “

    pitest .org Most popular framework for Java Active development & support Works with Ant , Maven , Gradle and others Human -readable reports containing line and mutation coverage information
  11. 5 . 2 Mutators Apply mutation operators to the byte

    code
  12. 5 . 3 Conditionals Boundary Mutator i f ( i

    > = 0 ) { r e t u r n " f o o " ; } e l s e { r e t u r n " b a r " ; } mutated into i f ( i > 0 ) { r e t u r n " f o o " ; } e l s e { r e t u r n " b a r " ; }
  13. 5 . 4 Mutants Java classes containing the injected fault

  14. 5 . 5 Available Mutators http ://pitest .org /quickstart /mutators

    / Conditionals Boundary Mutator Increments Mutator Invert Negatives Mutator Math Mutator Return Values Mutator Void Method Calls Mutator ...
  15. 5 . 6 Return Values Mutator p u b l

    i c O b j e c t f o o ( ) { r e t u r n n e w O b j e c t ( ) ; } mutated into p u b l i c O b j e c t f o o ( ) { n e w O b j e c t ( ) ; r e t u r n n u l l ; }
  16. 5 . 7 Void Method Calls Mutator p u b

    l i c v o i d s o m e V o i d M e t h o d ( i n t i ) { / / s e r i o u s b u s i n e s s l o g i c } p u b l i c i n t f o o ( ) { i n t i = 5 ; d o S o m e t h i n g ( i ) ; r e t u r n i ; } mutated into p u b l i c v o i d s o m e V o i d M e t h o d ( i n t i ) { / / s e r i o u s b u s i n e s s l o g i c } p u b l i c i n t f o o ( ) { i n t i = 5 ; r e t u r n i ; }
  17. 5 . 8 Running The T ests Attack the suspicious

    ones !
  18. 6 . 1 Summing up T raditional test coverage only

    addresses which code is executed By contrast , Mutation testing detects whether the code is meaningfully tested
  19. 6 . 2 Thank you for your attention ! Any

    questions left ? Slides @github .com /qabbasi