Introduction to Visual Regression Testing

For nodefest 17.
How introduce Visual Regression Testing into out products.

Transcript

1. Introduction to
   Visual Regression Testing
   #nodefest ‘17
   

   View Slide

2. 5PEBZ`TUIFNF
   

   View Slide

3. )PXXFJOUSPEVDF
   l7JTVBM3FHSFTTJPO5FTUz
   UPPVSQSPEVDU 41"
   
   

   View Slide

4. 8IZ7JTVBM3FHSFTTJPO
   

   View Slide

5. 1. How assert components ?
   w *Gtype: B XIBULJOEQSPQFSUZEPZPVBTTFSU 

   nodeName or textContent, classList
   
   
   
   Good evening, {{name}}.
   
   
   Bon soir, {{name}}.
   
   Other...
   
   
   

   View Slide

6. 1. How assert DOM ?
   w 0WFSBTTFSUJPO-PXNBJOUBJOBCJMJUZʜ

   'FXBTTFSUJPO6OFBTJOFTTʜ
   w 4P l4OBQTIPUUFTUJOHz
   ƭ
   The latest snapshot (DOM String)
   ƭ
   The actual snapshot (DOM String)
   Update if snapshot is accepted
   Assertion
   

   View Slide

7. 2. We want to test Styles
   w 0VSVTFSTBSFOPUJOUFSFTUFEJO$44DMBTTOBNF
   .hoge {
   padding: 15px;
   color: #808080;
   border: 1px solid currentColor;
   }
   
   ABCDEF
   
   JOUFSFTUFE
   OPUJOUFSFTUFE
   

   View Slide

8. That is question:

   To create or not to create
   w *OJUJBMMZXFUSJFETPNFTFSWJDFTPSUPPMT

   )PXFWFSUIFSFXBTOPUXIBUXFXBOU
   w 8FEFDJEFEUPEFWFMPQPVSl7JTVBM3FHSFTTJPO
   5FTUJOHzTZTUFN
   

   View Slide

9. Divide into smaller problems
   w $BQUVSFTOBQTIPUT TDSFFOTIPUJNBHFT
   
   w DPOTJEFSJOHJOUPl)PXJNQMFNFOUBOEUFTUz
   w $PNQBSFTOBQTIPUT TDSFFOTIPUJNBHFT
   
   w DPOTJEFSJOHJOUPl%FWFMPQNFOUXPSLqPXz
   

   View Slide

10. $BQUVSFTOBQTIPUT
    

    View Slide

11. Which tools do we use?
    

    View Slide

12. (e.g.) Stack of our product
    'PSFYBNQMF
    "QQ'SBNFXPSL "OHVMBS
    5FTU'SBNFXPSL +BTNJOF
    5FTU3VOOFS IBOENBEFUPPM
    
    #SPXTFS .BOJQVMBUPS
    1VQQFUFFS
    $POUJOVPVT*OUFHSBUJPO 0SBDMF8FSDLFS$*
    

    View Slide

13. 5IFSFBSFWBSJPVTXBZT
    8FEPOPUJOUSPEVDFNFUIPETJOTQFDJpDMJCSBSJFT
    
    

    View Slide

14. 5JQTGPSBVUPNBUFEDBQUVSF
    JOEFQFOEFOUPGUPPMT
    
    

    View Slide

15. 1. Web browsers in your CI env
    1MFBTFSFGFSXIFOZPVDSFBUF%PDLFS*NBHFT
    
    w *GZPVXBOUVTFlSFBMzXFCCSPXTFST
    w :PVOFFEYWGC 9XJOEPX7JSUVBM'SBNF#V⒎FS
    
    w *GZPVXBOUUPVTF$ISPNFXJUIlIFBEMFTT
    w :PVOFFEYDC 9QSPUPDPM$MBOH#JOEJOHT
    
    

    View Slide

16. 2. □□□□□□□□□
    ˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘
    $*˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘
    ˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘
    ˘˘˘˘˘˘˘˘.JH.JY˘˘˘˘˘˘˘˘˘
    

    View Slide

17. 2. ϑΥϯτೖΕ͓ͯ͘
    ϚϧνόΠτݍΛλʔήοτʹ։ൃ͍ͯ͠ΔͷͰ͋Ε͹
    $*ʹ΋ϑΥϯτΛೖΕ͓͖ͯ·͠ΐ͏ɻ
    ં֯ͷΩϟϓνϟը૾͕౾෗ͩΒ͚ʹͳΓ·͢ɻ
    ͏ͪͷνʔϜͰ͸.JH.JYϑΥϯτΛ࢖͍ͬͯ·͢ɻ
    

    View Slide

18. 3. Avoid “false positive”
    5PHVBSBOUFFl4BNFTPVSDF TBNFJNBHFz
    w 4VQQSFTT$44"OJNBUJPO
    w &YFDVUFDBQUVSJOHBTMBUFBTQPTTJCMF
    * {
    transition: none !important;
    animation: none !important;
    }
    requestIdleCallback(() => screenshot(), { timeout: 30000 });
    

    View Slide

19. $PNQBSFTOBQTIPUT
    

    View Slide

20. We’ve released a tool
    https://reg-viz.github.io/reg-suit/
    

    View Slide

21. reg-suit features
    SFHTVJUJOUFHSBUFTUIFGPMMPXJOHUBTLT
    w %FUFDUEJ⒎FSFODF
    w )FMQUPWJFXEJ⒎FSFODF
    w 6QEBUFTOBQTIPUT
    

    View Slide

22. Difference is not necessarily
    failure
    SFHTVJUQSPWJEFTUIJTqPX
    

    View Slide

23. %FNP
    

    View Slide

24. Demo 1. Push ~ Detection
    

    View Slide

25. Demo 2. Notify to reviewers
    

    View Slide

26. Demo 3. Review the difference
    

    View Slide

27. Integration with GitHub flow
    

    View Slide

28. (almost all)
    Built with JavaScript
    w $-* /PEFKT
    
    w $PNQBSFJNBHFTQOHKTQJYFMNBUDI
    w 4UPSFJNBHFT"844%,
    w (JU)VC"QQ "84-BNCEB
    
    w "QPMMP(SBQI2-$MJFOU
    w $PNQBSJTPO3FQPSU 8FC#SPXTFS
    
    w 'SPOUFOE7VFKT
    w %J⒎FSFODFNBSLFST0QFO$7 8FC"TTFNCMZ
    
    

    View Slide

29. 4VNNBSZ
    

    View Slide

30. Summary
    w "EWBOUBHFPGWJTVBMUFTUJOH
    w &OIBODFUIFNPUJWBUJPOUPXSJUFUFTUDPEFT
    w 3FEVDFUIFMPBEPGSFWJFXFST
    w SFHTVJUJTTUJMMJODPNQMFUFʜ
    w 8F`SFXFMDPNFZPVSGFFECBDLBOE13T
    

    View Slide

31. Thank you !
    by @Quramy
    

    View Slide