$30 off During Our Annual Pro Sale. View Details »

Introduction to Visual Regression Testing

Yosuke Kurami
November 25, 2017

Introduction to Visual Regression Testing

For nodefest 17.
How introduce Visual Regression Testing into out products.

Yosuke Kurami

November 25, 2017
Tweet

More Decks by Yosuke Kurami

Other Decks in Programming

Transcript

  1. Introduction to
    Visual Regression Testing
    #nodefest ‘17

    View Slide

  2. 5PEBZ`TUIFNF

    View Slide

  3. )PXXFJOUSPEVDF
    l7JTVBM3FHSFTTJPO5FTUz
    UPPVSQSPEVDU 41"

    View Slide

  4. 8IZ7JTVBM3FHSFTTJPO

    View Slide

  5. 1. How assert components ?
    w *Gtype: B XIBULJOEQSPQFSUZEPZPVBTTFSU 

    nodeName or textContent, classList



    Good evening, {{name}}.


    Bon soir, {{name}}.

    Other...


    View Slide

  6. 1. How assert DOM ?
    w 0WFSBTTFSUJPO-PXNBJOUBJOBCJMJUZʜ

    'FXBTTFSUJPO6OFBTJOFTTʜ
    w 4P l4OBQTIPUUFTUJOHz
    ƭ
    The latest snapshot (DOM String)
    ƭ
    The actual snapshot (DOM String)
    Update if snapshot is accepted
    Assertion

    View Slide

  7. 2. We want to test Styles
    w 0VSVTFSTBSFOPUJOUFSFTUFEJO$44DMBTTOBNF
    .hoge {
    padding: 15px;
    color: #808080;
    border: 1px solid currentColor;
    }

    ABCDEF

    JOUFSFTUFE
    OPUJOUFSFTUFE

    View Slide

  8. That is question:

    To create or not to create
    w *OJUJBMMZXFUSJFETPNFTFSWJDFTPSUPPMT

    )PXFWFSUIFSFXBTOPUXIBUXFXBOU
    w 8FEFDJEFEUPEFWFMPQPVSl7JTVBM3FHSFTTJPO
    5FTUJOHzTZTUFN

    View Slide

  9. Divide into smaller problems
    w $BQUVSFTOBQTIPUT TDSFFOTIPUJNBHFT

    w DPOTJEFSJOHJOUPl)PXJNQMFNFOUBOEUFTUz
    w $PNQBSFTOBQTIPUT TDSFFOTIPUJNBHFT

    w DPOTJEFSJOHJOUPl%FWFMPQNFOUXPSLqPXz

    View Slide

  10. $BQUVSFTOBQTIPUT

    View Slide

  11. Which tools do we use?

    View Slide

  12. (e.g.) Stack of our product
    'PSFYBNQMF
    "QQ'SBNFXPSL "OHVMBS
    5FTU'SBNFXPSL +BTNJOF
    5FTU3VOOFS IBOENBEFUPPM

    #SPXTFS .BOJQVMBUPS
    1VQQFUFFS
    $POUJOVPVT*OUFHSBUJPO 0SBDMF8FSDLFS$*

    View Slide

  13. 5IFSFBSFWBSJPVTXBZT
    8FEPOPUJOUSPEVDFNFUIPETJOTQFDJpDMJCSBSJFT

    View Slide

  14. 5JQTGPSBVUPNBUFEDBQUVSF
    JOEFQFOEFOUPGUPPMT

    View Slide

  15. 1. Web browsers in your CI env
    1MFBTFSFGFSXIFOZPVDSFBUF%PDLFS*NBHFT

    w *GZPVXBOUVTFlSFBMzXFCCSPXTFST
    w :PVOFFEYWGC 9XJOEPX7JSUVBM'SBNF#V⒎FS

    w *GZPVXBOUUPVTF$ISPNFXJUIlIFBEMFTT
    w :PVOFFEYDC 9QSPUPDPM$MBOH#JOEJOHT

    View Slide

  16. 2. □□□□□□□□□
    ˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘
    $*˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘
    ˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘˘
    ˘˘˘˘˘˘˘˘.JH.JY˘˘˘˘˘˘˘˘˘

    View Slide

  17. 2. ϑΥϯτೖΕ͓ͯ͘
    ϚϧνόΠτݍΛλʔήοτʹ։ൃ͍ͯ͠ΔͷͰ͋Ε͹
    $*ʹ΋ϑΥϯτΛೖΕ͓͖ͯ·͠ΐ͏ɻ
    ં֯ͷΩϟϓνϟը૾͕౾෗ͩΒ͚ʹͳΓ·͢ɻ
    ͏ͪͷνʔϜͰ͸.JH.JYϑΥϯτΛ࢖͍ͬͯ·͢ɻ

    View Slide

  18. 3. Avoid “false positive”
    5PHVBSBOUFFl4BNFTPVSDF TBNFJNBHFz
    w 4VQQSFTT$44"OJNBUJPO
    w &YFDVUFDBQUVSJOHBTMBUFBTQPTTJCMF
    * {
    transition: none !important;
    animation: none !important;
    }
    requestIdleCallback(() => screenshot(), { timeout: 30000 });

    View Slide

  19. $PNQBSFTOBQTIPUT

    View Slide

  20. We’ve released a tool
    https://reg-viz.github.io/reg-suit/

    View Slide

  21. reg-suit features
    SFHTVJUJOUFHSBUFTUIFGPMMPXJOHUBTLT
    w %FUFDUEJ⒎FSFODF
    w )FMQUPWJFXEJ⒎FSFODF
    w 6QEBUFTOBQTIPUT

    View Slide

  22. Difference is not necessarily
    failure
    SFHTVJUQSPWJEFTUIJTqPX

    View Slide

  23. %FNP

    View Slide

  24. Demo 1. Push ~ Detection

    View Slide

  25. Demo 2. Notify to reviewers

    View Slide

  26. Demo 3. Review the difference

    View Slide

  27. Integration with GitHub flow

    View Slide

  28. (almost all)
    Built with JavaScript
    w $-* /PEFKT

    w $PNQBSFJNBHFTQOHKTQJYFMNBUDI
    w 4UPSFJNBHFT"844%,
    w (JU)VC"QQ "84-BNCEB

    w "QPMMP(SBQI2-$MJFOU
    w $PNQBSJTPO3FQPSU 8FC#SPXTFS

    w 'SPOUFOE7VFKT
    w %J⒎FSFODFNBSLFST0QFO$7 8FC"TTFNCMZ

    View Slide

  29. 4VNNBSZ

    View Slide

  30. Summary
    w "EWBOUBHFPGWJTVBMUFTUJOH
    w &OIBODFUIFNPUJWBUJPOUPXSJUFUFTUDPEFT
    w 3FEVDFUIFMPBEPGSFWJFXFST
    w SFHTVJUJTTUJMMJODPNQMFUFʜ
    w 8F`SFXFMDPNFZPVSGFFECBDLBOE13T

    View Slide

  31. Thank you !
    by @Quramy

    View Slide