VRT in Action

VRT in Action

About VRT(Visual Regression Testing)

893f54413c2bd9ba41d11d753aacaf2c?s=128

Yosuke Kurami

November 16, 2019
Tweet

Transcript

  1. 6.

    How to test views? w ͜ͷ6*ͷςετΛॻ͍ͯΈΑ͏ʂ <template> <div v-if="user"> <span

    class="user-name">{{user.name}}</span> <img :src="user.avatarUrl" /> </div> <div v-else> <button>Signup</button> </div> </template>
  2. 7.

    How to test views? w ͜ͷ6*ͷςετΛॻ͍ͯΈΑ͏ʂ <template> <div v-if="user"> <span

    class="user-name">{{user.name}}</span> <img :src="user.avatarUrl" /> </div> <div v-else> <button>Signup</button> </div> </template> DMBTT͕VTFSOBNFͰ͋Δ͜ͱ VTFS໊͕಺ૠ͞Ε͍ͯΔ͜ͱ 63-͕όΠϯυ͞Ε͍ͯΔ͜ͱ
  3. 8.

    How to test views? w ͜ͷ6*ͷςετΛॻ͍ͯΈΑ͏ʂ <template> <div v-if="user"> <span

    class="user-name">{{user.name}}</span> <img :src="user.avatarUrl" /> </div> <div v-else> <button>Signup</button> </div> </template> DMBTT͕VTFSOBNFͰ͋Δ͜ͱ VTFS໊͕಺ૠ͞Ε͍ͯΔ͜ͱ 63-͕όΠϯυ͞Ε͍ͯΔ͜ͱ VTFS͕OVMMͷͱ͖ɺCVUUPOλά͕ඳը͞Ε͍ͯΔ͜ͱ JNHλάͰ͋Δ͜ͱ EJWλάͰ͋Δ͜ͱ TQBOλάͰ͋Δ͜ͱ ΩϦ͕ແ͍ʂ
  4. 9.

    Problems w Ͳ͜·ͰΞαʔγϣϯ͢Ε͹͍͍ͷʁ w UFYUOPEF ཁૉ໊ $44DMBTT໊΍ଐੑ஋ FUD w ΞαʔγϣϯΛॆ࣮ͤ͞Δͱ

    w ը໘࢓༷͕มΘͬͨΒΞαʔγϣϯ͕յΕͯϝϯς͠ΜͲ͍ ʮ7JFXͷࣗಈςετ͸ίεύѱ͍͔Βը໘ͷςετ͸ਓྗͰʂʯ ͱͳΔͱ࠷ѱ
  5. 11.

    Pros/Cons w 1SPT w ಋೖ͕༰қɻ+FTUͳͲʹ͸ඪ४૷උ͞Ε͍ͯΔ w $POT w จࣈྻͰςετ͢Δ͜ͱʹҙຯ͕ͲΕ͚ͩ͋Δͷ͔ʁ w

    ͦ΋ͦ΋ΤϯυϢʔβʔ͸$44DMBTT໊ʹڵຯ͕͋Δ༁Ͱ͸ͳ͍ w ϑϩϯτΤϯυͷ੹຿͸6*ͷఏڙͷ͸ͣͳͷʹɺݟͨ໨Λςετ͠ͳͯ͘Α͍ͷ͔ʁ
  6. 15.

    What should I do? w ెखۭݓͰ735ʹऔΓ૊Ήͷ͸݁ߏେมɻɻɻ w ͦ΋ͦ΋ࡢࠓͷϑϩϯτͷελοΫ͕ෳࡶ w 6*ϥΠϒϥϦ3FBDU

    "OHVMBS 7VFKT FUD w ςεςΟϯάϥΠϒϥϦ4FMFOJVN $ZQSFTTJP ,BSNB +FTU "WB +BTNJOF  .PDIB FUD w $*5SBWJT$* $JSDMF$* 8FSDLFS$* (JU)VC"DUJPOT (JUMBC$* FUD w 735ݕ౼࣌ɺࣗ෼ͨͪͷ։ൃελοΫʹ޲͍ͨπʔϧ͕ແ͔ͬͨͨΊɺࣗ࡞͢Δ͜ͱʹ
  7. 19.

    Weapon of choice w େ͖͘෼͚ͯ௨Γ w ࣮ΞϓϦέʔγϣϯΛΩϟϓνϟ͢Δύλʔϯ w 4FMFOJVN΍$ZQSFTTJPɺ#SPXTFS4UBDLͳͲͰࣗಈԽεΫϦϓτΛ༻ҙʢ&&ʣ w

    ը໘ͷύʔπຖʹΩϟϓνϟ͍ͯ͘͠ύλʔϯ w 1VQQFUFFS΍,BSNBɺ/JHIUNBSFͱ֤ςεςΟϯάϥΠϒϥϦΛ૊Έ߹Θ͍ͤͯ͘ w 4UPSZCPPL
  8. 20.

    Why Storybook? w 4UPSZCPPLͷར఺ w 3FBDU 7VFKT "OHVMBS 3/ͳͲɺओཁͳϑϨʔϜϫʔΫ͕αϙʔτ͞Ε͍ͯΔ w

    4UPSZTIPUTʢ4UPSZCPPLʹ͓͚Δ%0.จࣈྻϕʔεͷεφοϓγϣοτςετπʔϧʣ ͱͷซ༻΋Մೳ w ςετҎ֎Ͱ໾ʹཱͭγʔϯ͕ଟ͍ಋೖͷ߹ҙΛಘ΍͍͢ w ը໘ͷΧλϩάͱͯ͠ར༻Մೳ w ).3͕ޮ͘ͷͰɺ6*։ൃͦΕࣗମʹ΋ศར
  9. 24.

    Suppress false positive w 'BMTFQPTJUJWFʢِཅੑޡݕग़ʣͱ͸ w ຊདྷਖ਼͍͠΋ͷΛؒҧ͍ͱͯ͠ѻͬͯ͠·͏͜ͱɻཁ͸ΦΦΧϛগ೥ w ʮιʔείʔυΛม͍͑ͯͳ͍ͷʹΩϟϓνϟը૾ͷൺֱ࣌ʹࠩ෼͕ݕग़͞Εͯ͠·͏ʯ Λແ͘͞ͳͯ͘͸ͳΒͳ͍

    w จࣈྻϕʔεͷ4OBQTIPUςετͱൺֱͨ͠ͱ͖ʹɺ࣮ϒϥ΢βΛ࢖͏735ͷ೉қ౓͕ߴ ͘ͳͬͯ͠·͏ཧ༝ͷͭ w 4UPSZDBQͰ͸1VQQFUFFSͷ"1*Λ࢖ͬͯɺGBMTFQPTJUJWFΛ๷͙࢓૊ΈΛೖΕ͍ͯΔ
  10. 25.

    1. Turn off animation w $44Ξχϝʔγϣϯ͸ڧ੍ఀࢭ w +4ʹΑΔΞχϝʔγϣϯʢFHIJHIDIBSUT 3PUUJFͳͲʣ͸ϢʔβʔଆͰ੾Γସ͑ͯ΋Β ͏͔͠ͳ͍

    /* animation-off.css */ *, *::before, *::after { transition: none !important; animation: none !important; } page.addStyleTags({ path: 'animation-off.css' });
  11. 26.

    2. Wait for assets loaded w ը૾΍ϑΥϯτͳͲͷϦιʔεಡΈࠐΈ׬ྃલ
 ʹΩϟϓνϟͯ͠͠·͏ͱɺGBMTFQPTJUJWFʹͳͬͯ͠·͏ w page.on('request')Λ؂ࢹ

    w request.resourceType()Ͱ$ISPNF͕൑ผͨ͠Ϧιʔεछผ͕ࣄલʹΘ͔Δ w ಛఆͷϦιʔελΠϓʢJNBHFͳͲʣʹ͍ͭͯɺͦͷϦΫΤετ͕׬͔ྃͨ͠Ͳ͏͔Λݕ ূ͔ͯ͠ΒɺTDSFFOTIPUΛ࣮ߦ͢ΔΑ͏ʹ͍ͯ͠Δ w ϦιʔελΠϓΛݟ͓͔ͯͳ͍ͱɺTFSWFSTFOUFWFOUTʢFH4UPSZCPPLͷ).3ʣ ͳͲɺʮ׬ྃ͠ͳ͍ʯϦΫΤετΛ؂ࢹ͠ଓ͚Δ͜ͱʹͳΔ
  12. 27.

    3. Wait til metrics stabled w .FUSJDT"1*ΛͰ$ISPNFSFOEFSJOHQJQFMJOFʹ͓͚Δॲཧ࣮ߦճ਺͕ܭଌ w ௚ۙͷϑϨʔϜͱଌఆ஋Λൺֱͯࠩ͠෼͕ແ͘ͳ͔ͬͯΒTDSFFOTIPUΛ࣮ߦ͢ΔΑ͏ʹ ͍ͯ͠Δ


    ʢ྘৭ͷ෦෼͕ఆৗ͔Ͳ͏͔Λ஌Δज़͕ͳ͍ͨΊɺͪΐͬͱෆ׬શʣ const { Nodes, RecalcStyleCount, LayoutCount } = await page.metrics();
  13. 40.
  14. 41.

    Maintainers w SFHWJ[ͷϝϯόʔͱओͳ୲౰ൣғ w !2VSBNZSFHTVJU 4UPSZDBQWͳͲ w !CPLVXFCSFHDMJͷॳ൛ SFHTVJUͷFYQFDUFEDPNNJUݕग़෦෼ͳͲ w

    !XBEBDLFM4UPSZDBQͷॳ൛ SFHDMJͷϨϙʔτ6* SFHWJ[શମͷσβΠϯͳͲ w 13ͳͲ͸͍ͭͰ΋΢ΣϧΧϜͰ͢ʂ
  15. 42.

    Other tools w IUUQTHJUIVCDPNSFHWJ[OPEFYDC w $ISPNFΛIFBEMFTTͰಈ࡞ͤ͞ΔͨΊʹඞཁͳMJCYDCͳͲΛࣄલʹম͖෇͚ͨ %PDLFSpMFɻ$*ͷCBTFΠϝʔδʹ͢Δͱศར w IUUQTHJUIVCDPNSFHWJ[JNHEJ⒎KT w

    /PEFKT͚ͩͰ׬݁ͯ͠ը૾ͷEJ⒎ੜ੒͕ߦ͑ΔϥΠϒϥϦɻSFHDJM͕಺ଆͰ͜ΕΛୟ ͍͍ͯΔ w ࠷ॳظɺSFHDMJ͕JNBHFNBHJDLͷDPNQBSFίϚϯυʹґଘ͓ͯ͠Γɺ͜ΕΛണ͕͢ ͨΊʹ࣮૷
  16. 45.
  17. 51.

    Tuning: Storycap w  ຕͷੈքͰ͸ʮը૾ͷൺֱʯΑΓ΋ʮը૾ͷੜ੒ʯ͕ࢧ഑త w 1VQQFUFFSͷϓϩηεΛฒྻՔಇͤͯ͞$16Λ༡͹ͤͳ͍ w খωλ 1VQQFUFFSͰ͸1BHFΛෳ਺࡞ͬͯ΋5$1ίωΫγϣϯ੍ݶͷ౎߹্ɺMPBE

    BWFSBHF͕Ք͛ͳ͍ɻ#SPXTFSຖฒྻԽ͢Δํ͕݁Ռͱͯ͠ޮ཰త w 4UPSZͷ੾Γସ͑͸JGSBNFʹ௚઀QPTUNFTTBHF͢Δ w 4UPSZCPPLͷXFCQBDLʹCVOEMF͞Εͨ+BWB4DSJQUͷධՁ͕Ұ౓͚ͩʹͳΔ
  18. 53.

    Tuning: reg-cli w $-*ຊମ w ը૾ͷNEϋογϡ͕Ұகͨ͠৔߹ɺϐΫηϧൺֱΛߦΘͳ͍Α͏ʹ͍ͯ͠Δ w 3FQPSU6*΋ ຕͰ΋αΫαΫݟΕΔΑ͏ʹ޻෉ w

    ը૾ͷ஗Ԇϩʔυʢ*OUFSTFDUJPO0CTFSWFS MPBEJOHMB[Z  w 4΍($4΁ͷը૾(&5ճ਺͕࠷খԽ͞ΕΔͷͰɺ͓ࡒ෍ʹ΋༏͍͠ w Ծ૝εΫϩʔϧʢࣗલ࣮૷ʣ
  19. 54.
  20. 58.

    Scaffolding w TDIFNBUJDTͰͷTUPSJFTUTYͷྫɿ import React from 'react'; import { storiesOf

    } from '@storybook/react'; import { action } from '@storybook/addon-actions'; import <%= classify(name) %>, { Props } from './<%= classify(name) %>'; const stories = storiesOf('<%= target %>/components/<%= classify(name) %>', module); const props: Props = { }; stories.add('<%= classify(name) %>', () => <<%= classify(name) %> {...props} />);
  21. 63.