Haunted House of Angular unit testing

How to reduce execution time Angular Karma testing

Transcript

1. Haunted House of
 Angular unit testing by @Quramy #ng_sake 2017.05.10
2. About me
3. ͦΕ͸ڈ೥ͷࢣ૸ͷ͜ͱ…
4. feature branch࡞ͬͯͶɺ ŝƄŕŰŲƄʹcommitͨ͠ΜͰ͢Α
5. ͦͨ͠ΒCI͕૸Γग़ͯ͠Ͷɺ
6. Ͱ΋Ͷɺ ͳʔΜ͔༷ࢠ͕͓͔͍͠
7. มͩͳʔɺා͍ͳʔ
8. ڪΔڪΔCIͷ؅ཧը໘Λ ։͍ͯΈΔͱ…
9. ςετ͕ऴΘΒͳ͍ΜͰ͢ Α͓͓͓͓͒͒͒͒͒
10. Angular
11. Scale of our product: ~130,000 loc ~30 angular modules ~700 components ~1,000 cases
12. 1. Docker pull, run container 2. Webpack bundle 3. Run Karma (with capture screenshots) 4. Upload screenshot images to AWS S3 Workflow of unit test on CI 1 2 3 4
13. Question: workflow࣮ߦʹͲΕ͘Βֻ͍͔Δͷ͔
14. Changes of duration ྘ઢ: CIͰͷ୯ମςετ࣮ߦ࣌ؒ [msec]: ࠨ࣠, 
 ੨ઢ: masterͷιʔείʔυߦ਺ [loc]: ӈ࣠
15. Changes of duration ྘ઢ: CIͰͷ୯ମςετ࣮ߦ࣌ؒ [msec]: ࠨ࣠, 
 ੨ઢ: masterͷιʔείʔυߦ਺ [loc]: ӈ࣠ 1 upgrade ng 4 3 2
16. ɹͰԿΛͨ͠ͷ͔?
17. Today’s theme: 
 ୯ମςετߴ଎ԽςΫχοΫ
 (ςετίʔυͷॻ͖ํͱ͔͸Ұ੾આ໌͠ͳ͍Α!)
18. 1. Good bye Phantom, welcome to Nightmare
19. Unit testing on CI • Angularͷ୯ମςετͰ͸KarmaΛ༻͍Δ • Karmaͷಈ࡞ʹ͸ϒϥ΢β͕ඞཁ • CIͰ͸headlessʹಈ࡞ͤ͞ͳͯ͘͸ͳΒͳ͍
20. PhantomJS • Prjॳظ͸karma-phantomjs-launcher Λར༻ • PhantomJSࣗମɺChrome౳ͱൺ΂Δͱ஗͍ • ࠷େͷܽ఺͸ͦͷෆ҆ఆੑɻ
 “๢ྶ”ͱ͔ݴ͏ׂʹ΍ͨΒࢮ͵ɻ
21. PhantomJS -> hangs up
22. Nightmare • karma-nightmare[1] ʹ৐Γ׵͑ • Nightmare: Electronϕʔεͷscraping tool • Ҏલ͸PhantomJSϕʔεͰ͕͋ͬͨɺੑೳվ ળͷͨΊʹElectronϕʔεʹ੾Γସ͑ΒΕͨ[2] <>
    https://github.com/bokuweb/karma-nightmare
    <>
    https://github.com/segmentio/nightmare/issues/484#issuecomment-184519591
23. How effective ? • ໿10%~20%ͷ࣮ߦ࣌ؒ࡟ݮޮՌ • Pros: • ϋϯάൃੜ͕ແ͘ͳΓɺίϯελϯτʹςε τ͕׬ྃ͢ΔΑ͏ʹͳͬͨ • ಋೖ͕༰қ. karma-confͷઃఆ + Ծ૝ϑϨʔ ϜόοϑΝઃఆͰCIಈ࡞͢Δ
24. 2. More Karma, more and more…!
25. Multiple cores, single thread • Karma͸ϒϥ΢β্ͰςετΛ࣮ߦ͢Δ • جຊతʹγϯάϧεϨου • ϚϧνίΞ؀ڥͰ͸ଞͷίΞ͕༡ͼ͕ͪʹ • ͋Εɺϓϩηε෼͚Ε͹Α͘Ͷʁ
26. CPU Load ϩʔΧϧ(MBP, 4 core)ͰͷCPUෛՙ ۀ ۀ
27. ͦ͏ɺզʑʹ͸ ۀ͕଍Γ͍ͯͳ͔ͬͨͷͩ
28. Split test bundles • test bundleΛmodule୯ҐͰ෼ׂ,
 bundle୯ҐͰKarmaϓϩηεΛىಈ /* test.ts */ const context = require.context('./', true, /\.spec\.ts$/); context.keys().map(context); /* test-a.ts */ const context = require.context('./module-a', true, /\.spec\.ts$/); context.keys().map(context); /* test-b.ts */ const context = require.context('./module-b', true, /\.spec\.ts$/); context.keys().map(context);
29. How effective ? • ໿50%~60%ͷ࣮ߦ࣌ؒ࡟ݮޮՌ • Cons: • ʮCPUΛ࢖͑Δ͚ͩ࢖͏ʯઓུͷͨΊɺ CI(Wercker)ͷՔಇঢ়گ࣍ୈͳ෦෼ΞϦ • ಋೖোนߴΊɻbundle࡞੒ॲཧΛΠδΔͨΊɺ
 angular-cliͷ৔߹͸ཁng eject
30. 3. Zombie compiler
31. What does make tests slow? JiT Compiler.
 CPU profile with ~300 components, 15 test cases
32. What TestBed does? it(…)ͷ౎౓ɺFooModuleͷશcomponent͕compile͞ΕΔ beforeEach(async(() => { TestBed.configureTestingModule({ imports: [ FooModule, ], }).compileComponents(); })); it('should ...', () => { /* ... /* }); .component.spec.tsͰimportsΛॻ͘ͱ…
33. Avoid to compile components componentͷcompileճ਺ΛݮΒͤ͹, ͦͷ෼ߴ଎Խ • moduleΛద੾ͳཻ౓ʹ෼ׂͯ͠ґଘର৅Λখ͘͢͞Δ • componentΛcompile͠ͳ͍. ௨ৗͷclassͱͯ͠test͢Δ
 (isolated-component-tests) • ෳࡶͳॲཧΛcomponent͔Βservice΁Ҡߦ͢Δ 
 service testͷׂ߹Λ૿΍͢
35. ΋͏গ͠଍ૡ͍ͯΈΑ͏
36. TestingCompiler impl • JiT Compiler͸ΩϟογϡػߏΛඋ͍͑ͯΔ • Compiler Lifecycle͸it(…)ຖͰ͋ΔͨΊɺςε τέʔεΛލ͙ͱΩϟογϡ͝ͱഁغ͞ΕΔ • it(“should do A”) ͱ it(“should do B”) Ͱ
 Ωϟογϡ͕࠶ར༻͞ΕΔࣄ͸ͳ͍
37. Ͱ΋ςετର৅ͷఆٛͳΜͯɺ Karma࣮ߦத͸ෆม͡ΌͶʁ
 ※ overrideComponent౳ͷར༻͕ແ͍ͱ͍͏લఏͰ
38. Make compiler “undead” ngx-zombie-compiler[1] • ςετέʔεΛލ͍Ͱ΋ʮࢮͳͳ͍ʯcompiler • (ՄೳͳݶΓ)cacheͨ͠compile݁ՌΛ࢖͍ճͨ͢Ίɺߴ଎ import { ZOMBIE_COMPILER_PROVIDERS } from "ngx-zombie-compiler"; beforeEach(() => { TestBed.configureCompiler({ providers: ZOMBIE_COMPILER_PROVIDERS, }); }); [1] https://github.com/Quramy/ngx-zombie-compiler
39. Demonstration normal with zombie
40. How effective ? • ࣮prjͰ͸໿50~60%ͷ࣮ߦ࣌ؒ࡟ݮޮՌ • Pros: • ಋೖ༰қɻtest.tsͷमਖ਼ͷΈͰಈ࡞͢Δ • Cons: • Angular 5.xͷࠒʹ͸௨༻͠ͳ͘ͳΔ΍΋… • ୯ମςετͷႈ౳ੑΛߟྀ͢ΔͱΩϫϞϊ…
41. Summary
42. Improvement techniques ҙ֎ͱڭՊॻʹԊͬͯΔײɿ • Nightmare < - > ߴ଎ͳ࣮ߦج൫ͷಋೖ • Parallel Karma < - > ௚ྻॲཧͷฒྻԽ • Zombie Compiler < - > Flyweightύλʔϯద༻ ʮ஗͍͔Βճ͞ͳ͍ʯ͸؁͑ɻͲ͠Ͳ͠ςετॻ͜͏ʂ
43. Thank you!