What's new in testing WWDC2018

Transcript

1. What’s new in Testing WWDC18 #wwdc_rusuban 2018/06/11 @yokoyas000
2. ◾ ·ͱΊ ◾ Code Coverage ɾ xccovίϚϯυͰΧόϨοδϨϙʔτΛ͍͍ײ͡ʹͰ͖Δ ◾ Test Selection And Ordering ɾ ৽͍͠ςετΛ࣮ߦର৅ʹؚΊΔ/ؚΊͳ͍ Λબ୒Ͱ͖Δ ɾ ςετ͕ϥϯμϜ࣮ߦͰ͖ΔΑ͏ʹͳͬͨɻݎ࿚ੑ61⤴ ◾ Parallel Testing ɾ ฒྻͰςετΛ࣮ߦͰ͖ͯૣ͍
3. Code Coverage
4. ◾ Code Coverage ※ Ҏલ͔Β͋Δػೳ ෼ذॲཧΛςετͰͲΕ͚ͩ໢ཏͰ͖ͯΔ͔ݟΕΔػೳ
5. ◾ Code Coverage ▪ Code coverage Λ༗ޮʹͨ͠ঢ়ଶͰςετ࣮ߦͨ͠ޙ ɾςετͰ࣮ߦ͞Εͨߦ͸྘ ɾςετͰ࣮ߦ͞Εͳ͔ͬͨߦ͸੺
6. ◾ Code Coverage(Xcode 9.3 ~) ▪ ύϑΥʔϚϯεͱਖ਼֬͞ͷվળ ▪ ςετλʔήοτΛબ΂Δ ▪ xccovίϚϯυͰΧόϨοδ৘ใΛݟΕΔ
7. ◾ Code Coverage(Xcode 9.3 ~) ▪ ύϑΥʔϚϯεͱਖ਼֬͞ͷվળ ▪ ςετλʔήοτΛબ΂Δ ▪ xccovίϚϯυͰΧόϨοδ৘ใΛݟΕΔ
8. ◾ xccov ▪ xccov ΧόϨοδϨϙʔτΛ࢖ͬͯͳΜ΍͔Μ΍Ͱ͖ΔίϚϯυ ▪ ΧόϨοδϨϙʔτ ΧόϨοδΛ༗ޮʹͨ͠ঢ়ଶͰςετΛ࣮ߦ͢Δͱੜ੒͞ΕΔϑΝΠϧ ɾ.xccovreport֦ுࢠ ɾΧόϨοδͷׂ߹͕ॻ͔Ε͍ͯΔ ɾ.xccovarchive֦ுࢠ ɾ ΞϓϦέʔγϣϯίʔυͷ֤ϑΝΠϧͷ࣮ߦΧ΢ϯτ
9. ◾ xccov ▪ xccovίϚϯυͰΧόϨοδϨϙʔτΛৄ͘͠ݟΕΔ $ man xccov NAME xccov - view Xcode coverage data in human-readable or machine-parseable format.
10. ◾ xccov ▪ xccov Λ࢖ͬͯΧόϨοδϨϙʔτΛਓ͕ؒݟΕΔܗͰग़ྗ͢Δ $ xcrun xccov view XXXX.xccovreport Name Coverage ------------------------------------------------------------------ --------------- HelloWorld.app 57.89% (11/19) /tmp/HelloWorld/HelloWorld/AppDelegate.m 50.00% (3/6) -[AppDelegate applicationWillTerminate:] 0.00% (0/3) -[AppDelegate applicationDidFinishLaunching:] 100.00% (3/3) /tmp/HelloWorld/HelloWorld/ViewController.m 50.00% (5/10) -[ViewController setRepresentedObject:] 0.00% (0/5) -[ViewController viewDidLoad] 100.00% (5/5) /tmp/HelloWorld/HelloWorld/main.m 100.00% (3/3) main
11. ◾ xccov ▪ xccov Φϓγϣϯ xccov view --json XXXX.xccovreport ɾKTPOܗࣜͰग़ྗ xccov view --file-list XXXX.xccovarchive/ ɾΧόϨοδσʔλΛ࣋ͭ͢΂ͯͷϑΝΠϧΛҰཡදࣔ xccov view --file /tmp/HelloWorld/HelloWorldTests/HelloWorldTests.m XXXX.xccovarchive/` ɾ̍ϑΝΠϧͷΧόϨοδ͚ͩग़ྗ Լͭͷ࢖͍ॴ͕Α͘Θ͔Βͳ͍...
12. Test Selection And Ordering
13. ◾ Test Selection And Ordering ※ Xcode 10 ͔Βͷ৽ػೳ ▪ Test Selection ▪ Test Ordering
14. ◾ Test Selection And Ordering ▪ Test Selection ▪ Test Ordering
15. ◾ Test “Selection” εΩʔϜผʹ৽͘͠௥Ճͨ͠ςετΛ࣮ߦ͢Δ/͠ͳ͍Λબ΂Δ ɾEdit Scheme -> Test -> Info -> Options -> Automatically includes new tests ▪ ׆༻ྫ ɾUIςετ͸φΠτϦʔϏϧυͰར༻͢ΔεΩʔϜͷΈ Automatically includes new tests Λ ON ʹ͢Δ
16. ◾ Test Selection And Ordering ▪ Test Selection ▪ Test Ordering
17. ◾ Test “Ordering” XCTestCase ͷαϒΫϥε୯Ґ ͰɺςετΛϥϯμϜ࣮ߦ͢Δ ※ࠓ·Ͱ͸ΞϧϑΝϕοτॱͩͬͨ ɾEdit Scheme -> Test -> Info -> Options -> Randomize execution order ▪ ར఺ ɾຖճಉ͡ॱ൪ͩͱʮͨ·ͨ·͏·͍ͬͯ͘Δςετʯͷଘࡏʹؾ͚ͮͳ͍
18. ◾ Test “Ordering” ▪ ʮͨ·ͨ·͏·͍ͬͯ͘Δςετʯͱ͸ʁ ҎԼͷͭͷςετΫϥε͕͋ͬͨͱ͠·͢ɻ ɾATestsΫϥε: UserςʔϒϧΛ࡞੒ ɾBTestsΫϥε: Userςʔϒϧ΁௥Ճ ɾCTestsΫϥε: Userςʔϒϧͷ࡟আ ্هςετ͸ɺ A->B->C ͷॱ൪ʹ࣮ߦͨ࣌͠͸੒ޭ͠·͢ɻ ͔͠͠ɺ B,C͔Β࢝·Δͱࣦഊ͠·͢ɻ
19. ◾ Test “Ordering” ▪ ʮͨ·ͨ·͏·͍ͬͯ͘Δςετʯͱ͸ʁ ɾຊ౰͸ ࣦഊ͢΂͖ςετ ͕੒ޭͯ͠͠·͍ͬͯΔɻ ɾςετؒʹ ґଘ ͕͋ΔͷʹɺͦΕΛݕग़Ͱ͖͍ͯͳ͍ɻ
20. ◾ Test “Ordering” ▪ ʮͨ·ͨ·͏·͍ͬͯ͘Δςετʯͱ͸ʁ ɾຊ౰͸ ࣦഊ͢΂͖ςετ ͕੒ޭͯ͠͠·͍ͬͯΔɻ ɾςετؒʹ ґଘ ͕͋ΔͷʹɺͦΕΛݕग़Ͱ͖͍ͯͳ͍ɻ ϥϯμϜ࣮ߦͰʮͨ·ͨ·͏·͍ͬͯ͘ΔςετʯΛແ͘͢ ݎ࿚ੑ61⤴
21. ◾ Test “Ordering” ɾςετؒͰґଘ͕ແ͍Α͏ʹ͠·͠ΐ͏ ɾࣄલ/ࣄޙॲཧ͕ඞཁͰ͋Ε͹ɺTFUVQ teardown Λ࢖͍·͠ΐ͏
22. Parallel Testing
23. ◾ Parallel Testing ※ࠓ·Ͱͷ1BSBMMFM Testing γϛϡϨʔλΛෳ਺ ্ཱͪ͛ͯฒྻ࣮ߦ ▪ Parallel “Destination” Testing xcodebuild test -destination ‘platform=iOS,name=iPhone X’ -destination ‘platform=iOS,name=iPad’
24. ◾ Parallel Testing ※ࠓ·Ͱͷ1BSBMMFM Testing γϛϡϨʔλΛෳ਺ ্ཱͪ͛ͯฒྻ࣮ߦ ▪ Parallel “Destination” Testing xcodebuild test -destination ‘platform=iOS,name=iPhone X’ -destination ‘platform=iOS,name=iPad’ ɾiPhone X ͱ iPad ͷ γϛϡϨʔλͰฒྻ࣮ߦ ɾ֤γϛϡϨʔλͰ࣮ߦ͢Δςετέʔε͸ಉ͡
25. ◾ Parallel Testing ※৽͍͠1BSBMMFM Testing ̍ͭͷγϛϡϨʔλͷ ΫϩʔϯΛෳ਺ ্ཱͪ͛ͯฒྻ࣮ߦ ɾiPhone X ͷΫϩʔϯΛෳ਺্ཱͪ͛ͯฒྻ࣮ߦ ɾ֤ΫϩʔϯͰ ςετέʔεΛ෼୲ͯ͠ ࣮ߦ͢Δ ɾૣ͍ ▪ Parallel “Distributed” Testing
26. ◾ Parallel “Distributed” Testing ▪ ֤ΫϩʔϯͰςετέʔεΛ෼୲࣮ͯ͠ߦ͢Δ XCTestCase ͷαϒΫϥε୯Ґ Ͱ෼୲͢Δɻ TestClass TestClass TestClass TestClass TestClass TestClass
27. ◾ Parallel “Distributed” Testing ▪ ֤ΫϩʔϯͰςετέʔεΛ෼୲࣮ͯ͠ߦ͢Δ TestClass TestClass TestClass TestClass ֤ΫϩʔϯͰࢀর͢ΔϑΝΠϧ͸෼͔Ε͍ͯΔͷͰɺ ʮଞͷΫϩʔϯ͕ઌʹϑΝΠϧ"Λ࿔͚ͬͨ࣌ͩςετ͕མͪΔ…ʯ ͱ͍͏Α͏ͳ͜ͱ͸ແ͍ɻ
28. ◾ Parallel “Distributed” Testing ▪ Ϋϥε୯ҐͰόϥόϥʹ͢Δཧ༝ ɾςετΫϥεؒͷґଘΛͳ͘͢(ϥϯμϜςετͱಉ͡) ɾsetup, teardown ͸ͦΕͧΕ̍ճͣͭݺΜͰཉ͍͠ ɾΫϥε୯ҐΑΓখ͘͢͞ΔͱɺTFUVQ teardownΛෳ਺ճݺͿ͜ͱʹ…
29. ◾ Parallel “Distributed” Testing ▪ xcodebuild ίϚϯυͰɺฒྻςετʹؔ͢ΔΦϓγϣϯΛมߋͰ ͖Δ -parallel-testing-worker-count n ɾฒྻςετதʹੜ੒͞ΕΔςετϥϯφʔͷ਺ -parallel-testing-enabled YES | NO ɾεΩʔϜͷλʔήοτ͝ͱͷઃఆΛ্ॻ͖͢Δ -maximum-concurrent-test-device-destinations n ɾಉ࣌ʹςετ͢ΔσόΠεѼઌͷ࠷େ਺ -maximum-concurrent-test-simulator-destinations n ɾಉ࣌ʹςετ͢ΔγϛϡϨʔλѼઌͷ࠷େ਺
30. ◾ Parallel Testing Tips ▪ ̍ͭͷେ͖ͳςετΫϥε͸ͭҎ্ʹ෼ׂ͠·͠ΐ͏ ɹɾʮେ͖ͳςετΫϥεʯ͕ऴΘΒͳ͍ݶΓςετऴྃʹͳΒͳ͍ ▪ ύϑΥʔϚϯεςετ͸ฒྻςετͱผʹςετ͠·͠ΐ͏ ▪ ύϥϨϧʹ ޲͍͍ͯͳ͍ ςετΛ஌Γ·͠ΐ͏ ɾʮෳ਺ͷΫϩʔϯ͕ ڞ௨ͷ%#ʹΞΫηε͢ΔʯͳͲ
31. ◾ Testing Tips & Tricks ͔Βͷൈਮ ... Λ͠Α͏ͱࢥ͚ͬͨͲ͕࣌ؒ଍Γͳͦ͏ɻ Optimazation App Launch When Testing("Testing Tips & Tricks"ͷ 34: 57 ~) ͸ྑ͛͞ͳΞϓϩʔνͰͨ͠(ײ૝) ɾεΩʔϜ͝ͱʹ؀ڥม਺Λࢦఆ͠ɺʮ୯ମςετ࣌ʹෆཁͳॲཧ͸࣮ߦ͠ͳ͍ʯͳͲ Λߦ͏͜ͱͰςετΛૣ͘͢Δ
32. What's New in Testing https://developer.apple.com/videos/play/wwdc2018/403 Testing Tips & Tricks https://developer.apple.com/videos/play/wwdc2018/417