Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What's new in testing WWDC2018

yokoya-s000
June 11, 2018
600

What's new in testing WWDC2018

yokoya-s000

June 11, 2018
Tweet

Transcript

  1. What’s new in Testing
    WWDC18
    #wwdc_rusuban 2018/06/11
    @yokoyas000

    View full-size slide

  2. ◾ ·ͱΊ
    ◾ Code Coverage
    ɾ xccovίϚϯυͰΧόϨοδϨϙʔτΛ͍͍ײ͡ʹͰ͖Δ
    ◾ Test Selection And Ordering
    ɾ ৽͍͠ςετΛ࣮ߦର৅ʹؚΊΔ/ؚΊͳ͍ Λબ୒Ͱ͖Δ
    ɾ ςετ͕ϥϯμϜ࣮ߦͰ͖ΔΑ͏ʹͳͬͨɻݎ࿚ੑ61⤴
    ◾ Parallel Testing
    ɾ ฒྻͰςετΛ࣮ߦͰ͖ͯૣ͍

    View full-size slide

  3. Code Coverage

    View full-size slide

  4. ◾ Code Coverage
    ※ Ҏલ͔Β͋Δػೳ
    ෼ذॲཧΛςετͰͲΕ͚ͩ໢ཏͰ͖ͯΔ͔ݟΕΔػೳ

    View full-size slide

  5. ◾ Code Coverage
    ■ Code coverage Λ༗ޮʹͨ͠ঢ়ଶͰςετ࣮ߦͨ͠ޙ
    ɾςετͰ࣮ߦ͞Εͨߦ͸྘
    ɾςετͰ࣮ߦ͞Εͳ͔ͬͨߦ͸੺

    View full-size slide

  6. ◾ Code Coverage(Xcode 9.3 ~)
    ■ ύϑΥʔϚϯεͱਖ਼֬͞ͷվળ
    ■ ςετλʔήοτΛબ΂Δ
    ■ xccovίϚϯυͰΧόϨοδ৘ใΛݟΕΔ

    View full-size slide

  7. ◾ Code Coverage(Xcode 9.3 ~)
    ■ ύϑΥʔϚϯεͱਖ਼֬͞ͷվળ
    ■ ςετλʔήοτΛબ΂Δ
    ■ xccovίϚϯυͰΧόϨοδ৘ใΛݟΕΔ

    View full-size slide

  8. ◾ xccov
    ■ xccov
    ΧόϨοδϨϙʔτΛ࢖ͬͯͳΜ΍͔Μ΍Ͱ͖ΔίϚϯυ
    ■ ΧόϨοδϨϙʔτ
    ΧόϨοδΛ༗ޮʹͨ͠ঢ়ଶͰςετΛ࣮ߦ͢Δͱੜ੒͞ΕΔϑΝΠϧ
    ɾ.xccovreport֦ுࢠ
    ɾΧόϨοδͷׂ߹͕ॻ͔Ε͍ͯΔ
    ɾ.xccovarchive֦ுࢠ
    ɾ ΞϓϦέʔγϣϯίʔυͷ֤ϑΝΠϧͷ࣮ߦΧ΢ϯτ

    View full-size slide

  9. ◾ xccov
    ■ xccovίϚϯυͰΧόϨοδϨϙʔτΛৄ͘͠ݟΕΔ
    $ man xccov
    NAME
    xccov - view Xcode coverage data in human-readable or
    machine-parseable format.

    View full-size slide

  10. ◾ xccov
    ■ xccov Λ࢖ͬͯΧόϨοδϨϙʔτΛਓ͕ؒݟΕΔܗͰग़ྗ͢Δ
    $ xcrun xccov view XXXX.xccovreport
    Name Coverage
    ------------------------------------------------------------------ ---------------
    HelloWorld.app 57.89% (11/19)
    /tmp/HelloWorld/HelloWorld/AppDelegate.m 50.00% (3/6)
    -[AppDelegate applicationWillTerminate:] 0.00% (0/3)
    -[AppDelegate applicationDidFinishLaunching:] 100.00% (3/3)
    /tmp/HelloWorld/HelloWorld/ViewController.m 50.00% (5/10)
    -[ViewController setRepresentedObject:] 0.00% (0/5)
    -[ViewController viewDidLoad] 100.00% (5/5)
    /tmp/HelloWorld/HelloWorld/main.m 100.00% (3/3)
    main

    View full-size slide

  11. ◾ xccov
    ■ xccov Φϓγϣϯ
    xccov view --json XXXX.xccovreport
    ɾKTPOܗࣜͰग़ྗ
    xccov view --file-list XXXX.xccovarchive/
    ɾΧόϨοδσʔλΛ࣋ͭ͢΂ͯͷϑΝΠϧΛҰཡදࣔ
    xccov view --file /tmp/HelloWorld/HelloWorldTests/HelloWorldTests.m
    XXXX.xccovarchive/`
    ɾ̍ϑΝΠϧͷΧόϨοδ͚ͩग़ྗ
    Լͭͷ࢖͍ॴ͕Α͘Θ͔Βͳ͍...

    View full-size slide

  12. Test Selection And
    Ordering

    View full-size slide

  13. ◾ Test Selection And Ordering
    ※ Xcode 10 ͔Βͷ৽ػೳ
    ■ Test Selection
    ■ Test Ordering

    View full-size slide

  14. ◾ Test Selection And Ordering
    ■ Test Selection
    ■ Test Ordering

    View full-size slide

  15. ◾ Test “Selection”
    εΩʔϜผʹ৽͘͠௥Ճͨ͠ςετΛ࣮ߦ͢Δ/͠ͳ͍Λબ΂Δ
    ɾEdit Scheme -> Test -> Info -> Options -> Automatically includes new tests
    ■ ׆༻ྫ
    ɾUIςετ͸φΠτϦʔϏϧυͰར༻͢ΔεΩʔϜͷΈ Automatically includes new
    tests Λ ON ʹ͢Δ

    View full-size slide

  16. ◾ Test Selection And Ordering
    ■ Test Selection
    ■ Test Ordering

    View full-size slide

  17. ◾ Test “Ordering”
    XCTestCase ͷαϒΫϥε୯Ґ ͰɺςετΛϥϯμϜ࣮ߦ͢Δ
    ※ࠓ·Ͱ͸ΞϧϑΝϕοτॱͩͬͨ
    ɾEdit Scheme -> Test -> Info -> Options -> Randomize execution order
    ■ ར఺
    ɾຖճಉ͡ॱ൪ͩͱʮͨ·ͨ·͏·͍ͬͯ͘Δςετʯͷଘࡏʹؾ͚ͮͳ͍

    View full-size slide

  18. ◾ Test “Ordering”
    ■ ʮͨ·ͨ·͏·͍ͬͯ͘Δςετʯͱ͸ʁ
    ҎԼͷͭͷςετΫϥε͕͋ͬͨͱ͠·͢ɻ
    ɾATestsΫϥε: UserςʔϒϧΛ࡞੒
    ɾBTestsΫϥε: Userςʔϒϧ΁௥Ճ
    ɾCTestsΫϥε: Userςʔϒϧͷ࡟আ
    ্هςετ͸ɺ A->B->C ͷॱ൪ʹ࣮ߦͨ࣌͠͸੒ޭ͠·͢ɻ
    ͔͠͠ɺ B,C͔Β࢝·Δͱࣦഊ͠·͢ɻ

    View full-size slide

  19. ◾ Test “Ordering”
    ■ ʮͨ·ͨ·͏·͍ͬͯ͘Δςετʯͱ͸ʁ
    ɾຊ౰͸ ࣦഊ͢΂͖ςετ ͕੒ޭͯ͠͠·͍ͬͯΔɻ
    ɾςετؒʹ ґଘ ͕͋ΔͷʹɺͦΕΛݕग़Ͱ͖͍ͯͳ͍ɻ

    View full-size slide

  20. ◾ Test “Ordering”
    ■ ʮͨ·ͨ·͏·͍ͬͯ͘Δςετʯͱ͸ʁ
    ɾຊ౰͸ ࣦഊ͢΂͖ςετ ͕੒ޭͯ͠͠·͍ͬͯΔɻ
    ɾςετؒʹ ґଘ ͕͋ΔͷʹɺͦΕΛݕग़Ͱ͖͍ͯͳ͍ɻ
    ϥϯμϜ࣮ߦͰʮͨ·ͨ·͏·͍ͬͯ͘ΔςετʯΛແ͘͢
    ݎ࿚ੑ61⤴

    View full-size slide

  21. ◾ Test “Ordering”
    ɾςετؒͰґଘ͕ແ͍Α͏ʹ͠·͠ΐ͏
    ɾࣄલ/ࣄޙॲཧ͕ඞཁͰ͋Ε͹ɺTFUVQ teardown Λ࢖͍·͠ΐ͏

    View full-size slide

  22. Parallel Testing

    View full-size slide

  23. ◾ Parallel Testing
    ※ࠓ·Ͱͷ1BSBMMFM Testing
    γϛϡϨʔλΛෳ਺ ্ཱͪ͛ͯฒྻ࣮ߦ
    ■ Parallel “Destination” Testing
    xcodebuild test
    -destination ‘platform=iOS,name=iPhone X’
    -destination ‘platform=iOS,name=iPad’

    View full-size slide

  24. ◾ Parallel Testing
    ※ࠓ·Ͱͷ1BSBMMFM Testing
    γϛϡϨʔλΛෳ਺ ্ཱͪ͛ͯฒྻ࣮ߦ
    ■ Parallel “Destination” Testing
    xcodebuild test
    -destination ‘platform=iOS,name=iPhone X’
    -destination ‘platform=iOS,name=iPad’
    ɾiPhone X ͱ iPad ͷ γϛϡϨʔλͰฒྻ࣮ߦ
    ɾ֤γϛϡϨʔλͰ࣮ߦ͢Δςετέʔε͸ಉ͡

    View full-size slide

  25. ◾ Parallel Testing
    ※৽͍͠1BSBMMFM Testing
    ̍ͭͷγϛϡϨʔλͷ ΫϩʔϯΛෳ਺ ্ཱͪ͛ͯฒྻ࣮ߦ
    ɾiPhone X ͷΫϩʔϯΛෳ਺্ཱͪ͛ͯฒྻ࣮ߦ
    ɾ֤ΫϩʔϯͰ ςετέʔεΛ෼୲ͯ͠ ࣮ߦ͢Δ
    ɾૣ͍
    ■ Parallel “Distributed” Testing

    View full-size slide

  26. ◾ Parallel “Distributed” Testing
    ■ ֤ΫϩʔϯͰςετέʔεΛ෼୲࣮ͯ͠ߦ͢Δ
    XCTestCase ͷαϒΫϥε୯Ґ Ͱ෼୲͢Δɻ
    TestClass TestClass
    TestClass TestClass
    TestClass TestClass

    View full-size slide

  27. ◾ Parallel “Distributed” Testing
    ■ ֤ΫϩʔϯͰςετέʔεΛ෼୲࣮ͯ͠ߦ͢Δ
    TestClass TestClass
    TestClass TestClass
    ֤ΫϩʔϯͰࢀর͢ΔϑΝΠϧ͸෼͔Ε͍ͯΔͷͰɺ
    ʮଞͷΫϩʔϯ͕ઌʹϑΝΠϧ"Λ࿔͚ͬͨ࣌ͩςετ͕མͪΔ…ʯ
    ͱ͍͏Α͏ͳ͜ͱ͸ແ͍ɻ

    View full-size slide

  28. ◾ Parallel “Distributed” Testing
    ■ Ϋϥε୯ҐͰόϥόϥʹ͢Δཧ༝
    ɾςετΫϥεؒͷґଘΛͳ͘͢(ϥϯμϜςετͱಉ͡)
    ɾsetup, teardown ͸ͦΕͧΕ̍ճͣͭݺΜͰཉ͍͠
    ɾΫϥε୯ҐΑΓখ͘͢͞ΔͱɺTFUVQ teardownΛෳ਺ճݺͿ͜ͱʹ…

    View full-size slide

  29. ◾ Parallel “Distributed” Testing
    ■ xcodebuild ίϚϯυͰɺฒྻςετʹؔ͢ΔΦϓγϣϯΛมߋͰ
    ͖Δ
    -parallel-testing-worker-count n
    ɾฒྻςετதʹੜ੒͞ΕΔςετϥϯφʔͷ਺
    -parallel-testing-enabled YES | NO
    ɾεΩʔϜͷλʔήοτ͝ͱͷઃఆΛ্ॻ͖͢Δ
    -maximum-concurrent-test-device-destinations n
    ɾಉ࣌ʹςετ͢ΔσόΠεѼઌͷ࠷େ਺
    -maximum-concurrent-test-simulator-destinations n
    ɾಉ࣌ʹςετ͢ΔγϛϡϨʔλѼઌͷ࠷େ਺

    View full-size slide

  30. ◾ Parallel Testing Tips
    ■ ̍ͭͷେ͖ͳςετΫϥε͸ͭҎ্ʹ෼ׂ͠·͠ΐ͏
    ɹɾʮେ͖ͳςετΫϥεʯ͕ऴΘΒͳ͍ݶΓςετऴྃʹͳΒͳ͍
    ■ ύϑΥʔϚϯεςετ͸ฒྻςετͱผʹςετ͠·͠ΐ͏
    ■ ύϥϨϧʹ ޲͍͍ͯͳ͍ ςετΛ஌Γ·͠ΐ͏
    ɾʮෳ਺ͷΫϩʔϯ͕ ڞ௨ͷ%#ʹΞΫηε͢ΔʯͳͲ

    View full-size slide

  31. ◾ Testing Tips & Tricks ͔Βͷൈਮ
    ... Λ͠Α͏ͱࢥ͚ͬͨͲ͕࣌ؒ଍Γͳͦ͏ɻ
    Optimazation App Launch When Testing("Testing Tips & Tricks"ͷ 34: 57 ~)
    ͸ྑ͛͞ͳΞϓϩʔνͰͨ͠(ײ૝)
    ɾεΩʔϜ͝ͱʹ؀ڥม਺Λࢦఆ͠ɺʮ୯ମςετ࣌ʹෆཁͳॲཧ͸࣮ߦ͠ͳ͍ʯͳͲ
    Λߦ͏͜ͱͰςετΛૣ͘͢Δ

    View full-size slide

  32. What's New in Testing
    https://developer.apple.com/videos/play/wwdc2018/403
    Testing Tips & Tricks
    https://developer.apple.com/videos/play/wwdc2018/417

    View full-size slide