Testing without Xcode - CMD+U 2016

Transcript

1. 1.

   Testing without Xcode Kyle Fuller

2. 2.

   None
3. 3.

   None
4. 4.

   None
5. 5.

   None
6. 6.

   None
7. 7.

   None
8. 8.

   Open Source Libraries

9. 9.

   Existing code on other Platforms

10. 10.

    Editors

11. 11.

    Who am I

12. 12.

    None
13. 13.

    Agenda • How other communities do testing • How testing

    is done in Swift today • How we can do testing in Swift in the future
14. 14.

    How do other communities solve testing? • Ruby ! •

    C / C++ ䷢
15. 15.

    Testing in Ruby describe '.authenticate' do context 'when logged in'

    do it { is_expected.to respond_with 200 } end end
16. 16.

    Testing in Ruby $ rspec test.rb Finished in 0.0016 seconds

    (files took 0.2153 seconds to load) 5 examples, 0 failures
17. 17.

    Testing in C / C++ int main(int argc, const char

    *argv[]) { user_t user = user_alloc("kyle"); // Test User's name assert(user.name == "kyle"); // Test User's priviledges assert(user.is_admin, 0); return 0; }
18. 18.

    Testing in C / C++ $ clang test.c -o tests

    $ ./tests Tests passed
19. 19.

    Testing in C / C++ #define CATCH_CONFIG_MAIN #include "catch.hpp" TEST_CASE(

    "Factorials are computed", "[factorial]" ) { REQUIRE( Factorial(1) == 1 ); REQUIRE( Factorial(2) == 2 ); REQUIRE( Factorial(3) == 6 ); REQUIRE( Factorial(10) == 3628800 ); }
20. 20.

    Test Framework ~= Test Runner

21. 21.

    Testing in Swift • assert • XCTest • Quick (uses

    XCTest) • Spectre • Ploughman • Custom testing • CLI tool
22. 22.

    Testing with assert

23. 23.

    assert()

24. 24.

    assert(person.name == "Kyle")

25. 25.

    assert(person.name == "Kyle", "Persons name should be Kyle")

26. 26.

    struct Person: CustomStringConvertible { let name: String var description: String

    { return name } } let kyle = Person(name: "Kyle") assert(kyle.description == "Kyle", "conforms to CustomStringConvertible")
27. 27.

    $ swift test.swift

28. 28.

    Testing in a playground

29. 29.

    Demo

30. 30.

    assert() testing Upsides • Extremely simple • Built into stdlib

    Downsides • Hard to manage • Crashes your process, not all tests are ran
31. 31.

    XCTest

32. 32.

    CompileSwift normal x86_64 /Users/kyle/Projects/QueryKit/QueryKit/Tests/SortDescriptorTests.swift cd /Users/kyle/Projects/QueryKit/QueryKit /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift -frontend -c /Users/kyle/Projects/QueryKit/QueryKit/Tests/AttributeTests.swift

    /Users/kyle/Projects/QueryKit/QueryKit/Tests/QueryKitTests.swift -primary-file /Users/kyle/Projects/QueryKit/QueryKit/Tests/SortDescriptorTests.swift ... Test Suite 'All tests' started at 2016-07-08 01:59:05.062 Test Suite 'QueryKitTests.xctest' started at 2016-07-08 01:59:05.063 Test Suite 'AttributeTests' started at 2016-07-08 01:59:05.063 Test Case '-[QueryKitTests.AttributeTests testAscendingSortDescriptor]' started. Test Case '-[QueryKitTests.AttributeTests testAscendingSortDescriptor]' passed (0.002 seconds). Test Case '-[QueryKitTests.AttributeTests testAttributeExpression]' started. Test Case '-[QueryKitTests.AttributeTests testAttributeExpression]' passed (0.001 seconds). Test Suite 'SortDescriptorTests' passed at 2016-07-08 01:59:05.362. Executed 2 tests, with 0 failures (0 unexpected) in 0.001 (0.003) seconds Test Suite 'QueryKitTests.xctest' passed at 2016-07-08 01:59:05.363. Executed 75 tests, with 0 failures (0 unexpected) in 0.240 (0.300) seconds Test Suite 'All tests' passed at 2016-07-08 01:59:05.364. Executed 75 tests, with 0 failures (0 unexpected) in 0.240 (0.302) seconds ** TEST SUCCEEDED **
33. 33.

    xctool & xcpretty

34. 34.

    https://github.com/supermarin/xcpretty

35. 35.

    None
36. 36.

    Test Anywhere Protocol (TAP) ... ok 71 - testTypeSafeOrderBySortDescriptor ok

    72 - testTypeSafeOrderBySortDescriptors ok 73 - testTypeSafeRelatedFilterPredicate ok 74 - testAscendingSortDescriptor ok 75 - testDescendingSortDescriptor 1..75
37. 37.

    None
38. 38.

    How does XCTest work? $ xcrun -sdk macosx xctest Usage:

    xctest <path of unit to be tested>
39. 39.

    import XCTest class PersonTests : XCTestCase { func testCustomStringConvertible() {

    let kyle = Person(name: "Kyle") XCTAssertEqual(kyle.description, "Kyle") } }
40. 40.

    How does XCTest work without Objective-C? import XCTest extension PersonTests

    { static var allTests : [(String, PersonTests -> () throws -> Void)] { return [ ("testCustomStringConvertible", testCustomStringConvertible), ] } } XCTMain([ testCase(PersonTests.allTests), ])
41. 41.

    None
42. 42.

    Spectre http://spectre.fuller.li/

43. 43.

    describe("Person") { let kyle = Person(name: "Kyle") $0.it("has a description")

    { try expect(kyle.description) == "Kyle" } }
44. 44.

    describe("Person") { let kyle = Person(name: "Kyle") $0.it("has a description")

    { try expect(kyle.description) == "Kyle" } }
45. 45.

    try expect(person.description) == "Kyle"

46. 46.

    func == <T> (lhs: Expectation<T>, rhs: T?) throws

47. 47.

    Reporters

48. 48.

    None
49. 49.

    None
50. 50.

    Custom Reporters

51. 51.

    Testing in Playground

52. 52.

    None
53. 53.

    Demo

54. 54.

    Spectre Runner

55. 55.

    $ ./tests --help FLAGS: --tap - Test Anywhere Protocol Reporter

    --t - Dot Reporter
56. 56.

    https://github.com/Quick/Quick/

57. 57.

    import Quick import Nimble class TableOfContentsSpec: QuickSpec { override func

    spec() { describe("the 'Documentation' directory") { it("has everything you need to get started") { let sections = Directory("Documentation").sections expect(sections).to(contain("Installing Quick")) } } } }
58. 58.

    Cucumber

59. 59.

    Feature: An array Scenario: Appending to an array Given I

    have an empty array When I add 1 to the array Then I should have 1 item in the array Scenario: Filtering an array Given I have an array with the numbers 1 through 5 When I filter the array for even numbers Then I should have 2 items in the array
60. 60.

    Ploughman https://github.com/kylef/Ploughman/

61. 61.

    Given I have an empty array var array: [Int]? =

    nil given("^I have an empty array$") { _ in array = [] }
62. 62.

    When I add X to the array when("^I add (\\d)

    to the array$") { match in let number = Int(match.groups[1])! array.append(number) }
63. 63.

    Then I should have X items in the array then("^I

    should have (\\d) items? in the array$") { match in let count = Int(match.groups[1])! try expect(array.count) == count }
64. 64.

    $ ./runner scenarios/array.gherkin

65. 65.

    Real World Example

66. 66.

    Feature: Basic Nest compliance Scenario: Success response status code When

    I make a GET request to /hello Then I should have a 200 response Scenario: Response header When I make a GET request to /hello Then I should see the header 'Content-Type' with the value 'text/plain' Scenario: Response body When I make a GET request to /hello Then The contents of the body should be 'Hello World' Scenario: Response body (POST) When I make a POST request with body 'Swift' to /hello Then The contents of the body should be 'Hello Swift'
67. 67.

    $ NestTestSuite --host http://localhost:8080 Features/*.feature -> Basic Nest compliance ->

    Performing a simple GET request 1 scenarios passed, 0 scenarios failed.
68. 68.

    Build Tools

69. 69.

    Xcode

70. 70.

    DIY $ swiftc Sources/*.swift -module-name Hello -emit-library -emit-module -o Hello.dylib

    $ swiftc Tests/*.swift -lHello -o run-tests $ ./run-tests
71. 71.

    DIY - Make (Makefile) Hello.dylib: swiftc Sources/*.swift -module-name Hello -emit-library

    -emit-module -o Hello.dylib run-tests: Hello.dylib swiftc Tests/*.swift -lHello -o run-tests .PHONY: test test: run-tests ./run-tests
72. 72.

    $ make test

73. 73.

    Swift Package Manager

74. 74.

    import PackageDescription let package = Package( name: "Curassow", dependencies: [

    .Package(url: "https://github.com/kylef/Commander.git", majorVersion: 0, minor: 4), ] )
75. 75.

    $ swift build

76. 76.

    import PackageDescription let package = Package( name: "Curassow", testDependencies: [

    .Package(url: "https://github.com/kylef/Spectre.git", majorVersion: 0, minor: 7), ] )
77. 77.

    $ swift test

78. 78.

    LinuxMain.swift import XCTest extension PersonTests { static var allTests :

    [(String, PersonTests -> () throws -> Void)] { return [ ("testCustomStringConvertible", testCustomStringConvertible), ] } } XCTMain([ testCase(PersonTests.allTests), ])
79. 79.

    Custom Testing with swift test

80. 80.

    ! swim https://github.com/kylef/swim/

81. 81.

    swim Goals • swim is compatible with multiple versions of

    Swift (2.x, 3.x, etc) • swim allows you to test your Swift project with third-party testing frameworks. • swim is compatible with common Swift Package Manager libraries. • swim is easy to install (does not depend on Swift)
82. 82.

    $ pip install swim

83. 83.

    Package.swift let package = Package( name: "Stencil", testDependencies: [ .Package(url:

    "https://github.com/kylef/Spectre", majorVersion: 0), ] ) Terminal $ swim build $ swim test
84. 84.

    swiftenv https://swiftenv.fuller.li/

85. 85.

    $ swiftenv install 2.2.1

86. 86.

    $ swiftenv local DEVELOPMENT-SNAPSHOT-2016-02-08-a

87. 87.

    $ cat .swift-version DEVELOPMENT-SNAPSHOT-2016-02-08-a

88. 88.

    $ swift --version Apple Swift version 2.2.1 $ cd Stencil

    $ swift --version Apple Swift version 3.0-dev
89. 89.

    swiftenv https://swiftenv.fuller.li/

90. 90.

    $ brew install kylef/formulae/swiftenv

91. 91.

    Testing against multiple versions of Swift

92. 92.

    $ env SWIFT_VERSION=2.2 swift test --clean $ env SWIFT_VERSION=2.3 swift

    test --clean $ env SWIFT_VERSION=3.0 swift test --clean
93. 93.

    Swift on Travis CI https://swiftenv.fuller.li/en/latest/ integrations/travis-ci.html

94. 94.

    os: - linux - osx install: - eval "$(curl -sL

    https://github.com/kylef/.../swiftenv-install.sh)"
95. 95.

    script: swift test

96. 96.

    env: - SWIFT_VERSION=2.2 - SWIFT_VERSION=2.2.1

97. 97.

    • Early stages for testing OSS Swift • New tools

    are emerging • Swift Package Manager is evolving • Test against all platforms
98. 98.

    kylefuller https://fuller.li/talks