$30 off During Our Annual Pro Sale. View Details »

Testing without Xcode - CMD+U 2016

Testing without Xcode - CMD+U 2016

Kyle Fuller

July 08, 2016
Tweet

More Decks by Kyle Fuller

Other Decks in Technology

Transcript

  1. Testing without
    Xcode
    Kyle Fuller

    View Slide

  2. View Slide

  3. View Slide

  4. View Slide

  5. View Slide

  6. View Slide

  7. View Slide

  8. Open Source
    Libraries

    View Slide

  9. Existing code on
    other Platforms

    View Slide

  10. Editors

    View Slide

  11. Who am I

    View Slide

  12. View Slide

  13. Agenda
    • How other communities do testing
    • How testing is done in Swift today
    • How we can do testing in Swift in the future

    View Slide

  14. How do other communities
    solve testing?
    • Ruby !
    • C / C++ ䷢

    View Slide

  15. Testing in Ruby
    describe '.authenticate' do
    context 'when logged in' do
    it { is_expected.to respond_with 200 }
    end
    end

    View Slide

  16. Testing in Ruby
    $ rspec test.rb
    Finished in 0.0016 seconds (files took 0.2153 seconds to load)
    5 examples, 0 failures

    View Slide

  17. Testing in C / C++
    int main(int argc, const char *argv[]) {
    user_t user = user_alloc("kyle");
    // Test User's name
    assert(user.name == "kyle");
    // Test User's priviledges
    assert(user.is_admin, 0);
    return 0;
    }

    View Slide

  18. Testing in C / C++
    $ clang test.c -o tests
    $ ./tests
    Tests passed

    View Slide

  19. Testing in C / C++
    #define CATCH_CONFIG_MAIN
    #include "catch.hpp"
    TEST_CASE( "Factorials are computed", "[factorial]" ) {
    REQUIRE( Factorial(1) == 1 );
    REQUIRE( Factorial(2) == 2 );
    REQUIRE( Factorial(3) == 6 );
    REQUIRE( Factorial(10) == 3628800 );
    }

    View Slide

  20. Test Framework ~= Test Runner

    View Slide

  21. Testing in Swift
    • assert
    • XCTest
    • Quick (uses XCTest)
    • Spectre
    • Ploughman
    • Custom testing
    • CLI tool

    View Slide

  22. Testing with
    assert

    View Slide

  23. assert()

    View Slide

  24. assert(person.name == "Kyle")

    View Slide

  25. assert(person.name == "Kyle",
    "Persons name should be Kyle")

    View Slide

  26. struct Person: CustomStringConvertible {
    let name: String
    var description: String {
    return name
    }
    }
    let kyle = Person(name: "Kyle")
    assert(kyle.description == "Kyle",
    "conforms to CustomStringConvertible")

    View Slide

  27. $ swift test.swift

    View Slide

  28. Testing in a
    playground

    View Slide

  29. Demo

    View Slide

  30. assert() testing
    Upsides
    • Extremely simple
    • Built into stdlib
    Downsides
    • Hard to manage
    • Crashes your process, not all tests are ran

    View Slide

  31. XCTest

    View Slide

  32. CompileSwift normal x86_64
    /Users/kyle/Projects/QueryKit/QueryKit/Tests/SortDescriptorTests.swift cd
    /Users/kyle/Projects/QueryKit/QueryKit
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift
    -frontend -c /Users/kyle/Projects/QueryKit/QueryKit/Tests/AttributeTests.swift
    /Users/kyle/Projects/QueryKit/QueryKit/Tests/QueryKitTests.swift -primary-file
    /Users/kyle/Projects/QueryKit/QueryKit/Tests/SortDescriptorTests.swift
    ...
    Test Suite 'All tests' started at 2016-07-08 01:59:05.062
    Test Suite 'QueryKitTests.xctest' started at 2016-07-08 01:59:05.063
    Test Suite 'AttributeTests' started at 2016-07-08 01:59:05.063
    Test Case '-[QueryKitTests.AttributeTests testAscendingSortDescriptor]' started.
    Test Case '-[QueryKitTests.AttributeTests testAscendingSortDescriptor]' passed (0.002 seconds).
    Test Case '-[QueryKitTests.AttributeTests testAttributeExpression]' started.
    Test Case '-[QueryKitTests.AttributeTests testAttributeExpression]' passed (0.001 seconds).
    Test Suite 'SortDescriptorTests' passed at 2016-07-08 01:59:05.362.
    Executed 2 tests, with 0 failures (0 unexpected) in 0.001 (0.003) seconds
    Test Suite 'QueryKitTests.xctest' passed at 2016-07-08 01:59:05.363.
    Executed 75 tests, with 0 failures (0 unexpected) in 0.240 (0.300) seconds
    Test Suite 'All tests' passed at 2016-07-08 01:59:05.364.
    Executed 75 tests, with 0 failures (0 unexpected) in 0.240 (0.302) seconds
    ** TEST SUCCEEDED **

    View Slide

  33. xctool & xcpretty

    View Slide

  34. https://github.com/supermarin/xcpretty

    View Slide

  35. View Slide

  36. Test Anywhere Protocol (TAP)
    ...
    ok 71 - testTypeSafeOrderBySortDescriptor
    ok 72 - testTypeSafeOrderBySortDescriptors
    ok 73 - testTypeSafeRelatedFilterPredicate
    ok 74 - testAscendingSortDescriptor
    ok 75 - testDescendingSortDescriptor
    1..75

    View Slide

  37. View Slide

  38. How does XCTest work?
    $ xcrun -sdk macosx xctest
    Usage: xctest

    View Slide

  39. import XCTest
    class PersonTests : XCTestCase {
    func testCustomStringConvertible() {
    let kyle = Person(name: "Kyle")
    XCTAssertEqual(kyle.description, "Kyle")
    }
    }

    View Slide

  40. How does XCTest work without
    Objective-C?
    import XCTest
    extension PersonTests {
    static var allTests : [(String, PersonTests -> () throws -> Void)] {
    return [
    ("testCustomStringConvertible", testCustomStringConvertible),
    ]
    }
    }
    XCTMain([
    testCase(PersonTests.allTests),
    ])

    View Slide

  41. View Slide

  42. Spectre
    http://spectre.fuller.li/

    View Slide

  43. describe("Person") {
    let kyle = Person(name: "Kyle")
    $0.it("has a description") {
    try expect(kyle.description) == "Kyle"
    }
    }

    View Slide

  44. describe("Person") {
    let kyle = Person(name: "Kyle")
    $0.it("has a description") {
    try expect(kyle.description) == "Kyle"
    }
    }

    View Slide

  45. try expect(person.description) == "Kyle"

    View Slide

  46. func == (lhs: Expectation, rhs: T?) throws

    View Slide

  47. Reporters

    View Slide

  48. View Slide

  49. View Slide

  50. Custom Reporters

    View Slide

  51. Testing in
    Playground

    View Slide

  52. View Slide

  53. Demo

    View Slide

  54. Spectre Runner

    View Slide

  55. $ ./tests --help
    FLAGS:
    --tap - Test Anywhere Protocol Reporter
    --t - Dot Reporter

    View Slide

  56. https://github.com/Quick/Quick/

    View Slide

  57. import Quick
    import Nimble
    class TableOfContentsSpec: QuickSpec {
    override func spec() {
    describe("the 'Documentation' directory") {
    it("has everything you need to get started") {
    let sections = Directory("Documentation").sections
    expect(sections).to(contain("Installing Quick"))
    }
    }
    }
    }

    View Slide

  58. Cucumber

    View Slide

  59. Feature: An array
    Scenario: Appending to an array
    Given I have an empty array
    When I add 1 to the array
    Then I should have 1 item in the array
    Scenario: Filtering an array
    Given I have an array with the numbers 1 through 5
    When I filter the array for even numbers
    Then I should have 2 items in the array

    View Slide

  60. Ploughman
    https://github.com/kylef/Ploughman/

    View Slide

  61. Given I have an empty array
    var array: [Int]? = nil
    given("^I have an empty array$") { _ in
    array = []
    }

    View Slide

  62. When I add X to the array
    when("^I add (\\d) to the array$") { match in
    let number = Int(match.groups[1])!
    array.append(number)
    }

    View Slide

  63. Then I should have X items in the
    array
    then("^I should have (\\d) items? in the array$") { match in
    let count = Int(match.groups[1])!
    try expect(array.count) == count
    }

    View Slide

  64. $ ./runner scenarios/array.gherkin

    View Slide

  65. Real World Example

    View Slide

  66. Feature: Basic Nest compliance
    Scenario: Success response status code
    When I make a GET request to /hello
    Then I should have a 200 response
    Scenario: Response header
    When I make a GET request to /hello
    Then I should see the header 'Content-Type' with the value 'text/plain'
    Scenario: Response body
    When I make a GET request to /hello
    Then The contents of the body should be 'Hello World'
    Scenario: Response body (POST)
    When I make a POST request with body 'Swift' to /hello
    Then The contents of the body should be 'Hello Swift'

    View Slide

  67. $ NestTestSuite --host http://localhost:8080 Features/*.feature
    -> Basic Nest compliance
    -> Performing a simple GET request
    1 scenarios passed, 0 scenarios failed.

    View Slide

  68. Build Tools

    View Slide

  69. Xcode

    View Slide

  70. DIY
    $ swiftc Sources/*.swift -module-name Hello -emit-library -emit-module -o Hello.dylib
    $ swiftc Tests/*.swift -lHello -o run-tests
    $ ./run-tests

    View Slide

  71. DIY - Make (Makefile)
    Hello.dylib:
    swiftc Sources/*.swift -module-name Hello -emit-library -emit-module -o Hello.dylib
    run-tests: Hello.dylib
    swiftc Tests/*.swift -lHello -o run-tests
    .PHONY: test
    test: run-tests
    ./run-tests

    View Slide

  72. $ make test

    View Slide

  73. Swift Package Manager

    View Slide

  74. import PackageDescription
    let package = Package(
    name: "Curassow",
    dependencies: [
    .Package(url: "https://github.com/kylef/Commander.git",
    majorVersion: 0, minor: 4),
    ]
    )

    View Slide

  75. $ swift build

    View Slide

  76. import PackageDescription
    let package = Package(
    name: "Curassow",
    testDependencies: [
    .Package(url: "https://github.com/kylef/Spectre.git",
    majorVersion: 0, minor: 7),
    ]
    )

    View Slide

  77. $ swift test

    View Slide

  78. LinuxMain.swift
    import XCTest
    extension PersonTests {
    static var allTests : [(String, PersonTests -> () throws -> Void)] {
    return [
    ("testCustomStringConvertible", testCustomStringConvertible),
    ]
    }
    }
    XCTMain([
    testCase(PersonTests.allTests),
    ])

    View Slide

  79. Custom Testing
    with swift test

    View Slide

  80. !
    swim
    https://github.com/kylef/swim/

    View Slide

  81. swim Goals
    • swim is compatible with multiple versions of
    Swift (2.x, 3.x, etc)
    • swim allows you to test your Swift project with
    third-party testing frameworks.
    • swim is compatible with common Swift Package
    Manager libraries.
    • swim is easy to install (does not depend on
    Swift)

    View Slide

  82. $ pip install swim

    View Slide

  83. Package.swift
    let package = Package(
    name: "Stencil",
    testDependencies: [
    .Package(url: "https://github.com/kylef/Spectre", majorVersion: 0),
    ]
    )
    Terminal
    $ swim build
    $ swim test

    View Slide

  84. swiftenv
    https://swiftenv.fuller.li/

    View Slide

  85. $ swiftenv install 2.2.1

    View Slide

  86. $ swiftenv local DEVELOPMENT-SNAPSHOT-2016-02-08-a

    View Slide

  87. $ cat .swift-version
    DEVELOPMENT-SNAPSHOT-2016-02-08-a

    View Slide

  88. $ swift --version
    Apple Swift version 2.2.1
    $ cd Stencil
    $ swift --version
    Apple Swift version 3.0-dev

    View Slide

  89. swiftenv
    https://swiftenv.fuller.li/

    View Slide

  90. $ brew install kylef/formulae/swiftenv

    View Slide

  91. Testing against
    multiple versions
    of Swift

    View Slide

  92. $ env SWIFT_VERSION=2.2 swift test --clean
    $ env SWIFT_VERSION=2.3 swift test --clean
    $ env SWIFT_VERSION=3.0 swift test --clean

    View Slide

  93. Swift on Travis CI
    https://swiftenv.fuller.li/en/latest/
    integrations/travis-ci.html

    View Slide

  94. os:
    - linux
    - osx
    install:
    - eval "$(curl -sL https://github.com/kylef/.../swiftenv-install.sh)"

    View Slide

  95. script: swift test

    View Slide

  96. env:
    - SWIFT_VERSION=2.2
    - SWIFT_VERSION=2.2.1

    View Slide

  97. • Early stages for testing OSS Swift
    • New tools are emerging
    • Swift Package Manager is evolving
    • Test against all platforms

    View Slide

  98. kylefuller
    https://fuller.li/talks

    View Slide