Testing without Xcode Kyle Fuller 

No content

No content

No content

No content

No content

No content

Open Source Libraries 

Existing code on other Platforms 

Editors 

Who am I 

No content

Agenda • How other communities do testing • How testing is done in Swift today • How we can do testing in Swift in the future 

How do other communities solve testing? • Ruby ! • C / C++ ䷢ 

Testing in Ruby describe '.authenticate' do context 'when logged in' do it { is_expected.to respond_with 200 } end end 

Testing in Ruby $ rspec test.rb Finished in 0.0016 seconds (files took 0.2153 seconds to load) 5 examples, 0 failures 

Testing in C / C++ int main(int argc, const char *argv[]) { user_t user = user_alloc("kyle"); // Test User's name assert(user.name == "kyle"); // Test User's priviledges assert(user.is_admin, 0); return 0; } 

Testing in C / C++ $ clang test.c -o tests $ ./tests Tests passed 

Testing in C / C++ #define CATCH_CONFIG_MAIN #include "catch.hpp" TEST_CASE( "Factorials are computed", "[factorial]" ) { REQUIRE( Factorial(1) == 1 ); REQUIRE( Factorial(2) == 2 ); REQUIRE( Factorial(3) == 6 ); REQUIRE( Factorial(10) == 3628800 ); } 

Test Framework ~= Test Runner 

Testing in Swift • assert • XCTest • Quick (uses XCTest) • Spectre • Ploughman • Custom testing • CLI tool 

Testing with assert 

assert() 

assert(person.name == "Kyle") 

assert(person.name == "Kyle", "Persons name should be Kyle") 

struct Person: CustomStringConvertible { let name: String var description: String { return name } } let kyle = Person(name: "Kyle") assert(kyle.description == "Kyle", "conforms to CustomStringConvertible") 

$ swift test.swift 

Testing in a playground 

Demo 

assert() testing Upsides • Extremely simple • Built into stdlib Downsides • Hard to manage • Crashes your process, not all tests are ran 

XCTest 

CompileSwift normal x86_64 /Users/kyle/Projects/QueryKit/QueryKit/Tests/SortDescriptorTests.swift cd /Users/kyle/Projects/QueryKit/QueryKit /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift -frontend -c /Users/kyle/Projects/QueryKit/QueryKit/Tests/AttributeTests.swift /Users/kyle/Projects/QueryKit/QueryKit/Tests/QueryKitTests.swift -primary-file /Users/kyle/Projects/QueryKit/QueryKit/Tests/SortDescriptorTests.swift ... Test Suite 'All tests' started at 2016-07-08 01:59:05.062 Test Suite 'QueryKitTests.xctest' started at 2016-07-08 01:59:05.063 Test Suite 'AttributeTests' started at 2016-07-08 01:59:05.063 Test Case '-[QueryKitTests.AttributeTests testAscendingSortDescriptor]' started. Test Case '-[QueryKitTests.AttributeTests testAscendingSortDescriptor]' passed (0.002 seconds). Test Case '-[QueryKitTests.AttributeTests testAttributeExpression]' started. Test Case '-[QueryKitTests.AttributeTests testAttributeExpression]' passed (0.001 seconds). Test Suite 'SortDescriptorTests' passed at 2016-07-08 01:59:05.362. Executed 2 tests, with 0 failures (0 unexpected) in 0.001 (0.003) seconds Test Suite 'QueryKitTests.xctest' passed at 2016-07-08 01:59:05.363. Executed 75 tests, with 0 failures (0 unexpected) in 0.240 (0.300) seconds Test Suite 'All tests' passed at 2016-07-08 01:59:05.364. Executed 75 tests, with 0 failures (0 unexpected) in 0.240 (0.302) seconds ** TEST SUCCEEDED ** 

xctool & xcpretty 

https://github.com/supermarin/xcpretty 

No content

Test Anywhere Protocol (TAP) ... ok 71 - testTypeSafeOrderBySortDescriptor ok 72 - testTypeSafeOrderBySortDescriptors ok 73 - testTypeSafeRelatedFilterPredicate ok 74 - testAscendingSortDescriptor ok 75 - testDescendingSortDescriptor 1..75 

No content

How does XCTest work? $ xcrun -sdk macosx xctest Usage: xctest 

import XCTest class PersonTests : XCTestCase { func testCustomStringConvertible() { let kyle = Person(name: "Kyle") XCTAssertEqual(kyle.description, "Kyle") } } 

How does XCTest work without Objective-C? import XCTest extension PersonTests { static var allTests : [(String, PersonTests -> () throws -> Void)] { return [ ("testCustomStringConvertible", testCustomStringConvertible), ] } } XCTMain([ testCase(PersonTests.allTests), ]) 

No content

Spectre http://spectre.fuller.li/ 

describe("Person") { let kyle = Person(name: "Kyle") $0.it("has a description") { try expect(kyle.description) == "Kyle" } } 

describe("Person") { let kyle = Person(name: "Kyle") $0.it("has a description") { try expect(kyle.description) == "Kyle" } } 

try expect(person.description) == "Kyle" 

func == (lhs: Expectation, rhs: T?) throws 

Reporters 

No content

No content

Custom Reporters 

Testing in Playground 

No content

Demo 

Spectre Runner 

$ ./tests --help FLAGS: --tap - Test Anywhere Protocol Reporter --t - Dot Reporter 

https://github.com/Quick/Quick/ 

import Quick import Nimble class TableOfContentsSpec: QuickSpec { override func spec() { describe("the 'Documentation' directory") { it("has everything you need to get started") { let sections = Directory("Documentation").sections expect(sections).to(contain("Installing Quick")) } } } } 

Cucumber 

Feature: An array Scenario: Appending to an array Given I have an empty array When I add 1 to the array Then I should have 1 item in the array Scenario: Filtering an array Given I have an array with the numbers 1 through 5 When I filter the array for even numbers Then I should have 2 items in the array 

Ploughman https://github.com/kylef/Ploughman/ 

Given I have an empty array var array: [Int]? = nil given("^I have an empty array$") { _ in array = [] } 

When I add X to the array when("^I add (\\d) to the array$") { match in let number = Int(match.groups[1])! array.append(number) } 

Then I should have X items in the array then("^I should have (\\d) items? in the array$") { match in let count = Int(match.groups[1])! try expect(array.count) == count } 

$ ./runner scenarios/array.gherkin 

Real World Example 

Feature: Basic Nest compliance Scenario: Success response status code When I make a GET request to /hello Then I should have a 200 response Scenario: Response header When I make a GET request to /hello Then I should see the header 'Content-Type' with the value 'text/plain' Scenario: Response body When I make a GET request to /hello Then The contents of the body should be 'Hello World' Scenario: Response body (POST) When I make a POST request with body 'Swift' to /hello Then The contents of the body should be 'Hello Swift' 

$ NestTestSuite --host http://localhost:8080 Features/*.feature -> Basic Nest compliance -> Performing a simple GET request 1 scenarios passed, 0 scenarios failed. 

Build Tools 

Xcode 

DIY $ swiftc Sources/*.swift -module-name Hello -emit-library -emit-module -o Hello.dylib $ swiftc Tests/*.swift -lHello -o run-tests $ ./run-tests 

DIY - Make (Makefile) Hello.dylib: swiftc Sources/*.swift -module-name Hello -emit-library -emit-module -o Hello.dylib run-tests: Hello.dylib swiftc Tests/*.swift -lHello -o run-tests .PHONY: test test: run-tests ./run-tests 

$ make test 

Swift Package Manager 

import PackageDescription let package = Package( name: "Curassow", dependencies: [ .Package(url: "https://github.com/kylef/Commander.git", majorVersion: 0, minor: 4), ] ) 

$ swift build 

import PackageDescription let package = Package( name: "Curassow", testDependencies: [ .Package(url: "https://github.com/kylef/Spectre.git", majorVersion: 0, minor: 7), ] ) 

$ swift test 

LinuxMain.swift import XCTest extension PersonTests { static var allTests : [(String, PersonTests -> () throws -> Void)] { return [ ("testCustomStringConvertible", testCustomStringConvertible), ] } } XCTMain([ testCase(PersonTests.allTests), ]) 

Custom Testing with swift test 

! swim https://github.com/kylef/swim/ 

swim Goals • swim is compatible with multiple versions of Swift (2.x, 3.x, etc) • swim allows you to test your Swift project with third-party testing frameworks. • swim is compatible with common Swift Package Manager libraries. • swim is easy to install (does not depend on Swift) 

$ pip install swim 

Package.swift let package = Package( name: "Stencil", testDependencies: [ .Package(url: "https://github.com/kylef/Spectre", majorVersion: 0), ] ) Terminal $ swim build $ swim test 

swiftenv https://swiftenv.fuller.li/ 

$ swiftenv install 2.2.1 

$ swiftenv local DEVELOPMENT-SNAPSHOT-2016-02-08-a 

$ cat .swift-version DEVELOPMENT-SNAPSHOT-2016-02-08-a 

$ swift --version Apple Swift version 2.2.1 $ cd Stencil $ swift --version Apple Swift version 3.0-dev 

swiftenv https://swiftenv.fuller.li/ 

$ brew install kylef/formulae/swiftenv 

Testing against multiple versions of Swift 

$ env SWIFT_VERSION=2.2 swift test --clean $ env SWIFT_VERSION=2.3 swift test --clean $ env SWIFT_VERSION=3.0 swift test --clean 

Swift on Travis CI https://swiftenv.fuller.li/en/latest/ integrations/travis-ci.html 

os: - linux - osx install: - eval "$(curl -sL https://github.com/kylef/.../swiftenv-install.sh)" 

script: swift test 

env: - SWIFT_VERSION=2.2 - SWIFT_VERSION=2.2.1 

• Early stages for testing OSS Swift • New tools are emerging • Swift Package Manager is evolving • Test against all platforms 

kylefuller https://fuller.li/talks