Testing without Xcode - CMD+U 2016

Testing without Xcode - CMD+U 2016

D200a17dd269fd4001bacb11662dab4b?s=128

Kyle Fuller

July 08, 2016
Tweet

Transcript

  1. Testing without Xcode Kyle Fuller

  2. None
  3. None
  4. None
  5. None
  6. None
  7. None
  8. Open Source Libraries

  9. Existing code on other Platforms

  10. Editors

  11. Who am I

  12. None
  13. Agenda • How other communities do testing • How testing

    is done in Swift today • How we can do testing in Swift in the future
  14. How do other communities solve testing? • Ruby ! •

    C / C++ ䷢
  15. Testing in Ruby describe '.authenticate' do context 'when logged in'

    do it { is_expected.to respond_with 200 } end end
  16. Testing in Ruby $ rspec test.rb Finished in 0.0016 seconds

    (files took 0.2153 seconds to load) 5 examples, 0 failures
  17. Testing in C / C++ int main(int argc, const char

    *argv[]) { user_t user = user_alloc("kyle"); // Test User's name assert(user.name == "kyle"); // Test User's priviledges assert(user.is_admin, 0); return 0; }
  18. Testing in C / C++ $ clang test.c -o tests

    $ ./tests Tests passed
  19. Testing in C / C++ #define CATCH_CONFIG_MAIN #include "catch.hpp" TEST_CASE(

    "Factorials are computed", "[factorial]" ) { REQUIRE( Factorial(1) == 1 ); REQUIRE( Factorial(2) == 2 ); REQUIRE( Factorial(3) == 6 ); REQUIRE( Factorial(10) == 3628800 ); }
  20. Test Framework ~= Test Runner

  21. Testing in Swift • assert • XCTest • Quick (uses

    XCTest) • Spectre • Ploughman • Custom testing • CLI tool
  22. Testing with assert

  23. assert()

  24. assert(person.name == "Kyle")

  25. assert(person.name == "Kyle", "Persons name should be Kyle")

  26. struct Person: CustomStringConvertible { let name: String var description: String

    { return name } } let kyle = Person(name: "Kyle") assert(kyle.description == "Kyle", "conforms to CustomStringConvertible")
  27. $ swift test.swift

  28. Testing in a playground

  29. Demo

  30. assert() testing Upsides • Extremely simple • Built into stdlib

    Downsides • Hard to manage • Crashes your process, not all tests are ran
  31. XCTest

  32. CompileSwift normal x86_64 /Users/kyle/Projects/QueryKit/QueryKit/Tests/SortDescriptorTests.swift cd /Users/kyle/Projects/QueryKit/QueryKit /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift -frontend -c /Users/kyle/Projects/QueryKit/QueryKit/Tests/AttributeTests.swift

    /Users/kyle/Projects/QueryKit/QueryKit/Tests/QueryKitTests.swift -primary-file /Users/kyle/Projects/QueryKit/QueryKit/Tests/SortDescriptorTests.swift ... Test Suite 'All tests' started at 2016-07-08 01:59:05.062 Test Suite 'QueryKitTests.xctest' started at 2016-07-08 01:59:05.063 Test Suite 'AttributeTests' started at 2016-07-08 01:59:05.063 Test Case '-[QueryKitTests.AttributeTests testAscendingSortDescriptor]' started. Test Case '-[QueryKitTests.AttributeTests testAscendingSortDescriptor]' passed (0.002 seconds). Test Case '-[QueryKitTests.AttributeTests testAttributeExpression]' started. Test Case '-[QueryKitTests.AttributeTests testAttributeExpression]' passed (0.001 seconds). Test Suite 'SortDescriptorTests' passed at 2016-07-08 01:59:05.362. Executed 2 tests, with 0 failures (0 unexpected) in 0.001 (0.003) seconds Test Suite 'QueryKitTests.xctest' passed at 2016-07-08 01:59:05.363. Executed 75 tests, with 0 failures (0 unexpected) in 0.240 (0.300) seconds Test Suite 'All tests' passed at 2016-07-08 01:59:05.364. Executed 75 tests, with 0 failures (0 unexpected) in 0.240 (0.302) seconds ** TEST SUCCEEDED **
  33. xctool & xcpretty

  34. https://github.com/supermarin/xcpretty

  35. None
  36. Test Anywhere Protocol (TAP) ... ok 71 - testTypeSafeOrderBySortDescriptor ok

    72 - testTypeSafeOrderBySortDescriptors ok 73 - testTypeSafeRelatedFilterPredicate ok 74 - testAscendingSortDescriptor ok 75 - testDescendingSortDescriptor 1..75
  37. None
  38. How does XCTest work? $ xcrun -sdk macosx xctest Usage:

    xctest <path of unit to be tested>
  39. import XCTest class PersonTests : XCTestCase { func testCustomStringConvertible() {

    let kyle = Person(name: "Kyle") XCTAssertEqual(kyle.description, "Kyle") } }
  40. How does XCTest work without Objective-C? import XCTest extension PersonTests

    { static var allTests : [(String, PersonTests -> () throws -> Void)] { return [ ("testCustomStringConvertible", testCustomStringConvertible), ] } } XCTMain([ testCase(PersonTests.allTests), ])
  41. None
  42. Spectre http://spectre.fuller.li/

  43. describe("Person") { let kyle = Person(name: "Kyle") $0.it("has a description")

    { try expect(kyle.description) == "Kyle" } }
  44. describe("Person") { let kyle = Person(name: "Kyle") $0.it("has a description")

    { try expect(kyle.description) == "Kyle" } }
  45. try expect(person.description) == "Kyle"

  46. func == <T> (lhs: Expectation<T>, rhs: T?) throws

  47. Reporters

  48. None
  49. None
  50. Custom Reporters

  51. Testing in Playground

  52. None
  53. Demo

  54. Spectre Runner

  55. $ ./tests --help FLAGS: --tap - Test Anywhere Protocol Reporter

    --t - Dot Reporter
  56. https://github.com/Quick/Quick/

  57. import Quick import Nimble class TableOfContentsSpec: QuickSpec { override func

    spec() { describe("the 'Documentation' directory") { it("has everything you need to get started") { let sections = Directory("Documentation").sections expect(sections).to(contain("Installing Quick")) } } } }
  58. Cucumber

  59. Feature: An array Scenario: Appending to an array Given I

    have an empty array When I add 1 to the array Then I should have 1 item in the array Scenario: Filtering an array Given I have an array with the numbers 1 through 5 When I filter the array for even numbers Then I should have 2 items in the array
  60. Ploughman https://github.com/kylef/Ploughman/

  61. Given I have an empty array var array: [Int]? =

    nil given("^I have an empty array$") { _ in array = [] }
  62. When I add X to the array when("^I add (\\d)

    to the array$") { match in let number = Int(match.groups[1])! array.append(number) }
  63. Then I should have X items in the array then("^I

    should have (\\d) items? in the array$") { match in let count = Int(match.groups[1])! try expect(array.count) == count }
  64. $ ./runner scenarios/array.gherkin

  65. Real World Example

  66. Feature: Basic Nest compliance Scenario: Success response status code When

    I make a GET request to /hello Then I should have a 200 response Scenario: Response header When I make a GET request to /hello Then I should see the header 'Content-Type' with the value 'text/plain' Scenario: Response body When I make a GET request to /hello Then The contents of the body should be 'Hello World' Scenario: Response body (POST) When I make a POST request with body 'Swift' to /hello Then The contents of the body should be 'Hello Swift'
  67. $ NestTestSuite --host http://localhost:8080 Features/*.feature -> Basic Nest compliance ->

    Performing a simple GET request 1 scenarios passed, 0 scenarios failed.
  68. Build Tools

  69. Xcode

  70. DIY $ swiftc Sources/*.swift -module-name Hello -emit-library -emit-module -o Hello.dylib

    $ swiftc Tests/*.swift -lHello -o run-tests $ ./run-tests
  71. DIY - Make (Makefile) Hello.dylib: swiftc Sources/*.swift -module-name Hello -emit-library

    -emit-module -o Hello.dylib run-tests: Hello.dylib swiftc Tests/*.swift -lHello -o run-tests .PHONY: test test: run-tests ./run-tests
  72. $ make test

  73. Swift Package Manager

  74. import PackageDescription let package = Package( name: "Curassow", dependencies: [

    .Package(url: "https://github.com/kylef/Commander.git", majorVersion: 0, minor: 4), ] )
  75. $ swift build

  76. import PackageDescription let package = Package( name: "Curassow", testDependencies: [

    .Package(url: "https://github.com/kylef/Spectre.git", majorVersion: 0, minor: 7), ] )
  77. $ swift test

  78. LinuxMain.swift import XCTest extension PersonTests { static var allTests :

    [(String, PersonTests -> () throws -> Void)] { return [ ("testCustomStringConvertible", testCustomStringConvertible), ] } } XCTMain([ testCase(PersonTests.allTests), ])
  79. Custom Testing with swift test

  80. ! swim https://github.com/kylef/swim/

  81. swim Goals • swim is compatible with multiple versions of

    Swift (2.x, 3.x, etc) • swim allows you to test your Swift project with third-party testing frameworks. • swim is compatible with common Swift Package Manager libraries. • swim is easy to install (does not depend on Swift)
  82. $ pip install swim

  83. Package.swift let package = Package( name: "Stencil", testDependencies: [ .Package(url:

    "https://github.com/kylef/Spectre", majorVersion: 0), ] ) Terminal $ swim build $ swim test
  84. swiftenv https://swiftenv.fuller.li/

  85. $ swiftenv install 2.2.1

  86. $ swiftenv local DEVELOPMENT-SNAPSHOT-2016-02-08-a

  87. $ cat .swift-version DEVELOPMENT-SNAPSHOT-2016-02-08-a

  88. $ swift --version Apple Swift version 2.2.1 $ cd Stencil

    $ swift --version Apple Swift version 3.0-dev
  89. swiftenv https://swiftenv.fuller.li/

  90. $ brew install kylef/formulae/swiftenv

  91. Testing against multiple versions of Swift

  92. $ env SWIFT_VERSION=2.2 swift test --clean $ env SWIFT_VERSION=2.3 swift

    test --clean $ env SWIFT_VERSION=3.0 swift test --clean
  93. Swift on Travis CI https://swiftenv.fuller.li/en/latest/ integrations/travis-ci.html

  94. os: - linux - osx install: - eval "$(curl -sL

    https://github.com/kylef/.../swiftenv-install.sh)"
  95. script: swift test

  96. env: - SWIFT_VERSION=2.2 - SWIFT_VERSION=2.2.1

  97. • Early stages for testing OSS Swift • New tools

    are emerging • Swift Package Manager is evolving • Test against all platforms
  98. kylefuller https://fuller.li/talks