Testing without Xcode - CMD+U 2016

Transcript

1. Testing without Xcode Kyle Fuller
8. Open Source Libraries
9. Existing code on other Platforms
10. Editors
11. Who am I
13. Agenda • How other communities do testing • How testing is done in Swift today • How we can do testing in Swift in the future
14. How do other communities solve testing? • Ruby ! • C / C++ ䷢
15. Testing in Ruby describe '.authenticate' do context 'when logged in' do it { is_expected.to respond_with 200 } end end
16. Testing in Ruby $ rspec test.rb Finished in 0.0016 seconds (files took 0.2153 seconds to load) 5 examples, 0 failures
17. Testing in C / C++ int main(int argc, const char *argv[]) { user_t user = user_alloc("kyle"); // Test User's name assert(user.name == "kyle"); // Test User's priviledges assert(user.is_admin, 0); return 0; }
18. Testing in C / C++ $ clang test.c -o tests $ ./tests Tests passed
19. Testing in C / C++ #define CATCH_CONFIG_MAIN #include "catch.hpp" TEST_CASE( "Factorials are computed", "[factorial]" ) { REQUIRE( Factorial(1) == 1 ); REQUIRE( Factorial(2) == 2 ); REQUIRE( Factorial(3) == 6 ); REQUIRE( Factorial(10) == 3628800 ); }
20. Test Framework ~= Test Runner
21. Testing in Swift • assert • XCTest • Quick (uses XCTest) • Spectre • Ploughman • Custom testing • CLI tool
22. Testing with assert
23. assert()
24. assert(person.name == "Kyle")
25. assert(person.name == "Kyle", "Persons name should be Kyle")
26. struct Person: CustomStringConvertible { let name: String var description: String { return name } } let kyle = Person(name: "Kyle") assert(kyle.description == "Kyle", "conforms to CustomStringConvertible")
27. $ swift test.swift
28. Testing in a playground
29. Demo
30. assert() testing Upsides • Extremely simple • Built into stdlib Downsides • Hard to manage • Crashes your process, not all tests are ran
31. XCTest
32. CompileSwift normal x86_64 /Users/kyle/Projects/QueryKit/QueryKit/Tests/SortDescriptorTests.swift cd /Users/kyle/Projects/QueryKit/QueryKit /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift -frontend -c /Users/kyle/Projects/QueryKit/QueryKit/Tests/AttributeTests.swift /Users/kyle/Projects/QueryKit/QueryKit/Tests/QueryKitTests.swift -primary-file /Users/kyle/Projects/QueryKit/QueryKit/Tests/SortDescriptorTests.swift ... Test Suite 'All tests' started at 2016-07-08 01:59:05.062 Test Suite 'QueryKitTests.xctest' started at 2016-07-08 01:59:05.063 Test Suite 'AttributeTests' started at 2016-07-08 01:59:05.063 Test Case '-[QueryKitTests.AttributeTests testAscendingSortDescriptor]' started. Test Case '-[QueryKitTests.AttributeTests testAscendingSortDescriptor]' passed (0.002 seconds). Test Case '-[QueryKitTests.AttributeTests testAttributeExpression]' started. Test Case '-[QueryKitTests.AttributeTests testAttributeExpression]' passed (0.001 seconds). Test Suite 'SortDescriptorTests' passed at 2016-07-08 01:59:05.362. Executed 2 tests, with 0 failures (0 unexpected) in 0.001 (0.003) seconds Test Suite 'QueryKitTests.xctest' passed at 2016-07-08 01:59:05.363. Executed 75 tests, with 0 failures (0 unexpected) in 0.240 (0.300) seconds Test Suite 'All tests' passed at 2016-07-08 01:59:05.364. Executed 75 tests, with 0 failures (0 unexpected) in 0.240 (0.302) seconds ** TEST SUCCEEDED **
33. xctool & xcpretty
34. https://github.com/supermarin/xcpretty
36. Test Anywhere Protocol (TAP) ... ok 71 - testTypeSafeOrderBySortDescriptor ok 72 - testTypeSafeOrderBySortDescriptors ok 73 - testTypeSafeRelatedFilterPredicate ok 74 - testAscendingSortDescriptor ok 75 - testDescendingSortDescriptor 1..75
38. How does XCTest work? $ xcrun -sdk macosx xctest Usage: xctest <path of unit to be tested>
39. import XCTest class PersonTests : XCTestCase { func testCustomStringConvertible() { let kyle = Person(name: "Kyle") XCTAssertEqual(kyle.description, "Kyle") } }
40. How does XCTest work without Objective-C? import XCTest extension PersonTests { static var allTests : [(String, PersonTests -> () throws -> Void)] { return [ ("testCustomStringConvertible", testCustomStringConvertible), ] } } XCTMain([ testCase(PersonTests.allTests), ])
42. Spectre http://spectre.fuller.li/
43. describe("Person") { let kyle = Person(name: "Kyle") $0.it("has a description") { try expect(kyle.description) == "Kyle" } }
44. describe("Person") { let kyle = Person(name: "Kyle") $0.it("has a description") { try expect(kyle.description) == "Kyle" } }
45. try expect(person.description) == "Kyle"
46. func == <T> (lhs: Expectation<T>, rhs: T?) throws
47. Reporters
50. Custom Reporters
51. Testing in Playground
53. Demo
54. Spectre Runner
55. $ ./tests --help FLAGS: --tap - Test Anywhere Protocol Reporter --t - Dot Reporter
56. https://github.com/Quick/Quick/
57. import Quick import Nimble class TableOfContentsSpec: QuickSpec { override func spec() { describe("the 'Documentation' directory") { it("has everything you need to get started") { let sections = Directory("Documentation").sections expect(sections).to(contain("Installing Quick")) } } } }
58. Cucumber
59. Feature: An array Scenario: Appending to an array Given I have an empty array When I add 1 to the array Then I should have 1 item in the array Scenario: Filtering an array Given I have an array with the numbers 1 through 5 When I filter the array for even numbers Then I should have 2 items in the array
60. Ploughman https://github.com/kylef/Ploughman/
61. Given I have an empty array var array: [Int]? = nil given("^I have an empty array$") { _ in array = [] }
62. When I add X to the array when("^I add (\\d) to the array$") { match in let number = Int(match.groups[1])! array.append(number) }
63. Then I should have X items in the array then("^I should have (\\d) items? in the array$") { match in let count = Int(match.groups[1])! try expect(array.count) == count }
64. $ ./runner scenarios/array.gherkin
65. Real World Example
66. Feature: Basic Nest compliance Scenario: Success response status code When I make a GET request to /hello Then I should have a 200 response Scenario: Response header When I make a GET request to /hello Then I should see the header 'Content-Type' with the value 'text/plain' Scenario: Response body When I make a GET request to /hello Then The contents of the body should be 'Hello World' Scenario: Response body (POST) When I make a POST request with body 'Swift' to /hello Then The contents of the body should be 'Hello Swift'
67. $ NestTestSuite --host http://localhost:8080 Features/*.feature -> Basic Nest compliance -> Performing a simple GET request 1 scenarios passed, 0 scenarios failed.
68. Build Tools
69. Xcode
70. DIY $ swiftc Sources/*.swift -module-name Hello -emit-library -emit-module -o Hello.dylib $ swiftc Tests/*.swift -lHello -o run-tests $ ./run-tests
71. DIY - Make (Makefile) Hello.dylib: swiftc Sources/*.swift -module-name Hello -emit-library -emit-module -o Hello.dylib run-tests: Hello.dylib swiftc Tests/*.swift -lHello -o run-tests .PHONY: test test: run-tests ./run-tests
72. $ make test
73. Swift Package Manager
74. import PackageDescription let package = Package( name: "Curassow", dependencies: [ .Package(url: "https://github.com/kylef/Commander.git", majorVersion: 0, minor: 4), ] )
75. $ swift build
76. import PackageDescription let package = Package( name: "Curassow", testDependencies: [ .Package(url: "https://github.com/kylef/Spectre.git", majorVersion: 0, minor: 7), ] )
77. $ swift test
78. LinuxMain.swift import XCTest extension PersonTests { static var allTests : [(String, PersonTests -> () throws -> Void)] { return [ ("testCustomStringConvertible", testCustomStringConvertible), ] } } XCTMain([ testCase(PersonTests.allTests), ])
79. Custom Testing with swift test
80. ! swim https://github.com/kylef/swim/
81. swim Goals • swim is compatible with multiple versions of Swift (2.x, 3.x, etc) • swim allows you to test your Swift project with third-party testing frameworks. • swim is compatible with common Swift Package Manager libraries. • swim is easy to install (does not depend on Swift)
82. $ pip install swim
83. Package.swift let package = Package( name: "Stencil", testDependencies: [ .Package(url: "https://github.com/kylef/Spectre", majorVersion: 0), ] ) Terminal $ swim build $ swim test
84. swiftenv https://swiftenv.fuller.li/
85. $ swiftenv install 2.2.1
86. $ swiftenv local DEVELOPMENT-SNAPSHOT-2016-02-08-a
87. $ cat .swift-version DEVELOPMENT-SNAPSHOT-2016-02-08-a
88. $ swift --version Apple Swift version 2.2.1 $ cd Stencil $ swift --version Apple Swift version 3.0-dev
89. swiftenv https://swiftenv.fuller.li/
90. $ brew install kylef/formulae/swiftenv
91. Testing against multiple versions of Swift
92. $ env SWIFT_VERSION=2.2 swift test --clean $ env SWIFT_VERSION=2.3 swift test --clean $ env SWIFT_VERSION=3.0 swift test --clean
93. Swift on Travis CI https://swiftenv.fuller.li/en/latest/ integrations/travis-ci.html
94. os: - linux - osx install: - eval "$(curl -sL https://github.com/kylef/.../swiftenv-install.sh)"
95. script: swift test
96. env: - SWIFT_VERSION=2.2 - SWIFT_VERSION=2.2.1
97. • Early stages for testing OSS Swift • New tools are emerging • Swift Package Manager is evolving • Test against all platforms
98. kylefuller https://fuller.li/talks