Controllable Randomness in Unit Tests

Controllable Randomness in Unit Tests

Talk by Alexander Voronov

Originally posted here: https://speakerdeck.com/alexandervoronov/controllable-randomness-in-unit-tests

Речь пойдет о:
- В чем польза случайных данных в юнит тестах.
- Зачем и как контролировать случайность.
- Какие инструменты нам помогут достичь этого.
- Как это внедрить в свой проект.

Source code:
https://gist.github.com/a-voronov/12fcc2139fa2d14e31b256b57ef83f27

This talk was made for CocoaFriday #2 ( https://cocoaheads.org.ua/cocoafriday/2 ) which took place Apr 5, 2019.

Video: https://youtu.be/JaRMS8J87WE

Db84cf61fdada06b63f43f310b68b462?s=128

CocoaHeads Ukraine

April 05, 2019
Tweet

Transcript

  1. 4.

    { "id": 6253282, "id_str": "6253282", "name": "Twitter API", "screen_name": "twitterapi",

    "location": "San Francisco, CA", "url": "https://dev.twitter.com", "description": "The Real Twitter API.", "derived": { "locations": [ {} ] }, "protected": true, "verified": false, "followers_count": 21, "friends_count": 32, "listed_count": 9274, "favourites_count": 13, "statuses_count": 42, "created_at": "Mon Nov 29 21:18:15 +0000 2010", "geo_enabled": true, "lang": "zh-cn", "contributors_enabled": false, "profile_background_color": "e8f2f7", "profile_background_image_url": "http://a2.twimg.com/profile_background_images/229557229/twitterapi-bg.png", "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/229557229/twitterapi-bg.png", "profile_background_tile": false, "profile_banner_url": "https://si0.twimg.com/profile_banners/819797/1348102824", "profile_image_url": "http://abs.twimg.com/sticky/default_profile_images/default_profile_normal.png", "profile_image_url_https": "https://abs.twimg.com/sticky/default_profile_images/default_profile_normal.png", "profile_link_color": "0094C2", "profile_sidebar_border_color": "0094C2", "profile_sidebar_fill_color": "a9d9f1", "profile_text_color": "437792", "profile_use_background_image": true, "default_profile": false, "default_profile_image": false, "withheld_in_countries": [ "GR", "HK", "MY" ], "withheld_scope": "user" } @aleks_voronov • CocoaFriday#2
  2. 7.

    DATA GENERATED ON THE FLY ! ▸ Specification OpenAPI •

    Swift Type System ▸ Behavior Properties SwiftCheck @aleks_voronov • CocoaFriday#2
  3. 8.

    MIXED APPROACH Generated by specification • using predefined data sets

    • and manual clarifications @aleks_voronov • CocoaFriday#2
  4. 12.

    func random(from: Int, to: Int) -> Int { return Int(arc4random_uniform(UInt32(to)

    - UInt32(from) + 1) + UInt32(from)) } @aleks_voronov • CocoaFriday#2
  5. 14.

    4.2 protocol RandomNumberGenerator { mutating func next() -> UInt64 ...

    } github.com/apple/swift-evolution/blob/master/proposals/0202-random-unification.md @aleks_voronov • CocoaFriday#2
  6. 16.

    4.2 Bool • Int • Double • Array • Set

    • Dictionary random() -> T random<G: RandomNumberGenerator>(inout G) -> T github.com/apple/swift-evolution/blob/master/proposals/0202-random-unification.md @aleks_voronov • CocoaFriday#2
  7. 17.

    GENERIC 'RANDOM' INTERFACE protocol Random { static func random<G: RandomNumberGenerator>(

    using generator: inout G ) -> Self } @aleks_voronov • CocoaFriday#2
  8. 18.

    CONFORMING BASICS extension Character: Random { ... } extension UnicodeScalar:

    Random { ... } extension FloatingPoint where Self: Random { ... } extension FixedWidthInteger where Self: Random { ... } @aleks_voronov • CocoaFriday#2
  9. 19.

    extension Bool: Random { } extension Float: Random { }

    extension Double: Random { } extension Int: Random { } extension Int64: Random { } extension Int32: Random { } extension Int16: Random { } extension Int8: Random { } extension UInt: Random { } extension UInt64: Random { } extension UInt32: Random { } extension UInt16: Random { } extension UInt8: Random { } @aleks_voronov • CocoaFriday#2
  10. 20.

    RANDOM COLLECTIONS extension Array: Random where Element: Random { static

    func random<G: RandomNumberGenerator>( using generator: inout G ) -> [Element] { ... } } @aleks_voronov • CocoaFriday#2
  11. 21.

    RANDOM ENUMS protocol RandomAll: Random { static func allRandom<G: RandomNumberGenerator>(

    using generator: inout G ) -> [Self] } @aleks_voronov • CocoaFriday#2
  12. 22.

    OPTIONAL EXAMPLE extension Optional: RandomAll where Wrapped: Random { public

    static func allRandom<G: RandomNumberGenerator>( using generator: inout G ) -> [Wrapped?] { return [.none, .some(.random(using: &generator))] } } extension Optional: Random where Wrapped: Random {} @aleks_voronov • CocoaFriday#2
  13. 23.

    RANDOM TUPLES func randomTuple< A: Random, B: Random, G: RandomNumberGenerator

    >(using generator: inout G) -> (A, B) { return ( .random(using: &generator), .random(using: &generator) ) } @aleks_voronov • CocoaFriday#2
  14. 25.

    LET'S USE IT struct User { let id: String let

    name: String let age: Int let friends: (followers: Int, followees: Int) } @aleks_voronov • CocoaFriday#2
  15. 26.

    extension User: Random { static func random<G: RandomNumberGenerator>( using generator:

    inout G ) -> User { return User( id: .random(using: &generator), name: .random(using: &generator), age: .random(using: &generator), friends: randomTuple(using: &generator) ) } } @aleks_voronov • CocoaFriday#2
  16. 33.

    struct PRNG: RandomNumberGenerator { typealias State = UInt64 private(set) var

    state: State init() { var generator = SystemRandomNumberGenerator() state = .random(using: &generator) } init(seed: State) { state = seed } mutating func next() -> UInt64 { state = magic(state) return state } } @aleks_voronov • CocoaFriday#2
  17. 35.

    If an NSPrincipalClass key is declared in the test bundle's

    Info.plist file, XCTest automatically creates a single instance of that class when the test bundle is loaded ... – developer.apple.com/documentation/... @aleks_voronov • CocoaFriday#2
  18. 36.

    public var R = PRNG() class RandomSeed: NSObject { override

    init() { super.init() let seed = "\(R.state)" print(seed) } } @aleks_voronov • CocoaFriday#2
  19. 39.

    GET EVERY FAILED TEST SEED ✅ ✅ ❌ ✅ ❌

    –––––––8–––––12–––––––––––23–42 @aleks_voronov • CocoaFriday#2
  20. 41.

    If an NSPrincipalClass key is declared in the test bundle's

    Info.plist file, XCTest automatically creates a single instance of that class when the test bundle is loaded. You can use this instance as a place to register observers or do other pretesting global setup before testing for that bundle begins. – developer.apple.com/documentation/xctest/xctestobservationcenter @aleks_voronov • CocoaFriday#2
  21. 42.

    XCTESTOBSERVATION ▸ START: Save Seed 1 ▸ FAIL: Log Seed

    * ▸ SUCCESS: Do nothing * developer.apple.com/documentation/xctest/xctestobservation @aleks_voronov • CocoaFriday#2
  22. 45.

    import Foundation {# So far getting imports hardcoded from config,

    correct approach might be found here: https://github.com/krzysztofzablocki/Sourcery/issues/670 #} {% for import in argument.imports %} import {{ import }} {% endfor %} {% if argument.testable %}{% for testable in argument.testable %} @testable import {{ testable }} {% endfor %}{% endif %} {% macro randomValue type %}{% if type.kind == "protocol" %}{{ type.inheritedTypes.0.name }}{% endif %}{% endmacro %} // MARK: - Structs {# Random Struct #} {% macro rng %}&{{ argument.rng }}{% endmacro %} {% macro customRandomValueType variable %}{% if variable.annotations.random %}{{ variable.annotations.random }}{% endif %}{% endmacro %} {% macro randomValueUsingGenerator variable %}{% call customRandomValueType variable %}{% if variable.isTuple %}randomTuple(using: &generator){% else %}.random(using: &generator){% endif %}{% endmacro %} {% macro randomValueUsingRNG variable %}{% call customRandomValueType variable %}{% if variable.isTuple %}randomTuple(using: {% call rng %}){% else %}.random(using: {% call rng %}){% endif %}{% endmacro %} {% for type in types.structs where type|annotated:"Random" %} extension {{ type.name }}: Random { public static func random<G: RandomNumberGenerator>(using generator: inout G) -> {{ type.name }} { return {{ type.name }}( {% for variable in type.variables where not variable.isComputed %} {{ variable.name }}: {% call randomValueUsingGenerator variable %}{% if not forloop.last %},{% endif %} {% endfor %} ) } {% if argument.rng %} {# user-friendly version, so that you don't have to bother with closures and incoming generators. But it's tightly coupled to `rng` argument value from sourcery config #} public static func random( {% for variable in type.variables where not variable.isComputed %} {{ variable.name }}: {{ variable.typeName }} = {% call randomValueUsingRNG variable %}{% if not forloop.last %},{% endif %} {% endfor %} ) -> {{ type.name }} { return {{ type.name }}( {% for variable in type.variables where not variable.isComputed %} {{ variable.name }}: {{ variable.name }}{% if not forloop.last %},{% endif %} {% endfor %} ) } {% else %} {# this one is generated additionally, so that you can have everything random except for some selected fields #} public static func random<G: RandomNumberGenerator>( _ generator: inout G, {% for variable in type.variables where not variable.isComputed %} {{ variable.name }}: (inout G) -> {{ variable.typeName }} = { generator in {% call randomValueUsingGenerator variable %} }{% if not forloop.last %},{% endif %} {% endfor %} ) -> {{ type.name }} { return {{ type.name }}( {% for variable in type.variables where not variable.isComputed %} {{ variable.name }}: {{ variable.name }}(&generator){% if not forloop.last %},{% endif %} {% endfor %} ) } {% endif %} } {% endfor %} // MARK: - Enums {# Random Enum #} {% macro randomEnumCaseUsingGenerator case %}.{{ case.name }}{% if case.hasAssociatedValue %}({% for value in case.associatedValues %}{% if value.localName %}{{ value.localName }}: {% endif %}{% call randomValueUsingGenerator value %}{% if not forloop.last %}, {% endif %}{% endfor %}){% endif %}{% endmacro %} {% macro randomIfCaseUsingGenerator case %}{% if case.hasAssociatedValue %}{{ case.name }}(&generator){% else %}.{{ case.name }}{% endif %}{% endmacro %} {% macro randomEnumCaseUsingRNG case %}.{{ case.name }}{% if case.hasAssociatedValue %}({% for value in case.associatedValues %}{% if value.localName %}{{ value.localName }}: {% endif %}{% call randomValueUsingRNG value %}{% if not forloop.last %}, {% endif %}{% endfor %}){% endif %}{% endmacro %} {% macro randomIfCaseUsingRNG case %}{% if case.hasAssociatedValue %}{{ case.name }}{% else %}.{{ case.name }}{% endif %}{% endmacro %} {% for enum in types.enums where enum|annotated:"Random" %} {% if enum.cases.count == 0 %} #warning("`{{ enum.name }}` is an uninhabitant type and no value of it can be created, thus it can't have random value as well.") // extension {{ enum.name }}: Random { } {% elif not enum.hasAssociatedValues %} extension {{ enum.name }}: Random { {# I'd expect to check `enum.based` property, but it's always empty ¯\_(ϑ)_/¯ #} {% if enum.rawTypeName.name != "CaseIterable" and enum.inheritedTypes|join:" "|!contains:"CaseIterable" %} #warning("Please, conform to `CaseIterable` protocol, so that compiler takes care of this") public static let allCases: [{{ enum.name }}] = [{% for case in enum.cases %}.{{ case.name }}{% if not forloop.last %}, {% endif %}{% endfor %}] {% endif %} public static func random<G: RandomNumberGenerator>(using generator: inout G) -> {{ enum.name }} { return allCases.randomElement(using: &generator)! } } {% else %} extension {{ enum.name }}: RandomAll { public static func allRandom<G: RandomNumberGenerator>(using generator: inout G) -> [{{ enum.name }}] { {% if enum.cases.count == 1 %} return [{% call randomEnumCaseUsingGenerator enum.cases.0 %}] {% else %} return [{% for case in enum.cases %} {% call randomEnumCaseUsingGenerator case %}{% if not forloop.last %},{% endif %}{% endfor %} ] {% endif %} } {# alternative version, so that you don't have to bother with closures and incoming generators but it's tightly coupled to `rng` argument value from sourcery config and it executes and calculates randoms for all cases even though only one will be needed (thus a bit more expensive) #} {% if argument.rng %} public static func random( {% for case in enum.cases where case.associatedValues %} {{ case.name }}: {{ enum.name }} = {% call randomEnumCaseUsingRNG case %}{% if not forloop.last %},{% endif %} {% endfor %} ) -> {{ enum.name }} { {% if enum.cases.count == 1 %} return {% call randomIfCaseUsingRNG enum.cases.0 %} {% else %} return [ {% for case in enum.cases %} {% call randomIfCaseUsingRNG case %}{% if not forloop.last %},{% endif %} {% endfor %} ].randomElement(using: {% call rng %})! {% endif %} } {% else %} {# same for enum - you can override `random` for some cases and leave others generated by default #} public static func random<G: RandomNumberGenerator>( _ generator: inout G, {% for case in enum.cases where case.associatedValues %} {{ case.name }}: (inout G) -> {{ enum.name }} = { generator in {% call randomEnumCaseUsingGenerator case %} }{% if not forloop.last %},{% endif %} {% endfor %} ) -> {{ enum.name }} { {% if enum.cases.count == 1 %} return {% call randomIfCaseUsingGenerator enum.cases.0 %} {% else %} return [ {% for case in enum.cases %} {% call randomIfCaseUsingGenerator case %}{% if not forloop.last %},{% endif %} {% endfor %} ].randomElement(using: &generator)! {% endif %} } {% endif %} } {% endif %} {% endfor %} @aleks_voronov • CocoaFriday#2
  23. 48.

    INTERESTING ▸ pointfree.co : ep30 - ep32 • ep47 -

    ep50 ▸ github.com/pointfreeco/swift-gen ▸ github.com/typelift/SwiftCheck ▸ ieeexplore.ieee.org/document/6963470 : Oracles @aleks_voronov • CocoaFriday#2