Upgrade to Pro — share decks privately, control downloads, hide ads and more …

iOS BDD Beatdown

iOS BDD Beatdown

Figuring out how to do BDD for iOS isn't exactly clear cut. There are a number of contenders for framework of choice for both the integration and unit layers. Even when you've figured out which framework, it's not always easy to figure out what parts of your application to test and why. You're in luck! I've spent the last 1.5 years doing all the hard work for you! I'll offer a comparison of the popular BDD frameworks so you can figure out what will work best for you. I'll also be going over how to effectively BDD iOS applications as well as looking at how to retrofit your application with tests to cover the application's high value paths.

65df4ac904d6a73e5a70b0db927a2917?s=128

Stephen Caudill

February 16, 2013
Tweet

Transcript

  1. Stephen Caudill I’ll be your announcer for today’s

  2. iOS BDD Beatdown Stephen Caudill Ask the audience what their

    level of experience with BDD is Show of hands: if you know what BDD is. If you practice BDD. If you do it on iOS. BDD is an evolution on Test Driven Development that provides a specific shared vocabulary that bridges the divide between Business and Technology
  3. 2 Years of BDD on iOS Stephen Caudill I’m here

    to help you leverage my experience in your selection of a BDD framework
  4. Evaluation Criteria Stephen Caudill

  5. Ubiquitous Language Stephen Caudill This is our shared vocabulary and

    the thing that separates BDD from TDD
  6. • Run our specs atomically • Pend specs • Set

    expectations • Mocks/Doubles • Run our suite on devices • With a minimal amount of pain Stephen Caudill as developers we also need a suite of other tools
  7. TDD with BDD Language? Stephen Caudill Seems like an acceptable

    shim, let’s look at some of the offerings
  8. OCUnit Stephen Caudill http://www.sente.ch/software/ocunit

  9. Ease of Installation Stephen Caudill A+

  10. Ubiquitous Language Stephen Caudill Not so much... Note about “Ubiquitous

    Language” at the unit level... Terse external assertions lead to thinking at a low level... and I don’t know about you, but I constantly forget which order the expected and actual results should go in.
  11. Stephen Caudill

  12. Focused Test Runs Stephen Caudill No The closest you’ll come

    is having a duplicate OCUnit target and adding only the class you want to run to that
  13. Expectations Stephen Caudill Not so much What is an expectation?

  14. Mocks/Doubles Stephen Caudill BYOF OCMock is a popular solution for

    this... also provides expectations.
  15. Pending Tests Stephen Caudill Nope Common workaround is to use

    #warning imperatives
  16. Run tests on Device? Stephen Caudill No This is disappointing,

    given the “official” nature of the framework
  17. Documentation Stephen Caudill NO OFFICIAL DOCS! The closest you’ll get

    is the “sentesting macro reference” from apple. Googling specific cases is easy due to the sheer amount of people using it.
  18. Verdict Stephen Caudill Serviceable But far from ideal

  19. GHUnit Stephen Caudill http://github.com/gabriel/gh-unit

  20. Ease of Installation Stephen Caudill C- Readme says to download

    the framework (presumably prebuilt)... It doesn’t say where from though and googling didn’t help. You can make it yourself by cloning the repo, running make from the Project-iOS directory, then running a script in the Scripts directory... there is no documentation around this.
  21. Ubiquitous Language Stephen Caudill TDD, so no.

  22. Focused Runs Stephen Caudill Yes!

  23. And it looks like this

  24. Expectations Stephen Caudill Same issue here as with OCUnit...

  25. Mocks/Doubles Stephen Caudill Bring Your Own Again, OCMock is the

    goto solution here.
  26. Pending Tests Stephen Caudill Nope #warning imperatives

  27. Run tests on Device? Stephen Caudill Yes!

  28. Documentation Stephen Caudill Hit or miss... Often outdated or incorrect,

    with respect to installation
  29. Verdict Stephen Caudill Better than OCUnit Transitioning to this from

    OCUnit is a breeze! And lo, there was a pretty test runner.
  30. OCHamcrest Stephen Caudill http://github.com/hamcrest/OCHamcrest

  31. Ease of Installation Stephen Caudill A it's initially unclear what

    target to install to… this augments SenTestingKit though, so just add it to a new or existing SenTestingKit target
  32. Ubiquitous Language Stephen Caudill Getting Closer

  33. Stephen Caudill We’re starting to disambiguate between our expectation and

    our actual result, at least...
  34. Focused Runs Stephen Caudill Sadly, no.

  35. Expectations Stephen Caudill No, but... http://github.com/jonreid/OCMockito Jon Reid (author of

    OCHamcrest) also wrote OCMockito, which handles expectations.
  36. Mocks/Doubles Stephen Caudill Bring Your Own OCMockito is our huckleberry

    here as well. OCMock is another option here.
  37. Pending Tests Stephen Caudill Uh-uh. our old friend the #warning

    imperative
  38. Run tests on Device? Stephen Caudill Yes, indeed.

  39. Documentation Stephen Caudill Good! Readme calls out the important stuff,

    there are lots of tutorials and the API has full documentation.
  40. Verdict Stephen Caudill Best of the TDD lot

  41. Let’s talk BDD! Stephen Caudill

  42. Specta Stephen Caudill http://github.com/petejkim/specta

  43. Ease of Installation Stephen Caudill A Cocoapods is the recommended

    manner of installation and it makes it a breeze!
  44. Cocoapods http://cocoapods.org/ Stephen Caudill quick aside...

  45. Ubiquitous Language Stephen Caudill And how!

  46. Stephen Caudill

  47. Focused Runs Stephen Caudill Negative

  48. Expectations Stephen Caudill Bring Your Own http://github.com/petejkim/expecta Sister project called

    Expecta, from the same developer. Worth noting that Specta isn’t really useful without Expecta.
  49. Stephen Caudill

  50. Mocks/Doubles Stephen Caudill No

  51. Pending Tests Stephen Caudill Yay!

  52. Run tests on Device? Stephen Caudill :sadtrombone:

  53. Nice Xcode Integration Stephen Caudill Success and Failure feedback is

    integrated nicely with Xcode.
  54. Documentation Stephen Caudill Very little... It’s pretty much just the

    readme, but it identifies the most important aspects of the API.
  55. Verdict Stephen Caudill Pretty good (with Expecta)

  56. Kiwi Stephen Caudill http://github.com/allending/Kiwi

  57. Ease of Installation Stephen Caudill B- - Using Kiwi 2,

    which is young - Install documentation is sparse - Cocoapods (pay attention to the output of pod install)
  58. Ubiquitous Language Stephen Caudill Definitely.

  59. Stephen Caudill - similar to Specta - expectations are idiomatic

    Obj-C
  60. Focused Runs Stephen Caudill Still No

  61. Expectations Stephen Caudill Yes! As we just saw...

  62. Mocks/Doubles Stephen Caudill Yes, 1000 times! Yes!

  63. Mocks Protocol Mocks Stubs Oh my! Stephen Caudill

  64. Pending Tests Stephen Caudill Yay!

  65. Run tests on Device? Stephen Caudill Yes!

  66. Documentation Stephen Caudill As long as you don’t need to

    install it... http://kiwi-lib.info Kiwi 2 is new and documentation for it is pretty sparse Luckily most documentation for Kiwi 1 is still valid.
  67. Verdict Stephen Caudill Ya das is gut!

  68. Cedar Stephen Caudill http://github.com/pivotal/cedar

  69. Ease of Installation Stephen Caudill A+ Super easy! Between install

    script for the templates and the templates themselves, it's really painless.
  70. Stephen Caudill

  71. Ubiquitous Language Stephen Caudill Real BDD

  72. Stephen Caudill C++ ?!?! - So as we can see

    syntax is standardizing... - BUT! C++!?!? - Just because you can do something...
  73. Focused Runs Stephen Caudill http://github.com/cppforlife/CedarShortcuts For sure! let’s you do

    things like ctrl+alt+u to run the spec under the cursor
  74. Expectations Stephen Caudill Yepper

  75. Mocks/Doubles Stephen Caudill Yessiree Bob!

  76. Spies Mocks Stubs Stephen Caudill

  77. Pending Tests Stephen Caudill Yes!

  78. Run tests on Device? Stephen Caudill Yes!

  79. Documentation Stephen Caudill In the form of a README in

    it’s defense, it’s a very good readme
  80. Verdict Stephen Caudill It does all of the things.

  81. Acceptance Tests! Stephen Caudill

  82. Calabash-iOS Stephen Caudill http://github.com/calabash Android developers take heed!

  83. Ease of Installation Stephen Caudill A++ Would install again. Installation

    was very straightforward scripts work well and readme is accurate
  84. Ubiquitous Language Stephen Caudill Yes. Sir. - Ubiquitous language in

    the context of acceptance tests - Acts as an interpretation layer for Cucumber
  85. Stephen Caudill You may have seen this before... Feature implementation

    is in Ruby
  86. Focused Runs Stephen Caudill By the power of cucumber

  87. Pending Features Stephen Caudill Yessiree

  88. Run on Device? Stephen Caudill Yes! $ NO_LAUNCH=1 cucumber

  89. Documentation Stephen Caudill Very Good Extensive wiki

  90. Verdict Stephen Caudill Very good. Autostarting the simulator is broken

    - passing `NO_LAUNCH=1 cucumber` will get around it, but you have to manually start/stop the simulator
  91. Frank Stephen Caudill http://testingwithfrank.com

  92. Ease of Installation Stephen Caudill A- If you have an

    OCUnit target, it’ll try to copy your octest socket (using Ruby) and fail as it’s an abnormal file type. Outside of that, the frank CLI and it’s subcommands make it easy to set things up
  93. Ubiquitous Language Stephen Caudill Customer read/write? Check! We’re sitting on

    top of Cucumber again here, so yeah...
  94. Stephen Caudill This probably looks familiar (again) Tangent about web

    steps and the steps frank ships with - they’re difficult to remove
  95. Focused Runs Stephen Caudill Yes! Thank you cucumber.

  96. Pending Features Stephen Caudill Yeah

  97. Run on Device? Stephen Caudill NO! WHY, SCIENCE? WHY??

  98. Documentation Stephen Caudill Mediocre writing custom steps is a PITA,

    Ruby API isn’t documented
  99. Verdict Stephen Caudill Serviceable requiring the objective-c dev to know

    Ruby to implement steps is a big drawback
  100. KIF Stephen Caudill http://github.com/square/KIF Keep It Functional

  101. Ease of Installation Stephen Caudill D KIF is a PITA

    to install. Cocoapods helps, but only if you cobble together the incantations from github issues and selectively ignore the README
  102. Stephen Caudill * only head works currently, due to changes

    in Xcode 4.5 (current version is 4.6) * ensure you specify the target for the pod
  103. Ubiquitous Language Stephen Caudill Well...

  104. Stephen Caudill Can your customer write that?

  105. Stephen Caudill How do they feel about giant walls of

    text for output?
  106. Focused Runs Stephen Caudill No

  107. Pending Features Stephen Caudill No

  108. Run on Device? Stephen Caudill Yes

  109. Documentation Stephen Caudill Poor very fragmented

  110. Verdict Stephen Caudill Meh. Hard to set up Hard to

    learn No use of ubiquitous language But hey, it’s in objective C May be unfair to judge as an “acceptance” framework
  111. Recap Stephen Caudill

  112. Installation Ubiquitous Language Focused Tests Pending Tests Expectations Mocking Runs

    on Device Docs OCUnit GHUnit OCHamcrest Kiwi Cedar Specta/Expecta KIF Calabash Frank A+    N/A   D- C    N/A   C A    N/A   B B       C A+       B- A       B- D     N/A  D A+     N/A  A A-     N/A  C
  113. Installation Ubiquitous Language Focused Tests Pending Tests Expectations Mocking Runs

    on Device Docs OCUnit GHUnit OCHamcrest Kiwi Cedar Specta/Expecta KIF Calabash Frank A+    N/A   D- C    N/A   C A    N/A   B B       C A+       B- A       B- D     N/A  D A+     N/A  A A-     N/A  C Only Use the TDD frameworks if you *have* to
  114. Installation Ubiquitous Language Focused Tests Pending Tests Expectations Mocking Runs

    on Device Docs OCUnit GHUnit OCHamcrest Kiwi Cedar Specta/Expecta KIF Calabash Frank A+    N/A   D- C    N/A   C A    N/A   B B       C A+       B- A       B- D     N/A  D A+     N/A  A A-     N/A  C
  115. Installation Ubiquitous Language Focused Tests Pending Tests Expectations Mocking Runs

    on Device Docs OCUnit GHUnit OCHamcrest Kiwi Cedar Specta/Expecta KIF Calabash Frank A+    N/A   D- C    N/A   C A    N/A   B B       C A+       B- A       B- D     N/A  D A+     N/A  A A-     N/A  C Because C++ is groooooss.
  116. Installation Ubiquitous Language Focused Tests Pending Tests Expectations Mocking Runs

    on Device Docs OCUnit GHUnit OCHamcrest Kiwi Cedar Specta/Expecta KIF Calabash Frank A+    N/A   D- C    N/A   C A    N/A   B B       C A+       B- A       B- D     N/A  D A+     N/A  A A-     N/A  C Only Use the TDD frameworks if you *have* to
  117. Installation Ubiquitous Language Focused Tests Pending Tests Expectations Mocking Runs

    on Device Docs OCUnit GHUnit OCHamcrest Kiwi Cedar Specta/Expecta KIF Calabash Frank A+    N/A   D- C    N/A   C A    N/A   B B       C A+       B- A       B- D     N/A  D A+     N/A  A A-     N/A  C
  118. Installation Ubiquitous Language Focused Tests Pending Tests Expectations Mocking Runs

    on Device Docs OCUnit GHUnit OCHamcrest Kiwi Cedar Specta/Expecta KIF Calabash Frank A+    N/A   D- C    N/A   C A    N/A   B B       C A+       B- A       B- D     N/A  D A+     N/A  A A-     N/A  C Automates with actual touch events
  119. Honorable Mentions Stephen Caudill

  120. UIAutomation Wrappers Stephen Caudill

  121. Zucchini http://www.zucchiniframework.org/ Stephen Caudill Cucumber-like stories Maps to Coffeescript definitions

    Looks promising, avoided because it’s javascript
  122. Bwoken http://bendyworks.github.com/bwoken/ Stephen Caudill Very nice tooling built in Ruby

    Implement in Coffeescript or Javascript
  123. Other Honorable Mentions Stephen Caudill

  124. UISpec/UISpecRunner http://code.google.com/p/uispec/ Stephen Caudill Idiomatic Objective-C Code several other frameworks

    were originally based on (e.g. KIF)
  125. Simon http://drekka.github.com/simon/ Stephen Caudill Looks promising Feature files Native Objective-C

    mappings Just couldn’t get it to work
  126. Go forth and Specify! Stephen Caudill

  127. Stephen Caudill @voxdolo @voxdolo http://hashrocket.com So you know where to

    deliver your hatemail