Tests as Documentation (IPC14)

With automatic software tests, we ensure that our application (or certain parts of it) behaves like intended. Even if new requirements are added, our tests tell us if our old requirements are still fulfilled. A well maintained set of tests may also serve as a catalog of requirements, but usually only the developers are able to read those tests. BDD tools like Behat might change this. Let's have a look at how human-readable tests could be used as a communication tool for developers and stakeholders.

Transcript

1. [AMT] Alexander M. Turek Alexander M. Turek Tests as Documentation International php Conference — Oct 28th, 2014 1
2. [AMT] Alexander M. Turek about:me freelance software developer. builds web applications with php. likes the Symfony framework. lives in Munich Berlin, Germany. 2
3. [AMT] Alexander M. Turek Software Testing 3 Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test. http://en.wikipedia.org/wiki/Software_testing software
 meets
 requirements
4. [AMT] Alexander M. Turek Manual Testing • Just click through your application and verify that it (still) works as expected. • great for explorative development
 (prototypes, spike solutions) • time consuming • hard to repeat 4
5. [AMT] Alexander M. Turek Test Automation 5
6. [AMT] Alexander M. Turek Automated UI Tests • record a click path through your application • replay it in different browsers • awesome
 if (!$tester.isDeveloper()) • great for legacy applications 6
7. [AMT] Alexander M. Turek Automated UI Tests • slow execution • fragile • no help for debugging • complex setup 7
8. [AMT] Alexander M. Turek Test Pyramid • system tests
 (e.g. through the UI) • integration tests
 (headless) • unit tests
 (mocked dependencies) 8 http://martinfowler.com/bliki/TestPyramid.html
9. [AMT] Alexander M. Turek Unit Tests • Isolate small parts of your software (units) and verify their correctness. • Idea: Your software can only work as expected if each of those units works as expected. • Unit tests should execute fast to make sure that they are executed often. 9
10. [AMT] Alexander M. Turek Unit Tests • Interaction between units is not tested. • Tests do not correspond to requirements. 10
11. [AMT] Alexander M. Turek Integration Tests • Multiple components tested together • More realistic results • Errors are harder to locate • Slower execution 11
12. [AMT] Alexander M. Turek PHPUnit • originally a unit test framework • mock library • assertions • used as a universal
 test runner 12
13. [AMT] Alexander M. Turek Continuous Integration • Tests are run regularly • No failing tests • Early notification • Tests are kept up to date 13
14. [AMT] Alexander M. Turek Documentation? 14
15. [AMT] Alexander M. Turek Unit Tests A unit test documents how the code under test is meant to be used. 15 $cart = new Cart($this->connection);
 $cart->add('13374711');
 $cart->add('13370815', 5);
 
 $this->assertEquals(
 2,
 $cart->getPositionsCount()
 );
 $this->assertEquals(
 6,
 $cart->getItemsCount()
 );
 $this->assertEquals(
 25.94,
 $cart->getTotalPrice()
 );
16. [AMT] Alexander M. Turek your test is a documentation 16 user input database
 content calculated
 values
17. [AMT] Alexander M. Turek Specification By Example • natural language is ambiguous • much room for misunderstandings • examples help us understand • find edge cases 17
18. [AMT] Alexander M. Turek Given — When — Then • The given part describes the state of the world before. • The when section is the behavior that you're specifying. • The then section describes the changes you expect. 18 http://martinfowler.com/bliki/GivenWhenThen.html user input database
 content calculated
 values
19. [AMT] Alexander M. Turek Gherkin 19 Given my cart is empty
 And there is a product with ID 13374711 and a price of 0.99
 And there is a product with ID 13370815 and a price of 4.99
 
 When I add 1 item of product 13374711 to my cart
 And I add 5 items of product 13370815 to my cart
 
 Then I should have 2 positions in my cart
 And I should have 6 items in my cart
 And the total price should be 25.94 Feature: Shopping cart
 
 Scenario: I want the correct total price to be calculated.
20. [AMT] Alexander M. Turek A "Test"? Remember the Gherkin code we just saw? • Did it describe a unit test? • Was it an integration test? • Or maybe a UI test? 20
21. [AMT] Alexander M. Turek Behavior Driven Development 21
22. [AMT] Alexander M. Turek Behat • BDD framework for php • Implementation of the Gherkin language 22
23. [AMT] Alexander M. Turek Feature Files 23 Feature: Shopping cart
 
 Scenario: When I place items into my cart, I want the correct total price to be calculated.
 
 Given my cart is empty
 And there is a product with ID 13374711 and a price of 0.99
 And there is a product with ID 13370815 and a price of 4.99
 
 When I add 1 item of product 13374711 to my cart
 And I add 5 items of product 13370815 to my cart
 
 Then I should have 2 positions in my cart
 And I should have 6 items in my cart
 And the total price should be 25.94
 
 Scenario: When I add a product that is already in my cart, I want those positions to be merged.
 
 Given my cart is empty
 And there is a product with ID 13374711 and a price of 0.99
 
 When I add 1 item of product 13374711 to my cart
 And I add 2 item of product 13374711 to my cart
 
 Then I should have 3 items in my cart
 And I should have 1 position in my cart Scenario Step
24. [AMT] Alexander M. Turek Step • Starts with Given, When, Then, And or But • Behat tries to map the step to a piece of code and execute it, one after the other. • Step could not be mapped => scenario is skipped. • Step throws an exception => scenario fails 24
25. [AMT] Alexander M. Turek Feature Context This is where the magic happens 25 public function __construct()
 {
 $this->connection = new PDO('sqlite::memory:');
 $this->connection->query('CREATE TABLE products
 }
 
 /**
 * @Given /^my cart is empty$/
 */
 public function myCartIsEmpty()
 {
 $this->cart = new Cart($this->connection);
 }
 
 /**
 * @Given /^there is a product with ID (\d+) and a
 */
 public function thereIsAProductWithIdAndAPriceOf($p
 {
 $stmt = $this->connection->prepare('INSERT INTO
 $stmt->execute([$productId, $price]);
 }
 
 /**
 * @When /^I add (\d+) item[s]? of product (\d+) to
 */
 public function iAddItemsOfProductToMyCart($quantit
 {
 Assert::assertNotNull($this->cart);
 $this->cart->add($productId, $quantity);
 }
26. [AMT] Alexander M. Turek Feature Context 26 class IntegrationFeatureContext extends BehatContext
 {
 /**
 * @var PDO
 */
 private $connection;
 
 /**
 * @var Cart
 */
 private $cart;
 
 public function __construct()
 {
 $this->connection = new PDO('sqlite::memory:');
 $this->connection->query(
 'CREATE TABLE products (id INT PRIMARY KEY, price DECIMAL(10, 2))'
 );
 } run before each scenario
27. [AMT] Alexander M. Turek A Simple Step 27 /**
 * @Given /^my cart is empty$/
 */
 public function myCartIsEmpty()
 {
 $this->cart = new Cart($this->connection);
 } regular expression Given my cart is empty
28. [AMT] Alexander M. Turek Parameters 28 /**
 * @Given /^there is a product with ID (\d+) and a price of (\d+\.\d+)$/
 */
 public function thereIsAProductWithIdAndAPriceOf($productId, $price)
 {
 $stmt = $this->connection->prepare(
 'INSERT INTO products (id, price) VALUES (?, ?);'
 );
 $stmt->execute([$productId, $price]);
 } capturing subpatterns And there is a product with ID 13374711 and a price of 0.99
 And there is a product with ID 13370815 and a price of 4.99
29. [AMT] Alexander M. Turek Assertions 29 /**
 * @When /^I add (\d+) item[s]? of product (\d+) to my cart$/
 */
 public function iAddItemsOfProductToMyCart($quantity, $productId)
 {
 Assert::assertNotNull($this->cart);
 $this->cart->add($productId, $quantity);
 }
 use PHPUnit_Framework_Assert as Assert; When I add 1 item of product 13374711 to my cart
 And I add 5 items of product 13370815 to my cart
30. [AMT] Alexander M. Turek Testing Expectations 30 /**
 * @Then /^the total price should be (\d+\.\d+)$/
 */
 public function theTotalPriceShouldBe($price)
 {
 Assert::assertNotNull($this->cart);
 Assert::assertEquals(
 $price,
 $this->cart->getTotalPrice()
 );
 }
31. [AMT] Alexander M. Turek Let's Run The Tests 31
32. [AMT] Alexander M. Turek Once More 32
33. [AMT] Alexander M. Turek The Feature File, Revisited 33 Feature: Shopping cart
 
 Scenario: When I place items into my cart, I want the correct total price to be calculated.
 
 Given my cart is empty
 And there is a product with ID 13374711 and a price of 0.99
 And there is a product with ID 13370815 and a price of 4.99
 
 When I add 1 item of product 13374711 to my cart
 And I add 5 items of product 13370815 to my cart
 
 Then I should have 2 positions in my cart
 And I should have 6 items in my cart
 And the total price should be 25.94
 
 Scenario: When I add a product that is already in my cart, I want those positions to be merged.
 
 Given my cart is empty
 And there is a product with ID 13374711 and a price of 0.99
 
 When I add 1 item of product 13374711 to my cart
 And I add 2 item of product 13374711 to my cart
 
 Then I should have 3 items in my cart
 And I should have 1 position in my cart
34. [AMT] Alexander M. Turek Mink abstraction layer for
 web acceptance testing frameworks provides an extension for behat 34
35. [AMT] Alexander M. Turek UI Testing With Behat/Mink 35 /**
 * @When /^I add (\d+) item[s]? of product (\d+) to my cart$/
 */
 public function iAddItemsOfProductToMyCart($quantity, $productId)
 {
 $this->getSession()->visit('http://myshop/product/' . $productId);
 $this->getSession()->getPage()->fillField('Quantity', $quantity);
 $this->getSession()->getPage()->pressButton('Add to cart');
 }
 
 /**
 * @Then /^the total price should be (\d+\.\d+)$/
 */
 public function theTotalPriceShouldBe($price)
 {
 $this->getSession()->visit('http://myshop/cart/');
 $element = $this->getSession()->getPage()->find('css', '#totalPrice');
 
 Assert::assertNotNull($element);
 Assert::assertEquals($price, $element->getText());
 }
36. [AMT] Alexander M. Turek Behat in PhpStorm 36 http://blog.jetbrains.com/phpstorm/2014/07/using-behat-in-phpstorm/
37. [AMT] Alexander M. Turek Behat in PhpStorm 37 http://blog.jetbrains.com/phpstorm/2014/07/using-behat-in-phpstorm/
38. [AMT] Alexander M. Turek Behat in PhpStorm 38 http://blog.jetbrains.com/phpstorm/2014/07/using-behat-in-phpstorm/
39. [AMT] Alexander M. Turek Behat in Jenkins • Use Jenkins to run your behat tests regularly. • Behat exports test results in a JUnit-compatible XML format. • Jenkins is able to parse that file and displays failing scenarios with stack traces. 39
40. [AMT] Alexander M. Turek Questions? 40
41. [AMT] Alexander M. Turek Literature • Martin Fowler: Test Pyramid
 http://martinfowler.com/bliki/TestPyramid.html • Martin Fowler: GivenWhenThen
 http://martinfowler.com/bliki/GivenWhenThen.html • Dan North: Introducing BDD
 http://dannorth.net/introducing-bdd/ 41
42. [AMT] Alexander M. Turek Thank You spam me: me@derrabus.de follow me: @derrabus hire me: https://derrabus.de 42