$30 off During Our Annual Pro Sale. View Details »

Essential Testing Know-How for Testing on PHP 8+

Essential Testing Know-How for Testing on PHP 8+

Presented on March 12th 2021 for employees of Automattic (closed learnup).

Juliette Reinders Folmer

March 12, 2021
Tweet

More Decks by Juliette Reinders Folmer

Other Decks in Programming

Transcript

  1. Essential Testing Know-How for testing on PHP 8+ Juliette Reinders

    Folmer Tweet about it: @jrf_nl
  2. test Test TEST

  3. None
  4. Assessing & Improving Test Quality marktimemedia

  5. Test Types Unit Tests Integration Tests E2E Tests Acceptance Tests

  6. Test Requirements Have tests Strict assertions High code coverage, strictly

    measured Happy & unhappy path
  7. Test Requirements Have tests Strict assertions High code coverage, strictly

    measured Happy & unhappy path
  8. Have the basic setup in place [1] In composer.json: {

    "require-dev" : { "phpunit/phpunit": "^8.0 || ^9.0" }, "autoload": { "classmap": ["src/"] }, "autoload-dev": { "classmap": ["tests/"] } }
  9. Have the basic setup in place [2] In phpunit.xml.dist: <?xml

    version="1.0" encoding="UTF-8"?> <phpunit bootstrap="./vendor/autoload.php" colors="true" > <testsuites> <testsuite name="Foo"> <directory suffix="Test.php"> ./tests/ </directory> </testsuite> </testsuites> </phpunit> --generate-configuration
  10. Start Small (but start somewhere) <?php namespace PHPUnit_Demo; class Foo

    { static function stripQuotes($string) { return preg_replace( '`^([\'"])(.*)\1$`Ds', '$2', $string ); } }
  11. Start Small (but start somewhere) <?php namespace PHPUnit_Demo\Tests; use PHPUnit\Framework\TestCase;

    use PHPUnit_Demo\Foo; class FooTest extends TestCase { public function testStripQuotes() { $result = Foo::stripQuotes('"text"'); $this->assertEquals('text', $result); } }
  12. Don't Use Your Own Code To Create Test Data

  13. Test Requirements Have tests Strict assertions High code coverage, strictly

    measured Happy & unhappy path
  14. assertEquals() == assertSame() === Most Common Issue

  15. Available Assertions

  16. Use the Right Assertion <?php namespace PHPUnit_Demo\Tests; use PHPUnit\Framework\TestCase; use

    PHPUnit_Demo\Foo; class FooTest extends TestCase { public function testStripQuotes() { $result = Foo::stripQuotes('"text"'); $this->assertEquals('text', $result); $this->assertSame('text', $result); } }
  17. Test Requirements Have tests Strict assertions High code coverage, strictly

    measured Happy & unhappy path
  18. Don't Trust Code Coverage But do examine it

  19. <?php namespace PHPUnit_Demo; class Foo { static function stripQuotes($string) {

    return preg_replace( '`^([\'"])(.*)\1$`Ds', '$2', $string ); } }
  20. Enabling Code Coverage [1] <?xml version="1.0" encoding="UTF-8"?> <phpunit beStrictAboutCoversAnnotation="true" forceCoversAnnotation="true"

    > ... <filter> <whitelist addUncoveredFilesFromWhitelist="true"> <directory suffix=".php">src</directory> </whitelist> </filter> <logging> <log type="coverage-clover" target="build/logs/clover.xml"/> </logging> </phpunit>
  21. Enabling Code Coverage [2] <?php class FooTest extends TestCase {

    /** * Test Foo::stripQuotes(). * * @dataProvider dataStripQuotes * * @covers \PHPUnit_Demo\Foo::stripQuotes */ public function testStripQuotes($in, $out) { $result = Foo::stripQuotes($in); $this->assertSame($out, $result); }
  22. Simplify { "scripts" : { "test": [ "vendor/bin/phpunit --no-coverage" ],

    "coverage": [ "vendor/bin/phpunit" ], "coverage-local": [ "vendor/bin/phpunit --coverage-html ./build/coverage-html" ] } }
  23. None
  24. PHPUnit >= 9.3 ? phpunit --migrate-configuration

  25. Test Requirements Have tests Strict assertions High code coverage, strictly

    measured Happy & unhappy path
  26. Tests Document Expectations

  27. <?php public function testStripQuotes() { $result = Foo::stripQuotes('"some text"'); $this->assertSame('some

    text', $result); $result = Foo::stripQuotes("some 'text'"); $this->assertSame("some 'text'", $result); $result = Foo::stripQuotes(false); $this->assertSame('', $result); } Don't Just Test the Happy Path
  28. Allow for Testing the Unhappy Path ▪ strict_types ▪ Parameter

    type declarations mensatic
  29. Use Data Providers

  30. Test Method Data Provider /** * Test Foo::stripQuotes(). * *

    @dataProvider dataStripQuotes * * @param mixed $in Function input. * @param string $out Expected output. * * @return void */ public function testStripQ($in, $out) { $result = Foo::stripQuotes($in); $this->assertSame($out, $result); } /** * Data provider. * * @return array[] */ public function dataStripQuotes() { return [ ['"some text"', 'some text'], ["some 'text'", "some 'text'"], [false, ''], ]; }
  31. Without Data Provider With Data Provider

  32. Your Tests Are Limited By Your Own Imagination

  33. Test Requirements Have tests Strict assertions High code coverage, strictly

    measured Happy & unhappy path
  34. Test Your Tests ▪ Infection https://infection.github.io/ https://youtu.be/ADKyTlaH6e4 ▪ PHPUnitCompatibility (upcoming)

    ▪ PHPUnitQA (upcoming) lisaleo
  35. Test Quality of WordPress Plugins & Themes marktimemedia

  36. WordPress Test Suite Have tests Strict assertions High code coverage,

    strictly measured Happy & unhappy path
  37. WordPress Test Suite Have tests Strict assertions High code coverage,

    strictly measured Happy & unhappy path
  38. None
  39. None
  40. Plugins and Themes Happy & unhappy path High code coverage,

    strictly measured Strict assertions Have tests
  41. Running Your Tests on PHP 8.0 marktimemedia

  42. PHPUnit Support v Compatible with: 10 PHP >= 7.4 (expected

    April 2021) 9 PHP >= 7.3 8 PHP >= 7.2 7 PHP 7.1, 7.2, 7.3 [EOL] 6 PHP 7.0, 7.1, 7.2 [EOL] 5 PHP 5.6, 7.0, 7.1 [EOL] Seemann
  43. PHPUnit vs PHP PHPUnit / PHP 5.6 7.0 7.1 7.2

    7.3 7.4 8.0 5.x 6.x 7.x 8.x 9.x 9.3+ 10.x void 8.5+
  44. Constraints and Roadblocks » [OS] Supporting PHP 5.6 + 7.0

    still required » Plugin tests often extend WP integration test suite » PHPUnit < 8.5/9.3 not compatible with PHP 8.0 - Mocking - @runInSeparateProcess » PHPUnit 7 Phar will not run on PHP 8.0 » Committed composer.lock file
  45. Solution WP Core: "autoload-dev": { "files": [ "tests/includes/MockObject/Builder/NamespaceMatch.php", "tests/includes/MockObject/Builder/ParametersMatch.php", "tests/includes/MockObject/InvocationMocker.php",

    "tests/includes/MockObject/MockMethod.php" ], "exclude-from-classmap": [ "vendor/phpunit/…/MockObject/Builder/NamespaceMatch.php", "vendor/phpunit/…/MockObject/Builder/ParametersMatch.php", "vendor/phpunit/…/MockObject/InvocationMocker.php", "vendor/phpunit/…/MockObject/MockMethod.php" ] }, ▪ Composer, not phar ▪ Lock at PHPUnit 7.5 ▪ Downgrade to PHPUnit 5 in CI for PHP 5.6, 7.0 ▪ Use MockObject classes from PHPUnit 9.3 ▪ Separate process => separate group
  46. Constraints For Public/OS projects Support multiple WP versions Support multiple

    PHP versions No control over run environment
  47. Mock-based Tests PHPUnit 5.x composer.lock Platform - php 5.6 PHPUnit

    5.x – 9.x CI: composer update … --ignore-platform-reqs Needs PHPUnit cross-version compatibility layer
  48. WP Integration Tests PHPUnit 5.x composer.lock Platform - php 5.6

    PHPUnit 5.x – 7.x CI: composer update … --ignore-platform-reqs Need PHPUnit cross-version compatibility layer Need custom autoload for MockObject
  49. New Tooling PHPUnit Polyfills WP Test Utils Sponsored by: Alternative:

    Symfony Bridge * Supports PHPUnit 4.8 – 9.x, PHP 5.5 – 8.0
  50. Implementing PHPUnit Polyfills In composer.json: { "require-dev" : { "phpunit/phpunit":

    "^5.0 || ^7.0", "yoast/phpunit-polyfills": "^0.2.0" } }
  51. Implementing PHPUnit Polyfills What you get: ▪ Polyfills via traits

    for all new assertions and expectations in PHPUnit ▪ Helper to work round removal of assertAttribute*() methods ▪ An cross-version compatible abstract base TestCase (to get round void) which includes all polyfills ▪ A cross-version compatible TestListenerImplementation
  52. Implementing PHPUnit Polyfills <?php use PHPUnit\Framework\TestCase; use Yoast\PHPUnitPolyfills\TestCases\TestCase; class MyTest

    extends TestCase { protected function setUp() { protected function set_up() { parent::setUp(); parent::set_up(); // Set up function mocks which need to be // available for all tests in this class. } }
  53. Implementing WP Test Utils In composer.json: { "require-dev" : {

    "phpunit/phpunit": "^5.0 || ^7.0", "brain/monkey": "^2.6.0", "yoast/wp-test-utils": "^0.2.0" } }
  54. Implementing WP Test Utils for BrainMonkey tests What you get:

    ▪ PHPUnit Polyfills ▪ BrainMonkey and Mockery set up and teardown ▪ Mockery tests not marked as risky ▪ Choice between BrainMonkey or opaque escape/translation stubs ▪ expectOutputContains() helper ▪ [YoastTestCase] Additional function stubs
  55. Implementing WP Test Utils for BrainMonkey Based Tests <?php use

    PHPUnit\Framework\TestCase; use Yoast\WPTestUtils\BrainMonkey\[Yoast]TestCase; class MyTest extends TestCase { protected function setUp() { protected function set_up() { parent::setUp(); parent::set_up(); // Set up function mocks which need to be // available for all tests in this class. } }
  56. Implementing WP Test Utils for WP Integration tests What you

    get: ▪ PHPUnit Polyfills ▪ Access to all WP native test utilities ▪ expectOutputContains() helper ▪ Bootstrap utilities
  57. Implementing WP Test Utils for WP Integration Based Tests <?php

    use Yoast\WPTestUtils\WPIntegration; if ( getenv( 'WP_PLUGIN_DIR' ) !== false ) { define( 'WP_PLUGIN_DIR', getenv( 'WP_PLUGIN_DIR' ) ); } $GLOBALS['wp_tests_options'] = [ 'active_plugins' => [ 'plugin-name/main-file.php' ], ]; require_once dirname( __DIR__ ) . '/vendor/yoast/wp-test-utils/src/ WPIntegration/bootstrap-functions.php'; WPIntegration\bootstrap_it(); if ( ! defined( 'WP_PLUGIN_DIR' ) || file_exists( WP_PLUGIN_DIR . '/plugin- name/main-file.php' ) === false ) { echo 'ERROR: …', PHP_EOL; exit( 1 ); }
  58. Implementing WP Test Utils for WP Integration Based Tests <?php

    use WP_UnitTestCase; use Yoast\WPTestUtils\WPIntegration\TestCase; class MyTest extends WP_UnitTestCase { class MyTest extends TestCase { protected function setUp() { parent::setUp(); // Set up function mocks which need to be // available for all tests in this class. } }
  59. Further Reading ▪ The Grumpy Programmer's Guide to Testing PHP

    Applications https://grumpy-learning.com/ ▪ Your Mocks Won't Save You! https://24daysindecember.net/2020/12/08/your-mocks-wont-save-you/ ▪ My Top 10 PHPUnit Tips & Tricks https://speakerdeck.com/jrf/my-top-10-phpunit-tips-and-tricks-e6ea54ce- 2515-4ea9-aacf-9bf7ab3b3141 ▪ PHPUnit Documentation https://phpunit.readthedocs.io/ ▪ Path Coverage in PHPUnit https://doug.codes/php-code-coverage
  60. Tooling ▪ PHPUnit Polyfills https://packagist.org/packages/yoast/phpunit-polyfills ▪ WP Test Utils https://packagist.org/packages/yoast/wp-test-utils

  61. Thanks! @jrf_nl @jrfnl Any questions ? Slides: https://speakerdeck.com/jrf