Save 37% off PRO during our Black Friday Sale! »

Essential Testing Know-How for Testing on PHP 8+

Essential Testing Know-How for Testing on PHP 8+

Presented on March 12th 2021 for employees of Automattic (closed learnup).

2776198ea9584b6c0d4b494293b8d635?s=128

Juliette Reinders Folmer

March 12, 2021
Tweet

Transcript

  1. Essential Testing Know-How for testing on PHP 8+ Juliette Reinders

    Folmer Tweet about it: @jrf_nl
  2. test Test TEST

  3. None
  4. Assessing & Improving Test Quality marktimemedia

  5. Test Types Unit Tests Integration Tests E2E Tests Acceptance Tests

  6. Test Requirements Have tests Strict assertions High code coverage, strictly

    measured Happy & unhappy path
  7. Test Requirements Have tests Strict assertions High code coverage, strictly

    measured Happy & unhappy path
  8. Have the basic setup in place [1] In composer.json: {

    "require-dev" : { "phpunit/phpunit": "^8.0 || ^9.0" }, "autoload": { "classmap": ["src/"] }, "autoload-dev": { "classmap": ["tests/"] } }
  9. Have the basic setup in place [2] In phpunit.xml.dist: <?xml

    version="1.0" encoding="UTF-8"?> <phpunit bootstrap="./vendor/autoload.php" colors="true" > <testsuites> <testsuite name="Foo"> <directory suffix="Test.php"> ./tests/ </directory> </testsuite> </testsuites> </phpunit> --generate-configuration
  10. Start Small (but start somewhere) <?php namespace PHPUnit_Demo; class Foo

    { static function stripQuotes($string) { return preg_replace( '`^([\'"])(.*)\1$`Ds', '$2', $string ); } }
  11. Start Small (but start somewhere) <?php namespace PHPUnit_Demo\Tests; use PHPUnit\Framework\TestCase;

    use PHPUnit_Demo\Foo; class FooTest extends TestCase { public function testStripQuotes() { $result = Foo::stripQuotes('"text"'); $this->assertEquals('text', $result); } }
  12. Don't Use Your Own Code To Create Test Data

  13. Test Requirements Have tests Strict assertions High code coverage, strictly

    measured Happy & unhappy path
  14. assertEquals() == assertSame() === Most Common Issue

  15. Available Assertions

  16. Use the Right Assertion <?php namespace PHPUnit_Demo\Tests; use PHPUnit\Framework\TestCase; use

    PHPUnit_Demo\Foo; class FooTest extends TestCase { public function testStripQuotes() { $result = Foo::stripQuotes('"text"'); $this->assertEquals('text', $result); $this->assertSame('text', $result); } }
  17. Test Requirements Have tests Strict assertions High code coverage, strictly

    measured Happy & unhappy path
  18. Don't Trust Code Coverage But do examine it

  19. <?php namespace PHPUnit_Demo; class Foo { static function stripQuotes($string) {

    return preg_replace( '`^([\'"])(.*)\1$`Ds', '$2', $string ); } }
  20. Enabling Code Coverage [1] <?xml version="1.0" encoding="UTF-8"?> <phpunit beStrictAboutCoversAnnotation="true" forceCoversAnnotation="true"

    > ... <filter> <whitelist addUncoveredFilesFromWhitelist="true"> <directory suffix=".php">src</directory> </whitelist> </filter> <logging> <log type="coverage-clover" target="build/logs/clover.xml"/> </logging> </phpunit>
  21. Enabling Code Coverage [2] <?php class FooTest extends TestCase {

    /** * Test Foo::stripQuotes(). * * @dataProvider dataStripQuotes * * @covers \PHPUnit_Demo\Foo::stripQuotes */ public function testStripQuotes($in, $out) { $result = Foo::stripQuotes($in); $this->assertSame($out, $result); }
  22. Simplify { "scripts" : { "test": [ "vendor/bin/phpunit --no-coverage" ],

    "coverage": [ "vendor/bin/phpunit" ], "coverage-local": [ "vendor/bin/phpunit --coverage-html ./build/coverage-html" ] } }
  23. None
  24. PHPUnit >= 9.3 ? phpunit --migrate-configuration

  25. Test Requirements Have tests Strict assertions High code coverage, strictly

    measured Happy & unhappy path
  26. Tests Document Expectations

  27. <?php public function testStripQuotes() { $result = Foo::stripQuotes('"some text"'); $this->assertSame('some

    text', $result); $result = Foo::stripQuotes("some 'text'"); $this->assertSame("some 'text'", $result); $result = Foo::stripQuotes(false); $this->assertSame('', $result); } Don't Just Test the Happy Path
  28. Allow for Testing the Unhappy Path ▪ strict_types ▪ Parameter

    type declarations mensatic
  29. Use Data Providers

  30. Test Method Data Provider /** * Test Foo::stripQuotes(). * *

    @dataProvider dataStripQuotes * * @param mixed $in Function input. * @param string $out Expected output. * * @return void */ public function testStripQ($in, $out) { $result = Foo::stripQuotes($in); $this->assertSame($out, $result); } /** * Data provider. * * @return array[] */ public function dataStripQuotes() { return [ ['"some text"', 'some text'], ["some 'text'", "some 'text'"], [false, ''], ]; }
  31. Without Data Provider With Data Provider

  32. Your Tests Are Limited By Your Own Imagination

  33. Test Requirements Have tests Strict assertions High code coverage, strictly

    measured Happy & unhappy path
  34. Test Your Tests ▪ Infection https://infection.github.io/ https://youtu.be/ADKyTlaH6e4 ▪ PHPUnitCompatibility (upcoming)

    ▪ PHPUnitQA (upcoming) lisaleo
  35. Test Quality of WordPress Plugins & Themes marktimemedia

  36. WordPress Test Suite Have tests Strict assertions High code coverage,

    strictly measured Happy & unhappy path
  37. WordPress Test Suite Have tests Strict assertions High code coverage,

    strictly measured Happy & unhappy path
  38. None
  39. None
  40. Plugins and Themes Happy & unhappy path High code coverage,

    strictly measured Strict assertions Have tests
  41. Running Your Tests on PHP 8.0 marktimemedia

  42. PHPUnit Support v Compatible with: 10 PHP >= 7.4 (expected

    April 2021) 9 PHP >= 7.3 8 PHP >= 7.2 7 PHP 7.1, 7.2, 7.3 [EOL] 6 PHP 7.0, 7.1, 7.2 [EOL] 5 PHP 5.6, 7.0, 7.1 [EOL] Seemann
  43. PHPUnit vs PHP PHPUnit / PHP 5.6 7.0 7.1 7.2

    7.3 7.4 8.0 5.x 6.x 7.x 8.x 9.x 9.3+ 10.x void 8.5+
  44. Constraints and Roadblocks » [OS] Supporting PHP 5.6 + 7.0

    still required » Plugin tests often extend WP integration test suite » PHPUnit < 8.5/9.3 not compatible with PHP 8.0 - Mocking - @runInSeparateProcess » PHPUnit 7 Phar will not run on PHP 8.0 » Committed composer.lock file
  45. Solution WP Core: "autoload-dev": { "files": [ "tests/includes/MockObject/Builder/NamespaceMatch.php", "tests/includes/MockObject/Builder/ParametersMatch.php", "tests/includes/MockObject/InvocationMocker.php",

    "tests/includes/MockObject/MockMethod.php" ], "exclude-from-classmap": [ "vendor/phpunit/…/MockObject/Builder/NamespaceMatch.php", "vendor/phpunit/…/MockObject/Builder/ParametersMatch.php", "vendor/phpunit/…/MockObject/InvocationMocker.php", "vendor/phpunit/…/MockObject/MockMethod.php" ] }, ▪ Composer, not phar ▪ Lock at PHPUnit 7.5 ▪ Downgrade to PHPUnit 5 in CI for PHP 5.6, 7.0 ▪ Use MockObject classes from PHPUnit 9.3 ▪ Separate process => separate group
  46. Constraints For Public/OS projects Support multiple WP versions Support multiple

    PHP versions No control over run environment
  47. Mock-based Tests PHPUnit 5.x composer.lock Platform - php 5.6 PHPUnit

    5.x – 9.x CI: composer update … --ignore-platform-reqs Needs PHPUnit cross-version compatibility layer
  48. WP Integration Tests PHPUnit 5.x composer.lock Platform - php 5.6

    PHPUnit 5.x – 7.x CI: composer update … --ignore-platform-reqs Need PHPUnit cross-version compatibility layer Need custom autoload for MockObject
  49. New Tooling PHPUnit Polyfills WP Test Utils Sponsored by: Alternative:

    Symfony Bridge * Supports PHPUnit 4.8 – 9.x, PHP 5.5 – 8.0
  50. Implementing PHPUnit Polyfills In composer.json: { "require-dev" : { "phpunit/phpunit":

    "^5.0 || ^7.0", "yoast/phpunit-polyfills": "^0.2.0" } }
  51. Implementing PHPUnit Polyfills What you get: ▪ Polyfills via traits

    for all new assertions and expectations in PHPUnit ▪ Helper to work round removal of assertAttribute*() methods ▪ An cross-version compatible abstract base TestCase (to get round void) which includes all polyfills ▪ A cross-version compatible TestListenerImplementation
  52. Implementing PHPUnit Polyfills <?php use PHPUnit\Framework\TestCase; use Yoast\PHPUnitPolyfills\TestCases\TestCase; class MyTest

    extends TestCase { protected function setUp() { protected function set_up() { parent::setUp(); parent::set_up(); // Set up function mocks which need to be // available for all tests in this class. } }
  53. Implementing WP Test Utils In composer.json: { "require-dev" : {

    "phpunit/phpunit": "^5.0 || ^7.0", "brain/monkey": "^2.6.0", "yoast/wp-test-utils": "^0.2.0" } }
  54. Implementing WP Test Utils for BrainMonkey tests What you get:

    ▪ PHPUnit Polyfills ▪ BrainMonkey and Mockery set up and teardown ▪ Mockery tests not marked as risky ▪ Choice between BrainMonkey or opaque escape/translation stubs ▪ expectOutputContains() helper ▪ [YoastTestCase] Additional function stubs
  55. Implementing WP Test Utils for BrainMonkey Based Tests <?php use

    PHPUnit\Framework\TestCase; use Yoast\WPTestUtils\BrainMonkey\[Yoast]TestCase; class MyTest extends TestCase { protected function setUp() { protected function set_up() { parent::setUp(); parent::set_up(); // Set up function mocks which need to be // available for all tests in this class. } }
  56. Implementing WP Test Utils for WP Integration tests What you

    get: ▪ PHPUnit Polyfills ▪ Access to all WP native test utilities ▪ expectOutputContains() helper ▪ Bootstrap utilities
  57. Implementing WP Test Utils for WP Integration Based Tests <?php

    use Yoast\WPTestUtils\WPIntegration; if ( getenv( 'WP_PLUGIN_DIR' ) !== false ) { define( 'WP_PLUGIN_DIR', getenv( 'WP_PLUGIN_DIR' ) ); } $GLOBALS['wp_tests_options'] = [ 'active_plugins' => [ 'plugin-name/main-file.php' ], ]; require_once dirname( __DIR__ ) . '/vendor/yoast/wp-test-utils/src/ WPIntegration/bootstrap-functions.php'; WPIntegration\bootstrap_it(); if ( ! defined( 'WP_PLUGIN_DIR' ) || file_exists( WP_PLUGIN_DIR . '/plugin- name/main-file.php' ) === false ) { echo 'ERROR: …', PHP_EOL; exit( 1 ); }
  58. Implementing WP Test Utils for WP Integration Based Tests <?php

    use WP_UnitTestCase; use Yoast\WPTestUtils\WPIntegration\TestCase; class MyTest extends WP_UnitTestCase { class MyTest extends TestCase { protected function setUp() { parent::setUp(); // Set up function mocks which need to be // available for all tests in this class. } }
  59. Further Reading ▪ The Grumpy Programmer's Guide to Testing PHP

    Applications https://grumpy-learning.com/ ▪ Your Mocks Won't Save You! https://24daysindecember.net/2020/12/08/your-mocks-wont-save-you/ ▪ My Top 10 PHPUnit Tips & Tricks https://speakerdeck.com/jrf/my-top-10-phpunit-tips-and-tricks-e6ea54ce- 2515-4ea9-aacf-9bf7ab3b3141 ▪ PHPUnit Documentation https://phpunit.readthedocs.io/ ▪ Path Coverage in PHPUnit https://doug.codes/php-code-coverage
  60. Tooling ▪ PHPUnit Polyfills https://packagist.org/packages/yoast/phpunit-polyfills ▪ WP Test Utils https://packagist.org/packages/yoast/wp-test-utils

  61. Thanks! @jrf_nl @jrfnl Any questions ? Slides: https://speakerdeck.com/jrf