Essential Testing Know-How for Testing on PHP 8+

Presented on March 12th 2021 for employees of Automattic (closed learnup).

Transcript

1. Essential Testing Know-How for testing on PHP 8+ Juliette Reinders Folmer Tweet about it: @jrf_nl
2. test Test TEST
4. Assessing & Improving Test Quality marktimemedia
5. Test Types Unit Tests Integration Tests E2E Tests Acceptance Tests
6. Test Requirements Have tests Strict assertions High code coverage, strictly measured Happy & unhappy path
7. Test Requirements Have tests Strict assertions High code coverage, strictly measured Happy & unhappy path
8. Have the basic setup in place [1] In composer.json: { "require-dev" : { "phpunit/phpunit": "^8.0 || ^9.0" }, "autoload": { "classmap": ["src/"] }, "autoload-dev": { "classmap": ["tests/"] } }
9. Have the basic setup in place [2] In phpunit.xml.dist: <?xml version="1.0" encoding="UTF-8"?> <phpunit bootstrap="./vendor/autoload.php" colors="true" > <testsuites> <testsuite name="Foo"> <directory suffix="Test.php"> ./tests/ </directory> </testsuite> </testsuites> </phpunit> --generate-configuration
10. Start Small (but start somewhere) <?php namespace PHPUnit_Demo; class Foo { static function stripQuotes($string) { return preg_replace( '`^([\'"])(.*)\1$`Ds', '$2', $string ); } }
11. Start Small (but start somewhere) <?php namespace PHPUnit_Demo\Tests; use PHPUnit\Framework\TestCase; use PHPUnit_Demo\Foo; class FooTest extends TestCase { public function testStripQuotes() { $result = Foo::stripQuotes('"text"'); $this->assertEquals('text', $result); } }
12. Don't Use Your Own Code To Create Test Data
13. Test Requirements Have tests Strict assertions High code coverage, strictly measured Happy & unhappy path
14. assertEquals() == assertSame() === Most Common Issue
15. Available Assertions
16. Use the Right Assertion <?php namespace PHPUnit_Demo\Tests; use PHPUnit\Framework\TestCase; use PHPUnit_Demo\Foo; class FooTest extends TestCase { public function testStripQuotes() { $result = Foo::stripQuotes('"text"'); $this->assertEquals('text', $result); $this->assertSame('text', $result); } }
17. Test Requirements Have tests Strict assertions High code coverage, strictly measured Happy & unhappy path
18. Don't Trust Code Coverage But do examine it
19. <?php namespace PHPUnit_Demo; class Foo { static function stripQuotes($string) { return preg_replace( '`^([\'"])(.*)\1$`Ds', '$2', $string ); } }
20. Enabling Code Coverage [1] <?xml version="1.0" encoding="UTF-8"?> <phpunit beStrictAboutCoversAnnotation="true" forceCoversAnnotation="true" > ... <filter> <whitelist addUncoveredFilesFromWhitelist="true"> <directory suffix=".php">src</directory> </whitelist> </filter> <logging> <log type="coverage-clover" target="build/logs/clover.xml"/> </logging> </phpunit>
21. Enabling Code Coverage [2] <?php class FooTest extends TestCase { /** * Test Foo::stripQuotes(). * * @dataProvider dataStripQuotes * * @covers \PHPUnit_Demo\Foo::stripQuotes */ public function testStripQuotes($in, $out) { $result = Foo::stripQuotes($in); $this->assertSame($out, $result); }
22. Simplify { "scripts" : { "test": [ "vendor/bin/phpunit --no-coverage" ], "coverage": [ "vendor/bin/phpunit" ], "coverage-local": [ "vendor/bin/phpunit --coverage-html ./build/coverage-html" ] } }
24. PHPUnit >= 9.3 ? phpunit --migrate-configuration
25. Test Requirements Have tests Strict assertions High code coverage, strictly measured Happy & unhappy path
26. Tests Document Expectations
27. <?php public function testStripQuotes() { $result = Foo::stripQuotes('"some text"'); $this->assertSame('some text', $result); $result = Foo::stripQuotes("some 'text'"); $this->assertSame("some 'text'", $result); $result = Foo::stripQuotes(false); $this->assertSame('', $result); } Don't Just Test the Happy Path
28. Allow for Testing the Unhappy Path ▪ strict_types ▪ Parameter type declarations mensatic
29. Use Data Providers
30. Test Method Data Provider /** * Test Foo::stripQuotes(). * * @dataProvider dataStripQuotes * * @param mixed $in Function input. * @param string $out Expected output. * * @return void */ public function testStripQ($in, $out) { $result = Foo::stripQuotes($in); $this->assertSame($out, $result); } /** * Data provider. * * @return array[] */ public function dataStripQuotes() { return [ ['"some text"', 'some text'], ["some 'text'", "some 'text'"], [false, ''], ]; }
31. Without Data Provider With Data Provider
32. Your Tests Are Limited By Your Own Imagination
33. Test Requirements Have tests Strict assertions High code coverage, strictly measured Happy & unhappy path
34. Test Your Tests ▪ Infection https://infection.github.io/ https://youtu.be/ADKyTlaH6e4 ▪ PHPUnitCompatibility (upcoming) ▪ PHPUnitQA (upcoming) lisaleo
35. Test Quality of WordPress Plugins & Themes marktimemedia
36. WordPress Test Suite Have tests Strict assertions High code coverage, strictly measured Happy & unhappy path
37. WordPress Test Suite Have tests Strict assertions High code coverage, strictly measured Happy & unhappy path
40. Plugins and Themes Happy & unhappy path High code coverage, strictly measured Strict assertions Have tests
41. Running Your Tests on PHP 8.0 marktimemedia
42. PHPUnit Support v Compatible with: 10 PHP >= 7.4 (expected April 2021) 9 PHP >= 7.3 8 PHP >= 7.2 7 PHP 7.1, 7.2, 7.3 [EOL] 6 PHP 7.0, 7.1, 7.2 [EOL] 5 PHP 5.6, 7.0, 7.1 [EOL] Seemann
43. PHPUnit vs PHP PHPUnit / PHP 5.6 7.0 7.1 7.2 7.3 7.4 8.0 5.x 6.x 7.x 8.x 9.x 9.3+ 10.x void 8.5+
44. Constraints and Roadblocks » [OS] Supporting PHP 5.6 + 7.0 still required » Plugin tests often extend WP integration test suite » PHPUnit < 8.5/9.3 not compatible with PHP 8.0 - Mocking - @runInSeparateProcess » PHPUnit 7 Phar will not run on PHP 8.0 » Committed composer.lock file
45. Solution WP Core: "autoload-dev": { "files": [ "tests/includes/MockObject/Builder/NamespaceMatch.php", "tests/includes/MockObject/Builder/ParametersMatch.php", "tests/includes/MockObject/InvocationMocker.php", "tests/includes/MockObject/MockMethod.php" ], "exclude-from-classmap": [ "vendor/phpunit/…/MockObject/Builder/NamespaceMatch.php", "vendor/phpunit/…/MockObject/Builder/ParametersMatch.php", "vendor/phpunit/…/MockObject/InvocationMocker.php", "vendor/phpunit/…/MockObject/MockMethod.php" ] }, ▪ Composer, not phar ▪ Lock at PHPUnit 7.5 ▪ Downgrade to PHPUnit 5 in CI for PHP 5.6, 7.0 ▪ Use MockObject classes from PHPUnit 9.3 ▪ Separate process => separate group
46. Constraints For Public/OS projects Support multiple WP versions Support multiple PHP versions No control over run environment
47. Mock-based Tests PHPUnit 5.x composer.lock Platform - php 5.6 PHPUnit 5.x – 9.x CI: composer update … --ignore-platform-reqs Needs PHPUnit cross-version compatibility layer
48. WP Integration Tests PHPUnit 5.x composer.lock Platform - php 5.6 PHPUnit 5.x – 7.x CI: composer update … --ignore-platform-reqs Need PHPUnit cross-version compatibility layer Need custom autoload for MockObject
49. New Tooling PHPUnit Polyfills WP Test Utils Sponsored by: Alternative: Symfony Bridge * Supports PHPUnit 4.8 – 9.x, PHP 5.5 – 8.0
50. Implementing PHPUnit Polyfills In composer.json: { "require-dev" : { "phpunit/phpunit": "^5.0 || ^7.0", "yoast/phpunit-polyfills": "^0.2.0" } }
51. Implementing PHPUnit Polyfills What you get: ▪ Polyfills via traits for all new assertions and expectations in PHPUnit ▪ Helper to work round removal of assertAttribute*() methods ▪ An cross-version compatible abstract base TestCase (to get round void) which includes all polyfills ▪ A cross-version compatible TestListenerImplementation
52. Implementing PHPUnit Polyfills <?php use PHPUnit\Framework\TestCase; use Yoast\PHPUnitPolyfills\TestCases\TestCase; class MyTest extends TestCase { protected function setUp() { protected function set_up() { parent::setUp(); parent::set_up(); // Set up function mocks which need to be // available for all tests in this class. } }
53. Implementing WP Test Utils In composer.json: { "require-dev" : { "phpunit/phpunit": "^5.0 || ^7.0", "brain/monkey": "^2.6.0", "yoast/wp-test-utils": "^0.2.0" } }
54. Implementing WP Test Utils for BrainMonkey tests What you get: ▪ PHPUnit Polyfills ▪ BrainMonkey and Mockery set up and teardown ▪ Mockery tests not marked as risky ▪ Choice between BrainMonkey or opaque escape/translation stubs ▪ expectOutputContains() helper ▪ [YoastTestCase] Additional function stubs
55. Implementing WP Test Utils for BrainMonkey Based Tests <?php use PHPUnit\Framework\TestCase; use Yoast\WPTestUtils\BrainMonkey\[Yoast]TestCase; class MyTest extends TestCase { protected function setUp() { protected function set_up() { parent::setUp(); parent::set_up(); // Set up function mocks which need to be // available for all tests in this class. } }
56. Implementing WP Test Utils for WP Integration tests What you get: ▪ PHPUnit Polyfills ▪ Access to all WP native test utilities ▪ expectOutputContains() helper ▪ Bootstrap utilities
57. Implementing WP Test Utils for WP Integration Based Tests <?php use Yoast\WPTestUtils\WPIntegration; if ( getenv( 'WP_PLUGIN_DIR' ) !== false ) { define( 'WP_PLUGIN_DIR', getenv( 'WP_PLUGIN_DIR' ) ); } $GLOBALS['wp_tests_options'] = [ 'active_plugins' => [ 'plugin-name/main-file.php' ], ]; require_once dirname( __DIR__ ) . '/vendor/yoast/wp-test-utils/src/ WPIntegration/bootstrap-functions.php'; WPIntegration\bootstrap_it(); if ( ! defined( 'WP_PLUGIN_DIR' ) || file_exists( WP_PLUGIN_DIR . '/plugin- name/main-file.php' ) === false ) { echo 'ERROR: …', PHP_EOL; exit( 1 ); }
58. Implementing WP Test Utils for WP Integration Based Tests <?php use WP_UnitTestCase; use Yoast\WPTestUtils\WPIntegration\TestCase; class MyTest extends WP_UnitTestCase { class MyTest extends TestCase { protected function setUp() { parent::setUp(); // Set up function mocks which need to be // available for all tests in this class. } }
59. Further Reading ▪ The Grumpy Programmer's Guide to Testing PHP Applications https://grumpy-learning.com/ ▪ Your Mocks Won't Save You! https://24daysindecember.net/2020/12/08/your-mocks-wont-save-you/ ▪ My Top 10 PHPUnit Tips & Tricks https://speakerdeck.com/jrf/my-top-10-phpunit-tips-and-tricks-e6ea54ce- 2515-4ea9-aacf-9bf7ab3b3141 ▪ PHPUnit Documentation https://phpunit.readthedocs.io/ ▪ Path Coverage in PHPUnit https://doug.codes/php-code-coverage
60. Tooling ▪ PHPUnit Polyfills https://packagist.org/packages/yoast/phpunit-polyfills ▪ WP Test Utils https://packagist.org/packages/yoast/wp-test-utils
61. Thanks! @jrf_nl @jrfnl Any questions ? Slides: https://speakerdeck.com/jrf