Essential Testing Know-How for Testing on PHP 8+

Presented on March 12th 2021 for employees of Automattic (closed learnup).

Transcript

1. Essential Testing Know-How
   for testing on PHP 8+
   Juliette Reinders Folmer Tweet about it: @jrf_nl
   

   View Slide

2. test
   Test
   TEST
   

   View Slide

3. View Slide

4. Assessing
   & Improving
   Test
   Quality
   marktimemedia
   

   View Slide

5. Test
   Types Unit
   Tests Integration Tests
   E2E Tests
   Acceptance Tests
   

   View Slide

6. Test Requirements
   Have tests Strict assertions
   High code
   coverage,
   strictly measured
   Happy &
   unhappy path
   

   View Slide

7. Test Requirements
   Have tests Strict assertions
   High code
   coverage,
   strictly measured
   Happy &
   unhappy path
   

   View Slide

8. Have the basic
   setup in place [1]
   In composer.json:
   {
   "require-dev" : {
   "phpunit/phpunit": "^8.0 || ^9.0"
   },
   "autoload": {
   "classmap": ["src/"]
   },
   "autoload-dev": {
   "classmap": ["tests/"]
   }
   }
   

   View Slide

9. Have the basic
   setup in place [2]
   In phpunit.xml.dist:
   
   bootstrap="./vendor/autoload.php"
   colors="true"
   >
   
   
   
   ./tests/
   
   
   
   
   --generate-configuration
   

   View Slide

10. Start Small
    (but start somewhere)
    namespace PHPUnit_Demo;
    class Foo
    {
    static function stripQuotes($string)
    {
    return preg_replace(
    '`^([\'"])(.*)\1$`Ds',
    '$2',
    $string
    );
    }
    }
    

    View Slide

11. Start Small
    (but start somewhere)
    namespace PHPUnit_Demo\Tests;
    use PHPUnit\Framework\TestCase;
    use PHPUnit_Demo\Foo;
    class FooTest extends TestCase
    {
    public function testStripQuotes()
    {
    $result = Foo::stripQuotes('"text"');
    $this->assertEquals('text', $result);
    }
    }
    

    View Slide

12. Don't Use Your
    Own Code To
    Create Test Data
    

    View Slide

13. Test Requirements
    Have tests Strict assertions
    High code
    coverage,
    strictly measured
    Happy &
    unhappy path
    

    View Slide

14. assertEquals()
    ==
    assertSame()
    ===
    Most Common Issue
    

    View Slide

15. Available Assertions
    

    View Slide

16. Use the
    Right Assertion
    namespace PHPUnit_Demo\Tests;
    use PHPUnit\Framework\TestCase;
    use PHPUnit_Demo\Foo;
    class FooTest extends TestCase
    {
    public function testStripQuotes()
    {
    $result = Foo::stripQuotes('"text"');
    $this->assertEquals('text', $result);
    $this->assertSame('text', $result);
    }
    }
    

    View Slide

17. Test Requirements
    Have tests Strict assertions
    High code
    coverage,
    strictly measured
    Happy &
    unhappy path
    

    View Slide

18. Don't Trust Code
    Coverage
    But do examine it
    

    View Slide

19. namespace PHPUnit_Demo;
    class Foo
    {
    static function stripQuotes($string)
    {
    return preg_replace(
    '`^([\'"])(.*)\1$`Ds',
    '$2',
    $string
    );
    }
    }
    

    View Slide

20. Enabling Code
    Coverage [1]
    
    beStrictAboutCoversAnnotation="true"
    forceCoversAnnotation="true"
    >
    ...
    
    addUncoveredFilesFromWhitelist="true">
    src
    
    
    
    target="build/logs/clover.xml"/>
    
    
    

    View Slide

21. Enabling Code
    Coverage [2]
    class FooTest extends TestCase
    {
    /**
    * Test Foo::stripQuotes().
    *
    * @dataProvider dataStripQuotes
    *
    * @covers \PHPUnit_Demo\Foo::stripQuotes
    */
    public function testStripQuotes($in, $out)
    {
    $result = Foo::stripQuotes($in);
    $this->assertSame($out, $result);
    }
    

    View Slide

22. Simplify
    {
    "scripts" : {
    "test": [
    "vendor/bin/phpunit --no-coverage"
    ],
    "coverage": [
    "vendor/bin/phpunit"
    ],
    "coverage-local": [
    "vendor/bin/phpunit
    --coverage-html ./build/coverage-html"
    ]
    }
    }
    

    View Slide

23. View Slide

24. PHPUnit >= 9.3 ?
    phpunit --migrate-configuration
    

    View Slide

25. Test Requirements
    Have tests Strict assertions
    High code
    coverage,
    strictly measured
    Happy &
    unhappy path
    

    View Slide

26. Tests
    Document
    Expectations
    

    View Slide

27. public function testStripQuotes()
    {
    $result = Foo::stripQuotes('"some text"');
    $this->assertSame('some text', $result);
    $result = Foo::stripQuotes("some 'text'");
    $this->assertSame("some 'text'", $result);
    $result = Foo::stripQuotes(false);
    $this->assertSame('', $result);
    }
    Don't Just Test the Happy Path
    

    View Slide

28. Allow for Testing the
    Unhappy Path
    ▪ strict_types
    ▪ Parameter type
    declarations
    mensatic
    

    View Slide

29. Use Data Providers
    

    View Slide

30. Test Method Data Provider
    /**
    * Test Foo::stripQuotes().
    *
    * @dataProvider dataStripQuotes
    *
    * @param mixed $in Function input.
    * @param string $out Expected output.
    *
    * @return void
    */
    public function testStripQ($in, $out)
    {
    $result = Foo::stripQuotes($in);
    $this->assertSame($out, $result);
    }
    /**
    * Data provider.
    *
    * @return array[]
    */
    public function dataStripQuotes()
    {
    return [
    ['"some text"', 'some text'],
    ["some 'text'", "some 'text'"],
    [false, ''],
    ];
    }
    

    View Slide

31. Without Data Provider
    With Data Provider
    

    View Slide

32. Your Tests Are
    Limited By Your
    Own Imagination
    

    View Slide

33. Test Requirements
    Have tests Strict assertions
    High code
    coverage,
    strictly measured
    Happy &
    unhappy path
    

    View Slide

34. Test Your Tests
    ▪ Infection
    https://infection.github.io/
    https://youtu.be/ADKyTlaH6e4
    ▪ PHPUnitCompatibility
    (upcoming)
    ▪ PHPUnitQA
    (upcoming)
    lisaleo
    

    View Slide

35. Test
    Quality of
    WordPress
    Plugins & Themes
    marktimemedia
    

    View Slide

36. WordPress Test Suite
    Have tests Strict assertions
    High code
    coverage,
    strictly measured
    Happy &
    unhappy path
    

    View Slide

37. WordPress Test Suite
    Have tests Strict assertions
    High code
    coverage,
    strictly measured
    Happy &
    unhappy path
    

    View Slide

38. View Slide

39. View Slide

40. Plugins and Themes
    Happy &
    unhappy path
    High code
    coverage,
    strictly measured
    Strict assertions
    Have tests
    

    View Slide

41. Running
    Your Tests
    on PHP 8.0
    marktimemedia
    

    View Slide

42. PHPUnit Support
    v Compatible with:
    10 PHP >= 7.4
    (expected April 2021)
    9 PHP >= 7.3
    8 PHP >= 7.2
    7 PHP 7.1, 7.2, 7.3 [EOL]
    6 PHP 7.0, 7.1, 7.2 [EOL]
    5 PHP 5.6, 7.0, 7.1 [EOL]
    Seemann
    

    View Slide

43. PHPUnit vs PHP
    PHPUnit / PHP 5.6 7.0 7.1 7.2 7.3 7.4 8.0
    5.x
    6.x
    7.x
    8.x
    9.x 9.3+
    10.x
    void
    8.5+
    

    View Slide

44. Constraints
    and
    Roadblocks
    » [OS] Supporting PHP 5.6 + 7.0
    still required
    » Plugin tests often extend
    WP integration test suite
    » PHPUnit < 8.5/9.3 not
    compatible with PHP 8.0
    - Mocking
    - @runInSeparateProcess
    » PHPUnit 7 Phar will not run on
    PHP 8.0
    » Committed composer.lock
    file
    

    View Slide

45. Solution WP Core:
    "autoload-dev": {
    "files": [
    "tests/includes/MockObject/Builder/NamespaceMatch.php",
    "tests/includes/MockObject/Builder/ParametersMatch.php",
    "tests/includes/MockObject/InvocationMocker.php",
    "tests/includes/MockObject/MockMethod.php"
    ],
    "exclude-from-classmap": [
    "vendor/phpunit/…/MockObject/Builder/NamespaceMatch.php",
    "vendor/phpunit/…/MockObject/Builder/ParametersMatch.php",
    "vendor/phpunit/…/MockObject/InvocationMocker.php",
    "vendor/phpunit/…/MockObject/MockMethod.php"
    ]
    },
    ▪ Composer, not phar
    ▪ Lock at PHPUnit 7.5
    ▪ Downgrade to PHPUnit 5
    in CI for PHP 5.6, 7.0
    ▪ Use MockObject classes
    from PHPUnit 9.3
    ▪ Separate process =>
    separate group
    

    View Slide

46. Constraints For Public/OS projects
    Support multiple
    WP versions
    Support multiple
    PHP versions
    No control over run
    environment
    

    View Slide

47. Mock-based Tests
    PHPUnit 5.x
    composer.lock
    Platform - php 5.6
    PHPUnit 5.x – 9.x
    CI: composer update …
    --ignore-platform-reqs
    Needs PHPUnit cross-version
    compatibility layer
    

    View Slide

48. WP Integration Tests
    PHPUnit 5.x
    composer.lock
    Platform - php 5.6
    PHPUnit 5.x – 7.x
    CI: composer update …
    --ignore-platform-reqs
    Need PHPUnit cross-version
    compatibility layer
    Need custom autoload for MockObject
    

    View Slide

49. New Tooling
    PHPUnit
    Polyfills
    WP Test Utils
    Sponsored by:
    Alternative:
    Symfony Bridge
    * Supports PHPUnit 4.8 – 9.x, PHP 5.5 – 8.0
    

    View Slide

50. Implementing
    PHPUnit Polyfills
    In composer.json:
    {
    "require-dev" : {
    "phpunit/phpunit": "^5.0 || ^7.0",
    "yoast/phpunit-polyfills": "^0.2.0"
    }
    }
    

    View Slide

51. Implementing
    PHPUnit Polyfills
    What you get:
    ▪ Polyfills via traits for all new
    assertions and expectations in
    PHPUnit
    ▪ Helper to work round removal
    of assertAttribute*()
    methods
    ▪ An cross-version compatible
    abstract base TestCase (to get
    round void) which includes all
    polyfills
    ▪ A cross-version compatible
    TestListenerImplementation
    

    View Slide

52. Implementing PHPUnit Polyfills
    use PHPUnit\Framework\TestCase;
    use Yoast\PHPUnitPolyfills\TestCases\TestCase;
    class MyTest extends TestCase {
    protected function setUp() {
    protected function set_up() {
    parent::setUp();
    parent::set_up();
    // Set up function mocks which need to be
    // available for all tests in this class.
    }
    }
    

    View Slide

53. Implementing
    WP Test Utils
    In composer.json:
    {
    "require-dev" : {
    "phpunit/phpunit": "^5.0 || ^7.0",
    "brain/monkey": "^2.6.0",
    "yoast/wp-test-utils": "^0.2.0"
    }
    }
    

    View Slide

54. Implementing
    WP Test Utils for
    BrainMonkey tests
    What you get:
    ▪ PHPUnit Polyfills
    ▪ BrainMonkey and Mockery
    set up and teardown
    ▪ Mockery tests not marked as
    risky
    ▪ Choice between
    BrainMonkey or opaque
    escape/translation stubs
    ▪ expectOutputContains()
    helper
    ▪ [YoastTestCase] Additional
    function stubs
    

    View Slide

55. Implementing WP Test Utils for BrainMonkey Based Tests
    use PHPUnit\Framework\TestCase;
    use Yoast\WPTestUtils\BrainMonkey\[Yoast]TestCase;
    class MyTest extends TestCase {
    protected function setUp() {
    protected function set_up() {
    parent::setUp();
    parent::set_up();
    // Set up function mocks which need to be
    // available for all tests in this class.
    }
    }
    

    View Slide

56. Implementing
    WP Test Utils for
    WP Integration tests
    What you get:
    ▪ PHPUnit Polyfills
    ▪ Access to all WP native test
    utilities
    ▪ expectOutputContains()
    helper
    ▪ Bootstrap utilities
    

    View Slide

57. Implementing WP Test Utils for WP Integration Based Tests
    use Yoast\WPTestUtils\WPIntegration;
    if ( getenv( 'WP_PLUGIN_DIR' ) !== false ) {
    define( 'WP_PLUGIN_DIR', getenv( 'WP_PLUGIN_DIR' ) );
    }
    $GLOBALS['wp_tests_options'] = [
    'active_plugins' => [ 'plugin-name/main-file.php' ],
    ];
    require_once dirname( __DIR__ ) . '/vendor/yoast/wp-test-utils/src/
    WPIntegration/bootstrap-functions.php';
    WPIntegration\bootstrap_it();
    if ( ! defined( 'WP_PLUGIN_DIR' ) || file_exists( WP_PLUGIN_DIR . '/plugin-
    name/main-file.php' ) === false ) {
    echo 'ERROR: …', PHP_EOL;
    exit( 1 );
    }
    

    View Slide

58. Implementing WP Test Utils for WP Integration Based Tests
    use WP_UnitTestCase;
    use Yoast\WPTestUtils\WPIntegration\TestCase;
    class MyTest extends WP_UnitTestCase {
    class MyTest extends TestCase {
    protected function setUp() {
    parent::setUp();
    // Set up function mocks which need to be
    // available for all tests in this class.
    }
    }
    

    View Slide

59. Further Reading
    ▪ The Grumpy Programmer's Guide to Testing PHP Applications
    https://grumpy-learning.com/
    ▪ Your Mocks Won't Save You!
    https://24daysindecember.net/2020/12/08/your-mocks-wont-save-you/
    ▪ My Top 10 PHPUnit Tips & Tricks
    https://speakerdeck.com/jrf/my-top-10-phpunit-tips-and-tricks-e6ea54ce-
    2515-4ea9-aacf-9bf7ab3b3141
    ▪ PHPUnit Documentation
    https://phpunit.readthedocs.io/
    ▪ Path Coverage in PHPUnit
    https://doug.codes/php-code-coverage
    

    View Slide

60. Tooling
    ▪ PHPUnit Polyfills
    https://packagist.org/packages/yoast/phpunit-polyfills
    ▪ WP Test Utils
    https://packagist.org/packages/yoast/wp-test-utils
    

    View Slide

61. Thanks!
    @jrf_nl @jrfnl
    Any
    questions ?
    Slides: https://speakerdeck.com/jrf
    

    View Slide