Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Writing Tests For PHP Source - ZendCon 2017

Writing Tests For PHP Source - ZendCon 2017

Talk given at ZendCon 2017

As you might have already heard from a grumpy individual, adding more tests to your codebase is crucial to its stability over time. This session isn't about adding tests to your PHP codebase, but adding tests to the PHP language itself. And the best part is, all the tests are written in PHP so you don't even have to dust off that old C book from college. In this session you'll see how to find untested parts of the PHP source code, how to write a test for the untested feature, and how to submit your tests to PHP internals. Not only will you be making PHP more stable and reliable, but there's a good chance the tests you write will give you a deeper understanding of PHP; knowledge you can use to improve your own codebases. Not to mention you'll be able to call yourself an internals contributor. Come on in! The water's fine!

Sammy Kaye Powers

October 26, 2017
Tweet

More Decks by Sammy Kaye Powers

Other Decks in Programming

Transcript

  1. joind.in/talk/c6253
    @SammyK #ZendCon2017
    For PHP Source
    Writing Tests
    Sammy Kaye Powers
    2017-10-26

    View full-size slide

  2. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Slides
    Get the
    joind.in/talk/c6253

    View full-size slide

  3. joind.in/talk/c6253
    @SammyK #ZendCon2017
    PHP Internals
    This is a
    talk

    View full-size slide

  4. Write more
    TESTS!

    View full-size slide

  5. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Unit tests?

    View full-size slide

  6. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View full-size slide

  7. joind.in/talk/c6253
    @SammyK #ZendCon2017
    @dhh
    “TDD is dead.”
    http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html

    View full-size slide

  8. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Acceptance tests?

    View full-size slide

  9. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Integration tests?
    *Sort of

    View full-size slide

  10. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Black-box
    Testing

    View full-size slide

  11. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Testing
    …you could probably argue that it is
    Integration

    View full-size slide

  12. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Testing
    …but it’s more like
    System

    View full-size slide

  13. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Black-box
    Testing?
    What is this

    View full-size slide

  14. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View full-size slide

  15. joind.in/talk/c6253
    @SammyK #ZendCon2017
    What is
    in there?

    View full-size slide

  16. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Input
    Expected
    Output

    View full-size slide

  17. joind.in/talk/c6253
    @SammyK #ZendCon2017
    run-tests.php

    View full-size slide

  18. joind.in/talk/c6253
    @SammyK #ZendCon2017
    .phpt
    Expected
    Hello
    World
    Output
    Hello
    World

    View full-size slide

  19. joind.in/talk/c6253
    @SammyK #ZendCon2017
    run-tests.php
    Development

    View full-size slide

  20. joind.in/talk/c6253
    @SammyK #ZendCon2017
    run-tests.php

    View full-size slide

  21. STFU Operator

    View full-size slide

  22. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View full-size slide

  23. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View full-size slide

  24. joind.in/talk/c6253
    @SammyK #ZendCon2017
    run-tests.php
    Development

    View full-size slide

  25. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Files

    View full-size slide

  26. joind.in/talk/c6253
    @SammyK #ZendCon2017
    /ext/{name}/tests
    /Zend/tests /sapi/{name}/tests
    Extensions
    SAPI’s
    Zend Engine
    /tests

    View full-size slide

  27. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Input Output

    View full-size slide

  28. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Running
    run-tests
    with

    View full-size slide

  29. joind.in/talk/c6253
    @SammyK #ZendCon2017
    1 source code
    Clone the

    View full-size slide

  30. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ git clone \
    [email protected]:php/php-src.git

    View full-size slide

  31. joind.in/talk/c6253
    @SammyK #ZendCon2017
    2source
    Compile from

    View full-size slide

  32. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ ./buildconf

    View full-size slide

  33. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ ./configure

    View full-size slide

  34. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ ./configure \
    --disable-all

    View full-size slide

  35. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli

    View full-size slide

  36. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ make

    View full-size slide

  37. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View full-size slide

  38. $ sapi/cli/php --version

    View full-size slide

  39. joind.in/talk/c6253
    @SammyK #ZendCon2017
    3tests
    Run the

    View full-size slide

  40. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Two Ways
    to run
    tests

    View full-size slide

  41. joind.in/talk/c6253
    @SammyK #ZendCon2017
    run-tests.php
    Run the with…
    directly

    View full-size slide

  42. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Run the with…
    the Makefile
    Recommended!

    View full-size slide

  43. Recommended!

    View full-size slide

  44. Recommended!

    View full-size slide

  45. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Let’s
    Make a test!

    View full-size slide

  46. joind.in/talk/c6253
    @SammyK #ZendCon2017
    test-echo.phpt
    Save as…
    php
    Run from

    View full-size slide

  47. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Test it!
    $ make test TESTS=test-echo.phpt

    View full-size slide

  48. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Fail
    When tests

    View full-size slide

  49. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Input Output
    Expected

    View full-size slide

  50. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Output
    Examining the
    !==

    View full-size slide

  51. Files
    test-echo.out
    test-echo.exp
    test-echo.diff
    test-echo.log
    Output

    View full-size slide

  52. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ less test-echo.out
    Hello world
    Script output

    View full-size slide

  53. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ less test-echo.exp
    Hello Jane!
    --EXPECT--

    View full-size slide

  54. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ less test-echo.diff
    001+ Hello world
    001- Hello Jane!

    View full-size slide

  55. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ less test-echo.log
    ---- EXPECTED OUTPUT
    Hello Jane!
    ---- ACTUAL OUTPUT
    Hello world
    ---- FAILED

    View full-size slide

  56. Files
    test-echo.php
    test-echo.sh
    Executable

    View full-size slide

  57. joind.in/talk/c6253
    @SammyK #ZendCon2017
    test-echo.php
    --FILE--

    View full-size slide

  58. test-echo.sh

    View full-size slide

  59. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Example
    A more complex

    View full-size slide

  60. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Fails
    So say it

    View full-size slide

  61. $ less test-dt.diff
    001+ Boy you know I'm
    from Europe/Oslo
    001- Boy you know I'm
    from America/Chicago

    View full-size slide

  62. test-dt.php
    Executable!

    View full-size slide

  63. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Debugging
    $ gdb …
    failed tests with

    View full-size slide

  64. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Conventions
    Naming

    View full-size slide

  65. joind.in/talk/c6253
    @SammyK #ZendCon2017
    {function-name}_basic.phpt
    {function-name}_error.phpt
    {function-name}_variation.phpt
    {ext-name}{number}.phpt
    bug{bug-id}.phpt
    http://qa.php.net/write-test.php#naming-conventions

    View full-size slide

  66. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Contribute
    How-to

    View full-size slide

  67. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Tests!
    Add some

    View full-size slide

  68. gcov.php.net

    View full-size slide

  69. PHP_FUNCTION

    View full-size slide

  70. joind.in/talk/c6253
    @SammyK #ZendCon2017
    ?
    readline()

    View full-size slide

  71. joind.in/talk/c6253
    @SammyK #ZendCon2017
    php.net/readline

    View full-size slide

  72. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Let’s
    Make a test!

    View full-size slide

  73. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View full-size slide

  74. joind.in/talk/c6253
    @SammyK #ZendCon2017
    for ext/readline
    Find the
    ext/readline/tests

    View full-size slide

  75. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Save as…
    ext/readline/tests/readline_basic.phpt

    View full-size slide

  76. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ less ext/readline/tests/readline_basic.out
    Fatal error: Uncaught Error: Call
    to undefined function readline() in
    /Users/SammyK/Developer/php-src/
    ext/readline/tests/
    readline_basic.php:2

    View full-size slide

  77. joind.in/talk/c6253
    @SammyK #ZendCon2017
    php.net/readline

    View full-size slide

  78. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View full-size slide

  79. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli

    View full-size slide

  80. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli \
    --with-readline

    View full-size slide

  81. joind.in/talk/c6253
    @SammyK #ZendCon2017
    OR

    View full-size slide

  82. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli \
    --with-libedit

    View full-size slide

  83. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ less ext/readline/tests/readline_basic.out
    You said:

    View full-size slide

  84. joind.in/talk/c6253
    @SammyK #ZendCon2017
    readline_basic.sh
    Let’s try

    View full-size slide

  85. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View full-size slide

  86. Now send a
    pull request

    View full-size slide

  87. joind.in/talk/c6253
    @SammyK #ZendCon2017
    internals contributor
    You’re an

    View full-size slide

  88. joind.in/talk/c6253
    @SammyK #ZendCon2017
    phptestfest.org

    View full-size slide

  89. joind.in/talk/c6253
    @SammyK #ZendCon2017
    sammyk.me
    Six-part
    screencast
    series

    View full-size slide

  90. refactor
    github.com/SammyK/run-tests
    run-tests

    View full-size slide

  91. refactor
    run-tests

    View full-size slide

  92. wishlist
    * Concurrency
    * Unit tests
    * Colored output
    run-tests

    View full-size slide

  93. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Code
    with me today!
    Uncon Room
    @ 9:15am
    (by Paradise tower)

    View full-size slide

  94. Sammy Kaye Powers
    Thanks!
    /talk/c6253
    @SammyK
    SammyK.me
    Host of @PHPRoundtable
    Co-admin of @ChiPHPUG
    I ❤ West Coast Swing

    View full-size slide