Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Writing Tests For PHP Source - Northeast PHP 2017

Writing Tests For PHP Source - Northeast PHP 2017

Talk given at Northeast PHP on August 8, 2017.

As you might have already heard from a grumpy individual, adding more tests to your codebase is crucial to its stability over time. This talk isn't about adding tests to your PHP codebase, but adding tests to the PHP language itself. And the best part is, all the tests are written in PHP so you don't even have to dust off that old C book from college.

In this talk I will show you how to find untested parts of the PHP source code, how to write a test for the untested feature and how to submit your tests to PHP internals. Not only will you be making PHP more stable and reliable, but there's a good chance the tests you write will give you a deeper understanding of PHP; knowledge you can use to improve your own codebases. Not to mention you'll be able to call yourself an internals contributor. Come on in! The water's fine!

Sammy Kaye Powers

August 11, 2017
Tweet

More Decks by Sammy Kaye Powers

Other Decks in Programming

Transcript

  1. joind.in/talk/7c85b
    @SammyK #nephp17
    For PHP Source
    Writing Tests
    Sammy Kaye Powers
    2017-08-11

    View full-size slide

  2. joind.in/talk/7c85b
    @SammyK #nephp17
    Slides
    Get the
    joind.in/talk/7c85b

    View full-size slide

  3. joind.in/talk/7c85b
    @SammyK #nephp17
    PHP Internals
    This is a
    talk

    View full-size slide

  4. Write more
    TESTS!

    View full-size slide

  5. joind.in/talk/7c85b
    @SammyK #nephp17
    Unit tests?

    View full-size slide

  6. joind.in/talk/7c85b
    @SammyK #nephp17

    View full-size slide

  7. joind.in/talk/7c85b
    @SammyK #nephp17
    @dhh
    “TDD is dead.”
    http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html

    View full-size slide

  8. joind.in/talk/7c85b
    @SammyK #nephp17
    Acceptance tests?

    View full-size slide

  9. joind.in/talk/7c85b
    @SammyK #nephp17
    Integration tests?
    *Sort of

    View full-size slide

  10. joind.in/talk/7c85b
    @SammyK #nephp17
    Black-box
    Testing

    View full-size slide

  11. joind.in/talk/7c85b
    @SammyK #nephp17
    Testing
    …you could probably argue that it is
    Integration

    View full-size slide

  12. joind.in/talk/7c85b
    @SammyK #nephp17
    Testing
    …but it’s more like
    System

    View full-size slide

  13. joind.in/talk/7c85b
    @SammyK #nephp17
    Black-box
    Testing?
    What is this

    View full-size slide

  14. joind.in/talk/7c85b
    @SammyK #nephp17

    View full-size slide

  15. joind.in/talk/7c85b
    @SammyK #nephp17
    What is
    in there?

    View full-size slide

  16. joind.in/talk/7c85b
    @SammyK #nephp17
    Input
    Expected
    Output

    View full-size slide

  17. joind.in/talk/7c85b
    @SammyK #nephp17
    run-tests.php

    View full-size slide

  18. joind.in/talk/7c85b
    @SammyK #nephp17
    .phpt
    Expected
    Hello
    World
    Output
    Hello
    World

    View full-size slide

  19. joind.in/talk/7c85b
    @SammyK #nephp17
    run-tests.php
    Development

    View full-size slide

  20. joind.in/talk/7c85b
    @SammyK #nephp17
    run-tests.php

    View full-size slide

  21. STFU Operator

    View full-size slide

  22. joind.in/talk/7c85b
    @SammyK #nephp17

    View full-size slide

  23. joind.in/talk/7c85b
    @SammyK #nephp17

    View full-size slide

  24. joind.in/talk/7c85b
    @SammyK #nephp17
    run-tests.php
    Development

    View full-size slide

  25. joind.in/talk/7c85b
    @SammyK #nephp17
    Files

    View full-size slide

  26. joind.in/talk/7c85b
    @SammyK #nephp17
    /ext/{name}/tests
    /Zend/tests /sapi/{name}/tests
    Extensions
    SAPI’s
    Zend Engine
    /tests

    View full-size slide

  27. joind.in/talk/7c85b
    @SammyK #nephp17
    Input Output

    View full-size slide

  28. joind.in/talk/7c85b
    @SammyK #nephp17
    Running
    run-tests
    with

    View full-size slide

  29. joind.in/talk/7c85b
    @SammyK #nephp17
    1 source code
    Clone the

    View full-size slide

  30. joind.in/talk/7c85b
    @SammyK #nephp17
    $ git clone \
    [email protected]:php/php-src.git

    View full-size slide

  31. joind.in/talk/7c85b
    @SammyK #nephp17
    2source
    Compile from

    View full-size slide

  32. joind.in/talk/7c85b
    @SammyK #nephp17
    $ ./buildconf

    View full-size slide

  33. joind.in/talk/7c85b
    @SammyK #nephp17
    $ ./configure

    View full-size slide

  34. joind.in/talk/7c85b
    @SammyK #nephp17
    $ ./configure \
    --disable-all

    View full-size slide

  35. joind.in/talk/7c85b
    @SammyK #nephp17
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli

    View full-size slide

  36. joind.in/talk/7c85b
    @SammyK #nephp17
    $ make

    View full-size slide

  37. joind.in/talk/7c85b
    @SammyK #nephp17

    View full-size slide

  38. $ sapi/cli/php --version

    View full-size slide

  39. joind.in/talk/7c85b
    @SammyK #nephp17
    3tests
    Run the

    View full-size slide

  40. joind.in/talk/7c85b
    @SammyK #nephp17
    Two Ways
    to run
    tests

    View full-size slide

  41. joind.in/talk/7c85b
    @SammyK #nephp17
    run-tests.php
    Run the with…
    directly

    View full-size slide

  42. joind.in/talk/7c85b
    @SammyK #nephp17
    Run the with…
    the Makefile
    Recommended!

    View full-size slide

  43. Recommended!

    View full-size slide

  44. Recommended!

    View full-size slide

  45. joind.in/talk/7c85b
    @SammyK #nephp17
    Let’s
    Make a test!

    View full-size slide

  46. joind.in/talk/7c85b
    @SammyK #nephp17
    test-echo.phpt
    Save as…
    php
    Run from

    View full-size slide

  47. joind.in/talk/7c85b
    @SammyK #nephp17
    Test it!
    $ make test TESTS=test-echo.phpt

    View full-size slide

  48. joind.in/talk/7c85b
    @SammyK #nephp17
    Fail
    When tests

    View full-size slide

  49. joind.in/talk/7c85b
    @SammyK #nephp17
    Input Output
    Expected

    View full-size slide

  50. joind.in/talk/7c85b
    @SammyK #nephp17
    Output
    Examining the
    !==

    View full-size slide

  51. Files
    test-echo.out
    test-echo.exp
    test-echo.diff
    test-echo.log
    Output

    View full-size slide

  52. joind.in/talk/7c85b
    @SammyK #nephp17
    $ less test-echo.out
    Hello world
    Script output

    View full-size slide

  53. joind.in/talk/7c85b
    @SammyK #nephp17
    $ less test-echo.exp
    Hello Jane!
    --EXPECT--

    View full-size slide

  54. joind.in/talk/7c85b
    @SammyK #nephp17
    $ less test-echo.diff
    001+ Hello world
    001- Hello Jane!

    View full-size slide

  55. joind.in/talk/7c85b
    @SammyK #nephp17
    $ less test-echo.log
    ---- EXPECTED OUTPUT
    Hello Jane!
    ---- ACTUAL OUTPUT
    Hello world
    ---- FAILED

    View full-size slide

  56. Files
    test-echo.php
    test-echo.sh
    Executable

    View full-size slide

  57. joind.in/talk/7c85b
    @SammyK #nephp17
    test-echo.php
    --FILE--

    View full-size slide

  58. test-echo.sh

    View full-size slide

  59. joind.in/talk/7c85b
    @SammyK #nephp17
    Example
    A more complex

    View full-size slide

  60. joind.in/talk/7c85b
    @SammyK #nephp17
    Fails
    So say it

    View full-size slide

  61. $ less test-dt.diff
    001+ Boy you know I'm
    from Europe/Oslo
    001- Boy you know I'm
    from America/Chicago

    View full-size slide

  62. test-dt.php
    Executable!

    View full-size slide

  63. joind.in/talk/7c85b
    @SammyK #nephp17
    Debugging
    $ gdb …
    failed tests with

    View full-size slide

  64. joind.in/talk/7c85b
    @SammyK #nephp17
    Conventions
    Naming

    View full-size slide

  65. joind.in/talk/7c85b
    @SammyK #nephp17
    {function-name}_basic.phpt
    {function-name}_error.phpt
    {function-name}_variation.phpt
    {ext-name}{number}.phpt
    bug{bug-id}.phpt
    http://qa.php.net/write-test.php#naming-conventions

    View full-size slide

  66. joind.in/talk/7c85b
    @SammyK #nephp17
    Contribute
    How-to

    View full-size slide

  67. joind.in/talk/7c85b
    @SammyK #nephp17
    Tests!
    Add some

    View full-size slide

  68. gcov.php.net

    View full-size slide

  69. PHP_FUNCTION

    View full-size slide

  70. joind.in/talk/7c85b
    @SammyK #nephp17
    ?
    readline()

    View full-size slide

  71. joind.in/talk/7c85b
    @SammyK #nephp17
    php.net/readline

    View full-size slide

  72. joind.in/talk/7c85b
    @SammyK #nephp17
    Let’s
    Make a test!

    View full-size slide

  73. joind.in/talk/7c85b
    @SammyK #nephp17

    View full-size slide

  74. joind.in/talk/7c85b
    @SammyK #nephp17
    for ext/readline
    Find the
    ext/readline/tests

    View full-size slide

  75. joind.in/talk/7c85b
    @SammyK #nephp17
    Save as…
    ext/readline/tests/readline_basic.phpt

    View full-size slide

  76. joind.in/talk/7c85b
    @SammyK #nephp17
    $ less ext/readline/tests/readline_basic.out
    Fatal error: Uncaught Error: Call
    to undefined function readline() in
    /Users/SammyK/Developer/php-src/
    ext/readline/tests/
    readline_basic.php:2

    View full-size slide

  77. joind.in/talk/7c85b
    @SammyK #nephp17
    php.net/readline

    View full-size slide

  78. joind.in/talk/7c85b
    @SammyK #nephp17

    View full-size slide

  79. joind.in/talk/7c85b
    @SammyK #nephp17
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli

    View full-size slide

  80. joind.in/talk/7c85b
    @SammyK #nephp17
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli \
    --with-readline

    View full-size slide

  81. joind.in/talk/7c85b
    @SammyK #nephp17
    OR

    View full-size slide

  82. joind.in/talk/7c85b
    @SammyK #nephp17
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli \
    --with-libedit

    View full-size slide

  83. joind.in/talk/7c85b
    @SammyK #nephp17
    $ less ext/readline/tests/readline_basic.out
    You said:

    View full-size slide

  84. joind.in/talk/7c85b
    @SammyK #nephp17
    readline_basic.sh
    Let’s try

    View full-size slide

  85. joind.in/talk/7c85b
    @SammyK #nephp17

    View full-size slide

  86. Now send a
    pull request

    View full-size slide

  87. joind.in/talk/7c85b
    @SammyK #nephp17
    internals contributor
    You’re an

    View full-size slide

  88. joind.in/talk/7c85b
    @SammyK #nephp17
    phptestfest.org

    View full-size slide

  89. joind.in/talk/7c85b
    @SammyK #nephp17
    sammyk.me
    Six-part
    screencast
    series

    View full-size slide

  90. refactor
    github.com/SammyK/run-tests
    run-tests

    View full-size slide

  91. refactor
    run-tests

    View full-size slide

  92. wishlist
    * Concurrency
    * Unit tests
    * Colored output
    run-tests

    View full-size slide

  93. joind.in/talk/7c85b
    @SammyK #nephp17
    Code
    with me today!
    Kent Room
    after lunch @ 1pm
    (unconference room)

    View full-size slide

  94. THANKS!
    SAMMY KAYE POWERS
    @SammyK
    SammyK.me
    Host of @PHPRoundtable
    @ChiPHPUG
    West Coast Swing
    /talk/7c85b
    I have
    stickers!

    View full-size slide