Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Writing Tests For PHP Source - Northeast PHP 2017

Writing Tests For PHP Source - Northeast PHP 2017

Talk given at Northeast PHP on August 8, 2017.

As you might have already heard from a grumpy individual, adding more tests to your codebase is crucial to its stability over time. This talk isn't about adding tests to your PHP codebase, but adding tests to the PHP language itself. And the best part is, all the tests are written in PHP so you don't even have to dust off that old C book from college.

In this talk I will show you how to find untested parts of the PHP source code, how to write a test for the untested feature and how to submit your tests to PHP internals. Not only will you be making PHP more stable and reliable, but there's a good chance the tests you write will give you a deeper understanding of PHP; knowledge you can use to improve your own codebases. Not to mention you'll be able to call yourself an internals contributor. Come on in! The water's fine!

Sammy Kaye Powers

August 11, 2017
Tweet

More Decks by Sammy Kaye Powers

Other Decks in Programming

Transcript

  1. joind.in/talk/7c85b
    @SammyK #nephp17
    For PHP Source
    Writing Tests
    Sammy Kaye Powers
    2017-08-11

    View Slide

  2. joind.in/talk/7c85b
    @SammyK #nephp17
    Slides
    Get the
    joind.in/talk/7c85b

    View Slide

  3. joind.in/talk/7c85b
    @SammyK #nephp17
    PHP Internals
    This is a
    talk

    View Slide

  4. Write more
    TESTS!

    View Slide

  5. joind.in/talk/7c85b
    @SammyK #nephp17
    Unit tests?

    View Slide

  6. joind.in/talk/7c85b
    @SammyK #nephp17

    View Slide

  7. joind.in/talk/7c85b
    @SammyK #nephp17
    @dhh
    “TDD is dead.”
    http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html

    View Slide

  8. joind.in/talk/7c85b
    @SammyK #nephp17
    Acceptance tests?

    View Slide

  9. joind.in/talk/7c85b
    @SammyK #nephp17
    Integration tests?
    *Sort of

    View Slide

  10. joind.in/talk/7c85b
    @SammyK #nephp17
    Black-box
    Testing

    View Slide

  11. joind.in/talk/7c85b
    @SammyK #nephp17
    Testing
    …you could probably argue that it is
    Integration

    View Slide

  12. joind.in/talk/7c85b
    @SammyK #nephp17
    Testing
    …but it’s more like
    System

    View Slide

  13. joind.in/talk/7c85b
    @SammyK #nephp17
    Black-box
    Testing?
    What is this

    View Slide

  14. joind.in/talk/7c85b
    @SammyK #nephp17

    View Slide

  15. joind.in/talk/7c85b
    @SammyK #nephp17
    What is
    in there?

    View Slide

  16. joind.in/talk/7c85b
    @SammyK #nephp17
    Input
    Expected
    Output

    View Slide

  17. joind.in/talk/7c85b
    @SammyK #nephp17
    run-tests.php

    View Slide

  18. joind.in/talk/7c85b
    @SammyK #nephp17
    .phpt
    Expected
    Hello
    World
    Output
    Hello
    World

    View Slide

  19. joind.in/talk/7c85b
    @SammyK #nephp17
    run-tests.php
    Development

    View Slide

  20. joind.in/talk/7c85b
    @SammyK #nephp17
    run-tests.php

    View Slide

  21. View Slide

  22. View Slide

  23. STFU Operator

    View Slide

  24. joind.in/talk/7c85b
    @SammyK #nephp17

    View Slide

  25. joind.in/talk/7c85b
    @SammyK #nephp17

    View Slide

  26. joind.in/talk/7c85b
    @SammyK #nephp17
    run-tests.php
    Development

    View Slide

  27. Legacy
    Code

    View Slide

  28. joind.in/talk/7c85b
    @SammyK #nephp17
    Files

    View Slide

  29. joind.in/talk/7c85b
    @SammyK #nephp17
    /ext/{name}/tests
    /Zend/tests /sapi/{name}/tests
    Extensions
    SAPI’s
    Zend Engine
    /tests

    View Slide

  30. View Slide

  31. joind.in/talk/7c85b
    @SammyK #nephp17
    Input Output

    View Slide

  32. joind.in/talk/7c85b
    @SammyK #nephp17
    Running
    run-tests
    with

    View Slide

  33. joind.in/talk/7c85b
    @SammyK #nephp17
    1 source code
    Clone the

    View Slide

  34. joind.in/talk/7c85b
    @SammyK #nephp17
    $ git clone \
    [email protected]:php/php-src.git

    View Slide

  35. View Slide

  36. joind.in/talk/7c85b
    @SammyK #nephp17
    2source
    Compile from

    View Slide

  37. View Slide

  38. joind.in/talk/7c85b
    @SammyK #nephp17
    $ ./buildconf

    View Slide

  39. View Slide

  40. joind.in/talk/7c85b
    @SammyK #nephp17
    $ ./configure

    View Slide

  41. joind.in/talk/7c85b
    @SammyK #nephp17
    $ ./configure \
    --disable-all

    View Slide

  42. joind.in/talk/7c85b
    @SammyK #nephp17
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli

    View Slide

  43. View Slide

  44. joind.in/talk/7c85b
    @SammyK #nephp17
    $ make

    View Slide

  45. View Slide

  46. joind.in/talk/7c85b
    @SammyK #nephp17

    View Slide

  47. $ sapi/cli/php --version

    View Slide

  48. joind.in/talk/7c85b
    @SammyK #nephp17
    3tests
    Run the

    View Slide

  49. joind.in/talk/7c85b
    @SammyK #nephp17
    Two Ways
    to run
    tests

    View Slide

  50. joind.in/talk/7c85b
    @SammyK #nephp17
    run-tests.php
    Run the with…
    directly

    View Slide

  51. View Slide

  52. joind.in/talk/7c85b
    @SammyK #nephp17
    Run the with…
    the Makefile
    Recommended!

    View Slide

  53. Recommended!

    View Slide

  54. Recommended!

    View Slide

  55. joind.in/talk/7c85b
    @SammyK #nephp17
    Let’s
    Make a test!

    View Slide

  56. joind.in/talk/7c85b
    @SammyK #nephp17
    test-echo.phpt
    Save as…
    php
    Run from

    View Slide

  57. View Slide

  58. joind.in/talk/7c85b
    @SammyK #nephp17
    Test it!
    $ make test TESTS=test-echo.phpt

    View Slide

  59. View Slide

  60. joind.in/talk/7c85b
    @SammyK #nephp17
    Fail
    When tests

    View Slide

  61. joind.in/talk/7c85b
    @SammyK #nephp17
    Input Output
    Expected

    View Slide

  62. View Slide

  63. View Slide

  64. joind.in/talk/7c85b
    @SammyK #nephp17
    Output
    Examining the
    !==

    View Slide

  65. Files
    test-echo.out
    test-echo.exp
    test-echo.diff
    test-echo.log
    Output

    View Slide

  66. joind.in/talk/7c85b
    @SammyK #nephp17
    $ less test-echo.out
    Hello world
    Script output

    View Slide

  67. joind.in/talk/7c85b
    @SammyK #nephp17
    $ less test-echo.exp
    Hello Jane!
    --EXPECT--

    View Slide

  68. joind.in/talk/7c85b
    @SammyK #nephp17
    $ less test-echo.diff
    001+ Hello world
    001- Hello Jane!

    View Slide

  69. joind.in/talk/7c85b
    @SammyK #nephp17
    $ less test-echo.log
    ---- EXPECTED OUTPUT
    Hello Jane!
    ---- ACTUAL OUTPUT
    Hello world
    ---- FAILED

    View Slide

  70. Files
    test-echo.php
    test-echo.sh
    Executable

    View Slide

  71. joind.in/talk/7c85b
    @SammyK #nephp17
    test-echo.php
    --FILE--

    View Slide

  72. View Slide

  73. test-echo.sh

    View Slide

  74. joind.in/talk/7c85b
    @SammyK #nephp17
    Example
    A more complex

    View Slide

  75. View Slide

  76. View Slide

  77. View Slide

  78. joind.in/talk/7c85b
    @SammyK #nephp17
    Fails
    So say it

    View Slide

  79. View Slide

  80. View Slide

  81. $ less test-dt.diff
    001+ Boy you know I'm
    from Europe/Oslo
    001- Boy you know I'm
    from America/Chicago

    View Slide

  82. test-dt.php
    Executable!

    View Slide

  83. View Slide

  84. test-dt.sh

    View Slide

  85. View Slide

  86. View Slide

  87. joind.in/talk/7c85b
    @SammyK #nephp17
    Debugging
    $ gdb …
    failed tests with

    View Slide

  88. View Slide

  89. View Slide

  90. joind.in/talk/7c85b
    @SammyK #nephp17
    Conventions
    Naming

    View Slide

  91. joind.in/talk/7c85b
    @SammyK #nephp17
    {function-name}_basic.phpt
    {function-name}_error.phpt
    {function-name}_variation.phpt
    {ext-name}{number}.phpt
    bug{bug-id}.phpt
    http://qa.php.net/write-test.php#naming-conventions

    View Slide

  92. joind.in/talk/7c85b
    @SammyK #nephp17
    Contribute
    How-to

    View Slide

  93. joind.in/talk/7c85b
    @SammyK #nephp17
    Tests!
    Add some

    View Slide

  94. gcov.php.net

    View Slide

  95. PHP_FUNCTION

    View Slide

  96. readline.c

    View Slide

  97. joind.in/talk/7c85b
    @SammyK #nephp17
    ?
    readline()

    View Slide

  98. joind.in/talk/7c85b
    @SammyK #nephp17
    php.net/readline

    View Slide

  99. View Slide

  100. joind.in/talk/7c85b
    @SammyK #nephp17
    Let’s
    Make a test!

    View Slide

  101. joind.in/talk/7c85b
    @SammyK #nephp17

    View Slide

  102. joind.in/talk/7c85b
    @SammyK #nephp17
    for ext/readline
    Find the
    ext/readline/tests

    View Slide

  103. View Slide

  104. joind.in/talk/7c85b
    @SammyK #nephp17
    Save as…
    ext/readline/tests/readline_basic.phpt

    View Slide

  105. View Slide

  106. joind.in/talk/7c85b
    @SammyK #nephp17
    $ less ext/readline/tests/readline_basic.out
    Fatal error: Uncaught Error: Call
    to undefined function readline() in
    /Users/SammyK/Developer/php-src/
    ext/readline/tests/
    readline_basic.php:2

    View Slide

  107. joind.in/talk/7c85b
    @SammyK #nephp17
    php.net/readline

    View Slide

  108. joind.in/talk/7c85b
    @SammyK #nephp17

    View Slide

  109. View Slide

  110. joind.in/talk/7c85b
    @SammyK #nephp17
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli

    View Slide

  111. joind.in/talk/7c85b
    @SammyK #nephp17
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli \
    --with-readline

    View Slide

  112. joind.in/talk/7c85b
    @SammyK #nephp17
    OR

    View Slide

  113. joind.in/talk/7c85b
    @SammyK #nephp17
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli \
    --with-libedit

    View Slide

  114. View Slide

  115. joind.in/talk/7c85b
    @SammyK #nephp17
    $ less ext/readline/tests/readline_basic.out
    You said:

    View Slide

  116. joind.in/talk/7c85b
    @SammyK #nephp17
    readline_basic.sh
    Let’s try

    View Slide

  117. View Slide

  118. joind.in/talk/7c85b
    @SammyK #nephp17

    View Slide

  119. View Slide

  120. Now send a
    pull request

    View Slide

  121. joind.in/talk/7c85b
    @SammyK #nephp17
    internals contributor
    You’re an

    View Slide

  122. joind.in/talk/7c85b
    @SammyK #nephp17
    phptestfest.org

    View Slide

  123. joind.in/talk/7c85b
    @SammyK #nephp17
    sammyk.me
    Six-part
    screencast
    series

    View Slide

  124. Legacy
    Code

    View Slide

  125. refactor
    github.com/SammyK/run-tests
    run-tests

    View Slide

  126. refactor
    run-tests

    View Slide

  127. wishlist
    * Concurrency
    * Unit tests
    * Colored output
    run-tests

    View Slide

  128. joind.in/talk/7c85b
    @SammyK #nephp17
    Code
    with me today!
    Kent Room
    after lunch @ 1pm
    (unconference room)

    View Slide

  129. THANKS!
    SAMMY KAYE POWERS
    @SammyK
    SammyK.me
    Host of @PHPRoundtable
    @ChiPHPUG
    West Coast Swing
    /talk/7c85b
    I have
    stickers!

    View Slide