Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Writing Tests For PHP Source - ZendCon 2017

Writing Tests For PHP Source - ZendCon 2017

Talk given at ZendCon 2017

As you might have already heard from a grumpy individual, adding more tests to your codebase is crucial to its stability over time. This session isn't about adding tests to your PHP codebase, but adding tests to the PHP language itself. And the best part is, all the tests are written in PHP so you don't even have to dust off that old C book from college. In this session you'll see how to find untested parts of the PHP source code, how to write a test for the untested feature, and how to submit your tests to PHP internals. Not only will you be making PHP more stable and reliable, but there's a good chance the tests you write will give you a deeper understanding of PHP; knowledge you can use to improve your own codebases. Not to mention you'll be able to call yourself an internals contributor. Come on in! The water's fine!

Sammy Kaye Powers

October 26, 2017
Tweet

More Decks by Sammy Kaye Powers

Other Decks in Programming

Transcript

  1. joind.in/talk/c6253
    @SammyK #ZendCon2017
    For PHP Source
    Writing Tests
    Sammy Kaye Powers
    2017-10-26

    View Slide

  2. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Slides
    Get the
    joind.in/talk/c6253

    View Slide

  3. joind.in/talk/c6253
    @SammyK #ZendCon2017
    PHP Internals
    This is a
    talk

    View Slide

  4. Write more
    TESTS!

    View Slide

  5. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Unit tests?

    View Slide

  6. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View Slide

  7. joind.in/talk/c6253
    @SammyK #ZendCon2017
    @dhh
    “TDD is dead.”
    http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html

    View Slide

  8. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Acceptance tests?

    View Slide

  9. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Integration tests?
    *Sort of

    View Slide

  10. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Black-box
    Testing

    View Slide

  11. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Testing
    …you could probably argue that it is
    Integration

    View Slide

  12. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Testing
    …but it’s more like
    System

    View Slide

  13. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Black-box
    Testing?
    What is this

    View Slide

  14. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View Slide

  15. joind.in/talk/c6253
    @SammyK #ZendCon2017
    What is
    in there?

    View Slide

  16. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Input
    Expected
    Output

    View Slide

  17. joind.in/talk/c6253
    @SammyK #ZendCon2017
    run-tests.php

    View Slide

  18. joind.in/talk/c6253
    @SammyK #ZendCon2017
    .phpt
    Expected
    Hello
    World
    Output
    Hello
    World

    View Slide

  19. joind.in/talk/c6253
    @SammyK #ZendCon2017
    run-tests.php
    Development

    View Slide

  20. joind.in/talk/c6253
    @SammyK #ZendCon2017
    run-tests.php

    View Slide

  21. View Slide

  22. View Slide

  23. STFU Operator

    View Slide

  24. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View Slide

  25. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View Slide

  26. joind.in/talk/c6253
    @SammyK #ZendCon2017
    run-tests.php
    Development

    View Slide

  27. Legacy
    Code

    View Slide

  28. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Files

    View Slide

  29. joind.in/talk/c6253
    @SammyK #ZendCon2017
    /ext/{name}/tests
    /Zend/tests /sapi/{name}/tests
    Extensions
    SAPI’s
    Zend Engine
    /tests

    View Slide

  30. View Slide

  31. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Input Output

    View Slide

  32. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Running
    run-tests
    with

    View Slide

  33. joind.in/talk/c6253
    @SammyK #ZendCon2017
    1 source code
    Clone the

    View Slide

  34. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ git clone \
    [email protected]:php/php-src.git

    View Slide

  35. View Slide

  36. joind.in/talk/c6253
    @SammyK #ZendCon2017
    2source
    Compile from

    View Slide

  37. View Slide

  38. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ ./buildconf

    View Slide

  39. View Slide

  40. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ ./configure

    View Slide

  41. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ ./configure \
    --disable-all

    View Slide

  42. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli

    View Slide

  43. View Slide

  44. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ make

    View Slide

  45. View Slide

  46. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View Slide

  47. $ sapi/cli/php --version

    View Slide

  48. joind.in/talk/c6253
    @SammyK #ZendCon2017
    3tests
    Run the

    View Slide

  49. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Two Ways
    to run
    tests

    View Slide

  50. joind.in/talk/c6253
    @SammyK #ZendCon2017
    run-tests.php
    Run the with…
    directly

    View Slide

  51. View Slide

  52. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Run the with…
    the Makefile
    Recommended!

    View Slide

  53. Recommended!

    View Slide

  54. Recommended!

    View Slide

  55. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Let’s
    Make a test!

    View Slide

  56. joind.in/talk/c6253
    @SammyK #ZendCon2017
    test-echo.phpt
    Save as…
    php
    Run from

    View Slide

  57. View Slide

  58. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Test it!
    $ make test TESTS=test-echo.phpt

    View Slide

  59. View Slide

  60. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Fail
    When tests

    View Slide

  61. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Input Output
    Expected

    View Slide

  62. View Slide

  63. View Slide

  64. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Output
    Examining the
    !==

    View Slide

  65. Files
    test-echo.out
    test-echo.exp
    test-echo.diff
    test-echo.log
    Output

    View Slide

  66. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ less test-echo.out
    Hello world
    Script output

    View Slide

  67. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ less test-echo.exp
    Hello Jane!
    --EXPECT--

    View Slide

  68. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ less test-echo.diff
    001+ Hello world
    001- Hello Jane!

    View Slide

  69. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ less test-echo.log
    ---- EXPECTED OUTPUT
    Hello Jane!
    ---- ACTUAL OUTPUT
    Hello world
    ---- FAILED

    View Slide

  70. Files
    test-echo.php
    test-echo.sh
    Executable

    View Slide

  71. joind.in/talk/c6253
    @SammyK #ZendCon2017
    test-echo.php
    --FILE--

    View Slide

  72. View Slide

  73. test-echo.sh

    View Slide

  74. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Example
    A more complex

    View Slide

  75. View Slide

  76. View Slide

  77. View Slide

  78. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Fails
    So say it

    View Slide

  79. View Slide

  80. View Slide

  81. $ less test-dt.diff
    001+ Boy you know I'm
    from Europe/Oslo
    001- Boy you know I'm
    from America/Chicago

    View Slide

  82. test-dt.php
    Executable!

    View Slide

  83. View Slide

  84. test-dt.sh

    View Slide

  85. View Slide

  86. View Slide

  87. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Debugging
    $ gdb …
    failed tests with

    View Slide

  88. View Slide

  89. View Slide

  90. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Conventions
    Naming

    View Slide

  91. joind.in/talk/c6253
    @SammyK #ZendCon2017
    {function-name}_basic.phpt
    {function-name}_error.phpt
    {function-name}_variation.phpt
    {ext-name}{number}.phpt
    bug{bug-id}.phpt
    http://qa.php.net/write-test.php#naming-conventions

    View Slide

  92. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Contribute
    How-to

    View Slide

  93. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Tests!
    Add some

    View Slide

  94. gcov.php.net

    View Slide

  95. PHP_FUNCTION

    View Slide

  96. readline.c

    View Slide

  97. joind.in/talk/c6253
    @SammyK #ZendCon2017
    ?
    readline()

    View Slide

  98. joind.in/talk/c6253
    @SammyK #ZendCon2017
    php.net/readline

    View Slide

  99. View Slide

  100. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Let’s
    Make a test!

    View Slide

  101. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View Slide

  102. joind.in/talk/c6253
    @SammyK #ZendCon2017
    for ext/readline
    Find the
    ext/readline/tests

    View Slide

  103. View Slide

  104. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Save as…
    ext/readline/tests/readline_basic.phpt

    View Slide

  105. View Slide

  106. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ less ext/readline/tests/readline_basic.out
    Fatal error: Uncaught Error: Call
    to undefined function readline() in
    /Users/SammyK/Developer/php-src/
    ext/readline/tests/
    readline_basic.php:2

    View Slide

  107. joind.in/talk/c6253
    @SammyK #ZendCon2017
    php.net/readline

    View Slide

  108. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View Slide

  109. View Slide

  110. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli

    View Slide

  111. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli \
    --with-readline

    View Slide

  112. joind.in/talk/c6253
    @SammyK #ZendCon2017
    OR

    View Slide

  113. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ ./configure \
    --enable-maintainer-zts \
    --enable-debug \
    --enable-cli \
    --with-libedit

    View Slide

  114. View Slide

  115. joind.in/talk/c6253
    @SammyK #ZendCon2017
    $ less ext/readline/tests/readline_basic.out
    You said:

    View Slide

  116. joind.in/talk/c6253
    @SammyK #ZendCon2017
    readline_basic.sh
    Let’s try

    View Slide

  117. View Slide

  118. joind.in/talk/c6253
    @SammyK #ZendCon2017

    View Slide

  119. View Slide

  120. Now send a
    pull request

    View Slide

  121. joind.in/talk/c6253
    @SammyK #ZendCon2017
    internals contributor
    You’re an

    View Slide

  122. joind.in/talk/c6253
    @SammyK #ZendCon2017
    phptestfest.org

    View Slide

  123. joind.in/talk/c6253
    @SammyK #ZendCon2017
    sammyk.me
    Six-part
    screencast
    series

    View Slide

  124. Legacy
    Code

    View Slide

  125. refactor
    github.com/SammyK/run-tests
    run-tests

    View Slide

  126. refactor
    run-tests

    View Slide

  127. wishlist
    * Concurrency
    * Unit tests
    * Colored output
    run-tests

    View Slide

  128. joind.in/talk/c6253
    @SammyK #ZendCon2017
    Code
    with me today!
    Uncon Room
    @ 9:15am
    (by Paradise tower)

    View Slide

  129. Sammy Kaye Powers
    Thanks!
    /talk/c6253
    @SammyK
    SammyK.me
    Host of @PHPRoundtable
    Co-admin of @ChiPHPUG
    I ❤ West Coast Swing

    View Slide