Writing Tests For PHP Source - ZendCon 2017

Writing Tests For PHP Source - ZendCon 2017

Talk given at ZendCon 2017

As you might have already heard from a grumpy individual, adding more tests to your codebase is crucial to its stability over time. This session isn't about adding tests to your PHP codebase, but adding tests to the PHP language itself. And the best part is, all the tests are written in PHP so you don't even have to dust off that old C book from college. In this session you'll see how to find untested parts of the PHP source code, how to write a test for the untested feature, and how to submit your tests to PHP internals. Not only will you be making PHP more stable and reliable, but there's a good chance the tests you write will give you a deeper understanding of PHP; knowledge you can use to improve your own codebases. Not to mention you'll be able to call yourself an internals contributor. Come on in! The water's fine!


Sammy Kaye Powers

October 26, 2017


  1. joind.in/talk/c6253 @SammyK #ZendCon2017 For PHP Source Writing Tests Sammy Kaye

    Powers 2017-10-26
  2. joind.in/talk/c6253 @SammyK #ZendCon2017 Slides Get the joind.in/talk/c6253

  3. joind.in/talk/c6253 @SammyK #ZendCon2017 PHP Internals This is a talk

  4. Write more TESTS!

  5. joind.in/talk/c6253 @SammyK #ZendCon2017 Unit tests?

  6. joind.in/talk/c6253 @SammyK #ZendCon2017

  7. joind.in/talk/c6253 @SammyK #ZendCon2017 @dhh “TDD is dead.” http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html

  8. joind.in/talk/c6253 @SammyK #ZendCon2017 Acceptance tests?

  9. joind.in/talk/c6253 @SammyK #ZendCon2017 Integration tests? *Sort of

  10. joind.in/talk/c6253 @SammyK #ZendCon2017 Black-box Testing

  11. joind.in/talk/c6253 @SammyK #ZendCon2017 Testing …you could probably argue that it

    is Integration
  12. joind.in/talk/c6253 @SammyK #ZendCon2017 Testing …but it’s more like System

  13. joind.in/talk/c6253 @SammyK #ZendCon2017 Black-box Testing? What is this

  14. joind.in/talk/c6253 @SammyK #ZendCon2017

  15. joind.in/talk/c6253 @SammyK #ZendCon2017 What is in there?

  16. joind.in/talk/c6253 @SammyK #ZendCon2017 Input Expected Output ✓

  17. joind.in/talk/c6253 @SammyK #ZendCon2017 run-tests.php

  18. joind.in/talk/c6253 @SammyK #ZendCon2017 .phpt Expected Hello World Output Hello World

  19. joind.in/talk/c6253 @SammyK #ZendCon2017 run-tests.php Development

  20. joind.in/talk/c6253 @SammyK #ZendCon2017 run-tests.php

  21. None
  22. None
  23. STFU Operator

  24. joind.in/talk/c6253 @SammyK #ZendCon2017

  25. joind.in/talk/c6253 @SammyK #ZendCon2017

  26. joind.in/talk/c6253 @SammyK #ZendCon2017 run-tests.php Development

  27. Legacy Code

  28. joind.in/talk/c6253 @SammyK #ZendCon2017 Files

  29. joind.in/talk/c6253 @SammyK #ZendCon2017 /ext/{name}/tests /Zend/tests /sapi/{name}/tests Extensions SAPI’s Zend Engine

  30. None
  31. joind.in/talk/c6253 @SammyK #ZendCon2017 Input Output

  32. joind.in/talk/c6253 @SammyK #ZendCon2017 Running run-tests with

  33. joind.in/talk/c6253 @SammyK #ZendCon2017 1 source code Clone the

  34. joind.in/talk/c6253 @SammyK #ZendCon2017 $ git clone \ git@github.com:php/php-src.git

  35. None
  36. joind.in/talk/c6253 @SammyK #ZendCon2017 2source Compile from

  37. None
  38. joind.in/talk/c6253 @SammyK #ZendCon2017 $ ./buildconf

  39. None
  40. joind.in/talk/c6253 @SammyK #ZendCon2017 $ ./configure

  41. joind.in/talk/c6253 @SammyK #ZendCon2017 $ ./configure \ --disable-all

  42. joind.in/talk/c6253 @SammyK #ZendCon2017 $ ./configure \ --enable-maintainer-zts \ --enable-debug \

  43. None
  44. joind.in/talk/c6253 @SammyK #ZendCon2017 $ make

  45. None
  46. joind.in/talk/c6253 @SammyK #ZendCon2017

  47. $ sapi/cli/php --version

  48. joind.in/talk/c6253 @SammyK #ZendCon2017 3tests Run the

  49. joind.in/talk/c6253 @SammyK #ZendCon2017 Two Ways to run tests

  50. joind.in/talk/c6253 @SammyK #ZendCon2017 run-tests.php Run the with… directly

  51. None
  52. joind.in/talk/c6253 @SammyK #ZendCon2017 Run the with… the Makefile Recommended!

  53. Recommended!

  54. Recommended!

  55. joind.in/talk/c6253 @SammyK #ZendCon2017 Let’s Make a test!

  56. joind.in/talk/c6253 @SammyK #ZendCon2017 test-echo.phpt Save as… php Run from

  57. None
  58. joind.in/talk/c6253 @SammyK #ZendCon2017 Test it! $ make test TESTS=test-echo.phpt

  59. None
  60. joind.in/talk/c6253 @SammyK #ZendCon2017 Fail When tests

  61. joind.in/talk/c6253 @SammyK #ZendCon2017 Input Output Expected

  62. None
  63. None
  64. joind.in/talk/c6253 @SammyK #ZendCon2017 Output Examining the !==

  65. Files test-echo.out test-echo.exp test-echo.diff test-echo.log Output

  66. joind.in/talk/c6253 @SammyK #ZendCon2017 $ less test-echo.out Hello world Script output

  67. joind.in/talk/c6253 @SammyK #ZendCon2017 $ less test-echo.exp Hello Jane! --EXPECT--

  68. joind.in/talk/c6253 @SammyK #ZendCon2017 $ less test-echo.diff 001+ Hello world 001-

    Hello Jane!
  69. joind.in/talk/c6253 @SammyK #ZendCon2017 $ less test-echo.log ---- EXPECTED OUTPUT Hello

    Jane! ---- ACTUAL OUTPUT Hello world ---- FAILED
  70. Files test-echo.php test-echo.sh Executable

  71. joind.in/talk/c6253 @SammyK #ZendCon2017 test-echo.php --FILE--

  72. None
  73. test-echo.sh

  74. joind.in/talk/c6253 @SammyK #ZendCon2017 Example A more complex

  75. None
  76. None
  77. None
  78. joind.in/talk/c6253 @SammyK #ZendCon2017 Fails So say it

  79. None
  80. None
  81. $ less test-dt.diff 001+ Boy you know I'm from Europe/Oslo

    001- Boy you know I'm from America/Chicago
  82. test-dt.php Executable!

  83. None
  84. test-dt.sh

  85. None
  86. None
  87. joind.in/talk/c6253 @SammyK #ZendCon2017 Debugging $ gdb … failed tests with

  88. None
  89. None
  90. joind.in/talk/c6253 @SammyK #ZendCon2017 Conventions Naming

  91. joind.in/talk/c6253 @SammyK #ZendCon2017 {function-name}_basic.phpt {function-name}_error.phpt {function-name}_variation.phpt {ext-name}{number}.phpt bug{bug-id}.phpt http://qa.php.net/write-test.php#naming-conventions

  92. joind.in/talk/c6253 @SammyK #ZendCon2017 Contribute How-to

  93. joind.in/talk/c6253 @SammyK #ZendCon2017 Tests! Add some

  94. gcov.php.net


  96. readline.c

  97. joind.in/talk/c6253 @SammyK #ZendCon2017 ? readline()

  98. joind.in/talk/c6253 @SammyK #ZendCon2017 php.net/readline

  99. None
  100. joind.in/talk/c6253 @SammyK #ZendCon2017 Let’s Make a test!

  101. joind.in/talk/c6253 @SammyK #ZendCon2017

  102. joind.in/talk/c6253 @SammyK #ZendCon2017 for ext/readline Find the ext/readline/tests

  103. None
  104. joind.in/talk/c6253 @SammyK #ZendCon2017 Save as… ext/readline/tests/readline_basic.phpt

  105. None
  106. joind.in/talk/c6253 @SammyK #ZendCon2017 $ less ext/readline/tests/readline_basic.out Fatal error: Uncaught Error:

    Call to undefined function readline() in /Users/SammyK/Developer/php-src/ ext/readline/tests/ readline_basic.php:2
  107. joind.in/talk/c6253 @SammyK #ZendCon2017 php.net/readline

  108. joind.in/talk/c6253 @SammyK #ZendCon2017

  109. None
  110. joind.in/talk/c6253 @SammyK #ZendCon2017 $ ./configure \ --enable-maintainer-zts \ --enable-debug \

  111. joind.in/talk/c6253 @SammyK #ZendCon2017 $ ./configure \ --enable-maintainer-zts \ --enable-debug \

    --enable-cli \ --with-readline
  112. joind.in/talk/c6253 @SammyK #ZendCon2017 OR

  113. joind.in/talk/c6253 @SammyK #ZendCon2017 $ ./configure \ --enable-maintainer-zts \ --enable-debug \

    --enable-cli \ --with-libedit
  114. None
  115. joind.in/talk/c6253 @SammyK #ZendCon2017 $ less ext/readline/tests/readline_basic.out You said:

  116. joind.in/talk/c6253 @SammyK #ZendCon2017 readline_basic.sh Let’s try

  117. None
  118. joind.in/talk/c6253 @SammyK #ZendCon2017

  119. None
  120. Now send a pull request

  121. joind.in/talk/c6253 @SammyK #ZendCon2017 internals contributor You’re an

  122. joind.in/talk/c6253 @SammyK #ZendCon2017 phptestfest.org

  123. joind.in/talk/c6253 @SammyK #ZendCon2017 sammyk.me Six-part screencast series

  124. Legacy Code

  125. refactor github.com/SammyK/run-tests run-tests

  126. refactor run-tests

  127. wishlist * Concurrency * Unit tests * Colored output run-tests

  128. joind.in/talk/c6253 @SammyK #ZendCon2017 Code with me today! Uncon Room @

    9:15am (by Paradise tower)
  129. Sammy Kaye Powers Thanks! /talk/c6253 @SammyK SammyK.me Host of @PHPRoundtable

    Co-admin of @ChiPHPUG I ❤ West Coast Swing