Writing Tests For PHP Source - Northeast PHP 2017

Writing Tests For PHP Source - Northeast PHP 2017

Talk given at Northeast PHP on August 8, 2017.

As you might have already heard from a grumpy individual, adding more tests to your codebase is crucial to its stability over time. This talk isn't about adding tests to your PHP codebase, but adding tests to the PHP language itself. And the best part is, all the tests are written in PHP so you don't even have to dust off that old C book from college.

In this talk I will show you how to find untested parts of the PHP source code, how to write a test for the untested feature and how to submit your tests to PHP internals. Not only will you be making PHP more stable and reliable, but there's a good chance the tests you write will give you a deeper understanding of PHP; knowledge you can use to improve your own codebases. Not to mention you'll be able to call yourself an internals contributor. Come on in! The water's fine!

8c090cc1ccd623a146ddd9159b1bf7e2?s=128

Sammy Kaye Powers

August 11, 2017
Tweet

Transcript

  1. joind.in/talk/7c85b @SammyK #nephp17 For PHP Source Writing Tests Sammy Kaye

    Powers 2017-08-11
  2. joind.in/talk/7c85b @SammyK #nephp17 Slides Get the joind.in/talk/7c85b

  3. joind.in/talk/7c85b @SammyK #nephp17 PHP Internals This is a talk

  4. Write more TESTS!

  5. joind.in/talk/7c85b @SammyK #nephp17 Unit tests?

  6. joind.in/talk/7c85b @SammyK #nephp17

  7. joind.in/talk/7c85b @SammyK #nephp17 @dhh “TDD is dead.” http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html

  8. joind.in/talk/7c85b @SammyK #nephp17 Acceptance tests?

  9. joind.in/talk/7c85b @SammyK #nephp17 Integration tests? *Sort of

  10. joind.in/talk/7c85b @SammyK #nephp17 Black-box Testing

  11. joind.in/talk/7c85b @SammyK #nephp17 Testing …you could probably argue that it

    is Integration
  12. joind.in/talk/7c85b @SammyK #nephp17 Testing …but it’s more like System

  13. joind.in/talk/7c85b @SammyK #nephp17 Black-box Testing? What is this

  14. joind.in/talk/7c85b @SammyK #nephp17

  15. joind.in/talk/7c85b @SammyK #nephp17 What is in there?

  16. joind.in/talk/7c85b @SammyK #nephp17 Input Expected Output ✓

  17. joind.in/talk/7c85b @SammyK #nephp17 run-tests.php

  18. joind.in/talk/7c85b @SammyK #nephp17 .phpt Expected Hello World Output Hello World

  19. joind.in/talk/7c85b @SammyK #nephp17 run-tests.php Development

  20. joind.in/talk/7c85b @SammyK #nephp17 run-tests.php

  21. None
  22. None
  23. STFU Operator

  24. joind.in/talk/7c85b @SammyK #nephp17

  25. joind.in/talk/7c85b @SammyK #nephp17

  26. joind.in/talk/7c85b @SammyK #nephp17 run-tests.php Development

  27. Legacy Code

  28. joind.in/talk/7c85b @SammyK #nephp17 Files

  29. joind.in/talk/7c85b @SammyK #nephp17 /ext/{name}/tests /Zend/tests /sapi/{name}/tests Extensions SAPI’s Zend Engine

    /tests
  30. None
  31. joind.in/talk/7c85b @SammyK #nephp17 Input Output

  32. joind.in/talk/7c85b @SammyK #nephp17 Running run-tests with

  33. joind.in/talk/7c85b @SammyK #nephp17 1 source code Clone the

  34. joind.in/talk/7c85b @SammyK #nephp17 $ git clone \ git@github.com:php/php-src.git

  35. None
  36. joind.in/talk/7c85b @SammyK #nephp17 2source Compile from

  37. None
  38. joind.in/talk/7c85b @SammyK #nephp17 $ ./buildconf

  39. None
  40. joind.in/talk/7c85b @SammyK #nephp17 $ ./configure

  41. joind.in/talk/7c85b @SammyK #nephp17 $ ./configure \ --disable-all

  42. joind.in/talk/7c85b @SammyK #nephp17 $ ./configure \ --enable-maintainer-zts \ --enable-debug \

    --enable-cli
  43. None
  44. joind.in/talk/7c85b @SammyK #nephp17 $ make

  45. None
  46. joind.in/talk/7c85b @SammyK #nephp17

  47. $ sapi/cli/php --version

  48. joind.in/talk/7c85b @SammyK #nephp17 3tests Run the

  49. joind.in/talk/7c85b @SammyK #nephp17 Two Ways to run tests

  50. joind.in/talk/7c85b @SammyK #nephp17 run-tests.php Run the with… directly

  51. None
  52. joind.in/talk/7c85b @SammyK #nephp17 Run the with… the Makefile Recommended!

  53. Recommended!

  54. Recommended!

  55. joind.in/talk/7c85b @SammyK #nephp17 Let’s Make a test!

  56. joind.in/talk/7c85b @SammyK #nephp17 test-echo.phpt Save as… php Run from

  57. None
  58. joind.in/talk/7c85b @SammyK #nephp17 Test it! $ make test TESTS=test-echo.phpt

  59. None
  60. joind.in/talk/7c85b @SammyK #nephp17 Fail When tests

  61. joind.in/talk/7c85b @SammyK #nephp17 Input Output Expected

  62. None
  63. None
  64. joind.in/talk/7c85b @SammyK #nephp17 Output Examining the !==

  65. Files test-echo.out test-echo.exp test-echo.diff test-echo.log Output

  66. joind.in/talk/7c85b @SammyK #nephp17 $ less test-echo.out Hello world Script output

  67. joind.in/talk/7c85b @SammyK #nephp17 $ less test-echo.exp Hello Jane! --EXPECT--

  68. joind.in/talk/7c85b @SammyK #nephp17 $ less test-echo.diff 001+ Hello world 001-

    Hello Jane!
  69. joind.in/talk/7c85b @SammyK #nephp17 $ less test-echo.log ---- EXPECTED OUTPUT Hello

    Jane! ---- ACTUAL OUTPUT Hello world ---- FAILED
  70. Files test-echo.php test-echo.sh Executable

  71. joind.in/talk/7c85b @SammyK #nephp17 test-echo.php --FILE--

  72. None
  73. test-echo.sh

  74. joind.in/talk/7c85b @SammyK #nephp17 Example A more complex

  75. None
  76. None
  77. None
  78. joind.in/talk/7c85b @SammyK #nephp17 Fails So say it

  79. None
  80. None
  81. $ less test-dt.diff 001+ Boy you know I'm from Europe/Oslo

    001- Boy you know I'm from America/Chicago
  82. test-dt.php Executable!

  83. None
  84. test-dt.sh

  85. None
  86. None
  87. joind.in/talk/7c85b @SammyK #nephp17 Debugging $ gdb … failed tests with

  88. None
  89. None
  90. joind.in/talk/7c85b @SammyK #nephp17 Conventions Naming

  91. joind.in/talk/7c85b @SammyK #nephp17 {function-name}_basic.phpt {function-name}_error.phpt {function-name}_variation.phpt {ext-name}{number}.phpt bug{bug-id}.phpt http://qa.php.net/write-test.php#naming-conventions

  92. joind.in/talk/7c85b @SammyK #nephp17 Contribute How-to

  93. joind.in/talk/7c85b @SammyK #nephp17 Tests! Add some

  94. gcov.php.net

  95. PHP_FUNCTION

  96. readline.c

  97. joind.in/talk/7c85b @SammyK #nephp17 ? readline()

  98. joind.in/talk/7c85b @SammyK #nephp17 php.net/readline

  99. None
  100. joind.in/talk/7c85b @SammyK #nephp17 Let’s Make a test!

  101. joind.in/talk/7c85b @SammyK #nephp17

  102. joind.in/talk/7c85b @SammyK #nephp17 for ext/readline Find the ext/readline/tests

  103. None
  104. joind.in/talk/7c85b @SammyK #nephp17 Save as… ext/readline/tests/readline_basic.phpt

  105. None
  106. joind.in/talk/7c85b @SammyK #nephp17 $ less ext/readline/tests/readline_basic.out Fatal error: Uncaught Error:

    Call to undefined function readline() in /Users/SammyK/Developer/php-src/ ext/readline/tests/ readline_basic.php:2
  107. joind.in/talk/7c85b @SammyK #nephp17 php.net/readline

  108. joind.in/talk/7c85b @SammyK #nephp17

  109. None
  110. joind.in/talk/7c85b @SammyK #nephp17 $ ./configure \ --enable-maintainer-zts \ --enable-debug \

    --enable-cli
  111. joind.in/talk/7c85b @SammyK #nephp17 $ ./configure \ --enable-maintainer-zts \ --enable-debug \

    --enable-cli \ --with-readline
  112. joind.in/talk/7c85b @SammyK #nephp17 OR

  113. joind.in/talk/7c85b @SammyK #nephp17 $ ./configure \ --enable-maintainer-zts \ --enable-debug \

    --enable-cli \ --with-libedit
  114. None
  115. joind.in/talk/7c85b @SammyK #nephp17 $ less ext/readline/tests/readline_basic.out You said:

  116. joind.in/talk/7c85b @SammyK #nephp17 readline_basic.sh Let’s try

  117. None
  118. joind.in/talk/7c85b @SammyK #nephp17

  119. None
  120. Now send a pull request

  121. joind.in/talk/7c85b @SammyK #nephp17 internals contributor You’re an

  122. joind.in/talk/7c85b @SammyK #nephp17 phptestfest.org

  123. joind.in/talk/7c85b @SammyK #nephp17 sammyk.me Six-part screencast series

  124. Legacy Code

  125. refactor github.com/SammyK/run-tests run-tests

  126. refactor run-tests

  127. wishlist * Concurrency * Unit tests * Colored output run-tests

  128. joind.in/talk/7c85b @SammyK #nephp17 Code with me today! Kent Room after

    lunch @ 1pm (unconference room)
  129. THANKS! SAMMY KAYE POWERS @SammyK SammyK.me Host of @PHPRoundtable @ChiPHPUG

    West Coast Swing /talk/7c85b I have stickers!