Writing Tests For PHP Source - Northeast PHP 2017

Talk given at Northeast PHP on August 8, 2017.

As you might have already heard from a grumpy individual, adding more tests to your codebase is crucial to its stability over time. This talk isn't about adding tests to your PHP codebase, but adding tests to the PHP language itself. And the best part is, all the tests are written in PHP so you don't even have to dust off that old C book from college.

In this talk I will show you how to find untested parts of the PHP source code, how to write a test for the untested feature and how to submit your tests to PHP internals. Not only will you be making PHP more stable and reliable, but there's a good chance the tests you write will give you a deeper understanding of PHP; knowledge you can use to improve your own codebases. Not to mention you'll be able to call yourself an internals contributor. Come on in! The water's fine!

Transcript

1. joind.in/talk/7c85b @SammyK #nephp17 For PHP Source Writing Tests Sammy Kaye Powers 2017-08-11
2. joind.in/talk/7c85b @SammyK #nephp17 Slides Get the joind.in/talk/7c85b
3. joind.in/talk/7c85b @SammyK #nephp17 PHP Internals This is a talk
4. Write more TESTS!
5. joind.in/talk/7c85b @SammyK #nephp17 Unit tests?
6. joind.in/talk/7c85b @SammyK #nephp17
7. joind.in/talk/7c85b @SammyK #nephp17 @dhh “TDD is dead.” http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html
8. joind.in/talk/7c85b @SammyK #nephp17 Acceptance tests?
9. joind.in/talk/7c85b @SammyK #nephp17 Integration tests? *Sort of
10. joind.in/talk/7c85b @SammyK #nephp17 Black-box Testing
11. joind.in/talk/7c85b @SammyK #nephp17 Testing …you could probably argue that it is Integration
12. joind.in/talk/7c85b @SammyK #nephp17 Testing …but it’s more like System
13. joind.in/talk/7c85b @SammyK #nephp17 Black-box Testing? What is this
14. joind.in/talk/7c85b @SammyK #nephp17
15. joind.in/talk/7c85b @SammyK #nephp17 What is in there?
16. joind.in/talk/7c85b @SammyK #nephp17 Input Expected Output ✓
17. joind.in/talk/7c85b @SammyK #nephp17 run-tests.php
18. joind.in/talk/7c85b @SammyK #nephp17 .phpt Expected Hello World Output Hello World
19. joind.in/talk/7c85b @SammyK #nephp17 run-tests.php Development
20. joind.in/talk/7c85b @SammyK #nephp17 run-tests.php
23. STFU Operator
24. joind.in/talk/7c85b @SammyK #nephp17
25. joind.in/talk/7c85b @SammyK #nephp17
26. joind.in/talk/7c85b @SammyK #nephp17 run-tests.php Development
27. Legacy Code
28. joind.in/talk/7c85b @SammyK #nephp17 Files
29. joind.in/talk/7c85b @SammyK #nephp17 /ext/{name}/tests /Zend/tests /sapi/{name}/tests Extensions SAPI’s Zend Engine /tests
31. joind.in/talk/7c85b @SammyK #nephp17 Input Output
32. joind.in/talk/7c85b @SammyK #nephp17 Running run-tests with
33. joind.in/talk/7c85b @SammyK #nephp17 1 source code Clone the
34. joind.in/talk/7c85b @SammyK #nephp17 $ git clone \ git@github.com:php/php-src.git
36. joind.in/talk/7c85b @SammyK #nephp17 2source Compile from
38. joind.in/talk/7c85b @SammyK #nephp17 $ ./buildconf
40. joind.in/talk/7c85b @SammyK #nephp17 $ ./configure
41. joind.in/talk/7c85b @SammyK #nephp17 $ ./configure \ --disable-all
42. joind.in/talk/7c85b @SammyK #nephp17 $ ./configure \ --enable-maintainer-zts \ --enable-debug \ --enable-cli
44. joind.in/talk/7c85b @SammyK #nephp17 $ make
46. joind.in/talk/7c85b @SammyK #nephp17
47. $ sapi/cli/php --version
48. joind.in/talk/7c85b @SammyK #nephp17 3tests Run the
49. joind.in/talk/7c85b @SammyK #nephp17 Two Ways to run tests
50. joind.in/talk/7c85b @SammyK #nephp17 run-tests.php Run the with… directly
52. joind.in/talk/7c85b @SammyK #nephp17 Run the with… the Makefile Recommended!
53. Recommended!
54. Recommended!
55. joind.in/talk/7c85b @SammyK #nephp17 Let’s Make a test!
56. joind.in/talk/7c85b @SammyK #nephp17 test-echo.phpt Save as… php Run from
58. joind.in/talk/7c85b @SammyK #nephp17 Test it! $ make test TESTS=test-echo.phpt
60. joind.in/talk/7c85b @SammyK #nephp17 Fail When tests
61. joind.in/talk/7c85b @SammyK #nephp17 Input Output Expected
64. joind.in/talk/7c85b @SammyK #nephp17 Output Examining the !==
65. Files test-echo.out test-echo.exp test-echo.diff test-echo.log Output
66. joind.in/talk/7c85b @SammyK #nephp17 $ less test-echo.out Hello world Script output
67. joind.in/talk/7c85b @SammyK #nephp17 $ less test-echo.exp Hello Jane! --EXPECT--
68. joind.in/talk/7c85b @SammyK #nephp17 $ less test-echo.diff 001+ Hello world 001- Hello Jane!
69. joind.in/talk/7c85b @SammyK #nephp17 $ less test-echo.log ---- EXPECTED OUTPUT Hello Jane! ---- ACTUAL OUTPUT Hello world ---- FAILED
70. Files test-echo.php test-echo.sh Executable
71. joind.in/talk/7c85b @SammyK #nephp17 test-echo.php --FILE--
73. test-echo.sh
74. joind.in/talk/7c85b @SammyK #nephp17 Example A more complex
78. joind.in/talk/7c85b @SammyK #nephp17 Fails So say it
81. $ less test-dt.diff 001+ Boy you know I'm from Europe/Oslo 001- Boy you know I'm from America/Chicago
82. test-dt.php Executable!
84. test-dt.sh
87. joind.in/talk/7c85b @SammyK #nephp17 Debugging $ gdb … failed tests with
90. joind.in/talk/7c85b @SammyK #nephp17 Conventions Naming
91. joind.in/talk/7c85b @SammyK #nephp17 {function-name}_basic.phpt {function-name}_error.phpt {function-name}_variation.phpt {ext-name}{number}.phpt bug{bug-id}.phpt http://qa.php.net/write-test.php#naming-conventions
92. joind.in/talk/7c85b @SammyK #nephp17 Contribute How-to
93. joind.in/talk/7c85b @SammyK #nephp17 Tests! Add some
94. gcov.php.net
95. PHP_FUNCTION
96. readline.c
97. joind.in/talk/7c85b @SammyK #nephp17 ? readline()
98. joind.in/talk/7c85b @SammyK #nephp17 php.net/readline
100. joind.in/talk/7c85b @SammyK #nephp17 Let’s Make a test!
101. joind.in/talk/7c85b @SammyK #nephp17
102. joind.in/talk/7c85b @SammyK #nephp17 for ext/readline Find the ext/readline/tests
104. joind.in/talk/7c85b @SammyK #nephp17 Save as… ext/readline/tests/readline_basic.phpt
106. joind.in/talk/7c85b @SammyK #nephp17 $ less ext/readline/tests/readline_basic.out Fatal error: Uncaught Error: Call to undefined function readline() in /Users/SammyK/Developer/php-src/ ext/readline/tests/ readline_basic.php:2
107. joind.in/talk/7c85b @SammyK #nephp17 php.net/readline
108. joind.in/talk/7c85b @SammyK #nephp17
110. joind.in/talk/7c85b @SammyK #nephp17 $ ./configure \ --enable-maintainer-zts \ --enable-debug \ --enable-cli
111. joind.in/talk/7c85b @SammyK #nephp17 $ ./configure \ --enable-maintainer-zts \ --enable-debug \ --enable-cli \ --with-readline
112. joind.in/talk/7c85b @SammyK #nephp17 OR
113. joind.in/talk/7c85b @SammyK #nephp17 $ ./configure \ --enable-maintainer-zts \ --enable-debug \ --enable-cli \ --with-libedit
115. joind.in/talk/7c85b @SammyK #nephp17 $ less ext/readline/tests/readline_basic.out You said:
116. joind.in/talk/7c85b @SammyK #nephp17 readline_basic.sh Let’s try
118. joind.in/talk/7c85b @SammyK #nephp17
120. Now send a pull request
121. joind.in/talk/7c85b @SammyK #nephp17 internals contributor You’re an
122. joind.in/talk/7c85b @SammyK #nephp17 phptestfest.org
123. joind.in/talk/7c85b @SammyK #nephp17 sammyk.me Six-part screencast series
124. Legacy Code
125. refactor github.com/SammyK/run-tests run-tests
126. refactor run-tests
127. wishlist * Concurrency * Unit tests * Colored output run-tests
128. joind.in/talk/7c85b @SammyK #nephp17 Code with me today! Kent Room after lunch @ 1pm (unconference room)
129. THANKS! SAMMY KAYE POWERS @SammyK SammyK.me Host of @PHPRoundtable @ChiPHPUG West Coast Swing /talk/7c85b I have stickers!