Writing Tests For PHP Source - ZendCon 2017

Talk given at ZendCon 2017

As you might have already heard from a grumpy individual, adding more tests to your codebase is crucial to its stability over time. This session isn't about adding tests to your PHP codebase, but adding tests to the PHP language itself. And the best part is, all the tests are written in PHP so you don't even have to dust off that old C book from college. In this session you'll see how to find untested parts of the PHP source code, how to write a test for the untested feature, and how to submit your tests to PHP internals. Not only will you be making PHP more stable and reliable, but there's a good chance the tests you write will give you a deeper understanding of PHP; knowledge you can use to improve your own codebases. Not to mention you'll be able to call yourself an internals contributor. Come on in! The water's fine!

Transcript

1. joind.in/talk/c6253 @SammyK #ZendCon2017 For PHP Source Writing Tests Sammy Kaye Powers 2017-10-26
2. joind.in/talk/c6253 @SammyK #ZendCon2017 Slides Get the joind.in/talk/c6253
3. joind.in/talk/c6253 @SammyK #ZendCon2017 PHP Internals This is a talk
4. Write more TESTS!
5. joind.in/talk/c6253 @SammyK #ZendCon2017 Unit tests?
6. joind.in/talk/c6253 @SammyK #ZendCon2017
7. joind.in/talk/c6253 @SammyK #ZendCon2017 @dhh “TDD is dead.” http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html
8. joind.in/talk/c6253 @SammyK #ZendCon2017 Acceptance tests?
9. joind.in/talk/c6253 @SammyK #ZendCon2017 Integration tests? *Sort of
10. joind.in/talk/c6253 @SammyK #ZendCon2017 Black-box Testing
11. joind.in/talk/c6253 @SammyK #ZendCon2017 Testing …you could probably argue that it is Integration
12. joind.in/talk/c6253 @SammyK #ZendCon2017 Testing …but it’s more like System
13. joind.in/talk/c6253 @SammyK #ZendCon2017 Black-box Testing? What is this
14. joind.in/talk/c6253 @SammyK #ZendCon2017
15. joind.in/talk/c6253 @SammyK #ZendCon2017 What is in there?
16. joind.in/talk/c6253 @SammyK #ZendCon2017 Input Expected Output ✓
17. joind.in/talk/c6253 @SammyK #ZendCon2017 run-tests.php
18. joind.in/talk/c6253 @SammyK #ZendCon2017 .phpt Expected Hello World Output Hello World
19. joind.in/talk/c6253 @SammyK #ZendCon2017 run-tests.php Development
20. joind.in/talk/c6253 @SammyK #ZendCon2017 run-tests.php
23. STFU Operator
24. joind.in/talk/c6253 @SammyK #ZendCon2017
25. joind.in/talk/c6253 @SammyK #ZendCon2017
26. joind.in/talk/c6253 @SammyK #ZendCon2017 run-tests.php Development
27. Legacy Code
28. joind.in/talk/c6253 @SammyK #ZendCon2017 Files
29. joind.in/talk/c6253 @SammyK #ZendCon2017 /ext/{name}/tests /Zend/tests /sapi/{name}/tests Extensions SAPI’s Zend Engine /tests
31. joind.in/talk/c6253 @SammyK #ZendCon2017 Input Output
32. joind.in/talk/c6253 @SammyK #ZendCon2017 Running run-tests with
33. joind.in/talk/c6253 @SammyK #ZendCon2017 1 source code Clone the
34. joind.in/talk/c6253 @SammyK #ZendCon2017 $ git clone \ git@github.com:php/php-src.git
36. joind.in/talk/c6253 @SammyK #ZendCon2017 2source Compile from
38. joind.in/talk/c6253 @SammyK #ZendCon2017 $ ./buildconf
40. joind.in/talk/c6253 @SammyK #ZendCon2017 $ ./configure
41. joind.in/talk/c6253 @SammyK #ZendCon2017 $ ./configure \ --disable-all
42. joind.in/talk/c6253 @SammyK #ZendCon2017 $ ./configure \ --enable-maintainer-zts \ --enable-debug \ --enable-cli
44. joind.in/talk/c6253 @SammyK #ZendCon2017 $ make
46. joind.in/talk/c6253 @SammyK #ZendCon2017
47. $ sapi/cli/php --version
48. joind.in/talk/c6253 @SammyK #ZendCon2017 3tests Run the
49. joind.in/talk/c6253 @SammyK #ZendCon2017 Two Ways to run tests
50. joind.in/talk/c6253 @SammyK #ZendCon2017 run-tests.php Run the with… directly
52. joind.in/talk/c6253 @SammyK #ZendCon2017 Run the with… the Makefile Recommended!
53. Recommended!
54. Recommended!
55. joind.in/talk/c6253 @SammyK #ZendCon2017 Let’s Make a test!
56. joind.in/talk/c6253 @SammyK #ZendCon2017 test-echo.phpt Save as… php Run from
58. joind.in/talk/c6253 @SammyK #ZendCon2017 Test it! $ make test TESTS=test-echo.phpt
60. joind.in/talk/c6253 @SammyK #ZendCon2017 Fail When tests
61. joind.in/talk/c6253 @SammyK #ZendCon2017 Input Output Expected
64. joind.in/talk/c6253 @SammyK #ZendCon2017 Output Examining the !==
65. Files test-echo.out test-echo.exp test-echo.diff test-echo.log Output
66. joind.in/talk/c6253 @SammyK #ZendCon2017 $ less test-echo.out Hello world Script output
67. joind.in/talk/c6253 @SammyK #ZendCon2017 $ less test-echo.exp Hello Jane! --EXPECT--
68. joind.in/talk/c6253 @SammyK #ZendCon2017 $ less test-echo.diff 001+ Hello world 001- Hello Jane!
69. joind.in/talk/c6253 @SammyK #ZendCon2017 $ less test-echo.log ---- EXPECTED OUTPUT Hello Jane! ---- ACTUAL OUTPUT Hello world ---- FAILED
70. Files test-echo.php test-echo.sh Executable
71. joind.in/talk/c6253 @SammyK #ZendCon2017 test-echo.php --FILE--
73. test-echo.sh
74. joind.in/talk/c6253 @SammyK #ZendCon2017 Example A more complex
78. joind.in/talk/c6253 @SammyK #ZendCon2017 Fails So say it
81. $ less test-dt.diff 001+ Boy you know I'm from Europe/Oslo 001- Boy you know I'm from America/Chicago
82. test-dt.php Executable!
84. test-dt.sh
87. joind.in/talk/c6253 @SammyK #ZendCon2017 Debugging $ gdb … failed tests with
90. joind.in/talk/c6253 @SammyK #ZendCon2017 Conventions Naming
91. joind.in/talk/c6253 @SammyK #ZendCon2017 {function-name}_basic.phpt {function-name}_error.phpt {function-name}_variation.phpt {ext-name}{number}.phpt bug{bug-id}.phpt http://qa.php.net/write-test.php#naming-conventions
92. joind.in/talk/c6253 @SammyK #ZendCon2017 Contribute How-to
93. joind.in/talk/c6253 @SammyK #ZendCon2017 Tests! Add some
94. gcov.php.net
95. PHP_FUNCTION
96. readline.c
97. joind.in/talk/c6253 @SammyK #ZendCon2017 ? readline()
98. joind.in/talk/c6253 @SammyK #ZendCon2017 php.net/readline
100. joind.in/talk/c6253 @SammyK #ZendCon2017 Let’s Make a test!
101. joind.in/talk/c6253 @SammyK #ZendCon2017
102. joind.in/talk/c6253 @SammyK #ZendCon2017 for ext/readline Find the ext/readline/tests
104. joind.in/talk/c6253 @SammyK #ZendCon2017 Save as… ext/readline/tests/readline_basic.phpt
106. joind.in/talk/c6253 @SammyK #ZendCon2017 $ less ext/readline/tests/readline_basic.out Fatal error: Uncaught Error: Call to undefined function readline() in /Users/SammyK/Developer/php-src/ ext/readline/tests/ readline_basic.php:2
107. joind.in/talk/c6253 @SammyK #ZendCon2017 php.net/readline
108. joind.in/talk/c6253 @SammyK #ZendCon2017
110. joind.in/talk/c6253 @SammyK #ZendCon2017 $ ./configure \ --enable-maintainer-zts \ --enable-debug \ --enable-cli
111. joind.in/talk/c6253 @SammyK #ZendCon2017 $ ./configure \ --enable-maintainer-zts \ --enable-debug \ --enable-cli \ --with-readline
112. joind.in/talk/c6253 @SammyK #ZendCon2017 OR
113. joind.in/talk/c6253 @SammyK #ZendCon2017 $ ./configure \ --enable-maintainer-zts \ --enable-debug \ --enable-cli \ --with-libedit
115. joind.in/talk/c6253 @SammyK #ZendCon2017 $ less ext/readline/tests/readline_basic.out You said:
116. joind.in/talk/c6253 @SammyK #ZendCon2017 readline_basic.sh Let’s try
118. joind.in/talk/c6253 @SammyK #ZendCon2017
120. Now send a pull request
121. joind.in/talk/c6253 @SammyK #ZendCon2017 internals contributor You’re an
122. joind.in/talk/c6253 @SammyK #ZendCon2017 phptestfest.org
123. joind.in/talk/c6253 @SammyK #ZendCon2017 sammyk.me Six-part screencast series
124. Legacy Code
125. refactor github.com/SammyK/run-tests run-tests
126. refactor run-tests
127. wishlist * Concurrency * Unit tests * Colored output run-tests
128. joind.in/talk/c6253 @SammyK #ZendCon2017 Code with me today! Uncon Room @ 9:15am (by Paradise tower)
129. Sammy Kaye Powers Thanks! /talk/c6253 @SammyK SammyK.me Host of @PHPRoundtable Co-admin of @ChiPHPUG I ❤ West Coast Swing