Getting started with PHP core development - 010PHP January 2018

Joind.in: https://joind.in/talk/0649d

While PHP’s open source community is thriving, only a relatively small number of PHP developers contribute to the number one tool they use on a daily basis: PHP itself. Although it may seem daunting to help out on a project written in C, requiring compilation and linking, getting started is actually pretty easy.

In this talk I would like to share my own recent experiences of beginning to contribute to PHP itself. We will cover some practical tips, workflows and resources, so you can start giving back to your programming language straight away!

Transcript

1. Getting started with PHP core development @arnoutboks Arnout Boks #010php 18 jan 2018
2. @arnoutboks #010php PHP Open Source projects
3. @arnoutboks #010php PHP Open Source projects 4175 contributors 418 contributors 689 contributors 1497 contributors
4. @arnoutboks #010php PHP Open Source projects 4175 contributors 418 contributors 689 contributors 1497 contributors
5. @arnoutboks #010php PHP Open Source projects 4175 contributors 418 contributors 689 contributors 1497 contributors 497 contributors?
6. @arnoutboks #010php We all use PHP intensively Why not contribute something back?
7. @arnoutboks #010php The catch
8. @arnoutboks #010php The core
9. @arnoutboks #010php The core
10. @arnoutboks #010php My own journey to the core
11. @arnoutboks #010php Once upon a time… <something xmlns:foo="http://www.example.com"> <foo:bar /> <bar /> </something> <?php $doc = /* … (DOMDocument) */; $doc->getElementsByTagName( "bar" )->length;
12. @arnoutboks #010php Once upon a time… <something xmlns:foo="http://www.example.com"> <foo:bar /> <bar /> </something> <?php $doc = /* … (DOMDocument) */; $doc->getElementsByTagName( "bar" )->length; // returns 2
13. @arnoutboks #010php Once upon a time… <something xmlns:foo="http://www.example.com"> <foo:bar /> <bar /> </something> <?php $doc = /* … (DOMDocument) */; $doc->getElementsByTagNameNS( "http://www.example.com", "bar" )->length;
14. @arnoutboks #010php Once upon a time… <something xmlns:foo="http://www.example.com"> <foo:bar /> <bar /> </something> <?php $doc = /* … (DOMDocument) */; $doc->getElementsByTagNameNS( "http://www.example.com", "bar" )->length; // returns 1
15. @arnoutboks #010php Once upon a time… <something xmlns:foo="http://www.example.com"> <foo:bar /> <bar /> </something> <?php $doc = /* … (DOMDocument) */; $doc->getElementsByTagNameNS( "", "bar" )->length;
16. @arnoutboks #010php Once upon a time… <something xmlns:foo="http://www.example.com"> <foo:bar /> <bar /> </something> <?php $doc = /* … (DOMDocument) */; $doc->getElementsByTagNameNS( "", "bar" )->length; // returns 0!
17. @arnoutboks #010php Once upon a time…
18. Reporting bugs Gotta squash them all
19. @arnoutboks #010php bugs.php.net PHP bug tracker
20. @arnoutboks #010php Write good bug reports https://bugs.php.net/how-to-report.php
21. @arnoutboks #010php Search for existing bugs
22. @arnoutboks #010php Sharpen bug details Report different platform/version: Don’t post “me too” comments – Do post other relevant details
23. @arnoutboks #010php Help triaging bugs! https://bugs.php.net/random
24. Building PHP from git Living on the edge
25. @arnoutboks #010php 1. Install dependencies • git • build-essential • autoconf • automake • libtool • re2c • bison • libxml2-dev Using apt-get, yum or similar
26. @arnoutboks #010php 2. Clone php-src $ git clone https://github.com/php/php- src.git
27. @arnoutboks #010php 3. Generate ./configure script $ ./buildconf Uses autoconf to build ./configure from: • configure.ac • acinclude.m4 • sapi/*/config.m4 • ext/*/config.m4
28. @arnoutboks #010php 4. Run ./configure script $ ./configure Configure a default PHP build: $ ./configure --disable-cgi --disable-dom --enable-opcache --with-sqlite3=/path /to/sqlite3 Customize extensions/SAPIs: $ ./configure --help See all options
29. @arnoutboks #010php 5. Compile $ make $ make –j`nproc` To use all available cores (faster):
30. @arnoutboks #010php 6. Run! $ sapi/cli/php --version Binaries are in sapi/cli/php and sapi/cgi/php-cgi:
31. @arnoutboks #010php Subsequent builds $ make Just run Recompiles only files that have changed
32. @arnoutboks #010php Cleanup scripts $ make clean # try again $ make distclean # try again $ ./buildconf –force $ ./configure # try again If compilation fails, try (subsequently): Might be needed after git pull/git checkout
33. @arnoutboks #010php https://wiki.php.net/ internals/windows/stepbystepbuild Building for Windows
34. @arnoutboks #010php
35. @arnoutboks #010php Vagrant boxes for compiling PHP core
36. @arnoutboks #010php ramsey/vagrant-php-src-dev
37. @arnoutboks #010php rlerdorf/php7dev
38. Running tests for PHP Human-driven CI
39. @arnoutboks #010php Running tests $ sapi/cli/php run-tests.php –p `pwd`/sapi/cli/php
40. @arnoutboks #010php Running tests $ sapi/cli/php run-tests.php –p `pwd`/sapi/cli/php $ sapi/cli/php run-tests.php -P
41. @arnoutboks #010php Running tests $ sapi/cli/php run-tests.php –p `pwd`/sapi/cli/php $ sapi/cli/php run-tests.php -P $ make test
42. @arnoutboks #010php Running tests $ make test […] PASS Trivial "Hello World" test [tests/basic/001.phpt] SKIP Check libcurl config on windows [ext/curl/tests/check_win_config.phpt] reason: for windows only FAIL Compiled regex cache limit [ext/pcre/tests/cache_limit.phpt] XFAIL Inconsistencies when accessing protected members [Zend/tests/access_modifiers_008.phpt] XFAIL REASON: Discussion: http://marc.info/?l=php- internals&m=120221184420957&w=2 […10000 more lines of output…]
43. @arnoutboks #010php Running tests $ make test […] ========================================= TEST RESULT SUMMARY ----------------------------------------- Number of tests : 14896 12520 Tests skipped : 2376 ( 16.0%) -------- Tests warned : 2 ( 0.0%) ( 0.0%) Tests failed : 175 ( 1.2%) ( 1.4%) Expected fail : 44 ( 0.3%) ( 0.4%) Tests passed : 12299 ( 82.6%) ( 98.2%) ----------------------------------------- Time taken : 2937 seconds
44. @arnoutboks #010php Running tests $ make test […] You may have found a problem in PHP. This report can be automatically sent to the PHP QA team at http://qa.php.net/reports and http://news.php.net/php.qa.reports This gives us a better understanding of PHP's behavior. If you don't want to send the report immediately you can choose option "s" to save it. You can then email it to qa- reports@lists.php.net later. Do you want to send this report now? [Yns]:|
45. @arnoutboks #010php Options for running tests $ make test TESTS=path/to/my/test.phpt $ make test TESTS=ext/dom $ make test TESTS="--verbose ext/dom" $ make test TESTS=--help
46. Writing tests for PHP “the best bug report”
47. @arnoutboks #010php Steps to reproduce for bug #67474 Test script: --------------- $doc = new DOMDocument(); $doc->loadXML('<root xmlns:x="x"><a/><x:a/></root>'); $list = $doc->getElementsByTagNameNS('', 'a'); echo $list->length; Expected result: ---------------- 1 Actual result: -------------- 0
48. @arnoutboks #010php Testing tools Typical PHP project: • Written in PHP • Tests written in PHP (PHPUnit, PHPSpec, CodeCeption) The PHP core: • Written in C • Tests written in …
49. @arnoutboks #010php PHP is tested using PHPT tests No C skills required
50. @arnoutboks #010php PHPT tests • Basically plaintext with sections • Some sections (can) contain PHP code • Let PHP print some output… • …and check against expected output
51. @arnoutboks #010php Very simple PHPT test --TEST-- Basic arithmetic - addition --FILE-- <?php var_dump(42 + 1); ?> --EXPECT-- int(43)
52. @arnoutboks #010php Very simple PHPT test --TEST-- Basic arithmetic - addition --FILE-- <?php var_dump(42 + 1); ?> --EXPECT-- int(43)
53. @arnoutboks #010php Folder structure tests/ and Zend/tests/ tests for the Zend engine ext/standard/tests /<group>/ tests for ‘standard’ extension (array, string, filesystem functions, etc.) ext/<name>/tests/ tests for extensions sapi/<name>/tests/ tests for specific SAPI’s (CGI, CLI, PHP-FPM, etc.)
54. @arnoutboks #010php Naming PHPT files bug<bugid>.phpt tests for bugs <func>_basic[<no>].phpt <func>_variation[<no>].phpt <func>_error[<no>].phpt tests for specific function: basic behavior, variations (edge cases, etc.) and errors <extname><no>.phpt general tests for extension
55. @arnoutboks #010php PHPT test for bug #67474 --TEST-- Bug #67474 getElementsByTagNameNS and default namespace --FILE-- <?php $doc = new DOMDocument(); $doc->loadXML('<root xmlns:x="x"><a/><x:a/></root>'); $list = $doc->getElementsByTagNameNS('', 'a'); var_dump($list->length); ?> --EXPECT-- int(1)
56. @arnoutboks #010php Running our PHPT test $ make test TESTS=ext/dom/tests/bug67474.phpt […] ============================================= FAILED TEST SUMMARY --------------------------------------------- Bug #67474 getElementsByTagNameNS and default namespace [ext/dom/tests/bug67474.phpt] =============================================
57. @arnoutboks #010php Running our PHPT test Files generated for our failing test: bug67474.php contents of the FILE section bug67474.sh shell script for running the .php file bug67474.exp expected output bug67474.out actual output bug67474.diff diff between expected and actual bug67474.log log file, including the expected and actual output
58. @arnoutboks #010php Running our PHPT test $ cat ext/dom/tests/bug67474.log ---- EXPECTED OUTPUT int(1) ---- ACTUAL OUTPUT int(0) ---- FAILED
59. @arnoutboks #010php Running our PHPT test
60. @arnoutboks #010php Sections in PHPT files SKIPIF: Skips the test if the code in SKIPIF generates output containing ‘skip’ somewhere --SKIPIF-- <?php if(!extension_loaded('dom')) { die('skip dom extension not available'); } ?>
61. @arnoutboks #010php Sections in PHPT files Tip: Extensions usually have a skipif.inc shared between tests: --SKIPIF-- <?php require_once('skipif.inc'); ?>
62. @arnoutboks #010php Sections in PHPT files GET, POST, PUT, COOKIE, HEADERS: Simulate HTTP input to the PHP script in FILE --GET-- foo=bar&baz=qux --COOKIE-- foo=bar;baz=qux --FILE-- <?php var_dump($_GET['foo'] . $_COOKIE['foo']); ?> --EXPECT-- string(6) "barbar"
63. @arnoutboks #010php Sections in PHPT files EXPECTF, EXPECTREGEX: Expect the output to match a certain format --FILE-- <?php $fp = fopen(__FILE__, 'r'); var_dump($fp); ?> --EXPECTF-- resource(%d) of type (stream)
64. @arnoutboks #010php Sections in PHPT files CLEAN: Cleans up after the test --FILE-- <?php touch(__DIR__ . '/foo.tmp'); var_dump(file_exists(__DIR__ . '/foo.tmp')); ?> --EXPECT-- bool(true) --CLEAN-- <?php unlink(__DIR__ . '/foo.tmp'); ?>
65. @arnoutboks #010php Sections in PHPT files INI: Specifies custom php.ini directives for the test --INI-- precision=7 --FILE-- <?php var_dump(pi()); ?> --EXPECT-- float(3.141593)
66. @arnoutboks #010php Sections in PHPT files XFAIL: Indicates the test is expected to fail, and gives an explanation why This is usually used for hard-to-fix bugs or bugs in upstream code. --XFAIL-- See bug #xxxx
67. @arnoutboks #010php Easy way to get started without C skills
68. @arnoutboks #010php Finding untested code gcov.php.net
69. @arnoutboks #010php Resources • https://qa.php.net/write-test.php • https://www.sammyk.me/compiling-php-from-source- writing-tests-for-php-source • https://phptestfest.org/tutorials/ • http://www.phpinternalsbook.com/#testing-php- source • https://github.com/herdphp/docker-phpqa • https://gist.github.com/SammyK/4a5cf70d0973731d0c 85b151a323ea2d
70. Editing the PHP source Your own custom PHP
71. @arnoutboks #010php A shot at fixing #67474: recap <something xmlns:foo="http://www.example.com"> <foo:bar /> <bar /> </something> <?php $doc = /* … (DOMDocument) */; $doc->getElementsByTagNameNS( "", "bar" )->length; // returns 0!
72. @arnoutboks #010php A shot at fixing #67474 php-src/ext/dom/ • attr.c • document.c • dom_ce.h • dom_fe.h • element.c • node.c • php_dom.c • xpath.c • …
73. @arnoutboks #010php A shot at fixing #67474 const zend_function_entry php_dom_document_class_functions[] = { // ... PHP_FALIAS(createAttributeNS, dom_document_create_attribute_ns, arginfo_dom_document_create_attribute_ns) PHP_FALIAS(getElementsByTagNameNS, dom_document_get_elements_by_tag_name_ns, arginfo_dom_document_get_elements_by_tag_name_ns) PHP_FALIAS(getElementById, dom_document_get_element_by_id, arginfo_dom_document_get_element_by_id) // ... PHP_ME(domdocument, __construct, arginfo_dom_document_construct, ZEND_ACC_PUBLIC) PHP_FE_END };
74. @arnoutboks #010php A shot at fixing #67474
75. @arnoutboks #010php A shot at fixing #67474 const zend_function_entry php_dom_document_class_functions[] = { // ... PHP_FALIAS(createAttributeNS, dom_document_create_attribute_ns, arginfo_dom_document_create_attribute_ns) PHP_FALIAS(getElementsByTagNameNS, dom_document_get_elements_by_tag_name_ns, arginfo_dom_document_get_elements_by_tag_name_ns) PHP_FALIAS(getElementById, dom_document_get_element_by_id, arginfo_dom_document_get_element_by_id) // ... PHP_ME(domdocument, __construct, arginfo_dom_document_construct, ZEND_ACC_PUBLIC) PHP_FE_END };
76. @arnoutboks #010php A shot at fixing #67474 PHP_FUNCTION(dom_document_get_elements_by_tag_name_ns) { // ... if (zend_parse_method_parameters(ZEND_NUM_ARGS(), getThis(), "Oss", &id, dom_document_class_entry, &uri, &uri_len, &name, &name_len) == FAILURE) { return; } // ... local = xmlCharStrndup(name, name_len); nsuri = xmlCharStrndup(uri, uri_len); dom_namednode_iter(intern, 0, namednode, NULL, local, nsuri); }
77. @arnoutboks #010php zend_parse_parameters (‘ZPP’) PHP_FUNCTION(dom_document_get_elements_by_tag_name_ns) { // ... if (zend_parse_method_parameters(ZEND_NUM_ARGS(), getThis(), "Oss", &id, dom_document_class_entry, &uri, &uri_len, &name, &name_len) == FAILURE) { return; } // ... local = xmlCharStrndup(name, name_len); nsuri = xmlCharStrndup(uri, uri_len); dom_namednode_iter(intern, 0, namednode, NULL, local, nsuri); } s: string s: string O: object of given class
78. @arnoutboks #010php zend_parse_parameters (‘ZPP’) Specifier Type a array b boolean d double/float l long/int o object (any type) O object (specific type) r resource s string z mixed
79. @arnoutboks #010php zend_parse_parameters (‘ZPP’) Modifier Effect * variable number of arguments (0 or more) + variable number of arguments (1 or more) ! optional argument | remaining arguments are optional See README.PARAMETER_PARSING_API
80. @arnoutboks #010php A shot at fixing #67474 PHP_FUNCTION(dom_document_get_elements_by_tag_name_ns) { // ... if (zend_parse_method_parameters(ZEND_NUM_ARGS(), getThis(), "Oss", &id, dom_document_class_entry, &uri, &uri_len, &name, &name_len) == FAILURE) { return; } // ... local = xmlCharStrndup(name, name_len); nsuri = xmlCharStrndup(uri, uri_len); dom_namednode_iter(intern, 0, namednode, NULL, local, nsuri); }
81. @arnoutboks #010php A shot at fixing #67474 xmlNode *dom_get_elements_by_tag_name_ns_raw( xmlNodePtr nodep, char *ns, char *local /* ... */ ) { // ... if ( ns == NULL || (nodep->ns != NULL && ( xmlStrEqual(nodep->ns->href, (xmlChar *)ns) || xmlStrEqual((xmlChar *)"*", (xmlChar *)ns) ))) { { } // ... } Eventually:
82. @arnoutboks #010php A shot at fixing #67474 xmlNode *dom_get_elements_by_tag_name_ns_raw( xmlNodePtr nodep, char *ns, char *local /* ... */ ) { // ... if ( ns == NULL || (nodep->ns != NULL && ( xmlStrEqual(nodep->ns->href, (xmlChar *)ns) || xmlStrEqual((xmlChar *)"*", (xmlChar *)ns) ))) { { } // ... } Eventually: element to ‘test’ filter namespace and local name
83. @arnoutboks #010php A shot at fixing #67474 xmlNode *dom_get_elements_by_tag_name_ns_raw( xmlNodePtr nodep, char *ns, char *local /* ... */ ) { // ... if ( ns == NULL || (nodep->ns != NULL && ( xmlStrEqual(nodep->ns->href, (xmlChar *)ns) || xmlStrEqual((xmlChar *)"*", (xmlChar *)ns) ))) { { } // ... } Eventually: “no namespace filter”
84. @arnoutboks #010php A shot at fixing #67474 xmlNode *dom_get_elements_by_tag_name_ns_raw( xmlNodePtr nodep, char *ns, char *local /* ... */ ) { // ... if ( ns == NULL || (nodep->ns != NULL && ( xmlStrEqual(nodep->ns->href, (xmlChar *)ns) || xmlStrEqual((xmlChar *)"*", (xmlChar *)ns) ))) { { } // ... } Eventually: “node has a namespace”, and “it matches the filter” or “filter is a wildcard”
85. @arnoutboks #010php A shot at fixing #67474 xmlNode *dom_get_elements_by_tag_name_ns_raw( xmlNodePtr nodep, char *ns, char *local /* ... */ ) { // ... if ( ns == NULL || (nodep->ns == NULL && !strcmp(ns, "")) || (nodep->ns != NULL && ( xmlStrEqual(nodep->ns->href, (xmlChar *)ns) || xmlStrEqual((xmlChar *)"*", (xmlChar *)ns) ))) { { } // ... } Fix: “node has empty namespace”, and “filter is empty namespace”
86. @arnoutboks #010php A shot at fixing #67474 $ make test TESTS=ext/dom/tests/bug67474.phpt […] ============================================= Running selected tests. PASS Bug #67474 getElementsByTagNameNS and default namespace [ext/dom/tests/bug67474.phpt] ============================================= Number of tests : 1 1 Tests passed : 1 (100.0%) (100.0%)
87. @arnoutboks #010php PR for #67474 Submit bug fixes to the oldest supported branch: http://php.net/supported-versions.php
88. @arnoutboks #010php PR for #67474: 2 hours later…
89. @arnoutboks #010php It’s not THAT difficult • Read past the C details • See where variables come from and go to • Try and see what happens • Rely on the tests
90. @arnoutboks #010php Bug fixes vs. features • Bug fixes • PR against oldest supported branch • No RFC needed • Refer to bug in bug tracker • Features • PR against master • Formal RFC needed
91. @arnoutboks #010php RFC Process • Introduce idea, measure initial reaction • Formal proposal • Discussion period • Voting See https://wiki.php.net/rfc/howto & https://benramsey.com/talks/2015/06/dpc-contributing-core/
92. @arnoutboks #010php Resources • http://php.net/internals • https://wiki.php.net/internals • https://wiki.php.net/internals/references • http://www.phpinternalsbook.com/ • http://blog.jpauli.tech/ • https://nikic.github.io/ • https://blog.ircmaxell.com/search/label/PHP-Internals • https://www.sammyk.me/how-to-find-and-patch-a- bug-in-php-source-php-internals • https://externals.io
93. @arnoutboks #010php Some time later…
94. Documentation for PHP Contributing to the manual
95. @arnoutboks #010php edit.php.net PHP Docbook Online Editor ‘Easiest’ way to get started (if you can figure out how it works)
96. @arnoutboks #010php edit.php.net
97. @arnoutboks #010php edit.php.net
98. @arnoutboks #010php edit.php.net
99. @arnoutboks #010php 1. Find the correct file Under “All files” in the left menu Top-level directory per language • reference/<ext|group>/ • functions/<function>.xml • <class>.xml • <class>/<method>.xml • language/<feature>.xml
100. @arnoutboks #010php 1. Find the correct file Tip: Use the “edit” link from the documentation itself
101. @arnoutboks #010php 2. Make your changes Documentation is written using DocBook XML http://tdg.docbook.org/tdg/5.2/
102. @arnoutboks #010php 2. Make your changes Documentation is written using DocBook XML (or just peek in other docs)
103. @arnoutboks #010php 3. Preview changes Seems broken at the moment
104. @arnoutboks #010php 4. Submit as patch
105. @arnoutboks #010php Tip: submit a docs patch instead of user note
106. @arnoutboks #010php Documentation karma Commit changes yourself? Apply for docs karma! http://doc.php.net/tutorial/ joining.php
107. @arnoutboks #010php Don’t like edit.php.net? Make it better!
108. @arnoutboks #010php Don’t like edit.php.net? Or set up your own development environment: https://www.sammyk.me/how-to-contribute-to-php- documentation
109. @arnoutboks #010php Where to help? • Bugs of type “Doc” in bug tracker • check-missing-docs.php • “Failures to meet strict standards” (in edit.php.net sidebar) • Translations • Missing translations • Out-of-date translations • Translations needing review • Translations with errors
110. @arnoutboks #010php Where to help?
111. Recap Ways to contribute to PHP
112. @arnoutboks #010php Numerous ways to contribute • Bug reports • Bug triaging • Running tests • Writing tests • Bug fixes • New features • Documentation patches • Documentation translations • Tools
113. @arnoutboks #010php Why contribute? • Get ‘own’ bugs fixed • Fix ‘own’ bugs • Make PHP better • Make PHP more popular • Learn PHP • Learn C • Give back to the community
114. @arnoutboks #010php You don’t have to be an expert to contribute to the PHP core
115. @arnoutboks #010php Feedback & Questions @arnoutboks @arnoutboks @aboks Arnout Boks Please leave your feedback on joind.in: https://joind.in/talk/0649d
116. @arnoutboks #010php Image Credits • https://photojournal.jpl.nasa.gov/jpeg/PIA19058.jpg • https://www.flickr.com/photos/vivianejl/22990832 • https://www.flickr.com/photos/dhuiz/13609080315/ • https://www.flickr.com/photos/gotovan/8671348572 • https://www.flickr.com/photos/sidelong/246816211 • https://www.flickr.com/photos/pezon64/23764268828 • https://www.flickr.com/photos/crdot/6855538268/ • https://www.flickr.com/photos/borkurdotnet/9682277597