Tweet
Tweet

Slide 1

Slide 1 text

Getting started with PHP core development @arnoutboks Arnout Boks #dpc18 08-06-2018

Slide 2

Slide 2 text

@arnoutboks #dpc18 PHP Open Source projects

Slide 3

Slide 3 text

@arnoutboks #dpc18 PHP Open Source projects 4175 contributors 418 contributors 689 contributors 1497 contributors

Slide 4

Slide 4 text

@arnoutboks #dpc18 PHP Open Source projects 4175 contributors 418 contributors 689 contributors 1497 contributors

Slide 5

Slide 5 text

@arnoutboks #dpc18 PHP Open Source projects 4175 contributors 418 contributors 689 contributors 1497 contributors 497 contributors?

Slide 6

Slide 6 text

@arnoutboks #dpc18 We all use PHP intensively Why not contribute something back?

Slide 7

Slide 7 text

@arnoutboks #dpc18 The catch

Slide 8

Slide 8 text

@arnoutboks #dpc18 The catch

Slide 9

Slide 9 text

@arnoutboks #dpc18 The core

Slide 10

Slide 10 text

@arnoutboks #dpc18 The core

Slide 11

Slide 11 text

@arnoutboks #dpc18 My own journey to the core

Slide 12

Slide 12 text

@arnoutboks #dpc18 Once upon a time… getElementsByTagName( "bar" )->length;

Slide 13

Slide 13 text

@arnoutboks #dpc18 Once upon a time… getElementsByTagName( "bar" )->length; // returns 2

Slide 14

Slide 14 text

@arnoutboks #dpc18 Once upon a time… getElementsByTagNameNS( "http://www.example.com", "bar" )->length;

Slide 15

Slide 15 text

@arnoutboks #dpc18 Once upon a time… getElementsByTagNameNS( "http://www.example.com", "bar" )->length; // returns 1

Slide 16

Slide 16 text

@arnoutboks #dpc18 Once upon a time… getElementsByTagNameNS( "", "bar" )->length;

Slide 17

Slide 17 text

@arnoutboks #dpc18 Once upon a time… getElementsByTagNameNS( "", "bar" )->length; // returns 0!

Slide 18

Slide 18 text

@arnoutboks #dpc18 Once upon a time…

Slide 19

Slide 19 text

Reporting bugs Gotta squash them all

Slide 20

Slide 20 text

@arnoutboks #dpc18 bugs.php.net PHP bug tracker

Slide 21

Slide 21 text

@arnoutboks #dpc18 Write good bug reports https://bugs.php.net/how-to-report.php

Slide 22

Slide 22 text

@arnoutboks #dpc18 Search for existing bugs

Slide 23

Slide 23 text

@arnoutboks #dpc18 Sharpen bug details Report different platform/version: Don’t post “me too” comments – Do post other relevant details

Slide 24

Slide 24 text

@arnoutboks #dpc18 Help triaging bugs! https://bugs.php.net/random

Slide 25

Slide 25 text

Building PHP from git Living on the edge

Slide 26

Slide 26 text

@arnoutboks #dpc18 1. Install dependencies • git • build-essential • autoconf • automake • libtool • re2c • bison • libxml2-dev Using apt-get, yum or similar

Slide 27

Slide 27 text

@arnoutboks #dpc18 2. Clone php-src $ git clone https://github.com/php/php- src.git

Slide 28

Slide 28 text

@arnoutboks #dpc18 3. Generate ./configure script $ ./buildconf Uses autoconf to build ./configure from: • configure.ac • acinclude.m4 • sapi/*/config.m4 • ext/*/config.m4

Slide 29

Slide 29 text

@arnoutboks #dpc18 4. Run ./configure script $ ./configure Configure a default PHP build: $ ./configure --disable-cgi --disable-dom --enable-opcache --with-sqlite3=/path /to/sqlite3 Customize extensions/SAPIs: $ ./configure --help See all options

Slide 30

Slide 30 text

@arnoutboks #dpc18 5. Compile $ make $ make –j`nproc` To use all available cores (faster):

Slide 31

Slide 31 text

@arnoutboks #dpc18 6. Run! $ sapi/cli/php --version Binaries are in sapi/cli/php and sapi/cgi/php-cgi:

Slide 32

Slide 32 text

@arnoutboks #dpc18 Subsequent builds $ make Just run Recompiles only files that have changed

Slide 33

Slide 33 text

@arnoutboks #dpc18 Cleanup scripts $ make clean # try again $ make distclean # try again $ ./buildconf –force $ ./configure # try again If compilation fails, try (subsequently): Might be needed after git pull/git checkout

Slide 34

Slide 34 text

@arnoutboks #dpc18 https://wiki.php.net/ internals/windows/stepbystepbuild Building for Windows

Slide 35

Slide 35 text

@arnoutboks #dpc18

Slide 36

Slide 36 text

@arnoutboks #dpc18 • ramsey/vagrant-php-src- dev • rlerdorf/php7dev Vagrant boxes for compiling PHP

Slide 37

Slide 37 text

@arnoutboks #dpc18 Or use Docker: aboks/php-src-devtools

Slide 38

Slide 38 text

@arnoutboks #dpc18 Using php-src-devtools $ composer global require aboks/php-src- devtools $ git clone https://github.com/php/php- src.git && cd php-src $ php-src-devtools build

Slide 39

Slide 39 text

Running tests for PHP Human-driven CI

Slide 40

Slide 40 text

@arnoutboks #dpc18 Running tests $ sapi/cli/php run-tests.php –p `pwd`/sapi/cli/php

Slide 41

Slide 41 text

@arnoutboks #dpc18 Running tests $ sapi/cli/php run-tests.php –p `pwd`/sapi/cli/php $ sapi/cli/php run-tests.php -P

Slide 42

Slide 42 text

@arnoutboks #dpc18 Running tests $ sapi/cli/php run-tests.php –p `pwd`/sapi/cli/php $ sapi/cli/php run-tests.php -P $ make test # (or, with php-src-devtools) $ php-src-devtools test

Slide 43

Slide 43 text

@arnoutboks #dpc18 Running tests $ make test […] PASS Trivial "Hello World" test [tests/basic/001.phpt] SKIP Check libcurl config on windows [ext/curl/tests/check_win_config.phpt] reason: for windows only FAIL Compiled regex cache limit [ext/pcre/tests/cache_limit.phpt] XFAIL Inconsistencies when accessing protected members [Zend/tests/access_modifiers_008.phpt] XFAIL REASON: Discussion: http://marc.info/?l=php- internals&m=120221184420957&w=2 […10000 more lines of output…]

Slide 44

Slide 44 text

@arnoutboks #dpc18 Running tests $ make test […] ========================================= TEST RESULT SUMMARY ----------------------------------------- Number of tests : 14896 12520 Tests skipped : 2376 ( 16.0%) -------- Tests warned : 2 ( 0.0%) ( 0.0%) Tests failed : 175 ( 1.2%) ( 1.4%) Expected fail : 44 ( 0.3%) ( 0.4%) Tests passed : 12299 ( 82.6%) ( 98.2%) ----------------------------------------- Time taken : 2937 seconds

Slide 45

Slide 45 text

@arnoutboks #dpc18 Running tests $ make test […] You may have found a problem in PHP. This report can be automatically sent to the PHP QA team at http://qa.php.net/reports and http://news.php.net/php.qa.reports This gives us a better understanding of PHP's behavior. If you don't want to send the report immediately you can choose option "s" to save it. You can then email it to qa- [email protected] later. Do you want to send this report now? [Yns]:|

Slide 46

Slide 46 text

@arnoutboks #dpc18 Options for running tests $ make test TESTS=path/to/my/test.phpt $ make test TESTS=ext/dom $ make test TESTS="--verbose ext/dom" $ make test TESTS=--help

Slide 47

Slide 47 text

Writing tests for PHP “the best bug report”

Slide 48

Slide 48 text

@arnoutboks #dpc18 Steps to reproduce for bug #67474 Test script: --------------- $doc = new DOMDocument(); $doc->loadXML(''); $list = $doc->getElementsByTagNameNS('', 'a'); echo $list->length; Expected result: ---------------- 1 Actual result: -------------- 0

Slide 49

Slide 49 text

@arnoutboks #dpc18 Testing tools Typical PHP project: • Written in PHP • Tests written in PHP (PHPUnit, PHPSpec, CodeCeption) The PHP core: • Written in C • Tests written in …

Slide 50

Slide 50 text

@arnoutboks #dpc18 PHP is tested using PHPT tests No C skills required

Slide 51

Slide 51 text

@arnoutboks #dpc18 PHPT tests • Basically plaintext with sections • Some sections (can) contain PHP code • Let PHP print some output… • …and check against expected output

Slide 52

Slide 52 text

@arnoutboks #dpc18 Very simple PHPT test --TEST-- Basic arithmetic - addition --FILE-- --EXPECT-- int(43)

Slide 53

Slide 53 text

@arnoutboks #dpc18 Very simple PHPT test --TEST-- Basic arithmetic - addition --FILE-- --EXPECT-- int(43)

Slide 54

Slide 54 text

@arnoutboks #dpc18 PHPT test for bug #67474 --TEST-- Bug #67474 getElementsByTagNameNS and default namespace --FILE-- loadXML(''); $list = $doc->getElementsByTagNameNS('', 'a'); var_dump($list->length); ?> --EXPECT-- int(1)

Slide 55

Slide 55 text

@arnoutboks #dpc18 Running our PHPT test $ make test TESTS=ext/dom/tests/bug67474.phpt […] ============================================= FAILED TEST SUMMARY --------------------------------------------- Bug #67474 getElementsByTagNameNS and default namespace [ext/dom/tests/bug67474.phpt] =============================================

Slide 56

Slide 56 text

@arnoutboks #dpc18 Running our PHPT test Files generated for our failing test: bug67474.php contents of the FILE section bug67474.sh shell script for running the .php file bug67474.exp expected output bug67474.out actual output bug67474.diff diff between expected and actual bug67474.log log file, including the expected and actual output

Slide 57

Slide 57 text

@arnoutboks #dpc18 Running our PHPT test $ cat ext/dom/tests/bug67474.log ---- EXPECTED OUTPUT int(1) ---- ACTUAL OUTPUT int(0) ---- FAILED

Slide 58

Slide 58 text

@arnoutboks #dpc18 Running our PHPT test

Slide 59

Slide 59 text

@arnoutboks #dpc18 Sections in PHPT files GET, POST, PUT, COOKIE, HEADERS: Simulate HTTP input to the PHP script in FILE --GET-- foo=bar&baz=qux --COOKIE-- foo=bar;baz=qux --FILE-- --EXPECT-- string(6) "barbar"

Slide 60

Slide 60 text

@arnoutboks #dpc18 Sections in PHPT files EXPECTF, EXPECTREGEX: Expect the output to match a certain format --FILE-- --EXPECTF-- resource(%d) of type (stream)

Slide 61

Slide 61 text

@arnoutboks #dpc18 More sections in PHPT files SKIPIF: Skips the test if the code in SKIPIF generates output containing ‘skip’ somewhere CLEAN: Cleans up after the test INI: Specifies custom php.ini directives for the test XFAIL: Indicates the test is expected to fail, and gives an explanation why

Slide 62

Slide 62 text

@arnoutboks #dpc18 Easy way to get started without C skills

Slide 63

Slide 63 text

@arnoutboks #dpc18 Finding untested code gcov.php.net

Slide 64

Slide 64 text

@arnoutboks #dpc18 Resources • https://qa.php.net/write-test.php • https://www.sammyk.me/compiling-php-from-source- writing-tests-for-php-source • https://phptestfest.org/tutorials/ • http://www.phpinternalsbook.com/#testing-php- source • https://github.com/PHPTestFestBrasil/phptt • https://gist.github.com/SammyK/4a5cf70d0973731d0c 85b151a323ea2d

Slide 65

Slide 65 text

Editing the PHP source Your own custom PHP

Slide 66

Slide 66 text

@arnoutboks #dpc18 A shot at fixing #67474: recap getElementsByTagNameNS( "", "bar" )->length; // returns 0!

Slide 67

Slide 67 text

@arnoutboks #dpc18 A shot at fixing #67474 php-src/ext/dom/ • attr.c • document.c • dom_ce.h • dom_fe.h • element.c • node.c • php_dom.c • xpath.c • …

Slide 68

Slide 68 text

@arnoutboks #dpc18 A shot at fixing #67474 const zend_function_entry php_dom_document_class_functions[] = { // ... PHP_FALIAS(createAttributeNS, dom_document_create_attribute_ns, arginfo_dom_document_create_attribute_ns) PHP_FALIAS(getElementsByTagNameNS, dom_document_get_elements_by_tag_name_ns, arginfo_dom_document_get_elements_by_tag_name_ns) PHP_FALIAS(getElementById, dom_document_get_element_by_id, arginfo_dom_document_get_element_by_id) // ... PHP_ME(domdocument, __construct, arginfo_dom_document_construct, ZEND_ACC_PUBLIC) PHP_FE_END };

Slide 69

Slide 69 text

@arnoutboks #dpc18 C macros

Slide 70

Slide 70 text

@arnoutboks #dpc18 C macros #define MIN(a,b) a < b ? a : b int c = MIN(42,7) // ^ is converted into // int c = 42 < 7 ? 42 : 7

Slide 71

Slide 71 text

@arnoutboks #dpc18 A shot at fixing #67474 const zend_function_entry php_dom_document_class_functions[] = { // ... PHP_FALIAS(createAttributeNS, dom_document_create_attribute_ns, arginfo_dom_document_create_attribute_ns) PHP_FALIAS(getElementsByTagNameNS, dom_document_get_elements_by_tag_name_ns, arginfo_dom_document_get_elements_by_tag_name_ns) PHP_FALIAS(getElementById, dom_document_get_element_by_id, arginfo_dom_document_get_element_by_id) // ... PHP_ME(domdocument, __construct, arginfo_dom_document_construct, ZEND_ACC_PUBLIC) PHP_FE_END };

Slide 72

Slide 72 text

@arnoutboks #dpc18 A shot at fixing #67474 PHP_FUNCTION(dom_document_get_elements_by_tag_name_ns) { // ... if (zend_parse_method_parameters(ZEND_NUM_ARGS(), getThis(), "Oss", &id, dom_document_class_entry, &uri, &uri_len, &name, &name_len) == FAILURE) { return; } // ... local = xmlCharStrndup(name, name_len); nsuri = xmlCharStrndup(uri, uri_len); dom_namednode_iter(intern, 0, namednode, NULL, local, nsuri); }

Slide 73

Slide 73 text

@arnoutboks #dpc18 zend_parse_parameters (‘ZPP’) PHP_FUNCTION(dom_document_get_elements_by_tag_name_ns) { // ... if (zend_parse_method_parameters(ZEND_NUM_ARGS(), getThis(), "Oss", &id, dom_document_class_entry, &uri, &uri_len, &name, &name_len) == FAILURE) { return; } // ... local = xmlCharStrndup(name, name_len); nsuri = xmlCharStrndup(uri, uri_len); dom_namednode_iter(intern, 0, namednode, NULL, local, nsuri); } s: string s: string O: object of given class

Slide 74

Slide 74 text

@arnoutboks #dpc18 zend_parse_parameters (‘ZPP’) Specifier Type a array b boolean d double/float l long/int o object (any type) O object (specific type) r resource s string z mixed

Slide 75

Slide 75 text

@arnoutboks #dpc18 zend_parse_parameters (‘ZPP’) Modifier Effect * variable number of arguments (0 or more) + variable number of arguments (1 or more) ! optional argument | remaining arguments are optional See README.PARAMETER_PARSING_API

Slide 76

Slide 76 text

@arnoutboks #dpc18 A shot at fixing #67474 PHP_FUNCTION(dom_document_get_elements_by_tag_name_ns) { // ... if (zend_parse_method_parameters(ZEND_NUM_ARGS(), getThis(), "Oss", &id, dom_document_class_entry, &uri, &uri_len, &name, &name_len) == FAILURE) { return; } // ... local = xmlCharStrndup(name, name_len); nsuri = xmlCharStrndup(uri, uri_len); dom_namednode_iter(intern, 0, namednode, NULL, local, nsuri); }

Slide 77

Slide 77 text

@arnoutboks #dpc18 A shot at fixing #67474 xmlNode *dom_get_elements_by_tag_name_ns_raw( xmlNodePtr nodep, char *ns, char *local /* ... */ ) { // ... if ( ns == NULL || (nodep->ns != NULL && ( xmlStrEqual(nodep->ns->href, (xmlChar *)ns) || xmlStrEqual((xmlChar *)"*", (xmlChar *)ns) )) ) { /* namespace matches */ } // ... } Eventually:

Slide 78

Slide 78 text

@arnoutboks #dpc18 A shot at fixing #67474 xmlNode *dom_get_elements_by_tag_name_ns_raw( xmlNodePtr nodep, char *ns, char *local /* ... */ ) { // ... if ( ns == NULL || (nodep->ns != NULL && ( xmlStrEqual(nodep->ns->href, (xmlChar *)ns) || xmlStrEqual((xmlChar *)"*", (xmlChar *)ns) )) ) { /* namespace matches */ } // ... } Eventually: element to ‘test’ filter namespace and local name

Slide 79

Slide 79 text

@arnoutboks #dpc18 A shot at fixing #67474 xmlNode *dom_get_elements_by_tag_name_ns_raw( xmlNodePtr nodep, char *ns, char *local /* ... */ ) { // ... if ( ns == NULL || (nodep->ns != NULL && ( xmlStrEqual(nodep->ns->href, (xmlChar *)ns) || xmlStrEqual((xmlChar *)"*", (xmlChar *)ns) )) ) { /* namespace matches */ } // ... } Eventually: “no namespace filter”

Slide 80

Slide 80 text

@arnoutboks #dpc18 A shot at fixing #67474 xmlNode *dom_get_elements_by_tag_name_ns_raw( xmlNodePtr nodep, char *ns, char *local /* ... */ ) { // ... if ( ns == NULL || (nodep->ns != NULL && ( xmlStrEqual(nodep->ns->href, (xmlChar *)ns) || xmlStrEqual((xmlChar *)"*", (xmlChar *)ns) )) ) { /* namespace matches */ } // ... } Eventually: “node has a namespace”, and “it matches the filter” or “filter is a wildcard”

Slide 81

Slide 81 text

@arnoutboks #dpc18 A shot at fixing #67474 xmlNode *dom_get_elements_by_tag_name_ns_raw( xmlNodePtr nodep, char *ns, char *local /* ... */ ) { // ... if ( ns == NULL || (nodep->ns == NULL && !strcmp(ns, "")) || (nodep->ns != NULL && ( xmlStrEqual(nodep->ns->href, (xmlChar *)ns) || xmlStrEqual((xmlChar *)"*", (xmlChar *)ns) )) ) { /* namespace matches */ } // ... } Fix: “node has empty namespace”, and “filter is empty namespace”

Slide 82

Slide 82 text

@arnoutboks #dpc18 A shot at fixing #67474 $ make test TESTS=ext/dom/tests/bug67474.phpt […] ============================================= Running selected tests. PASS Bug #67474 getElementsByTagNameNS and default namespace [ext/dom/tests/bug67474.phpt] ============================================= Number of tests : 1 1 Tests passed : 1 (100.0%) (100.0%)

Slide 83

Slide 83 text

@arnoutboks #dpc18 PR for #67474 Submit bug fixes to the oldest supported branch: http://php.net/supported-versions.php

Slide 84

Slide 84 text

@arnoutboks #dpc18 The Release Managers PHP 7.1 Davey Shafik @dshafik Joe Watkins @krakjoe PHP 7.2 Remi Collet @RemiCollet Sara Golemon @SaraMG PHP 7.3 Christoph M. Becker @cmbecker69 Stanislav Malyshev @smalyshev

Slide 85

Slide 85 text

@arnoutboks #dpc18 PR for #67474: 2 hours later…

Slide 86

Slide 86 text

@arnoutboks #dpc18 It’s not THAT difficult • Read past the C details and macros • Focus on the logic • See where variables come from and go to • Try and see what happens • Rely on the tests

Slide 87

Slide 87 text

@arnoutboks #dpc18 You can do this!

Slide 88

Slide 88 text

@arnoutboks #dpc18 Bug fixes vs. features • Bug fixes • PR against oldest supported branch • No RFC needed • Refer to bug in bug tracker • Features • PR against master • Formal RFC needed

Slide 89

Slide 89 text

@arnoutboks #dpc18 RFC Process • Introduce idea, measure initial reaction • Formal proposal • Discussion period • Voting See https://wiki.php.net/rfc/howto & https://benramsey.com/talks/2015/06/dpc-contributing-core/

Slide 90

Slide 90 text

@arnoutboks #dpc18 Resources • http://php.net/internals • https://wiki.php.net/internals • https://wiki.php.net/internals/references • http://www.phpinternalsbook.com/ • http://blog.jpauli.tech/ • https://nikic.github.io/ • https://blog.ircmaxell.com/search/label/PHP-Internals • https://www.sammyk.me/how-to-find-and-patch-a- bug-in-php-source-php-internals • https://externals.io

Slide 91

Slide 91 text

@arnoutboks #dpc18 Some time later…

Slide 92

Slide 92 text

Documentation for PHP Contributing to the manual

Slide 93

Slide 93 text

@arnoutboks #dpc18 edit.php.net PHP Docbook Online Editor ‘Easiest’ way to get started (if you can figure out how it works)

Slide 94

Slide 94 text

@arnoutboks #dpc18 edit.php.net

Slide 95

Slide 95 text

@arnoutboks #dpc18 edit.php.net

Slide 96

Slide 96 text

@arnoutboks #dpc18 edit.php.net

Slide 97

Slide 97 text

@arnoutboks #dpc18 1. Find the correct file Under “All files” in the left menu Top-level directory per language • reference// • functions/.xml • .xml • /.xml • language/.xml

Slide 98

Slide 98 text

@arnoutboks #dpc18 1. Find the correct file Tip: Use the “edit” link from the documentation itself

Slide 99

Slide 99 text

@arnoutboks #dpc18 2. Make your changes Documentation is written using DocBook XML http://tdg.docbook.org/tdg/5.2/

Slide 100

Slide 100 text

@arnoutboks #dpc18 2. Make your changes Documentation is written using DocBook XML (or just peek in other docs)

Slide 101

Slide 101 text

@arnoutboks #dpc18 3. Preview changes Seems broken at the moment

Slide 102

Slide 102 text

@arnoutboks #dpc18 4. Submit as patch

Slide 103

Slide 103 text

@arnoutboks #dpc18 Tip: submit a docs patch instead of user note

Slide 104

Slide 104 text

@arnoutboks #dpc18 Don’t like edit.php.net? Make it better!

Slide 105

Slide 105 text

@arnoutboks #dpc18 Don’t like edit.php.net? Or set up your own development environment: https://www.sammyk.me/how-to-contribute-to-php- documentation

Slide 106

Slide 106 text

@arnoutboks #dpc18 Where to help? • Bugs of type “Doc” in bug tracker • check-missing-docs.php • “Failures to meet strict standards” (in edit.php.net sidebar) • Translations • Missing translations • Out-of-date translations • Translations needing review • Translations with errors

Slide 107

Slide 107 text

@arnoutboks #dpc18 Where to help?

Slide 108

Slide 108 text

Recap Ways to contribute to PHP

Slide 109

Slide 109 text

@arnoutboks #dpc18 Numerous ways to contribute • Bug reports • Bug triaging • Running tests • Writing tests • Bug fixes • New features • Documentation patches • Documentation translations • Tools

Slide 110

Slide 110 text

@arnoutboks #dpc18 Why contribute? • Get ‘own’ bugs fixed • Fix ‘own’ bugs • Make PHP better • Make PHP more popular • Learn PHP • Learn C • Give back to the community

Slide 111

Slide 111 text

@arnoutboks #dpc18 You don’t have to be an expert to contribute to the PHP core

Slide 112

Slide 112 text

@arnoutboks #dpc18 Feedback & Questions @arnoutboks @arnoutboks @aboks Arnout Boks Please leave your feedback on joind.in: https://joind.in/talk/4783a

Slide 113

Slide 113 text

@arnoutboks #dpc18 Image Credits • https://photojournal.jpl.nasa.gov/jpeg/PIA19058.jpg • https://www.flickr.com/photos/vivianejl/22990832 • https://www.flickr.com/photos/dhuiz/13609080315/ • https://www.flickr.com/photos/gotovan/8671348572 • https://www.flickr.com/photos/sidelong/246816211 • https://www.flickr.com/photos/pezon64/23764268828 • https://www.flickr.com/photos/crdot/6855538268/ • https://www.flickr.com/photos/borkurdotnet/9682277597