PHP Internals for the Inquisitive Developer

PHP Internals for the Inquisitive Developer

Presented February 8, 2019 at Sunshine PHP.

Presented December 4, 2018 at Lisbon PHP meetup.

Presented October 11, 2018 at Symfony Loves PHP 2018 - San Francisco.

F23700b51dc0c196c1dc02f84aeeecdf?s=128

Jeremy Mikola

February 08, 2019
Tweet

Transcript

  1. PHP Internals for the Inquisitive Developer Jeremy Mikola @jmikola

  2. A Little About Myself

  3. None
  4. Some Topics to Cover • Request lifecycle • Types of

    PHP extensions • Internal data structures • Navigating PHP’s source • Executing a PHP file • Opcodes, caching, and optimization • Examining how PHP interacts with C • Debugging crashes
  5. None
  6. Everything Starts Here* int main(int argc, char *argv[]) { return

    0; }
  7. None
  8. None
  9. None
  10. None
  11. Server APIs • Apache mod_php • Command Line Interface (CLI)

    • Common Gateway Interface (CGI) ◦ FastCGI allows for persistent processes ◦ Apache mod_fcgid • FastCGI Process Manager (FPM) ◦ Apache mod_proxy_fcgi
  12. None
  13. Types of Extensions • Extensions (aka “modules”) ◦ Allows new

    concepts to be added to PHP ◦ Integration with C, system libraries ◦ e.g. APCu, MongoDB, OpenSSL • Zend Extensions ◦ More hooks for changing PHP’s behavior ◦ Commonly used for debuggers and profilers ◦ May also register normal extensions for user APIs ◦ e.g. OPCache, Xdebug, phpdbg, Blackfire
  14. Module Scope • Module initialization ◦ Allocate persistent read-only globals

    ◦ Register INI entries, classes, constants, etc. ◦ Initialize third-party libraries • Module shutdown ◦ Free persistent allocations ◦ Unregister INI entries
  15. Request Scope • Request initialization ◦ Allocate request-bound memory ◦

    Reset globals as needed ◦ Avoid unintentionally altering global state • Request shutdown ◦ Free request-bound allocations ◦ Zend Memory Manager helps catch leaks
  16. Process Model

  17. Thread Model

  18. Global Scope • Globals ◦ Use persistent allocation, initialize to

    zero ◦ Requests access globals via TSRM macros ◦ Zend hash globals can be used as a cache • Process model ◦ GINIT is called once, before MINIT ◦ GSHUTDOWN is called once, during MSHUTDOWN • Thread model ◦ Additionally called each time a thread spawns or dies ◦ TSRM macros take care of thread local storage
  19. PHP Process Manager • Uses ReactPHP to manage a pool

    of PHP processes (CLI) • Bootstraps application once per worker process • Each worker handles a series of HTTP requests • Leverages request/response design in frameworks • Operates entirely within one CLI “request” • Issues with memory leaks, resetting state
  20. None
  21. It looks like you're giving a presentation on PHP internals.

    Would you like help? Talk about zvals
  22. Zvals • 16-byte struct consisting of three union fields •

    value (8-bytes) ◦ Integer and double values are stored inline ◦ Pointers are used for other types (e.g. string, array, object) ◦ Not used for null and boolean values (denoted by type_info) • u1 contains type_info (4-bytes) ◦ Type byte and various bit flags • u2 is a multi-purpose union (4-bytes) ◦ Used for hash tables, AST line numbers, foreach iteration, etc.
  23. Zval Improvements in PHP 7 • Zvals are no longer

    individually heap-allocated ◦ Can now be directly embedded (e.g. hash buckets) • Zvals are no longer refcounted ◦ Refcounts stored on values themselves (e.g. zend_string) ◦ Values can be shared independently of the zval struct • Much less indirection and pointer traversal
  24. Improvements to Other Types • New string representation ◦ zend_string

    struct replaces char* ◦ Encapsulates refcount, string length, and data • Hash table redesigned ◦ Buckets allocated in sequence (less pointer traversal) ◦ Optimizations for “packed arrays” • Objects are more lightweight ◦ Declared properties embedded in zend_object
  25. Types and Attributes Refcounted Collectable Copyable Immutable Simple Types String

    ✘ ✘ Array ✘ ✘ ✘ Object ✘ ✘ Resource ✘ Reference ✘ Interned String Immutable Array ✘
  26. Navigating PHP

  27. None
  28. None
  29. None
  30. None
  31. None
  32. None
  33. None
  34. Executing PHP

  35. Interpretation vs. Compilation

  36. <?php function test() { echo "Hello!\n"; return true; echo "Not

    executed.\n"; } An Admittedly Contrived PHP Script
  37. T_OPEN_TAG T_WHITESPACE T_FUNCTION T_WHITESPACE T_STRING ( ) T_WHITESPACE { T_WHITESPACE

    T_ECHO T_WHITESPACE T_CONSTANT_ENCAPSED_STRING ; T_WHITESPACE T_RETURN T_WHITESPACE T_STRING ; T_WHITESPACE T_ECHO T_WHITESPACE T_CONSTANT_ENCAPSED_STRING ; T_WHITESPACE } T_WHITESPACE Parsing Tokens (token_get_all)
  38. None
  39. Vulcan Logic Dumper

  40. Examining Opcodes (vld) function name: test number of ops: 4

    compiled vars: none line #* E I O op fetch ext return operands ----------------------------------------------------- 5 0 E > ECHO 'Hello%21%0A' 6 1 > RETURN <true> 8 2* ECHO 'Not+executed.%0A' 9 3* > RETURN null
  41. Opcode Caching • Various caching strategies • OpArrays (i.e. opcode

    sequences) can be optimized • OpArrays still interpreted at runtime • JIT would allow machine code caching
  42. Pass Bit Optimization 1 1 << 0 Casts, operators, internal

    functions with constant and literal arguments 2 1 << 1 Type coercion in expressions, conditional elimination (e.g. if statements) 3 1 << 2 Optimize self-assignment, post-increment, and jumps 5 1 << 4 Block optimization of control flow graph (CFG) 9 1 << 8 Optimize usage of temporary variables (register allocation) 10 1 << 9 Remove NOPs - 1 << 14 Collect constants for future replacement OpCode Optimizations https://stackoverflow.com/a/21291587/162228 https://phpinternals.net/categories/opcache
  43. Approaching C

  44. System Calls • open and close file descriptors • read

    and write files, sockets, devices • fork, exec, or wait on another process • exit the current process • Send signals to other processes (kill) • Map files or devices to memory (mmap, munmap) • Allocate process memory (brk, sbrk)
  45. Tracing System Calls (strace) <?php echo "Hello!\n"; $ strace -e

    write php example.php > /dev/null write(1, "Hello!\n", 7) = 7 +++ exited with 0 +++
  46. Tracing System Calls (strace) $ strace -e openat php example.php

    2>&1 | grep ini openat(AT_FDCWD, "/usr/bin/php-cli.ini", O_RDONLY) = -1 openat(AT_FDCWD, "/etc/php/7.2/cli/php-cli.ini", O_RDONLY) = -1 openat(AT_FDCWD, "/usr/bin/php.ini", O_RDONLY) = -1 openat(AT_FDCWD, "/etc/php/7.2/cli/php.ini", O_RDONLY) = 3 openat(AT_FDCWD, "/etc/php/7.2/cli/conf.d/10-opcache.ini", … openat(AT_FDCWD, "/etc/php/7.2/cli/conf.d/10-pdo.ini", …
  47. Tracing Library Calls (ltrace) $ ltrace -l mongodb.so php example.php

    mongodb.so->mongoc_log_trace_disable(0, 1, 0x55a78b57a410, … mongodb.so->mongoc_log_set_handler(0, 0, 0x55a78b57a410, … mongodb.so->mongoc_init(0, 0, 0x55a78b552bb8, 0x7f1941ec6320 … mongodb.so->_mongoc_openssl_init(0x7ffcf6e60460, … mongodb.so->bson_malloc0(40, 0, 0, 0x7f1940ecf8e0) … mongodb.so->bson_malloc0(40, 0x55a78b42a018, 0x55a78b554bc0, … mongodb.so->_mongoc_counters_init(0x55a78b5a4710, …
  48. Debugging Crashes (gdb) • The most common crashes are segfaults

    ◦ C makes it trivially easy to access memory incorrectly • Ideally, crashes produce core dumps, which can be inspected ◦ https://bugs.php.net/bugs-generating-backtrace.php • PHP source includes a .gdbinit file with helpful macros
  49. None
  50. This’ll Do Just Fine <?php class Crash { public function

    __tostring() { return "".$this; } } "".(new Crash());
  51. Capturing a Core Dump $ php class-tostring-recursion.php Segmentation fault (core

    dumped) $ ls class-tostring-recursion.php core $ gdb `which php` core
  52. Capturing a Core Dump $ gdb -q --args php class-tostring-recursion.php

    Reading symbols from php...done. (gdb) run Starting program: /usr/bin/php class-tostring-recursion.php Program received signal SIGSEGV, Segmentation fault. 0x0000555555c9ab9e in zend_call_function (fci=<error reading variable: Cannot access memory at address 0x7fffff7fef78>, fci_cache=<error reading variable: Cannot access memory at address 0x7fffff7fef70>) at /tmp/build_php-7.2.6.sx8/php-7.2.6/Zend/zend_execute_API.c:659 (gdb)
  53. Traversing the Backtrace (gdb) bt -20 #45993 0x0000555555ce6e8a in zend_call_method

    (object=0x7ffff3423110, #45994 0x0000555555d13de3 in zend_std_cast_object_tostring (readobj=0x #45995 0x0000555555ca5c8f in _zval_get_string_func (op=0x7ffff3423110) #45996 0x0000555555cb4b68 in zend_make_printable_zval (expr=0x7ffff342 #45997 0x0000555555cae0a8 in concat_function (result=0x7ffff3423120, o #45998 0x0000555555d4069a in ZEND_CONCAT_SPEC_CONST_TMPVAR_HANDLER () #45999 0x0000555555db6336 in execute_ex (ex=0x7ffff34230c0) at /tmp/bu #46000 0x0000555555c9b9e1 in zend_call_function (fci=0x7fffffffb0b0, f #46001 0x0000555555ce6e8a in zend_call_method (object=0x7ffff3423090, #46002 0x0000555555d13de3 in zend_std_cast_object_tostring (readobj=0x #46003 0x0000555555ca5c8f in _zval_get_string_func (op=0x7ffff3423090)
  54. #45999 0x0000555555db6336 in execute_ex (ex=0x7ffff34230c0) at /tmp/bu #46000 0x0000555555c9b9e1 in

    zend_call_function (fci=0x7fffffffb0b0, f #46001 0x0000555555ce6e8a in zend_call_method (object=0x7ffff3423090, #46002 0x0000555555d13de3 in zend_std_cast_object_tostring (readobj=0x #46003 0x0000555555ca5c8f in _zval_get_string_func (op=0x7ffff3423090) #46004 0x0000555555cb4b68 in zend_make_printable_zval (expr=0x7ffff342 #46005 0x0000555555cae0a8 in concat_function (result=0x7ffff34230b0, o #46006 0x0000555555d4069a in ZEND_CONCAT_SPEC_CONST_TMPVAR_HANDLER () #46007 0x0000555555db6336 in execute_ex (ex=0x7ffff3423030) at /tmp/bu #46008 0x0000555555dbaa7d in zend_execute (op_array=0x7ffff3489300, re #46009 0x0000555555cb8f1e in zend_execute_scripts (type=8, retval=0x0, #46010 0x0000555555befaa9 in php_execute_script (primary_file=0x7fffff #46011 0x0000555555dbd88f in do_cli (argc=2, argv=0x5555568f9380) at / #46012 0x0000555555dbed26 in main (argc=2, argv=0x5555568f9380) at /tm
  55. (take a breath)

  56. Resources and Further Reading References about Maintaining and Extending PHP

    https://wiki.php.net/internals/references PHP Internals (Thomas Punt) https://phpinternals.net/ Derick Rethans’ Blog https://derickrethans.nl/ Nikita Popov’s Blog https://nikic.github.io/ PHP Internals Book (Nikita and Julien Pauli) http://www.phpinternalsbook.com/
  57. Thanks! Jeremy Mikola @jmikola

  58. Photo Credits • https://imgur.com/gallery/0hcTtiW • http://inception.wikia.com/wiki/Fischer_inception_job?file=Cobb_using_the_Mr._Charles_tactic.png • http://www.phpinternalsbook.com/ • https://support.cloud.engineyard.com/hc/en-us/articles/205411888-PHP-Performance-I-Everything-You-Need-to-Kno

    w-About-OpCode-Caches • http://www.leonarddavid.com/wp-content/uploads/2015/04/death-grip.jpg • http://weca.mp/2016/images/coach/sara.png