PHP Internals for the Inquisitive Developer

Transcript

1. PHP Internals for the Inquisitive Developer Jeremy Mikola @jmikola

2. A Little About Myself

3. None

4. Some Topics to Cover • Request lifecycle • Types of

   PHP extensions • Internal data structures • Navigating PHP’s source • Executing a PHP file • Opcodes, caching, and optimization • Examining how PHP interacts with C • Debugging crashes
5. None

6. Everything Starts Here* int main(int argc, char *argv[]) { return

   0; }
7. None
8. None
9. None
10. None

11. Server APIs • Apache mod_php • Command Line Interface (CLI)

    • Common Gateway Interface (CGI) ◦ FastCGI allows for persistent processes ◦ Apache mod_fcgid • FastCGI Process Manager (FPM) ◦ Apache mod_proxy_fcgi
12. None

13. Types of Extensions • Extensions (aka “modules”) ◦ Allows new

    concepts to be added to PHP ◦ Integration with C, system libraries ◦ e.g. APCu, MongoDB, OpenSSL • Zend Extensions ◦ More hooks for changing PHP’s behavior ◦ Commonly used for debuggers and profilers ◦ May also register normal extensions for user APIs ◦ e.g. OPCache, Xdebug, phpdbg, Blackfire

14. Module Scope • Module initialization ◦ Allocate persistent read-only globals

    ◦ Register INI entries, classes, constants, etc. ◦ Initialize third-party libraries • Module shutdown ◦ Free persistent allocations ◦ Unregister INI entries

15. Request Scope • Request initialization ◦ Allocate request-bound memory ◦

    Reset globals as needed ◦ Avoid unintentionally altering global state • Request shutdown ◦ Free request-bound allocations ◦ Zend Memory Manager helps catch leaks

16. Process Model

17. Thread Model

18. Global Scope • Globals ◦ Use persistent allocation, initialize to

    zero ◦ Requests access globals via TSRM macros ◦ Zend hash globals can be used as a cache • Process model ◦ GINIT is called once, before MINIT ◦ GSHUTDOWN is called once, during MSHUTDOWN • Thread model ◦ Additionally called each time a thread spawns or dies ◦ TSRM macros take care of thread local storage

19. PHP Process Manager • Uses ReactPHP to manage a pool

    of PHP processes (CLI) • Bootstraps application once per worker process • Each worker handles a series of HTTP requests • Leverages request/response design in frameworks • Operates entirely within one CLI “request” • Issues with memory leaks, resetting state
20. None

21. It looks like you're giving a presentation on PHP internals.

    Would you like help? Talk about zvals

22. Zvals • 16-byte struct consisting of three union fields •

    value (8-bytes) ◦ Integer and double values are stored inline ◦ Pointers are used for other types (e.g. string, array, object) ◦ Not used for null and boolean values (denoted by type_info) • u1 contains type_info (4-bytes) ◦ Type byte and various bit flags • u2 is a multi-purpose union (4-bytes) ◦ Used for hash tables, AST line numbers, foreach iteration, etc.

23. Zval Improvements in PHP 7 • Zvals are no longer

    individually heap-allocated ◦ Can now be directly embedded (e.g. hash buckets) • Zvals are no longer refcounted ◦ Refcounts stored on values themselves (e.g. zend_string) ◦ Values can be shared independently of the zval struct • Much less indirection and pointer traversal

24. Improvements to Other Types • New string representation ◦ zend_string

    struct replaces char* ◦ Encapsulates refcount, string length, and data • Hash table redesigned ◦ Buckets allocated in sequence (less pointer traversal) ◦ Optimizations for “packed arrays” • Objects are more lightweight ◦ Declared properties embedded in zend_object

25. Types and Attributes Refcounted Collectable Copyable Immutable Simple Types String

    ✘ ✘ Array ✘ ✘ ✘ Object ✘ ✘ Resource ✘ Reference ✘ Interned String Immutable Array ✘

26. Navigating PHP

27. None
28. None
29. None
30. None
31. None
32. None
33. None

34. Executing PHP

35. Interpretation vs. Compilation

36. <?php function test() { echo "Hello!\n"; return true; echo "Not

    executed.\n"; } An Admittedly Contrived PHP Script

37. T_OPEN_TAG T_WHITESPACE T_FUNCTION T_WHITESPACE T_STRING ( ) T_WHITESPACE { T_WHITESPACE

    T_ECHO T_WHITESPACE T_CONSTANT_ENCAPSED_STRING ; T_WHITESPACE T_RETURN T_WHITESPACE T_STRING ; T_WHITESPACE T_ECHO T_WHITESPACE T_CONSTANT_ENCAPSED_STRING ; T_WHITESPACE } T_WHITESPACE Parsing Tokens (token_get_all)
38. None

39. Vulcan Logic Dumper

40. Examining Opcodes (vld) function name: test number of ops: 4

    compiled vars: none line #* E I O op fetch ext return operands ----------------------------------------------------- 5 0 E > ECHO 'Hello%21%0A' 6 1 > RETURN <true> 8 2* ECHO 'Not+executed.%0A' 9 3* > RETURN null

41. Opcode Caching • Various caching strategies • OpArrays (i.e. opcode

    sequences) can be optimized • OpArrays still interpreted at runtime • JIT would allow machine code caching

42. Pass Bit Optimization 1 1 << 0 Casts, operators, internal

    functions with constant and literal arguments 2 1 << 1 Type coercion in expressions, conditional elimination (e.g. if statements) 3 1 << 2 Optimize self-assignment, post-increment, and jumps 5 1 << 4 Block optimization of control flow graph (CFG) 9 1 << 8 Optimize usage of temporary variables (register allocation) 10 1 << 9 Remove NOPs - 1 << 14 Collect constants for future replacement OpCode Optimizations https://stackoverflow.com/a/21291587/162228 https://phpinternals.net/categories/opcache

43. Approaching C

44. System Calls • open and close file descriptors • read

    and write files, sockets, devices • fork, exec, or wait on another process • exit the current process • Send signals to other processes (kill) • Map files or devices to memory (mmap, munmap) • Allocate process memory (brk, sbrk)

45. Tracing System Calls (strace) <?php echo "Hello!\n"; $ strace -e

    write php example.php > /dev/null write(1, "Hello!\n", 7) = 7 +++ exited with 0 +++

46. Tracing System Calls (strace) $ strace -e openat php example.php

    2>&1 | grep ini openat(AT_FDCWD, "/usr/bin/php-cli.ini", O_RDONLY) = -1 openat(AT_FDCWD, "/etc/php/7.2/cli/php-cli.ini", O_RDONLY) = -1 openat(AT_FDCWD, "/usr/bin/php.ini", O_RDONLY) = -1 openat(AT_FDCWD, "/etc/php/7.2/cli/php.ini", O_RDONLY) = 3 openat(AT_FDCWD, "/etc/php/7.2/cli/conf.d/10-opcache.ini", … openat(AT_FDCWD, "/etc/php/7.2/cli/conf.d/10-pdo.ini", …

47. Tracing Library Calls (ltrace) $ ltrace -l mongodb.so php example.php

    mongodb.so->mongoc_log_trace_disable(0, 1, 0x55a78b57a410, … mongodb.so->mongoc_log_set_handler(0, 0, 0x55a78b57a410, … mongodb.so->mongoc_init(0, 0, 0x55a78b552bb8, 0x7f1941ec6320 … mongodb.so->_mongoc_openssl_init(0x7ffcf6e60460, … mongodb.so->bson_malloc0(40, 0, 0, 0x7f1940ecf8e0) … mongodb.so->bson_malloc0(40, 0x55a78b42a018, 0x55a78b554bc0, … mongodb.so->_mongoc_counters_init(0x55a78b5a4710, …

48. Debugging Crashes (gdb) • The most common crashes are segfaults

    ◦ C makes it trivially easy to access memory incorrectly • Ideally, crashes produce core dumps, which can be inspected ◦ https://bugs.php.net/bugs-generating-backtrace.php • PHP source includes a .gdbinit file with helpful macros
49. None

50. This’ll Do Just Fine <?php class Crash { public function

    __tostring() { return "".$this; } } "".(new Crash());

51. Capturing a Core Dump $ php class-tostring-recursion.php Segmentation fault (core

    dumped) $ ls class-tostring-recursion.php core $ gdb `which php` core

52. Capturing a Core Dump $ gdb -q --args php class-tostring-recursion.php

    Reading symbols from php...done. (gdb) run Starting program: /usr/bin/php class-tostring-recursion.php Program received signal SIGSEGV, Segmentation fault. 0x0000555555c9ab9e in zend_call_function (fci=<error reading variable: Cannot access memory at address 0x7fffff7fef78>, fci_cache=<error reading variable: Cannot access memory at address 0x7fffff7fef70>) at /tmp/build_php-7.2.6.sx8/php-7.2.6/Zend/zend_execute_API.c:659 (gdb)

53. Traversing the Backtrace (gdb) bt -20 #45993 0x0000555555ce6e8a in zend_call_method

    (object=0x7ffff3423110, #45994 0x0000555555d13de3 in zend_std_cast_object_tostring (readobj=0x #45995 0x0000555555ca5c8f in _zval_get_string_func (op=0x7ffff3423110) #45996 0x0000555555cb4b68 in zend_make_printable_zval (expr=0x7ffff342 #45997 0x0000555555cae0a8 in concat_function (result=0x7ffff3423120, o #45998 0x0000555555d4069a in ZEND_CONCAT_SPEC_CONST_TMPVAR_HANDLER () #45999 0x0000555555db6336 in execute_ex (ex=0x7ffff34230c0) at /tmp/bu #46000 0x0000555555c9b9e1 in zend_call_function (fci=0x7fffffffb0b0, f #46001 0x0000555555ce6e8a in zend_call_method (object=0x7ffff3423090, #46002 0x0000555555d13de3 in zend_std_cast_object_tostring (readobj=0x #46003 0x0000555555ca5c8f in _zval_get_string_func (op=0x7ffff3423090)

54. #45999 0x0000555555db6336 in execute_ex (ex=0x7ffff34230c0) at /tmp/bu #46000 0x0000555555c9b9e1 in

    zend_call_function (fci=0x7fffffffb0b0, f #46001 0x0000555555ce6e8a in zend_call_method (object=0x7ffff3423090, #46002 0x0000555555d13de3 in zend_std_cast_object_tostring (readobj=0x #46003 0x0000555555ca5c8f in _zval_get_string_func (op=0x7ffff3423090) #46004 0x0000555555cb4b68 in zend_make_printable_zval (expr=0x7ffff342 #46005 0x0000555555cae0a8 in concat_function (result=0x7ffff34230b0, o #46006 0x0000555555d4069a in ZEND_CONCAT_SPEC_CONST_TMPVAR_HANDLER () #46007 0x0000555555db6336 in execute_ex (ex=0x7ffff3423030) at /tmp/bu #46008 0x0000555555dbaa7d in zend_execute (op_array=0x7ffff3489300, re #46009 0x0000555555cb8f1e in zend_execute_scripts (type=8, retval=0x0, #46010 0x0000555555befaa9 in php_execute_script (primary_file=0x7fffff #46011 0x0000555555dbd88f in do_cli (argc=2, argv=0x5555568f9380) at / #46012 0x0000555555dbed26 in main (argc=2, argv=0x5555568f9380) at /tm

55. (take a breath)

56. Resources and Further Reading References about Maintaining and Extending PHP

    https://wiki.php.net/internals/references PHP Internals (Thomas Punt) https://phpinternals.net/ Derick Rethans’ Blog https://derickrethans.nl/ Nikita Popov’s Blog https://nikic.github.io/ PHP Internals Book (Nikita and Julien Pauli) http://www.phpinternalsbook.com/

57. Thanks! Jeremy Mikola @jmikola

58. Photo Credits • https://imgur.com/gallery/0hcTtiW • http://inception.wikia.com/wiki/Fischer_inception_job?file=Cobb_using_the_Mr._Charles_tactic.png • http://www.phpinternalsbook.com/ • https://support.cloud.engineyard.com/hc/en-us/articles/205411888-PHP-Performance-I-Everything-You-Need-to-Kno

    w-About-OpCode-Caches • http://www.leonarddavid.com/wp-content/uploads/2015/04/death-grip.jpg • http://weca.mp/2016/images/coach/sara.png