How to debug Xdebug... or any other weird bug in PHP

What to do when the debugger has a bug? That's the question I had to answer when my GitHub notifications went haywire because a new version of Xdebug was crashing FrankenPHP and therefore all the projects that use it.

Together, we'll retrace the epic story of this debugging debugger and discover a method that can debug just about any bug:

- isolate the problem
- code a "minimal reproducer"
- install a development environment that allows you to debug PHP and its extensions
- use the right tools to understand what's going on
- make a detailed bug report
- establish a theory
- write a patch
- contribute the patch to the upstream project

This method can be used to debug any problem, in any language! We'll take advantage of this adventure to discover the inner workings of the PHP runtime engine, and its extensions, as well as some debugging tools, in particular GDB.

Transcript

1. How to Debug Bizarre PHP Bugs Photo by Dmitry Bukhantsov
2. Kévin Dunglas ➔ Co-founder of Les-Tilleuls.coop ➔ Creator of FrankenPHP and API Platform ➔ Co-maintainer of Caddy and Symfony ➔ Contributor to PHP, and by necessity, to Xdebug 🐞 @dunglas
3. PHP, Web and Cloud Experts ➔ Development: PHP, JS, Go, Rust, C... ➔ DevOps and SRE ➔ Consultancy and maintenance ➔ Agile management, UX and UI design… ➔ contact@les-tilleuls.coop 💌
4. Where's the Bug?
5. As a user: is the bug in my code or in the library I'm using? There might be a bug in the lib (but could be in your code) The bug is probably in your code (but could be in the lib) Is this lib popular? Has a new version been published recently? Does downgrading solve this issue? Are you using the lib in a weird or advanced way? Have others users reported this bug? YES NO YES NO YES NO YES NO NO YES
6. As a maintainer: is the bug in my lib or in user’s code? The bug is likely in their code It is a regular contributor? How many users reported the bug? SOME MANY A SINGLE USER NO YES
7. Bizarre Bugs!
8. A bizarre bug
9. More Bizarre Bugs
10. 🔥🔥
11. Reminder The bug is likely in their code It is a regular contributor? How many users reported the bug? SOME MANY A SINGLE USER NO YES W e’re here!
12. But What Is Xdebug Anyway?
13. A must-have for PHP development: ➔ Step debugging ➔ Better error reporting ➔ Function calls tracing ➔ Profiling ➔ Code coverage ➔ Installed by default with Symfony Docker, API Platform and Laravel Herd ➔ Supported by PhpStorm, VSCode and (Neo)vim Xdebug
16. Asking For Help
17. Let’s Try To Find the Issue Together!
18. Step 1: Reproducing
19. A Minimal Reproducer <?php // index.php error_reporting(0); sleep(1); phpinfo(); // In the same directory, with Xdebug enabled: // frankenphp php-server
20. Step 2: Gathering Information
22. ➔ Debug symbols • Make stack traces readable • Available as extra packages for some (but not all) distributions ◦ Debian: "How To Get A Backtrace" • Not available (yet?) for official Docker images: docker-library/php#1538 ➔ The GNU Debugger (or lldb on Mac) Let Me Introduce You To:
23. My Reco: Compile PHP with Debug Symbols From Source gh repo clone php/php-src cd php-src/ ./buildconf -f ./configure \ --enable-embed \ --enable-zts \ --disable-zend-signals \ --enable-debug make sudo make install
24. Compile FrankenPHP to Use Our libphp gh repo clone dunglas/frankenphp cd frankenphp/caddy/frankenphp export PATH="/usr/local/bin/:$PATH" export CGO_CFLAGS=$(php-config --includes) export CGO_LDFLAGS="$(php-config --ldflags) $(php-config --libs)" go build
25. Compile Xdebug gh repo clone xdebug/xdebug cd xdebug git checkout 3.3.1 phpize ./configure make #sudo make install
26. Load Xdebug ; php.ini, in the root directory of the project zend_extension = /path/to/xdebug.so xdebug_mode = debug xdebug.start_with_request = yes
27. Run GDB to Gather a Stack Trace gdb --args ./frankenphp run # Or on Apple Silicon lldb -- ./frankenphp run # “run” to start or resume execution # “bt” to show the back trace # “break example.c:17” to set a breakpoint # “p” to print variable values
29. When You Can’t Use GDB… Sometimes (e.g. in prod) you can’t install GDB. ➔ Enable verbose logs ➔ In case of segfault, enable core dumps: ulimit -S -c unlimited ➔ lsof: list open files and network sockets ➔ strace: list system calls (I/O…) ➔ top / htop: inspect CPU and memory usage ➔ …
30. Step 3: Doing Our Research
31. ➔ Symfony Docker and Laravel Octane uses FrankenPHP ➔ FrankenPHP uses POSIX threads and GoRoutines (GoRoutines also use threads) ➔ Using threads requires a thread-safe (ZTS) build of PHP and of PHP extensions ➔ PHP extensions must be compatible with non-thread-safe (NTS) and thread-safe (ZTS) builds of PHP Some Context
33. The PHP Parallelism Model (NTS)
34. The PHP Parallelism Model (ZTS)
35. From The PHP Internals Book
36. Step 4: Develop Theories
38. grep -r error_reporting xdebug static void xdebug_base_overloaded_functions_restore(void) { // ... if (XG_BASE(orig_error_reporting_func)) { orig = zend_hash_str_find_ptr( EG(function_table), "error_reporting", sizeof("error_reporting") - 1 ); if (orig) { orig->internal_function.handler = XG_BASE(orig_error_reporting_func); // ...
40. Step 5: Write and Submit a Patch
41. 🎉
43. Summary ➔ Step 0: COMMUNICATE ➔ Step 1: gather evidences to create a reproducer ➔ Step 2: use the right tools ➔ Step 3: use a search engine (and read books) ➔ Step 4: develop theories ➔ Step 5: contribute!
44. Thank you! Come to our booth or call us if you have weird bugs you can’t fix! les-tilleuls.coop contact@les-tilleuls.coop @coopTilleuls