Slide 1
Slide 1 text
Profiling PHP
Michael Heap (@mheap)
Developer at DataSift
Presented at FOSDEM 2016
Slide 2
Slide 2 text
Me!
I’m Michael
I’m @mheap
Developer at DataSift
Slide 3
Slide 3 text
Disclaimer
Slide 4
Slide 4 text
Profiling PHP
Slide 5
Slide 5 text
We Won't Cover
Frontend optimisation
Database queries
TCP Negotiation
Slide 6
Slide 6 text
We Will Cover
Quick wins
Naive profiling
XHProf
XHGUI
Other considerations
Slide 7
Slide 7 text
Quick Wins
Things to do before you start profiling
Slide 8
Slide 8 text
$ php --go-faster
Slide 9
Slide 9 text
$ php --go-faster
$ apt-get upgrade php
$ yum upgrade php
$ pacman -S php
Slide 10
Slide 10 text
Async Workers
Gearman
Beanstalkd
RabbitMQ
ZeroMQ
Slide 11
Slide 11 text
Setting the scene
Slide 12
Slide 12 text
{
“interaction”: {
“content": "Hello World”,
”author”: {
“name”: ”Michael"
}
}
}
Slide 13
Slide 13 text
{
“interaction”: {
“content": "Hello World”,
”author”: {
“name”: ”Michael"
}
}
}
[“interaction.content”, “interaction.author.name”]
Slide 14
Slide 14 text
{
“interaction”: {
“content": "Hello World”,
”author”: {
“name”: ”Michael"
}
}
}
Slide 15
Slide 15 text
{
“interaction”: {
“content": "Hello World”,
”author”: {
“name”: ”Michael"
}
}
}
Slide 16
Slide 16 text
{
“interaction”: {
“content": "Hello World”,
”author”: {
“name”: ”Michael"
}
}
}
Slide 17
Slide 17 text
{
“interaction”: {
“content": "Hello World”,
”author”: {
“name”: ”Michael"
}
}
}
Slide 18
Slide 18 text
{
“interaction”: {
“content": "Hello World”,
”author”: {
“name”: ”Michael"
}
}
}
Slide 19
Slide 19 text
[“interaction.content”]
{
“interaction”: {
“content": "Hello World”,
”author”: {
“name”: ”Michael"
}
}
}
Slide 20
Slide 20 text
{
“interaction”: {
“content": "Hello World”,
”author”: {
“name”: ”Michael"
}
}
}
[“interaction.content”]
Slide 21
Slide 21 text
{
“interaction”: {
“content": "Hello World”,
”author”: {
“name”: ”Michael"
}
}
}
[“interaction.content”]
Slide 22
Slide 22 text
{
“interaction”: {
“content": "Hello World”,
”author”: {
“name”: ”Michael"
}
}
}
[“interaction.content”]
Slide 23
Slide 23 text
{
“interaction”: {
“content": "Hello World”,
”author”: {
“name”: ”Michael"
}
}
}
[“interaction.content”]
Slide 24
Slide 24 text
{
“interaction”: {
“content": "Hello World”,
”author”: {
“name”: ”Michael"
}
}
}
[“interaction.content”]
Slide 25
Slide 25 text
{
“interaction”: {
“content": "Hello World”,
”author”: {
“name”: ”Michael"
}
}
}
[“interaction.content”, “interaction.author.name”]
Slide 26
Slide 26 text
Getting Started
Slide 27
Slide 27 text
$start = microtime(true);
$input = file_get_contents('./small.json');
foreach (explode("\n", $input) as $line){
$tmp = json_decode($line, true);
if ($tmp) { $data[] = $tmp; }
}
echo "Took: ". (microtime(true) - $start);
Slide 28
Slide 28 text
$start = microtime(true);
$input = file_get_contents('./small.json');
foreach (explode("\n", $input) as $line){
$tmp = json_decode($line, true);
if ($tmp) { $data[] = $tmp; }
}
echo "Took: ". (microtime(true) - $start);
Slide 29
Slide 29 text
$start = microtime(true);
$input = file_get_contents('./small.json');
foreach (explode("\n", $input) as $line){
$tmp = json_decode($line, true);
if ($tmp) { $data[] = $tmp; }
}
echo "Took: ". (microtime(true) - $start);
Slide 30
Slide 30 text
$start = microtime(true);
$input = file_get_contents('./small.json');
foreach (explode("\n", $input) as $line){
$tmp = json_decode($line, true);
if ($tmp) { $data[] = $tmp; }
}
echo "Took: ". (microtime(true) - $start);
Slide 31
Slide 31 text
$start = microtime(true);
$input = file_get_contents('./small.json');
foreach (explode("\n", $input) as $line){
$tmp = json_decode($line, true);
if ($tmp) { $data[] = $tmp; }
}
echo "Took: ". (microtime(true) - $start);
Slide 32
Slide 32 text
Run Times
50,000 items: 5.7440850734711
50,000 items: 5.8537809848785
50,000 items: 5.5094730854034
50,000 items: 5.8217489719391
50,000 items: 5.8287329673767
Slide 33
Slide 33 text
$start = microtime(true);
$input = file_get_contents('./small.json');
foreach (explode("\n", $input) as $line){
$tmp = json_decode($line, true);
if ($tmp) { $data[] = $tmp; }
}
echo "Took: ". (microtime(true) - $start);
Slide 34
Slide 34 text
$start = microtime(true);
$input = file_get_contents('./small.json');
foreach (explode("\n", $input) as $line){
$tmp = json_decode($line, true);
if ($tmp) { $data[] = $tmp; }
}
echo "Took: ". (microtime(true) - $start);
Slide 35
Slide 35 text
if (file_exists("./cache")){
$data = json_decode(
file_get_contents("./cache"), true
);
} else {
$input = file_get_contents('./small.json');
foreach (explode("\n", $input) as $line){
// Decode and add to array
}
file_put_contents("./cache",
json_encode($data)
);
}
Slide 36
Slide 36 text
Run Times
Without Cache: 11.717740058899
With Cache: 8.7964880466461
Slide 37
Slide 37 text
Guessing
Slide 38
Slide 38 text
XHProf
sudo pecl install xhprof-0.9.4
extension=xhprof.so
/usr/share/php/xhprof_lib
Slide 39
Slide 39 text
xhprof_enable(
XHPROF_FLAGS_CPU +
XHPROF_FLAGS_MEMORY
)
Slide 40
Slide 40 text
$data = xhprof_disable();
print_r($data);
Slide 41
Slide 41 text
[main()==>json_decode] => Array
(
[ct] => 50001
[wt] => 4379326
[cpu] => 4710383
[mu] => 807834192
[pmu] => 807572808
)
Slide 42
Slide 42 text
include_once '/usr/share/php/xhprof_lib/utils/
xhprof_lib.php';
include_once '/usr/share/php/xhprof_lib/utils/
xhprof_runs.php';
$runs = new XHProfRuns_Default();
$runs->save_run($xhprof_data, "michael");
Slide 43
Slide 43 text
No Cache
Slide 44
Slide 44 text
Total Incl. Wall Time (microsec): 12,638,896 microsecs
Total Incl. CPU (microsecs): 9,692,792 microsecs
Total Incl. MemUse (bytes): 929,483,336 bytes
Total Incl. PeakMemUse (bytes): 1,170,441,880 bytes
Number of Function Calls: 50,008
No Cache
Slide 45
Slide 45 text
Total Incl. Wall Time (microsec): 9,473,467 microsecs
Total Incl. CPU (microsecs): 7,010,122 microsecs
Total Incl. MemUse (bytes): 815,163,360 bytes
Total Incl. PeakMemUse (bytes): 935,510,968 bytes
Number of Function Calls: 5
Cache
Slide 46
Slide 46 text
No Cache
Slide 47
Slide 47 text
No Cache
Slide 48
Slide 48 text
No Cache
Slide 49
Slide 49 text
No Cache
Cache
Slide 50
Slide 50 text
Back to the task
Slide 51
Slide 51 text
{
“interaction”: {
“content": "Hello World”,
”author”: {
“name”: ”Michael"
}
}
}
[“interaction.content”, “interaction.author.name”]
Slide 52
Slide 52 text
Version 1
Slide 53
Slide 53 text
public function generate($interaction, $parents = []) {
$keys = [];
foreach ($interaction as $k => $v) {
$newFields = [];
if (!is_numeric($k)) {
$newFields[] = $k;
}
if (is_array($v) || is_object($v)) {
$keys = array_merge($keys, $this-
>generate($v, array_merge($parents, $newFields)));
} else {
$keys[] = implode(array_merge($parents ,
$newFields), ".");
}
}
return $keys;
}
Slide 54
Slide 54 text
public function generate($interaction, $parents = []) {
$keys = [];
foreach ($interaction as $k => $v) {
$newFields = [];
if (!is_numeric($k)) {
$newFields[] = $k;
}
if (is_array($v) || is_object($v)) {
$keys = array_merge($keys, $this-
>generate($v, array_merge($parents, $newFields)));
} else {
$keys[] = implode(array_merge($parents ,
$newFields), ".");
}
}
return $keys;
}
Slide 55
Slide 55 text
public function generate($interaction, $parents = []) {
$keys = [];
foreach ($interaction as $k => $v) {
$newFields = [];
if (!is_numeric($k)) {
$newFields[] = $k;
}
if (is_array($v) || is_object($v)) {
$keys = array_merge($keys, $this-
>generate($v, array_merge($parents, $newFields)));
} else {
$keys[] = implode(array_merge($parents ,
$newFields), ".");
}
}
return $keys;
}
Slide 56
Slide 56 text
public function generate($interaction, $parents = []) {
$keys = [];
foreach ($interaction as $k => $v) {
$newFields = [];
if (!is_numeric($k)) {
$newFields[] = $k;
}
if (is_array($v) || is_object($v)) {
$keys = array_merge($keys, $this-
>generate($v, array_merge($parents, $newFields)));
} else {
$keys[] = implode(array_merge($parents ,
$newFields), ".");
}
}
return $keys;
}
Slide 57
Slide 57 text
public function generateAll($interactions) {
$all = [];
foreach ($interactions as $i) {
$all = array_merge($all, $this-
>generate($i));
}
return array_filter(array_unique($all));
}
Slide 58
Slide 58 text
public function generateAll($interactions) {
$all = [];
foreach ($interactions as $i) {
$all = array_merge($all, $this-
>generate($i));
}
return array_filter(array_unique($all));
}
Slide 59
Slide 59 text
public function generateAll($interactions) {
$all = [];
foreach ($interactions as $i) {
$all = array_merge($all, $this-
>generate($i));
}
return array_filter(array_unique($all));
}
Slide 60
Slide 60 text
Total Incl. Wall Time (microsec): 8,046,823 microsecs
Total Incl. CPU (microsecs): 8,018,211 microsecs
Total Incl. MemUse (bytes): 19,617,760 bytes
Total Incl. PeakMemUse (bytes): 37,543,384 bytes
Number of Function Calls: 355,764
Slide 61
Slide 61 text
public function generate($interaction, $parents = []) {
$keys = [];
foreach ($interaction as $k => $v) {
$newFields = [];
if (!is_numeric($k)) {
$newFields[] = $k;
}
if (is_array($v) || is_object($v)) {
$keys = array_merge($keys, $this-
>generate($v, array_merge($parents, $newFields)));
} else {
$keys[] = implode(array_merge($parents ,
$newFields), ".");
}
}
return $keys;
}
Slide 62
Slide 62 text
No content
Slide 63
Slide 63 text
No content
Slide 64
Slide 64 text
Version 2
Slide 65
Slide 65 text
public function generate($interaction, $parents = []) {
$keys = [];
foreach ($interaction as $k => $v) {
$newFields = [];
if (!is_numeric($k)) {
$newFields[] = $k;
}
if (is_array($v) || is_object($v)) {
$keys = array_merge($keys, $this-
>generate($v, array_merge($parents, $newFields)));
} else {
$keys[] = implode(array_merge($parents ,
$newFields), ".");
}
}
return $keys;
}
Slide 66
Slide 66 text
public function generate($interaction, $parents = []) {
$keys = [];
foreach ($interaction as $k => $v) {
$newFields = [];
if (!is_numeric($k)) {
$newFields[] = $k;
}
if (is_array($v) || is_object($v)) {
foreach ($this->generate($v,
array_merge($parents, $newFields) as $val) {
$keys[] = $val;
}
} else {
$keys[] = implode(array_merge($parents ,
$newFields), ".");
}
}
return $keys;
}
Slide 67
Slide 67 text
public function generateAll($interactions) {
$all = [];
foreach ($interactions as $i) {
$all = array_merge($all, $this-
>generate($i));
}
return array_filter(array_unique($all));
}
Slide 68
Slide 68 text
public function generateAll($interactions) {
$all = [];
foreach ($interactions as $i) {
foreach ($this->generate($i) as $g){
$all[] = $g;
}
}
return array_filter(array_unique($all));
}
Slide 69
Slide 69 text
Total Incl. Wall Time (microsec): 3,617,157 microsecs
Total Incl. CPU (microsecs): 3,539,472 microsecs
Total Incl. MemUse (bytes): 20,135,232 bytes
Total Incl. PeakMemUse (bytes): 37,407,936 bytes
Number of Function Calls: 334,278
Slide 70
Slide 70 text
No content
Slide 71
Slide 71 text
No content
Slide 72
Slide 72 text
Version 2.5
Slide 73
Slide 73 text
public function generate($interaction, $parents = []) {
$keys = [];
foreach ($interaction as $k => $v) {
$newFields = [];
if (!is_numeric($k)) {
$newFields[] = $k;
}
if ($v instanceof Traversable) {
foreach ($this->generate($v,
array_merge($parents, $newFields) as $val) {
$keys[] = $val;
}
} else {
$keys[] = implode(array_merge($parents ,
$newFields), ".");
}
}
return $keys;
}
Slide 74
Slide 74 text
Total Incl. Wall Time (microsec): 274,998 microsecs
Total Incl. CPU (microsecs): 221,910 microsecs
Total Incl. MemUse (bytes): 19,510,464 bytes
Total Incl. PeakMemUse (bytes): 22,173,048 bytes
Number of Function Calls: 10,922
Slide 75
Slide 75 text
Version 3
Slide 76
Slide 76 text
public function generate($interaction, $parents = []) {
$keys = [];
foreach ($interaction as $k => $v) {
$newFields = [];
if (!is_numeric($k)) {
$newFields[] = $k;
}
if (is_array($v) || is_object($v)) {
foreach ($this->generate($v,
array_merge($parents, $newFields) as $val) {
$keys[] = $val;
}
} else {
$keys[] = implode(array_merge($parents ,
$newFields), ".");
}
}
return $keys;
}
Slide 77
Slide 77 text
public function generate($interaction, $parents = []) {
$keys = [];
foreach ($interaction as $k => $v) {
$newFields = [];
if (!is_numeric($k)) {
$newFields[] = $k;
}
if (!is_scalar($v)) {
foreach ($this->generate($v,
array_merge($parents, $newFields) as $val) {
$keys[] = $val;
}
} else {
$keys[] = implode(array_merge($parents ,
$newFields), ".");
}
}
return $keys;
}
Slide 78
Slide 78 text
Total Incl. Wall Time (microsec): 2,926,818 microsecs
Total Incl. CPU (microsecs): 2,857,804 microsecs
Total Incl. MemUse (bytes): 20,126,960 bytes
Total Incl. PeakMemUse (bytes): 37,362,384 bytes
Number of Function Calls: 284,214
Slide 79
Slide 79 text
Total Incl. Wall Time (microsec): 3,617,157 microsecs
Total Incl. CPU (microsecs): 3,539,472 microsecs
Total Incl. MemUse (bytes): 20,135,232 bytes
Total Incl. PeakMemUse (bytes): 37,407,936 bytes
Number of Function Calls: 334,278
Slide 80
Slide 80 text
Total Incl. Wall Time (microsec): 2,926,818 microsecs
Total Incl. CPU (microsecs): 2,857,804 microsecs
Total Incl. MemUse (bytes): 20,126,960 bytes
Total Incl. PeakMemUse (bytes): 37,362,384 bytes
Number of Function Calls: 284,214
Slide 81
Slide 81 text
Version 4
Slide 82
Slide 82 text
Total Incl. Wall Time (microsec): 8,046,823 microsecs
Total Incl. CPU (microsecs): 8,018,211 microsecs
Total Incl. MemUse (bytes): 19,617,760 bytes
Total Incl. PeakMemUse (bytes): 37,543,384 bytes
Number of Function Calls: 355,764
Slide 83
Slide 83 text
Total Incl. Wall Time (microsec): 1,274,903 microsecs
Total Incl. CPU (microsecs): 1,195,136 microsecs
Total Incl. MemUse (bytes): 20,086,232 bytes
Total Incl. PeakMemUse (bytes): 34,597,576 bytes
Number of Function Calls: 93,053
Slide 84
Slide 84 text
$currentDepth = [];
foreach($parents as $x){ $currentDepth[] = $x; }
foreach($newFields as $x){ $currentDepth[] = $x; }
Slide 85
Slide 85 text
$xx = \gettype($v)[0];
if ($xx == 'a' || $xx == 'o') {
Slide 86
Slide 86 text
The UI
Slide 87
Slide 87 text
No content
Slide 88
Slide 88 text
No content
Slide 89
Slide 89 text
No content
Slide 90
Slide 90 text
No content
Slide 91
Slide 91 text
XHGui
Slide 92
Slide 92 text
No content
Slide 93
Slide 93 text
No content
Slide 94
Slide 94 text
No content
Slide 95
Slide 95 text
No content
Slide 96
Slide 96 text
blackfire.io
Slide 97
Slide 97 text
No content
Slide 98
Slide 98 text
No content
Slide 99
Slide 99 text
No content
Slide 100
Slide 100 text
No content
Slide 101
Slide 101 text
No content
Slide 102
Slide 102 text
No content
Slide 103
Slide 103 text
Other Tools
XDebug/Cachegrind
tideways.io
New Relic
catchy.io
PHP Bench
Slide 104
Slide 104 text
Optimisation
Slide 105
Slide 105 text
Caching
The fastest code is code that doesn't run
Slide 106
Slide 106 text
Should it run?
The fastest code is code that doesn't run
Slide 107
Slide 107 text
Measuring Production
Enable 1 in 1000 requests (XHProf)
Blackfire production mode (untested)
Slide 108
Slide 108 text
Logging
Work out which code paths are popular
Optimise them
Slide 109
Slide 109 text
Track Everything
Used/available memory
Response time
Timeout
Successful/failed requests
Open socket count
Bandwidth usage
Slide 110
Slide 110 text
Considerations
Cache stampedes
Circuit breakers
Slide 111
Slide 111 text
Process
Measure
Refactor
Measure again
Without measurements, you're just guessing
Slide 112
Slide 112 text
Is it worth it?
Slide 113
Slide 113 text
Thanks!
I’ve been @mheap, you’ve been awesome.
Please leave feedback on Joind.in
https://joind.in/16814