How to serialize()
a \Closure and Get
a Million Downloads
by Jeremy Lindblom – @jeremeamia
Slide 2
Slide 2 text
How to serialize()
a \Closure and Get
3.6 Million Downloads
by Jeremy Lindblom – @jeremeamia
Slide 3
Slide 3 text
Let me tell you about myself...
Slide 4
Slide 4 text
@jeremeamia
Jeremy Lindblom the Software Developer
Slide 5
Slide 5 text
@jeremeamia
Jeremy Lindblom the Software Developer
AWS SDK for PHP at Amazon
@awsforphp
Slide 6
Slide 6 text
@jeremeamia
Jeremy Lindblom the Software Developer
AWS SDK for PHP at Amazon
@awsforphp
President of the Seattle PHP User Group
@seaphp & #pnwphp
Slide 7
Slide 7 text
@jeremeamia
Jeremy Lindblom the Software Developer
AWS SDK for PHP at Amazon
@awsforphp
President of the Seattle PHP User Group
@seaphp & #pnwphp
PHPoetry and PHParodies
@phpbard
Slide 8
Slide 8 text
Once upon a time...
Slide 9
Slide 9 text
No content
Slide 10
Slide 10 text
No content
Slide 11
Slide 11 text
No content
Slide 12
Slide 12 text
No content
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
Closures
Slide 15
Slide 15 text
Closures
—or—
Lambdas (λ)
Slide 16
Slide 16 text
Closures
—or—
Lambdas
—or—
Anonymous Functions
Slide 17
Slide 17 text
function ($a, $b) {
return $a + $b;
};
Anonymous (a.k.a. no name)
is_callable($closure);
// true
is_object($closure);
// true
get_class($closure);
// Closure
Closure Objects
“This fact used to be considered an implementation
detail, but it can now be relied upon.” - PHP Manual
Fatal error: Uncaught
exception 'Exception' with
message 'Serialization of
'Closure' is not allowed'
Slide 31
Slide 31 text
Back to the Story
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
serialize($closure);
Slide 34
Slide 34 text
Fatal error: Uncaught
exception 'Exception' with
message 'Serialization of
'Closure' is not allowed'
Slide 35
Slide 35 text
Challenge Accepted
Slide 36
Slide 36 text
No content
Slide 37
Slide 37 text
No content
Slide 38
Slide 38 text
No content
Slide 39
Slide 39 text
No content
Slide 40
Slide 40 text
No content
Slide 41
Slide 41 text
Really ?
Slide 42
Slide 42 text
No content
Slide 43
Slide 43 text
Thanks, Laravel!
Slide 44
Slide 44 text
SuperClosure Today
Slide 45
Slide 45 text
How to Get
a Million Downloads
Slide 46
Slide 46 text
★ Solve a problem
★ Find a niche
★ Make it easy to use
How to Get a Million Downloads
If you build it, they will come
Slide 47
Slide 47 text
★ Build trust in your project
○ Address issues promptly
○ Follow SemVer
★ Build a community
○ Become a dependency to a larger project
○ Recruit contributors; reduce “bus factor”
How to Get a Million Downloads (cont.)
If you maintain it, they will stay
Slide 48
Slide 48 text
How to serialize()
a \Closure
Slide 49
Slide 49 text
$fn = function ($a) use ($b, $c) {
return ($a + $b) * $c;
};
What we need to serialize
We need to know
the values of these.
$context
Slide 50
Slide 50 text
$fn = function ($a) use ($b, $c) {
return ($a + $b) * $c;
};
What we need to serialize (cont.)
And we need to
know all of this.
$code
Slide 51
Slide 51 text
extract($context);
eval(“\$closure = {$code};”);
How do we unserialize?
Slide 52
Slide 52 text
Interface for Serializing
class SerializableClosure
extends Closure
implements Serializable
{
// ...
}
Slide 53
Slide 53 text
Fatal error: Class
SerializableClosure may
not inherit from
final class (Closure)
Slide 54
Slide 54 text
No content
Slide 55
Slide 55 text
Interface for Serializing (Decorator)
class SerializableClosure implements Serializable
{
private $closure;
function __construct($closure) {…}
function __invoke() {…}
function serialize() {…}
function unserialize($serialized) {…}
}
Slide 56
Slide 56 text
Context is easiest to get
$a = 5; $b = 10;
$fn = function () use ($a, $b) { … };
$r = new ReflectionFunction($fn);
$context = $r->getStaticVariables();
// [‘a’ => 5, ‘b’ => 10]
Slide 57
Slide 57 text
Context is easiest to get… maybe
$a = 5; $b = 10;
$fn = function () use ($a, $b) {
static $c = 15;
};
// [‘a’ => 5, ‘b’ => 10, ‘c’ => 15]
Slide 58
Slide 58 text
OK, How do we get the code?
$r = new ReflectionFunction($closure);
$file = $r->getFileName();
$start = $r->getStartLine();
$end = $r->getEndLine();
$code = implode(‘’, array_slice(
file($file), $start, $end-$start+1));
Slide 59
Slide 59 text
token_get_all()
parses the given source string into
PHP language tokens using the
Zend engine's lexical scanner.
Slide 60
Slide 60 text
Use the lexer
$tokens = token_get_all(‘
Slide 61
Slide 61 text
Create a parser
foreach ($tokens as $token) {
switch ($step) {
case 0:
if ($token[0] === T_FUNCTION) {
$keep[] = $token; $step++;
}
break;
// More steps...
}
}
Slide 62
Slide 62 text
Create a parser (cont.)
★ Throw out tokens before 1st T_FUNCTION
★ Analyze function signature
○ Verify names of “used” variables (i.e., the context)
★ Keep all tokens within function body
○ Track the brace level to help identify the end.
Slide 63
Slide 63 text
Token parser pitfalls
★ Assumes first T_FUNCTION encountered
★ No context outside of function
○ What namespace/class is it in?
○ What happens to magic constants __CLASS__?
Slide 64
Slide 64 text
Use an AST
★ AST = Abstract Syntax Tree
★ nikic/PHP-Parser is awesome!
★ Create tree data structure from entire file
○ Traverse to closure, keeping track of scopes
○ Replace some nodes with literal values as visited
Slide 65
Slide 65 text
Example - Original
namespace Foo;
use Bar\Baz;
class Fizz
{
public function buzz() {
return function (Baz $baz) {
return $baz->boop(__CLASS__);
}
}
}
Slide 66
Slide 66 text
Example - Token-based Result
function (Baz $baz) {
return $baz->boop(__CLASS__);
}
Code no longer
knows the FQCN.
ERROR
Value is different
from original function
WTF
Slide 67
Slide 67 text
Example - AST-based Result
function (Bar\Baz $baz) {
return $baz->boop(‘Foo\Fizz’);
}
FQCN injected into
code. No loss of
information.
Magic constants
replaced with literal
values. No loss of
information.
Slide 68
Slide 68 text
AST parser pitfalls
★ SLOW. Really slow.
Slide 69
Slide 69 text
Parsing Caveats
★ Two closures declared on the same line is
too ambiguous handle. Oh well…
★ If closures use (&$vales, &$by, &$ref)
then those references cannot be restored
○ Unless ref is to the closure (e.g., recursive closures)
Slide 70
Slide 70 text
Unsolved Mysteries
★ If a closure is declared in a trait, how do
you resolve the value of __CLASS__ to a
literal value?
(If you figure this out, please let me know.)
Slide 71
Slide 71 text
Oh crap!
What about bindings?
(Curse you, PHP 5.4!)
Slide 72
Slide 72 text
class Foo {
private $bar;
public function baz() {
return function () {
return $this->bar;
};
}
}
Closure Bindings
Automatically
creates a closure
on $this.
Slide 73
Slide 73 text
class Foo {
private $bar;
public function baz() {
return function () {
return 5;
};
}
}
Closure Bindings
Automatically
creates a closure
on $this
Even when you
don’t need it.
Slide 74
Slide 74 text
class Foo {
private $bar;
public function baz() {
return function () {
return 5;
};
}
}
Closure Bindings
Automatically
creates a closure
on $this.
Even when you
don’t need it.
UNLESS...
Slide 75
Slide 75 text
class Foo {
private $bar;
public function baz() {
return static function () {
return 5;
};
}
}
Static Closures
UNLESS...
You make it static.
Slide 76
Slide 76 text
No content
Slide 77
Slide 77 text
$foo = new Foo;
$letters = new ArrayObject(range(‘a’, ‘z’));
$fn = $foo->baz(); // baz() returns a static closure
$fn->bindTo($letters);
“Warning: Cannot bind an
instance to a static closure”
Static is forever
Slide 78
Slide 78 text
Handling Closure Bindings
★ Two parts:
○ Object ($this) is the object it is bound to.
○ Scope determines what is visible.
★ Affects usage of $this, self::, and
static:: within the closure
Slide 79
Slide 79 text
Getting Binding Information
class Foo { public function bar() {
return function () { … };
}}
$foo = new Foo();
$fn = $foo->bar();
$r = new ReflectionFunction($fn);
$r->getClosureThis(); // $foo
$r->getClosureScopeClass()->getName(); // “Foo”
Slide 80
Slide 80 text
Setting Binding Information
$newClosure = $closure->bindTo($newThis, $newScope);
Notes:
★ You can omit $newScope, which means “keep the scope
the same as it is now”.
★ Using null for $newThis makes the closure static.
Remember, static is forever.
Slide 81
Slide 81 text
(Un)Serializing Closure Bindings
★ Including object and scope in serialization
★ Rebind closure after unserialization
○ Tricky, because the object can be null in two cases
■ It’s static
■ It was not declared within a class
Slide 82
Slide 82 text
How to unserialize()
a \Closure and Scare
the Hell Out of Everyone
Slide 83
Slide 83 text
public function unserialize($serialized) {
// Unserialize and reconstruct the closure.
$this->data = unserialize($serialized);
$this->closure = _reconstruct_closure($this->data);
// Rebind the closure to its former binding and scope.
if ($this->data['binding'] || $this->data['isStatic']) {
$this->closure = $this->closure->bindTo(
$this->data['binding'],
$this->data['scope']
);
}
}
Slide 84
Slide 84 text
And now, the most
frightening function
ever written...
The End
★ Closures are weird
★ Serializing closures “is not allowed”, but you
can still do it, if you really want to.
★ Reflection is awesome; so is PHP-Parser
★ Open Source is fun
Slide 91
Slide 91 text
How to serialize()
a \Closure and Get
a Million Downloads
by Jeremy Lindblom – @jeremeamia