Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to Serialize a Closure and Get a Million Downloads

How to Serialize a Closure and Get a Million Downloads

Did you know that you cannot serialize a Closure object! Well, that is what the PHP runtime tells you when you try to do it, anyway. Back in 2010, I set out to prove PHP wrong with my SuperClosure project, which allows you to serialize—and then actually unserialize and execute—PHP closures. I’ve learned a lot along the way, especially about about Closures, Reflection, and lexical parsing. I want to tell you the whole story, teach you some of PHP’s dark magic, and show you how some code snippets in a blog post evolved into a quirky open source project that has been downloaded over two million times, despite its usage of the infamous eval().

Jeremy Lindblom

May 21, 2015
Tweet

More Decks by Jeremy Lindblom

Other Decks in Programming

Transcript

  1. @jeremeamia Jeremy Lindblom the Software Developer AWS SDK for PHP

    at Amazon @awsforphp President of the Seattle PHP User Group @seaphp & #pnwphp
  2. @jeremeamia Jeremy Lindblom the Software Developer AWS SDK for PHP

    at Amazon @awsforphp President of the Seattle PHP User Group @seaphp & #pnwphp PHPoetry and PHParodies @phpbard
  3. $add = function ($a, $b) { return $a + $b;

    }; $result = $add(3, 7); // 10 First-class Functions
  4. $b = 7; $add7 = function ($a) use ($b) {

    return $a + $b; }; $result = $add7(3); // 10 Closures
  5. function getAdder($b) { return function ($a) use ($b) { return

    $a + $b; } } $add7 = getAdder(7); First-class Functions + Closures
  6. $nums = [3, 6, 9]; $res = array_map($add7, $nums); //

    [10, 13, 16] Higher-order Functions
  7. is_callable($closure); // true is_object($closure); // true get_class($closure); // Closure Closure

    Objects “This fact used to be considered an implementation detail, but it can now be relied upon.” - PHP Manual
  8. ★ Solve a problem ★ Find a niche ★ Make

    it easy to use How to Get a Million Downloads If you build it, they will come
  9. ★ Build trust in your project ◦ Address issues promptly

    ◦ Follow SemVer ★ Build a community ◦ Become a dependency to a larger project ◦ Recruit contributors; reduce “bus factor” How to Get a Million Downloads (cont.) If you maintain it, they will stay
  10. $fn = function ($a) use ($b, $c) { return ($a

    + $b) * $c; }; What we need to serialize We need to know the values of these. $context
  11. $fn = function ($a) use ($b, $c) { return ($a

    + $b) * $c; }; What we need to serialize (cont.) And we need to know all of this. $code
  12. Interface for Serializing (Decorator) class SerializableClosure implements Serializable { private

    $closure; function __construct($closure) {…} function __invoke() {…} function serialize() {…} function unserialize($serialized) {…} }
  13. Context is easiest to get $a = 5; $b =

    10; $fn = function () use ($a, $b) { … }; $r = new ReflectionFunction($fn); $context = $r->getStaticVariables(); // [‘a’ => 5, ‘b’ => 10]
  14. Context is easiest to get… maybe $a = 5; $b

    = 10; $fn = function () use ($a, $b) { static $c = 15; }; // [‘a’ => 5, ‘b’ => 10, ‘c’ => 15]
  15. OK, How do we get the code? $r = new

    ReflectionFunction($closure); $file = $r->getFileName(); $start = $r->getStartLine(); $end = $r->getEndLine(); $code = implode(‘’, array_slice( file($file), $start, $end-$start+1));
  16. Use the lexer $tokens = token_get_all(‘<?php ’.$code); [ [T_OPEN_TAG, '<?php

    ', 1], [T_VARIABLE, '$fn', 1], [T_WHITESPACE, ' ', 1], '=', [T_WHITESPACE, ' ', 1], [T_FUNCTION, 'function', 1], ...]
  17. Create a parser foreach ($tokens as $token) { switch ($step)

    { case 0: if ($token[0] === T_FUNCTION) { $keep[] = $token; $step++; } break; // More steps... } }
  18. Create a parser (cont.) ★ Throw out tokens before 1st

    T_FUNCTION ★ Analyze function signature ◦ Verify names of “used” variables (i.e., the context) ★ Keep all tokens within function body ◦ Track the brace level to help identify the end.
  19. Token parser pitfalls ★ Assumes first T_FUNCTION encountered ★ No

    context outside of function ◦ What namespace/class is it in? ◦ What happens to magic constants __CLASS__?
  20. Use an AST ★ AST = Abstract Syntax Tree ★

    nikic/PHP-Parser is awesome! ★ Create tree data structure from entire file ◦ Traverse to closure, keeping track of scopes ◦ Replace some nodes with literal values as visited
  21. Example - Original namespace Foo; use Bar\Baz; class Fizz {

    public function buzz() { return function (Baz $baz) { return $baz->boop(__CLASS__); } } }
  22. Example - Token-based Result function (Baz $baz) { return $baz->boop(__CLASS__);

    } Code no longer knows the FQCN. ERROR Value is different from original function WTF
  23. Example - AST-based Result function (Bar\Baz $baz) { return $baz->boop(‘Foo\Fizz’);

    } FQCN injected into code. No loss of information. Magic constants replaced with literal values. No loss of information.
  24. Parsing Caveats ★ Two closures declared on the same line

    is too ambiguous handle. Oh well… ★ If closures use (&$vales, &$by, &$ref) then those references cannot be restored ◦ Unless ref is to the closure (e.g., recursive closures)
  25. Unsolved Mysteries ★ If a closure is declared in a

    trait, how do you resolve the value of __CLASS__ to a literal value? (If you figure this out, please let me know.)
  26. class Foo { private $bar; public function baz() { return

    function () { return $this->bar; }; } } Closure Bindings Automatically creates a closure on $this.
  27. class Foo { private $bar; public function baz() { return

    function () { return 5; }; } } Closure Bindings Automatically creates a closure on $this Even when you don’t need it.
  28. class Foo { private $bar; public function baz() { return

    function () { return 5; }; } } Closure Bindings Automatically creates a closure on $this. Even when you don’t need it. UNLESS...
  29. class Foo { private $bar; public function baz() { return

    static function () { return 5; }; } } Static Closures UNLESS... You make it static.
  30. $foo = new Foo; $letters = new ArrayObject(range(‘a’, ‘z’)); $fn

    = $foo->baz(); // baz() returns a static closure $fn->bindTo($letters); “Warning: Cannot bind an instance to a static closure” Static is forever
  31. Handling Closure Bindings ★ Two parts: ◦ Object ($this) is

    the object it is bound to. ◦ Scope determines what is visible. ★ Affects usage of $this, self::, and static:: within the closure
  32. Getting Binding Information class Foo { public function bar() {

    return function () { … }; }} $foo = new Foo(); $fn = $foo->bar(); $r = new ReflectionFunction($fn); $r->getClosureThis(); // $foo $r->getClosureScopeClass()->getName(); // “Foo”
  33. Setting Binding Information $newClosure = $closure->bindTo($newThis, $newScope); Notes: ★ You

    can omit $newScope, which means “keep the scope the same as it is now”. ★ Using null for $newThis makes the closure static. Remember, static is forever.
  34. (Un)Serializing Closure Bindings ★ Including object and scope in serialization

    ★ Rebind closure after unserialization ◦ Tricky, because the object can be null in two cases ▪ It’s static ▪ It was not declared within a class
  35. public function unserialize($serialized) { // Unserialize and reconstruct the closure.

    $this->data = unserialize($serialized); $this->closure = _reconstruct_closure($this->data); // Rebind the closure to its former binding and scope. if ($this->data['binding'] || $this->data['isStatic']) { $this->closure = $this->closure->bindTo( $this->data['binding'], $this->data['scope'] ); } }
  36. function _reconstruct_closure(array $__data) { extract($__data['context']); if ($__fn = array_search(RECURSION, $__data['context']))

    { @eval("\${$__fn} = {$__data['code']};"); $__closure = $$__fn; } else { @eval("\$__closure = {$__data['code']};"); } return $__closure; }
  37. function _reconstruct_closure(array $__data) { extract($__data['context']); if ($__fn = array_search(RECURSION, $__data['context']))

    { @eval("\${$__fn} = {$__data['code']};"); $__closure = $$__fn; } else { @eval("\$__closure = {$__data['code']};"); } return $__closure; }
  38. function _reconstruct_closure(array $__data) { extract($__data['context']); if ($__fn = array_search(RECURSION, $__data['context']))

    { @eval("\${$__fn} = {$__data['code']};"); $__closure = $$__fn; } else { @eval("\$__closure = {$__data['code']};"); } return $__closure; }
  39. function _reconstruct_closure(array $__data) { extract($__data['context']); if ($__fn = array_search(RECURSION, $__data['context']))

    { @eval("\${$__fn} = {$__data['code']};"); $__closure = $$__fn; } else { @eval("\$__closure = {$__data['code']};"); } return $__closure; }
  40. function _reconstruct_closure(array $__data) { extract($__data['context']); if ($__fn = array_search(RECURSION, $__data['context']))

    { @eval("\${$__fn} = {$__data['code']};"); $__closure = $$__fn; } else { @eval("\$__closure = {$__data['code']};"); } return $__closure; }
  41. The End ★ Closures are weird ★ Serializing closures “is

    not allowed”, but you can still do it, if you really want to. ★ Reflection is awesome; so is PHP-Parser ★ Open Source is fun