Slide 1

Slide 1 text

How to serialize() a \Closure and Get a Million Downloads by Jeremy Lindblom – @jeremeamia

Slide 2

Slide 2 text

How to serialize() a \Closure and Get 3.6 Million Downloads by Jeremy Lindblom – @jeremeamia

Slide 3

Slide 3 text

Let me tell you about myself...

Slide 4

Slide 4 text

@jeremeamia Jeremy Lindblom the Software Developer

Slide 5

Slide 5 text

@jeremeamia Jeremy Lindblom the Software Developer AWS SDK for PHP at Amazon @awsforphp

Slide 6

Slide 6 text

@jeremeamia Jeremy Lindblom the Software Developer AWS SDK for PHP at Amazon @awsforphp President of the Seattle PHP User Group @seaphp & #pnwphp

Slide 7

Slide 7 text

@jeremeamia Jeremy Lindblom the Software Developer AWS SDK for PHP at Amazon @awsforphp President of the Seattle PHP User Group @seaphp & #pnwphp PHPoetry and PHParodies @phpbard

Slide 8

Slide 8 text

Once upon a time...

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

Closures

Slide 15

Slide 15 text

Closures —or— Lambdas (λ)

Slide 16

Slide 16 text

Closures —or— Lambdas —or— Anonymous Functions

Slide 17

Slide 17 text

function ($a, $b) { return $a + $b; }; Anonymous (a.k.a. no name)

Slide 18

Slide 18 text

$add = function ($a, $b) { return $a + $b; }; $result = $add(3, 7); // 10 First-class Functions

Slide 19

Slide 19 text

$b = 7; $add7 = function ($a) use ($b) { return $a + $b; }; $result = $add7(3); // 10 Closures

Slide 20

Slide 20 text

function getAdder($b) { return function ($a) use ($b) { return $a + $b; } } $add7 = getAdder(7); First-class Functions + Closures

Slide 21

Slide 21 text

$nums = [3, 6, 9]; $res = array_map($add7, $nums); // [10, 13, 16] Higher-order Functions

Slide 22

Slide 22 text

$closure = function () {...}; So what is this thing?

Slide 23

Slide 23 text

is_callable($closure); // true Closure Objects

Slide 24

Slide 24 text

is_callable($closure); // true is_object($closure); // true Closure Objects

Slide 25

Slide 25 text

is_callable($closure); // true is_object($closure); // true Closure Objects Like __invoke()

Slide 26

Slide 26 text

is_callable($closure); // true is_object($closure); // true get_class($closure); // Closure Closure Objects

Slide 27

Slide 27 text

is_callable($closure); // true is_object($closure); // true get_class($closure); // Closure Closure Objects “This fact used to be considered an implementation detail, but it can now be relied upon.” - PHP Manual

Slide 28

Slide 28 text

get_class_methods($closure); // array() Closure Objects (cont.)

Slide 29

Slide 29 text

serialize($closure); // ??? Closure Objects (still cont.)

Slide 30

Slide 30 text

Fatal error: Uncaught exception 'Exception' with message 'Serialization of 'Closure' is not allowed'

Slide 31

Slide 31 text

Back to the Story

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

serialize($closure);

Slide 34

Slide 34 text

Fatal error: Uncaught exception 'Exception' with message 'Serialization of 'Closure' is not allowed'

Slide 35

Slide 35 text

Challenge Accepted

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

Really ?

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

Thanks, Laravel!

Slide 44

Slide 44 text

SuperClosure Today

Slide 45

Slide 45 text

How to Get a Million Downloads

Slide 46

Slide 46 text

★ Solve a problem ★ Find a niche ★ Make it easy to use How to Get a Million Downloads If you build it, they will come

Slide 47

Slide 47 text

★ Build trust in your project ○ Address issues promptly ○ Follow SemVer ★ Build a community ○ Become a dependency to a larger project ○ Recruit contributors; reduce “bus factor” How to Get a Million Downloads (cont.) If you maintain it, they will stay

Slide 48

Slide 48 text

How to serialize() a \Closure

Slide 49

Slide 49 text

$fn = function ($a) use ($b, $c) { return ($a + $b) * $c; }; What we need to serialize We need to know the values of these. $context

Slide 50

Slide 50 text

$fn = function ($a) use ($b, $c) { return ($a + $b) * $c; }; What we need to serialize (cont.) And we need to know all of this. $code

Slide 51

Slide 51 text

extract($context); eval(“\$closure = {$code};”); How do we unserialize?

Slide 52

Slide 52 text

Interface for Serializing class SerializableClosure extends Closure implements Serializable { // ... }

Slide 53

Slide 53 text

Fatal error: Class SerializableClosure may not inherit from final class (Closure)

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

Interface for Serializing (Decorator) class SerializableClosure implements Serializable { private $closure; function __construct($closure) {…} function __invoke() {…} function serialize() {…} function unserialize($serialized) {…} }

Slide 56

Slide 56 text

Context is easiest to get $a = 5; $b = 10; $fn = function () use ($a, $b) { … }; $r = new ReflectionFunction($fn); $context = $r->getStaticVariables(); // [‘a’ => 5, ‘b’ => 10]

Slide 57

Slide 57 text

Context is easiest to get… maybe $a = 5; $b = 10; $fn = function () use ($a, $b) { static $c = 15; }; // [‘a’ => 5, ‘b’ => 10, ‘c’ => 15]

Slide 58

Slide 58 text

OK, How do we get the code? $r = new ReflectionFunction($closure); $file = $r->getFileName(); $start = $r->getStartLine(); $end = $r->getEndLine(); $code = implode(‘’, array_slice( file($file), $start, $end-$start+1));

Slide 59

Slide 59 text

token_get_all() parses the given source string into PHP language tokens using the Zend engine's lexical scanner.

Slide 60

Slide 60 text

Use the lexer $tokens = token_get_all(‘

Slide 61

Slide 61 text

Create a parser foreach ($tokens as $token) { switch ($step) { case 0: if ($token[0] === T_FUNCTION) { $keep[] = $token; $step++; } break; // More steps... } }

Slide 62

Slide 62 text

Create a parser (cont.) ★ Throw out tokens before 1st T_FUNCTION ★ Analyze function signature ○ Verify names of “used” variables (i.e., the context) ★ Keep all tokens within function body ○ Track the brace level to help identify the end.

Slide 63

Slide 63 text

Token parser pitfalls ★ Assumes first T_FUNCTION encountered ★ No context outside of function ○ What namespace/class is it in? ○ What happens to magic constants __CLASS__?

Slide 64

Slide 64 text

Use an AST ★ AST = Abstract Syntax Tree ★ nikic/PHP-Parser is awesome! ★ Create tree data structure from entire file ○ Traverse to closure, keeping track of scopes ○ Replace some nodes with literal values as visited

Slide 65

Slide 65 text

Example - Original namespace Foo; use Bar\Baz; class Fizz { public function buzz() { return function (Baz $baz) { return $baz->boop(__CLASS__); } } }

Slide 66

Slide 66 text

Example - Token-based Result function (Baz $baz) { return $baz->boop(__CLASS__); } Code no longer knows the FQCN. ERROR Value is different from original function WTF

Slide 67

Slide 67 text

Example - AST-based Result function (Bar\Baz $baz) { return $baz->boop(‘Foo\Fizz’); } FQCN injected into code. No loss of information. Magic constants replaced with literal values. No loss of information.

Slide 68

Slide 68 text

AST parser pitfalls ★ SLOW. Really slow.

Slide 69

Slide 69 text

Parsing Caveats ★ Two closures declared on the same line is too ambiguous handle. Oh well… ★ If closures use (&$vales, &$by, &$ref) then those references cannot be restored ○ Unless ref is to the closure (e.g., recursive closures)

Slide 70

Slide 70 text

Unsolved Mysteries ★ If a closure is declared in a trait, how do you resolve the value of __CLASS__ to a literal value? (If you figure this out, please let me know.)

Slide 71

Slide 71 text

Oh crap! What about bindings? (Curse you, PHP 5.4!)

Slide 72

Slide 72 text

class Foo { private $bar; public function baz() { return function () { return $this->bar; }; } } Closure Bindings Automatically creates a closure on $this.

Slide 73

Slide 73 text

class Foo { private $bar; public function baz() { return function () { return 5; }; } } Closure Bindings Automatically creates a closure on $this Even when you don’t need it.

Slide 74

Slide 74 text

class Foo { private $bar; public function baz() { return function () { return 5; }; } } Closure Bindings Automatically creates a closure on $this. Even when you don’t need it. UNLESS...

Slide 75

Slide 75 text

class Foo { private $bar; public function baz() { return static function () { return 5; }; } } Static Closures UNLESS... You make it static.

Slide 76

Slide 76 text

No content

Slide 77

Slide 77 text

$foo = new Foo; $letters = new ArrayObject(range(‘a’, ‘z’)); $fn = $foo->baz(); // baz() returns a static closure $fn->bindTo($letters); “Warning: Cannot bind an instance to a static closure” Static is forever

Slide 78

Slide 78 text

Handling Closure Bindings ★ Two parts: ○ Object ($this) is the object it is bound to. ○ Scope determines what is visible. ★ Affects usage of $this, self::, and static:: within the closure

Slide 79

Slide 79 text

Getting Binding Information class Foo { public function bar() { return function () { … }; }} $foo = new Foo(); $fn = $foo->bar(); $r = new ReflectionFunction($fn); $r->getClosureThis(); // $foo $r->getClosureScopeClass()->getName(); // “Foo”

Slide 80

Slide 80 text

Setting Binding Information $newClosure = $closure->bindTo($newThis, $newScope); Notes: ★ You can omit $newScope, which means “keep the scope the same as it is now”. ★ Using null for $newThis makes the closure static. Remember, static is forever.

Slide 81

Slide 81 text

(Un)Serializing Closure Bindings ★ Including object and scope in serialization ★ Rebind closure after unserialization ○ Tricky, because the object can be null in two cases ■ It’s static ■ It was not declared within a class

Slide 82

Slide 82 text

How to unserialize() a \Closure and Scare the Hell Out of Everyone

Slide 83

Slide 83 text

public function unserialize($serialized) { // Unserialize and reconstruct the closure. $this->data = unserialize($serialized); $this->closure = _reconstruct_closure($this->data); // Rebind the closure to its former binding and scope. if ($this->data['binding'] || $this->data['isStatic']) { $this->closure = $this->closure->bindTo( $this->data['binding'], $this->data['scope'] ); } }

Slide 84

Slide 84 text

And now, the most frightening function ever written...

Slide 85

Slide 85 text

function _reconstruct_closure(array $__data) { extract($__data['context']); if ($__fn = array_search(RECURSION, $__data['context'])) { @eval("\${$__fn} = {$__data['code']};"); $__closure = $$__fn; } else { @eval("\$__closure = {$__data['code']};"); } return $__closure; }

Slide 86

Slide 86 text

function _reconstruct_closure(array $__data) { extract($__data['context']); if ($__fn = array_search(RECURSION, $__data['context'])) { @eval("\${$__fn} = {$__data['code']};"); $__closure = $$__fn; } else { @eval("\$__closure = {$__data['code']};"); } return $__closure; }

Slide 87

Slide 87 text

function _reconstruct_closure(array $__data) { extract($__data['context']); if ($__fn = array_search(RECURSION, $__data['context'])) { @eval("\${$__fn} = {$__data['code']};"); $__closure = $$__fn; } else { @eval("\$__closure = {$__data['code']};"); } return $__closure; }

Slide 88

Slide 88 text

function _reconstruct_closure(array $__data) { extract($__data['context']); if ($__fn = array_search(RECURSION, $__data['context'])) { @eval("\${$__fn} = {$__data['code']};"); $__closure = $$__fn; } else { @eval("\$__closure = {$__data['code']};"); } return $__closure; }

Slide 89

Slide 89 text

function _reconstruct_closure(array $__data) { extract($__data['context']); if ($__fn = array_search(RECURSION, $__data['context'])) { @eval("\${$__fn} = {$__data['code']};"); $__closure = $$__fn; } else { @eval("\$__closure = {$__data['code']};"); } return $__closure; }

Slide 90

Slide 90 text

The End ★ Closures are weird ★ Serializing closures “is not allowed”, but you can still do it, if you really want to. ★ Reflection is awesome; so is PHP-Parser ★ Open Source is fun

Slide 91

Slide 91 text

How to serialize() a \Closure and Get a Million Downloads by Jeremy Lindblom – @jeremeamia