Iterators and Decorators and Generators, Oh My!

Iterators are an awesome and important feature of PHP, and PHP comes with a lot of them built in too. Let’s talk about what they are, how they’re used, and how to make your own. Then we'll talk about Generators, which were introduced in PHP 5.5, and that take Iterators to the next level. And… we can’t talk about Iterators without also discussing composition and the Decorator design pattern. After all, many of the SPL Iterator classes are Decorators too.

Transcript

1. Iterators and
   Decorators and
   Generators,
   Oh My!
   

   View Slide

2. Hello, World!
   I am Jeremy Lindblom
   Software Architect at McGraw-Hill Education
   @jeremeamia • I ♥ PHP, OSS, APIs, & AWS
   2
   

   View Slide

3. “
   An investment in knowledge
   pays the best interest.
   — Benjamin Franklin
   3
   

   View Slide

4. Mission Objectives
   ★ Understand and Apply Iterators
   Learn how they work, why they’re great, and how to make them
   ★ Understand the Decorator Design Pattern
   Learn how it allows for the composition of Iterator behavior
   ★ Understand and Apply Generators
   Learn how they work and how they make Iterators easier
   4
   

   View Slide

5. But first...
   5
   

   View Slide

6. 6
   [ ]
   The Array!
   

   View Slide

7. 7
   [ ]
   ♥ There’s no place like home. ♥
   

   View Slide

8. 8
   foreach ($data as $value) {
   echo "{$value}\n";
   }
   foreach ($data as $key => $value) {
   echo "{$key}: {$value}\n";
   }
   

   View Slide

9. 9
   $dir = new DirectoryIterator(__DIR__);
   foreach ($dir as $file) {
   echo "{$file->getFilename()}\n";
   }
   foreach-able objects?
   

   View Slide

10. Iterators
    The Scarecrow doesn’t have a brain, but you do.
    You will understand why Iterators are awesome.
    1
    

    View Slide

11. 11
    $dir = new DirectoryIterator(__DIR__);
    foreach ($dir as $file) {
    echo "{$file->getFilename()}\n";
    }
    

    View Slide

12. 12
    $dir = new DirectoryIterator(__DIR__);
    foreach ($dir as $file) {
    echo "{$file->getFilename()}\n";
    }
    var_dump($dir instanceof \Traversable);
    #> bool(true)
    

    View Slide

13. 13
    Traversable
    MyCoolIterator
    ?
    

    View Slide

14. 14
    Traversable
    MyCoolIterator
    

    View Slide

15. 15
    Traversable
    MyCoolIterator
    PHP Fatal error: Class MyCoolIterator
    cannot extend from interface Traversable
    

    View Slide

16. 16
    Traversable
    Iterator IteratorAggregate
    

    View Slide

17. 17
    Traversable
    Iterator IteratorAggregate
    Can’t directly
    extend this
    You extend one of these instead
    

    View Slide

18. 18
    Traversable
    Iterator IteratorAggregate
    DirectoryIterator
    

    View Slide

19. 19
    Traversable
    Iterator IteratorAggregate
    current();
    key();
    valid();
    next();
    rewind();
    getIterator();
    

    View Slide

20. 20
    Traversable
    Iterator IteratorAggregate
    current();
    key();
    valid();
    next();
    rewind();
    getIterator();
    

    View Slide

21. 21
    foreach ($iter as $key => $value) {
    echo "{$key}: {$value}\n";
    }
    

    View Slide

22. 22
    foreach ($iter as $key => $value) {
    echo "{$key}: {$value}\n";
    }
    // --- THIS IS THE SAME AS ---------------------
    $iter->rewind();
    while ($iter->valid()) {
    echo "{$iter->key()}: {$iter->current()}\n";
    $iter->next();
    }
    

    View Slide

23. 23
    $iter = new RangeIterator(3, 8);
    foreach ($iter as $key => $value) {
    echo "{$key}: {$value}\n";
    }
    #> 0: 3
    #> 1: 4
    #> 2: 5
    #> 3: 6
    #> 4: 7
    #> 5: 8
    

    View Slide

24. 24
    The following code is not PSR-2 compliant
    

    View Slide

25. 25
    class RangeIterator implements Iterator {
    private $low, $high, $index;
    function __construct(int $low, int $high) { … }
    function rewind() { $this->index = 0; }
    function next() { $this->index++; }
    function valid() {
    return $this->low + $this->index <= $this->high;
    }
    function key() { return $this->index; }
    function current() {
    return $this->low + $this->index;
    }
    }
    

    View Slide

26. 26
    class RangeIterator implements Iterator {
    private $low, $high, $index;
    function __construct(int $low, int $high) { … }
    function rewind() { $this->index = 0; }
    function next() { $this->index++; }
    function valid() {
    return $this->low + $this->index <= $this->high;
    }
    function key() { return $this->index; }
    function current() {
    return $this->low + $this->index;
    }
    }
    The Data
    

    View Slide

27. 27
    class RangeIterator implements Iterator {
    private $low, $high, $index;
    function __construct(int $low, int $high) { … }
    function rewind() { $this->index = 0; }
    function next() { $this->index++; }
    function valid() {
    return $this->low + $this->index <= $this->high;
    }
    function key() { return $this->index; }
    function current() {
    return $this->low + $this->index;
    }
    }
    The Cursor
    

    View Slide

28. 28
    class RangeIterator implements Iterator {
    private $low, $high, $index;
    function __construct(int $low, int $high) { … }
    function rewind() { $this->index = 0; }
    function next() { $this->index++; }
    function valid() {
    return $this->low + $this->index <= $this->high;
    }
    function key() { return $this->index; }
    function current() {
    return $this->low + $this->index;
    }
    }
    Moves the
    Cursor
    

    View Slide

29. 29
    class RangeIterator implements Iterator {
    private $low, $high, $index;
    function __construct(int $low, int $high) { … }
    function rewind() { $this->index = 0; }
    function next() { $this->index++; }
    function valid() {
    return $this->low + $this->index <= $this->high;
    }
    function key() { return $this->index; }
    function current() {
    return $this->low + $this->index;
    }
    }
    Yields
    the data
    

    View Slide

30. For valid(), key(), and current()
    ● Do “the work” in either valid() or current().
    ● These methods need to be idempotent per iteration.
    ● Avoid doing expensive work more than once
    (i.e., you should internally cache/memoize the current iteration’s data).
    30
    Implementation Tips
    

    View Slide

31. 31
    $iter = new RangeIterator(3, 8);
    foreach ($iter as $key => $value) {
    echo "{$key}: {$value}\n";
    }
    #> 0: 3
    #> 1: 4
    #> 2: 5
    #> 3: 6
    #> 4: 7
    #> 5: 8
    

    View Slide

32. 32
    $iter = new RangeIterator(3, 8);
    foreach ($iter as $key => $value) {
    echo "{$key}: {$value}\n";
    }
    // --- THIS IS THE SAME AS ---------------------
    $data = range(3, 8);
    foreach ($data as $key => $value) {
    echo "{$key}: {$value}\n";
    }
    

    View Slide

33. 33
    Why?
    

    View Slide

34. 34
    class RangeIterator implements Iterator {
    private $low, $high, $index;
    function __construct(int $low, int $high) { … }
    function rewind() { $this->index = 0; }
    function next() { $this->index++; }
    function valid() {
    return $this->low + $this->index <= $this->high;
    }
    function key() { return $this->index; }
    function current() {
    return $this->low + $this->index;
    }
    }
    

    View Slide

35. 35
    $data = range(1, 10);
    

    View Slide

36. 36
    $data = range(1, 1000000000);
    

    View Slide

37. 37
    $data = range(1, 1000000000);
    PHP Warning: range(): The supplied
    range exceeds the maximum array size
    

    View Slide

38. 38
    $iter = new RangeIterator(1, 1000000000);
    foreach ($iter as $key => $value) {
    echo "{$key}: {$value}\n";
    }
    #> 0: 1
    #> 1: 2
    #> 2: 3
    later...
    #> 999999998: 999999999
    #> 999999999: 1000000000
    

    View Slide

39. 39
    What are the benefits?
    

    View Slide

40. Benefits of Iterators vs. Arrays
    Lower Memory
    Handle data one iteration at a
    time. Don’t store the entire
    data set in memory at once.
    Allows for Infinity
    Because you handle data one
    iteration at a time, you can
    handle data sets that are large,
    have an unknown length, or
    are even infinitely long.
    Lazy/Deferred Work
    Defer data retrieval and
    transformation until the point in
    time it is actually needed.
    40
    Encapsulation
    Iterators are self-contained.
    Logic to work with a set of
    data lives in one class, and
    does not need to be repeated.
    Composition
    Using the Decorator and other
    composition design patterns,
    data transformation behavior
    can be composed from
    multiple iterator objects.
    SPL Implementations
    The Standard PHP Library comes
    with a plethora of iterator
    interfaces and classes for basic
    and advanced/specific use cases.
    

    View Slide

41. 41
    Traversable
    Iterator IteratorAggregate
    SPL Iterators
    

    View Slide

42. 42
    Traversable
    Iterator IteratorAggregate
    current();
    key();
    valid();
    next();
    rewind();
    getIterator();
    

    View Slide

43. 43
    class RangeIterator implements IteratorAggregate
    {
    private $low, $high;
    function __construct(int $low, int $high) { … }
    function getIterator() {
    $data = range($this->low, $this->high);
    return new ArrayIterator($data);
    }
    }
    

    View Slide

44. Decorators
    The Tinman is heartless, but your heart will be
    be warmed as you see objects collaborating.
    2
    

    View Slide

45. The Decorator
    Design Pattern
    Dynamically add behavior to an
    object instance by composing (or
    “wrapping”) it with another object of
    the same interface.
    45
    

    View Slide

46. 46
    ThingInterface
    ConcreteThing ThingDecorator
    

    View Slide

47. 47
    interface EventPublisher {
    public function publish(Event $event): bool;
    }
    class HttpPublisher implements EventPublisher {
    private $httpClient;
    public function __construct(Client $client) {…}
    public function publish(Event $event): bool {
    // …
    }
    }
    

    View Slide

48. 48
    class LoggingPublisher implements EventPublisher {
    public function __construct(
    EventPublisher $publisher,
    LoggerInterface $logger
    ) {…}
    public function publish(Event $event): bool {
    $this->logger->info('Published event!');
    $this->publisher->publish($event);
    }
    }
    

    View Slide

49. 49
    $client = new Client(…);
    $pub1 = new HttpPublisher($client);
    $db = new Database(…);
    $pub2 = new DbPublisher($db);
    $logger = new Logger(…);
    $pub1 = new LoggingPublisher($pub1, $logger);
    $pub2 = new LoggingPublisher($pub2, $logger);
    $event = new Event(…);
    $pub1->publish($event);
    $pub2->publish($event);
    

    View Slide

50. 50
    SPL Iterators
    CallbackFilterIterator
    AppendIterator NoRewindIterator
    IteratorIterator
    RegexIterator MultipleIterator
    ArrayIterator DirectoryIterator GlobIterator
    InfiniteIterator
    LimitIterator RecursiveIteratorIterator
    

    View Slide

51. 51
    class NotesRepository {
    // …
    public function getNotes(): array {
    $notes = [];
    foreach ($this->db->fetchNotes() as $note) {
    $notes[] = new Note($note);
    }
    return $notes;
    }
    }
    

    View Slide

52. 52
    $notes = $notesRepository->getNotes();
    foreach ($notes as $note) {
    // ...
    }
    

    View Slide

53. 53
    class NotesRepository {
    // …
    public function getNotes(): \Iterator {
    $notes = [];
    foreach ($this->db->fetchNotes() as $note) {
    $notes[] = new Note($note);
    }
    return new ArrayIterator($notes);
    }
    }
    

    View Slide

54. 54
    SPL Iterators
    CallbackFilterIterator
    AppendIterator NoRewindIterator
    IteratorIterator
    RegexIterator MultipleIterator
    ArrayIterator DirectoryIterator GlobIterator
    InfiniteIterator
    LimitIterator RecursiveIteratorIterator
    

    View Slide

55. 55
    $notes = $notesRepository->getNotes();
    $notes = new LimitIterator($notes, 0, 50);
    foreach ($notes as $note) {
    // ...
    }
    

    View Slide

56. 56
    And now…
    an epic poem by
    the phpbard
    

    View Slide

57. Generators
    The Lion lacks the courage, but you will be
    brave enough to use Generators in your code.
    3
    

    View Slide

58. ● An interruptible function
    ● A mechanism for cooperative multitasking
    ● An Iterator, but easier
    58
    What is a Generator?
    

    View Slide

59. ● An interruptible function
    ● A mechanism for cooperative multitasking
    ● An Iterator, but easier
    59
    What is a Generator?
    

    View Slide

60. 60
    Traversable
    Iterator IteratorAggregate
    SPL Iterators
    Generator
    

    View Slide

61. 61
    function names() {
    yield 'Joey';
    yield 'Izzy';
    yield 'Livy';
    }
    

    View Slide

62. 62
    function names(): \Generator {
    yield 'Joey';
    yield 'Izzy';
    yield 'Livy';
    }
    $generator = names();
    

    View Slide

63. 63
    function names(): \Generator {
    yield 'Joey';
    yield 'Izzy';
    yield 'Livy';
    }
    foreach (names() as $name) {
    echo "{$name}\n";
    }
    #> Joey
    #> Izzy
    #> Livy
    

    View Slide

64. 64
    /** @dataProvider provideTestCases */
    …
    public function provideTestCases() {
    return [
    [2, 6, 8],
    [0, 6, 6],
    [2, -6, -4],
    [2, 'a', null],
    ];
    }
    

    View Slide

65. 65
    /** @dataProvider provideTestCases */
    …
    public function provideTestCases() {
    yield [2, 6, 8];
    yield [0, 6, 6];
    yield [2, -6, -4];
    yield [2, 'a', null];
    }
    

    View Slide

66. 66
    /** @dataProvider provideTestCases */
    …
    public function provideTestCases() {
    yield 'with positives' => [2, 6, 8];
    yield 'with zeros' => [0, 6, 6];
    yield 'with negatives' => [2, -6, -4];
    yield 'with non-numbers' => [2, 'a', null];
    }
    

    View Slide

67. 67
    class RangeIterator implements Iterator {
    private $low, $high, $index;
    function __construct(int $low, int $high) { … }
    function rewind() { $this->index = 0; }
    function next() { $this->index++; }
    function valid() {
    return $this->low + $this->index <= $this->high;
    }
    function key() { return $this->index; }
    function current() {
    return $this->low + $this->index;
    }
    }
    Remember
    this?
    

    View Slide

68. 68
    function iter_range(int $low, int $high) {
    for ($n = $low; $n <= $high; $n++) {
    yield $n;
    }
    }
    

    View Slide

69. 69
    Achievement Unlocked
    You’ve struck gold on simplicity and robustness
    

    View Slide

70. 70
    class RangeIterator implements IteratorAggregate
    {
    private $low, $high;
    function __construct(int $low, int $high) { … }
    function getIterator() {
    $data = range($this->low, $this->high);
    return new ArrayIterator($data);
    }
    }
    Remember
    this?
    

    View Slide

71. 71
    class RangeIterator implements IteratorAggregate
    {
    private $low, $high;
    function __construct(int $low, int $high) { … }
    function getIterator() {
    for ($n = $this->low; $n <= $this->high; $n++) {
    yield $n;
    }
    }
    }
    Remember
    this?
    

    View Slide

72. 72
    class NotesRepository {
    // …
    public function getNotes(): \Iterator {
    $notes = [];
    foreach ($this->db->fetchNotes() as $note) {
    $notes[] = new Note($note);
    }
    return new ArrayIterator($notes);
    }
    }
    Remember
    this?
    

    View Slide

73. 73
    class NotesRepository {
    // …
    public function getNotes(): iterable {
    foreach ($this->db->fetchNotes() as $note) {
    yield new Note($note);
    }
    }
    }
    

    View Slide

74. 74
    class NotesRepository {
    // …
    public function getNotes(): iterable {
    foreach ($this->db->fetchNotes() as $note) {
    yield new Note($note);
    }
    }
    }
    

    View Slide

75. 75
    class NotesRepository {
    // …
    public function getNotes(): iterable {
    foreach ($this->db->fetchNotes() as $note) {
    yield new Note($note);
    }
    }
    }
    The “iterable” pseudo-type!
    

    View Slide

76. 76
    iterable
    array
    Traversable
    Iterator IteratorAggregate
    SPL Iterators
    Generator
    

    View Slide

77. Let’s get
    functional...
    77
    

    View Slide

78. 78
    function iter_filter(iterable $iter, callable $fn): iterable {
    foreach ($iter as $value) {
    if ($fn($value)) {
    yield $value;
    }
    }
    }
    

    View Slide

79. 79
    function iter_filter(iterable $iter, callable $fn): iterable {
    foreach ($iter as $value) {
    if ($fn($value)) {
    yield $value;
    }
    }
    }
    function iter_map(iterable $iter, callable $fn): iterable {
    foreach ($iter as $value) {
    yield $fn($value);
    }
    }
    

    View Slide

80. 80
    function iter_flatmap(iterable $iter, callable $fn) {
    foreach ($iter as $value) {
    yield from $fn($value);
    }
    }
    

    View Slide

81. 81
    function iter_flatmap(iterable $iter, callable $fn) {
    foreach ($iter as $value) {
    yield from $fn($value);
    }
    }
    

    View Slide

82. It works IRL!
    82
    

    View Slide

83. 83
    class ResultPaginator implements IteratorAggregate {
    public function __construct(
    Result $initialResult,
    callable $getNextPageFn
    ) {…}
    public function getIterator() {
    $current = $this->initialResult;
    while ($current) {
    yield $current;
    $current = ($this->getNextPageFn)($current);
    }
    }
    }
    

    View Slide

84. 84
    class ModelPaginator extends IteratorIterator {
    function __construct($paginator, $key, $mapper) {
    parent::__construct(iter_flatmap(
    $paginator,
    function (Result $result) use ($mapper, $key) {
    return iter_map($result[$key], $mapper);
    }
    ));
    }}
    

    View Slide

85. Mission Objectives
    ★ Understand and Apply Iterators
    Learn how they work, why they’re great, and how to make them
    ★ Understand the Decorator Design Pattern
    Learn how it allows for the composition of Iterator behavior
    ★ Understand and Apply Generators
    Learn how they work and how they make Iterators easier
    85
    

    View Slide

86. “There is no silver bullet. There
    are always options, and the
    options have consequences.
    — Ben Horowitz
    86
    

    View Slide

87. “There is no silver bullet. There
    are always options, and the
    options have consequences.
    — Ben Horowitz
    87
    tradeoffs
    

    View Slide

88. ● Using Arrays
    ● Implementing an Iterator
    ● Implementing IteratorAggregate
    ● Using an existing SPL Iterator
    ● Extending an SPL Iterator
    ● Composing/decorating multiple Iterators
    ● Implementing Generators
    ● Using general Generators: https://github.com/nikic/iter
    88
    What are your options?
    

    View Slide

89. 89
    iterable
    array
    Traversable
    Iterator IteratorAggregate
    SPL Iterators
    Generator
    All of them!
    

    View Slide

90. 90
    Thanks!
    Any questions?
    You can find me on Twitter & GitHub as @jeremeamia
    Give me feedback: https://joind.in/talk/ec8d0
    Yay!
    Iterables!
    

    View Slide

91. Credits
    Special thanks to all the people who made and
    released these awesome resources for free:
    ● Presentation template by SlidesCarnival
    ● Photographs from the Wizard of Oz
    91
    

    View Slide