Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Map, Filter, and Reduce in PHP

derek-b
January 25, 2020

Map, Filter, and Reduce in PHP

Even though Map Reduce was created for parallelization, which PHP does not natively support, the syntax is supported. In this talk I will outline how to use map, filter, and reduce with PHP collections and the benefits you can gain over traditional for loops.

In this talk we will explore different methods of processing and manipulating collections of data giving you new tools in your developer’s toolbox

derek-b

January 25, 2020
Tweet

More Decks by derek-b

Other Decks in Technology

Transcript

  1. @DerekB_WI
    Map, Filter, and
    Reduce in PHP
    Presented by Derek Binkley

    View Slide

  2. @DerekB_WI
    About Me
    • Father of 3, Husband

    • Software Consultant with Spark Labs

    • PHP, Java, Groovy, JavaScript …

    • He/Him/His

    View Slide

  3. View Slide

  4. @DerekB_WI

    View Slide

  5. @DerekB_WI

    View Slide

  6. @DerekB_WI
    Google in 2004
    • MapReduce: Simplified Data Processing on Large
    Clusters, Jeffrey Dean and Sanjay Ghemawat

    View Slide

  7. –Oscar Stiffelman - early Google engineer
    “Only when something is simple can you make it
    really efficient”

    View Slide

  8. View Slide

  9. @DerekB_WI
    Java Data Streams
    List numbers = Arrays.asList(5, 22, 457, 111233, 6);

    View Slide

  10. @DerekB_WI
    Java Data Streams
    List numbers = Arrays.asList(5, 22, 457, 111233, 6);
    List filteredNumbers = (List) numbers.stream()

    .filter(num -> (int)num > 10)

    .sorted()

    .map(num -> (int)num * -1)

    .collect(Collectors.toList());

    View Slide

  11. @DerekB_WI
    Java Data Streams
    List numbers = Arrays.asList(5, 22, 457, 111233, 6);
    Integer sum = numbers.stream()

    .reduce(0, (subtotal, element) -> subtotal + element);

    View Slide

  12. @DerekB_WI
    Java Data Streams
    List numbers = Arrays.asList(5, 22, 457, 111233, 6);
    List filteredNumbers = (List) numbers.parallelStream()

    .filter(num -> (int)num > 10)

    .sorted()

    .map(num -> (int)num * -1)

    .collect(Collectors.toList());
    Integer sum = numbers.parallelStream()

    .reduce(0, (subtotal, element) -> subtotal + element);

    View Slide

  13. View Slide

  14. @DerekB_WI
    JavaScript Promise
    function loadImage() {

    let imageUrl = 'https://live.staticflickr.com/
    65535/47813616341_6063c7b420_b_d.jpg';

    fetch(imageUrl)

    .then((data) => {

    // Save image and display

    ...

    })

    .catch((error) => {

    // Alert user of error

    ...

    });

    }

    View Slide

  15. View Slide

  16. @DerekB_WI
    Benefits
    • No Parallel Processing, So what else?

    • Functional, no side-effects

    • Easy to read

    • Single Responsibility

    • Reduce boilerplate

    View Slide

  17. View Slide

  18. @DerekB_WI
    Traditional Foreach
    $placesArray = $reader->getFileContents();


    $filteredLocations = [];

    foreach ($placesArray as $place) {

    if ($place['state'] == 'AK') {

    $filteredLocations[] = new Place($place);

    }

    }

    View Slide

  19. @DerekB_WI
    Filter
    $placesArray = $reader->getFileContents();

    $filteredAlaskaList = array_filter($placesArray,

    function($place) {

    return $place['state'] == 'AK';

    }

    );

    View Slide

  20. @DerekB_WI
    Filter
    $placesArray = $reader->getFileContents();

    $filteredAlaskaList = array_filter($placesArray,

    function($place) {

    return $place['state'] == 'AK';

    }

    );

    View Slide

  21. @DerekB_WI
    Filter - 7.4
    $placesArray = $reader->getFileContents();

    $filteredAlaskaList = array_filter($placesArray,

    fn($place) => $place['state'] == 'AK'

    );

    View Slide

  22. View Slide

  23. @DerekB_WI
    Traditional Foreach
    $placesArray = $reader->getFileContents();


    $filteredLocations = [];

    foreach ($placesArray as $place) {

    if ($place['state'] == 'AK') {

    $filteredLocations[] = new Place($place);

    }

    }

    View Slide

  24. @DerekB_WI
    Map
    $placesArray = $reader->getFileContents();

    $modifiedList = array_map(

    function($place) {

    return PlaceMapper::parse($place);

    }, 

    $placesArray);

    View Slide

  25. @DerekB_WI
    Map - 7.4
    $placesArray = $reader->getFileContents();

    $filteredAlaskaList = array_map(

    fn($place) => PlaceMapper::parse($place), 

    $placesArray);

    View Slide

  26. View Slide

  27. @DerekB_WI
    Reduce
    $myTrips = [

    new Trip("name" => "school", "distance" => 1.5),

    new Trip("name" => "the store", "distance" => 2.5)];

    $totalDriven = array_reduce($myTrips,

    function ($total, $trip) {

    return $total + $trip->distance;

    },

    0 // initial

    );

    echo "I biked {$totalDriven} kilometers today.";

    View Slide

  28. @DerekB_WI
    Reduce
    $myTrips = [

    new Trip("name" => "school", "distance" => 1.5),

    new Trip("name" => "the store", "distance" => 2.5)];

    $totalDriven = array_reduce($myTrips,

    function ($total, $trip) {

    return $total + $trip->distance;

    },

    0 // initial

    );

    echo "I biked {$totalDriven} kilometers today.";

    View Slide

  29. @DerekB_WI
    Reduce - 7.4
    $totalDriven = array_reduce($myTrips,

    fn($total, $trip) => $total + $trip->distance,

    0);

    View Slide

  30. @DerekB_WI
    Traditional Loop
    for ($i = 0; $i < count($placesArray); $i++) {

    $stateName = $placesArray[$i]['state'];

    if (!array_key_exists($stateName, $stateCounters)) 

    {

    $stateCounters[$stateName] = 0;

    }

    $stateCounters[$stateName]++;

    }

    View Slide

  31. @DerekB_WI
    Reduce
    $stateCounters = array_reduce($placesArray,

    function($totals, $location) {

    $totals[$location['state']] =

    isset($totals[$location['state']])

    ? $totals[$location['state']] + 1 : 1;

    return $totals;

    });

    View Slide

  32. @DerekB_WI
    Reduce
    $stateCounters = array_reduce($placesArray,

    function($totals, $location) {

    $totals[$location['state']] =

    isset($totals[$location['state']])

    ? $totals[$location['state']] + 1 : 1;

    return $totals;

    });

    View Slide

  33. @DerekB_WI
    Reduce
    $stateCounters = array_reduce($placesArray,

    function($totals, $location) {

    $totals[$location['state']] =

    isset($totals[$location['state']])

    ? $totals[$location['state']] + 1 : 1;

    return $totals;

    });

    View Slide

  34. @DerekB_WI
    Use an outside variable
    $searchWord = 'Bridge';

    $placesWithBridges = array_filter($placesArray,

    function($location) use($searchWord) {

    return strpos($location['name'], $searchWord);

    });

    View Slide

  35. @DerekB_WI
    Patterns
    ApiPlacesController extends AbstractController {



    }

    View Slide

  36. @DerekB_WI
    Patterns
    /**

    * route - /api/place

    */

    public function allPlaces() {

    $places = $this->repository->retrieveAll(); 

    $placesDtos = array_map(

    function($place) {

    return PlacesTransformer::toDTO($place);

    }, $places);

    return convertToJson($placesDtos);

    }

    View Slide

  37. @DerekB_WI
    Patterns
    /**

    * route - /api/user/$userId/place

    */

    public function placesByUser($userId) {

    $places = $this->repository->retrieveAll(); 

    $placesDtos = array_map(

    function($place) {

    return PlacesTransformer::toDTO($place);

    }, array_filter($places, function($place) {

    return LegacyMiddleware::shouldInclude($place);

    }

    ));

    return convertToJson($placesDtos);

    }

    View Slide

  38. View Slide

  39. @DerekB_WI
    Immutable
    $searchWord = 'Bridge';

    $placesWithBridges = array_filter($placesArray,

    function($location) use($searchWord) {

    $searchWord = ‘something else’;

    return strpos($location['name'], $searchWord);

    });

    View Slide

  40. @DerekB_WI
    Mutable
    array_walk(

    $placesArray,

    function(&$location, $key) {

    $location['distance'] = calcDistance($location);

    });

    array_multisort(

    array_column($placesArray, 'distance'),

    SORT_ASC,

    $placesArray);

    View Slide

  41. @DerekB_WI
    Generators
    function location_file_generator($filename)

    {

    $file = fopen($filename, 'r');

    while(($fileData = fgetcsv($file, 1000, "\t")) !== FALSE) {
    yield $fileData;

    }

    }

    View Slide

  42. @DerekB_WI
    Generators

    foreach(location_file_generator($filename) as $line) 

    {

    ...

    }

    View Slide

  43. @DerekB_WI
    Generators
    • Pro: Reduced Memory Usage

    • Pro: Easy syntax

    • Con: Cannot use array functions

    View Slide

  44. @DerekB_WI
    Generator Map
    function map(Traversable $data, Callable $func) {

    foreach($data as $item) {

    yield call_user_func($func, $item);

    }

    }

    View Slide

  45. @DerekB_WI
    Generator Map
    foreach(map(location_file_generator($filename), 

    function($item) {

    return $item[1];

    }) as $line) {



    }

    View Slide

  46. @DerekB_WI
    Generator Filter
    function filter(Traversable $data, Callable $func) {

    foreach($data as $item) {

    if (call_user_func($func)) {

    yield $item;

    }

    }

    }

    View Slide

  47. @DerekB_WI
    Generator Filter
    foreach(filter(location_file_generator($filename), 

    function($item) {

    return $item[1] == 1;

    }) as $line) {



    }

    View Slide

  48. @DerekB_WI
    Data Structures
    • Standard PHP Library - SPL

    • Common Structures - widely used in other languages

    • Many have filter and reduce functions

    • apt install php-ds

    • pecl install ds

    View Slide

  49. @DerekB_WI
    Hash Map
    $map = new DS\Map();

    $map->put('AK', PlaceFactory::make('Alaska'));

    $map->put('WI', PlaceFactory::make('Wisconsin'));

    $map->filter(function($key, $value) {

    return $value->population > 5000000;

    });

    View Slide

  50. @DerekB_WI
    Hash Map
    $map = new DS\Map();

    $map->put('AK', PlaceFactory::make('Alaska'));

    $map->put('WI', PlaceFactory::make('Wisconsin'));

    $map->reduce(function($carry, $key, $value) {

    return $carry + $value->population;

    }, 0);

    View Slide

  51. View Slide

  52. @DerekB_WI
    Iterators
    class MyIter implements Iterator {

    protected $data;

    protected $index;

    protected $valid = true;



    }

    View Slide

  53. @DerekB_WI
    Iterators
    public function __construct($inArray) {

    $this->data = $inArray;

    $this->index;

    }

    View Slide

  54. @DerekB_WI
    Iterators
    public function current() {

    return $this->data[$this->index];

    }

    View Slide

  55. @DerekB_WI
    Iterators
    public function next() {

    $this->index++;

    if ($this->index >= count($this->data)) {

    $this->valid = false;

    return null;

    }

    return $this->current();

    }

    View Slide

  56. @DerekB_WI
    Iterators
    public function key() {

    return $this->index;

    }

    public function valid() {

    return $this->valid;

    }

    public function rewind() {

    $this->index = 0;

    }

    View Slide

  57. @DerekB_WI
    Invoke Iterator
    foreach(new MyIter($someArray) as $it) {

    …

    }

    View Slide

  58. @DerekB_WI
    Invoke Iterator + Map
    foreach(map(new MyIter($someArray), function($item) {

    …

    }) as $it) {

    …

    }

    View Slide

  59. @DerekB_WI
    Laravel Collections

    View Slide

  60. @DerekB_WI
    Laravel Collections

    View Slide

  61. @DerekB_WI
    CakePHP MapReduce

    View Slide

  62. @DerekB_WI
    CakePHP MapReduce

    View Slide

  63. @DerekB_WI
    Doctrine Collections

    View Slide

  64. @DerekB_WI
    Doctrine Collections

    View Slide

  65. @DerekB_WI
    Async/Process
    • Symfony process component

    • Promise library

    • Job Queues

    View Slide

  66. @DerekB_WI
    Promise
    use GuzzleHttp\Promise\Promise;

    $promise = new Promise(function() { … });

    $result = $promise->then(

    function($value) {

    // handle result

    }

    );

    View Slide

  67. @DerekB_WI
    Resources
    • https://www.newyorker.com/magazine/2018/12/10/the-friendship-that-made-google-huge

    • https://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf

    • https://medium.com/@oscarstiffelman/a-brief-history-of-mapreduce-97aec97df8ff

    • https://www.sitepoint.com/parallel-programming-pthreads-php-fundamentals/

    • https://www.dragonbe.com/2015/07/speeding-up-database-calls-with-pdo-and.html

    • https://github.com/doctrine/collections/blob/master/lib/Doctrine/Common/Collections/Collection.php

    View Slide

  68. View Slide

  69. View Slide