Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Map, Filter, and Reduce in PHP

derek-b
January 25, 2020

Map, Filter, and Reduce in PHP

Even though Map Reduce was created for parallelization, which PHP does not natively support, the syntax is supported. In this talk I will outline how to use map, filter, and reduce with PHP collections and the benefits you can gain over traditional for loops.

In this talk we will explore different methods of processing and manipulating collections of data giving you new tools in your developer’s toolbox

derek-b

January 25, 2020
Tweet

More Decks by derek-b

Other Decks in Technology

Transcript

  1. @DerekB_WI
    Map, Filter, and
    Reduce in PHP
    Presented by Derek Binkley

    View full-size slide

  2. @DerekB_WI
    About Me
    • Father of 3, Husband

    • Software Consultant with Spark Labs

    • PHP, Java, Groovy, JavaScript …

    • He/Him/His

    View full-size slide

  3. @DerekB_WI
    Google in 2004
    • MapReduce: Simplified Data Processing on Large
    Clusters, Jeffrey Dean and Sanjay Ghemawat

    View full-size slide

  4. –Oscar Stiffelman - early Google engineer
    “Only when something is simple can you make it
    really efficient”

    View full-size slide

  5. @DerekB_WI
    Java Data Streams
    List numbers = Arrays.asList(5, 22, 457, 111233, 6);

    View full-size slide

  6. @DerekB_WI
    Java Data Streams
    List numbers = Arrays.asList(5, 22, 457, 111233, 6);
    List filteredNumbers = (List) numbers.stream()

    .filter(num -> (int)num > 10)

    .sorted()

    .map(num -> (int)num * -1)

    .collect(Collectors.toList());

    View full-size slide

  7. @DerekB_WI
    Java Data Streams
    List numbers = Arrays.asList(5, 22, 457, 111233, 6);
    Integer sum = numbers.stream()

    .reduce(0, (subtotal, element) -> subtotal + element);

    View full-size slide

  8. @DerekB_WI
    Java Data Streams
    List numbers = Arrays.asList(5, 22, 457, 111233, 6);
    List filteredNumbers = (List) numbers.parallelStream()

    .filter(num -> (int)num > 10)

    .sorted()

    .map(num -> (int)num * -1)

    .collect(Collectors.toList());
    Integer sum = numbers.parallelStream()

    .reduce(0, (subtotal, element) -> subtotal + element);

    View full-size slide

  9. @DerekB_WI
    JavaScript Promise
    function loadImage() {

    let imageUrl = 'https://live.staticflickr.com/
    65535/47813616341_6063c7b420_b_d.jpg';

    fetch(imageUrl)

    .then((data) => {

    // Save image and display

    ...

    })

    .catch((error) => {

    // Alert user of error

    ...

    });

    }

    View full-size slide

  10. @DerekB_WI
    Benefits
    • No Parallel Processing, So what else?

    • Functional, no side-effects

    • Easy to read

    • Single Responsibility

    • Reduce boilerplate

    View full-size slide

  11. @DerekB_WI
    Traditional Foreach
    $placesArray = $reader->getFileContents();


    $filteredLocations = [];

    foreach ($placesArray as $place) {

    if ($place['state'] == 'AK') {

    $filteredLocations[] = new Place($place);

    }

    }

    View full-size slide

  12. @DerekB_WI
    Filter
    $placesArray = $reader->getFileContents();

    $filteredAlaskaList = array_filter($placesArray,

    function($place) {

    return $place['state'] == 'AK';

    }

    );

    View full-size slide

  13. @DerekB_WI
    Filter
    $placesArray = $reader->getFileContents();

    $filteredAlaskaList = array_filter($placesArray,

    function($place) {

    return $place['state'] == 'AK';

    }

    );

    View full-size slide

  14. @DerekB_WI
    Filter - 7.4
    $placesArray = $reader->getFileContents();

    $filteredAlaskaList = array_filter($placesArray,

    fn($place) => $place['state'] == 'AK'

    );

    View full-size slide

  15. @DerekB_WI
    Traditional Foreach
    $placesArray = $reader->getFileContents();


    $filteredLocations = [];

    foreach ($placesArray as $place) {

    if ($place['state'] == 'AK') {

    $filteredLocations[] = new Place($place);

    }

    }

    View full-size slide

  16. @DerekB_WI
    Map
    $placesArray = $reader->getFileContents();

    $modifiedList = array_map(

    function($place) {

    return PlaceMapper::parse($place);

    }, 

    $placesArray);

    View full-size slide

  17. @DerekB_WI
    Map - 7.4
    $placesArray = $reader->getFileContents();

    $filteredAlaskaList = array_map(

    fn($place) => PlaceMapper::parse($place), 

    $placesArray);

    View full-size slide

  18. @DerekB_WI
    Reduce
    $myTrips = [

    new Trip("name" => "school", "distance" => 1.5),

    new Trip("name" => "the store", "distance" => 2.5)];

    $totalDriven = array_reduce($myTrips,

    function ($total, $trip) {

    return $total + $trip->distance;

    },

    0 // initial

    );

    echo "I biked {$totalDriven} kilometers today.";

    View full-size slide

  19. @DerekB_WI
    Reduce
    $myTrips = [

    new Trip("name" => "school", "distance" => 1.5),

    new Trip("name" => "the store", "distance" => 2.5)];

    $totalDriven = array_reduce($myTrips,

    function ($total, $trip) {

    return $total + $trip->distance;

    },

    0 // initial

    );

    echo "I biked {$totalDriven} kilometers today.";

    View full-size slide

  20. @DerekB_WI
    Reduce - 7.4
    $totalDriven = array_reduce($myTrips,

    fn($total, $trip) => $total + $trip->distance,

    0);

    View full-size slide

  21. @DerekB_WI
    Traditional Loop
    for ($i = 0; $i < count($placesArray); $i++) {

    $stateName = $placesArray[$i]['state'];

    if (!array_key_exists($stateName, $stateCounters)) 

    {

    $stateCounters[$stateName] = 0;

    }

    $stateCounters[$stateName]++;

    }

    View full-size slide

  22. @DerekB_WI
    Reduce
    $stateCounters = array_reduce($placesArray,

    function($totals, $location) {

    $totals[$location['state']] =

    isset($totals[$location['state']])

    ? $totals[$location['state']] + 1 : 1;

    return $totals;

    });

    View full-size slide

  23. @DerekB_WI
    Reduce
    $stateCounters = array_reduce($placesArray,

    function($totals, $location) {

    $totals[$location['state']] =

    isset($totals[$location['state']])

    ? $totals[$location['state']] + 1 : 1;

    return $totals;

    });

    View full-size slide

  24. @DerekB_WI
    Reduce
    $stateCounters = array_reduce($placesArray,

    function($totals, $location) {

    $totals[$location['state']] =

    isset($totals[$location['state']])

    ? $totals[$location['state']] + 1 : 1;

    return $totals;

    });

    View full-size slide

  25. @DerekB_WI
    Use an outside variable
    $searchWord = 'Bridge';

    $placesWithBridges = array_filter($placesArray,

    function($location) use($searchWord) {

    return strpos($location['name'], $searchWord);

    });

    View full-size slide

  26. @DerekB_WI
    Patterns
    ApiPlacesController extends AbstractController {



    }

    View full-size slide

  27. @DerekB_WI
    Patterns
    /**

    * route - /api/place

    */

    public function allPlaces() {

    $places = $this->repository->retrieveAll(); 

    $placesDtos = array_map(

    function($place) {

    return PlacesTransformer::toDTO($place);

    }, $places);

    return convertToJson($placesDtos);

    }

    View full-size slide

  28. @DerekB_WI
    Patterns
    /**

    * route - /api/user/$userId/place

    */

    public function placesByUser($userId) {

    $places = $this->repository->retrieveAll(); 

    $placesDtos = array_map(

    function($place) {

    return PlacesTransformer::toDTO($place);

    }, array_filter($places, function($place) {

    return LegacyMiddleware::shouldInclude($place);

    }

    ));

    return convertToJson($placesDtos);

    }

    View full-size slide

  29. @DerekB_WI
    Immutable
    $searchWord = 'Bridge';

    $placesWithBridges = array_filter($placesArray,

    function($location) use($searchWord) {

    $searchWord = ‘something else’;

    return strpos($location['name'], $searchWord);

    });

    View full-size slide

  30. @DerekB_WI
    Mutable
    array_walk(

    $placesArray,

    function(&$location, $key) {

    $location['distance'] = calcDistance($location);

    });

    array_multisort(

    array_column($placesArray, 'distance'),

    SORT_ASC,

    $placesArray);

    View full-size slide

  31. @DerekB_WI
    Generators
    function location_file_generator($filename)

    {

    $file = fopen($filename, 'r');

    while(($fileData = fgetcsv($file, 1000, "\t")) !== FALSE) {
    yield $fileData;

    }

    }

    View full-size slide

  32. @DerekB_WI
    Generators

    foreach(location_file_generator($filename) as $line) 

    {

    ...

    }

    View full-size slide

  33. @DerekB_WI
    Generators
    • Pro: Reduced Memory Usage

    • Pro: Easy syntax

    • Con: Cannot use array functions

    View full-size slide

  34. @DerekB_WI
    Generator Map
    function map(Traversable $data, Callable $func) {

    foreach($data as $item) {

    yield call_user_func($func, $item);

    }

    }

    View full-size slide

  35. @DerekB_WI
    Generator Map
    foreach(map(location_file_generator($filename), 

    function($item) {

    return $item[1];

    }) as $line) {



    }

    View full-size slide

  36. @DerekB_WI
    Generator Filter
    function filter(Traversable $data, Callable $func) {

    foreach($data as $item) {

    if (call_user_func($func)) {

    yield $item;

    }

    }

    }

    View full-size slide

  37. @DerekB_WI
    Generator Filter
    foreach(filter(location_file_generator($filename), 

    function($item) {

    return $item[1] == 1;

    }) as $line) {



    }

    View full-size slide

  38. @DerekB_WI
    Data Structures
    • Standard PHP Library - SPL

    • Common Structures - widely used in other languages

    • Many have filter and reduce functions

    • apt install php-ds

    • pecl install ds

    View full-size slide

  39. @DerekB_WI
    Hash Map
    $map = new DS\Map();

    $map->put('AK', PlaceFactory::make('Alaska'));

    $map->put('WI', PlaceFactory::make('Wisconsin'));

    $map->filter(function($key, $value) {

    return $value->population > 5000000;

    });

    View full-size slide

  40. @DerekB_WI
    Hash Map
    $map = new DS\Map();

    $map->put('AK', PlaceFactory::make('Alaska'));

    $map->put('WI', PlaceFactory::make('Wisconsin'));

    $map->reduce(function($carry, $key, $value) {

    return $carry + $value->population;

    }, 0);

    View full-size slide

  41. @DerekB_WI
    Iterators
    class MyIter implements Iterator {

    protected $data;

    protected $index;

    protected $valid = true;



    }

    View full-size slide

  42. @DerekB_WI
    Iterators
    public function __construct($inArray) {

    $this->data = $inArray;

    $this->index;

    }

    View full-size slide

  43. @DerekB_WI
    Iterators
    public function current() {

    return $this->data[$this->index];

    }

    View full-size slide

  44. @DerekB_WI
    Iterators
    public function next() {

    $this->index++;

    if ($this->index >= count($this->data)) {

    $this->valid = false;

    return null;

    }

    return $this->current();

    }

    View full-size slide

  45. @DerekB_WI
    Iterators
    public function key() {

    return $this->index;

    }

    public function valid() {

    return $this->valid;

    }

    public function rewind() {

    $this->index = 0;

    }

    View full-size slide

  46. @DerekB_WI
    Invoke Iterator
    foreach(new MyIter($someArray) as $it) {

    …

    }

    View full-size slide

  47. @DerekB_WI
    Invoke Iterator + Map
    foreach(map(new MyIter($someArray), function($item) {

    …

    }) as $it) {

    …

    }

    View full-size slide

  48. @DerekB_WI
    Laravel Collections

    View full-size slide

  49. @DerekB_WI
    Laravel Collections

    View full-size slide

  50. @DerekB_WI
    CakePHP MapReduce

    View full-size slide

  51. @DerekB_WI
    CakePHP MapReduce

    View full-size slide

  52. @DerekB_WI
    Doctrine Collections

    View full-size slide

  53. @DerekB_WI
    Doctrine Collections

    View full-size slide

  54. @DerekB_WI
    Async/Process
    • Symfony process component

    • Promise library

    • Job Queues

    View full-size slide

  55. @DerekB_WI
    Promise
    use GuzzleHttp\Promise\Promise;

    $promise = new Promise(function() { … });

    $result = $promise->then(

    function($value) {

    // handle result

    }

    );

    View full-size slide

  56. @DerekB_WI
    Resources
    • https://www.newyorker.com/magazine/2018/12/10/the-friendship-that-made-google-huge

    • https://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf

    • https://medium.com/@oscarstiffelman/a-brief-history-of-mapreduce-97aec97df8ff

    • https://www.sitepoint.com/parallel-programming-pthreads-php-fundamentals/

    • https://www.dragonbe.com/2015/07/speeding-up-database-calls-with-pdo-and.html

    • https://github.com/doctrine/collections/blob/master/lib/Doctrine/Common/Collections/Collection.php

    View full-size slide