Map, Filter, and Reduce in PHP

D12eaf3ef46e4f0fc6b714fd2b7ffe3b?s=47 derek-b
January 25, 2020

Map, Filter, and Reduce in PHP

Even though Map Reduce was created for parallelization, which PHP does not natively support, the syntax is supported. In this talk I will outline how to use map, filter, and reduce with PHP collections and the benefits you can gain over traditional for loops.

In this talk we will explore different methods of processing and manipulating collections of data giving you new tools in your developer’s toolbox

D12eaf3ef46e4f0fc6b714fd2b7ffe3b?s=128

derek-b

January 25, 2020
Tweet

Transcript

  1. @DerekB_WI Map, Filter, and Reduce in PHP Presented by Derek

    Binkley
  2. @DerekB_WI About Me • Father of 3, Husband • Software

    Consultant with Spark Labs • PHP, Java, Groovy, JavaScript … • He/Him/His
  3. None
  4. @DerekB_WI

  5. @DerekB_WI

  6. @DerekB_WI Google in 2004 • MapReduce: Simplified Data Processing on

    Large Clusters, Jeffrey Dean and Sanjay Ghemawat
  7. –Oscar Stiffelman - early Google engineer “Only when something is

    simple can you make it really efficient”
  8. None
  9. @DerekB_WI Java Data Streams List<Integer> numbers = Arrays.asList(5, 22, 457,

    111233, 6);
  10. @DerekB_WI Java Data Streams List<Integer> numbers = Arrays.asList(5, 22, 457,

    111233, 6); List<Integer> filteredNumbers = (List<Integer>) numbers.stream() .filter(num -> (int)num > 10) .sorted() .map(num -> (int)num * -1) .collect(Collectors.toList());
  11. @DerekB_WI Java Data Streams List<Integer> numbers = Arrays.asList(5, 22, 457,

    111233, 6); Integer sum = numbers.stream() .reduce(0, (subtotal, element) -> subtotal + element);
  12. @DerekB_WI Java Data Streams List<Integer> numbers = Arrays.asList(5, 22, 457,

    111233, 6); List<Integer> filteredNumbers = (List<Integer>) numbers.parallelStream() .filter(num -> (int)num > 10) .sorted() .map(num -> (int)num * -1) .collect(Collectors.toList()); Integer sum = numbers.parallelStream() .reduce(0, (subtotal, element) -> subtotal + element);
  13. None
  14. @DerekB_WI JavaScript Promise function loadImage() { let imageUrl = 'https://live.staticflickr.com/

    65535/47813616341_6063c7b420_b_d.jpg'; fetch(imageUrl) .then((data) => { // Save image and display ... }) .catch((error) => { // Alert user of error ... }); }
  15. None
  16. @DerekB_WI Benefits • No Parallel Processing, So what else? •

    Functional, no side-effects • Easy to read • Single Responsibility • Reduce boilerplate
  17. None
  18. @DerekB_WI Traditional Foreach $placesArray = $reader->getFileContents();
 
 $filteredLocations = [];

    foreach ($placesArray as $place) {
 if ($place['state'] == 'AK') {
 $filteredLocations[] = new Place($place);
 }
 }
  19. @DerekB_WI Filter $placesArray = $reader->getFileContents(); $filteredAlaskaList = array_filter($placesArray,
 function($place) {


    return $place['state'] == 'AK';
 }
 );
  20. @DerekB_WI Filter $placesArray = $reader->getFileContents(); $filteredAlaskaList = array_filter($placesArray,
 function($place) {


    return $place['state'] == 'AK';
 }
 );
  21. @DerekB_WI Filter - 7.4 $placesArray = $reader->getFileContents(); $filteredAlaskaList = array_filter($placesArray,


    fn($place) => $place['state'] == 'AK'
 );
  22. None
  23. @DerekB_WI Traditional Foreach $placesArray = $reader->getFileContents();
 
 $filteredLocations = [];

    foreach ($placesArray as $place) {
 if ($place['state'] == 'AK') {
 $filteredLocations[] = new Place($place);
 }
 }
  24. @DerekB_WI Map $placesArray = $reader->getFileContents(); $modifiedList = array_map( function($place) {


    return PlaceMapper::parse($place);
 }, 
 $placesArray);
  25. @DerekB_WI Map - 7.4 $placesArray = $reader->getFileContents(); $filteredAlaskaList = array_map(


    fn($place) => PlaceMapper::parse($place), 
 $placesArray);
  26. None
  27. @DerekB_WI Reduce $myTrips = [
 new Trip("name" => "school", "distance"

    => 1.5),
 new Trip("name" => "the store", "distance" => 2.5)]; $totalDriven = array_reduce($myTrips,
 function ($total, $trip) {
 return $total + $trip->distance;
 },
 0 // initial
 ); echo "I biked {$totalDriven} kilometers today.";
  28. @DerekB_WI Reduce $myTrips = [
 new Trip("name" => "school", "distance"

    => 1.5),
 new Trip("name" => "the store", "distance" => 2.5)]; $totalDriven = array_reduce($myTrips,
 function ($total, $trip) {
 return $total + $trip->distance;
 },
 0 // initial
 ); echo "I biked {$totalDriven} kilometers today.";
  29. @DerekB_WI Reduce - 7.4 $totalDriven = array_reduce($myTrips,
 fn($total, $trip) =>

    $total + $trip->distance,
 0);
  30. @DerekB_WI Traditional Loop for ($i = 0; $i < count($placesArray);

    $i++) {
 $stateName = $placesArray[$i]['state']; if (!array_key_exists($stateName, $stateCounters)) 
 {
 $stateCounters[$stateName] = 0;
 } $stateCounters[$stateName]++; }
  31. @DerekB_WI Reduce $stateCounters = array_reduce($placesArray,
 function($totals, $location) {
 $totals[$location['state']] =


    isset($totals[$location['state']])
 ? $totals[$location['state']] + 1 : 1;
 return $totals;
 });
  32. @DerekB_WI Reduce $stateCounters = array_reduce($placesArray,
 function($totals, $location) {
 $totals[$location['state']] =


    isset($totals[$location['state']])
 ? $totals[$location['state']] + 1 : 1;
 return $totals;
 });
  33. @DerekB_WI Reduce $stateCounters = array_reduce($placesArray,
 function($totals, $location) {
 $totals[$location['state']] =


    isset($totals[$location['state']])
 ? $totals[$location['state']] + 1 : 1;
 return $totals;
 });
  34. @DerekB_WI Use an outside variable $searchWord = 'Bridge'; $placesWithBridges =

    array_filter($placesArray,
 function($location) use($searchWord) {
 return strpos($location['name'], $searchWord);
 });
  35. @DerekB_WI Patterns ApiPlacesController extends AbstractController { … }

  36. @DerekB_WI Patterns /**
 * route - /api/place
 */ public function

    allPlaces() {
 $places = $this->repository->retrieveAll(); 
 $placesDtos = array_map(
 function($place) {
 return PlacesTransformer::toDTO($place);
 }, $places); return convertToJson($placesDtos); }
  37. @DerekB_WI Patterns /**
 * route - /api/user/$userId/place
 */ public function

    placesByUser($userId) {
 $places = $this->repository->retrieveAll(); 
 $placesDtos = array_map(
 function($place) {
 return PlacesTransformer::toDTO($place);
 }, array_filter($places, function($place) {
 return LegacyMiddleware::shouldInclude($place);
 }
 )); return convertToJson($placesDtos);
 }
  38. None
  39. @DerekB_WI Immutable $searchWord = 'Bridge'; $placesWithBridges = array_filter($placesArray,
 function($location) use($searchWord)

    { $searchWord = ‘something else’;
 return strpos($location['name'], $searchWord);
 });
  40. @DerekB_WI Mutable array_walk(
 $placesArray,
 function(&$location, $key) {
 $location['distance'] = calcDistance($location);


    }); array_multisort(
 array_column($placesArray, 'distance'),
 SORT_ASC,
 $placesArray);
  41. @DerekB_WI Generators function location_file_generator($filename)
 {
 $file = fopen($filename, 'r'); while(($fileData

    = fgetcsv($file, 1000, "\t")) !== FALSE) { yield $fileData;
 }
 }
  42. @DerekB_WI Generators 
 foreach(location_file_generator($filename) as $line) 
 {
 ...
 }

  43. @DerekB_WI Generators • Pro: Reduced Memory Usage • Pro: Easy

    syntax • Con: Cannot use array functions
  44. @DerekB_WI Generator Map function map(Traversable $data, Callable $func) { foreach($data

    as $item) {
 yield call_user_func($func, $item);
 }
 }
  45. @DerekB_WI Generator Map foreach(map(location_file_generator($filename), 
 function($item) {
 return $item[1];
 })

    as $line) { … }
  46. @DerekB_WI Generator Filter function filter(Traversable $data, Callable $func) { foreach($data

    as $item) {
 if (call_user_func($func)) {
 yield $item;
 }
 }
 }
  47. @DerekB_WI Generator Filter foreach(filter(location_file_generator($filename), 
 function($item) {
 return $item[1] ==

    1;
 }) as $line) { … }
  48. @DerekB_WI Data Structures • Standard PHP Library - SPL •

    Common Structures - widely used in other languages • Many have filter and reduce functions • apt install php-ds • pecl install ds
  49. @DerekB_WI Hash Map $map = new DS\Map();
 $map->put('AK', PlaceFactory::make('Alaska'));
 $map->put('WI',

    PlaceFactory::make('Wisconsin')); $map->filter(function($key, $value) {
 return $value->population > 5000000;
 });
  50. @DerekB_WI Hash Map $map = new DS\Map();
 $map->put('AK', PlaceFactory::make('Alaska'));
 $map->put('WI',

    PlaceFactory::make('Wisconsin')); $map->reduce(function($carry, $key, $value) {
 return $carry + $value->population;
 }, 0);
  51. None
  52. @DerekB_WI Iterators class MyIter implements Iterator {
 protected $data;
 protected

    $index;
 protected $valid = true; … }
  53. @DerekB_WI Iterators public function __construct($inArray) {
 $this->data = $inArray;
 $this->index;


    }
  54. @DerekB_WI Iterators public function current() {
 return $this->data[$this->index];
 }

  55. @DerekB_WI Iterators public function next() {
 $this->index++; if ($this->index >=

    count($this->data)) {
 $this->valid = false;
 return null;
 } return $this->current(); }
  56. @DerekB_WI Iterators public function key() {
 return $this->index;
 } public

    function valid() {
 return $this->valid;
 } public function rewind() {
 $this->index = 0;
 }
  57. @DerekB_WI Invoke Iterator foreach(new MyIter($someArray) as $it) {
 …
 }

  58. @DerekB_WI Invoke Iterator + Map foreach(map(new MyIter($someArray), function($item) {
 …


    }) as $it) {
 …
 }
  59. @DerekB_WI Laravel Collections

  60. @DerekB_WI Laravel Collections

  61. @DerekB_WI CakePHP MapReduce

  62. @DerekB_WI CakePHP MapReduce

  63. @DerekB_WI Doctrine Collections

  64. @DerekB_WI Doctrine Collections

  65. @DerekB_WI Async/Process • Symfony process component • Promise library •

    Job Queues
  66. @DerekB_WI Promise use GuzzleHttp\Promise\Promise; $promise = new Promise(function() { …

    }); $result = $promise->then(
 function($value) {
 // handle result
 }
 );
  67. @DerekB_WI Resources • https://www.newyorker.com/magazine/2018/12/10/the-friendship-that-made-google-huge • https://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf • https://medium.com/@oscarstiffelman/a-brief-history-of-mapreduce-97aec97df8ff • https://www.sitepoint.com/parallel-programming-pthreads-php-fundamentals/

    • https://www.dragonbe.com/2015/07/speeding-up-database-calls-with-pdo-and.html • https://github.com/doctrine/collections/blob/master/lib/Doctrine/Common/Collections/Collection.php
  68. None
  69. None