Tweet
Tweet

Slide 1

Slide 1 text

@DerekB_WI Map, Filter, and Reduce in PHP Presented by Derek Binkley

Slide 2

Slide 2 text

@DerekB_WI About Me • Father of 3, Husband • Software Consultant with Spark Labs • PHP, Java, Groovy, JavaScript … • He/Him/His

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

@DerekB_WI

Slide 5

Slide 5 text

@DerekB_WI

Slide 6

Slide 6 text

@DerekB_WI Google in 2004 • MapReduce: Simplified Data Processing on Large Clusters, Jeffrey Dean and Sanjay Ghemawat

Slide 7

Slide 7 text

–Oscar Stiffelman - early Google engineer “Only when something is simple can you make it really efficient”

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

@DerekB_WI Java Data Streams List numbers = Arrays.asList(5, 22, 457, 111233, 6);

Slide 10

Slide 10 text

@DerekB_WI Java Data Streams List numbers = Arrays.asList(5, 22, 457, 111233, 6); List filteredNumbers = (List) numbers.stream() .filter(num -> (int)num > 10) .sorted() .map(num -> (int)num * -1) .collect(Collectors.toList());

Slide 11

Slide 11 text

@DerekB_WI Java Data Streams List numbers = Arrays.asList(5, 22, 457, 111233, 6); Integer sum = numbers.stream() .reduce(0, (subtotal, element) -> subtotal + element);

Slide 12

Slide 12 text

@DerekB_WI Java Data Streams List numbers = Arrays.asList(5, 22, 457, 111233, 6); List filteredNumbers = (List) numbers.parallelStream() .filter(num -> (int)num > 10) .sorted() .map(num -> (int)num * -1) .collect(Collectors.toList()); Integer sum = numbers.parallelStream() .reduce(0, (subtotal, element) -> subtotal + element);

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

@DerekB_WI JavaScript Promise function loadImage() { let imageUrl = 'https://live.staticflickr.com/ 65535/47813616341_6063c7b420_b_d.jpg'; fetch(imageUrl) .then((data) => { // Save image and display ... }) .catch((error) => { // Alert user of error ... }); }

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

@DerekB_WI Benefits • No Parallel Processing, So what else? • Functional, no side-effects • Easy to read • Single Responsibility • Reduce boilerplate

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

@DerekB_WI Traditional Foreach $placesArray = $reader->getFileContents();
 
 $filteredLocations = []; foreach ($placesArray as $place) {
 if ($place['state'] == 'AK') {
 $filteredLocations[] = new Place($place);
 }
 }

Slide 19

Slide 19 text

@DerekB_WI Filter $placesArray = $reader->getFileContents(); $filteredAlaskaList = array_filter($placesArray,
 function($place) {
 return $place['state'] == 'AK';
 }
 );

Slide 20

Slide 20 text

@DerekB_WI Filter $placesArray = $reader->getFileContents(); $filteredAlaskaList = array_filter($placesArray,
 function($place) {
 return $place['state'] == 'AK';
 }
 );

Slide 21

Slide 21 text

@DerekB_WI Filter - 7.4 $placesArray = $reader->getFileContents(); $filteredAlaskaList = array_filter($placesArray,
 fn($place) => $place['state'] == 'AK'
 );

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

@DerekB_WI Traditional Foreach $placesArray = $reader->getFileContents();
 
 $filteredLocations = []; foreach ($placesArray as $place) {
 if ($place['state'] == 'AK') {
 $filteredLocations[] = new Place($place);
 }
 }

Slide 24

Slide 24 text

@DerekB_WI Map $placesArray = $reader->getFileContents(); $modifiedList = array_map( function($place) {
 return PlaceMapper::parse($place);
 }, 
 $placesArray);

Slide 25

Slide 25 text

@DerekB_WI Map - 7.4 $placesArray = $reader->getFileContents(); $filteredAlaskaList = array_map(
 fn($place) => PlaceMapper::parse($place), 
 $placesArray);

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

@DerekB_WI Reduce $myTrips = [
 new Trip("name" => "school", "distance" => 1.5),
 new Trip("name" => "the store", "distance" => 2.5)]; $totalDriven = array_reduce($myTrips,
 function ($total, $trip) {
 return $total + $trip->distance;
 },
 0 // initial
 ); echo "I biked {$totalDriven} kilometers today.";

Slide 28

Slide 28 text

@DerekB_WI Reduce $myTrips = [
 new Trip("name" => "school", "distance" => 1.5),
 new Trip("name" => "the store", "distance" => 2.5)]; $totalDriven = array_reduce($myTrips,
 function ($total, $trip) {
 return $total + $trip->distance;
 },
 0 // initial
 ); echo "I biked {$totalDriven} kilometers today.";

Slide 29

Slide 29 text

@DerekB_WI Reduce - 7.4 $totalDriven = array_reduce($myTrips,
 fn($total, $trip) => $total + $trip->distance,
 0);

Slide 30

Slide 30 text

@DerekB_WI Traditional Loop for ($i = 0; $i < count($placesArray); $i++) {
 $stateName = $placesArray[$i]['state']; if (!array_key_exists($stateName, $stateCounters)) 
 {
 $stateCounters[$stateName] = 0;
 } $stateCounters[$stateName]++; }

Slide 31

Slide 31 text

@DerekB_WI Reduce $stateCounters = array_reduce($placesArray,
 function($totals, $location) {
 $totals[$location['state']] =
 isset($totals[$location['state']])
 ? $totals[$location['state']] + 1 : 1;
 return $totals;
 });

Slide 32

Slide 32 text

@DerekB_WI Reduce $stateCounters = array_reduce($placesArray,
 function($totals, $location) {
 $totals[$location['state']] =
 isset($totals[$location['state']])
 ? $totals[$location['state']] + 1 : 1;
 return $totals;
 });

Slide 33

Slide 33 text

@DerekB_WI Reduce $stateCounters = array_reduce($placesArray,
 function($totals, $location) {
 $totals[$location['state']] =
 isset($totals[$location['state']])
 ? $totals[$location['state']] + 1 : 1;
 return $totals;
 });

Slide 34

Slide 34 text

@DerekB_WI Use an outside variable $searchWord = 'Bridge'; $placesWithBridges = array_filter($placesArray,
 function($location) use($searchWord) {
 return strpos($location['name'], $searchWord);
 });

Slide 35

Slide 35 text

@DerekB_WI Patterns ApiPlacesController extends AbstractController { … }

Slide 36

Slide 36 text

@DerekB_WI Patterns /**
 * route - /api/place
 */ public function allPlaces() {
 $places = $this->repository->retrieveAll(); 
 $placesDtos = array_map(
 function($place) {
 return PlacesTransformer::toDTO($place);
 }, $places); return convertToJson($placesDtos); }

Slide 37

Slide 37 text

@DerekB_WI Patterns /**
 * route - /api/user/$userId/place
 */ public function placesByUser($userId) {
 $places = $this->repository->retrieveAll(); 
 $placesDtos = array_map(
 function($place) {
 return PlacesTransformer::toDTO($place);
 }, array_filter($places, function($place) {
 return LegacyMiddleware::shouldInclude($place);
 }
 )); return convertToJson($placesDtos);
 }

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

@DerekB_WI Immutable $searchWord = 'Bridge'; $placesWithBridges = array_filter($placesArray,
 function($location) use($searchWord) { $searchWord = ‘something else’;
 return strpos($location['name'], $searchWord);
 });

Slide 40

Slide 40 text

@DerekB_WI Mutable array_walk(
 $placesArray,
 function(&$location, $key) {
 $location['distance'] = calcDistance($location);
 }); array_multisort(
 array_column($placesArray, 'distance'),
 SORT_ASC,
 $placesArray);

Slide 41

Slide 41 text

@DerekB_WI Generators function location_file_generator($filename)
 {
 $file = fopen($filename, 'r'); while(($fileData = fgetcsv($file, 1000, "\t")) !== FALSE) { yield $fileData;
 }
 }

Slide 42

Slide 42 text

@DerekB_WI Generators 
 foreach(location_file_generator($filename) as $line) 
 {
 ...
 }

Slide 43

Slide 43 text

@DerekB_WI Generators • Pro: Reduced Memory Usage • Pro: Easy syntax • Con: Cannot use array functions

Slide 44

Slide 44 text

@DerekB_WI Generator Map function map(Traversable $data, Callable $func) { foreach($data as $item) {
 yield call_user_func($func, $item);
 }
 }

Slide 45

Slide 45 text

@DerekB_WI Generator Map foreach(map(location_file_generator($filename), 
 function($item) {
 return $item[1];
 }) as $line) { … }

Slide 46

Slide 46 text

@DerekB_WI Generator Filter function filter(Traversable $data, Callable $func) { foreach($data as $item) {
 if (call_user_func($func)) {
 yield $item;
 }
 }
 }

Slide 47

Slide 47 text

@DerekB_WI Generator Filter foreach(filter(location_file_generator($filename), 
 function($item) {
 return $item[1] == 1;
 }) as $line) { … }

Slide 48

Slide 48 text

@DerekB_WI Data Structures • Standard PHP Library - SPL • Common Structures - widely used in other languages • Many have filter and reduce functions • apt install php-ds • pecl install ds

Slide 49

Slide 49 text

@DerekB_WI Hash Map $map = new DS\Map();
 $map->put('AK', PlaceFactory::make('Alaska'));
 $map->put('WI', PlaceFactory::make('Wisconsin')); $map->filter(function($key, $value) {
 return $value->population > 5000000;
 });

Slide 50

Slide 50 text

@DerekB_WI Hash Map $map = new DS\Map();
 $map->put('AK', PlaceFactory::make('Alaska'));
 $map->put('WI', PlaceFactory::make('Wisconsin')); $map->reduce(function($carry, $key, $value) {
 return $carry + $value->population;
 }, 0);

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

@DerekB_WI Iterators class MyIter implements Iterator {
 protected $data;
 protected $index;
 protected $valid = true; … }

Slide 53

Slide 53 text

@DerekB_WI Iterators public function __construct($inArray) {
 $this->data = $inArray;
 $this->index;
 }

Slide 54

Slide 54 text

@DerekB_WI Iterators public function current() {
 return $this->data[$this->index];
 }

Slide 55

Slide 55 text

@DerekB_WI Iterators public function next() {
 $this->index++; if ($this->index >= count($this->data)) {
 $this->valid = false;
 return null;
 } return $this->current(); }

Slide 56

Slide 56 text

@DerekB_WI Iterators public function key() {
 return $this->index;
 } public function valid() {
 return $this->valid;
 } public function rewind() {
 $this->index = 0;
 }

Slide 57

Slide 57 text

@DerekB_WI Invoke Iterator foreach(new MyIter($someArray) as $it) {
 …
 }

Slide 58

Slide 58 text

@DerekB_WI Invoke Iterator + Map foreach(map(new MyIter($someArray), function($item) {
 …
 }) as $it) {
 …
 }

Slide 59

Slide 59 text

@DerekB_WI Laravel Collections

Slide 60

Slide 60 text

@DerekB_WI Laravel Collections

Slide 61

Slide 61 text

@DerekB_WI CakePHP MapReduce

Slide 62

Slide 62 text

@DerekB_WI CakePHP MapReduce

Slide 63

Slide 63 text

@DerekB_WI Doctrine Collections

Slide 64

Slide 64 text

@DerekB_WI Doctrine Collections

Slide 65

Slide 65 text

@DerekB_WI Async/Process • Symfony process component • Promise library • Job Queues

Slide 66

Slide 66 text

@DerekB_WI Promise use GuzzleHttp\Promise\Promise; $promise = new Promise(function() { … }); $result = $promise->then(
 function($value) {
 // handle result
 }
 );

Slide 67

Slide 67 text

@DerekB_WI Resources • https://www.newyorker.com/magazine/2018/12/10/the-friendship-that-made-google-huge • https://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf • https://medium.com/@oscarstiffelman/a-brief-history-of-mapreduce-97aec97df8ff • https://www.sitepoint.com/parallel-programming-pthreads-php-fundamentals/ • https://www.dragonbe.com/2015/07/speeding-up-database-calls-with-pdo-and.html • https://github.com/doctrine/collections/blob/master/lib/Doctrine/Common/Collections/Collection.php

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

No content