Pythonic functional (iter)tools for your data challenges [email protected] @leriomaggio Image Source: https://scryfall.com/card/apc/112/mystic-snake All content released under the Wizards of the Coast’s Fan Content Policy Valerio Maggio 

Still • Researcher and Data Scientist • ML/DL for BioMedicine • Data Scientists Advocate • SSI Fellow • Python Geek 
 • Casual M:TG Player me pun Who 🧙 “a short summary of myself in logos” @mtg_lotus_vale I’m Valerio 

Refs & Mentions 
 & Stocktone to Malone… 

This talk is absolutely brilliant! And This (this 👇) talk can be considered as a RECAP + COMPLEMENT 

Pythonic functional (iter)tools for your data challenges [email protected] @leriomaggio Image Source: https://scryfall.com/card/vis/155/snake-basket All content released under the Wizards of the Coast’s Fan Content Policy Clumsy and Convoluted Python code (that only you can possibly understand (maybe)) (( )) Valerio Maggio 

Advent of Code 🎄https://adventofcode.com 

from Advent of Code to Coders of Advent Private Leaderboards to have fun with stats Public shout out to my friends and colleagues! You folks rock! 🫶 Honorable Mentions: @Taifu 
 @antigones, @edobld, @davrizzo, @Andrea 
 @paulox, @veggero, @greenkey, @akiross, @DaBenny, @SaltySpaghetti Special Thanks: 
 Daniel Holth & 
 Michael C. Grant 

from Advent of Code to Coders of Advent https://github.com/leriomaggio/AoC We had only one rule: 
 Just use the Standard Library Valerio: Why not using functional programming too sometimes? 

Algorithmic 
 Adventures Image Source:https://scryfall.com/card/unh/108/remodel All content released under the Wizards of the Coast’s Fan Content Policy What to expect in AoC 

Python is fantastic! 

Python is fantastic! Support for many programming styles (including functional programming) without enforcing any! 

1. Functions Functional Programming “(pure) functional programming” is a gradient among various languages Our definition: FP has to do with functions over stream of data 🙃 

Functional Programming 1. Pure functions •In FP, input fl ows through a set of functions •Every function should not have any side e ff ects 🧟 •Every output must depend only on its input 

ReLU function max(0, x) Functional Programming 1. Pure functions & Mutability ⚠ Side e ff ect 🫶 List Comprehension 

• FP languages require functions to: 1.To take another function as an argument 2.To return another function to its caller • A higher-order function takes one or more functions as input and returns a new function. Lambda functions Functions are fi rst-class citizens in FP & Functional Programming 1.5 High-Order Functions 

Iterable Stream of Data map map(f, ) f( ) f( ) f( ) f( ) 

f ilter fi lter(f, ) 

Image Source: https://scryfall.com/card/4ed/34/land-tax All content released under the Wizards of the Coast’s Fan Content Policy functools Higher-order functions and operations on callable objects 

Partial function application (“currying”) functools.partial 

Iterable Stream of Data functools.reduce reduce(f, ) f( , ) f( f( , ) , ) = 

1. Functions Functional Programming “(pure) functional programming” is a gradient among languages Our definition: FP has to do with functions over stream of data 2. Iterators and Iterables! ✔ 

Functional Programming 2. Iterators vs Iterables (vs Generators) • Iterator: an object representing a stream of data (key: Laziness) • Returns one element at a time - next(it) • If there are no more elements, a StopIteration exception is raised. • Iterable: an object is an iterable, if you could get an iterator for it • Generator: a convenient function to return an iterator over a stream of data • Function: return a value (i.e. data) • Generator: return an iterator which yields a value 

Functional Programming Common operations on iterator’s output: • Element wise ops (e.g. ReLU function) • Selection and fi ltering 2. Iterators vs Iterables (vs Generators) listcomps & genexps (borrowed from Haskell) 

Functional Programming • Iterators are stateful • Once you’ve consumed the iterator, it’s gone. You’d need a new one! many built-ins return iterators (e.g. map, f i lter, enumerate, reversed, zip) ⚠ range is not an Iterator! 2. Iterators vs Iterables (vs Generators) 

Functional Programming ⚠ range is not an Iterator! Cannot call next() Can call len() Is never consumed Is subscriptable 

The range_object is a lazy iterable • The distinction may matter for your program! 1.You always expect an Iterator to be consumed 2.You always expect to call next() on Iterators 

Functional built-ins 

Functional built-ins 

Functional built-ins 

itertools Image Source: https://scryfall.com/card/wth/130/harvest-wurm All content released under the Wizards of the Coast’s Fan Content Policy Functions creating iterators for e ff i cient looping 

Iterable Stream of Data itertools.accumulate accumulate(f, ) f( , ) f( , ) f( , ) f( , ) f( f( , ) , ) 

Itertools Functions creating iterators for e ff i cient looping 

Itertools Functions creating iterators for e ff i cient looping Very e ff i cient over large stream of data In itertools since Py3.10 

Itertools Functions creating iterators for e ff i cient looping Will be In itertools 
 in Py3.12 

Coding 
 Challenges Image Source:https://scryfall.com/card/mmq/61/brainstorm All content released under the Wizards of the Coast’s Fan Content Policy Putting (almost) everything in practice with AoC + new 💎 

Warm up from Sonar Sweep - adventofcode.com/2021/day/1 - part 1 Pairwise in 3.10 Iterator does not have len 

Warm up from Sonar Sweep - adventofcode.com/2021/day/1 - part 2 more-itertools Lookup 

Reduce at its best from Binary Diagnostic- adventofcode.com/2021/day/3 - part 1 We need to slice the data vertically! 

Reduce at its best from Binary Diagnostic- adventofcode.com/2021/day/3 - part 1 Walrus Operator Slice vertically 

Moving on an in f inite 2d Grid from Rope Bridge - adventofcode.com/2022/day/9 - part 1 Versors coords: (-1, -1), (-1, 0), … 

Moving on a f inite 2d f inite Grid from Rope Bridge - adventofcode.com/2021/day/9 - part 1 functools.starmap(f, * args) 

Moving on a f inite 2d f inite Grid (cont.) from Rope Bridge - adventofcode.com/2021/day/9 - part 1 

The key_fn is the key from Rope Bridge - adventofcode.com/2022/day/13 - part 2 

The key_fn is the key (cont.) from Rope Bridge - adventofcode.com/2022/day/13 - part 2 functools.cmp_to_key(f) Quicksort, Timsort, Powersort - Algorithmic ideas, engineering tricks, and trivia behind CPython's new sorting algorithm 
 Sebastian Wild | Saturday, 255ABC 3:15 

Why Functional Programming ? • High level: focus on the result rather than explicitly specifying the steps to get it. • Transparent: The behaviour of a pure function depends only on its inputs and outputs, without intermediary values. • That eliminates the possibility of side e ff ects, which facilitates debugging. • Parallelizable: 
 Executions with no side e ff ects can more easily run in parallel with one another. • Data / Coding challenges ? • FP o ff ers an alternative way to express a solution that is more natural 

Neural Networks as High-Order Functions NN - > linear(relu(linear(…(relu(linear(X)))))) Image: Jose-Luis Olivares/MIT linear ~ > sum(itertools.starmap(operator.mul, zip(W_i, x_j))) https://github.com/python/cpython/issues/100485 math.sumprod PR by Raimond Hattinger 

Functional Programming in Python By Example By Erik Welch https://bit.ly/functional-python-lightning 

References • Functional Programming HOWTO | https://docs.python.org/3/howto/functional.html • Joel Grus: Learning Data Science Using Functional Python | https://www.youtube.com/watch? v=ThS4juptJjQ • "The Joy of Functional Programming (for Data Science)" with Hadley Wickham 
 https://www.youtube.com/watch?v=bzUmK0Y07ck • Functional Programming in Python | https://realpython.com/python-functional-programming/ • Tour of Python Itertools | https://martinheinz.dev/blog/16 • (How to Write a (Lisp) Interpreter (in Python)) | https://norvig.com/lispy.html • (An ((Even Better) Lisp) Interpreter (in Python)) | https://norvig.com/lispy2.html • https://github.com/norvig/pytudes/blob/main/py/lis.py 

Thank you very much 
 for your kind attention Valerio Maggio [email protected] @leriomaggio