a language ‣ Helps do computation with arrays ‣ “Modern” means “uses software engineering practices from today, solving problems people have had for a while”
not NumPy 2.0 - NumPy is really good at what it was designed to do: broadcasting computation and stride-based manipulation - NumPy may not be “replaceable” — Fortran is still pretty widely used ‣ DyND is not a JIT for Python - See Numba for that ‣ DyND is not Boost.MultiArray - DyND is dynamic C++, templates are a hidden detail
‣ Example: Missing data - Values that may or may not be present - The masked arrays of numpy.ma are not sufficient, there is overhead related to how the masked array is stored and NumPy is not always consistent with how it treats mask arrays - Discussed at length in 2011, remains unsolved in 2016
‣ Example: Variable-length strings - NumPy can only efficiently represent strings of a predefined length - Variable-length strings have to be stored as Python objects
‣ Example: Custom types - NumPy dtypes are too primitive - How does one represent categorical data? Ragged dimensions? GPU data? - Cannot define user overloads on ufuncs, e.g. string concatenation
‣ Example: NumPy without the “Py” - Sometimes we don’t want to use Python - Why not have a representation of data that can go between several languages? (R, Julia, Javascript, …)
type system - A structured data description language, http://datashape.pydata.org type dimension * dtype var * int32 3 * string 4 * float64 Datashape: Struct type {x: int32, y: string, z: float64} Tabular data var * {x: int32, y: string, z: float64}
objects are reference- counted smart pointers that dynamically interpret data ‣ Types can be parameterized on other types - N * T, var[T], option[T] ‣ Callables can be transformed (in a functional sense) from inner operations to higher-order patterns - Define the innermost operation, then build out the behavior you want with predefined generic patterns - nd::functional::elwise([](int x, int y) { return x + y; });
Define custom types and callables (or namespaces thereof) directly ‣ Use nd::set(“my_amazing_callable”, f) for a custom callable or nd::set(“my_amazing_namespace”, {{“my_amazing_callable”, f}, {“my_other_amazing_callable”, g}}) for a custom namespace ‣ Callables are dynamically propagated to Python, entirely removing the need for any user wrapper code
classes - Write a class, get a type ‣ Types expose dynamic features to arrays - Either properties, like .real or .imag, or behavior, like .conj() ‣ Types can be kinds or patterns - Int, Scalar, Fixed, or Any; Fixed * T or (N * T, T) -> T
describe data other than strided - Offset (tuple or struct), indirect (pointer), ragged (variable-sized dimensions) ‣ Array data is poolable or allocatable in custom memory spaces - Variable-sized strings or dimensions; CUDA
data - Callables are first-class objects that can be dynamically published ‣ Enable user-defined functions with generic patterns - Functionals like apply, elwise, reduction, multidispatch, outer, neighborhood, and rolling transform one callable into another ‣ Built-in callable are overloadable - Users can define +, -, *, /, … for custom types
Irwin Zaid
izaid
reverentgeek
rmw
skipperchong
carmenhchung
keathley
productmarketing
clairelew
thoeni
jcasabona
cherdarchuk
lauravandoore
