DyND.pdf

Transcript

1. DyND C++ dynamic array library, with Python exposure (http://github.com/libdynd) Irwin Zaid September 18, 2015
2. This Talk • Discussion at a high level – No implementation details • Code examples
3. Data?
4. Data? Missing Values Composite Structures Heterogeneous Storage The Usual Case Irregularly Shaped
5. Principles of DyND • Store data in its native layout • Enable computation with generic patterns • Be a cross-language intermediary (C++, Python, R?, Julia?, JavaScript?, …)
6. Types, Metadata, and Data • Types are instances of simple classes – Write a class, get a type • Types expose dynamic features to arrays – Either properties, like .real() or .imag(), or behavior, like .conj() • Types can be kinds, patterns, or constraints – Int, Scalar, Fixed, or Any; Fixed * T or (N * T, T) -> T; C[3 * 2 * float64] • Array metadata can describe data other than strided – Offset (tuple or struct), indirect (pointer), ragged (variable-sized dimensions) • Array data is poolable or allocatable in custom memory spaces – Variable-sized strings or dimensions; CUDA
7. Fundamental Types
8. Dimension Types
9. Composite Types
10. Option Type
11. Symbolic Types
12. Callables and Functionals • Share functions alongside data – Callables are first-class objects that can be dynamically published • Enable user-defined functions with generic patterns – Functionals like apply, elwise, reduction, multidispatch, outer, neighborhood, and rolling transform one callable into another • Built-in callables are overloadable – Users can define +, -, *, /, … for custom types • Write once, retarget later – Capture LLVM IR from C++ at compile-time
13. Elementwise Functional
14. Reduction Functional
15. Multidispatch Functional
16. JSON Processing
17. Option Operations
18. LLVM Support
19. Just-in-Time and Ahead-of-Time • JITing happens automatically with Numba if it is importable – Previous examples of user-defined functions can then drop the return type, it will be deduced at runtime • AOT planned, it will work by specifying the argument types of user-defined functions
20. Contributors
21. Get DyND! conda install dynd-python –c dynd/channel/dev