209f54f0b8edd93e50c7bf9870037ac5?s=47 Irwin Zaid
September 18, 2015



Irwin Zaid

September 18, 2015


  1. DyND C++ dynamic array library, with Python exposure (http://github.com/libdynd) Irwin

    Zaid September 18, 2015
  2. This Talk • Discussion at a high level – No

    implementation details • Code examples
  3. Data?

  4. Data? Missing Values Composite Structures Heterogeneous Storage The Usual Case

    Irregularly Shaped
  5. Principles of DyND • Store data in its native layout

    • Enable computation with generic patterns • Be a cross-language intermediary (C++, Python, R?, Julia?, JavaScript?, …)
  6. Types, Metadata, and Data • Types are instances of simple

    classes – Write a class, get a type • Types expose dynamic features to arrays – Either properties, like .real() or .imag(), or behavior, like .conj() • Types can be kinds, patterns, or constraints – Int, Scalar, Fixed, or Any; Fixed * T or (N * T, T) -> T; C[3 * 2 * float64] • Array metadata can describe data other than strided – Offset (tuple or struct), indirect (pointer), ragged (variable-sized dimensions) • Array data is poolable or allocatable in custom memory spaces – Variable-sized strings or dimensions; CUDA
  7. Fundamental Types

  8. Dimension Types

  9. Composite Types

  10. Option Type

  11. Symbolic Types

  12. Callables and Functionals • Share functions alongside data – Callables

    are first-class objects that can be dynamically published • Enable user-defined functions with generic patterns – Functionals like apply, elwise, reduction, multidispatch, outer, neighborhood, and rolling transform one callable into another • Built-in callables are overloadable – Users can define +, -, *, /, … for custom types • Write once, retarget later – Capture LLVM IR from C++ at compile-time
  13. Elementwise Functional

  14. Reduction Functional

  15. Multidispatch Functional

  16. JSON Processing

  17. Option Operations

  18. LLVM Support

  19. Just-in-Time and Ahead-of-Time • JITing happens automatically with Numba if

    it is importable – Previous examples of user-defined functions can then drop the return type, it will be deduced at runtime • AOT planned, it will work by specifying the argument types of user-defined functions
  20. Contributors

  21. Get DyND! conda install dynd-python –c dynd/channel/dev