Numpy, the Python foundation for number crunching

Talk by Didrik Pinte, MD at Enthought at Data Science London meetup 18/10/12

Transcript

1. Numpy, the Python foundation for number crunching Didrik Pinte, Enthought London Data Science meetup Monday 22 October 2012
2. Number crunching? •High-level api •Interactivity & visualization •Performance •Low-level access Monday 22 October 2012
3. Evidence ? PyCuda PyTrilinos Joblib petsc4py PyAlgoTrade numpy-boost Clyther PyOpenGL Monday 22 October 2012
4. Evidence ? PyCuda PyTrilinos Joblib petsc4py PyAlgoTrade numpy-boost Clyther PyOpenGL Monday 22 October 2012
5. Why then? •The API ... •Simple but powerful memory model •Open access to the data Monday 22 October 2012
6. Array data structure Monday 22 October 2012
7. Let’s look at the code! •Examples: – API / interactivity – memory management with stride_tricks – pikos – extensions with talib –(joblib, memmap/multiprocessing, ipython //) Monday 22 October 2012
8. API / interactivity Monday 22 October 2012
9. Memory management 1 2 3 4 5 6 7 8 9 10 1 2 3 4 2 3 4 5 3 4 5 6 12 11 = 4 5 6 7 5 6 7 8 6 7 8 9 7 8 9 10 8 9 10 11 ... Monday 22 October 2012
10. Memory management 1 2 3 4 5 6 7 8 9 10 1 2 3 4 2 3 4 5 3 4 5 6 12 11 = Shape 12, Strides 8, Shape 9,4, Strides 8,8 4 5 6 7 5 6 7 8 6 7 8 9 7 8 9 10 8 9 10 11 ... Monday 22 October 2012
11. Memory management - pikos Monday 22 October 2012
12. Memory management - chaco Monday 22 October 2012
13. Low level access %timeit talib.moving_average(adj_close, optInTimePeriod=5) 100000 loops, best of 3: 7.67 us per loop %timeit as_strided(adj_close, shape=(len(adj_close)-4, 5), strides=(8, 8)).mean (axis=1) 10000 loops, best of 3: 28.2 us per loop Monday 22 October 2012
14. Conclusion •It’s obvious, no? Monday 22 October 2012
15. Q & A ? Monday 22 October 2012