Neat Analytics with Pandas [PyParis]

Pandas is the Swiss-Multipurpose Knife for Data Analysis in Python. In this talk we will look deeper into how to gain productivity utilising Pandas powerful indexing and make advanced analytics a piece of cake.

Transcript

1. Neat Analytics with Pandas A Closer Look at Pandas Indexing Alexander C. S. Hendorf @hendorf
2. Alexander C. S. Hendorf Königsweg GmbH Strategic data consulting for startups and the industry. EuroPython & PyConDE 
 Organisator + Programm Chair mongoDB master, PSF managing member Speaker mongoDB days, EuroPython, PyData… @hendorf
3. Today Closer Look at Indexes - Catch up on Pandas indexing - Accessing data using the index - Index Types - MultiIndex - Closer look at DateTimeIndex and Resampling
9. Structure: Index -the label of a series is usually called index -automatically created if not given -can be reset or replaced -immutable ndarray implementing an ordered, sliceable set -can only contain hashable objects -one or more dimensions -may contain a value more than once (NOT UNIQUE!)
10. Index Types -Index -MultiIndex -DateTimeIndex -TimeDelta -IntervalIndex -CategoricalIndex
11. Structure DataFrame 2D 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 … Panel 3D XXXXXXXXX pd.Series 1D 1 2 3 4 5 6 7 1 2 3 4 5 6 7 Index Data: Numpy array
15. Axes 1 2 3 0 1 0 1 2 3 4 5 6
18. no match: NaN only if index matches
23. MultiIndex
28. max value of each series (not row) get level 0 data get level 1 data get level 2 data • • • •
29. DateTimeIndex -index of datetime64 data
30. 2014-09-26T03:50:00,14.0 2014-08-10T05:00:00,14 2014-08-21T22:50:00,12.0 2014-08-17T13:20:00,16.0 2014-08-06T01:20:00,14.0 2014-09-27T06:50:00,11.0 2014-08-25T21:50:00,13.0 2014-08-14T05:20:00,13.0 2014-09-14T05:20:00,16.0 2014-08-03T02:50:00,21.0 2014-09-29T03:00:00,13 2014-09-06T08:20:00,16.0 2014-08-19T07:20:00,13.0 2014-09-27T22:50:00,10.0 2014-08-28T08:20:00,12.0 2014-08-17T01:00:00,14 2014-09-27T14:00:00,17 2014-09-10T18:00:00,18 2014-09-22T23:00:00,8 2014-09-20T03:00:00,9 2014-08-29T09:50:00,16.0 2014-08-16T01:50:00,13.0
32. DateTime format="%d.%m.%Y %H:%M:%S
33. before DateTimeIndex: unordered
41. Resampling
45. Resampling -H hourly frequency -T minutely frequency -S secondly frequency -L milliseonds -U microseconds -N nanoseconds -D calendar day frequency -W weekly frequency -M month end frequency -Q quarter end frequency -A year end frequency - B business day frequency - C custom business day frequency (experimenta - BM business month end frequency - CBM custom business month end frequency - MS month start frequency - BMS business month start frequency - CBMS custom business month start frequency - BQ business quarter endfrequency - QS quarter start frequency - BQS business quarter start frequency - BA business year end frequency - AS year start frequency - BAS business year start frequency - BH business hour frequency
46. Extra discounts for students & post docs #16 180+ sessions 18 free trainings panels open spaces 5d talks & trainings 2d sprints beginners’ day Tickets start @ 375€ Rimini . Venice ! Bologna ! ✈ . Florence ! . # $ Rome ! . Armin Rohnacher • Katharine Jarmul • Tracy Osborn Jan Willem Tulp • Aisha Bello & Daniele Procida interactive sessions
47. 25. - 27. October ZKM, Karlsruhe CfP is open!
48. Alexander C. S. Hendorf ah@koenigsweg.com @hendorf