Data Preparation with Pandas (BNCC Bandung CSR 2021)

Transcript

1. Data Preparation with Pandas Elvan Selvano Data Scientist at Teman Data | Graduate Student at BINUS @elvanselvano | linktr.ee/elvanselvano
2. https://towardsdatascience.com/data-science-methodology-101-ce9f0d660336
3. 80% of a data scientist’s valuable time is spent simply finding, cleansing, and organizing data, leaving only 20 percent to actually perform analysis. IBM Data Analytics https://www.ibm.com/cloud/blog/ibm-data-catalog-data-scientists-productivity
4. https://www.compassred.com/data-journal/explaining-the-world-of-data-through-memes
5. https://quantdare.com/data-cleansing-and-transformation/
6. https://medium.com/well-red/cleaning-a-messy-dataset-using-python-7d7ab0bf199b
7. https://i.imgur.com/xOJr921h.jpg https://www.rfwireless-world.com/images/Data-Quality-Problems.jpg
8. Libraries and Tools for Data Science https://dev.to/minchulkim87/my-data-science-tech-stack-2020-1poa
9. Panel Data https://miro.medium.com/max/1400/1*5Uza5wbRmOGl-05h5J2pAw.jpeg
10. https://colab.research.google.com/github/QuantEcon/lecture-python-programming.notebooks/blob/master/pandas.ipynb
11. Pandas pandas is a fast, powerful, flexible and easy to use open source data analysis and manipulation tool, built on top of the Python programming language. https://blogs.sap.com/2021/05/24/easy-descriptive-statistics-with-python-and-sap-hana-cloud/
12. Pandas • DataFrame object for data manipulation with integrated indexing. • Tools for reading and writing data between in-memory data structures and different file formats. • Data alignment and integrated handling of missing data. • Reshaping and pivoting of data sets. • Label-based slicing, fancy indexing, and subsetting of large data sets. • Data structure column insertion and deletion. • Group by engine allowing split-apply-combine operations on data sets. • Data set merging and joining. • Time series-functionality: Date range generation and frequency conversions, moving window statistics, moving window linear regressions, date shifting and lagging. • …. https://en.wikipedia.org/wiki/Pandas_(software)
13. Hands-on Data Cleaning bit.ly/handson-pandas-bncc https://ai.sophos.com/wp-content/uploads/2020/09/Data-cleaning-meme.png