Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction to Jupyter Notebooks & Data Analysis using Kaggle

Introduction to Jupyter Notebooks & Data Analysis using Kaggle

PyLadies Dublin Feb 2019 Meetup
Mon, Feb 18 @ Dogpatch Labs
Event page: https://www.meetup.com/PyLadiesDublin/events/dclgvlyzdbzb/

Title:
Introduction to Jupyter Notebooks & Data Analysis using Kaggle by Leticia Portella

Short Description:
We will give an overview on what are Jupyter Notebooks and how they work, how to read and user a dataset based on a spreadsheet and how can we explore the data using Pandas and Matplotlib. This is an introductory workshop for people interested in data analysis.

Setup before workshop
- An account to Kaggle (https://www.kaggle.com/) already verified. We will use the Titanic dataset (https://www.kaggle.com/sureshbhusare/titanic-dataset-from-kaggle) for exploratory analysis.

About Leticia Portella
Leticia is an oceanographer that fell in love with programming. It is one of the hosts of the first brazilian podcast on Data Science, Pizza de Dados and is currently working with Project Jupyter at the Outreachy program. leportella.com

3476530ee3199731f810cb41daadad79?s=128

PyLadies Dublin

February 19, 2019
Tweet

More Decks by PyLadies Dublin

Other Decks in Technology

Transcript

  1. Introduction to Jupyter Notebooks & Data Analysis using Kaggle

  2. LETICIA PORTELLA /in/leportella @leportella @leleportella leportella.com pizzadedados.com

  3. Kaggle is a place where you can find a lot

    of datasets, it already have installed most of tools you’ll need for a basic analysis, is a good place to see the people’s code and built a portfolio Why Kaggle?
  4. Choosing a dataset https://www.kaggle.com/datasets

  5. None
  6. https://www.kaggle.com/vikichocolate/ titanic-machine-learning-from-disaster

  7. None
  8. None
  9. Notebooks are a place where you can create code, show

    graphs, document your methodologies and findings… all in a single place
  10. You can write code You can write text

  11. And they work well together

  12. Numbers on code cells indicate the order in which each

    cell ran
  13. Notebooks always print the last line (event without the print

    statement)
  14. Jupyter Shortcuts Ctrl + Enter = Run cell ESC +

    B = New cell below ESC + dd = Delete cell
  15. Run the first cell

  16. Reading a document If you check the first cell, it

    will tell you that the documents are ready for you in ../input/. So, we can read the files by with a Pandas function and with the path of the file df = pd.read_csv(‘../input/train.csv')
  17. Dataframes Dataframes are similiar to what you find in Excel

    structures. You have rows indicated by numbers and columns with names. You can check the first 5 rows of a data frame to see the basic structure: df.head()
  18. Dataframes

  19. Dataframes Dataframe columns: PassengerId, Survived, Pclass…

  20. Dataframes Dataframe rows: 0, 1, 2…

  21. Dataframes You can check the structure of a dataframe, to

    get an idea of how many rows and columns it has: df.shape
  22. Dataframes You can check the main statistical characteristics of the

    numerical columns of a data frame df.describe()
  23. Dataframes df.describe()

  24. Series You can select a single column of the data

    frame to work with. A column of a Dataframe is called Series and have some special properties df['Age']
  25. Series You can also check the statistical characteristics of a

    Series df['Age'].describe()
  26. Series You can filter a series to see which rows

    have adults. This will return a Series of True and False. df[‘Age'] > 10
  27. Series And Series have functions that help you quickly plot

    some of it. We can, for instance, check the histogram of Ages. df[‘Age’].plot.hist()
  28. Exercise Plot the histogram of Fares

  29. Series And Series have functions that help you quickly plot

    some of it. We can, for instance, check the histogram of Ages. df[‘Age’].plot.hist()
  30. Series We can count how many passengers were on each

    class df[‘Pclass’].value_counts()
  31. Series Since the result of a value_counts is also a

    Series, we can store this value in a variable and use it to plot a pie chart :) passengers_per_class = df[‘Pclass’].value_counts() passengers_per_class.plot.pie()
  32. Exercise Plot a bar plot with the number of people

    that survived and didn’t survive (Column Survived)
  33. Series Remember we could filter a series? We could use

    it to checkout our variables. Let’s see which class survived the most survived = df[‘Survived'] > 0 filtered_df = df[survived] passenger_per_class = filtered_df[“Pclass”].value_counts() passenger_per_class.plot.pie()
  34. Function Let’s build a function that “breaks" the age into

    classes:
  35. Function Now if we pass an age to the function

    it returns a label:
  36. Series We can create a new column (Ageclass) using the

    Column Age and this function :) df[“Ageclass”] = df[“Age”].apply(age_to_ageclass)
  37. Exercise Now we have classes for age, we can check

    which sector survived the most, the same we did with Class :)
  38. Dataframes We can group two columns to count df[“Ageclass”] =

    df[“Age”].apply(age_to_ageclass)