Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction to (Spatial) Data Science

Introduction to (Spatial) Data Science

JD Godchaux

More Decks by Texas Natural Resources Information System

Other Decks in Technology

Transcript

  1. Texas GIS Forum Workshop, October 22, 2019 Introduction to (Spatial)

    Data Science JD Godchaux, Customer Success Manager @ CARTO https://tinyurl.com/carto-sds-workshop https://tinyurl.com/carto-sds-workshop-folder
  2. Texas GIS Forum Workshop, October 22, 2019 Data Science

  3. Texas GIS Forum Workshop, October 22, 2019 What is Data

    Science "What do you think of when you read the phrase ‘data science’? It’s probably some combination of keywords like statistics, machine learning, deep learning, and ‘sexiest job of the 21st century’. Or maybe it’s an image of a data scientist, sitting at her computer, putting together stunning visuals from well-run A/B tests. Either way, it’s glamorous, smart, and sophisticated. This is the narrative that data science has been selling since I entered the field almost ten years ago.” - Vicki Boykis
  4. Texas GIS Forum Workshop, October 22, 2019 Well...

  5. Texas GIS Forum Workshop, October 22, 2019 What is Data

    Science
  6. Texas GIS Forum Workshop, October 22, 2019 What is Data

    Science
  7. Texas GIS Forum Workshop, October 22, 2019 "Data Scientist" most

    of the time means "Data Janitor" - Dongjie Fan, Data Scientist @ CARTO
  8. Texas GIS Forum Workshop, October 22, 2019 Many different tools

  9. Texas GIS Forum Workshop, October 22, 2019 Storytelling is important

  10. Texas GIS Forum Workshop, October 22, 2019 SQL is still

    important
  11. Texas GIS Forum Workshop, October 22, 2019 What Data Scientists

    Do • Prediction • Inference • Optimization
  12. Texas GIS Forum Workshop, October 22, 2019 What Data Scientists

    Do • A/B Testing on Website Path • Image Recognition • Predictions (Supervised Learning) • Unsupervised Learning • Natural Language Processing • Recommendation System (Netflix) • Time Series Analysis (trading) • Operation Research (Uber)
  13. Texas GIS Forum Workshop, October 22, 2019 Who are "Data

    Scientists"
  14. Texas GIS Forum Workshop, October 22, 2019 Who are "Data

    Scientists"? • Data Analyst • Business Analyst • Data Engineer • Machine Learning Engineer • Data Visualization • Developers
  15. Texas GIS Forum Workshop, October 22, 2019 What do Data

    Scientists do?
  16. Texas GIS Forum Workshop, October 22, 2019 Being able to

    work with data to understand problems
  17. Texas GIS Forum Workshop, October 22, 2019 Use different tools

    & methods to accomplish these tasks
  18. Texas GIS Forum Workshop, October 22, 2019 Validate the methods

    used to solve the assumption, even if results look good
  19. Texas GIS Forum Workshop, October 22, 2019 Be able to

    explain the process of how you reached a conclusion
  20. Texas GIS Forum Workshop, October 22, 2019 (Spatial) Data Science

  21. Texas GIS Forum Workshop, October 22, 2019 Spatial data science

    contains at least two aspects. One is adding spatial variables; the other is adding spatial items in the model. For example, f(x)=aX+b+error this is simple linear regression. you can add as many as self-created spatial variables here in X ; f(x)=aX+Sigma+b+error we add a covariance item Sigma which represents the influences caused by the spatial relationships from the whole dataset (not necessarily related to specific input x)
  22. Texas GIS Forum Workshop, October 22, 2019 ! Bringing the

    spatial data in as an explicit part of the analysis or taking "the where", distances, spatial arrangement into account
  23. Texas GIS Forum Workshop, October 22, 2019 " If the

    location changes, and the context of the data also changes, then you need SDS
  24. Texas GIS Forum Workshop, October 22, 2019 Spatial Analysis Questions

    • Where are things and where do they happen: clusters, hot spots, disparities • Why do they happen where they happen: understanding locations decisions and movement patterns • How does where things happen affect other things (context/environment) and how does context affect what happens: "I am my neighbors neighbor" • Where should things be located: optimization
  25. Texas GIS Forum Workshop, October 22, 2019 What makes spatial

    data unique • Spatial Context: Understanding the effect that your neighbors have on an observation, and vice versa • Spatial Support Problem: where scales of data do not match (zip codes and block groups) • Spatial Scale of Observations: behavior does not match the unit of observation (you know what neighborhood you live in but not with block group)
  26. Texas GIS Forum Workshop, October 22, 2019 What makes spatial

    data unique • Spatial Spillover: the activity of one location will impact the costs of other locations (closing a road will have an increased cost for a distribution center) • Spatial Multiplier: a successful store will not just impact that store, but the nearby stores as well • Spatial Decay: Observations change and decay as you move away from an observation
  27. Texas GIS Forum Workshop, October 22, 2019 80% of the

    effort is data preparation
  28. Texas GIS Forum Workshop, October 22, 2019 GIS v. Location

    Intelligence v. Spatial Data Science
  29. Texas GIS Forum Workshop, October 22, 2019 GIS GIS is

    the toolkit to work spatial data and do something with it - here are the tools and we train people to use those tools. Much (but not all) of this work is descriptive in nature - where the things are. Traditional GIS programs trained you on these tools, then you found careers using those tools. And with ESRI there are many many tools.
  30. Texas GIS Forum Workshop, October 22, 2019 Location Intelligence Location

    Intelligence focuses on outcomes from location data - you take the data in and you get some valuable insight out. Think about human mobility, they market immediate insight from location data, when the real process is much more complex.
  31. Texas GIS Forum Workshop, October 22, 2019 "Location Intelligence"

  32. Texas GIS Forum Workshop, October 22, 2019

  33. Texas GIS Forum Workshop, October 22, 2019 Spatial Data Science

    Spatial Data Science focuses on the the journey and the data, or the underlying conditions leading to the insight and prescribes recommendations for optimization Looking at the factors that cause something to occur, and creating models for spatial "optimization".
  34. Texas GIS Forum Workshop, October 22, 2019 USC Spatial Data

    Science Curriculum
  35. Texas GIS Forum Workshop, October 22, 2019 Data Science Takeaways

    • Data science is a verbose space with many tools and skills • Spatial data science concepts are newer to traditional 'data scientists' • There are many different roles and titles that all triangulate on ‘data science’
  36. Texas GIS Forum Workshop, October 22, 2019

  37. Texas GIS Forum Workshop, October 22, 2019 Spatial Data Science

    Takeaways • Spatial Data Science is data science with spatial attributes (with a little more complexity) • In Spatial Data Science, the journey to discover the problem is just as, if not more important, than the answer • It is important to start with a problem, then work from the beginning to align the steps to get there
  38. Texas GIS Forum Workshop, October 22, 2019 Now to the

    Notebooks!
  39. Texas GIS Forum Workshop, October 22, 2019 Ways of Notebooking

    - Jupyter Notebook (Installation in a virtual environment with CARTOframes) - Docker - Google Colaboratory - nteract - Kaggle - Glitch
  40. Texas GIS Forum Workshop, October 22, 2019 Notebooks for Today

    • Intro to Python • NumPy, Pandas, Seaborn • Introduction to CARTOframes • Helper Functions in CARTOframes • Los Angeles Real Estate Data Cleaning • Los Angeles Real Estate Exploratory Data Analysis