Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Python Neogeography

Adam Fast
October 27, 2012

Python Neogeography

Short talk about Neogeography, with a focus on pointing you to batteries in Python that are available.

Adam Fast

October 27, 2012
Tweet

More Decks by Adam Fast

Other Decks in Programming

Transcript

  1. [Python] Neogeography Teaching your code about where Adam Fast pyArkansas

    2012 http://blog.adamfast.com/ Saturday, October 27, 12 Mostly a pep talk If you want to build a ship, don’t find people and assign them work - teach them to long for the sea. - Antoine de Saint Exupery Video from tutorial My blog
  2. Neogeography • “new geography” • “non-experts” • Examples: Google Earth

    / Google Maps • “experts” Saturday, October 27, 12
  3. Neogeography • “new geography” • “non-experts” • Examples: Google Earth

    / Google Maps • “experts” • ESRI / governments / architects Saturday, October 27, 12
  4. Neogeography • “new geography” • “non-experts” • Examples: Google Earth

    / Google Maps • “experts” • ESRI / governments / architects • GIS day November 14 - check it out Saturday, October 27, 12
  5. Neogeography • “new geography” • “non-experts” • Examples: Google Earth

    / Google Maps • “experts” • ESRI / governments / architects • GIS day November 14 - check it out • Ft. Smith / Memphis / Lawrence and others Saturday, October 27, 12
  6. • Photo of panel on trip down Copyright Cessna Aircraft

    http://www.cessna.com/single-engine/skyhawk/skyhawk-gallery.html Saturday, October 27, 12 Modern light aircraft are PACKED with GIS This is what attracted me to the field - I wanted to DIY my own Then the iPad happened, and while I have open sourced the core libraries, I never did display of data
  7. GIS? • Geographic Information Systems • Farming • Aviation •

    Driving • Police Dispatch / 911 / Emergency Management Saturday, October 27, 12
  8. http://www.flickr.com/photos/rlsdave/6480088333/ Saturday, October 27, 12 GPS makes this possible. Before

    that it was known landmarks, scopes and trigonometry (but now there are lasers too)
  9. Why should I care? • Information is easily accessible Saturday,

    October 27, 12 Map: Evernote Other: Ham radio repeaters
  10. Why should I care? • Information is easily accessible •

    Users growing to expect it Saturday, October 27, 12 Map: Evernote Other: Ham radio repeaters
  11. Why should I care? • Information is easily accessible •

    Users growing to expect it • GREAT for hackathons Saturday, October 27, 12 Map: Evernote Other: Ham radio repeaters
  12. Why should I care? • Information is easily accessible •

    Users growing to expect it • GREAT for hackathons • Location is valuable metadata Saturday, October 27, 12 Map: Evernote Other: Ham radio repeaters
  13. Gathering Geospatial Saturday, October 27, 12 TIGER/LINE - gathering this

    data is their job SimpleGeo shut down, you can find an API from factual.com or download the JSON uscampgrounds
  14. Gathering Geospatial • Census Bureau Saturday, October 27, 12 TIGER/LINE

    - gathering this data is their job SimpleGeo shut down, you can find an API from factual.com or download the JSON uscampgrounds
  15. Gathering Geospatial • Census Bureau • TIGER/LINE Saturday, October 27,

    12 TIGER/LINE - gathering this data is their job SimpleGeo shut down, you can find an API from factual.com or download the JSON uscampgrounds
  16. Gathering Geospatial • Census Bureau • TIGER/LINE • Local governments

    (city/county/state) Saturday, October 27, 12 TIGER/LINE - gathering this data is their job SimpleGeo shut down, you can find an API from factual.com or download the JSON uscampgrounds
  17. Gathering Geospatial • Census Bureau • TIGER/LINE • Local governments

    (city/county/state) • data.gov Saturday, October 27, 12 TIGER/LINE - gathering this data is their job SimpleGeo shut down, you can find an API from factual.com or download the JSON uscampgrounds
  18. Gathering Geospatial • Census Bureau • TIGER/LINE • Local governments

    (city/county/state) • data.gov • GeoNames Saturday, October 27, 12 TIGER/LINE - gathering this data is their job SimpleGeo shut down, you can find an API from factual.com or download the JSON uscampgrounds
  19. Gathering Geospatial • Census Bureau • TIGER/LINE • Local governments

    (city/county/state) • data.gov • GeoNames • SimpleGeo Places JSON Saturday, October 27, 12 TIGER/LINE - gathering this data is their job SimpleGeo shut down, you can find an API from factual.com or download the JSON uscampgrounds
  20. Gathering Geospatial • Census Bureau • TIGER/LINE • Local governments

    (city/county/state) • data.gov • GeoNames • SimpleGeo Places JSON • http://s3.amazonaws.com/simplegeo-public/places_dump_20110628.zip Saturday, October 27, 12 TIGER/LINE - gathering this data is their job SimpleGeo shut down, you can find an API from factual.com or download the JSON uscampgrounds
  21. Gathering Geospatial • Census Bureau • TIGER/LINE • Local governments

    (city/county/state) • data.gov • GeoNames • SimpleGeo Places JSON • http://s3.amazonaws.com/simplegeo-public/places_dump_20110628.zip • Google for it Saturday, October 27, 12 TIGER/LINE - gathering this data is their job SimpleGeo shut down, you can find an API from factual.com or download the JSON uscampgrounds
  22. Gathering Geospatial • Census Bureau • TIGER/LINE • Local governments

    (city/county/state) • data.gov • GeoNames • SimpleGeo Places JSON • http://s3.amazonaws.com/simplegeo-public/places_dump_20110628.zip • Google for it • User generated Saturday, October 27, 12 TIGER/LINE - gathering this data is their job SimpleGeo shut down, you can find an API from factual.com or download the JSON uscampgrounds
  23. User Generated? Saturday, October 27, 12 Be careful with usage

    of information you geocode Hardware loggers used by OSM, commonly available, or use a smartphone / tablet
  24. User Generated? • Textual geocoding* Saturday, October 27, 12 Be

    careful with usage of information you geocode Hardware loggers used by OSM, commonly available, or use a smartphone / tablet
  25. User Generated? • Textual geocoding* • Address to latitude/longitude Saturday,

    October 27, 12 Be careful with usage of information you geocode Hardware loggers used by OSM, commonly available, or use a smartphone / tablet
  26. User Generated? • Textual geocoding* • Address to latitude/longitude •

    Logging Saturday, October 27, 12 Be careful with usage of information you geocode Hardware loggers used by OSM, commonly available, or use a smartphone / tablet
  27. User Generated? • Textual geocoding* • Address to latitude/longitude •

    Logging • Hardware / software Saturday, October 27, 12 Be careful with usage of information you geocode Hardware loggers used by OSM, commonly available, or use a smartphone / tablet
  28. User Generated? • Textual geocoding* • Address to latitude/longitude •

    Logging • Hardware / software • GeoIP Saturday, October 27, 12 Be careful with usage of information you geocode Hardware loggers used by OSM, commonly available, or use a smartphone / tablet
  29. User Generated? • Textual geocoding* • Address to latitude/longitude •

    Logging • Hardware / software • GeoIP • Manual entry Saturday, October 27, 12 Be careful with usage of information you geocode Hardware loggers used by OSM, commonly available, or use a smartphone / tablet
  30. User Generated? • Textual geocoding* • Address to latitude/longitude •

    Logging • Hardware / software • GeoIP • Manual entry • HTML5 geolocation Saturday, October 27, 12 Be careful with usage of information you geocode Hardware loggers used by OSM, commonly available, or use a smartphone / tablet
  31. User Generated? • Textual geocoding* • Address to latitude/longitude •

    Logging • Hardware / software • GeoIP • Manual entry • HTML5 geolocation • Provided latitude / longitude Saturday, October 27, 12 Be careful with usage of information you geocode Hardware loggers used by OSM, commonly available, or use a smartphone / tablet
  32. >>> from geopy import geocoders >>> g = geocoders.Google() >>>

    place, (lat, lng) = g.geocode(“Conway, AR”) >>> place u'Conway, AR, USA' >>> lat 35.0886963 >>> lng -92.4421011 Saturday, October 27, 12
  33. >>> from django.contrib.gis.utils.geoip import GeoIP >>> GeoIP().city("amazon.com") {'city': 'Seattle', 'region':

    'WA', 'area_code': 206, 'longitude': -122.32839965820312, 'country_code3': 'USA', 'latitude': 47.60260009765625, 'postal_code': '98104', 'dma_code': 819, 'country_code': 'US', 'country_name': 'United States'} Saturday, October 27, 12 Makes a useful Django context processor http://blog.adamfast.com/2011/11/where-is-my-user-part-1-geoip/
  34. <script src='http://media.adamfast.com/QueryData.compressed.js'></script> <script language='JavaScript'> function checkLocation() { var getData =

    new QueryData(); if ('lat' in getData) { } else { if (navigator.geolocation) { navigator.geolocation.getCurrentPosition( function (ppos) { window.location.href = window.location.href + '?lat=' + ppos.coords.latitude + '&lon=' + ppos.coords.longitude; }, function (err) { switch(err.code) { case err.TIMEOUT: alert('Attempts to retrieve location timed out.') break; case err.POSITION_UNAVAILABLE: alert("Your browser doesn't know where you are.") break; case err.PERMISSION_DENIED: alert('You have to give us permission!') break; case err.UNKNOWN_ERROR: alert('Unknown error returned.') break; default: alert(err + ' ' + err.code) } } ); } } } </script> Saturday, October 27, 12 HTML5 geolocation - bounces you back to the URL with parameters WATCH OUT - will be FAR more precise than your GPS is. Remember significant figures
  35. WARNING • JavaScript floating point - now I have .9999999999

    problems because my cache didn’t hit http://localhost:8000/campgrounds/?lat=38.9591749&lon=-95.32675689999999 Saturday, October 27, 12 block ins’t a formal unit 2 digits ~0.69 miles, 3 digits ~360ft, 4 digits ~36ft, 5 digits ~3.6ft
  36. WARNING • JavaScript floating point - now I have .9999999999

    problems because my cache didn’t hit • This much precision (14 decimal places) is 1.1 nanometers, your computer CPU is likely a 32 nanometers process. Human hair 40,000 nanometers. http://localhost:8000/campgrounds/?lat=38.9591749&lon=-95.32675689999999 Saturday, October 27, 12 block ins’t a formal unit 2 digits ~0.69 miles, 3 digits ~360ft, 4 digits ~36ft, 5 digits ~3.6ft
  37. WARNING • JavaScript floating point - now I have .9999999999

    problems because my cache didn’t hit • This much precision (14 decimal places) is 1.1 nanometers, your computer CPU is likely a 32 nanometers process. Human hair 40,000 nanometers. • Remember significant figures - GPS will barely give 1m http://localhost:8000/campgrounds/?lat=38.9591749&lon=-95.32675689999999 Saturday, October 27, 12 block ins’t a formal unit 2 digits ~0.69 miles, 3 digits ~360ft, 4 digits ~36ft, 5 digits ~3.6ft
  38. WARNING • JavaScript floating point - now I have .9999999999

    problems because my cache didn’t hit • This much precision (14 decimal places) is 1.1 nanometers, your computer CPU is likely a 32 nanometers process. Human hair 40,000 nanometers. • Remember significant figures - GPS will barely give 1m • 2 digits = 1111m / 3 digits = 111m (1/4 block) / 4 digits = 11.1m / 5 digits 1.1m (*at equator) http://localhost:8000/campgrounds/?lat=38.9591749&lon=-95.32675689999999 Saturday, October 27, 12 block ins’t a formal unit 2 digits ~0.69 miles, 3 digits ~360ft, 4 digits ~36ft, 5 digits ~3.6ft
  39. Batteries Available • GDAL/OGR ctypes (Django, but reusable) • shapely

    • QGIS (scripting for) Saturday, October 27, 12
  40. Batteries Available • GDAL/OGR ctypes (Django, but reusable) • shapely

    • QGIS (scripting for) • geopy Saturday, October 27, 12
  41. Batteries Available • GDAL/OGR ctypes (Django, but reusable) • shapely

    • QGIS (scripting for) • geopy • GeoDjango Saturday, October 27, 12
  42. Batteries Available • GDAL/OGR ctypes (Django, but reusable) • shapely

    • QGIS (scripting for) • geopy • GeoDjango • pyproj Saturday, October 27, 12
  43. Saturday, October 27, 12 LA times allows users to draw

    borders on their neighborhoods Everyblock was heavy into this, more hidden in the interface now
  44. Faking it US: 24 deg N to 48 deg N

    67 deg W to 124 deg W Saturday, October 27, 12 This method includes 20 degrees (the height of the midwest) of results for a 20 mile query Then it uses Python to calculate a more accurate measurement and exclude as necessary
  45. Faking it • Always look and ask “does this make

    sense?” Saturday, October 27, 12
  46. Faking it • Always look and ask “does this make

    sense?” • Debug in Google Earth - slow, but good to visualize Saturday, October 27, 12
  47. Faking it • Always look and ask “does this make

    sense?” • Debug in Google Earth - slow, but good to visualize • Pre-filtering is the right idea Saturday, October 27, 12
  48. Faking it • Always look and ask “does this make

    sense?” • Debug in Google Earth - slow, but good to visualize • Pre-filtering is the right idea • I would use GeoPy when calculating distances Saturday, October 27, 12
  49. Faking it • Always look and ask “does this make

    sense?” • Debug in Google Earth - slow, but good to visualize • Pre-filtering is the right idea • I would use GeoPy when calculating distances • It is BSD. Attribute / acknowledge and bundle. Saturday, October 27, 12
  50. import math def latitude_degrees(miles): return miles / 69.172 # very

    little variance here. a degree is approximately 69.172 miles def longitude_degrees(latitude, miles): return miles / (math.cos(math.radians(latitude)) * 69.172) # dist is cosine of the latitude mult by length of latitude def basic_bounding_box(center_lat, center_lon, radius_mi, wiggle_room=None): if wiggle_room is not None: radius_mi = radius_mi + wiggle_room off_center = radius_mi / 2 # since the center is the center, we only add 1/2 the range to each end start_lat = center_lat - latitude_degrees(off_center) end_lat = center_lat + latitude_degrees(off_center) start_lon = center_lon - longitude_degrees(center_lat, off_center) end_lon = center_lon + longitude_degrees(center_lat, off_center) return ( (start_lat, start_lon), (end_lat, end_lon), ) Saturday, October 27, 12 not going too deeply into this
  51. Faking it • Why? • Platform (Heroku, shared hosting, MySQL,

    etc) • Need “real” search i.e. solr Saturday, October 27, 12
  52. Faking it • Why? • Platform (Heroku, shared hosting, MySQL,

    etc) • Need “real” search i.e. solr • No longer. Solr and Elastic Search both do spatial Saturday, October 27, 12
  53. Faking it • Why? • Platform (Heroku, shared hosting, MySQL,

    etc) • Need “real” search i.e. solr • No longer. Solr and Elastic Search both do spatial • Avoiding library dependencies Saturday, October 27, 12
  54. Displaying on a map • No real batteries here http://blog.adamfast.com/2011/11/on-the-map/

    http://blog.adamfast.com/2011/11/mapping-better-google-maps-api-v3/ http://blog.adamfast.com/2011/11/polygons-and-overlays-with-google-maps-v3-api/ Saturday, October 27, 12
  55. Displaying on a map • No real batteries here •

    There is a “hidden” Google Maps generator in GeoDjango, but it’s API v2 http://blog.adamfast.com/2011/11/on-the-map/ http://blog.adamfast.com/2011/11/mapping-better-google-maps-api-v3/ http://blog.adamfast.com/2011/11/polygons-and-overlays-with-google-maps-v3-api/ Saturday, October 27, 12
  56. Displaying on a map • No real batteries here •

    There is a “hidden” Google Maps generator in GeoDjango, but it’s API v2 • You’ll spend a LOT more time reading source code than you would doing a Google Maps v3 template http://blog.adamfast.com/2011/11/on-the-map/ http://blog.adamfast.com/2011/11/mapping-better-google-maps-api-v3/ http://blog.adamfast.com/2011/11/polygons-and-overlays-with-google-maps-v3-api/ Saturday, October 27, 12
  57. • Geographic Models • Importing Shapefiles • Importing lower-tech files

    • Basic poly/point queries • Geocoding • Distance Search • Radius Limited Searches • GeoIP User Location • Browser Geolocation • Faking spatial search • Slippy maps (Google v2/ v3 • Finding Data • Customizing the admin map (layers too) • Geo-front-end input All this and more... http://blog.adamfast.com Saturday, October 27, 12