Slide 1

Slide 1 text

Bulk Chris Herwig @hrwgc +open

Slide 2

Slide 2 text

Chris Herwig Satellite Team lead, MapBox

Slide 3

Slide 3 text

MapBox Satellite Phase 1, Launched 12/2012 • Global imagery base layer for MapBox users • Global satellite imagery, zoom 0-12 • Continental U.S. aerial imagery zoom 13-17 • Licensed to allow for OSM tracing

Slide 4

Slide 4 text

MapBox Satellite phase 1 was sourced entirely from public domain, open data.

Slide 5

Slide 5 text

Kuala Lumpur, Malaysia

Slide 6

Slide 6 text

Los Angeles, CA -

Slide 7

Slide 7 text

Brawley, CA

Slide 8

Slide 8 text

Cloudless Atlas • Cloudfree global mosaic, zoom 0-8 • NASA MODIS Aqua and Terra Satellites • 380,000 source satellite images

Slide 9

Slide 9 text

Open data is good.

Slide 10

Slide 10 text

“data is open if anyone is free to use, reuse, and redistribute it ...”

Slide 11

Slide 11 text

“subject to the requirement to attribute and/or share-alike” -Open Knowledge Definition

Slide 12

Slide 12 text

ACCESS

Slide 13

Slide 13 text

- open license - open format - available for download ACCESS

Slide 14

Slide 14 text

assumptions 3+1

Slide 15

Slide 15 text

There are different types of open data users.

Slide 16

Slide 16 text

Different users have different needs and abilities.

Slide 17

Slide 17 text

Data accessibility matters.

Slide 18

Slide 18 text

Open data is not truly open if it is inaccessible.

Slide 19

Slide 19 text

USERS 3

Slide 20

Slide 20 text

CASUAL

Slide 21

Slide 21 text

casual •least technical •dataset discovery •basic needs: ability to query and download

Slide 22

Slide 22 text

•geoportal •simple html table •solid metadata •intuitive interface casual

Slide 23

Slide 23 text

casual USGS EarthExplorer http://earthexplorer.usgs.gov

Slide 24

Slide 24 text

casual Massachusetts GIS http://gis.amherstma.gov/mgis/

Slide 25

Slide 25 text

casual The National Map http://nationalmap.gov

Slide 26

Slide 26 text

casual Utah AGRC Raster Data Discovery http://gis.utah.gov

Slide 27

Slide 27 text

casual New Hampshire Statewide GIS Clearinghouse http://www.granit.unh.edu/data/downloadfreedata/category/databycategory.html

Slide 28

Slide 28 text

PROGRAM MATIC

Slide 29

Slide 29 text

•Tech skills/API familiarity •spatial query •download sub-dataset based on parent process programmatic

Slide 30

Slide 30 text

programmatic • API • developer documentation • solid metadata • interface optional

Slide 31

Slide 31 text

USGS Application Services http://cumulus.cr.usgs.gov/app_services.php programmatic

Slide 32

Slide 32 text

USGS Application Services http://cumulus.cr.usgs.gov/app_services.php programmatic

Slide 33

Slide 33 text

BULK

Slide 34

Slide 34 text

bulk • Need entire datasets, not spatial intersections • Data APIs/manual retrieval workflows do not scale • Sometimes retrieve data via physical drives

Slide 35

Slide 35 text

bulk • interface optional • FTP-like access • reasonable bandwidth for download retrieval

Slide 36

Slide 36 text

New Hampshire Statewide GIS Clearinghouse http://www.granit.unh.edu/ Bulk

Slide 37

Slide 37 text

API

Slide 38

Slide 38 text

TYPES 3

Slide 39

Slide 39 text

CONTENT

Slide 40

Slide 40 text

ConTeNt Database REST Content

Slide 41

Slide 41 text

Content • Makes application content available for developers to integrate into existing/new applications

Slide 42

Slide 42 text

Content

Slide 43

Slide 43 text

DATA

Slide 44

Slide 44 text

Database REST Matching Rows Data

Slide 45

Slide 45 text

DATA • Allows users to query large datasets without having to have full dataset locally • Applications can be built on top of Live/real-time datasets

Slide 46

Slide 46 text

Data http://api.occupy-data.org/v1/? results&value=crossst&value=age&value=race&value=crimsusp&value=sex&value=build&value=frisked&results_p er_page=100

Slide 47

Slide 47 text

BULK

Slide 48

Slide 48 text

Bulk Database REST References

Slide 49

Slide 49 text

bulk • Key difference is user obtains reference to object requested, rather than object itself. • Download object(s) later • Can be relatively lightweight

Slide 50

Slide 50 text

SO?

Slide 51

Slide 51 text

Data API = Best Open Data MetHOD?

Slide 52

Slide 52 text

NO.

Slide 53

Slide 53 text

APIs, like geoportals, are not always the best option for disseminating open data.

Slide 54

Slide 54 text

Different USers

Slide 55

Slide 55 text

Different NEEds

Slide 56

Slide 56 text

Different Abilities

Slide 57

Slide 57 text

Different Access Endpoints

Slide 58

Slide 58 text

STUFF breaks

Slide 59

Slide 59 text

Permalinks != Permanent

Slide 60

Slide 60 text

WayBackMachine http://archive.org/web/web.php

Slide 61

Slide 61 text

So?

Slide 62

Slide 62 text

Open data users change as tech changes.

Slide 63

Slide 63 text

Access should be a policy and tech consideration.

Slide 64

Slide 64 text

NEXT STEPS

Slide 65

Slide 65 text

Strive to be SAD

Slide 66

Slide 66 text

SCALABLE Accessible Durable

Slide 67

Slide 67 text

- Open systems for access to open data - Can grow in response to changes in technology/user requirements SCALABLE

Slide 68

Slide 68 text

- Data access and retrieval is as quick and painless as possible - Options for users with different abilities, different desired results Accessible

Slide 69

Slide 69 text

- APIs, geoportals don’t always work - Low-maintenance, durable options - FTP-like directory access - Good documentation DURABLE

Slide 70

Slide 70 text

San Francisco, CA

Slide 71

Slide 71 text