Survey of Unique IDs in NYC Open Data: Joins, Sorrows, and Surprises

An overview of unique identifiers in NYC open data relating to the built environment.

Presentation: https://www.youtube.com/watch?v=Dhuc4fdIBqc

Transcript

1. Jonathan Pichot pichot.us Survey of Unique IDs in NYC Open Data Joins, Sorrows, and Surprises
2. What we’ll cover Borough/Block/Lot (BBL) Building ID Number (BIN) Address Street Segment IDs ⚙ With some data and tool demos along the way
3. Land Ordinance of 1785
5. 1811 Commissioner’s Plan
6. Lot maps ~1860
7. Alteration Books ~1900s Block Lot
8. Paper Tax Map 1968 Block Lot
9. Digital Tax Map (DOF) Block Lot
10. & Legal legibility… …for taxes Why all the trouble?
11. The BBL Borough Block Lot - Assigned to Tax Lots for purposes of property assessment and taxation - 3 types: Base, Billing, Unit
 (there’s actually a 4th, but don’t worry just yet) managed by the Department of Finance
12. The BBL Borough 1008350041 Block Lot 1/00835/0041 zola.planning.nyc.gov/lot/1/835/41 1/835/41 10 byte integer, beginning with boro code [1-5]
13. BBL Datasets MapPLUTO
 Department of City Planning
 ⚙ Tools: ZoLa Digital Tax Map
 Department of Finance
 ⚙ Tools: Digital Tax Map online
14. BBL Surprises: Condos 1 Billing BBL (red is from MapPLUTO)
15. BBL Surprises: Condos 1 Billing BBL (red is from MapPLUTO) 8 Base BBLs (orange from Digital Tax Map)
16. BBL Surprises: Condos 1 Billing BBL (red is from MapPLUTO) 8 Base BBLs (orange from Digital Tax Map) 27 Unit BBLs (from Digital Tax Map) 3080071004 3080071005 3080071006
17. BBL Surprises: Condos Base BBL - covers most cases, single standing building with one owner Billing BBL - Represents a legal entity, like a condo association, or co-op Unit BBL - represents a unit for tax liability managed by the Department of Finance
18. BBL Surprises: Marble Hill
19. BBL Surprises: Marble Hill
20. BBL Surprises: Marble Hill
21. BBL Sorrows: Breezy Point
22. BBL Sorrows: Breezy Point
23. The BIN
24. The BIN Columbia University
 (complex)
25. The BIN Building Identifier Number - Created in the 1990s, as a way to identify buildings with multiple addresses & entrances - Assigned to buildings and other structures (Mostly) generated by NYC Planning
 Fundamental to Department of Buildings
26. The BIN Borough 1015862 Building 7 byte integer, beginning with boro code [1-5]
27. BIN Datasets Building Footprints
 DoITT DOB Job Application Filings
 Department of Buildings
 ⚙ Tools: NYC Construction Dashboard Property Address Directory
 Department of City Planning
 ⚙ Tools: Geographic Online Address Translator (GOAT)
28. A BIN is born: construction Building permit filed with DOB DOB creates temporary BIN (TBIN) ✅ DOB issues a Certificate of Occupancy DCP adds BIN to address database (PAD) DoITT adds building footprints
29. A BIN is retired: demolition Demolition permit filed with DOB DOB confirms demolition completion DCP removes BIN from address database DoITT removes building footprint
30. Wait, weren’t there buildings in NYC before the 90s?
31. Geographic Research! - team of 9 at DCP
32. Buildings before BIN
33. The BIN
34. The BIN
35. The BIN
36. The BIN
37. Surprises: pseudo-BBLs Streetview
 Google
38. Surprises: pseudo-BBLs Building Footprints
 DoITT
39. Sorrows: not 100% coverage Building Footprints
 DoITT Breezy Point
40. Address
41. Address
42. Address
43. Address
44. Address
45. NYC Address Points
 DoITT
 Metadata Address Datasets Property Address Directory
 Department of City Planning
 ⚙ Tools: Geographic Online Address Translator (GOAT)
46. Address From GOAT
47. Address - Collection of information used to give the location of a building, apartment, or other structure or a plot of land - Generated by each Borough President’s office, integrated into data by NYC Planning Address
48. Address
49. Address
50. Sorrows: Local Law Street Designations
51. Surprises: Vanity Address
52. Surprises: Neighborhood Names
53. Surprises: Non-Addressable Place Names (NAPs)
54. Street Segment
55. Street Segment 2 types of IDs Segment ID
 - Older, more common Physical ID
 - Newer, composite of 1 or more Segment IDs Maintained by NYC Planning
56. Street Segment Datasets Citywide Street Centerline (CSCL)
 DoITT
 Metadata
 Physical ID
 Linear Integrated Ordered Network (LION)
 Department of City Planning
 Segment ID + Physical ID
57. Sorrows: Segment ID splits 101 E 20th St
58. Sorrows: Versioning
59. Street Segment Joins Street Pavement Rating
 Department of Transportation
 Segment ID (with version year)
 ⚙ NYC Streets
 Street Construction Permits
 Department of Transportation
 Segment ID
 ⚙ NYC Streets
60. Street Segment Joins PlowNYC
 Department of Sanitation
 Physical ID
 ⚙ Plow NYC
 Traffic Counts
 Department of Transportation
 Segment ID (no version )
61. Recap
62. Data and its limits Data will only ever approximate physical reality NYC an always changing reality Modeling such a complex reality, and maintaining that model over decades, let alone centuries, is extraordinary work
63. Resources NYC Public Data: A primer on property data by David Elner NYC GIS Metadata by DoITT
 github.com/CityOfNewYork/nyc-geo-metadata NYCityMap & Beyond by DoITT GIS
 nycitymap.wordpress.com
64. We’re hiring! Data Engineer Full Stack Developer Interaction Designer medium.com/nyc-planning-digital