Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MESH and OpenStreetMap - State of the Map Scotland 2015

MESH and OpenStreetMap - State of the Map Scotland 2015

MESH (Mapping Edinburgh’s Social History) is a path-breaking AHRC funded project that uses data based on addresses and areas to represent historical information. It provides a spatial dimension that enriches and enhances an understanding of the past.

This particular talk shows:
- why the MESH team chose OpenStreetMap (OSM) among different cartographic solutions,
- how the team improved OSM in Edinburgh using historical maps (out-of-copyright town plans and Ordnance Survey maps provided by the National Library of Scotland) and intensive ground surveying,
- the quality of the mapping including a comparison with current Ordnance Survey data,
- how the team and historians start to use it (tools, results),
- how the work presented here can be reused by everyone including academics, public institutions and private companies.

More info about the project: http://www.mesh.ed.ac.uk/

Transcript

  1. MESH Mapping Edinburgh's Social History Eric Grosso, Richard Rodger, Sophie

    McCallum
  2. MESH Mapping Edinburgh's Social History Co-editors: Richard Rodger, Bob Morris,

    Charlie Withers, Michael Lynch, Simon Stronach Partners: Team: Eric Grosso, Sophie McCallum, Eisa Esfanjari, Leila Amely, Marc di Tommasi, Michael Brown, Wilson Smith
  3. 3 Disclaimer What you'll see here is: - open data

    - open source - open data - open source - open data - open source
  4. 4 Disclaimer

  5. 5 Outline MESH MESH and OpenStreetMap Talk about history (Richard)

    Demo
  6. 6 History and Maps Maps and History About MESH

  7. 7 MESH – main objectives Atlas Data Maps Tools Networks

  8. 8 MESH – an atlas around 10 themes 1. Living

    and dying 2. Making, selling and earning 3. Teaching and learning 4. Worshipping 5. Feeding and drinking 6. Moving and communicating 7. Socialising 8. Managing and administering
  9. 9 MESH – constraints Atlas Data Maps Tools Networks

  10. 10 MESH – constraints... after 3 years... Atlas Data Maps

    Tools Networks
  11. 11 MESH – constraints... after 3 years... Atlas Data Maps

    Tools Networks Robustness (HGIS) Open source tools Open data Sustainability (5-10 years)
  12. 12 MESH – sustainability and legacy Atlas Data Maps Tools

    Networks
  13. 13 Constraints vs Questions / Solutions Open data and tools,

    Open licence, OpenStreetMap Georeferencing process Data integration Geocoding Robustness (HGIS) Open source tools Open data Sustainability (5-10 years)
  14. 14 Mapping Edinburgh (MESH and OSM)

  15. 15 MESH – how to build the HGIS? Historical needs,

    historical questions Choice of a period, delimitation of a space (international, national, regional, local), choice of a scale / level of detail Need for resources / data: which ones? Need to create them? Different resources: written documents, historical maps, surveys (archaeological, historical) Problem to know the content of these resources and to access these resources Problem solved more and more thanks to digitisation (parallel between digitisation and the emergence of HGIS)
  16. 16 MESH – how to build the HGIS? A solution

    could be... Current Google Maps + Advocates 1861, using the Google Maps API Overlay
  17. 17 Historical Geographic Information System Other resources Optical character recognition

    (OCR) Geocoding Old maps (OS / NLS) Georeferencing (GCP, transformation, evaluation, etc.) Vectorisation? MESH – how to build the HGIS?
  18. 18 MESH – which map or/and database to use? Google

    Maps (API) Ordnance Survey OpenStreetMap
  19. 19 MESH – which map or/and database to use? Google

    Maps
  20. 20 MESH – which map or/and database to use? Google

    Maps
  21. 21 MESH – which map or/and database to use? Google

    Maps
  22. 22 MESH – which map or/and database to use? Ordnance

    Survey (open data)
  23. 23 MESH – which map or/and database to use? Ordnance

    Survey (open data)
  24. 24 MESH – which map or/and database to use? OpenStreetMap,

    Edinburgh, September 2013
  25. 25 MESH – which map or/and database to use? OpenStreetMap,

    Edinburgh, September 2013
  26. 26 MESH – which map or/and database to use? OpenStreetMap,

    Edinburgh, September 2013
  27. 27 MESH – which map or/and database to use? OpenStreetMap,

    Edinburgh, September 2013
  28. 28 MESH – which map or/and database to use? GOOGLE

    ORDNANCE SURVEY MAPPING FROM SCRATCH OSM TYPE OF DATA Images only Raster and Vector Vector Vector COPYRIGHT Yes Yes Our choice No OPEN LICENCE No Only OpenData (poor quality) Our choice Yes (ODbL) DATA REDISTRIBUTION No No Our choice Yes (ODbL) QUALITY Relatively good (depends where) Very good (1m) Very poor Good (1-3m) – generally < 5m – DATABASE QUERY Only through Google APIs Yes but tools need to be developed Yes but tools need to be developed Yes with some already existing tools
  29. 29 OpenStreetMap, Edinburgh: resources John Wood map, 1831 Bartholomew plan,

    1912 OS town plan, 1876-1877 OS town plan, 1893-1894 Bartholomew plan, 1919 Bartholomew Post Office map, 1939-1940 Historical maps from the National Library of Scotland
  30. 30 Old maps: georeferencing Principle: ground control points and transformation(s)

  31. 31 OpenStreetMap, Edinburgh: resources OS town plan, 1849-1851 OS town

    plan, 1876-1877 OS town plan, 1893-1894 OS town plan, 1940s
  32. 32 Old maps: georeferencing What extent?

  33. 33 OpenStreetMap, Edinburgh: resources OS town plan, 1893-1894 OS town

    plan, 1940s
  34. 34 OSM vs Old map: vectorisation NLS, OS town plan,

    1940s
  35. 35 OSM vs Old map: vectorisation JOSM in action

  36. 36 OSM vs Old map: vectorisation JOSM in action

  37. 37 OSM vs Old map: vectorisation JOSM in action

  38. 38 OSM vs Old map: vectorisation JOSM in action

  39. 39 OSM vs Old map: vectorisation JOSM in action

  40. 40 OSM vs Old map: vectorisation JOSM in action

  41. 41 OSM vs Old map: vectorisation The MESH method Improve

    the roads → skeleton Footprint rather than roofprint Using lower elements (walls, roads) to manage the offset Tracing the walls and the gardens/plots help to understand the differences between the old map and the reality (Bing/survey) Using plots to create new buildings not even in Bing Adding tags Full survey
  42. 42 OSM vs Old map: vectorisation The MESH method (data

    model for addresses) And if there is a shop? 1 address = addr:country addr:city addr:street addr:housenumber source (level) (addr:postcode) (addr:housename) 1 address (amenity)= addr:street addr:housenumber source (level)
  43. 43 OSM: survey It's just a small part of our

    surveys All our surveys will be released to show that we walked each street
  44. 44 Old maps: vectorisation/data integration

  45. 45 Old maps: vectorisation/data integration

  46. 46 Old maps: vectorisation/data integration

  47. 47 OSM survey

  48. 48 OpenStreetMap, Edinburgh

  49. 49 OpenStreetMap, Edinburgh: MESH contributions

  50. 50 OpenStreetMap, Edinburgh: MESH contributions

  51. 51 OpenStreetMap, Edinburgh: MESH contributions

  52. 52 OpenStreetMap, Edinburgh: MESH contributions September 2013 Now

  53. 53 OpenStreetMap, Edinburgh: MESH contributions September 2013 Now

  54. 54 OpenStreetMap, Edinburgh: MESH contributions September 2013 Now

  55. 55 Towards a Digital Elevation Model NLS, OS town plan,

    1940s
  56. 56 Towards a Digital Elevation Model Benchmark (B.M.)

  57. 57 Towards a Digital Elevation Model

  58. 58 Towards a Digital Elevation Model

  59. 59 OpenStreetMap, Edinburgh: few numbers Edinburgh is the 14th most

    populated city in UK 3-4 man-years required to map it fully Accuracy: 1-3 meters Main enhancements and additions: - 15 000 road segments (25 000 in total) - 52 000 buildings (60 000 in total) - 50 000 addresses (including house numbers, shops, amenities) (52 000 addresses in total) - 11 000 POIs (12 500 POIs in total)
  60. 60 OpenStreetMap, Edinburgh: few numbers Other features of interest -

    railways, transport networks, waterways, administrative boundaries, green spaces, leisure facilities - gardens, walls, fences, etc. Ah! numbers... but what does really represent the survey? - 47,000 house numbers located at the door - total length of the roads: 530km / 330 miles = Edinburgh-London walk (as the crow flies) - total area of the buildigns: 8.3 km² / 3.2 sq miles = 200 Murrayfield stadiums 200
  61. 61 Is mapping Edinburgh in OSM a big challenge?

  62. 62 Certainly a big challenge but we try using... fdecomite

    / Brand new bricks (licence: CC BY 2.0) – https://www.flickr.com/photos/fdecomite/2710132377/
  63. 63 to build.... Rob Young / Legoland Windsor - Edinburgh

    Castle (licence: CC BY 2.0) – https://www.flickr.com/photos/rob-young/2835889828
  64. 64 and to avoid.... Eirik Newth / Lego City: Collapse

    (licence: CC BY 2.0) – https://www.flickr.com/photos/eiriknewth/238681391
  65. 65 How to provide a proof? vs

  66. 66 How to provide a proof? © Crown Copyright and

    database right. All rights reserved. Ordnance Survey License number 100023420.
  67. 67 How to provide a proof?

  68. 68 How to provide a proof?

  69. 69 How to provide a proof?

  70. 70 OpenStreetMap, Edinburgh And now, so what? 200

  71. 71 OpenStreetMap, Edinburgh 200 It proves something... our method can

    be applied everywhere in Scotland, in UK!
  72. 72 OpenStreetMap, Edinburgh 200

  73. 73 OpenStreetMap, Edinburgh 200

  74. 74 OpenStreetMap, Edinburgh 200

  75. 75 OpenStreetMap, Edinburgh 200

  76. 76 OpenStreetMap, Edinburgh 200 It also allows to do silly

    things...
  77. 77 OpenStreetMap, Edinburgh 200

  78. 78 OpenStreetMap, Edinburgh 200

  79. 79 OpenStreetMap, Edinburgh 200

  80. 80 OpenStreetMap, Edinburgh 200

  81. 81 OpenStreetMap, Edinburgh 200

  82. 82 OpenStreetMap, Edinburgh 200 It can be reused by others

  83. 83 OpenStreetMap, Edinburgh 200 Academics

  84. 84 OpenStreetMap, Edinburgh 200 Public institutions

  85. 85 Geocoding – CEC data (tables and chairs permits)

  86. 86 Improvement of existing databases (RCAHMS)

  87. 87 Improvement of existing databases (RCAHMS)

  88. 88 Improvement of existing databases (RCAHMS)

  89. 89 Improvement of existing databases (RCAHMS)

  90. 90 OpenStreetMap, Edinburgh 200 What is the next step for

    us? (OSM)
  91. 91 OpenStreetMap, Edinburgh: road map “Freeze” OpenStreetMap Go back in

    time
  92. 92 Historical GIS: data integration John Wood map, 1831 Bartholomew

    plan, 1912 Superimposition...
  93. 93 OpenStreetMap, Edinburgh 200 What is the next step for

    us? (geocoder)
  94. 94 Geocoder and historical geocoders Geocoder - first version based

    on Nominatim
  95. 95 Geocoder and historical geocoders Geocoder - first version based

    on Nominatim - second version based on addok Performance: at least 140-150 addresses per second = Maps in seconds addok include process to correct misspellings Rather easy to to keep good performances with different databases of addresses (historical addresses) Possibility to add “fake” interpolated addresses CSV import/export
  96. 96 House numbers in Edinburgh Geocoding at a house number

    level, until 1800-1820 Robert Kirkwood map (1817) Before 1800, geocoding at a street level
  97. 97 OSM survey

  98. 98 OSM survey

  99. 99 OSM survey

  100. 100 Routing Which routes to go to the coal mine?

    (Marc Di Tommasi)
  101. 101 Historical data Boundaries Abercrombie report Data from the old

    maps Data from historical sources Scottish Post Office Directories
  102. 102 Boundaries Registration districts (1865-1902)

  103. 103 Boundaries Municipal wards (1852-1902)

  104. 104 Abercrombie report Survey of dwelling sizes / family sizes

    / dwelling ownerships
  105. 105 Scottish Post Office Directories Content: - name/occupation/address(es) Licence: -

    CC BY-NC-SA - incompatible with OSM Post Office Directories (NLS) Optical Character Recognition (OCR) # old fonts / old letters / columns # one entry = several lines and Geocoding (Addresses) aka historical yellow pages (nearly 150 volumes)
  106. 106 Scottish Post Office Directories

  107. 107 Scottish Post Office Directories

  108. 108 Scottish Post Office Directories

  109. 109 Talk about history

  110. Bakers 1910-11

  111. 111 Edinburgh Retail Business Structure 1910-11 Butchers Bakers Spirit Merchants

    % % % women (Mrs, Miss) 1 3 10 family (& son, brother) 4 11 1 company (& Co.) 1 2 6 limited company (Ltd.) 3 3 0 have 2 premises 13 32 8 have 3 3 16 1 have 4 1 8 0 have 5 1 5 have 6 1 4 have 7+ ( Co-op) 15 7 have 8-12 premises 6 have 13+ (Co-op) 10 single premises 87 67 92
  112. 112 Spirit Dealers 1910-11 Central Area © MESH, NLS and

    OSM community
  113. 113 Data from the old maps: pubs Current pubs (South

    Edinburgh): exist (blue) or not (red) in 1940s
  114. 114 Data from the old maps: tram lines Tram lines:

    current (blue) and 1893 (red)
  115. 115 Data from the old maps: tram lines Tram lines:

    current (blue) and 1944 (orange)
  116. 116 Data from the old maps: comparison Current transport networks

    (OSM data)
  117. 117 Demo

  118. 118 want to contact us? www.mesh.ed.ac.uk @mesh_ed mesh@ed.ac.uk thank you