A Cook’s Tour of Open Data from a User’s Perspective (IT Shared Services)

Presentation to Prince Edward Island IT Shared Services staff on December 2, 2015, looking at open data not from a policy or regulatory perspective, but through the eyes of someone who has been consuming open data, and pointing to general principles that have been learned along the way.

Transcript

1. A Cook’s Tour of Open Data
   from a User’s Perspective
   Presentation to 

   IT Shared Services, Government of Prince Edward Island
   Peter Rukavina, Hacker in Residence, Robertson Library, University of PEI
   December 2, 2015
   http://hacker.vre.upei.ca/
   

   View Slide

2. I moved to PEI for data.
   1993
   

   View Slide

3. JULY 18, 1993
   CRAFTS SUPPLIES INFORMATION AVAILABLE
   Craftspersons looking for information on sources of supplies, materials, or
   equipment can now send electronic mail to address:
   info@crafts-council.pe.ca
   Please put "WANTED:" at the beginning of your subject line.
   The "PEI Crafts Information Service" is service of the non-profit PEI Crafts
   Council in Charlottetown, Prince Edward Island, Canada. We have an ever-growing
   database of suppliers, covering all of the various crafts media; while we are
   still in the midst of information gathering and data entry, there is much useful
   information available now and we will respond to inquires as best we can.
   The bulk of our suppliers-on-file are located in North America, roughly half in
   Canada and half in the U.S.A. We can answer questions like "give me a list of
   the ceramics suppliers east of the Mississippi," "where can I get a rug hooking
   kit," or "who sells ConeArt kilns?" We can provide address, telephone, fax,
   WATS number, a list of products sold and, where we have a catalogue on file,
   more detailed information about specific products. We also track consumer
   experiences with suppliers when we receive it. Actual contact with suppliers
   for information and ordering is left to craftspersons themselves; we'll just
   help find them.
   1993
   

   View Slide

4. JULY 7, 1994
   FOR IMMEDIATE RELEASE
   PEI CRAFTS COUNCIL JOINS THE WORLD WIDE WEB
   CHARLOTTETOWN - The PEI Crafts Council today became the first organization
   on Prince Edward Island to join the Internet computer network's "World Wide
   Web."
   The "World Wide Web" (WWW) is an international collection of
   interconnected information providers founded at the European Laboratory for
   Particle Physics in Geneva, Switzerland. The World Wide Web uses hypertext
   and multimedia techniques to make "the web" -- collections of information on
   computers around the world -- easy to roam through, browse, search and
   contribute to.
   The PEI Crafts Council, the Prince Edward Island organization
   offering services and support to crafts producers, will use its WWW server
   to offer information on crafts and the crafts industry to crafts producers
   around the world. The project will centre on information from the Council's
   "Crafts Information Service," a comprehensive database of sources of
   materials, tools, equipment and services for crafts producers.
   1994
   

   View Slide

5. View Slide

6. View Slide

7. View Slide

8. View Slide

9. Year of #OpenData
   • ARMA PEI in February
   • Canadian Cartographic Association in May
   • Queen’s Printers of Canada in June
   • Chief Electoral Officers of Canada in July
   • PEI Deputy Ministers in August
   • Open Data Book Club in September
   • Policy PEI Lunch and Learn in September
   • Green Party Caucus in September
   • Workforce and Advanced Learning in October
   • Open Access Week at Robertson Library in October
   • Directors Forum in December
   

   View Slide

10. Open Data Principles
    (from a user’s perspective)
    

    View Slide

11. Windmill by Stephen Downes is licensed under CC BY-NC 2.0
    

    View Slide

12. Photo of Richard Brown from Liberal Party of Prince Edward Island
    

    View Slide

13. Photo of Ron Estabrooks taken at the open house to officially launch the Hermanville/Clearspring wind development from www.gov.pe.ca.
    

    View Slide

14. View Slide

15. View Slide

16. View Slide

17. Golf at 50BPM by user707748013
    50 MW
    50 BPM
    

    View Slide

18. W&W - Bigfoot (Uriel M Deephouse Remix) by urielm
    122 MW
    122 BPM
    

    View Slide

19. Clip of audio from CBC Spark interview with Peter Rukavina by Nora Young, recorded June 19, 2011.
    

    View Slide

20. View Slide

21. [
    {
    "data1":164.2,
    "data2":17.63,
    "data3":0,
    "updateDate":1437139741,
    "error":0
    }
    ]
    

    View Slide

22. [
    {
    "on-island-load":164.2,
    "on-island-wind":17.63,
    "on-island-fossil":0,
    "updateDate":1437139741,
    "error":0
    }
    ]
    

    View Slide

23. View Slide

24. View Slide

25. 155% of our electricity
    load from the wind
    10:30 p.m. 7:30 a.m.
    8 Hours with wind at more
    than 100% of load
    }
    

    View Slide

26. View Slide

27. View Slide

28. View Slide

29. http://innovis.cpsc.ucalgary.ca/
    

    View Slide

30. “I have become slightly obsessed by
    your energy graph and am very
    grateful that you have found a way to
    record the load and wind power
    generated in PEI. Do you by any
    chance have a spread sheet of these
    values? I am looking at using storage to
    balance wind fluctuations and was
    hoping to try and using the real load
    and wind data to accomplish that.”
    

    View Slide

31. Open Data Principle № 1
    You have no idea (at all)
    what open data might be
    used for.
    

    View Slide

32. View Slide

33. View Slide

34. View Slide

35. View Slide

36. pdftotext -raw

    Weekly_approvals_webpage_21_Oct_2011.pdf \
    Weekly_approvals_webpage_21_Oct_2011.txt
    

    View Slide

37. 10-533 335067 402-bld-10 20-Oct-10 3-Oct-11 18-22 Water Street...
    10-569 363556 439-BLD-10 17-Nov-10 3-Oct-11 20 Lapthorne Avenue...
    11-002 1018274 001-bld-11 4-Jan-11 6-Oct-11 375 Mount Edward Road...
    11-136 342436 326-bld-11 26-Aug-11 3-Oct-11 134 Kent Street...
    

    View Slide

38. -63.12688,46.23066,22 WATER ST,"10-533 335067 402-bld-10…
    -63.12606,46.24454,20 LAPTHORN AV,"10-569 363556 439-BLD-10..
    -63.14558,46.27834,375 MOUNT EDWARD RD,"11-002 1018274 001-bod-11…
    -63.12808,46.23572,134 KENT ST,"11-136 342436 326-bld-11…
    

    View Slide

39. View Slide

40. View Slide

41. View Slide

42. Open Data Principle № 2
    PDFs are where 

    data goes to die.
    

    View Slide

43. I walked thru some testing with the
    admin assistant in planning.
    She had some concerns with data
    entry -- she found it cumbersome and
    also identified a couple of scenarios
    that were not possible to enter
    properly, or were not represented
    clearly in the result.
    

    View Slide

44. Open Data Principle № 2B
    Bureaucrats really like
    File > Save as PDF…
    

    View Slide

45. View Slide

46. scale drawings drawn to Parking Design Standards and certified
    (stamped) by a qualified engineer, architect, or public land surveyor Shall
    be submitted where there are ten (10) or more Parking Spaces or for less
    than ten (10) spaces as required by the Development Officer with the
    application for the Building permit showing entrances and exits to such
    Parking facilities, all proposed and Existing Parking Spaces, aisles,
    lighting, and drainage of the Lot;
    

    View Slide

47. scale drawings drawn to Parking Design Standards and certified
    (stamped) by a qualified engineer, architect, or public land surveyor Shall
    be submitted where there are ten (10) or more Parking Spaces or for less
    than ten (10) spaces as required by the Development Officer with the
    application for the Building permit showing entrances and exits to such
    Parking facilities, all proposed and Existing Parking Spaces, aisles,
    lighting, and drainage of the Lot;
    scale drawings drawn to Parking Design Standards and certified
    (stamped) by a qualified engineer, architect, or public land surveyor Shall
    be submitted where there are ten (10) or more Parking Spaces or for less
    than ten (10) spaces as required by the Development Officer with the
    application for the Building permit showing entrances and exits to such
    Parking facilities, all proposed and Existing Parking Spaces, aisles,
    lighting, and drainage of the Lot;
    

    View Slide

48. GENERAL PROVISIONS FOR BUILDINGS AND STRUCTURES ................................................ 26
    4.1 ACCESSORY BUILDINGS
    4.2 BUILDING HEIGHT
    4.3 DECKS, AND OTHER PROJECTIONS INTO YARDS
    4.4 EXISTING BUILDINGS OR STRUCTURES
    4.5 FENCES
    4.6 GROUPED DWELLINGS
    4.7 ONE MAIN BUILDING ON A LOT
    4.8 RESTORATION TO A SAFE CONDITION
    4.9 SATELLITE DISHES AND COMMUNICATION TOWERS
    4.10 WINDMILL TOWERS
    4.11 SUBDIVIDING OF ATTACHED DWELLINGS
    4.12 SWIMMING POOLS
    4.13 VEHICLE BODIES
    

    View Slide

49. GENERAL PROVISIONS FOR BUILDINGS AND STRUCTURES ................................................ 26
    4.1 ACCESSORY BUILDINGS
    4.2 BUILDING HEIGHT
    4.3 DECKS, AND OTHER PROJECTIONS INTO YARDS
    4.4 EXISTING BUILDINGS OR STRUCTURES
    4.5 FENCES
    4.6 GROUPED DWELLINGS
    4.7 ONE MAIN BUILDING ON A LOT
    4.8 RESTORATION TO A SAFE CONDITION
    4.9 SATELLITE DISHES AND COMMUNICATION TOWERS
    4.10 WINDMILL TOWERS
    4.11 SUBDIVIDING OF ATTACHED DWELLINGS
    4.12 SWIMMING POOLS
    4.13 VEHICLE BODIES
    General Provision for Buildings and Structures ................................................................................. 26
    4.1 Accessory Buildings
    4.2 Building Height
    4.3 Decks, and Other Projections into Yards
    4.4 Existing Buildings or Structures
    4.5 Fences
    4.6 Groups Dwellings
    4.7 One Main Building on a Lot
    4.8 Restoration to a Safe Condition
    4.9 Satellite Dishes and Communication Towers
    4.10 Windmill Towers
    4.11 Subdividing of Attached Dwellings
    4.12 Swimming Pools
    4.13 Vehicle Bodies
    

    View Slide

50. View Slide

51. Open Data Principle № 3
    Sometimes “open” can simply
    mean following 

    basic rules of design.
    

    View Slide

52. View Slide

53. View Slide

54. View Slide

55. View Slide

56. Metadata Issues
    • Lot 50 was mislabelled as Lot 66.
    • Lot 66 was unlabelled.
    • Georgetown Royalty was mislabelled as Lot 73
    • Princetown Royalty was missing.
    

    View Slide

57. I'm writing to report what I believe to be incorrect metadata for one of
    the GIS Catalog items that you are listed as contact for, the "Lot /
    Townships" layer:
    http://www.gov.pe.ca/gis/index.php3?number=1011342&lang=E
    There are two polygons that appear to have incorrect metadata; I have
    attached a screen shot labelling these, which are in eastern PEI in the
    Orwell Point area.
    One polygon currently has a LOT value of NULL, and I believe it should,
    instead, have a value of 66.
    The other polygon, currently labelled 66, should, I believe, be labelled
    50.
    As a reference, I am using the Samuel Holland map, a version of which has
    been digitized at:
    http://www.islandimagined.ca/fedora/repository/imagined:209165
    I am attaching a screen shot of that map showing the lot labels.
    As next year is the 250th anniversary of the Samuel Holland Map, I would
    expect this layer to attract more attention than it otherwise would, so it
    would be nice to have the issue rectified.
    

    View Slide

58. Correcting the Metadata
    • Relabelled Lot 50.
    • Added a label to Lot 66.
    • Changed Lot 73 to Georgetown Royalty.
    • Princetown Royalty was missing.
    

    View Slide

59. View Slide

60. View Slide

61. WhatsMyLot.com
    

    View Slide

62. View Slide

63. View Slide

64. View Slide

65. View Slide

66. View Slide

67. The Confederation Trail passes 

    through 47 lots and 2 royalties.
    

    View Slide

68. Open Data Principle № 4
    Open data is a
    conversation.
    

    View Slide

69. View Slide

70. View Slide

71. View Slide

72. View Slide

73. (902) 367-3694
    

    View Slide

74. View Slide

75. Open Data Principle № 5
    Sometimes your users will
    create open data for you. 

    Follow their lead.
    

    View Slide

76. Open Data Principles
    1. You have no idea (at all) what
    open data might be used for.
    2. PDFs are where data goes to die.
    3. Sometimes “open” can simply
    mean following rules of design.
    4. Open data is a conversation.
    5. Sometimes your users will create
    open data for you.
    

    View Slide

77. What’s Here Now?
    (open data on www.gov.pe.ca)
    

    View Slide

78. Civic Address Data
    

    View Slide

79. View Slide

80. View Slide

81. GIS Data Layers
    

    View Slide

82. View Slide

83. View Slide

84. Telephone Directory
    

    View Slide

85. View Slide

86. View Slide

87. View Slide

88. Electricity Load and
    Generation Data
    

    View Slide

89. [
    {
    "data1":164.2,
    "data2":17.63,
    "data3":0,
    "updateDate":1437139741,
    "error":0
    }
    ]
    

    View Slide

90. Spreadsheets
    

    View Slide

91. View Slide

92. View Slide

93. View Slide

94. View Slide

95. What’s Here Now?
    (not a lot, but what’s there is very helpful)
    

    View Slide

96. Open Data Principles
    1. You have no idea (at all) what
    open data might be used for.
    2. PDFs are where data goes to die.
    3. Sometimes “open” can simply
    mean following rules of design.
    4. Open data is a conversation.
    5. Sometimes your users will create
    open data for you.
    

    View Slide

97. Slides are online at
    http://l.ruk.ca/itss-slides
    These slides are released under a Creative Commons license.
    You are encouraged to distribute them as far and as 

    wide as possible under the terms of the license.
    

    View Slide