Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A Cook’s Tour of Open Data from a User’s Perspective (Green Party Caucus)

Peter Rukavina
September 29, 2015

A Cook’s Tour of Open Data from a User’s Perspective (Green Party Caucus)

Presentation to Prince Edward Island Green Party Caucus on September 29, 2015, looking at open data not from a policy or regulatory perspective, but through the eyes of someone who has been consuming open data, and pointing to general principles that have been learned along the way.

Peter Rukavina

September 29, 2015
Tweet

More Decks by Peter Rukavina

Other Decks in Education

Transcript

  1. A Cook’s Tour of Open Data
    from a User’s Perspective
    Presentation to 

    Green Party Caucus
    Peter Rukavina, Hacker in Residence, Robertson Library, University of PEI
    September 29, 2015
    http://hacker.vre.upei.ca/

    View Slide

  2. I moved to PEI for data.
    1993

    View Slide

  3. JULY 18, 1993
    CRAFTS SUPPLIES INFORMATION AVAILABLE
    Craftspersons looking for information on sources of supplies, materials, or
    equipment can now send electronic mail to address:
    [email protected]
    Please put "WANTED:" at the beginning of your subject line.
    The "PEI Crafts Information Service" is service of the non-profit PEI Crafts
    Council in Charlottetown, Prince Edward Island, Canada. We have an ever-growing
    database of suppliers, covering all of the various crafts media; while we are
    still in the midst of information gathering and data entry, there is much useful
    information available now and we will respond to inquires as best we can.
    The bulk of our suppliers-on-file are located in North America, roughly half in
    Canada and half in the U.S.A. We can answer questions like "give me a list of
    the ceramics suppliers east of the Mississippi," "where can I get a rug hooking
    kit," or "who sells ConeArt kilns?" We can provide address, telephone, fax,
    WATS number, a list of products sold and, where we have a catalogue on file,
    more detailed information about specific products. We also track consumer
    experiences with suppliers when we receive it. Actual contact with suppliers
    for information and ordering is left to craftspersons themselves; we'll just
    help find them.
    1993

    View Slide

  4. JULY 7, 1994
    FOR IMMEDIATE RELEASE
    PEI CRAFTS COUNCIL JOINS THE WORLD WIDE WEB
    CHARLOTTETOWN - The PEI Crafts Council today became the first organization
    on Prince Edward Island to join the Internet computer network's "World Wide
    Web."
    The "World Wide Web" (WWW) is an international collection of
    interconnected information providers founded at the European Laboratory for
    Particle Physics in Geneva, Switzerland. The World Wide Web uses hypertext
    and multimedia techniques to make "the web" -- collections of information on
    computers around the world -- easy to roam through, browse, search and
    contribute to.
    The PEI Crafts Council, the Prince Edward Island organization
    offering services and support to crafts producers, will use its WWW server
    to offer information on crafts and the crafts industry to crafts producers
    around the world. The project will centre on information from the Council's
    "Crafts Information Service," a comprehensive database of sources of
    materials, tools, equipment and services for crafts producers.
    1994

    View Slide

  5. 1996

    View Slide

  6. 2003

    View Slide

  7. 2012

    View Slide

  8. 2013

    View Slide

  9. Year of #OpenData
    • ARMA PEI in February
    • Canadian Cartographic Association in May
    • Queen’s Printers of Canada in June
    • Chief Electoral Officers of Canada in July
    • PEI Deputy Ministers in August
    • Open Data Book Club in September
    • Senior PEI Public Servants in September
    • Green Party Caucus in September

    View Slide

  10. Open Data Principles
    (from a user’s perspective)

    View Slide

  11. Windmill by Stephen Downes is licensed under CC BY-NC 2.0

    View Slide

  12. Photo of Richard Brown from Liberal Party of Prince Edward Island

    View Slide

  13. Photo of Ron Estabrooks taken at the open house to officially launch the Hermanville/Clearspring wind development from www.gov.pe.ca.

    View Slide

  14. View Slide

  15. View Slide

  16. View Slide

  17. Golf at 50BPM by user707748013
    50 MW
    50 BPM

    View Slide

  18. W&W - Bigfoot (Uriel M Deephouse Remix) by urielm
    122 MW
    122 BPM

    View Slide

  19. Clip of audio from CBC Spark interview with Peter Rukavina by Nora Young, recorded June 19, 2011.

    View Slide

  20. View Slide

  21. [
    {
    "data1":164.2,
    "data2":17.63,
    "data3":0,
    "updateDate":1437139741,
    "error":0
    }
    ]

    View Slide

  22. [
    {
    "on-island-load":164.2,
    "on-island-wind":17.63,
    "on-island-fossil":0,
    "updateDate":1437139741,
    "error":0
    }
    ]

    View Slide

  23. View Slide

  24. View Slide

  25. 155% of our electricity
    load from the wind
    10:30 p.m. 7:30 p.m.
    8 Hours

    View Slide

  26. View Slide

  27. View Slide

  28. View Slide

  29. View Slide

  30. http://innovis.cpsc.ucalgary.ca/

    View Slide

  31. “I have become slightly obsessed by
    your energy graph and am very
    grateful that you have found a way to
    record the load and wind power
    generated in PEI. Do you by any
    chance have a spread sheet of these
    values? I am looking at using storage to
    balance wind fluctuations and was
    hoping to try and using the real load
    and wind data to accomplish that.”

    View Slide

  32. Open Data Principle № 1
    You have no idea (at all)
    what open data might be
    used for.

    View Slide

  33. View Slide

  34. View Slide

  35. View Slide

  36. View Slide

  37. pdftotext -raw

    Weekly_approvals_webpage_21_Oct_2011.pdf \
    Weekly_approvals_webpage_21_Oct_2011.txt

    View Slide

  38. 10-533 335067 402-bld-10 20-Oct-10 3-Oct-11 18-22 Water Street...
    10-569 363556 439-BLD-10 17-Nov-10 3-Oct-11 20 Lapthorne Avenue...
    11-002 1018274 001-bld-11 4-Jan-11 6-Oct-11 375 Mount Edward Road...
    11-136 342436 326-bld-11 26-Aug-11 3-Oct-11 134 Kent Street...

    View Slide

  39. -63.12688,46.23066,22 WATER ST,"10-533 335067 402-bld-10…
    -63.12606,46.24454,20 LAPTHORN AV,"10-569 363556 439-BLD-10..
    -63.14558,46.27834,375 MOUNT EDWARD RD,"11-002 1018274 001-bod-11…
    -63.12808,46.23572,134 KENT ST,"11-136 342436 326-bld-11…

    View Slide

  40. View Slide

  41. View Slide

  42. View Slide

  43. Open Data Principle № 2
    PDFs are where 

    data goes to die.

    View Slide

  44. I walked thru some testing with the
    admin assistant in planning.
    She had some concerns with data
    entry -- she found it cumbersome and
    also identified a couple of scenarios
    that were not possible to enter
    properly, or were not represented
    clearly in the result.

    View Slide

  45. Open Data Principle № 2B
    Bureaucrats really like
    File > Save as PDF…

    View Slide

  46. View Slide

  47. scale drawings drawn to Parking Design Standards and certified
    (stamped) by a qualified engineer, architect, or public land surveyor Shall
    be submitted where there are ten (10) or more Parking Spaces or for less
    than ten (10) spaces as required by the Development Officer with the
    application for the Building permit showing entrances and exits to such
    Parking facilities, all proposed and Existing Parking Spaces, aisles,
    lighting, and drainage of the Lot;

    View Slide

  48. scale drawings drawn to Parking Design Standards and certified
    (stamped) by a qualified engineer, architect, or public land surveyor Shall
    be submitted where there are ten (10) or more Parking Spaces or for less
    than ten (10) spaces as required by the Development Officer with the
    application for the Building permit showing entrances and exits to such
    Parking facilities, all proposed and Existing Parking Spaces, aisles,
    lighting, and drainage of the Lot;
    scale drawings drawn to Parking Design Standards and certified
    (stamped) by a qualified engineer, architect, or public land surveyor Shall
    be submitted where there are ten (10) or more Parking Spaces or for less
    than ten (10) spaces as required by the Development Officer with the
    application for the Building permit showing entrances and exits to such
    Parking facilities, all proposed and Existing Parking Spaces, aisles,
    lighting, and drainage of the Lot;

    View Slide

  49. GENERAL PROVISIONS FOR BUILDINGS AND STRUCTURES ................................................ 26
    4.1 ACCESSORY BUILDINGS
    4.2 BUILDING HEIGHT
    4.3 DECKS, AND OTHER PROJECTIONS INTO YARDS
    4.4 EXISTING BUILDINGS OR STRUCTURES
    4.5 FENCES
    4.6 GROUPED DWELLINGS
    4.7 ONE MAIN BUILDING ON A LOT
    4.8 RESTORATION TO A SAFE CONDITION
    4.9 SATELLITE DISHES AND COMMUNICATION TOWERS
    4.10 WINDMILL TOWERS
    4.11 SUBDIVIDING OF ATTACHED DWELLINGS
    4.12 SWIMMING POOLS
    4.13 VEHICLE BODIES

    View Slide

  50. GENERAL PROVISIONS FOR BUILDINGS AND STRUCTURES ................................................ 26
    4.1 ACCESSORY BUILDINGS
    4.2 BUILDING HEIGHT
    4.3 DECKS, AND OTHER PROJECTIONS INTO YARDS
    4.4 EXISTING BUILDINGS OR STRUCTURES
    4.5 FENCES
    4.6 GROUPED DWELLINGS
    4.7 ONE MAIN BUILDING ON A LOT
    4.8 RESTORATION TO A SAFE CONDITION
    4.9 SATELLITE DISHES AND COMMUNICATION TOWERS
    4.10 WINDMILL TOWERS
    4.11 SUBDIVIDING OF ATTACHED DWELLINGS
    4.12 SWIMMING POOLS
    4.13 VEHICLE BODIES
    General Provision for Buildings and Structures ................................................................................. 26
    4.1 Accessory Buildings
    4.2 Building Height
    4.3 Decks, and Other Projections into Yards
    4.4 Existing Buildings or Structures
    4.5 Fences
    4.6 Groups Dwellings
    4.7 One Main Building on a Lot
    4.8 Restoration to a Safe Condition
    4.9 Satellite Dishes and Communication Towers
    4.10 Windmill Towers
    4.11 Subdividing of Attached Dwellings
    4.12 Swimming Pools
    4.13 Vehicle Bodies

    View Slide

  51. View Slide

  52. Open Data Principle № 3
    Sometimes “open” can simply
    mean following 

    basic rules of design.

    View Slide

  53. View Slide

  54. View Slide

  55. View Slide

  56. View Slide

  57. Metadata Issues
    • Lot 50 was mislabelled as Lot 66.
    • Lot 66 was unlabelled.
    • Georgetown Royalty was mislabelled as Lot 73
    • Princetown Royalty was missing.

    View Slide

  58. I'm writing to report what I believe to be incorrect metadata for one of
    the GIS Catalog items that you are listed as contact for, the "Lot /
    Townships" layer:
    http://www.gov.pe.ca/gis/index.php3?number=1011342&lang=E
    There are two polygons that appear to have incorrect metadata; I have
    attached a screen shot labelling these, which are in eastern PEI in the
    Orwell Point area.
    One polygon currently has a LOT value of NULL, and I believe it should,
    instead, have a value of 66.
    The other polygon, currently labelled 66, should, I believe, be labelled
    50.
    As a reference, I am using the Samuel Holland map, a version of which has
    been digitized at:
    http://www.islandimagined.ca/fedora/repository/imagined:209165
    I am attaching a screen shot of that map showing the lot labels.
    As next year is the 250th anniversary of the Samuel Holland Map, I would
    expect this layer to attract more attention than it otherwise would, so it
    would be nice to have the issue rectified.

    View Slide

  59. Correcting the Metadata
    • Relabelled Lot 50.
    • Added a label to Lot 66.
    • Changed Lot 73 to Georgetown Royalty.
    • Princetown Royalty was missing.

    View Slide

  60. View Slide

  61. View Slide

  62. WhatsMyLot.com

    View Slide

  63. View Slide

  64. View Slide

  65. View Slide

  66. View Slide

  67. View Slide

  68. The Confederation Trail passes 

    through 47 lots and 2 royalties.

    View Slide

  69. Open Data Principle № 4
    Open data is a
    conversation.

    View Slide

  70. View Slide

  71. View Slide

  72. View Slide

  73. View Slide

  74. (902) 367-3694

    View Slide

  75. View Slide

  76. Open Data Principle № 5
    Sometimes your users will
    create open data for you. 

    Follow their lead.

    View Slide

  77. “By 2010, our investment in Islanders’
    health care had increased by 25 per
    cent or over $700 per capita – to
    $3,655, putting us above the national
    average of $3,500.”
    2011 Budget Address

    View Slide

  78. I’m interested in obtaining a digital record
    of the transactions under what I would call,
    for lack of knowledge of the terminology, my
    “health care account” — in other words, all
    of the payments to doctors, specialists,
    hospitals, labs, etc. paid out by Health PEI
    attached to my Health Card account for as
    long as records are available.
    Email to Health PEI
    May 26, 2011 – Day 1

    View Slide

  79. Our Information Co-ordinator has indicated
    to me that you should submit a FOIPP
    (Freedom of Information and Protection
    of Privacy) request. Attached is the Access
    to Information form which you can complete
    and submit to our co-ordinator.
    Reply from Health PEI
    May 26, 2011 – Day 1

    View Slide

  80. FOIPP Request

    View Slide

  81. Digital record of financial transactions
    related to my health care (PEI Health Card #
    XXXXXX). I’m interested in all financial
    transactions related to doctors visits,
    hospital care, emergency room visits, lab
    tests, etc. with as much detail as available
    (date, amount, description, account, etc.).
    I would prefer the information in an open
    digital format, such as CSV or XML format.
    FOIPP Request

    View Slide

  82. 30 Days Later
    June 27, 2011 – Day 32

    View Slide

  83. What I Received…
    Physician Date Code
    Location
    (Printed report, not digital data)

    View Slide

  84. “However, a payment made to a physician
    is the physician’s personal information
    and is therefore not ours to disclose.
    General information related to payments for
    physicians can be found in the Master
    Agreement between the Medical Society of
    PEI and the Government of PEI.”
    But no financial data…

    View Slide

  85. So I appealed to the
    Information and Privacy
    Commissioner
    She agreed to open
    a review.
    August 18, 2011 – Day 84

    View Slide

  86. Clause 17(2)(c) of the Health Services Payments Act specifically
    authorizes the Minister to release the information I have requested:
    “the Minister may disclose information obtained in the
    administration of this Act, to the person who received the basic
    health services or to the legal representative or guardian of the
    person” and subsection 17(3) of the same act sets out the
    information that is authorized to be provided (emphasis mine):
    (a) basic health services provided;

    (b) the date on which the basic health services were provided;

    (c) the name and address of the person who provided the basic
    health services;
    (d) amounts paid under the plan; and

    (e) the person to whom payments were made from the plan.
    Sent more information…
    October 17, 2011 – Day 144

    View Slide

  87. Waited…

    View Slide

  88. Waited…
    Information and Privacy Commissioner informed
    me that her review would take longer than 90 days

    View Slide

  89. Waited…
    Waited…
    Information and Privacy Commissioner informed
    me that her review would take longer than 90 days

    View Slide

  90. Waited…
    Waited…
    Sent the Standing Committee on Legislative
    Management a letter asking for more resources
    for the Information and Privacy Commissioner
    Information and Privacy Commissioner informed
    me that her review would take longer than 90 days

    View Slide

  91. Waited…
    Waited…
    Sent the Standing Committee on Legislative
    Management a letter asking for more resources
    for the Information and Privacy Commissioner
    Waited…
    Information and Privacy Commissioner informed
    me that her review would take longer than 90 days

    View Slide

  92. “One more year…”
    Information and Privacy Commissioner sends a
    letter anticipating a ruling in February 2014
    January 24, 2013 – Day 609

    View Slide

  93. “9 more months…”
    Information and Privacy Commissioner sends a
    letter anticipating a ruling in September 2014
    January 16, 2014 – Day 966

    View Slide

  94. $879.37
    February 26, 2014 – Day 1007

    View Slide

  95. Open Data Principle № 6
    Open data is a necessary
    precondition for 

    tenacious citizenship.

    View Slide

  96. Open Data Principles
    1. You have no idea (at all) what open data
    might be used for.
    2. PDFs are where data goes to die.
    3. Sometimes “open” can simply mean
    following rules of design.
    4. Open data is a conversation.
    5. Sometimes your users will create open data
    for you.
    6. Open data is a necessary precondition for 

    tenacious citizenship.

    View Slide

  97. View Slide

  98. View Slide

  99. View Slide

  100. View Slide

  101. View Slide

  102. View Slide

  103. View Slide

  104. View Slide

  105. View Slide

  106. View Slide

  107. View Slide

  108. View Slide

  109. View Slide

  110. View Slide

  111. View Slide

  112. Open Data Principles
    1. You have no idea (at all) what open data
    might be used for.
    2. PDFs are where data goes to die.
    3. Sometimes “open” can simply mean
    following rules of design.
    4. Open data is a conversation.
    5. Sometimes your users will create open data
    for you.
    6. Open data is a necessary precondition for 

    tenacious citizenship.

    View Slide

  113. Slides are online at
    http://l.ruk.ca/gpc-slides
    These slides are released under a Creative Commons license.
    You are encouraged to distribute them as far and as 

    wide as possible under the terms of the license.

    View Slide