Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Open Data and CKAN

Open Data and CKAN

A short presentation about our experience in choosing an open data platform and using CKAN. Prepared by Foundation Obshtestvo.bg.

Dimitar Dimitrov

September 17, 2014
Tweet

More Decks by Dimitar Dimitrov

Other Decks in Technology

Transcript

  1. Agenda 1. Abstract 1. About us 2. How we understand

    the problem 3. The idea about an open data demo project
  2. Agenda 1. Abstract 1. About us 2. How we understand

    the problem 3. The idea about an open data demo project 2. The CKAN platform
  3. Agenda 1. Abstract 1. About us 2. How we understand

    the problem 3. The idea about an open data demo project 2. The CKAN platform 1. How we chose CKAN
  4. Agenda 1. Abstract 1. About us 2. How we understand

    the problem 3. The idea about an open data demo project 2. The CKAN platform 1. How we chose CKAN 2. Why open source and open technologies?
  5. Agenda 1. Abstract 1. About us 2. How we understand

    the problem 3. The idea about an open data demo project 2. The CKAN platform 1. How we chose CKAN 2. Why open source and open technologies? 3. About CKAN
  6. Agenda 1. Abstract 1. About us 2. How we understand

    the problem 3. The idea about an open data demo project 2. The CKAN platform 1. How we chose CKAN 2. Why open source and open technologies? 3. About CKAN 4. Our experience with the platform
  7. Agenda 1. Abstract 1. About us 2. How we understand

    the problem 3. The idea about an open data demo project 2. The CKAN platform 1. How we chose CKAN 2. Why open source and open technologies? 3. About CKAN 4. Our experience with the platform 3. Current status of the project
  8. Agenda 1. Abstract 1. About us 2. How we understand

    the problem 3. The idea about an open data demo project 2. The CKAN platform 1. How we chose CKAN 2. Why open source and open technologies? 3. About CKAN 4. Our experience with the platform 3. Current status of the project 4. Future work
  9. Agenda 1. Abstract 1. About us 2. How we understand

    the problem 3. The idea about an open data demo project 2. The CKAN platform 1. How we chose CKAN 2. Why open source and open technologies? 3. About CKAN 4. Our experience with the platform 3. Current status of the project 4. Future work 5. Questions and discussion
  10. 1.1. About Us • Joro and Dimitar, part of Obshtestvo.bg

    (translates as “community”) • A young (~1 year old) non-government, non-profit organization
  11. 1.1. About Us • Joro and Dimitar, part of Obshtestvo.bg

    (translates as “community”) • A young (~1 year old) non-government, non-profit organization • Software developers, designers, journalists, marketing specialists, lawyers and many others
  12. 1.1. About Us • Joro and Dimitar, part of Obshtestvo.bg

    (translates as “community”) • A young (~1 year old) non-government, non-profit organization • Software developers, designers, journalists, marketing specialists, lawyers and many others • Currently about 10 active projects and campaigns aimed at improving the interaction between citizens and the government
  13. 1.1. About Us • Joro and Dimitar, part of Obshtestvo.bg

    (translates as “community”) • A young (~1 year old) non-government, non-profit organization • Software developers, designers, journalists, marketing specialists, lawyers and many others • Currently about 10 active projects and campaigns aimed at improving the interaction between citizens and the government • We strongly believe in open data, open source, agile, transparency and collaboration with the government
  14. 1.2. The Problem • This is our very rough understanding

    of the overall picture in Bulgaria in relation to public and open data
  15. 1.2. The Problem • This is our very rough understanding

    of the overall picture in Bulgaria in relation to public and open data • A significant amount of data is public, but not in a machine readable format and it’s not updated regularly
  16. 1.2. The Problem • This is our very rough understanding

    of the overall picture in Bulgaria in relation to public and open data • A significant amount of data is public, but not in a machine readable format and it’s not updated regularly • No central hub to concentrate and categorize all available public data
  17. 1.2. The Problem • This is our very rough understanding

    of the overall picture in Bulgaria in relation to public and open data • A significant amount of data is public, but not in a machine readable format and it’s not updated regularly • No central hub to concentrate and categorize all available public data • Lots of misconceptions and fear about open data
  18. 1.2. The Problem • This is our very rough understanding

    of the overall picture in Bulgaria in relation to public and open data • A significant amount of data is public, but not in a machine readable format and it’s not updated regularly • No central hub to concentrate and categorize all available public data • Lots of misconceptions and fear about open data • Poor back-end software systems additionally complicate data management
  19. 1.2. Short-term Goals • Show the public that open data

    can easily aid in creating value-added products
  20. 1.2. Short-term Goals • Show the public that open data

    can easily aid in creating value-added products • Give citizens the incentive to demand open data more rigorously from the government
  21. 1.2. Short-term Goals • Show the public that open data

    can easily aid in creating value-added products • Give citizens the incentive to demand open data more rigorously from the government • Fight misconceptions regarding open data in the administration – given the proper infrastructure, open data becomes the easier option for data publishing
  22. 1.2. Short-term Goals • Show the public that open data

    can easily aid in creating value-added products • Give citizens the incentive to demand open data more rigorously from the government • Fight misconceptions regarding open data in the administration – given the proper infrastructure, open data becomes the easier option for data publishing • Lay the groundwork for creating a government operated central open data hub
  23. 1.3. Demo Project • An idea by the current deputy-minister

    of the Ministry of Transport, Information Technology and Communications, Kamen Spassov
  24. 1.3. Demo Project • An idea by the current deputy-minister

    of the Ministry of Transport, Information Technology and Communications, Kamen Spassov • Create an experimental central hub for open data
  25. 1.3. Demo Project • An idea by the current deputy-minister

    of the Ministry of Transport, Information Technology and Communications, Kamen Spassov • Create an experimental central hub for open data • Import existing public data sets, converting them in a machine-readable and API-accessible format
  26. 1.3. Demo Project • An idea by the current deputy-minister

    of the Ministry of Transport, Information Technology and Communications, Kamen Spassov • Create an experimental central hub for open data • Import existing public data sets, converting them in a machine-readable and API-accessible format • Create simple applications on top of the APIs
  27. 1.3. Demo Project • An idea by the current deputy-minister

    of the Ministry of Transport, Information Technology and Communications, Kamen Spassov • Create an experimental central hub for open data • Import existing public data sets, converting them in a machine-readable and API-accessible format • Create simple applications on top of the APIs • Demo the results and use the public interest to educate about the benefits of open data
  28. 2.1. Choosing a Platform • Existing solution or build our

    own? • Evaluated Socrata, CKAN, DKAN, Junar and others
  29. 2.1. Choosing a Platform • Existing solution or build our

    own? • Evaluated Socrata, CKAN, DKAN, Junar and others • CKAN seemed very stable and the most widely adopted one
  30. 2.1. Choosing a Platform • Existing solution or build our

    own? • Evaluated Socrata, CKAN, DKAN, Junar and others • CKAN seemed very stable and the most widely adopted one • Because of time and resource constraints choose an existing platform for the experiment
  31. 2.1. Why CKAN • Stable • Popular • Modular and

    extensible • Built on open technologies
  32. 2.1. Why CKAN • Stable • Popular • Modular and

    extensible • Built on open technologies • Open source with a permissive license
  33. 2.1. Why CKAN • Stable • Popular • Modular and

    extensible • Built on open technologies • Open source with a permissive license • Translated in many languages, including Bulgarian
  34. 2.2. Why Open • The use of open technologies in

    the platform was a key metric • Allows us to evaluate the technical quality of the product
  35. 2.2. Why Open • The use of open technologies in

    the platform was a key metric • Allows us to evaluate the technical quality of the product • Allows us to extend and modify our installation as needed
  36. 2.2. Why Open • The use of open technologies in

    the platform was a key metric • Allows us to evaluate the technical quality of the product • Allows us to extend and modify our installation as needed • Provides some level of guarantee against vendor lock-in
  37. 2.2. Why Open • The use of open technologies in

    the platform was a key metric • Allows us to evaluate the technical quality of the product • Allows us to extend and modify our installation as needed • Provides some level of guarantee against vendor lock-in • Allows us to validate our ideas quickly and in a low-cost manner
  38. 2.3. The CKAN Platform • Built on top of and

    runs on open technologies such as Linux, Python, and PostgreSQL
  39. 2.3. The CKAN Platform • Built on top of and

    runs on open technologies such as Linux, Python, and PostgreSQL • Extensible via plugins
  40. 2.3. The CKAN Platform • Built on top of and

    runs on open technologies such as Linux, Python, and PostgreSQL • Extensible via plugins • Licensed under AGPL v3
  41. 2.3. The CKAN Platform • Built on top of and

    runs on open technologies such as Linux, Python, and PostgreSQL • Extensible via plugins • Licensed under AGPL v3 • Built by Open Knowledge Foundation, Germany
  42. 2.3. The CKAN Platform • Built on top of and

    runs on open technologies such as Linux, Python, and PostgreSQL • Extensible via plugins • Licensed under AGPL v3 • Built by Open Knowledge Foundation, Germany • Used by many central and local governments, including the European Commission
  43. 2.3. CKAN in the EU • Austria – http://data.gv.at/ •

    Holland – https://data.overheid.nl/
  44. 2.3. CKAN in the EU • Austria – http://data.gv.at/ •

    Holland – https://data.overheid.nl/ • Italy – http://www.dati.gov.it/
  45. 2.3. CKAN in the EU • Austria – http://data.gv.at/ •

    Holland – https://data.overheid.nl/ • Italy – http://www.dati.gov.it/ • Romania – http://data.gov.ro/
  46. 2.3. CKAN in the EU • Austria – http://data.gv.at/ •

    Holland – https://data.overheid.nl/ • Italy – http://www.dati.gov.it/ • Romania – http://data.gov.ro/ • Slovakia – http://data.gov.sk/
  47. 2.3. CKAN in the EU • Austria – http://data.gv.at/ •

    Holland – https://data.overheid.nl/ • Italy – http://www.dati.gov.it/ • Romania – http://data.gov.ro/ • Slovakia – http://data.gov.sk/ • Sweden – http://oppnadata.se/
  48. 2.3. CKAN in the EU • Austria – http://data.gv.at/ •

    Holland – https://data.overheid.nl/ • Italy – http://www.dati.gov.it/ • Romania – http://data.gov.ro/ • Slovakia – http://data.gov.sk/ • Sweden – http://oppnadata.se/ • UK – http://data.gov.uk/
  49. 2.4. Our Experience • Deployment and configuration were not trivial

    • Some plugins (extensions) need to be activated for a sensible feature set, they’re not on by default
  50. 2.4. Our Experience • Deployment and configuration were not trivial

    • Some plugins (extensions) need to be activated for a sensible feature set, they’re not on by default • Bulgarian translation was only 50% complete, we fixed that
  51. 2.4. Our Experience • Deployment and configuration were not trivial

    • Some plugins (extensions) need to be activated for a sensible feature set, they’re not on by default • Bulgarian translation was only 50% complete, we fixed that • Documentation seems to be good
  52. 2.4. Our Experience • Deployment and configuration were not trivial

    • Some plugins (extensions) need to be activated for a sensible feature set, they’re not on by default • Bulgarian translation was only 50% complete, we fixed that • Documentation seems to be good • Will do the job for our purpose, but we still need time to evaluate it more in actual usage
  53. Use CKAN officially • CKAN only holds a copy of

    the data and the data is public
  54. Use CKAN officially • CKAN only holds a copy of

    the data and the data is public • There are practically no security risks in using CKAN officially as a central open data hub
  55. Use CKAN officially • CKAN only holds a copy of

    the data and the data is public • There are practically no security risks in using CKAN officially as a central open data hub • Migration from CKAN to other platforms is possible if needed
  56. Use CKAN officially • CKAN only holds a copy of

    the data and the data is public • There are practically no security risks in using CKAN officially as a central open data hub • Migration from CKAN to other platforms is possible if needed • Thus, no risks to use it officially and encourage data to be published there