Getting to understand Linked Open Data in 60 minutes

Guest lecture at Ghent University

Transcript

1. Open Data in 60 minutes https://pietercolpaert.be/#me Ghent University – Guest lecture 2017-05-08
2. Will engineering a better information system help us build a type I civilization? https://en.wikipedia.org/wiki/Kardashev_scale
3. Open Data in the world For example Data Portal from Worldbank http://data.worldbank.org
4. Open Data in Europe Public Sector Information INSPIRE directive PSI directive http://europeandataportal.eu
5. http://www.alterechos.be/bianca-debaets-lopen-data-a-bruxelles-un-potentiel-de-1-500-emplois/
6. Costs Benefits When are you going to reuse open data?
7. Public Sector Information vs. Open Data? PSI Open Data
8. SNCB STIB De Lijn TEC Schedules shared shared shared open Real-time shared shared shared planned Tickets no no yes no Historic no no no open Status of e.g., Public Transit in BE?
9. Open Data vs. data sharing?
10. Sharing data between 2 systems Your system Third party system Agree on a protocol Will determine which questions can be answered in a timely fashion Can ask questions to your system as previously agreed
11. Sharing data on the Web Your system ? ? ? ? ? ? Maximizing reuse → need to raise the interoperability
12. ↓ Querying syntactic semantic technical legal When I have got 2 datasets, how easy is it to use them as if they were 1?
13. As a reuser, you need certainty that you won’t get sued https://github.com/iRail/stations
15. OpenDefinition.org ↓ Querying syntactic semantic technical legal
16. Data licenses Interested in the full story? https://pietercolpaert.be/open%20data/2017/02/23/cc0.html
17. A story of raising interoperability ↓ Querying syntactic semantic technical legal When I have 2 datasets, how easy is it to turn them into 1 dataset? → Open Definition & open licenses → The Internet: exchanging data world-wide → JSON, XML, CSV, HTML… Open Standards
18. name type city population StP-Plein Parking Gent 257k { “StP-Plein” : { “type” : “Parking”, “city” : “Gent, “population” : “257k” } } <StP-Plein> <type>Parking</type> <city>Gent</city> <population> 257k </population> </StP-Plein> Table / CSV / Spreadsheet JSON XML Serialisations ↓ Querying syntactic semantic technical legal
19. { “StP-Plein” : { “type” : “Parking”, “city” : “Gent, “population” : “257k” } } name type city population StP-Plein Parking Gent 257k <StP-Plein> <type>Parking</type> <city>Gent</city> <population> 257k </population> </StP-Plein> <StP-Plein> <type> <Parking> . <StP-Plein> <city> <Gent> . <Gent> <population> “257k” . Table / CSV / Spreadsheet 3 time a datum Triples JSON XML
20. World Wide Web St-P Plein city Gent St Pietersplein type Parking Gent population 257k HTTP Machine 1 HTTP Machine 2 HTTP Machine 3 Thought experiment: decentralizing publishing A user agent visiting each machine knows more than any of the machines independently
21. Problem Sint-Pietersplein is a Parking Site ? ↓ Querying syntactic semantic technical legal
22. Problem Sint-Pietersplein is a Parking Site ? ↓ Querying syntactic semantic technical legal
23. Solution Sint Pietersplein → https://stad.gent/id/parking/P10 is a → http://www.w3.org/1999/02/22-rdf-syntax-ns#type Parking → http://vocab.datex.org/terms#UrbanParkingSite Uniform Resource Identifiers (URIs)
24. A story of raising interoperability ↓ Querying syntactic semantic technical legal When I have 2 datasets, how easy is it to turn them into 1 dataset? → Open Definition & open licenses → The Internet: exchanging data world-wide → JSON, XML, CSV, HTML… Open Standards → Linked Data
25. Open Data is only a legal definition… but: The 5 stars of Linked Open Data http://5stardata.info
26. Sharing data between 2 systems Your system Third party system Agree on a protocol Will determine which questions can be answered in a timely fashion Can ask questions to your system as previously agreed
27. Sharing data on the Web Your system ? ? ? ? ? ? Maximizing reuse → need to raise the interoperability
28. data dump Ask any question How to allow for asking any kind of query? Your system 3d party Your system ? ? ? ? ? ?
29. data dump Ask any question Asking questions Your system 3d party Your system ? ? ? ? ? ? Data publishing: Cacheable Dataset split in fragments
30. A long tail for for e.g., transport data services ... Hard to guess which kind of queries will be needed … More specific features Size of audience Google maps Proximus CityMapper Go-OV Ally Transit App NextTrain smartwatch
31. Proposal http://api.{mycompany}/?from={A}&to={B} &departuretime=2016-10-16T14:45.024Z &wheelchairaccessible=true &transit_modes=plane,railway,bus,car &algoritm_mode=shortest ... Yet this interface will need to answer all questions for all third party apps…
32. data dump Route planning algorithms as a service Asking questions Your system 3d party Your system ? ? ? ? ? ? Does not scale: Extra users comes with extra load Does not give necessary flexibility to companies
33. Discover all the necessary data on the Web Just like websites, we want your data to be high available
34. API fanboys Real data reusers Need Open Data Want services on top of data What we ask data owners
35. What we ask data owners Data dumps Smart servers Data publishing (cheap/reliable) Data services (rather expensive/unreliable) Entire query languages over HTTP Dataset split in fragments Smart agents algorithms as a service Read more at http://linkeddatafragments.org API fanboys Open Data
36. Business model? API fanboys Real data reusers Need Open Data Need services on top of data Business opportunity?
37. Servers publishing Open Data e.g., • all the planned and actual arrivals and departures • the network of roads in a certain region worldwide web-services e.g., • a route planner: from → to • the closest station to your current location? Scalable businesses $$$ $ $ $$$ end-users
38. We want a world where knowledge creates power for the many, not the few. Looking for a student job in Open Data? Check out http://summerofcode.be