LDES and the Flemish Smart Data Space

https://tree.linkeddatafragments.org

Presented at Future Summits 2022

Transcript

1. The Flemish Smart Data Space Pieter Colpaert https://pietercolpaert.be
2. What’s the best API to publish a dataset in a data space? My data space cache Query language APIs in the data space End-user device Full question
3. Dataset SPARQL 3d parties Graph-based application WFS GIS application ... ... Publisher Approach 1: Hosting querying APIs
4. Dataset SPARQL 3d parties Graph-based application WFS GIS application ... ... Publisher Maintenance hell
5. Data dump Autocompletion 3d parties Awesome application WFS Awesome application ... ... Approach 2: Sharing dumps Publisher
6. Data dump Autocompletion 3d parties Awesome application WFS Awesome application ... ... Publisher Replication hell
7. Autocompletion Awesome application WFS Awesome application ... ... Let’s not call it a data dump, but let’s call it an “event stream” 𝚫 𝚫 𝚫 𝚫 3d parties Publisher → Staying in-sync with an event stream is a core concern
8. The Linked Data Event Streams (LDES) specification https://w3id.org/ldes/specification
9. Consumers cannot replicate the entire web Triple store with everything SPARQL ingests everything End-user device Full question
10. Linked Data Event Stream ?page=1 3d parties Publisher 𝚫 𝚫 𝚫 𝚫 𝚫 𝚫 𝚫 𝚫 𝚫 𝚫 Autocompletion Awesome application Text index /a?page=1 a b … ?q=Gen Autocompletion API Substring fragmentations of Event Streams
11. Substring fragmentations for autocompletion Ruben, D., Delva, H., Colpaert, P., & Verborgh, R. (2021, May). “A File-Based Linked Data Fragments Approach to Prefix Search”. Proceedings of the 21th International Conference on Web Engineering. Live demo at tree.linkeddatafragments.org
12. III Querying interfaces II Reusable indexes I Linked Data Event Stream Third party indexes Third party querying interfaces Priority circles as pitched by the CoGhent museum project Subject pages & SPARQL endpoint By city of Ghent Enriched LDES with IIIF manifest and image links For every institution’s collection objects Indexed by substring By IDLab Van de Vyvere, B., Van D'Huynslager, O., Atauil, A., Segers, M., Van Campe, L., Vandekeybus, N., Teugels, S., Saenko, A., Pauwels, P.-J., & Colpaert, P.(2022, January). “Publishing cultural heritage collections of Ghent with Linked Data Event Streams”. Proceedings of the Metadata and Semantics Research Conference 2021.
13. III Querying interfaces II Reusable indexes I Linked Data Event Stream Third party indexes Third party querying interfaces When your users still want a SPARQL endpoint, GraphQL interface, OGC API, … You can still offer this to them as lowest priority, without any promise to keep them online because they can also host it themself.
14. Evolvability is key Powered by 3 specs: 1. Linked Data Event Streams https://w3id.org/ldes/specification 2. TREE hypermedia https://w3id.org/tree/specification 3. Smart Data Specification for lineage https://w3id.org/sds/specification
15. The Flemish Data Space The next 2 years we are building reusable workflow components Funded for ~€10M by Flemish government Adopted by EC’s SEMIC in its pilots programme
16. 1. Publish an LDES as the base API accept you cannot do everything your ecosystem can do the rest 2. Your fragmentations can evolve as fast as you want clients will find them just in time 3. When coding an app Code against API patterns, not 1 specific data source Three take-aways
17. The Flemish Smart Data Space Pieter Colpaert https://pietercolpaert.be