$30 off During Our Annual Pro Sale. View Details »

LDES and the Flemish Smart Data Space

LDES and the Flemish Smart Data Space

https://tree.linkeddatafragments.org

Presented at Future Summits 2022

Pieter Colpaert

May 18, 2022
Tweet

More Decks by Pieter Colpaert

Other Decks in Research

Transcript

  1. The Flemish Smart Data Space Pieter Colpaert https://pietercolpaert.be

  2. What’s the best API to publish a dataset in a

    data space? My data space cache Query language APIs in the data space End-user device Full question
  3. Dataset SPARQL 3d parties Graph-based application WFS GIS application ...

    ... Publisher Approach 1: Hosting querying APIs
  4. Dataset SPARQL 3d parties Graph-based application WFS GIS application ...

    ... Publisher Maintenance hell
  5. Data dump Autocompletion 3d parties Awesome application WFS Awesome application

    ... ... Approach 2: Sharing dumps Publisher
  6. Data dump Autocompletion 3d parties Awesome application WFS Awesome application

    ... ... Publisher Replication hell
  7. Autocompletion Awesome application WFS Awesome application ... ... Let’s not

    call it a data dump, but let’s call it an “event stream” 𝚫 𝚫 𝚫 𝚫 3d parties Publisher → Staying in-sync with an event stream is a core concern
  8. The Linked Data Event Streams (LDES) specification https://w3id.org/ldes/specification

  9. Consumers cannot replicate the entire web Triple store with everything

    SPARQL ingests everything End-user device Full question
  10. Linked Data Event Stream ?page=1 3d parties Publisher 𝚫 𝚫

    𝚫 𝚫 𝚫 𝚫 𝚫 𝚫 𝚫 𝚫 Autocompletion Awesome application Text index /a?page=1 a b … ?q=Gen Autocompletion API Substring fragmentations of Event Streams
  11. Substring fragmentations for autocompletion Ruben, D., Delva, H., Colpaert, P.,

    & Verborgh, R. (2021, May). “A File-Based Linked Data Fragments Approach to Prefix Search”. Proceedings of the 21th International Conference on Web Engineering. Live demo at tree.linkeddatafragments.org
  12. III Querying interfaces II Reusable indexes I Linked Data Event

    Stream Third party indexes Third party querying interfaces Priority circles as pitched by the CoGhent museum project Subject pages & SPARQL endpoint By city of Ghent Enriched LDES with IIIF manifest and image links For every institution’s collection objects Indexed by substring By IDLab Van de Vyvere, B., Van D'Huynslager, O., Atauil, A., Segers, M., Van Campe, L., Vandekeybus, N., Teugels, S., Saenko, A., Pauwels, P.-J., & Colpaert, P.(2022, January). “Publishing cultural heritage collections of Ghent with Linked Data Event Streams”. Proceedings of the Metadata and Semantics Research Conference 2021.
  13. III Querying interfaces II Reusable indexes I Linked Data Event

    Stream Third party indexes Third party querying interfaces When your users still want a SPARQL endpoint, GraphQL interface, OGC API, … You can still offer this to them as lowest priority, without any promise to keep them online because they can also host it themself.
  14. Evolvability is key Powered by 3 specs: 1. Linked Data

    Event Streams https://w3id.org/ldes/specification 2. TREE hypermedia https://w3id.org/tree/specification 3. Smart Data Specification for lineage https://w3id.org/sds/specification
  15. The Flemish Data Space The next 2 years we are

    building reusable workflow components Funded for ~€10M by Flemish government Adopted by EC’s SEMIC in its pilots programme
  16. 1. Publish an LDES as the base API accept you

    cannot do everything your ecosystem can do the rest 2. Your fragmentations can evolve as fast as you want clients will find them just in time 3. When coding an app Code against API patterns, not 1 specific data source Three take-aways
  17. The Flemish Smart Data Space Pieter Colpaert https://pietercolpaert.be