Data Product Catalog: from Data Contracts to Knowledge Graph

The management of data as a product is at the core of all modern approaches to data management. The objective is to build modular data architectures capable of effectively handling their complexity and evolving sustainably over time. While data contracts are crucial for achieving interoperability among data products, they alone fall short in ensuring their composability. To this end, a data product catalog is necessary.

In this talk at the Data Innovation Summit 2024, we will explore what a data product catalog is and how it leverages metadata contained in data contracts to make data products addressable, discoverable, and understandable for consumers or in simpler terms, to make them effortlessly composable to implement an unbounded set of use cases. Specifically, we will see how the data product catalog can link the data and metadata of each data product to a semantic data model (domain ontology), thus constructing an enterprise knowledge graph essential for enabling self-service analytics and generative AI use cases.

Transcript

1. Data product catalog from data contracts to knowledge graph
2. PART 1 > Unlocking AI potential through IA productization
3. Information architecture Data is useless Data is an asset that only unlocks its value when put to use. Data management is not limited to just managing data DATA RELATIONSHIPS + Meaning KNOWLEDGE INTELLIGENCE ALGORITHMS + Actions METADATA Context INFORMATION +
4. Data Products Moving from monolithic to modular architectures Monolithic solutions Modular solutions
5. Data DRIVES product’s functionalities Data Products Definition Digital products Data products Transactional products Pure data products Data SUPPORTS product’s functionalities Data IS the product
6. Syntactic & tech. interoperability Data Contracts Moving from integrability to interoperability Data Product Data Contract Schema Constraints API Populate Accepts & consume Shared Lifecycle Metadata Data
7. Knowledge graph Moving from interoperability to composability Upper ontology Semantic Interop. Data products Enterprise Ontology Data products 1. enable access to physical data asset 2. aggregate technical metadata related to exposed data 3. create the semantic link between physical data asset and business concepts modeled in the enterprise ontology Data products are a pivotal element in the incremental and distributed construction of a knowledge graph Domain ontology Physical Data Subdomain ontologies Syntactic Interop.
8. PART 2 > Boosting IA productization with a platform strategy
9. Platforming Rethinking the data value chain Factory Platform Ecosystem Linear value creation Economy of scope Economy of scale Non-Linear value creation (network effects) Consumers Producers Mediators DATA INFORMATION KNOWLEDGE INTELLIGENCE DATA INFORMATION KNOWLEDGE INTELLIGENCE
10. Data product catalog Platform Plays Bring back personalization of experience for business users Bring data producers on top of the Value Chain Standardization of Transactions Complex Process embedded into Software as a Service Enable leveraging on Identity, Reputation and Trust Aggregation of Demand and Supply Enterprise Data Marketplace Data Developer Platform Learning Engine Transaction Engine Trust Engine
11. PART 3 > Implementing the platform strategy with the right tools
12. Beyond slideware DPDS, ODM and Blindata Bring back personalization of experience for business users Bring data producers on top of the Value Chain Standardization of Transactions Complex Process embedded into Software as a Service Enable leveraging on Identity, Reputation and Trust Aggregation of Demand and Supply Enterprise Data Marketplace Data Developer Platform Data Product Descriptor Specification Open Data Mesh Platform Tell me more… Tell me more… Tell me more…
13. Bring back personalization of experience for business users Bring data producers on top of the Value Chain Standardization of Transactions Complex Process embedded into Software as a Service Enable leveraging on Identity, Reputation and Trust Aggregation of Demand and Supply Enterprise Data Marketplace Data Developer Platform Data Product Descriptor Specification Open Data Mesh Platform Learning Engine Transaction Engine
14. Data Product Catalog Data Developer Platform Data Product Builder Monitoring & Policies Management Deployment Management Data Product Registry Initialize data product blueprint Develop data product Publish descriptor Validate descriptor Deploy data product Monitor data product Data product lifecycle management Producer
15. Data Product Catalog Enterprise Data Marketplace Metadata Management Data Product Consumption Management Activation & Collaboration Ontology Management Search Understand Evaluate Access Compose Share Data product supply management Consumer
16. Data Product Catalog What About AI? SPARQL Cypher Vector Search SQL Data Product Catalog Ecosystem Platform disambiguate the term customer How can i find the count of new customers acquired last month? find relationship in the metadata graph follow the links to the related assets - definitions - assets - sql query Consumption Layer
17. Thanks! Book he demo Book a demo here….