Data Lakehouse 第三世代 E T E E L T E AI/ML L BI L T L E L T E T DI L L E T E T L E T L AI/ML T BI BI DI DI etc. etc. etc. データ種類 構造化データ 構造化、半構造、非構造データ 構造化、半構造、非構造データ 利用用途 BI/DI AI (ETL→DWH経由でBI/DI) AI/BI/DI
Data Mesh From Inter-Department Data User Data User BizDevOps BizDevOps Distribution Department Sales Department Wholesale Company Logistics Company BizDevOps Production Department Manufacturer
and How to Use: Data Addressability and Discoverability(DAD) Ontology and Semantic Interoperability(OSI) Identity and Usage Control (IUC) 分散型Data Managementにおける「ドメイン、企業横断の透過的なSSOT」と 「データ提供者が対価を得るメカニズム」を実現
Server) turtle # Company A { "altitude_m": 120.0, "datum": "AGL" } @prefix ex: <http:example.org/> Decouple Information Model ex: AltitudeObservation a owl: Class. ex: AltitudeDatum a owl: Class. #Accept only one datum ex: datum a owl: Objectproperty, owl: FunctionalProperty . # Company B { "altitude": 120.0, "unit" : "meter", "datum" : "MSL" } ex:AGL a ex:AltitudeDatum . ex:MSL a ex:AltitudeDatum . #Differentiate AGL and MSL ex:AGL owl:differentFrom ex:MSL. ETL Data User record id altitude type 1 120.0 AGL 2 120.0 MSL “Bugs” in Application json "@context" :{…}, "@id:": "ex:XXX", "@type": "AltitudeObservation", … T Data User Inconsistent ontology ex:AGL is different from ex:MSL. Error in Ontology Information Model Data Model json
1 Domain 2 Domain 3 ・・・ Domain N Apps Company N Company B Company N Company B Company A Domain 1 Domain 2 Discovery Service keyword Apps Domain 3 ・・・ Domain N Discovery Service keyword
in ODS Middleware Ontology Endpoint Query Server API Gateway PEP PEP Policy Engine PDP ODS-RAM L3 (Authentication) w/ ODS-RAM L3 (Authorization: PDP) w/ ODS-RAM L2 (PEP) Query Server API Gateway PEP PEP See details at Open Data Spaces Protocols (Gitbook) & Open Data Spaces Middleware (GitHub) (Multi-Modal) Data Endpoint Data Product Server
ODS Heuristic Contracting Protocol Access denied eContract Apps Optional redirects Optional No Yes PEP Optional: e-Contracting? status: 200 3rd-party Apps ODS Clearing and Payment Protocol Data User PDP Data Provider Apps Data User Example Save as a log Payment Apps redirects No Yes TX Optional: Need to charge? Log charges Apps