SemPart: Yet Another RDF Partitioning Strategy In Action.

With the continued growth of the Semantic Web, RDF (Resource Description Framework) has become a foundational standard for modeling structured knowledge. However,
managing large-scale RDF datasets remains a challenge, especially in distributed environments where effective data partitioning is critical to performance. Existing RDF
triplestores predominantly rely on triple-based partitioning strategies, which, although
simple, often scatter semantically related data across partitions, resulting in increased
query latency and communication overhead. Another important limitation is the lack of
involvement of database administrators (DBAs) in the partitioning process. Current systems do not provide mechanisms for DBAs to leverage their domain expertise to influence
or adjust the partitioning. As a result, partitioning remains static and rigid, reducing
the systems adaptability to evolving workloads or semantic structures.
To address these limitations, the PQDAG team at the LIAS laboratory (ISAEENSMA) proposed SemPart, a fragment-based RDF partitioning framework designed
to introduce semantic coherence and expert-driven control into the partitioning process
by enabling a range of physical transformations on RDF fragments. While SemPart had
been introduced as a theoretical concept, this work focuses on its practical implementation. Our contribution consists of designing and developing the full SemPart framework
and integrating it into the distributed PQDAG system. By transforming the theoretical
principles of SemPart into a functional system, this work enables flexible, semantically
rich, and performance-oriented RDF partitioning, with potential applications in dynamic
and resource-constrained environments.

Transcript

1. University of Abou Bekr Belkaid – Tlemcen Department of Computer Science With LIAS – France SemPart: Yet Another RDF Partitioning Strategy In Action. Presented by KHIAT Sara Nardjes HAMMOUMI Tarik Supervised by Mr. MATALLAH Houcine Mr. SAIDI Boumediene Mr. MESMOUDI Amin Master’s Thesis Jury Members Mrs. HALFAOUI AMAL Mrs. KHITRI Souad Option : Software Engineering 24 June 2025 1
2. 1 2 3 4 5 GENERAL CONTEXT SEMPART FRAMEWORK PROBLEM STATEMENT OUR CONTRIBUTION CONCLUSION TABLE OF CONTENT 2
3. GENERAL CONTEXT I believe that knowledge should be represented as a graph. Tim Berners-Lee From Knowledge to Large-Scale Graphs 3 Big Graph
4. national_team has_club knows has_country has_club national_team has_country GENERAL CONTEXT I believe that knowledge should be represented as a graph. Tim Berners-Lee From Knowledge to Large-Scale Graphs 4
5. GENERAL CONTEXT Real-World Applications of Large Graphs 5
6. GENERAL CONTEXT Google Knowledge Graph in Action 6
7. GENERAL CONTEXT RDF is a standard model for representing knowledge on the Web using triples: Subject, Predicate, and Object. What is the Resource Description Framework ? Subject Object Predicate 7
8. Has_club RDF is a standard model for representing knowledge on the Web using triples: Subject, Predicate, and Object. GENERAL CONTEXT What is the Resource Description Framework ? Riayd Mahrez AlAhli Subject Object Predicate 8
9. Has_club RDF is a standard model for representing knowledge on the Web using triples: Subject, Predicate, and Object. GENERAL CONTEXT What is the Resource Description Framework ? Riayd Mahrez AlAhli Subject Object Predicate national_team has_club knows has_country has_club national_team has_country 9
10. GENERAL CONTEXT 2010 203 datasets 2025 1314 datasets RDF Data Challenges 10
11. GENERAL CONTEXT Large RDF Knowledge Bases The use of RDF in real-world systems has grown significantly, raising new storage and performance challenges. ≈9.5 billion triples ≈11 billion triples ≈17.8 billion triples 11
12. PROPLEM STATEMENT Limits of Centralized RDF Systems RDF Data Explosion Centralized RDF Systems Centralized RDF systems cannot keep up with the explosive growth of data, leading to memory saturation, slow query processing, and limited scalability. 12
13. PROPLEM STATEMENT Distributed RDF and the Need for Partitioning Distributed RDF systems offer scalability, but their performance heavily depends on how data is partitioned. Poor partitioning leads to network overhead, unbalanced workloads, and slow query execution due to excessive inter-node communication. 13
14. PROPLEM STATEMENT Classic RDF Partitioning Approaches Triple-based Fragment-based RDF data can be partitioned in different ways, depending on the desired balance between simplicity, locality, and semantic grouping. 14
15. PROPLEM STATEMENT 1.Triple-based Partitioning (e.g., TriAD) Each triple is placed independently using a hash function. Simple to apply but leads to poor semantic grouping. Has_club national_team 15
16. PROPLEM STATEMENT 1.Triple-based Partitioning (e.g., TriAD) Each triple is placed independently using a hash function. Simple to apply but leads to poor semantic grouping. Has_club national_team Has_club national_team 16
17. PROPLEM STATEMENT 3. Fragment-based Partitioning(e.g., QDAG, SemStore) Creates meaningful subgraphs based on logic or usage. Better query performance but requires preprocessing. Has_club Has_club Has_country Has_country 17
18. PROPLEM STATEMENT 3. Fragment-based Partitioning(e.g., QDAG, SemStore) Creates meaningful subgraphs based on logic or usage. Better query performance but requires preprocessing. Has_club Has_club Has_club Has_club Has_country Has_country Has_country Has_country 18
19. PROPLEM STATEMENT Limitations of Existing RDF Partitioning Systems 01 03 02 04 SCATTERED SEMANTIC TRIPLES NO DYNAMIC RECONFIGURATION NO EXPERT-DRIVEN CUSTOMIZATION RIGID FRAMEWORKS 19
20. SemPart Framework Collaboration with PQDAG Team 20
21. SemPart Framework national_team has_club knows has_club national_team Characteristic Set national_team has_club knows has_club national_team 21 GraphStar
22. SemPart Framework national_team has_club has_club national_team GraphStar Characteristic Set national_team has_club knows has_club national_team 22
23. Characteristic Set national_team has_club knows has_club national_team SemPart Framework national_team has_club has_club national_team 23 GraphStar
24. SemPart Framework Phases of the SemPart Framework Logical Fragmentation Physical Fragmentation Allocation 24
25. SemPart Framework national_team has_club knows has_club national_team Logical Fragmentation Group RDF data into semantically coherent fragments using characteristic sets. 25
26. SemPart Framework Logical Fragmentation Group RDF data into semantically coherent fragments using characteristic sets. national_team has_club knows has_club national_team 26
27. SemPart Framework national_team has_club knows has_club national_team Physical Fragmentation Apply transformations (e.g., select, split, filter) on logical fragments example : Select has_club = ALAhli 27
28. SemPart Framework Physical Fragmentation Apply transformations (e.g., select, split, filter) on logical fragments national_team has_club has_club national_team example : Select has_club = ALAhli 28
29. Fragment 1 Fragment 2 Fragment 3 SemPart Framework Allocation Distribute fragments across computing nodes based on a defined strategy. MPC METIS 29
30. Our Contribution Physical Fragmentation Engine 01 02 Custom Partitioning Language 03 Interactive Web Platform 30
31. Physical Fragmentation Control Operators Core Operators 01 02 01 02 03 01 02 03 04 Select Operators Filter Operator Split Operators Set Operators Rmove Operator Integration Materialization 31
32. Core Operators 02 03 01 Selection by Fragment Properties Selection by Number of Graph Stars Selection by Graph Star Content Allows retaining fragments based on the properties of their characteristic sets. Allows retaining fragments based on the count of graph stars they contain. Allows retaining fragments that contain at least one graph star satisfying a condition on its content. 01 Select Operators 32
33. 01 Selection by Fragment Properties Select fragments with a characteristic set including the predicates "has_club" Example: Has_club national_team national_team 01 Select Operators 33
34. 01 Selection by Fragment Properties Select fragments with a characteristic set including the predicates "has_club" Example: Has_club national_team national_team 01 Select Operators 34
35. Physical Fragmentation Control Operators Core Operators 01 02 01 02 03 01 02 03 04 Select Operators Filter Operator Split Operators Set Operators Rmove Operator Integration Materialization 35
36. Core Operators 02 03 01 Simple Split Conditional Split GroupBy Split 04 Derived Split Divides fragments such that each sub-fragment does not exceed a maximum number N of graph stars. Groups graph stars into sub- fragments based on the distinct values of specified predicate. Partitions a fragment into two sub-fragments based on a boolean condition involving predicates. Applies a baseline splitting strategy to the current fragment, then extends this logic to connected fragments. 02 Split Operators 36
37. 04 02 Split Operators Derived Split national_team has_club knows has_country has_club national_team has_country Apply a derived split using a Group By on the predicate "national_team", and propagate the result along the path "national_team → has_country". 1.Group by “national_team” 37
38. 04 02 Split Operators Derived Split national_team has_club knows has_country has_club national_team has_country Apply a derived split using a Group By on the predicate "national_team", and propagate the result along the path "national_team → has_country". 1.Group by “national_team”. 2.Propagation via the path "national_team → has_country". 38
39. 04 02 Split Operators Derived Split national_team has_club knows has_country has_club national_team has_country Apply a derived split using a Group By on the predicate "national_team", and propagate the result along the path "national_team → has_country". 1.Group by “national_team” 2.Propagation via the path "national_team → has_country". 3.Conflict Resolution. 39
40. 04 02 Split Operators Derived Split Apply a derived split using a Group By on the predicate "national_team", and propagate the result along the path "national_team → has_country". national_team has_club has_country has_club national_team has_country 1.Group by “national_team” 2.Propagation via the path "national_team → has_country". 3.Conflict Resolution. 40
41. Physical Fragmentation Control Operators Core Operators 01 02 01 02 03 01 02 03 04 Select Operators Filter Operator Split Operators Set Operators Rmove Operator Integration Materialization 41
42. Core Operators 02 03 01 Union Operators Intersection Operators Difference Operators 03 Set Operators 42
43. Physical Fragmentation Control Operators Core Operators 01 02 01 02 03 01 02 03 04 Select Operators Filter Operator Split Operators Set Operators Rmove Operator Integration Materialization 43
44. Our Contribution Physical Fragmentation Engine 01 02 Custom Partitioning Language 03 Interactive Web Platform 44
45. RDPAL Language RDPAL is a domain-specific language (DSL) designed to empower DBAs to customize how RDF data is fragmented and allocated. RDPAL 45
46. What is DSL ? A Domain Specific Language is a programming language with a higher level of abstraction optimized for a specific class of problems. RDPAL Language 46
47. DSL Code Generator Parser Typechecker Code Generator DSL Typechecker DSL Parser RDPAL Language Internal DSL External DSL GPL Code DSL Syntax encountered Parser Typechecker Code Generator DSL Code DSL 47
48. Lexical Analyzer Parser RDPPAL Script Token getNextToken Grammar .g4 RDPAL Language 48
49. q0 q1 q2 q3 q4 q5 q6 q7 Parser Rule Lexical Analysis start CLASS_ID VARIABLE_ID = new CLASS_ID ( arg ) Deterministic Finite Automaton 49 Lexer Rule
50. Lexical Analyzer Parser Syntax Analysis RDPPAL Script Token getNextToken Parse Tree 50
51. Abstract Syntax Tree 51
52. Process Workflow RDPAL DBA Parser DAG Code Runner C++ Engine 01 02 03 04 05 06 52
53. Process Workflow RDPAL DBA Parser 53 01 02 03
54. Process Workflow DAG Directed Acyclic Graph Parser 54 03 04
55. Process Workflow DAG Directed Acyclic Graph 55 04 Id OriginalStatement Type Operation Params VariableName DependsOn
56. node 0 node 1 RDPAL to DAG Translation RDPAL Directed Acyclic Graph 56 1 2
57. node 0 node 1 RDPAL to DAG Translation RDPAL Directed Acyclic Graph 57 1 2
58. Process Workflow Code Runner Prepare commands based on the nodes' information DAG 58
59. Process Workflow C++ Engine Use the binary and the commands to execute them through the CLI Code Runner 59
60. From DAG to Execution Directed Acyclic Graph Parser Code runner 60 node 0 node 1 node 2
61. Our Contribution Physical Fragmentation Engine 01 02 Custom Partitioning Language 03 Interactive Web Platform 61
62. Web Walkthrough 62
63. Our Contribution Physical Fragmentation Engine 01 02 Custom Partitioning Language 03 Interactive Web Platform 63
64. Conclusion 64
65. Thank you 65