Portable Spatial-Semantic RAG for 3D City Models Using DuckDB

FOSS4G Hiroshima 2026で発表した資料です。
デモで作ったWebアプリのコードは以下で公開しています。
https://github.com/raokiey/hiroshima-bldg-rag-example

Transcript

1. FOSS4G Hiroshima 2026 2026.09.02 Portable Spatial-Semantic RAG for 3D City Models Using DuckDB Ryosuke Aoki / ぴっかりん (@ra0kley)
2. About Me Ryosuke Aoki ⚫ Committee Member of OSGeo Japan Chapter ⚫ Project PLATEAU ADVOCATE X @ra0kley (Username: ぴっかりん) GitHub raokiey FOSS4G Hiroshima 2026 2
3. Where This Started ⚫ It started with one question: how do you give map data to an AI? ⚫ I picked PLATEAU — as a PLATEAU advocate, I already knew this data well. I focused on what only 3D data can do. FOSS4G Hiroshima 2026 3
4. The Question “Find me sunny buildings near Hiroshima Station.” FOSS4G Hiroshima 2026 4
5. The Question “Find me sunny buildings near Hiroshima Station.” Easy to ask. Not easy to answer. FOSS4G Hiroshima 2026 5
6. Two Different Problems 1 Where 2 Which way “near the station” is a “sunny” is a spatial problem 3D shape problem north S within 500 m south FOSS4G Hiroshima 2026 6
7. Why This Is Hard ✓ AI search understands words, not maps ✓ 3D data can lose direction when it is converted WHAT THE AI SEES WHAT THE FILE KEEPS “near” the shape — but no wall labels “sunny” just words — no distance, no direction FOSS4G Hiroshima 2026 7
8. What is Project PLATEAU? ⚫ Japan’s open 3D city model data ⚫ Made by the government (MLIT) and partners ⚫ Free for anyone to use Image： Project PLATEAU official website （MLIT） FOSS4G Hiroshima 2026 8
9. Not Just a 3D Map ⚫ Google Earth: mesh only ⚫ CityGML: shape + meaning — Building, Wall Surface, Roof Surface. Image 4 Project 3 6 D令 PLATEAU ： 年 回 第 都 度和 市 モデ ルの整備・ 活用促進に関する検討分科会 Image： 令和4年度 Project PLATEAU 第6回 3D都市モデルの整備・活用促進に関する検討分科会 FOSS4G Hiroshima 2026 9
10. CityGML → GeoPackage ✓ CityGML labels every surface: wall, roof, ground →Convert it for GIS, and those labels can disappear FOSS4G Hiroshima 2026 10
11. What is RAG? 1 2 3 4 Question Search Give to AI Answer in your words find your data as context from that data • The AI answers from your data, not from memory. → But it still has no idea where things are. FOSS4G Hiroshima 2026 11
12. What is Vector Search? sunny wooden house south-facing ⚫ Turn text into numbers ⚫ Match by meaning, not by exact words bright flood risk close in meaning FOSS4G Hiroshima 2026 12
13. SQL and Vector Search EXACT FUZZY SQL Good at exact numbers Vector height > 20 m “sunny” within 500 m “quiet” Good at fuzzy meaning → A real question usually needs both. FOSS4G Hiroshima 2026 13
14. One File, Two Kinds of Search plateau_rag.duckdb geometry attributes embeddings ⚫ DuckDB + spatial and vss ⚫ Everything in one file ⚫ No server to set up — DuckDB runs right inside the program. ⚫ The embedding model is open (ruri-v3) FOSS4G Hiroshima 2026 14
15. How a Question Flows STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 Read it Find place Filter Rank Answer AI reads it geocode by location rule or meaning AI writes it Step 4 is not always vector search. When the question has one exact answer, the system uses plain SQL instead. FOSS4G Hiroshima 2026 15
16. Reading Direction from the Shape ⚫ Each face has a normal vector ⚫ Roof, ground or wall — from the angle ⚫ Walls also get N / E / S / W → The label was never stored. We compute it. FOSS4G Hiroshima 2026 16
17. How We Actually Did It DuckDB Spatial cannot split one shape into single faces yet. -- DuckDB SELECT id, ST_AsText(geom) FROM buildings; # Python nx, ny, nz = newell(face) roof = nz > 0.8 → Read the shape as text, do the maths in Python. Still one file, still no server. FOSS4G Hiroshima 2026 17
18. From Numbers to Words GEOMETRY SENTENCE EMBEDDING wall_s = 0.42 height = 24.3 “This building has 42% south-facing walls.” [0.13, -0.88, 0.42, …] → Now “sunny” can match a building. FOSS4G Hiroshima 2026 18
19. Not Every Question Needs Vector Search THE QUESTION SQL only “Tallest?” one exact answer “Sunny?” Vector only meaning, no hard rule Both an area plus a fuzzy word FOSS4G Hiroshima 2026 19
20. Checking the Answer ✓ ✓ Does the top building really match? Every candidate is tested again, outside the database. Did the AI invent a building? Every building ID in the answer is checked. → The search can be clever. The answer has to be true. FOSS4G Hiroshima 2026 20
21. Case Study: Hiroshima ⚫ PLATEAU LOD2 near the station ⚫ Flood, storm surge, tsunami risk 2,958 1 buildings file 出典：Project PLATEAU（国土交通省）広島市 LOD2 建物データ FOSS4G Hiroshima 2026 21
22. Demo: Chat + Map FOSS4G Hiroshima 2026 22
23. The Question, Answered “Sunny buildings near Hiroshima Station” ROUTE structured exact SQL 30 m 32% ✓ from the station south-facing walls sunlit in winter → Every number came from the shape. FOSS4G Hiroshima 2026 23
24. What Still Does Not Work ⚫ No terrain data — a hill can block a view ⚫ Sunlight is a simple angle rule, not a simulation ⚫ source data is broken — floor count, for example → These are choices about scope, not bugs. FOSS4G Hiroshima 2026 24
25. Open Source & What's Next SOURCE CODE https://github.com/raokiey/ hiroshima-bldg-rag-example WHAT’S NEXT ⚫ Other cities’ PLATEAU data ⚫ Proper floor counts FOSS4G Hiroshima 2026 25
26. Summary ⚫ This is Spatial-Semantic RAG for 3D city models — search by place and by meaning, together. ⚫ One file, DuckDB — spatial search and vector search, routed automatically. ⚫ Recovering direction from shape is one technique. It even turns fuzzy questions into exact ones. ⚫ Search runs locally, too — even the embedding model is open & free. FOSS4G Hiroshima 2026 26
27. Thank you!