Tweet
Tweet

Slide 1

Slide 1 text

Rust for Geospatial Data Processing A Case Study with CityGML Converter for PLATEAU, Japan's Open Digital Twin Models 2024-12-16 Sorami Hisamoto FOSS4G ASIA 2024

Slide 2

Slide 2 text

Rust PLATEAU nusamai Fast, reliable, & ef fi cient programming language Open-source CityGML data conversion tool Digital Twin project led by the Japanese ministry

Slide 3

Slide 3 text

https: // github.com/MIERUNE/plateau-gis-converter

Slide 4

Slide 4 text

https: // github.com/MIERUNE/plateau-gis-converter PLATEAU GIS Converter Convert 3D models of Japanese cities into various geospatial formats. codename “nusamai” Written in Rust, developed by MIERUNE Inc.

Slide 5

Slide 5 text

https: // github.com/MIERUNE/plateau-gis-converter github.com/MIERUNE/plateau-gis-converter OPEN SOURCE (MIT License)

Slide 6

Slide 6 text

@sorami HISAMOTO Sorami ‣ MIERUNE Inc., a geospatial company in Japan ‣ Exploring “Tools for Thought” with computers ‣ Love open source, love community! Hello, FOSS4G community!

Slide 7

Slide 7 text

We are MIERUNE ! ‣ A geospatial company in Japan ‣ Since 2016 - currently 30+ members ‣ A company born from the FOSS4G community Founded by the organizers of FOSS4G Hokkaido

Slide 8

Slide 8 text

Contributions to the community QGIS 5PUBM€ d € WJBUIFDFSUJ fi DBUJPOQSPHSBN MapLibre 5PUBM d 0UIFSTQPOTPSTJODM"84 .FUB .JDSPTPGU

Slide 9

Slide 9 text

Contributions to the community ‣ QGIS Country User Group, Voting Member ‣ MapLibre, Voting Member ‣ OSGeo, Charter Member ‣ OSGeo.JP, Institutional Member ‣ etc., ...

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

FOSS4G ASIA 2024 ‣ We are a GOLD SPONSOR ! ‣ plus 7 presentations !

Slide 12

Slide 12 text

Visit our SPONSOR BOOTH !

Slide 13

Slide 13 text

Rust PLATEAU nusamai Fast, reliable, & ef fi cient programming language Open-source CityGML data conversion tool Digital Twin project led by the Japanese ministry

Slide 14

Slide 14 text

Rust PLATEAU nusamai Fast, reliable, & ef fi

Slide 15

Slide 15 text

https: // www.mlit.go.jp/plateau/about/

Slide 16

Slide 16 text

https: // www.mlit.go.jp/plateau/about/ ‣A project to create and utilize 3D city models of Japanese cities ‣Led by the government (MLIT: Ministry of Land, Infrastructure, Transport & Tourism) ‣Started in year 2020

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

https: // plateauview.mlit.go.jp/

Slide 19

Slide 19 text

https: // plateauview.mlit.go.jp/ Not just simple geometry, but data re fl ecting the real world; year, altitude, land use, etc.

Slide 20

Slide 20 text

https: // www.mlit.go.jp/plateau/use-case/uc22-021/ Use Case: Potential Estimation of Solar Power Generation

Slide 21

Slide 21 text

https: // www.mlit.go.jp/plateau/use-case/uc23-12/ Use Case: Urban planning through gami fi cation; via simulation game “Cities: Skylines”

Slide 22

Slide 22 text

An example in the entertainment domain https: // www.youtube.com/watch?v=Jb6Zlg30rgk

Slide 23

Slide 23 text

https: // www.mlit.go.jp/plateau/open-data/ Open Data ! Currently 218 cities, 20 million features

Slide 24

Slide 24 text

Huge, complex speci fi cation https: // www.mlit.go.jp/plateaudocument/

Slide 25

Slide 25 text

Original data: CityGML format ‣ XML-based format for 3D models of cities and landscapes ‣ Classes & relationships for representing urban features Buildings, roads, terrain, vegetation, etc. ‣ Multi-Level of Detail (LoD) Simple footprints (LoD0) to highly detailed interior structures (LoD4) OGC Standard since 2008

Slide 26

Slide 26 text

Image source: Project PLATEAU, MLIT Japan. https: // www.mlit.go.jp/plateau/learning/tpc01-2/ Box (Building + Height) + Roof Shape + Exteriror struct. + Interior struct.

Slide 27

Slide 27 text

Example: Osaka city data ‣ 24.36GB (ZIP archive: 1.42GB) ‣ LOD1: 225.3kᶷ ‣ LOD2: 5.238kᶷ ‣ LOD3: 0.009kᶷ

Slide 28

Slide 28 text

‣ Overly complex and huge XML format data ‣ Intermediate data: Requires conversion for use ‣ Need costly commercial software for conversion → Handly, free & open tool ? PLATEAU data: The Problems

Slide 29

Slide 29 text

Rust PLATEAU nusamai Fast, reliable, & ef fi cient programming language Open-source CityGML data conversion tool Digital Twin project led by the Japanese ministry

Slide 30

Slide 30 text

Rust PLATEAU nusamai Fast, reliable, & ef fi cient programming language Open-source CityGML data conversion tool Digital Twin project led by the Japanese ministry

Slide 31

Slide 31 text

https: // www.rust-lang.org/

Slide 32

Slide 32 text

https: // www.rust-lang.org/ ‣Modern systems programming langauge First stable release in 2015 ‣Performance, reliability, and productivity Highly performant, comparable to C and C++ Reliable via its elegant language design Modern developer experience ‣Community-led, open source

Slide 33

Slide 33 text

Ferris the crab, unof fi cial mascot for Rust

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

https: // georust.org/

Slide 36

Slide 36 text

https: // github.com/georust/geo/pull/797 3D support is not (yet) available

Slide 37

Slide 37 text

https: // github.com/georust/geo/pull/797 Cross-platform apps with Rust & Web technologies (all major desktop & mobile platforms) (Think “Electron for Rust”)

Slide 38

Slide 38 text

Rust PLATEAU nusamai Fast, reliable, & ef fi cient programming language Open-source CityGML data conversion tool Digital Twin project led by the Japanese ministry

Slide 39

Slide 39 text

Rust PLATEAU nusamai Fast, reliable, & ef fi

Slide 40

Slide 40 text

https: // github.com/MIERUNE/plateau-gis-converter

Slide 41

Slide 41 text

https: // github.com/MIERUNE/plateau-gis-converter PLATEAU GIS Converter Convert 3D models of Japanese cities into various geospatial formats. codename “nusamai” Written in Rust

Slide 42

Slide 42 text

Cross-platform App, with Tauri https: // github.com/MIERUNE/plateau-gis-converter/releases/tag/v0.0.0-alpha.37

Slide 43

Slide 43 text

CityGML → Various data formats ‣ GeoPackage ‣ GeoJSON ‣ 3D Tiles ‣ MVT Mapbox Vector Tiles ‣ glTF ‣CZML ‣KML ‣Minecraft ‣Wavefront OBJ ‣Shapefile 🪦

Slide 44

Slide 44 text

Why Rust ? ‣Performance To process large data in realistic time ‣Developer Experience Nice type system & modern tooling helps a lot ‣Cross-platform, GUI Windows & macOS, GUI instead of CUI → Tauri!

Slide 45

Slide 45 text

Why Rust ? ‣Performance To process large data in realistic time ‣Developer Experience Nice type system & modern tooling helps a lot ‣Cross-platform, GUI Windows & macOS, GUI instead of CUI → Tauri! .. and for FUN ! The team wanted to try Rust

Slide 46

Slide 46 text

Python or JavaScript? ‣ Great GIS ecosystem ‣ But … ‣ Lack of nice GUI tooling ‣ Performance issue ‣ GUI with Electron ‣ But … ‣ Lack of libraries for the domain ‣ Performance issue (?)

Slide 47

Slide 47 text

Rust is great, but we had to create a lot of things from scratch MIERUNE/ fl atgeom-rs Ef fi cient Flat data structures instead of jagged arrays https: // github.com/MIERUNE/flatgeom-rs MIERUNE/earcut-rs A Rust port of the Earcut polygon triangulation library https: // github.com/MIERUNE/earcut-rs plus, Projection transformation, XML (CityGML) parsing, 3D Tiles generation, MVT generation, etc., …

Slide 48

Slide 48 text

Yes! In the end, are we happy with Rust? I don’t think this could have been achieved with other languages or setups.

Slide 49

Slide 49 text

In the end, are we happy with Rust? Rust was perfect for this speci fi c problem as we were building a fundamental tool. However, in many cases, you may be better off quickly using existing great libraries, such as those available in Python. The right tool for the right job; there is no language or setup that fi ts all problems.

Slide 50

Slide 50 text

In the end, are we happy with Rust? Learning and writing Rust was FUN! You truly feel like you are learning the real fundamentals of computing. As a software engineer, that is a great experience and will make you a better developer, even if you don’t use Rust on a day-to-day basis.

Slide 51

Slide 51 text

More detail in this talk: Rust.Tokyo 2024, Satoru Nishio (MIERUNE) (Sorry, in Japanese language; feel free to open an issue for questions!) https: // speakerdeck.com/nokonoko1203/ri-ben-quan-guo-du-shi-3dhua-hurosiekuto-plateau-todetabian-huan-oss-plateau-gis-converter-nogong-kai

Slide 52

Slide 52 text

https: // github.com/MIERUNE/plateau-gis-converter github.com/MIERUNE/plateau-gis-converter OPEN SOURCE (MIT License) Have a look !

Slide 53

Slide 53 text

Rust PLATEAU nusamai Fast, reliable, & ef fi cient programming language Digital Twin project led by the Japanese ministry Open-source CityGML data conversion tool