GeoBIM benchmark 2019: design and initial results

Presentation to the GeoSpatial Week 2019 conference.
The preparation of the data useful to the ISPRS-EuroSDR GeoBIM benchmark scientific initiative (https://3d.bk.tudelft.nl/projects/geobim-benchmark/) is described, together with the initial results obtained in the first 3 months from the beginning.

Transcript

1. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Reference study on software support for open standards of city and building models
2. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Overview The GeoBIM benchmark Preparing the data • The selected IFC data • The selected CityGML data Initial results • Initial results for Task 1 (support for IFC) • Initial results for Task 3 (support for CityGML) • Initial results for Task 4 (IFC à CityGML) Discussion and conclusions Progress status of the participation
3. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter The GeoBIM benchmark GeoBIM benchmark The data IFC data CityGML data Initial results Task 1 Task 3 Task 4 Conclusions Progress of participation
4. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Investigated features (in Task 1 and 3) While importing and exporting the data in the software, participants should check: How is georeferencing supported: how is it read by software? How is geometry managed: is it read correctly and exported correctly? For CityGML: LoDs management, ADEs management… How are semantics managed: does the conversion to native format leave semantics consistent? Is it exported consistently? Does it lose detail/granularity? What functionalities does the software offer to manage the file (view, analysis, editing…) Detailed instructions in the results templates • Deliver the data in the native format of the used software • Deliver the re-exported data to IFC (Task 1) or CityGML (Task 3) GeoBIM benchmark The data IFC data CityGML data Initial results Task 1 Task 3 Task 4 Conclusions Progress of participation
5. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Challenge: provision of adequate data to test software, tools and procedures without biases and effectively serve the purpose. Potentially, two kinds of data are available: 1. Data modelled and generated in academic environments clean and perfect Results unrealistically positive for the software’s behaviour in practice. 2. Data derived from real-world practice. Without the aim of being exchanged using open standards (many aspects disregarded: e.g. geometries validity, correctness of semantics…). Results negatively influenced: issues with software support by the inaccuracies in the models rather than the software itself. Preparing the data GeoBIM benchmark The data IFC data CityGML data Initial results Task 1 Task 3 Task 4 Conclusions Progress of participation
6. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Our compromise Datasets modelled by practitioners Inspected and analysed using 3D viewers and validation tools (e.g. IfcViewer, Solibrí Model Checker, FZK, Azul). Inspection of the files in their text format. Focus on geometry, semantics and georeferencing Understanding of the main limitations (inaccurate semantics, inaccuracies in the modelling of geometries…) Better models chosen Main issues resolved Other problems documented. Preparing the data GeoBIM benchmark The data IFC data CityGML data Initial results Task 1 Task 3 Task 4 Conclusions Progress of participation
7. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter The analysed models (different sources no reciprocal connections) have almost identical characteristics: IFC data Den Haag Witte de Withstraat(20150508) 23.259 MB Den Haag CUVO Ockenburghstraat KOW 5.389 MB Den Haag Haviklaan-6 0.062 MB Den Haag Rabarberstraat144 5.396 MB Rotterdam 15048 Uitvoering Up Town 16.231 MB Rotterdam KLA_Uptown 246.824 MB Rotterdam Bright Rotterdam_IFC 325.230 MB Rotterdam KralingseZoom – Hogeschool 1.337 MB Almere P120_SubZero_TOB_2017-10-23 29.521 MB Sweden Myran_2017-10-25 27.788 MB IFC version 2x3 Software producer Autodesk Revit 2014 (ENU) Included IFC entities IfcProject IfcSite IfcBuilding IfcBuildingStorey (8 no criteria for grouping objects) IfcSpace ifcWall IfcWallStandardCase IfcColumn IfcBeam IfcSlab IfcStair IfcRoof IfcCovering IfcOpeningElement IfcDoor IfcWindow IfcRailing IfcFlowSegment IfcStairFlight IfcAnnotation IfcBuildingElementProxy Note: Many IfcBuildingElementProxy and not always accurate definition(e.g. roof and slab) Inspection in some of the most common 3D viewers (RDF IFC Viewer, KIT FZK viewer, Solibri M. Viewer) Analysis through the NIST STEP File Analyser and Viewer (not supporting IFC4) Inspection of the text format of the IFC files (through common text editors). GeoBIM benchmark The data IFC data CityGML data Initial results Task 1 Task 3 Task 4 Conclusions Progress of participation
8. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter • no serious geometry issues (e.g. not many intersections, etc.), • the most complete semantics as possible, including the definition of attributes and relationships (hierarchies, groups, associated materials) • From different nations (no specific national rules or best practices could affect the results of the benchmark). • permission to be shared at least among the participants of the project. one representing a bigger project, in a heavier file, to test also the connected software-and-hardware performances. one describing a small building in order to test the software functionalities in the most reliable way as possible. Choice criteria: IFC data GeoBIM benchmark The data IFC data CityGML data Initial results Task 1 Task 3 Task 4 Conclusions Progress of participation
9. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Two-floor office building in Falun, Sweden. by MONDO arkitekter. Architectural model Autodesk Revit 2018 (ENU) IFC 2x3 (MVD: CoordinationView v.2.0). Model georeferenced through RefLatitude, RefLongitude, RefElevation in the IFCSITE entity. The IFCTrueNorth entity was added to the IFC file (by modifying the text format). Accurate semantics, many attributes filled. The Myran IFC model Grouping of entities in 3 storeys is little consistent. For the tests, the grouping is anyhow considered as the reference, and the self- compliancy checked. Storey 1 Storey 2 Storey 3 GeoBIM benchmark The data IFC data CityGML data Initial results Task 1 Task 3 Task 4 Conclusions Progress of participation
10. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Large residential building under construction in Rotterdam, the Netherlands. Provided by the Municipality of Rotterdam. Architectural model Autodesk Revit 2015 (ENU) IFC 2x3 (MVD CoordinationView v.2.0). Dimension of the file: 252,7 MB à useful to test the associated software and hardware performance. Not georeferenced The Up:Town building IFC model Semantics: a high number of IfcBuildingElementProxy (generic IFC entity used to cover the semantics of objects not explicitly defined by other IFC entities). GeoBIM benchmark The data IFC data CityGML data Initial results Task 1 Task 3 Task 4 Conclusions Progress of participation
11. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Residential building in Savigliano, Italy. by L. Polia (architect expert in BIM and IFC) within a research in Politecnico di Torino, (Department of Architecture and Design, prof. F. Rinaudo). Architectural model Autodesk Revit 2019 (ITA) IFC 4 (DesignTransferView v.1.0). No georeferencing. Many aspects (e.g. grouping of entities, semantics definition) more controlled than in common models The ‘Savigliano’ IFC model To test tools and functionalities for IFC4. Difference between IFC 2x3 and 4. GeoBIM benchmark The data IFC data CityGML data Initial results Task 1 Task 3 Task 4 Conclusions Progress of participation
12. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter To assess the diverging compliance within software, the ‘IFC geometries’ benchmark file includes: • Esoteric profile definition (removed in IFC4, IfcCraneRailAShapeProfileDef), • Negative extrusion depth, • Extrusion direction parallel to the base surface. • Sweep along a parametric range of a curve, • Conventional geometries (e.g. Boolean operations and revolutions). IFC2x3 geometries and IFC4 geometries GeoBIM benchmark The data IFC data CityGML data Initial results Task 1 Task 3 Task 4 Conclusions Progress of participation Not possible to completely analyse the building models geometries (many and very complex). To test the geometries whose management is theoretically allowed by IFC, and which could be widespread in the practice of building modelling, but which often give errors when used by software.
13. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter • Validated (through val3Dity, schema validation, FME validator, 3DCityDB validator) • Covering the most part of available CityGML classes. • All available LoDs CityGML data Choice criteria: GeoBIM benchmark The data IFC data CityGML data Initial results Task 1 Task 3 Task 4 Conclusions Progress of participation
14. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Entire city of Amsterdam, seamless 3D city model (buildings, roads, vegetation, and so on) Level of Detail (LoD) 1. CRS: Amersfoort / RD New (EPSG: 28992). Open data very heavy model (4.36 GB) CityGML Amsterdam LoD 1 Modelling: software 3dfier (3D geoinfo TUDelft) Footprints - open datasets BGT (large- scale topographic map of the Netherlands) Heights - AHN3 (laser altimetry point cloud) GeoBIM benchmark The data IFC data CityGML data Initial results Task 1 Task 3 Task 4 Conclusions Progress of participation One model covering one entire city (Amsterdam), which could be essential when running analysis on the whole city, even if the file is very heavy.
15. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Maritiem district in Rotterdam (Surrounding area of the Up:Town building). CityGML class Building CRS: Amersfoort / RD New (EPSG: 28992). CityGML Rotterdam LoD 1 and 2 Geometry validated through val3dity (3D geoinfo TUDelft) : errors on 3.5% of the primitives (non-planar polygons, open shells and self- intersections): cleaner model than most other 3D city models of that size and complexity. Provided by the City of Rotterdam Open data Level of Detail (LoD) 1 (extruded building footprints) Level of Detail (LoD) 2 (more detailed external surfaces of the buildings). GeoBIM benchmark The data IFC data CityGML data Initial results Task 1 Task 3 Task 4 Conclusions Progress of participation One model including different levels of details (LoDs), which is an interesting opportunity in CityGML, but not always well supported.
16. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Procedurally generated dataset CityGML class Building Level of Detail (LoD) 3. CRS: Amersfoort / RD New (EPSG: 28992). Open data CityGML Buildings LoD 3 Modelling: open source tool Random3Dcity (3D geoinfo TUDelft) Validated and corrected against residual errors. GeoBIM benchmark The data IFC data CityGML data Initial results Task 1 Task 3 Task 4 Conclusions Progress of participation One LoD3 model, requiring whole management of 3D objects.
17. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Time from the start: 3 Months Initial results – progress status Tools for Task 1 Software OS Kind of sw Autodesk Revit 2018 Windows 10 BIM software Graphisoft ArchiCAD 22.0.0 Windows 10 BIM software Vectorworks Designer 2019 Windows 10 BIM software KIT FZK Viewer Windows 10 3D Viewer Trimble SketchUP macOS Mojave CAD software Tools for Task 4 Software OS Kind of sw Used methods Safe Software FME Windows 10/Home ETL tools Standard conversion Specific workflow Tools for Task 3 Software OS Kind of sw Used methods ESRI ArcGIS 10.2 Windows 10 GIS software Data interoperability tb QGIS 3.6 Windows 10 GIS software Conversion to CityJSON Safe Software FME Data Inspector 2018.1 macOS 10.14.3 3D viewer - KIT FZK Viewer Windows 10 3D viewer - Kind of Participants: • Researchers • 1st year students of the MSc in Geomatics • MSc graduating students in Geomatics 0 1 2 3 4 5 6 7 8 9 Task 1 Task 3 Task 4 Submitted tests results n. submissions n.software GeoBIM benchmark The data IFC data CityGML data Initial results IR Task 1 IR Task 3 IR Task 4 Conclusions Progress of participation
18. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Common features for BIM sw • import and export functionalities • 2D and 3D view, query and analysis tools, editing of the model (attributes, geometry, georeferencing, scale). • IFC certification (2013 - 2015): IFC2x3 ‘Coordination View 2.0’ import IFC2x3 ‘Coordination View 2.0-Arch’ export. Similar results in testing the buildings models (Myran, UpTown, Savigliano). Similar tools for reading the georeferencing details SketchUP is not able to import the files Initial results for Task 1 – supports for IFC Export functionality All à Selection of the Model View Definition (MVD) and IFC version for exporting the model Revit, ArchiCAD, FZK Viewer à no processing required (by default) to enable a consistent export Revit and ArchiCAD à customization of MVD possible. Vectorworks à layer mapping pre-process to define storeys. only Revit à export to IFC 4. Differences in the tests Hardware+software performance (Management of the ‘UpTown’ building model) Revit, Vectorworks, FZK Viewer à system crashes ArchiCAD à no problems (operations in up to 5 minutes approximately). GeoBIM benchmark The data IFC data CityGML data Initial results IR Task 1 IR Task 3 IR Task 4 Conclusions Progress of participation
19. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Semantics Entities ArchiCAD, Vectorworks, FZK Viewer à Consistent with the corresponding IFC entities Revit à loss of specialization: e.g. Attributes Consistent attributes Relationships Hierarchical and other (e.g. grouping) relationships à not consistent with IFC ones. Elements different from the original IFC-hierarchy, once imported and ordered in the Revit families. (errors while importing the file, that required to separate elements to repair them). Not a straightforward functionality of the tools to understand relationships. FZK Viewer can read the relationships well [‘IfcTank’, ‘IfcTankType’, ‘IfcTransportElement’, ‘IfcTransportElementType’] à ‘Specialty Equipment’ [‘IfcWall’, IfcWallStandardCase’, ‘IfcWallType’] à ‘Wall’ The software modelling the original file (from which IFC data were exported) was Revit! à Possible to set (manually) the corresponding Revit Categories. Initial results for Task 1 – supports for IFC GeoBIM benchmark The data IFC data CityGML data Initial results IR Task 1 IR Task 3 IR Task 4 Conclusions Progress of participation
20. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter ‘IFC geometry’ files Extruded shapes having a footprint with oblique edges à None of the BIM software could read them Some of the geometries (e.g. swept H shapes), à only read as a footprint, or flattened on the plane, or not visualised Curved geometries à look smooth and correctly discretized Two small cylinders in the corner Curved surface (opposite behavior w.r.t. others) à in ArchiCAD look smooth à in Vectorworks, roughly discretized (becoming octagons). Different heights à none of the software can read this. Two H beams with different heights. à read in Revit and Vectorworks à Not visible in ArchiCAD and FZK. Initial results for Task 1 – supports for IFC FZK reads them correctly GeoBIM benchmark The data IFC data CityGML data Initial results IR Task 1 IR Task 3 IR Task 4 Conclusions Progress of participation
21. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Initial results for Task 3 – support for CityGML Common features CityGML format not directly supported in GIS sw (ArcGIS and QGIS): Rotterdam LoD 1+2 à set of read-only 2.5D points, representing the buildings (likely their centroids) with associated attributes. [Misinterpretation of geometries, stored with the two joint LoDs]. Buildings LoD 3 à no geometry is visualised; only an attribute table is shown. [high complexity of the data]. Amsterdam LoD 1 à imported directly, even if it takes some time (i.e. a couple of minutes). Geometries (triangulated surfaces) imported as a QGIS layer for each entity, visualised in 2D in usual map and in 3D in the 3D map (new functionality in QGIS 3). GeoBIM benchmark The data IFC data CityGML data Initial results IR Task 1 IR Task 3 IR Task 4 Conclusions Progress of participation
22. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Hardware+software performance Amsterdam.gml was too big to be processed: CityGML-tools for conversion to CityJSON à not possible to continue ArcGIS à struggled because of memory limitations. FZK Viewer and FME DI à it takes more than one hour to import and handle it à Specific translation tools to import all the 3 files consistently in GIS: ArcGIS à ‘Data Interoperability toolbox’ QGIS à conversion to CityJSON, through citygml-tools+ CityJSON Loader plugin for QGIS. Limitations: Problems with multiple LoDs, Loss of semantics: • hierarchical relationships are not maintained • attributes stored in different tables, connected through parent IDs. 3D view 3D viewer in QGIS still basic and detached from the main 2D view, operations. More advanced features in ArcGIS through ArcGlobe or ArcScene. Both GIS software and the viewers handle well many features, georeferencing, geometry and semantics. Initial results for Task 3 – support for CityGML GeoBIM benchmark The data IFC data CityGML data Initial results IR Task 1 IR Task 3 IR Task 4 Conclusions Progress of participation
23. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Hardware+software performance Myran.ifc and IFCgeometries.ifc were ok Uptown.ifc could be converted in +2 hours (result cannot be viewed in the FME viewer – Inspector) Initial results for Task 4: IFC à CityGML à Conversion tools for the three buildings files: Extended FME workflow with default transformers Data comment: In Myran: There is now connection between individual glass screens and steel frames. Therefore scripts has to be tailored not to loose the glass screens. Quality improvements in conversion Use of data inspector in FME is required to identify solid and surface parts for SIG 3D modelling instructions. Source: Ville-Pekka Soini, LU GeoBIM benchmark The data IFC data CityGML data Initial results IR Task 1 IR Task 3 IR Task 4 Conclusions Progress of participation
24. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Challenges were to obtain and suitably describe good test data coming from practice: Discussion and conclusions For BIMs • Practitioners usually don’t care the result of the export to IFC (geometry, semantics) • Small part of the available IFC entities is included in IFC files from practice • No georeferencing in the BIMs from practice • No IFC 4 data was available from practice • Few tools for IFC validation • Privacy and copy right issues. For 3D city models • Few multi-LoD data • Almost no data in LoD 3 • Often inaccurate structure of the GML files: e.g. ID formats; order of the attributes; XML structure For all data from practice (not generated by a software we know): MOSTLY DARK DATA (no metadata, no information about modelling, source, etc.) GeoBIM benchmark The data IFC data CityGML data Initial results IR Task 1 IR Task 3 IR Task 4 Conclusion Progress of participation
25. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter Registered participants from 19 Countries The GeoBIM benchmark is shaping up to become a useful reference for the current capabilities of software tools w.r.t. open standards. It is attracting interest! Discussion and conclusions GeoBIM benchmark The data IFC data CityGML data Initial results IR Task 1 IR Task 3 IR Task 4 Conclusion Progress of participation
26. GeoBIM benchmark 2019: design and initial results F. Noardo, K. Arroyo Ohori, F. Biljecki, T. Krijnen, C. Ellul, L. Harrie, J. Stoter You are invited!