GEOQUESTIONS1089 Sergios-Anestis Kefalidis1, Dharmen Punjani2, Eleni Tsalapati1,Konstantinos Plas1, Mariangela Pollali1, Michail Mitsios1, Myrto Tsokanaridou1, Manolis Koubarakis1, and Pierre Maret2/1 National and Kapodistrian University of Athens, 2 Universit´e St. Monnet • 一言でいうと 地理空間QAエンジンの評価用ベンチマークデータセットを作成, SOTAのエンジンを評価. • Resource Type (Benchmark data) • 動機 地理空間QAエンジンの有効性(effectiveness), 効率(efficiency)の評価のため, 意味 的に複雑なもの等, 多様なタイプの質問を含む大規模なベンチマークデータセットを作成. • リソースの設計方針 - 対象: YAGO2, YAGO2geo(YAGO2のうち, 緯度経度情報をもつエンティティのサブセット) - 従来のデータセット(GEOQUESTIONS201)より複雑な質問を多く含む.自然言語理解や GeoSPARQLの高度なfeatures(ネスト, no-existsフィルタ, 算術演算)の必要な質問等. • 新規性 地理空間QAの最大のベンチマーク(1089トリプル), 意味的に複雑な質問を含む • リソースの再利用性/活用実績 エンジン(GeoQA2,Hamzeiら(2021))を評価(複雑な質問には両者とも対応が不十分) . • 品質 クエリのタイプ別/作成方法別サブセットによりエンジンの詳細な分析が可能(報告者コメント). • リソースの可用性 https://github.com/AI-team-UoA/GeoQuestions1089 • 次にすべきこと 地理空間QAエンジンの深層学習のトレーニングに使えるようデータセットを拡張 カテ ゴリ 説明 (3.The GEOQUESTIONS1089 Datasetの本文より引用, 太字は報告者による) A Asking for a thematic or a spatial attribute of a feature, e.g., “Where is Loch Goil located?” B Asking whether a feature is in a geospatial relation with another feature or features, e.g., “Is Liverpool east of Ireland?”. C Asking for features of a given class that are in a geospatial relation with another feature. E.g., “Which counties border county Lincolnshire?” or “Which hotels in Belfast are at most 2km from George Best Belfast City Airport?” D Asking for features of a given class that are in a geospatial relation with any features of another class, e.g., “Which churches are near castles?”. E Asking for features of a given class that are in a geospatial relation with an unspecified feature of another class, and either one or both, is/are in another geospatial relation with a feature specified explicitly. E.g., “Which churches are near a castle in Scotland?” or “In Greece, which beaches are near villages?” F As in categories C, D and E above, plus more thematic and/or geospatial characteristics of the features expected as answers, e.g., “Which mountains in Scotland have height more than 1000 m?” G Questions with quantities and aggregates, e.g., “What is the total area of lakes in Monaghan?” or “How many lakes are there in Monaghan?”. H Questions with superlatives or comparatives, e.g., “Which is the largest island in Greece?” or “Is the largest island in France larger than Crete?”. I Questions with quantities, aggregates, and superlatives/comparatives, e.g.,“Which city in the UK has the most hospitals?” or “Is the total size of lakes in Greece larger than lake Loch Lomond in Scotland?”