The Development of Open Source Based Citizen Collaboration Applications for Infrastructure Management: My City Report Toshikazu Seto, Yoshihide Sekimoto, Hiroshi Omata, Hiroya Maeda, Takehiro Kashiyama, Shusaku HIGASHI, Masato Fujii, and Haruyuki SEKI. 1 2019/07/17 International Cartographic Conference 2019 @ Tokyo

l Open Government l Incident Reporting System n Chiba Report n FixMyStreet n 311 Chicago Etc… Citizens can report local problems (Pothole App. 311 App.) u Infrastructure maintenance u ICT l Lack of Experts l Expensive Cost • Research for citizen feedback systems （Patel, 2015; Goldstein,2013; Offenhuber,2015） Possibility of Citizens Collaboration for Infrastructure Management Collaborate with citizen and government together !

Background our studies… 3 https://doi.org/10.3390/ijgi8030115

Technological Seeds • Digital transformation and automation of administrative work • Image process using deep learning is quite high accuracy • AI (artificial intelligence) for supporting daily work is starting 4 (0)：損傷はない (1)：修繕すべき 損傷はない (2)：修繕すべき 損傷がある 市民からの投稿画像 点検業務による画像 （ラベル） ・損傷はない ・修繕すべき損傷はない ・修繕すべき損傷がある Joint-research by Chiba-city（March, 2016） Tweet by Mayor of Chiba City

The Progress of MCR project supported by NICT research grant FY2016 FY2017 FY2018 ・Prototyping ・Demonstration ・Development of for Road Manager application ■General Meeting (3 times) ・4 municipalities （+observer 4 municipalities） ・Construction of MCR application and backend systems ・Improvement of RM application ・ Improvement of MCR systems ・Continuous pubic trial (Muroran and Numazu) ・Improvement of RM application ・Development of Road AI dashboard system Call for Organization ■General Meeting (6 times) ・5 municipalities （+observer 4 municipalities） Call for Organization ■General Meeting (4 times) ・7 municipalities

https://www.mycityreport.jp/ ①MCR Smartphone App. ②MCR Management System ④ Road Damage Detector Dashboard ③ Road Damage Detector 6 Application for citizens Collaboration Application for road managers Developed by Georepublic Japan Developed by Sekimoto Lab.

Road Management Statistics of Local Government Name Total road length (km) Budget of Civil Engineering （千円） Budget of Civil management （千円） Budget of Road and bridge （千円） Employee Of Civil Engineer Civil Engine er Road Engineer Muroran 441.6 945,736 0 740,249 28 28 0 Numazu 1,135.5 403,891 74 263,137 106 106 0 Chiba 3,195.0 5,720,467 72,223 2,424,234 294 276 18 Hanamaki 3,302.5 748,276 4,896 701,009 22 22 0 Kaga 684.8 209,061 0 186,624 48 45 3 Higashi Hiroshima 2180.6 1,213,502 0 964,968 120 120 0 Shinagaw a 326.1 574,923 0 226,031 87 0 0 ■Source︓ ・Total road length（2015）︓【総務省】公共施設状況調経年⽐較表H27 ・Budget（2016）︓【総務省】地⽅財政状況調査2017 ・Employee（2017）︓【総務省】平成29年地⽅公共団体定員管理調査 7

The basic function of MCR application Default interface Filtering mode Reporting Select of the Map Select of categories Text and photo Status confirmation Publish API of Open311 8 Notifications of the municipality

The Interface for Municipal Managers Default Interface Confirmation of Status Detailed status management Printing Response for citizen 連絡表 受付月日 2018-08-16 担当者 室蘭全職員 現地住所 住宅地図 通報者氏名 dai 連絡先 通報方法 スマホ 602 いつも雨の後はこのようになっていて歩けません。ご対応お願いします。 タイトル 水たまり Response/ Repair 9

Main Reporting Category of MCR • Report for Problems – Mainly reporting issues of the infrastructure (decided by municipality such as road, park, river etc) • Report for Self-solving – Citizen reported to themselves solve the problem of the city (garbage picking and minor repair) • Specific Theme Report – Things that municipality side particularly calls for contribution (city recommendation spot, etc.) 10 ➔ Now, adjusting the functions for full-scale operation by multiple local governments. You can use only testing Report for Problems

Feedback of MCR function development from participating local governments カテゴリ FY2016 FY2017 FY2018 Total Smartphone 33 150 87 270 Administrative function 39 32 119 190 Public mode function - 8 9 17 Both of system - 6 9 15 Operation - 22 - 22 Total 72 218 224 514

The Statistics of public trial reporting using MCR application periods: April 2018 to February 2019 Category Incomplete Finished Total Road 58 21 79 Park 10 6 16 Other 3 9 12 Category Incomplete Finished Total Self-solving 0 1 1 Road 14 14 28 Park 3 5 8 River 0 1 1 Tourism facilities 0 0 0 ■Muroran City：27 users （Total 107 reports) ■Numazu City：14 users （Total 38 reports） Until the solution is initially demonstrated about from two months to within 1 month Mainly posts are not major roads, it is difficult to judge to repair 10 new problem & solved in February

Distribution of public trial reporting by MCR April 2018 to February 2019 Incomplete Finished Observation

Reporting Example of MCR Finished Incomplete Incomplete of Park •Displaced spot alone and remarkably discovered •Continuous damage •Section outside the jurisdiction of municipality •The broken of guide board or a bench, relatively repair is fast •The play equipment tends to take long to repair 14

③Road Damage Detector 15 • Automatic judgment of damage and classification within 1-2 seconds within smartphone • SSD Inception V2 / SSD MobileNet • Launched as an Android app with MIT License • https://github.com/sekilab/RoadDamageDetector

16

17 Road damage types in RDD dataset and their definitions Source: Road Maintenance and Repair Guidebook 2013 (JRA, 2013) in Japan.

18 Number of damage instances in each class in each municipality (FY2017) D00 D01 D10 D11 D20 D40 D43 D44 Recall of SSD Inception V2 0.22 0.60 0.10 0.05 0.68 0.03 0.81 0.62 Precision of SSD Inception V2 0.73 0.84 0.99 0.95 0.73 0.67 0.77 0.81 Accuracy of SSD Inception V2 0.78 0.80 0.94 0.92 0.85 0.95 0.95 0.83 Recall of SSD MobileNet 0.40 0.89 0.20 0.05 0.68 0.02 0.71 0.85 Precision of SSD MobileNet 0.73 0.64 0.99 0.95 0.68 0.99 0.85 0.66 Accuracy of SSD MobileNet 0.81 0.77 0.92 0.94 0.83 0.95 0.95 0.81 Detection and classification results for each class

Reporting by MCR & municipality patrol using RDD (September 2017) Incomplete Finished Observation Finding road damage Patrol route

• patrol cars do not go in micro area units • Expects posts from citizens, especially where they are off the patrol's regular route 20 Incomplete Finished Observation Finding road damage Patrol route

③ The results of Road Damage Detector • Maeda, H., Sekimoto, Y. and Seto, T.: Lightweight Road Manager: Smartphone-based Automatic Determination of Road Damage Status by Deep Neural Network, Proceedings of the 5th ACM SIGSPATIAL International Workshop on Mobile Geographic Information Systems (MobiGIS '16), pp.37-45, 2016.10.31 https://doi.org/10.1145/3004725.3004729 • Maeda, H., Sekimoto, Y., Seto, T., Kashiyama, T. and Omata, H.: Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images, Computer-Aided Civil and Infrastructure Engineering, https://doi.org/10.1111/mice.12387 • SSD Inception V2＆SSD MobileNet model and 10,000 trained images are published CC-BY-SA4.0 license. https://github.com/sekilab/RoadDamageDetector • Organized by IEEE Big Data Conference2018 ”Road Damage Detection and Classification Challenge” 15 countries, 52 teams participated. https://bdc2018.mycityreport.net/overview/ 21

Connecting with researchers all over the world by open data ! Published road damage data with 10,000 images (CC-BY-SA 4.0) https://github.com/sekilab/RoadDamageDetector） • IEEE Big Data Conference ”Road Damage Detection and Classification Challenge” • Participated with 52teams from 15 countries • The winning team is an ensemble of multiple deep learning methods, and the winning team is a Faster R-CNN base • https://bdc2018.mycityreport.net/ ※プライバシー保護のため、⼈の顔、 ⾞のナンバープレートにモザイクをかけています。 Announcement of the result Gold Prize Silver Prize Bronze Prize Special Prize IMSC@USC (F1 = 0.64) Abdullah Alfarrarjeh, Dweep Trivedi, Seon Ho Kim, and Cyrus Shahabi CMBC_CHALLENGERS (F1 = 0.68) Yanbo J. Wang, Ming Ding, Shichao Kan, Shifeng Zhang, Chenyue Lu, and Qi Hong DSSC@BUPT (F1 = 0.65) Wenzhe Wang, Bin Wu, Sixiong Yang, Zhixiang Wang Bodo Rosenhahn's group (F1 = 0.63) Florian Kluger, Christoph Reinders, Kevin Raetz, Philipp Schelske, Bastian Wandt, Hanno Ackermann, and Bodo Rosenhahn

④ Development of Road Damage Detector Dashboard: Real-time Display of Patrol results using RDD App. 23

④ Development of Road Damage Detector Dashboard: Road Evaluation and Cost Estimation by Road Data of the City 24

https://www.mycityreport.jp/ ①MCR Smartphone App. ②MCR Management System ④ Road Damage Detector Dashboard ③ Road Damage Detector 25 Application for citizens Collaboration Application for road managers Conclusions

Future Works • Comparing citizen reporting by some cities – Quantitative and qualitative – Analysis of long-term activities in Chiba-repo • Designing for Motivation of Citizen Reporting – Wellbeing, positive computing… – Peters D, Calvo RA and Ryan RM (2018) Designing for Motivation, Engagement and Wellbeing in Digital Experience. Front. Psychol. 9:797. doi: 10.3389/fpsyg.2018.00797 26 Peters et al. Designing for Motivation and Wellbeing FIGURE 2 | Taxonomy of Human Motivation; (A) Type of regulation, (B) Type of motivation, and (C) Examples translated to the user experience context (Adapted from Ryan and Deci, 2000a).

Launched My City Report Consortium! Thank you ! https://mycityreport.jp [email protected] 27