Vegetation management for powerlines with remote sensing data Datageeks Meetup March 2023 Dr. Roland Rodde

2 Agenda Project motivation and setup I Satellite images and Lidar data II Modelling and usage of data III IV V V Architecture IV

No content

Project motivation 4 Vegetation can grow into the powerlines from below or fall into the the powerlines from the sides. This can lead to outages. Therefore the vegetation near to the powerlines has to be monitored

Project goals 5

One project team 6 - Jose Moreno Ortega - Roland Rodde - Martin Cleven - Stefan Bietz - Anders Richter - Christer Friberg - Cecilia Pihl - Nils Näsström - Anton Westholm - Wilhelm Rosendahl - Alejandro Ruete - Jonas Josefsson - Louise Westerberg - Tobias Dannerstedt - Anders Vannfält - Johan Höök

Product development 7 Agile development: mixture of Kanban and Scrum, Sprints of 3 weeks Different workstreams: Frontend/UI, GIS systems, Ecological corridor management, Data processing and ML Quality standards: Unit testing of code, peer reviews before code acceptance Toolset: Jira (Task management), Confluence (documentation), Gitlab (source code management)

8 Agenda Project motivation and setup I Satellite images and Lidar data II Modelling and usage of data III IV V V Architecture IV

High resolution satellite images 9 Satellite images can be tasked for areas of interest Skysat image constellation operated by planet A full set of APIs is available to do the tasking, track the tasking and to obtain the image data Time to delivery mainly dependent on weather conditions and satellite orbits

Planets image processing chain 10

Example data 11

What is Lidar data? 12

How do we obtain the Lidar data? 13

Lidar data example 14 Short demo of lidar data

15 Agenda Project motivation and setup I Satellite images and Lidar data II Modelling and usage of data III IV V V Architecture IV

Generating training data set Preprocessing of satellite images 16 Tiling Cloud cover removal Selection of tiles with ground truth Split of dataset Input Train Test Validation

Generating training data set Generating ground truth from Lidar data 17 Reprojection of lidar data Map projection of heights Map projection of lidar classes Creation of segmentation classes Tiling of segmentation images Split of dataset Input Train Test Validation

Semantic segmentation model 18 DeeplabV3 model for semantic segmentation Resnet101 backbone Model is pretrained on COCO train2017 dataset Can be easily obtained through torchvision library Transfer learning is done to adapt model parameters to task at hand

Model training 19 • We use cross entropy loss as loss function • As our groundtruth data contains a lot of missing data, we ignore these pixels in loss calculation • We use a learning rate scheduler to adapt learning rate during training (exponential decay) • Adam is used as optimiser • To increase the diversity of the training dataset we apply image augmentations on the training images (Rotations, flips, distortion of colours) • Early stopping done after model does not improve over several epochs • Dataset pretty balanced, no need for class weights • Hyperparameter tuning was apllied to find the best set of hyperparameters (learning rate, learning rate scheduler, image augmentation probabilities, …)

Model evaluation 20 Our main metric is IoU (Intersection over Union) Groundtruth Prediction Red IoU: 5 / (5 + 2 + 1) = 0.63 Blue IoU: 4 / (4 + 1 + 2) = 0.57 Mean IoU: Mean of class IoU = 0.6 We also log the best and worst performing tiles to get an idea for model improvements

Generating heightmaps 21 Height normalised Lidar Input Determine DTM Resolution binning Quantile based height Output

Generating growth maps 22 Height map date 1 (2015) Height map date 2 (2019) Weather data (temperature and precipitation) Growthmap

Calculating risks 23 Growth Model HeightMap1 GrowthMap PowerLine Risk Model Based on the initial heightmap, the GrowthMap will be used to estimate the future height for each pixel. These pixel will be associated with a risk value, which will be projected into points with a equidistant distance along a line in the centre of the corridor. Based on the accumulated risk for each point, areas in need of clearing are identified which will be assumed cleared and used as feedback to future years risk_year interpolation_distance Projected Risk HeightMap0 Cut down of high risk areas

24 Agenda Project motivation and setup I Satellite images and Lidar data II Modelling and usage of data III IV V V Architecture IV

Azure architecture 25

Python environment 26 rasterio

Scaling and rollout 27

Thank you for your attention!