Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Satellite Data and Super-Resolution to enhance a Slope Soaring Simulator by Schalk Heunis

Pycon ZA
October 09, 2020

Satellite Data and Super-Resolution to enhance a Slope Soaring Simulator by Schalk Heunis

Imagine soaring the slopes of el-Capitan in Yosemite, Rainbow mountain in Peru or even replicating the wingsuit glide path of Jeb Corliss! And doing it under lockdown

Slope soaring requires one to go outside and spend time with friends which was not possible during lockdown, so the next best thing is a simulator.

This talk will cover how a 10 year old, abandoned open source simulator (SSS - http://www.rowlhouse.co.uk/sss/index.html) was enhanced using Python with:

* Digital Elevation Models from Google Earth Engine
Enhancing Elevation Model Super Resolution using Deep Learning
* Satellite imagery as scenery from anywhere in the world courtesy of Bing
* Multi-player networking

The result is stunning scenery and the ability to fly any slope you can dream of.

The talk will cover:

* Introduction to the slope soaring simulator (SSS) and motivation for the wanting to use this under COVID'19 lockdown
* Multi-player networking using Python cstruct and socket libraries
* Extracting digital elevation models (DEM) and satelite data using Google Earth Engine Python API
* Downloading aerial photography from Bing using Python and mapping it to the DEM
* Downloading Hi-res LIDAR point cloud data and processing as target for Super Resolution using PDAL Python API
* Enhancing the elevation model using Deep Learning Super Resolution from 30m to 5m accuracy
* Adding skybox scenery to complete the picture

This talk is for anyone interested in working with satellite data, super-resolution and slope soaring simulations

Pycon ZA

October 09, 2020
Tweet

More Decks by Pycon ZA

Other Decks in Programming

Transcript

  1. Satellite data & Super resolution to
    enhance a Slope Soaring Simulator
    Schalk Heunis

    View Slide

  2. Previous Pycon Talks
    • 2019 -Edge Computing at the Edge of the World

    View Slide

  3. View Slide

  4. View Slide

  5. Amphitheatre – Climb & Fly 12 April

    View Slide

  6. Slope Soaring Simulator

    View Slide

  7. Slope Soaring Simulator

    View Slide

  8. View Slide

  9. Multi Two player

    View Slide

  10. Two player à Multi player
    Python Server
    SSS SSS SSS SSS
    struct

    View Slide

  11. cstruct
    • pip install cstruct

    View Slide

  12. View Slide

  13. Import DEM
    • Add slide with orog documention

    View Slide

  14. Digital Elevation Models

    View Slide

  15. Digital Elevation Model

    View Slide

  16. Digital Elevation Model

    View Slide

  17. Digital Elevation Model - Python
    • pip install rasterio

    View Slide

  18. Digital Elevation Model - Python

    View Slide

  19. https://touchterrain.geol.iastate.edu

    View Slide

  20. Satellite Data

    View Slide

  21. Google Earth Engine
    • pip install earthengine-api

    View Slide

  22. Google Earth Engine

    View Slide

  23. View Slide

  24. View Slide

  25. Higher Resolution Data: LiDAR

    View Slide

  26. LiDAR

    View Slide

  27. https://usgs.entwine.io/

    View Slide

  28. Super Resolution
    Learn transform
    Low Res (30m) High Res (5m)

    View Slide

  29. Super Resolution
    • GAN
    • Unsupervised
    • Hard to train
    • Cool
    • Supervised
    • Simple
    • Fast to train

    View Slide

  30. Fast.AI

    View Slide

  31. Fast.ai
    • conda install -c fastai -c pytorch -c anaconda fastai gh
    anaconda
    • Lesson 7: Resnets from scratch; U-net; Generative
    (adversarial) networks
    • https://course19.fast.ai/
    • https://github.com/fastai/course-
    v3/blob/master/nbs/dl1/lesson7-superres.ipynb

    View Slide

  32. U-Net: Convolutional Networks for Biomedical Image Segmentation
    41m parameters
    Fast AI: Pretrained
    Resnet34

    View Slide

  33. Super Resolution – fast.ai – UNet

    View Slide

  34. LiDAR - Python
    • pip install PDAL

    View Slide

  35. PDAL : resource
    • https://usgs-lidar-public.s3-us-west-2.amazonaws.com/{resource}/ept.json
    • resource:

    View Slide

  36. Training data
    • 16 446 tiles
    • Each tile 128x128 @ 5m
    • 0.4 km2
    • 6 736 km2

    View Slide

  37. Train & Validate
    Low Res (30m) Predicted Hi Res (5m)

    View Slide

  38. Amphitheatre
    Low Res (30m) Predicted (5m)

    View Slide

  39. Amphitheatre
    Low Res (30m) Predicted (5m)

    View Slide

  40. Add ground texture
    • Bing Maps
    • https://github.com/llgeek/Satellite-Aerial-Image-Retrieval

    View Slide

  41. Ampitheatre

    View Slide

  42. View Slide

  43. View Slide

  44. View Slide

  45. El-Capitan Yosemite

    View Slide

  46. Rainbow Mountain Peru

    View Slide

  47. Glider & cockpit view

    View Slide

  48. Jeb Corliss – “Grinding The Crack”

    View Slide

  49. View Slide

  50. View Slide