Satellite Data and Super-Resolution to enhance a Slope Soaring Simulator by Schalk Heunis

Imagine soaring the slopes of el-Capitan in Yosemite, Rainbow mountain in Peru or even replicating the wingsuit glide path of Jeb Corliss! And doing it under lockdown

Slope soaring requires one to go outside and spend time with friends which was not possible during lockdown, so the next best thing is a simulator.

This talk will cover how a 10 year old, abandoned open source simulator (SSS - http://www.rowlhouse.co.uk/sss/index.html) was enhanced using Python with:

* Digital Elevation Models from Google Earth Engine
Enhancing Elevation Model Super Resolution using Deep Learning
* Satellite imagery as scenery from anywhere in the world courtesy of Bing
* Multi-player networking

The result is stunning scenery and the ability to fly any slope you can dream of.

The talk will cover:

* Introduction to the slope soaring simulator (SSS) and motivation for the wanting to use this under COVID'19 lockdown
* Multi-player networking using Python cstruct and socket libraries
* Extracting digital elevation models (DEM) and satelite data using Google Earth Engine Python API
* Downloading aerial photography from Bing using Python and mapping it to the DEM
* Downloading Hi-res LIDAR point cloud data and processing as target for Super Resolution using PDAL Python API
* Enhancing the elevation model using Deep Learning Super Resolution from 30m to 5m accuracy
* Adding skybox scenery to complete the picture

This talk is for anyone interested in working with satellite data, super-resolution and slope soaring simulations

Transcript

1. Satellite data & Super resolution to
   enhance a Slope Soaring Simulator
   Schalk Heunis
   

   View Slide

2. Previous Pycon Talks
   • 2019 -Edge Computing at the Edge of the World
   

   View Slide

3. View Slide

4. View Slide

5. Amphitheatre – Climb & Fly 12 April
   

   View Slide

6. Slope Soaring Simulator
   

   View Slide

7. Slope Soaring Simulator
   

   View Slide

8. View Slide

9. Multi Two player
   

   View Slide

10. Two player à Multi player
    Python Server
    SSS SSS SSS SSS
    struct
    

    View Slide

11. cstruct
    • pip install cstruct
    

    View Slide

12. View Slide

13. Import DEM
    • Add slide with orog documention
    

    View Slide

14. Digital Elevation Models
    

    View Slide

15. Digital Elevation Model
    

    View Slide

16. Digital Elevation Model
    

    View Slide

17. Digital Elevation Model - Python
    • pip install rasterio
    

    View Slide

18. Digital Elevation Model - Python
    

    View Slide

19. https://touchterrain.geol.iastate.edu
    

    View Slide

20. Satellite Data
    

    View Slide

21. Google Earth Engine
    • pip install earthengine-api
    

    View Slide

22. Google Earth Engine
    

    View Slide

23. View Slide

24. View Slide

25. Higher Resolution Data: LiDAR
    

    View Slide

26. LiDAR
    

    View Slide

27. https://usgs.entwine.io/
    

    View Slide

28. Super Resolution
    Learn transform
    Low Res (30m) High Res (5m)
    

    View Slide

29. Super Resolution
    • GAN
    • Unsupervised
    • Hard to train
    • Cool
    • Supervised
    • Simple
    • Fast to train
    

    View Slide

30. Fast.AI
    

    View Slide

31. Fast.ai
    • conda install -c fastai -c pytorch -c anaconda fastai gh
    anaconda
    • Lesson 7: Resnets from scratch; U-net; Generative
    (adversarial) networks
    • https://course19.fast.ai/
    • https://github.com/fastai/course-
    v3/blob/master/nbs/dl1/lesson7-superres.ipynb
    

    View Slide

32. U-Net: Convolutional Networks for Biomedical Image Segmentation
    41m parameters
    Fast AI: Pretrained
    Resnet34
    

    View Slide

33. Super Resolution – fast.ai – UNet
    

    View Slide

34. LiDAR - Python
    • pip install PDAL
    

    View Slide

35. PDAL : resource
    • https://usgs-lidar-public.s3-us-west-2.amazonaws.com/{resource}/ept.json
    • resource:
    

    View Slide

36. Training data
    • 16 446 tiles
    • Each tile 128x128 @ 5m
    • 0.4 km2
    • 6 736 km2
    

    View Slide

37. Train & Validate
    Low Res (30m) Predicted Hi Res (5m)
    

    View Slide

38. Amphitheatre
    Low Res (30m) Predicted (5m)
    

    View Slide

39. Amphitheatre
    Low Res (30m) Predicted (5m)
    

    View Slide

40. Add ground texture
    • Bing Maps
    • https://github.com/llgeek/Satellite-Aerial-Image-Retrieval
    

    View Slide

41. Ampitheatre
    

    View Slide

42. View Slide

43. View Slide

44. View Slide

45. El-Capitan Yosemite
    

    View Slide

46. Rainbow Mountain Peru
    

    View Slide

47. Glider & cockpit view
    

    View Slide

48. Jeb Corliss – “Grinding The Crack”
    

    View Slide

49. View Slide

50. View Slide